Job Role Separation (JRS) - RAC/Non-RAC environments


Job Role Separation (JRS) environment when deploying Oracle Grid Infrastructure is about having 2 different users for GI and DB installation.
In our scenario, we will have oracle for database installation and oraasm for Grid Infrastructure installation


Checklist for Job Role Separation (JRS) environment-


1. setuid bit - As root, change the file permissions of the oracle executable under the <Grid_Home>/bin and the <db_home>/bin,
to 6751
This is basically used on binaries when you would like OS user A to execute binary X which owned by user B preserving privileges of user B.

# cd <Grid_Home>/bin
# ls -l oracle
# chmod 6751 oracle
# ls -l oracle
-rwsr-s--x 1 grid oinstall 173515905 May 21 17:04 oracle

This is the setuid bit, and this must be set in order for users, other than "Grid" user to have it work.
(same applies to the db_home)
# cd <ORACLE_Home>/bin
# ls -l oracle
# chmod 6751 oracle
# ls -l oracle


2. Both users (oracle,oraasm) should be part of oinstall,asmdba,asmadmin groups.
We perform Grid Infrastructure installation first and when installing database, you will not be able to locate diskgroups.
This happens when oracle user is not secondary member of group which is assigned when configuring oracleasm.
In my scenario,
ls -ltr /dev/oracleasm/disks
total 0
brw-rw---- 1 oraasm asmadmin 253,  7 Aug 17 14:27 PLNDWSTG_DATA8

id oracle
uid=1006(oracle) gid=1013(oinstall) groups=1013(oinstall),1006(dbaplstg),1015(asmdba),1017(racdb)

id oraasm
uid=1002(oraasm) gid=1013(oinstall) groups=1013(oinstall),1002(asmadmin),1003(dbadvetl),1004(dbadvndw),1005(dbaplpub),1006(dbaplstg),1007(dbautpub),1008(dbautstg),1015(asmdba),1016(asmoper),1017(racdba)

We added asmadmin as secondary group to oracle user here.


3. Check for /etc/fstab to make sure grid home filesystem was mounted with option 'nosuid'

4. /etc/oracle/olr.loc must have correct GI home location.
This can occur during upgrades and we must make sure we have correct location under olr.loc for CRS_HOME and not old one.

5. Correct permissions for -
- GI_BASE
- GI_HOME

6. sqlnet.ora under GI_HOME should have below -
- DIAG_ADR_ENABLED=ON
- ORA_CLIENTTRACE_DIR to any valid directory that is having write permissions for oraasm in your GI_HOME
7. RDBMS_HOME/dbs/init.ora should only contain diskgroup name for db_create_online_log_dest_n.
To avoid below error, you can have init.ora parameter file checked to point to 'diskgroup name' and not complete path.


8. Both RDBMS_HOME/lib and GRID_HOME/lib must have correct permissions (755)


9. Check if CRS_HOME or ORA_CRS_HOME environments are set and make sure they are point to right GI_HOME. (typically during upgrades).

10. Last but not the least, we should check asm instance status using crsctl -
./crsct stat res -t


Reference notes -


ASM Diskgroup Can Not Be Shown When Creating Database With DBCA (Doc ID 1269734.1)
DBCA Does Not Display ASM Disk Groups In 11.2 (non-Windows environments) (Doc ID 1177483.1)
Remote Diagnostic Agent (RDA) - Getting Started (Doc ID 314422.1)
ORA-01261: Parameter Db_create_online_log_dest_1 Destination String Cannot Be Translated (Doc ID 2369000.1)




Lessons learnt from fresh ebs r12.2 installation on Oracle Linux 7



1. Preinstaller rpms are great and your files are backed up under /var/log/oracle-ebs-server-R12-preinstall/backup
sysctl.conf
resolv.conf
grub.cfg

2. orakernel.log often ignored should be reviewed after installing both rpms
oracle-rdbms-server-12cR1-preinstall
oracle-ebs-server-R12-preinstall

3. applmgr and oracle have oinstall as common primary OS group. applmgr does not have dba as secondary group.

4. Starting from OL7, user limits are not stored in /etc/security/limits.conf. There will be .conf files created under /etc/security/limits.d 
that are read by systemd during server startup 

5. resolve.conf file will be overridden if you have DNS1/2 directives in your network configuration files -
/etc/sysconfig/network-scripts/ifcfg-*
You can make resolv.conf immutable to changes or remove directives DNS1/2 in your network configuration files.