EXADATA一体机新建数据库遇到的报错ORA-00603 AND ORA-27515
j近期在一个新的EXADATA环境19.25版本,创建了数据库后,观察到后台ALERT日志中出现有ORA-00603 AND ORA-27515报错,报错信息如下:Errors in file /u01/app/oracle/diag/rdbms/xxxx/xxxx1/trace/xxxx1_ora_9460.trc (incident=1737933) (PDBNAME=CDB$ROOT):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27515: inadequate memlock limit or driver settings
Incident details in: /u01/app/oracle/diag/rdbms/xxxx/xxxx1/incident/incdir_1737933/xxxx1_ora_9460_i1737933.trc
2020-03-24T01:03:25.650686-04:00
opiodr aborting process unknown ospid (9460) as a result of ORA-603
对应截图如下,数据库运行正常未受影响:
怀疑是BUG,在MOS上搜索可以看到文档:EXADATA : ORA-00603 AND ORA-27515 during EXAChk run with Autonomous Health Framework (AHF) 20.1.1. (Doc ID 2662144.1)、Instance crashes with ORA-00603, ORA-27515 (Doc ID 2700814.1)对此问题有描述:问题原因是AHF BUG引起的,按照workaround可以打补丁或者禁用AHF解决;考虑到AHF用的不多,这里通过禁用AHF,后续观察没有再报错。
Applies to:
Gen 1 Exadata Cloud at Customer (Oracle Exadata Database Cloud Machine) - Version N/A to N/A
Information in this document applies to any platform.
Symptoms
Instance failure with following errors reported in alert log.
2020-07-14T01:11:58.695144-05:00
Archived Log entry 1275 added for T-1.S-2367 ID 0x2c5d558c LAD:1
2020-07-14T01:13:53.787461-05:00
Errors in file /u02/app/oracle/diag/rdbms/cd1sfm/cd1sfm1/trace/cd1sfm1_ora_167850.trc (incident=770730) (PDBNAME=CDB$ROOT):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27515: inadequate memlock limit or driver settings
Incident details in: /u02/app/oracle/diag/rdbms/cd1sfm/cd1sfm1/incident/incdir_770730/cd1sfm1_ora_167850_i770730.trc
.....
2020-07-14T01:14:58.631500-05:00
opiodr aborting process unknown ospid (167850) as a result of ORA-603
2020-07-14T01:17:23.410311-05:00
Errors in file /u02/app/oracle/diag/rdbms/cd1sfm/cd1sfm1/trace/cd1sfm1_ora_172703.trc (incident=770475) (PDBNAME=SFDAT):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27515: inadequate memlock limit or driver settings
SFDAT(3):Incident details in: /u02/app/oracle/diag/rdbms/cd1sfm/cd1sfm1/incident/incdir_770475/cd1sfm1_ora_172703_i770475.trc
2020-07-14T01:17:25.642557-05:00
SFDAT(3):opiodr aborting process unknown ospid (172703) as a result of ORA-603
2020-07-14T01:19:19.671875-05:00
Cause
Internal Bug 31118086 involving TFA
Solution
AHF version 20.1.3 or above contains the fix.
Please upgrade to the latest available versions of AHF
===
Applies to:
Oracle Database - Enterprise Edition - Version 12.2.0.1 to 19.6.0.0.0 [Release 12.2 to 19]
Linux x86-64
Symptoms
We see following errors in the RAC Databases :
Excerpts from database Alert log:
Errors in file /u01/app/oracle/diag/rdbms/xxxx/xxxx1/trace/xxxx1_ora_9460.trc (incident=1737933) (PDBNAME=CDB$ROOT):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27515: inadequate memlock limit or driver settings
Incident details in: /u01/app/oracle/diag/rdbms/xxxx/xxxx1/incident/incdir_1737933/xxxx1_ora_9460_i1737933.trc
2020-03-24T01:03:25.650686-04:00
opiodr aborting process unknown ospid (9460) as a result of ORA-603
oracle$ ulimit -a
max locked memory (kbytes, -l) unlimited
root# ulimit -a
max locked memory (kbytes, -l) 32768
Changes
Autonomous Health Framework (AHF) 20.1.1 Installed
exachk -v
EXAchk Version 20.1.1_20200317 <<<<<<<<<
Cause
ORA-27515 is hit when EXAChk is either run manually or TFA initiated.
The issue is due to following unpublished EXAChk bug:
Bug 31118086 - AHF 20.1.1: ORA-00603, ORA-27515 OCCURRED ONLY DURING EXACHK RUN
Solution
Bug 31118086 is fixed in AHF 20.1.3.
Upgrade to Autonomous Health Framework (AHF) 20.1.3 version.
= OR =
Use the following workaround:
#exachk -autostop
#tfactl stop