Dear Friends,
We have a very peculiar case here with our PI system.
Whenever we go complete restart(ABAP+JAVA).....our JAVA stack fails everytime. No matter what, it wont come.
All the server nodes fail immediately during 'starting framework' phase itself.
Interesting thing is, as a workaround, we go for server reboot everytime and then JAVA comes up without any problem.
Please note that this is not the case when we go for only JAVA restart i.e. there is no problem while restarting JAVA stack alone.
Problem only occurs whenever we go for complete restart.
I have went through possible each and every thread on this in SCN, but no help.
We are in dire need of any help.
Please find below the default trace of server node.
Could not register com.sap.engine.session.mbeans.DatashareMBean. Exception is: java.lang.InstantiationException: com.sap.engine.session.mbeans.MonitorSessionsOpenMBe
an#
#2.#2014 04 11 20:01:18:832#+0200#Error#com.sap.engine.core.service630.container.DescriptorHelperImpl#
com.sap.ASJ.krn_srv.000002#BC-JAS-COR#kernel.sda#C0003547E44E00020000000100000AF9#602208850000000023##com.sap.engine.core.service630.container.DescriptorHelperImpl##
###436fd3bac1a311e38fb6005056ba50f8##0#Service Runner [deploy]#Plain##
Error reading [containers-info.xml] file entry from [tc~je~cfs] component [descriptor] container#
#2.#2014 04 11 20:01:18:832#+0200#Error#com.sap.engine.core.service630.container.DescriptorHelperImpl#
#BC-JAS-COR#kernel.sda#C0003547E44E00020000000200000AF9#602208850000000023##com.sap.engine.core.service630.container.DescriptorHelperImpl#####436fd3bac1a311e38fb6005
056ba50f8##0#Service Runner [deploy]#Plain##
Error reading [containers-info.xml] file entry from [tc~je~cfs] component [descriptor] container
[EXCEPTION]
com.sap.engine.lib.config.api.exceptions.ClusterConfigurationException: The entry with the specified name containers-info.xml cannot be retrieved in tc~je~cfs/descri
ptors.
at com.sap.engine.lib.config.impl.component.PersistentEntryContainerImpl.getPersistentEntryAsStream(PersistentEntryContainerImpl.java:62)
at com.sap.engine.core.service630.container.DescriptorHelperImpl.getPersistentEntryStream(DescriptorHelperImpl.java:57)
at com.sap.engine.services.deploy.server.cache.containers.Containers.addContainers(Containers.java:90)
at com.sap.engine.services.deploy.server.cache.containers.Containers.initContainers(Containers.java:197)
at com.sap.engine.services.deploy.server.DeployServiceFrame.init(DeployServiceFrame.java:175)
at com.sap.engine.services.deploy.server.DeployServiceFrame.start(DeployServiceFrame.java:56)
at com.sap.engine.core.service630.container.ServiceRunner.startFrame(ServiceRunner.java:165)
at com.sap.engine.core.service630.container.ServiceRunner.startService(ServiceRunner.java:118)
at com.sap.engine.core.service630.container.ServiceRunner.run(ServiceRunner.java:65)
at com.sap.engine.frame.core.thread.Task.run(Task.java:73)
at com.sap.engine.core.thread.impl5.SingleThread.execute(SingleThread.java:178)
at com.sap.engine.core.thread.impl5.SingleThread.run(SingleThread.java:316)
Caused by: com.sap.engine.frame.core.configuration.ConfigurationException: Permission denied
at com.sap.engine.core.configuration.impl.ConfigurationImpl.getFileFromCopy(ConfigurationImpl.java:346)
at com.sap.engine.lib.config.impl.ConfigurationAccessor.getEntryStream(ConfigurationAccessor.java:2267)
at com.sap.engine.lib.config.impl.component.PersistentEntryContainerImpl.getPersistentEntryAsStream(PersistentEntryContainerImpl.java:58)
... 11 more
Caused by: java.io.IOException: Permission denied
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.checkAndCreate(File.java:1715)
at java.io.File.createTempFile0(File.java:1736)
at java.io.File.createTempFile(File.java:1813)
at com.sap.engine.core.configuration.impl.ConfigurationImpl.getFileFromCopy(ConfigurationImpl.java:292)
... 13 more
#
#2.#2014 04 11 20:01:18:839#+0200#Error#com.sap.engine.core.service630.container.DescriptorHelperImpl#
com.sap.ASJ.krn_srv.000002#BC-JAS-COR#kernel.sda#C0003547E44E00020000000400000AF9#602208850000000023##com.sap.engine.core.service630.container.DescriptorHelperImpl##
###436fd3bac1a311e38fb6005056ba50f8##0#Service Runner [deploy]#Plain##
Error reading [containers-info.xml] file entry from [tc~je~sca~integration] component [descriptor] container#
#2.#2014 04 11 20:01:18:839#+0200#Error#com.sap.engine.core.service630.container.DescriptorHelperImpl#
#BC-JAS-COR#kernel.sda#C0003547E44E00020000000500000AF9#602208850000000023##com.sap.engine.core.service630.container.DescriptorHelperImpl#####436fd3bac1a311e38fb6005
056ba50f8##0#Service Runner [deploy]#Plain##
Error reading [containers-info.xml] file entry from [tc~je~sca~integration] component [descriptor] container
[EXCEPTION]
com.sap.engine.lib.config.api.exceptions.ClusterConfigurationException: The entry with the specified name containers-info.xml cannot be retrieved in tc~je~sca~integr
ation/descriptors.
at com.sap.engine.lib.config.impl.component.PersistentEntryContainerImpl.getPersistentEntryAsStream(PersistentEntryContainerImpl.java:62)
at com.sap.engine.core.service630.container.DescriptorHelperImpl.getPersistentEntryStream(DescriptorHelperImpl.java:57)
at com.sap.engine.services.deploy.server.cache.containers.Containers.addContainers(Containers.java:90)
at com.sap.engine.services.deploy.server.cache.containers.Containers.initContainers(Containers.java:197)
at com.sap.engine.services.deploy.server.DeployServiceFrame.init(DeployServiceFrame.java:175)
at com.sap.engine.services.deploy.server.DeployServiceFrame.start(DeployServiceFrame.java:56)
at com.sap.engine.core.service630.container.ServiceRunner.startFrame(ServiceRunner.java:165)
at com.sap.engine.core.service630.container.ServiceRunner.startService(ServiceRunner.java:118)
at com.sap.engine.core.service630.container.ServiceRunner.run(ServiceRunner.java:65)
at com.sap.engine.frame.core.thread.Task.run(Task.java:73)
at com.sap.engine.core.thread.impl5.SingleThread.execute(SingleThread.java:178)
at com.sap.engine.core.thread.impl5.SingleThread.run(SingleThread.java:316)
Caused by: com.sap.engine.frame.core.configuration.ConfigurationException: Permission denied
at com.sap.engine.core.configuration.impl.ConfigurationImpl.getFileFromCopy(ConfigurationImpl.java:346)
at com.sap.engine.lib.config.impl.ConfigurationAccessor.getEntryStream(ConfigurationAccessor.java:2267)
at com.sap.engine.lib.config.impl.component.PersistentEntryContainerImpl.getPersistentEntryAsStream(PersistentEntryContainerImpl.java:58)
... 11 more
Caused by: java.io.IOException: Permission denied
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.checkAndCreate(File.java:1715)
at java.io.File.createTempFile0(File.java:1736)
at java.io.File.createTempFile(File.java:1813)
at com.sap.engine.core.configuration.impl.ConfigurationImpl.getFileFromCopy(ConfigurationImpl.java:292)
... 13 more
#
#2.#2014 04 11 20:01:18:845#+0200#Error#com.sap.engine.core.service630.container.DescriptorHelperImpl#
com.sap.ASJ.krn_srv.000002#BC-JAS-COR#kernel.sda#C0003547E44E00020000000700000AF9#602208850000000023##com.sap.engine.core.service630.container.DescriptorHelperImpl##
###436fd3bac1a311e38fb6005056ba50f8##0#Service Runner [deploy]#Plain##
Error reading [containers-info.xml] file entry from [jmsconnector] component [descriptor] container#
#2.#2014 04 11 20:01:18:845#+0200#Error#com.sap.engine.core.service630.container.DescriptorHelperImpl#
#BC-JAS-COR#kernel.sda#C0003547E44E00020000000800000AF9#602208850000000023##com.sap.engine.core.service630.container.DescriptorHelperImpl#####436fd3bac1a311e38fb6005
056ba50f8##0#Service Runner [deploy]#Plain##
Error reading [containers-info.xml] file entry from [jmsconnector] component [descriptor] container
[EXCEPTION]
com.sap.engine.lib.config.api.exceptions.ClusterConfigurationException: The entry with the specified name containers-info.xml cannot be retrieved in jmsconnector/des
criptors.
And finally, this message pops in trace
#2.#2014 04 11 20:01:24:187#+0200#Fatal#com.sap.engine.core.Framework#
com.sap.ASJ.krn_frm.000019#BC-JAS-COR#kernel.sda#C0003547E44E00020000005900000AF9#602208850000000830##com.sap.engine.core.Framework#####436fd3bac1a311e38fb6005056ba5
0f8##0#Service Runner [dbpool]#Plain##
Critical shutdown with exit code [2150] was requested due to [AS Java cannot be started; core service [dbpool] failed to start]#
#2.#2014 04 11 20:01:34:262#+0200#Warning#com.sap.engine.core.thread.impl3.ErrorQueueHandler#
com.sap.ASJ.krn_thd.000025#BC-JAS-COR#kernel.sda#C0003547E44E00000000000000001064###com.sap.engine.core.thread.impl3.ErrorQueueHandler#####51832c54c1a311e38428005056
ba50f8###Thread[ErrorQueueWatchDog,5,main]#Plain##
The following Error is thrown by the VM [java.lang.InternalError]. ErrorQueueHandler will just trace it. The caller component should take care to process it properly
.
java.lang.InternalError
#
#2.#2014 04 11 20:01:37:170#+0200#Warning#com.sap.engine.core.session.Manager#
com.sap.ASJ.ses.cf0023#BC-JAS-COR-SES#kernel.sda#C0003547E44E00010000000000001064###com.sap.engine.core.session.Manager########Thread[main,5,main]#Plain##
Could not register com.sap.engine.session.mbeans.SessDomRecMBean. Exception is: java.lang.InstantiationException: com.sap.engine.session.mbeans.MonitorSessionsOpenMB
ean#
#2.#2014 04 11 20:01:41:477#+0200#Error#com.sap.engine.core.service630.container.DescriptorHelperImpl#
com.sap.ASJ.krn_srv.000002#BC-JAS-COR#kernel.sda#C0003547E44E00020000000100001064#602208850000000021##com.sap.engine.core.service630.container.DescriptorHelperImpl##
###544c3391c1a311e3a22b005056ba50f8##0#Service Runner [deploy]#Plain##
Error reading [containers-info.xml] file entry from [component.info.service] component [descriptor] container#
#2.#2014 04 11 20:01:41:477#+0200#Error#com.sap.engine.core.service630.container.DescriptorHelperImpl#
#BC-JAS-COR#kernel.sda#C0003547E44E00020000000200001064#602208850000000021##com.sap.engine.core.service630.container.DescriptorHelperImpl#####544c3391c1a311e3a22b005
056ba50f8##0#Service Runner [deploy]#Plain##
Error reading [containers-info.xml] file entry from [component.info.service] component [descriptor] container
[EXCEPTION]
com.sap.engine.lib.config.api.exceptions.ClusterConfigurationException: The entry with the specified name containers-info.xml cannot be retrieved in component.info.s
ervice/descriptors.
at com.sap.engine.lib.config.impl.component.PersistentEntryContainerImpl.getPersistentEntryAsStream(PersistentEntryContainerImpl.java:62)
at com.sap.engine.core.service630.container.DescriptorHelperImpl.getPersistentEntryStream(DescriptorHelperImpl.java:57)
at com.sap.engine.services.deploy.server.cache.containers.Containers.addContainers(Containers.java:90)
at com.sap.engine.services.deploy.server.cache.containers.Containers.initContainers(Containers.java:197)
at com.sap.engine.services.deploy.server.DeployServiceFrame.init(DeployServiceFrame.java:175)
at com.sap.engine.services.deploy.server.DeployServiceFrame.start(DeployServiceFrame.java:56)
at com.sap.engine.core.service630.container.ServiceRunner.startFrame(ServiceRunner.java:165)
at com.sap.engine.core.service630.container.ServiceRunner.startService(ServiceRunner.java:118)
at com.sap.engine.core.service630.container.ServiceRunner.run(ServiceRunner.java:65)
at com.sap.engine.frame.core.thread.Task.run(Task.java:73)
at com.sap.engine.core.thread.impl5.SingleThread.execute(SingleThread.java:178)
at com.sap.engine.core.thread.impl5.SingleThread.run(SingleThread.java:316)
Caused by: com.sap.engine.frame.core.configuration.ConfigurationException: Permission denied
at com.sap.engine.core.configuration.impl.ConfigurationImpl.getFileFromCopy(ConfigurationImpl.java:346)
at com.sap.engine.lib.config.impl.ConfigurationAccessor.getEntryStream(ConfigurationAccessor.java:2267)
Kindly help.
Thanks
Message was edited by: Moderator