node manager inactive status with error javax.net.ssl.SSLHandshakeException

 

Nodemanger startup script has started node manager and it started listening on 5556 port but trying to start managed server was throwing error.
 
Checking log manager status from nodemanager monitoring tab was showing nodemanager status as inactive with a problem discritpion javax.net.ssl.SSLHandshakeException


Admin server keypair on KSS has expired this was causing the issue. Keypair can be renewed using em console



Admin server log


] [partition-name: DOMAIN] > <BEA-090171> <Loading the identity certificate and private key stored under the alias DemoIdentity from the kss keystore fil

e kss://system/demoidentity.>

####<Apr 30, 2023 10:41:41,019 AM AST> <Info> <JMS> <tfictapps.test.com> <AdminServer> <[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <7c5c05c1-09ae-410f-8280-57d9ca8f0c13-00000002> <1682840501019> <[severity-value: 64] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-040305> <JMS service is initialized and in standby mode.>

####<Apr 30, 2023 10:41:41,353 AM AST> <Info> <JMS> <tfictapps.test.com> <AdminServer> <[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <7c5c05c1-09ae-410f-8280-57d9ca8f0c13-00000002> <1682840501353> <[severity-value: 64] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-040090> <Deployed 8 default connection factories.>

####<Apr 30, 2023 10:41:41,391 AM AST> <Alert> <Security> <tfictapps.test.com> <AdminServer> <[STANDBY] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <7c5c05c1-09ae-410f-8280-57d9ca8f0c13-00000006> <1682840501391> <[severity-value: 2] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-090154> <Identity certificate has expired: [

[

Version: V3

Subject: CN=DemoCertFor_tfict_dmn

 

Admin server log


] [partition-name: DOMAIN] > <BEA-090171> <Loading the identity certificate and private key stored under the alias DemoIdentity from the kss keystore fil

e kss://system/demoidentity.>

####<Apr 30, 2023 10:41:41,019 AM AST> <Info> <JMS> <tfictapps.test.com> <AdminServer> <[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <7c5c05c1-09ae-410f-8280-57d9ca8f0c13-00000002> <1682840501019> <[severity-value: 64] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-040305> <JMS service is initialized and in standby mode.>

####<Apr 30, 2023 10:41:41,353 AM AST> <Info> <JMS> <tfictapps.test.com> <AdminServer> <[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <7c5c05c1-09ae-410f-8280-57d9ca8f0c13-00000002> <1682840501353> <[severity-value: 64] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-040090> <Deployed 8 default connection factories.>

####<Apr 30, 2023 10:41:41,391 AM AST> <Alert> <Security> <tfictapps.test.com> <AdminServer> <[STANDBY] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <7c5c05c1-09ae-410f-8280-57d9ca8f0c13-00000006> <1682840501391> <[severity-value: 2] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-090154> <Identity certificate has expired: [

[

Version: V3

Subject: CN=DemoCertFor_tfict_dmn

Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11


Key: SunPKCS11-Solaris RSA public key, 2048 bits (id 4339940272, session object)

modulus: 21408857851587150114404800711916396981234035324566008254053351703135484889399951208979218644764266450588661516780376730237840558111101324774037789131266182846581901973783538821981750745180286668961190097518386861892017337639883815271532776343865226767693910870969605579726433117738210963922929769064994204182927127052801356133915404947456368142590302429619930586437089279527372384271953898170922916960021080734084952567756874047878559665825520572591198777888093123454491326111170011337244408804777405506274497243147110630201931525871672589221434271484445198122800461138353744077919118956045712068414287816997002855591

public exponent: 65537

Validity: [From: Thu Apr 05 12:49:35 AST 2018,

To: Tue Apr 04 12:49:35 AST 2023]

Issuer: CN=CertGenCA, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US

SerialNumber: [ 01629536 2204]

 

Weblogic log


<Apr 29, 2023 3:39:11,624 PM AST> <Notice> <Security> <BEA-090171> <Loading the identity certificate and private key stored under the alias DemoIdentity from the kss keystore file kss://system/demoidentity.>

Apr 29, 2023 3:39:11 PM oracle.security.opss.internal.runtime.ServiceContextManagerImpl getContext

WARNING: Bootstrap services are used by OPSS internally and clients should never need to directly read/write bootstrap credentials. If required, use Wlst or configuration management interfaces.

<Apr 29, 2023 3:39:12,080 PM AST> <Alert> <Security> <BEA-090154> <Identity certificate has expired: [


 

Both Admin log and weblogic had error "Identity certificate has expired"

 

Use FMW em conosle to renew the certificate

 

1. Log in to em console

2. Weblogic domain > Security > Keystore

 

 

 

Click on keystore 

 

this will bring up keystore page

Select the demoidentiy and click manage

identity store password is required for default identity use "DemoIdentityKeyStorePassPhrase"

 

take a screen shot of existing for reference

select the expired certificate and delete. the  password for default "DemoIdentityPassPhrase"

 

Generate Keypair

Make sure you are providing same information as before

alias: DemoIdentity

Subject Name CN=DemoCertFor_tdept5_domain
Password as : DemoIdentityPassPhrase


New keypair is generated with an 5 year expiration period


Deleted entries are shown on the screen as long as we are on this page, once moved to different page and coming back will only show certificates which are not deleted


Take a backup of domain folder

sync system KSS keystore


apps@tfictapps:/u01/app/apps/middleware/product/12.2.1/oracle_common/common/bin$ ./wlst.sh

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server Administration Scripting Shell


Type help() for help on available commands


wls:/offline> connect()

Please enter your username :weblogic

weblogic

Please enter your password :

Please enter your server URL [t3://localhost:7001] :


Connecting to t3://localhost:7001 with userid weblogic ...

Successfully connected to Admin Server "AdminServer" that belongs to domain "tfict_dmn".


Warning: An insecure protocol was used to connect to the server.

To ensure on-the-wire security, the SSL port or Admin port should be used instead.


wls:/tfict_dmn/serverConfig/> syncKeyStores(appStripe='system', keystoreFormat='KSS')

Location changed to domainRuntime tree. This is a read-only tree

with DomainMBean as the root MBean.

For more help, use help('domainRuntime')


Keystore sync successful.

 

KSS keystore certificate can be checked with wlst as well with below command

 

wls:/offline> connect()

Please enter your username :weblogic

weblogic

Please enter your password :

Please enter your server URL [t3://localhost:7001] :


Connecting to t3://localhost:7001 with userid weblogic ...

Successfully connected to Admin Server "AdminServer" that belongs to domain "tfict_dmn".


Warning: An insecure protocol was used to connect to the server.

To ensure on-the-wire security, the SSL port or Admin port should be used instead.


wls:/tfict_dmn/serverConfig/> svc = getOpssService(name='KeyStoreService')

wls:/tfict_dmn/serverConfig/> svc.listKeyStores(appStripe='*')

Location changed to domainRuntime tree. This is a read-only tree

with DomainMBean as the root MBean.

For more help, use help('domainRuntime')


system/trust

system/demoidentity

system/castore

system/publiccacerts

opss/trustservice_ts

opss/trustservice_ks

wls:/tfict_dmn/domainRuntime/> svc.getKeyStoreCertificates(appStripe='system', name='demoidentity', password='DemoIdentityKeyStorePassPhrase', alias='DemoIdentity')


 

[

[

Version: V3

Subject: CN=DemoCertFor_tfict_dmn

Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11


Key: SunPKCS11-Solaris RSA public key, 2048 bits (id 4366650656, session object)

modulus: 20379721393075119294342523920733124149526129157306822816689239529207787209461790849731737776234409712668665041731880858542757631161516102061268200694326006329095242213090475880686213820655344095057857225151036339065592830486963521083067630769019901881680023059664634682081997897857794303568029329022490726981605589736972450407560568875024153767833732903271219481637866963158480254020115670064108614500835150642396034913549892097560012500228178889805701207213707288446778119476219588507011094722810271316820206398873895091020950181009222149259774303243516363118011489609706764537554758126642888814787270875642309976079

public exponent: 65537

Validity: [From: Sun Apr 30 16:27:40 AST 2023,

To: Fri Apr 28 16:27:40 AST 2028]

Issuer: CN=CertGenCA, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US

SerialNumber: [ 0187d258 ff60]

 the node manager status is reachable now






Comments

Post a Comment

Popular posts from this blog

APP-FND-01436: List of Values cannot find any values for you to choose error from concurrent request program parameter (R12 12.1.3)

Image
APP-FND-01436: List of Values cannot find any values for you to choose error from concurrent request program parameter After upgrading oracle application 11i(11.5.10.2) to R12(12.1.3) running concurrent program request caused this issue When concurrent program is run for a concurrent request after upgrading to R12 12.1.3 the lov, which is attached to a parameter, gives APP-FND-01436: “List of Values cannot find any values for you to choose”. When the same request is run from oracle applications 11i 11.5.10.2 the same value set is working fine. The value set is based on a table. If you go and test the value set query from the value set definition screen it returns complete successful message yet if you go to the parameter of the concurrent request program form where the particular value set is used it gives you the above error. After searching google and oracle support I found metalink note 1400963.1, EBS R12.1: Parameter for Concurrent Request Using Table Based LOV G...
Read more

WEB ADI - issues encountered during initial access

Image
On accessing create document link under Desktop Integrator responsibility browser was showing following error Set BNE UIX Physical Directory profile value to proper directory at site level Location = $COMMON_TOP/html/cabo/  (11i) Location = $COMMON_TOP/webapps/oacore/html/cabo/  (R12) As this was a cloned instance the value of this profile setting was pointing to a wrong location. set as BNE UIX Physical Directory /u03/oracle/prodcomn/html/cabo/ Access to functions are controlled by function security so to access and work with web adi the user should have proper access. You should have access to particular integrator to access and work with it Find out responsibility which is created to work with web adi check the main menu for this responsibility. Modify the responsibility or Menu if default is not fitting to your needs. For instance Desktop Integration Menu has create document, Define Lay out, Define Mapping, Manage Documents, Setup options Note : you...
Read more

Error processing request - Contact your application administrator apex 20.1

Image
After upgrading 11.2.0.4 to 19c with apex 20.1 accessing apex_admin/apex pages was throwing “Error processing request” Contact your application administrator. There were three invalid objects for APEX_200100 WWV_FLOW_SESSION_RAS WWV_FLOW_WIZARD_API WWV_FLOW_DATA_LOADER Tried compiling these objects but compile was completing with PL/SQL: ORA-00942: table or view does not exist To resolve the issue following rant needs to be given then compile (ran both grant and compile as sysdba) grant select on SYS.DBA_XS_DYNAMIC_ROLES to APEX_200100; grant select on SYS.DBA_TAB_IDENTITY_COLS to APEX_200100; alter package APEX_200100.WWV_FLOW_SESSION_RAS compile body; alter package APEX_200100.WWV_FLOW_WIZARD_API compile body; alter package APEX_200100.WWV_FLOW_DATA_LOADER compile body; Returned 0 invalid object for APEX_200100  select * from dba_objects where status = 'INVALID' and owner = 'APEX_200100' ; Now both admin and apex pages ar...
Read more