Weblogic, 1 managed server gives HTTP Error 401 Unauthorized, other works fine.

I encountered this weird behaviour last week where one managed server in a cluster of two servers constantly gave the error “HTTP Error 401 Unauthorized“.

The setup was a standard Oracle Service Bus installation on Oracle Weblogic.
The domain consisted out of one Admin server, one cluster with two managed servers.

Managed Server #1 was acting without problems but all the requests which needed authentication and which where pointed to Managed Server #2 failed and responded with a 401 Unauthorized message.

There was an error in the logs of MS#2, which is displayed at the bottom of this post as well.

#### <> <Inbound http BASIC authentication failed
javax.security.auth.login.FailedLoginException: [Security:090304]Authentication Failed: User webhosting javax.security.auth.login.FailedLoginException: [Security:090302]Authentication Failed: User webhosting denied

The solution to this problem lies in the internal LDAP configuration of the faulty Managed Server. I did not find an answer to what might have caused this problem, but the solution was to rebuild the LDAP setting of the Managed Server.

This is done by following these steps:

  1. Shut down Managed Server via Weblogic Console
  2. Log in via SSH
  3. Rename the following folder: %domain_directory%/servers/%osb_managed_server_1%/data/ldap
  4. Start the Managed Server via Weblogic Console

These steps will rebuild the LDAP folder, which is the internal LDAP to which Weblogic authenticates.

You can then remove the folder you’ve backed up in step 3.

To be complete; This is the full error in the log of the faulty managed server:

####<Feb 20, 2015 10:35:20 AM CET> <Error> <WliSbTransports> <bctest61.rdc.local> <esb11g_osb_tst_server_1> <[ACTIVE] ExecuteThread: '14' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <e718323cf85f0ef3:57b17433:14ba1f26810:-7c68-00000000000020e6> <
1424424920363> <BEA-381328> <Inbound http BASIC authentication failed
javax.security.auth.login.FailedLoginException: [Security:090304]Authentication Failed: User webhosting javax.security.auth.login.FailedLoginException: [Security:090302]Authentication Failed: User webhosting denied
        at weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.login(LDAPAtnLoginModuleImpl.java:261)
        at com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:110)
        at java.security.AccessController.doPrivileged(Native Method)
        at com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:106)
        at sun.reflect.GeneratedMethodAccessor738.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at javax.security.auth.login.LoginContext.invoke(LoginContext.java:769)
        at javax.security.auth.login.LoginContext.access$000(LoginContext.java:186)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:683)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
        at javax.security.auth.login.LoginContext.login(LoginContext.java:579)
        at com.bea.common.security.internal.service.JAASLoginServiceImpl.login(JAASLoginServiceImpl.java:113)
        at sun.reflect.GeneratedMethodAccessor736.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at com.bea.common.security.internal.utils.Delegator$ProxyInvocationHandler.invoke(Delegator.java:57)
        at $Proxy36.login(Unknown Source)
        at weblogic.security.service.internal.WLSJAASLoginServiceImpl$ServiceImpl.login(WLSJAASLoginServiceImpl.java:89)
        at com.bea.common.security.internal.service.JAASAuthenticationServiceImpl.authenticate(JAASAuthenticationServiceImpl.java:82)
        at sun.reflect.GeneratedMethodAccessor743.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at com.bea.common.security.internal.utils.Delegator$ProxyInvocationHandler.invoke(Delegator.java:57)
        at $Proxy54.authenticate(Unknown Source)
        at weblogic.security.service.WLSJAASAuthenticationServiceWrapper.authenticate(WLSJAASAuthenticationServiceWrapper.java:40)
        at weblogic.security.service.PrincipalAuthenticator.authenticate(PrincipalAuthenticator.java:338)
        at com.bea.wli.sb.security.WLSAuthenticationServiceImpl.authenticate(WLSAuthenticationServiceImpl.java:47)
        at com.bea.wli.sb.security.authentication.BoundedTTLAuthenticationCache.authenticate(BoundedTTLAuthenticationCache.java:129)
        at com.bea.wli.sb.security.CachedWLSAuthenticationService.authenticate(CachedWLSAuthenticationService.java:72)
        at com.bea.wli.sb.transports.http.wls.HttpBasicAuthRuntime.doInboundAuthentication(HttpBasicAuthRuntime.java:101)
        at com.bea.wli.sb.transports.http.generic.RequestHelperBase.enforceSecurity(RequestHelperBase.java:240)
        at com.bea.wli.sb.transports.http.generic.RequestHelperBase.service(RequestHelperBase.java:74)
        at com.bea.wli.sb.transports.http.wls.HttpTransportServlet.service(HttpTransportServlet.java:129)
        at weblogic.servlet.FutureResponseServlet.service(FutureResponseServlet.java:24)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
        at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301)
        at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:184)
        at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3741)
        at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3705)
        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
        at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
        at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2282)
        at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2181)
        at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1491)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:252)
        at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Advertisements

One thought on “Weblogic, 1 managed server gives HTTP Error 401 Unauthorized, other works fine.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s