BEA-010213, Message-Driven EJB: MyProcessMDB’s transaction was rolled back

I encountered this error last week and it was quite a headache to find out what the source of the problem was.

Short summary:

I thought the BEA-010213 error and the rollback log entries meant that we had a database problem. This was a wrong assumption, there’s another storage type in weblogic domains which use transactions (or state transitions?), which are JMS queues, which live in persistent stores in the managed servers.
That was the source of our problem; our persistent stores became corrupted because of storage problems.

Continue reading “BEA-010213, Message-Driven EJB: MyProcessMDB’s transaction was rolled back”

Weblogic, 1 managed server gives HTTP Error 401 Unauthorized, other works fine.

I encountered this weird behaviour last week where one managed server in a cluster of two servers constantly gave the error “HTTP Error 401 Unauthorized“.

The setup was a standard Oracle Service Bus installation on Oracle Weblogic.
The domain consisted out of one Admin server, one cluster with two managed servers.

Managed Server #1 was acting without problems but all the requests which needed authentication and which where pointed to Managed Server #2 failed and responded with a 401 Unauthorized message.

There was an error in the logs of MS#2, which is displayed at the bottom of this post as well.

#### <> <Inbound http BASIC authentication failed
javax.security.auth.login.FailedLoginException: [Security:090304]Authentication Failed: User webhosting javax.security.auth.login.FailedLoginException: [Security:090302]Authentication Failed: User webhosting denied

The solution to this problem lies in the internal LDAP configuration of the faulty Managed Server. I did not find an answer to what might have caused this problem, but the solution was to rebuild the LDAP setting of the Managed Server.

This is done by following these steps:

  1. Shut down Managed Server via Weblogic Console
  2. Log in via SSH
  3. Rename the following folder: %domain_directory%/servers/%osb_managed_server_1%/data/ldap
  4. Start the Managed Server via Weblogic Console

These steps will rebuild the LDAP folder, which is the internal LDAP to which Weblogic authenticates.

You can then remove the folder you’ve backed up in step 3.

Continue reading “Weblogic, 1 managed server gives HTTP Error 401 Unauthorized, other works fine.”

Dynamic HTTP endpoint in Oracle Service Bus 12c based on values in a database routing table

This article outlines how to set a dynamic endpoint in an OSB HTTP Business Service. The endpoint is retrieved from a routing table which resides in an Oracle 12c database.

Components used for this solution:

  1. Ubuntu Linux 14.04 64bit
    1. JDeveloper, running the Quick Start Oracle Fusion Middleware suite
      1. Oracle Service Bus 12c
      2. Oracle Weblogic 12c
      3. OSB Project location:
        1. https://github.com/visscher/Fusion/tree/master/DBRouting
    2. Oracle Virtualbox Developer Days image for DB 12c, running:
      1. Oracle Database 12c
    3. Oracle SQL Developer 4

This picture shows the running solution in the OSB test console:

Oracle Service Bus Console 12c : Pipeline Testing - DBRouting_v1Pipeline - Google Chrome_019

Database table preparation

We need a routing table in our schema, I’m using this table setup:

CREATE TABLE "C##JORIS"."ROUTINGTABLE"
  (
    "ROUTE" VARCHAR2(50),
    "ENDPOINT" VARCHAR2(100)
  );
Where ROUTE stands for the identifier and ENDPOINT is the HTTP endpoint we try to reach.
I’ve inserted two rows:
Insert into ROUTINGTABLE (ROUTE,ENDPOINT) values ('SalesOrder','http://localhost:7101/salesEndpoint');
Insert into ROUTINGTABLE (ROUTE,ENDPOINT) values ('FinanceReceipt','http://localhost:7101/financeEndpoint');
These two endpoints will point to very simple OSB services which we will create in a moment.
Selection_022

Weblogic configuration: JDBC Data Source

We need to configure a JDBC data source in our Weblogic server, this data source is used by the XQuery function to execute SQL.
Start JDEVeloper, select your integrated Weblogic Server and start it up.
When your domain is started, open the WLS Console:
http://127.0.0.1:7101/console/
Login and open the Data Sources summary:
Selection_020
Navigation in Console: DefaultDomain - Services - Data Sources
Create a new datasource, in my example I use the JNDI name “LocalDB
When you’re done with the configuration, test the datasource to make sure all is well:
Selection_021
The status message will be green and show a check mark if you’ve configured your data source correctly.

JDeveloper: Oracle Service Bus project

If you import the DBRouting project from here, you should have all the necessary services.
I will only discuss the assign steps which are needed in the DBRouting_v1Pipeline.pipeline.
Selection_023
There are three assign actions:
1.) Assign $route: node-name($body/*[1])

This assign determines our routing key. It is the same key as 
the first column in the routing table.
The XPath here is used to select the name of the first node 
but you can change this to what you want to route on.
2.) Assign $query: 
fn:concat("select ENDPOINT from ROUTINGTABLE where ROUTE = '", $route, "'")

This assign determines the query which will be executed in 
the next step. We want to select the ENDPOINT which belongs 
to the ROUTE which was assigned in step 1.
3.) Assign $query: 
(fn-bea:execute-sql(
 xs:string("LocalDB"), 
 xs:string("ENDPOINT"), 
 $query
 )/*:ENDPOINT)[1]

This assign actually executes the SQL query to our database, 
which is the first argument.
The second argument names the re-occurring rows, in this 
case "ENDPOINT".
The thirst argument is the query to execute.
The XPath after the execute-sql statement is to make sure 
we only get one endpoint.
4.) After those assigns, we use place a task “Routing Options” in the HTTP Route node:
We only use the “URI” Routing Option:
$endpoint/text()
Selection_024
This ends the article, if you execute the pipeline you will see the endpoint has become dynamic, it is retreived from the routing table:
Oracle Service Bus Console 12c : Pipeline Testing - DBRouting_v1Pipeline - Google Chrome_019

Weblogic does not start, java.lang.NumberFormatException, EmbeddedLDAP

I encountered the following exception when trying to start our AdminServer today:

<May 30, 2013 1:04:27 PM CEST> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: java.lang.NumberFormatException: null
java.lang.NumberFormatException: null
 at java.lang.Integer.parseInt(Integer.java:454)
 at java.lang.Integer.parseInt(Integer.java:527)
 at weblogic.ldap.EmbeddedLDAP.validateVDEDirectories(EmbeddedLDAP.java:1104)
 at weblogic.ldap.EmbeddedLDAP.start(EmbeddedLDAP.java:242)
 at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
 Truncated. see log file for complete stacktrace
> 
<May 30, 2013 1:04:27 PM CEST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED> 
<May 30, 2013 1:04:27 PM CEST> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down> 
<May 30, 2013 1:04:27 PM CEST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>

This does not really explain what went wrong, but the solution is to remove the replicas.prop file and restart the AdminServer again.

The replicas.prop file is located at :

$DOMAIN/servers/AdminServer/data/ldap/conf/replicas.prop

Rename (move) replicas.prop to replicas.prop.bak and retry the startWeblogic. In my case this solved the not starting server.