Thursday, May 17, 2007

BPEL easy as it sounds?

In my previous post I discussed the basics of BPEL correlations. In this post, I want to present the challenge I faced trying to apply correlations to my business process, which unfortunately for me does not align with any vendor tutorials I have seen. Imagine that :)

The portion of my business process which requires correlation involves calling out to a notification web service and supplying an unbounded set of people to which I want to notify. This notification web service then responds with a single notification identifier (notification ID) representing the notification sent. At some future point in time, the people notified will respond (or the notification system will let me know there was no response). These responses can be received in any order at any time, and are received via a web service call into my ESB. The payload of this web service call includes the notification ID and the response of the person. After receiving the response, my business process updates the person information with the response and continues on. This is exactly what correlations are made for. Perfect!

Challenge 1: If I define correlation initialization on the notification web service invocation activity, this will persist 1 instance of the business process. But I have multiple responses coming back. After the first response is received, I have no business process instances waiting.

Solution 1a: I'll just loop around the web service invocation for each person in my set of people and make individual calls out to the notification web service. This will create multiple notification IDs and therefore persist multiple business process instances. So as responses come back, each one will correspond to its own instance. The business process would look something like this:

Problems: First, the business process will always run to completion as the response web service implementation gets invoked after the while loop (i.e. there is no receive task for the process to wait on). So, if we replace the web service implementation with a JMS receive task we can get over that hurdle. All we have to do is create another business process that implements a web service, and within the implementation it uses a JMS send activity to send the notification response to the corresponding JMS queue that this process is monitoring. The new business processes look like this:

Now the problem is that we really only have one business process instance being created. Although it seems like multiple business process instances are being persisted, each one really maps to the same instance. So, once the 1st response is received, the persisted instance is gone and other responses are left hanging.

Solution 1b: So, we remove the loop here and will need to create another business process (we are now up to 3 business processes) that sends messages with a single person to the inbound JMS queue. This will cause a new business process instance to be created for each person, so each response can now be correlated to one of these instances.

Challenge 2: How do we define the correlation set so that it actually works? Our correlation aliases were initially defined as the notification ID returning from the notification web service invocation and the JMS receive message text. Since the message text value (an XML message) does not match the notification ID (an int data type), the correlation is not working.

Solution: This was a difficult one. I went through many iterations with problems....err challenges encountered with each one. Here they are:

(1) Add an unmarshal task so that we can correlate on the resulting notification ID value. This will require using correlation on the Unmarshal activity. Since the JMS receive activity, which occurs prior to the unmarshal, causes the BPEL engine to retrieve an instance from persistence, and no correlation set is defined for this task, the engine grabs any instance. So essentially there is no real correlation going on.

(2) Try using the JMS message header correlation ID. This involved mapping the response notification ID to the correlation ID of the JMS message header prior to placing this on the response queue (in our second business process). The issue here was the correlation ID message property is defined as a string, while the notification ID is an int. So CAPS will not let you define an appropriate correlation key/correlation set that will work. The data types of each alias need to be the same, which makes sense. Also, since my business process has 2 JMS receive tasks, the BPEL engine gets confused when trying to create the instance identifier using the correlation ID of the JMS message header.

(3) Call for help. I discussed this problem with someone I know from Sun and he was able to provide a pre-release of some documentation that provided a good amount of insight into how to deal with correlations. The result of this new found knowledge was the creation of one more business process (for a total of 4) that actually had 2 JMS receive tasks. I could then create a correlation set based on the JMS message header correlationID and use correlations on each of the JMS receive tasks. The following images show 3 of the 4 business processes (the one not shown is simply a loop that places messages on the inbound JMS queue for the notification web service invocation process).

Notification Web Service Invocation
Notification Response Web Service Implementation

Combined Business Process Using Correlation

First, a message is placed on the inbound queue of the notification web service invocation process. This calls the external web service and uses the response to place a message on the correlation.queue JMS queue. Also, the notification ID that is the response from the web service invocation is set to the JMS message header correlationID property.

Recall that the Combined Business Process has a correlation set defined to be the correlationID message header property. The 2 JMS receive activities are set to use correlations. The first receive task initializes it and the second uses it.

So, once the first message is received, an instance of the Combine Business Process is created and then persisted (since it is now waiting on the second JMS receive). Now, at some future point in time, a person responds to the notification (or the service responds with "no response"). The response includes the notification ID and the response. The Notification Response Web Service Implementation maps the notification ID to the correlationID message property and places the response on the response.queue.

The second JMS receive task on the Combined Business Process is using this queue. This causes the BPEL engine to retrieve the business process instance that corresponds to the notification ID contained within the persons response. The process instance is retrieved (along with its state), and the business process continues to completion. This implementation of correlation finally worked.

From initial design to final implementation took longer (and was more painful) than it sounds. As a result I got a great understanding of how correlation works within CAPS and I am confident that this knowledge will transfer to other products as well.


Ron said...

Hi Steve,

After reading through your posts here, I would recommend (re?-)reading the current version of the WS-BPEL 2.0 Standard. Most of the questions you seem to have run into might be answered via a deeper familiarity with BPEL itself. For instance, to my knowledge there is nothing in the standard that precludes BPEL from supporting RPC/Encoded. We (ActiveBPEL) do it all the time. That said, it is not WS-Interoperability Basic profile 1.1 compliant, but that's a different issue.

Correlation and correlation sets are difficult for everyone - especially if you're coming at them from the messaging world. Again, your best bet there is to completely familiarize yourself with what's in the standard, and not rely on any one vendor's tools to demonstrate what they are or how to use them.

Finally, your last post indicates that your process is getting more complex than necessary. I'd suggest looking into the Scope or Process level On Event event handler to asynchronously receive your notification acknowledgments. Using that, you'll only need one process, and one instance of that process per meeting notice.

TechnoMind said...

Hi Steve,

I am new to SOA and for that matter to Glassfish ESB as well.

What I Intend: I am trying to route my call to a web service having multiple operations VIA Glassfish ESB.

Problems: There are a few problems for whom I have not found any answer to the net. Hope I will end my quest here.

Problem 1: Is it mandatory to have as many BPEL processes in ths ESB as the number of operations in my WSDL? I have eg. 2 operations in my WSDL. I have configured 2 processes for each operation. The first operation works fine but the second operation process gives me error saying:
[b]BPJBI-7017:Could not find a business process providing service for the endpoint specified in the message exchange[/b]. Service name: {}PartnerLink1, endpoint name: ExternalInterfaceSoap_ESBRole_myRole, operation name: ModifyAccount"

I think the BPEL service engine does not recognise the 2nd BPEL process meant for the desired operation. What to do in this case?

Vaneet Bansal

Steve Smith said...

I am not entirely clear on your question.

BPEL processes refer to external WSDLs via the partner link. This external WSDL will have some implementation behind it to provide the service specified in the WSDL.

So, if you have a WSDL that defines several operations, your BPEL process can execute any of the operations as required by your BPEL process. It is not a 1 to 1 mapping. The BPEL process can orchestrate calls out to numerous service providers (or partners).