Sequential convoy - sort of

I wish to set up a sequential convoy pattern in an orchestration, but instead
of aggregating the messages, I want to process each message as it is
received.

For example, the orchestration receives the first message and immediately
begins processing it. As the message is processing, another correlated
message arrives and is queued. When the first message is completed processing
the second message is dequeued and begins processing. This pattern repeats
until no more messages have arrived.

Can anyone offer a fairly straightforward solution to this? How can I
simultaneously process and receive messages in the same orchestration
instance?

Thanks!

By it is very nature a sequential convoy will do this automatically.

By it is very nature a sequential convoy will do this automatically.
Messages will queue (waiting for processing) from the Orchestration
that is running the convoy (even if it is busy processing another
message already).  They just queue in the message box for you.

So all you have to do is have the loop for your sequential convoy and
do your processing within it and you are done.  That's easy huh?

Your biggest issue is going to be knowing when to terminate this
convoy (which message is the last).  There are a lot of ways to do
this such as time or count.

Lastly I would do my processing in a child Orchestration that you call
using the Call Orchestration shape.  This will help give yo some
encapsulation.  Do you really need to process the messages one at a
time?  Often when I do with a sequential convoy is to use Start
Orchestration instead of Call and then pass in a self-correlating
receive port as a parameter to the called Orchestration.

You keep a counter to count how many times you called Start
Orchestration, then you have a second loop that receives the responses
(if you need responses at all or need to know when all the processing
is done).  What's great about this pattern is that you get parallel
processing, which increases throughput and scales much better in
larger environments.

Kind Regards,
-Dan

Dan, I am intrigued.

Dan, I am intrigued. Can you point me to any samples of how the
self-correlating port/start orchestration implementation works? This really
sounds like just the solution I need.

Thank you much for responding!

-m

Sorry for the delayed response. Check this out:http://msdn.microsoft.

Sorry for the delayed response.  Check this out:
http://msdn.microsoft.com/en-us/library/ms942189(BTS.10).aspx and
http://www.thoughtworks.com/pdfs/integrationpatterns_biztalk.pdf

The Thoughtworks paper actually runs through a Broker example (which
is what you want to do).  If these are too verbose or do not make
sense, I can send you something quickly.

Basically the main Orchestration calls Start on the child sending the
message and a self-correlating one way port as Orchestration
parameters.  it is a Receive in the main and a Send port in the child.
The main Orchestration would call Start in a loop, sending out many
messages and incrementing a counter.  When it is done it would do a
second loop with that counter to receive the responses.  it is really a
lot easier than it sounds.  Simplest multiprocessing I have ever seen!

Kind Regards,
-Dan

Hi Dan, I would like to ask you how do you handle errors in the

Hi Dan, I would like to ask you how do you handle errors in the child
orchestration.

I am using a Sequential Convoy Main Orchestration and when this orchestration
suspends for any reason, it suspends the other messages that're waiting to be
processing too (are waiting in the orchestration work queue).

That's OK, but if I terminate the orchestration (because I cannot resume the
message with the problem), all the other messages are lost (the one with the
error and all the others waiting to be processed)! This is not OK for me.

I suppose that having a child orchestration with self correlating as you
said, the behavior is the same.

Any idea or workaround?

Thanks in advance,
Diego

That's a really good question. Sorry for the delayed response.

That's a really good question.  Sorry for the delayed response.  I do
handle this and it is by sending back an empty processing message
(probably should have used a Fault) from the child orchestration.
This way I still get a response, but I know it is bad by checking a
promoted property.  I guess if I were really smart I would regenerate
the processing message and send it out.  Frankly I have never had the
child orchestration in my case throw an exception.  All it does is
make a SQL call and run a lot of custom .NET code for a trading
model.

Kind Regards,
-Dan