Scattered processing - Taking out the sequentiality from asynchronous sequential flows

With Do<>() you build an async sequential flow. It´s guaranteed that values entering the flow are processed in order. This avoids races among intermediate results. A flow thus basically is an "intelligent" FIFO data structure.

Sometimes though you don´t care if input is processed in order. Instead you want them processed in parallel. That´s what the CCR is good at.

With CCR Flows you switch from sequential to parallel processing by replacing Do<>() with Scatter<>(). Here´s how you would process the words of a text in parallel:

var flow = Flow<string>
    .Do<string>(SplitIntoWords)
    .Scatter<string>(ProcessWord);

Usually you would translate

SplitIntoWords | ProcessWord

into

var flow = Flow<string>
    .Do<string>(SplitIntoWords)
    .Do<string>(ProcessWord);

But then you would not gain any speedup on a multi core machine. All words would be processed one after another.

By using Scatter<>() instead of Do<>(), though, you take out the sequentiality from the flow.

fig3.gif

Each word is potentially processed on its own thread and concurrently. But be careful with this feature! It´s making it harder to reason about what´s going on in a flow. That´s why CCR Flows are sequential by default.

But now enough with single flows. Let´s move on to ParallelFlows.

Last edited Jul 1, 2009 at 6:04 PM by ralfw, version 3

Comments

No comments yet.