Concurrency and Workflow in .NET

I’ve been reviewing the Visual Studio 2008 IDE, mostly learning LINQ and C# 3 to maintain developer edge.  I’ve also been checking periodically for the PLINQ/PFX/System.Concurrency release.  I follow the GridLINQ developments, and other easy-to-use tools direct or tangential to concurrency, parallel programming (such as SyncLinq), and distributed databases.  I downloaded MPI.NET as soon I heard it had been released – quality – added to a long list of emerging technologies I’m planning on looking more closely at.   It’s been a while since MPI coding on a Beowulf for me.

I noticed the System.Workflow platform.  Because of the other things I’m working on at the same time, it hit me that workflow could be the solution to verifiable distributed algorithm and systems design.  For those that haven’t had the pleasure of distributed computer programming, it’s like multi-threading, except many things are actually happening at the same time, and logic errors can include temporal mixemups and all problems cascade to where you’re not certain which machine and when an error arose to begin detecting where in the code to investigate.  Some of these development issues appear to be on the horizon with the multi-core chips inviting developers into the world of concurrency.

You had to learn how to imagine the state of 64 machines at the same time for fly-by-the-seat-of-your-pants distributed application development.  This is why distributed design has, for the most part, utilized algorithms where nodes are doing similar subtasks in parallel, as opposed to what we could call more “exotic” algorithms – to keep the states on each of the 64 machines running the application within a range of possibilities at each time slice.  These are memories from C/C++, things are hopefully less stressful with .NET practices.  I am optimistic about P2P-HPC programming becoming increasingly popular.

So, what’s workflow got to do with all this?  Well, my first impression of it was that it’s easy to use.  It’s like drawing the blueprint of an algorithm; just drag and drop programming structures from the toolbox onto the graphics area.  Except, instead of an illustration, like a class diagram, you’re developing the code at the same time.  When it got interesting was when visual cues popped up telling me why the code wasn’t going to compile.  This was like the the intellisense features but right in the visualization space.  I began to imagine an IDE that could put up visual cues when distributed programming logic, concurrency issues, and systems-level design problems arose.  For the IDE to remind me that I haven’t typed something yet is convenient, but for an indication that an algorithm, as designed, has concurrency bloopers when running on 64 machines… that would be so nice.  I can only imagine the developer joy in OWL-S/XOML/Workflow analysis techniques mapped to distributed systems with the IDE also maintaining the states of computers in all possible logic branches.