Skip to main content

Managing Change in Long Running Workflows Part 2

Image
By

 

Recently I wrote about how some of the Serialization problems you can face when dealing with long running workflows. In this post I’m going to cover how I deal with logic changes in persisted workflows and also how you can make logic changes to your workflows without having to deploy code.

Hosting the Workflow

When it comes to hosting your workflows there are really only two options to consider.

Self-Hosting
WCF Workflow Service

To be honest I only looked at the WCF Workflow Service model briefly and decided it wasn’t the best fit for the requirements as it couldn’t provide the same benefits of self-hosting within a Workflow Application.

Workflow Instance Store

In order to persist Workflow instances you need to first create the Workflow Instance Store database.

For this you will need to use the following scripts:

C:\Windows\Microsoft.NET\Framework\v4.0.30319\SQL\en\SqlWorkflowInstanceStoreSchema.sql
C:\Windows\Microsoft.NET\Framework\v4.0.30319\SQL\en\SqlWorkflowInstanceStoreLogic.sql

 

The Workflow

The scenario I’m going to use for this post is that of an order approval workflow. So after an order is created it goes into a “Pending” state, a workflow is then started requiring two users to approve before it can be changed to an “Approved” state.

This is represented in the below diagram.

OrderApprovalWorkflow

Bookmarks

If you already know how to implement long running workflows using Bookmarks then you can skip down to the next section.

The way to have points in your workflows that trigger persistence and can wait for external input are called Bookmarks.

In above diagram the “Wait for Approval” activity is a Bookmark and looks like the following:

 [Serializable]
 public sealed class WaitForApproval : NativeActivity<bool>
 {
     public InArgument<int> UserId { get; set; }

     protected override bool CanInduceIdle
     {
         get { return true; }
     }

     protected override void Execute(NativeActivityContext context)
     {
         var bookmarkName = "waitingFor_" + UserId.Get(context);

         context.CreateBookmark(bookmarkName, OnReadComplete);
     }

     private void OnReadComplete(NativeActivityContext context, Bookmark bookmark, object state)
     {
         var input = Convert.ToBoolean(state, new CultureInfo("EN-us"));

         context.SetValue(Result, input);
     }
 }

Take note of the bookmark name as this will be used later when resuming the workflow.

TIP: The bookmark is stored in the BlockingBookmarks field on your InstancesTable in your workflow instance database.

The Problem

The problem you can face after you’ve persisted your workflow instance is when the workflow definition changes and you try to load the instance using WorkflowApplication.Load.

A number of changes detailed below can lead to invalidating the workflow instance and will cause a System.Activities.ValidationException to be thrown.

The somewhat “official" word on this found here is that “WF4 is not able to handle runtime changes to the workflow definition”. This was elaborated a bit more and here are the “official” lists of breaking and non-breaking changes.

Non-Breaking Changes:

  • Changes to the activities code
  • Changes which don’t affect the number or types of arguments/variables
  • Addition or removal of member fields or properties

Breaking Changes:

  • Renaming or removing methods that had been used as bookmark, fault or completion callbacks
  • Adding new arguments
  • Adding new variables
  • Adding new children

Basically the rules of serialization/deserialization versioning apply also to workflows, which when you think about it makes sense.

Don’t fret though as this can be avoided without too much effort, so keep reading to find out how.

Incorporating Workflow into the Architecture

Very early on in my foray with Workflow Foundation I realised the power and flexibility it could to add to the application, specifically the ability to use different workflows for different users/accounts and changing workflows at run-time.

Because of this it was decided that the workflows should be treated as a first class citizen in our domain model and not just left as an abstract technical implementation of the business process.

Below is a simple class diagram which should give you a good idea of the approach taken.

ClassDiagram1

As you can see in the Order class there is the WorkflowInstanceId and WorkflowXaml.
The WorkflowInstanceId is there because it allows to identify the Workflow by using the OrderId meaning that you don’t have to expose the InstanceId in other parts of your application.

The WorkflowXaml is the raw definition of the workflow that gets copied from the WorkflowDefinition at instantiation time. The purpose of this is to allow the WorkflowDefinition Xaml to be changed without affecting the persisted workflows and solves the problem described above.

Arguably you could add a separate Class/Table which stores the active Xaml and InstanceId for the Order, but for simplicities sake I’ve just added them to the Order.

It should also be said the WorkflowInstanceId and WorkflowXaml can be cleared from the Order table once the workflow has completed.

This will all become clearer as you read on.

Generic Workflow Host

For the hosting I have come up with a GenericWorkflowHost which allows to start instances and resume from bookmarks. There’s not much that is very exciting in here although do take note of the following methods:

  • CreateActivityFrom
  • StartPersistableInsance
  • LoadInstanceWithBookmark

The key part is the XamlServices.Load method which allows to create an Activity using just the Xaml. This is important because this allows you to change the Xaml without re-compiling and deploying code. 

using System;
using System.Activities;
using System.Activities.DurableInstancing;
using System.Activities.XamlIntegration;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.Reflection;
using System.Runtime.DurableInstancing;
using System.Xaml;

public static class GenericWorkflowHost
{
    private static ConcurrentDictionary<Guid, WorkflowApplication> runningWorkflows;

    #region Private Helper Methods 

    private static Activity CreateActivityFrom(string xaml)
    {
        var sr = new StringReader(xaml);

        //Change LocalAssembly to where the Activities reside
        var xamlSettings = new XamlXmlReaderSettings
                               {LocalAssembly = Assembly.GetExecutingAssembly()};

        var xamlReader = ActivityXamlServices
            .CreateReader(new XamlXmlReader(sr, xamlSettings));

        var result = XamlServices.Load(xamlReader);

        var activity = result as Activity;

        return activity;
    }

    private static InstanceStore CreateInstanceStore()
    {
        var conn = ConfigurationManager.ConnectionStrings["WorkflowDbConn"].ConnectionString;

        var store = new SqlWorkflowInstanceStore(conn)
                        {
                            InstanceLockedExceptionAction = InstanceLockedExceptionAction.AggressiveRetry,
                            InstanceCompletionAction = InstanceCompletionAction.DeleteNothing,
                            HostLockRenewalPeriod = TimeSpan.FromSeconds(20),
                            RunnableInstancesDetectionPeriod = TimeSpan.FromSeconds(3)
                        };

        var handle = store.CreateInstanceHandle();

        var view = store.Execute(handle, new CreateWorkflowOwnerCommand(),
                                 TimeSpan.FromSeconds(60));

        store.DefaultInstanceOwner = view.InstanceOwner;

        handle.Free();

        return store;
    }

   
    #endregion

    public static void InvokeInstance(object input, string xaml, Guid instanceId)
    {
        var inputs = new Dictionary<string, object>();

        if (input != null)
        {
            inputs.Add(input.GetType().Name, input);
        }

        var wf = CreateActivityFrom(xaml);

        var activity = wf;

        WorkflowInvoker.Invoke(activity, inputs);
    }

    public static Guid StartPersistableInstance(IDictionary<string, object> inputs, string xaml)
    {
        if (runningWorkflows == null)
        {
            runningWorkflows = new ConcurrentDictionary<Guid, WorkflowApplication>();
        }


        var activity = CreateActivityFrom(xaml);

        var workflowApp = new WorkflowApplication(activity, inputs)
                              {
                                  InstanceStore = CreateInstanceStore(),
                                  PersistableIdle = OnIdleAndPersistable,
                                  Completed = OnWorkflowCompleted,
                                  Aborted = OnWorkflowAborted,
                                  Unloaded = OnWorkflowUnloaded,
                                  OnUnhandledException = OnWorkflowException
                              };

        workflowApp.Persist();

        var instanceId = workflowApp.Id;

        workflowApp.Run();

        runningWorkflows.TryAdd(instanceId, workflowApp);

        return workflowApp.Id;
    }

    public static bool LoadInstanceWithBookmark(string bookmarkName,
                                                Guid instanceId,
                                                object input,
                                                string xaml)
    {
        if (runningWorkflows == null)
        {
            runningWorkflows = new ConcurrentDictionary<Guid, WorkflowApplication>();
        }

        BookmarkResumptionResult result;

        if (runningWorkflows.ContainsKey(instanceId))
        {
            var workflow = runningWorkflows[instanceId];
            workflow.Completed = OnWorkflowCompleted;
            workflow.PersistableIdle = OnIdleAndPersistable;

            result = workflow.ResumeBookmark(bookmarkName, input, TimeSpan.FromSeconds(60));

            Console.WriteLine(instanceId + " resumed @ " + bookmarkName);
        }
        else
        {
            // Setup the persistance
            var store = CreateInstanceStore();

            var activity = CreateActivityFrom(xaml);

            var application = new WorkflowApplication(activity)
                                  {
                                      InstanceStore = store,
                                      Completed = OnWorkflowCompleted,
                                      Unloaded = OnWorkflowUnloaded,
                                      PersistableIdle = OnIdleAndPersistable,
                                  };

            application.Load(instanceId, TimeSpan.FromSeconds(60));

            result = application.ResumeBookmark(bookmarkName, input, TimeSpan.FromSeconds(60));

            runningWorkflows.TryAdd(instanceId, application);

            Console.WriteLine(instanceId + " resumed @ " + bookmarkName);
        }

        return result == BookmarkResumptionResult.Success;
    }

    public static void UnloadInstance(Guid instanceId)
    {
        if (!runningWorkflows.ContainsKey(instanceId))
        {
            return;
        }

        var workflow = runningWorkflows[instanceId];
        workflow.Unload();

        runningWorkflows.TryRemove(instanceId, out workflow);
    }

    #region Events 

    public static void OnWorkflowCompleted(WorkflowApplicationCompletedEventArgs e)
    {
        if (runningWorkflows != null && runningWorkflows.ContainsKey(e.InstanceId))
        {
            WorkflowApplication workflowApp;
            runningWorkflows.TryRemove(e.InstanceId, out workflowApp);
        }

        Console.WriteLine(e.CompletionState);
    }

    public static PersistableIdleAction OnIdleAndPersistable(WorkflowApplicationIdleEventArgs e)
    {
        return PersistableIdleAction.Unload;
    }

    public static void OnWorkflowAborted(WorkflowApplicationAbortedEventArgs e)
    {
        Console.WriteLine(e.Reason);
    }

    public static void OnWorkflowUnloaded(WorkflowApplicationEventArgs e)
    {
        if (runningWorkflows != null && runningWorkflows.ContainsKey(e.InstanceId))
        {
            WorkflowApplication workflowApp;
            runningWorkflows.TryRemove(e.InstanceId, out workflowApp);
        }

        Console.WriteLine(e.InstanceId + " unloaded");
    }

    public static UnhandledExceptionAction OnWorkflowException(WorkflowApplicationUnhandledExceptionEventArgs e)
    {
        //log the exception here using e.UnhandledException 
        return UnhandledExceptionAction.Terminate;
    }

    #endregion
}

As you may have noticed the GenericWorkflowHost is a static class so I choose to abstract this away behind an interface so that it’s nice and Unit-Test friendly: 

   public interface IOrderApprovalWorkflowHost
   {
       void Resume(Guid instanceId, string xaml, int userId, bool isApproved);

       Guid Start(int orderId, string xaml);
   }

Which is implemented like so: 

    public class OrderWorkflowApprovalHost : IOrderApprovalWorkflowHost
    {
        public void Resume(Guid instanceId, string xaml, int userId, bool isApproved)
        {
            var bookmark = string.Format("waitingFor_{0}", userId);

            GenericWorkflowHost.LoadInstanceWithBookmark(bookmark, instanceId, isApproved, xaml); 
        }

        public Guid Start(int orderId, string xaml)
        {
            var inputs = new Dictionary<string, object>()
                             {
                                 {"OrderId", orderId}
                             };  
            return GenericWorkflowHost.StartPersistableInstance(inputs, xaml);
        }

    }

 

Starting the Workflow Instance

Now that we have our workflow host it’s time to start the workflow. I like to abstract any workflow calls behind a service interface that the other layers can consume without the hard-dependency on the workflow libraries. 

    public interface IOrderApprovalService
    {
        void SubmitApproval(int orderId, int userId, bool isApproved);

        void StartApprovalWorkflow(int orderId); 
    }

The StartApprovalWorkflow method is implemented like so: 

        public void StartApprovalWorkflow(int orderId)
        {
            using (var unitOfWork = unitOfWorkFactory.Create())
            {
                var order = orderRepository.Get(orderId);

                var workflow = workflowRepository.Get(1); // this id should come from configuration 

                var instanceId = orderApprovalWorkflowHost.Start(orderId, workflow.Xaml);

                order.StartWorkflow(workflow, instanceId);// this sets the workflow xaml & instanceid

                unitOfWork.Commit(); 

            }
        }

All we do here is load the Order & workflow from their respective repositories and then start the workflow and assign the workflow to the order. All pretty simple stuff.

Resuming the Workflow Instance

Resuming the instance is as easy as loading the Order and then calling the OrderApprovalWorkflowHost passing the persisted Xaml & InstanceId that was saved when starting the workflow. 

        public void SubmitApproval(int orderId, int userId, bool isApproved)
        {
            var order = orderRepository.Get(orderId);

            orderApprovalWorkflowHost
                .Resume(order.WorkflowInstanceId, order.WorkflowXaml, userId, isApproved);
        }

 

Conclusion

The approach I’ve outlined in this post has some major benefits:

  • Ability to change the workflow definition without code deployment
  • Ability to change workflow definition without invalidating persisted instances
  • Provides a simple way to view active instances

When you want to change the logic flow in a persisted instance the simplest approach is either to force the workflow to end and then restart it or to just rollback any state changes the workflow has done and simply restart a new instance then delete the old instance. This needs to be carefully thought out though especially if any of the activities send emails.

Well that’s all I’ve got for this post, I’m really keen to hear any feedback and as always I’d be happy to answer any questions.

Labels:

Popular posts from this blog

ASP.NET MVC Release Candidate - Upgrade issues - Spec#

By
First of all, great news that the ASP.NET MVC Release Candidate has finally been released.  Full credit to the team for the hard work on this.  You can get the download here  However this is the first time I have had upgrade issues.  Phil Haack has noted some of the issues here   If like me you have lot's of CTP's and Add-Ins then you might experience some pain in Uninstalling MVC Beta on Vista SP1  This is the list of Add-Ins / CTP's I had to uninstall to get it to work  Spec# PEX Resharper 4.1  Sourcelinks ANTS Profiler 4   Can't say I'm too impressed as it wasted over an hour of my time.  As it turned out Spec# turned out to be the offending culprit, it's forgiveable to have issues with a third party product but a Microsoft one? Guess no-one on the ASP.NET team has Spec# installed. 
Read more

Freeing Disk Space on C:\ Windows Server 2008

By Will Beattie
  I just spent the last little while trying to clear space on our servers in order to install .NET 4.5 . Decided to post so my future self can find the information when I next have to do this. I performed all the usual tasks: Deleting any files/folders from C:\windows\temp and C:\Users\%UserName%\AppData\Local\Temp Delete all EventViewer logs Save to another Disk if you want to keep them Remove any unused programs, e.g. Firefox Remove anything in C:\inetpub\logs Remove any file/folders C:\Windows\System32\LogFiles Remove any file/folders from C:\Users\%UserName%\Downloads Remove any file/folders able to be removed from C:\Users\%UserName%\Desktop Remove any file/folders able to be removed from C:\Users\%UserName%\My Documents Stop Windows Update service and remove all files/folders from C:\Windows\SoftwareDistribution Deleting an Event Logs Run COMPCLN.exe Move the Virtual Memory file to another disk However this wasn’t enough & I found the most space...
Read more

CPF Contribution Rates for new Singapore Permanent Residents (SPR’s)

By Will Beattie
Recently my wife and I applied and got approved for Singapore Permanent Residency. After completing the formalities the most significant immediate change is the contribution to CPF which is Singapore’s mandatory social security savings scheme requiring contributions from employers and employees. CPF contributions start from the date you obtain SPR status, which is the date of the entry permit.   Being a relentless budgeter I needed to know exactly how much I and my employer would have to contribute so that I could adjust my budget accordingly as the employee contributions get deducted from the monthly salary. After doing some research I discovered that there is a “graduated” approach to CPF contributions for new SPR’s where the contributions gradually increase in the first and second year and then upon reaching the third year are at the full amount. Note: There is an option for employers to contribute the full amount for year 1 and year 2 and the employee can use the gra...
Read more