Skip to main content

Managing Change in Long Running Workflows Part 2

 

Recently I wrote about how some of the Serialization problems you can face when dealing with long running workflows. In this post I’m going to cover how I deal with logic changes in persisted workflows and also how you can make logic changes to your workflows without having to deploy code.

Hosting the Workflow

When it comes to hosting your workflows there are really only two options to consider.

Self-Hosting
WCF Workflow Service

To be honest I only looked at the WCF Workflow Service model briefly and decided it wasn’t the best fit for the requirements as it couldn’t provide the same benefits of self-hosting within a Workflow Application.

Workflow Instance Store

In order to persist Workflow instances you need to first create the Workflow Instance Store database.

For this you will need to use the following scripts:

C:\Windows\Microsoft.NET\Framework\v4.0.30319\SQL\en\SqlWorkflowInstanceStoreSchema.sql
C:\Windows\Microsoft.NET\Framework\v4.0.30319\SQL\en\SqlWorkflowInstanceStoreLogic.sql

 

The Workflow

The scenario I’m going to use for this post is that of an order approval workflow. So after an order is created it goes into a “Pending” state, a workflow is then started requiring two users to approve before it can be changed to an “Approved” state.

This is represented in the below diagram.

OrderApprovalWorkflow

Bookmarks

If you already know how to implement long running workflows using Bookmarks then you can skip down to the next section.

The way to have points in your workflows that trigger persistence and can wait for external input are called Bookmarks.

In above diagram the “Wait for Approval” activity is a Bookmark and looks like the following:

 [Serializable]
 public sealed class WaitForApproval : NativeActivity<bool>
 {
     public InArgument<int> UserId { get; set; }

     protected override bool CanInduceIdle
     {
         get { return true; }
     }

     protected override void Execute(NativeActivityContext context)
     {
         var bookmarkName = "waitingFor_" + UserId.Get(context);

         context.CreateBookmark(bookmarkName, OnReadComplete);
     }

     private void OnReadComplete(NativeActivityContext context, Bookmark bookmark, object state)
     {
         var input = Convert.ToBoolean(state, new CultureInfo("EN-us"));

         context.SetValue(Result, input);
     }
 }

Take note of the bookmark name as this will be used later when resuming the workflow.

TIP: The bookmark is stored in the BlockingBookmarks field on your InstancesTable in your workflow instance database.

The Problem

The problem you can face after you’ve persisted your workflow instance is when the workflow definition changes and you try to load the instance using WorkflowApplication.Load.

A number of changes detailed below can lead to invalidating the workflow instance and will cause a System.Activities.ValidationException to be thrown.

The somewhat “official" word on this found here is that “WF4 is not able to handle runtime changes to the workflow definition”. This was elaborated a bit more and here are the “official” lists of breaking and non-breaking changes.

Non-Breaking Changes:

  • Changes to the activities code
  • Changes which don’t affect the number or types of arguments/variables
  • Addition or removal of member fields or properties

Breaking Changes:

  • Renaming or removing methods that had been used as bookmark, fault or completion callbacks
  • Adding new arguments
  • Adding new variables
  • Adding new children

Basically the rules of serialization/deserialization versioning apply also to workflows, which when you think about it makes sense.

Don’t fret though as this can be avoided without too much effort, so keep reading to find out how.

Incorporating Workflow into the Architecture

Very early on in my foray with Workflow Foundation I realised the power and flexibility it could to add to the application, specifically the ability to use different workflows for different users/accounts and changing workflows at run-time.

Because of this it was decided that the workflows should be treated as a first class citizen in our domain model and not just left as an abstract technical implementation of the business process.

Below is a simple class diagram which should give you a good idea of the approach taken.

ClassDiagram1

As you can see in the Order class there is the WorkflowInstanceId and WorkflowXaml.
The WorkflowInstanceId is there because it allows to identify the Workflow by using the OrderId meaning that you don’t have to expose the InstanceId in other parts of your application.

The WorkflowXaml is the raw definition of the workflow that gets copied from the WorkflowDefinition at instantiation time. The purpose of this is to allow the WorkflowDefinition Xaml to be changed without affecting the persisted workflows and solves the problem described above.

Arguably you could add a separate Class/Table which stores the active Xaml and InstanceId for the Order, but for simplicities sake I’ve just added them to the Order.

It should also be said the WorkflowInstanceId and WorkflowXaml can be cleared from the Order table once the workflow has completed.

This will all become clearer as you read on.

Generic Workflow Host

For the hosting I have come up with a GenericWorkflowHost which allows to start instances and resume from bookmarks. There’s not much that is very exciting in here although do take note of the following methods:

  • CreateActivityFrom
  • StartPersistableInsance
  • LoadInstanceWithBookmark

The key part is the XamlServices.Load method which allows to create an Activity using just the Xaml. This is important because this allows you to change the Xaml without re-compiling and deploying code.

using System;
using System.Activities;
using System.Activities.DurableInstancing;
using System.Activities.XamlIntegration;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.Reflection;
using System.Runtime.DurableInstancing;
using System.Xaml;

public static class GenericWorkflowHost
{
    private static ConcurrentDictionary<Guid, WorkflowApplication> runningWorkflows;

    #region Private Helper Methods 

    private static Activity CreateActivityFrom(string xaml)
    {
        var sr = new StringReader(xaml);

        //Change LocalAssembly to where the Activities reside
        var xamlSettings = new XamlXmlReaderSettings
                               {LocalAssembly = Assembly.GetExecutingAssembly()};

        var xamlReader = ActivityXamlServices
            .CreateReader(new XamlXmlReader(sr, xamlSettings));

        var result = XamlServices.Load(xamlReader);

        var activity = result as Activity;

        return activity;
    }

    private static InstanceStore CreateInstanceStore()
    {
        var conn = ConfigurationManager.ConnectionStrings["WorkflowDbConn"].ConnectionString;

        var store = new SqlWorkflowInstanceStore(conn)
                        {
                            InstanceLockedExceptionAction = InstanceLockedExceptionAction.AggressiveRetry,
                            InstanceCompletionAction = InstanceCompletionAction.DeleteNothing,
                            HostLockRenewalPeriod = TimeSpan.FromSeconds(20),
                            RunnableInstancesDetectionPeriod = TimeSpan.FromSeconds(3)
                        };

        var handle = store.CreateInstanceHandle();

        var view = store.Execute(handle, new CreateWorkflowOwnerCommand(),
                                 TimeSpan.FromSeconds(60));

        store.DefaultInstanceOwner = view.InstanceOwner;

        handle.Free();

        return store;
    }

   
    #endregion

    public static void InvokeInstance(object input, string xaml, Guid instanceId)
    {
        var inputs = new Dictionary<string, object>();

        if (input != null)
        {
            inputs.Add(input.GetType().Name, input);
        }

        var wf = CreateActivityFrom(xaml);

        var activity = wf;

        WorkflowInvoker.Invoke(activity, inputs);
    }

    public static Guid StartPersistableInstance(IDictionary<string, object> inputs, string xaml)
    {
        if (runningWorkflows == null)
        {
            runningWorkflows = new ConcurrentDictionary<Guid, WorkflowApplication>();
        }


        var activity = CreateActivityFrom(xaml);

        var workflowApp = new WorkflowApplication(activity, inputs)
                              {
                                  InstanceStore = CreateInstanceStore(),
                                  PersistableIdle = OnIdleAndPersistable,
                                  Completed = OnWorkflowCompleted,
                                  Aborted = OnWorkflowAborted,
                                  Unloaded = OnWorkflowUnloaded,
                                  OnUnhandledException = OnWorkflowException
                              };

        workflowApp.Persist();

        var instanceId = workflowApp.Id;

        workflowApp.Run();

        runningWorkflows.TryAdd(instanceId, workflowApp);

        return workflowApp.Id;
    }

    public static bool LoadInstanceWithBookmark(string bookmarkName,
                                                Guid instanceId,
                                                object input,
                                                string xaml)
    {
        if (runningWorkflows == null)
        {
            runningWorkflows = new ConcurrentDictionary<Guid, WorkflowApplication>();
        }

        BookmarkResumptionResult result;

        if (runningWorkflows.ContainsKey(instanceId))
        {
            var workflow = runningWorkflows[instanceId];
            workflow.Completed = OnWorkflowCompleted;
            workflow.PersistableIdle = OnIdleAndPersistable;

            result = workflow.ResumeBookmark(bookmarkName, input, TimeSpan.FromSeconds(60));

            Console.WriteLine(instanceId + " resumed @ " + bookmarkName);
        }
        else
        {
            // Setup the persistance
            var store = CreateInstanceStore();

            var activity = CreateActivityFrom(xaml);

            var application = new WorkflowApplication(activity)
                                  {
                                      InstanceStore = store,
                                      Completed = OnWorkflowCompleted,
                                      Unloaded = OnWorkflowUnloaded,
                                      PersistableIdle = OnIdleAndPersistable,
                                  };

            application.Load(instanceId, TimeSpan.FromSeconds(60));

            result = application.ResumeBookmark(bookmarkName, input, TimeSpan.FromSeconds(60));

            runningWorkflows.TryAdd(instanceId, application);

            Console.WriteLine(instanceId + " resumed @ " + bookmarkName);
        }

        return result == BookmarkResumptionResult.Success;
    }

    public static void UnloadInstance(Guid instanceId)
    {
        if (!runningWorkflows.ContainsKey(instanceId))
        {
            return;
        }

        var workflow = runningWorkflows[instanceId];
        workflow.Unload();

        runningWorkflows.TryRemove(instanceId, out workflow);
    }

    #region Events 

    public static void OnWorkflowCompleted(WorkflowApplicationCompletedEventArgs e)
    {
        if (runningWorkflows != null && runningWorkflows.ContainsKey(e.InstanceId))
        {
            WorkflowApplication workflowApp;
            runningWorkflows.TryRemove(e.InstanceId, out workflowApp);
        }

        Console.WriteLine(e.CompletionState);
    }

    public static PersistableIdleAction OnIdleAndPersistable(WorkflowApplicationIdleEventArgs e)
    {
        return PersistableIdleAction.Unload;
    }

    public static void OnWorkflowAborted(WorkflowApplicationAbortedEventArgs e)
    {
        Console.WriteLine(e.Reason);
    }

    public static void OnWorkflowUnloaded(WorkflowApplicationEventArgs e)
    {
        if (runningWorkflows != null && runningWorkflows.ContainsKey(e.InstanceId))
        {
            WorkflowApplication workflowApp;
            runningWorkflows.TryRemove(e.InstanceId, out workflowApp);
        }

        Console.WriteLine(e.InstanceId + " unloaded");
    }

    public static UnhandledExceptionAction OnWorkflowException(WorkflowApplicationUnhandledExceptionEventArgs e)
    {
        //log the exception here using e.UnhandledException 
        return UnhandledExceptionAction.Terminate;
    }

    #endregion
}

As you may have noticed the GenericWorkflowHost is a static class so I choose to abstract this away behind an interface so that it’s nice and Unit-Test friendly:

   public interface IOrderApprovalWorkflowHost
   {
       void Resume(Guid instanceId, string xaml, int userId, bool isApproved);

       Guid Start(int orderId, string xaml);
   }

Which is implemented like so:

    public class OrderWorkflowApprovalHost : IOrderApprovalWorkflowHost
    {
        public void Resume(Guid instanceId, string xaml, int userId, bool isApproved)
        {
            var bookmark = string.Format("waitingFor_{0}", userId);

            GenericWorkflowHost.LoadInstanceWithBookmark(bookmark, instanceId, isApproved, xaml); 
        }

        public Guid Start(int orderId, string xaml)
        {
            var inputs = new Dictionary<string, object>()
                             {
                                 {"OrderId", orderId}
                             };  
            return GenericWorkflowHost.StartPersistableInstance(inputs, xaml);
        }

    }

 

Starting the Workflow Instance

Now that we have our workflow host it’s time to start the workflow. I like to abstract any workflow calls behind a service interface that the other layers can consume without the hard-dependency on the workflow libraries.

    public interface IOrderApprovalService
    {
        void SubmitApproval(int orderId, int userId, bool isApproved);

        void StartApprovalWorkflow(int orderId); 
    }

The StartApprovalWorkflow method is implemented like so:

        public void StartApprovalWorkflow(int orderId)
        {
            using (var unitOfWork = unitOfWorkFactory.Create())
            {
                var order = orderRepository.Get(orderId);

                var workflow = workflowRepository.Get(1); // this id should come from configuration 
                var instanceId = orderApprovalWorkflowHost.Start(orderId, workflow.Xaml);

                order.StartWorkflow(workflow, instanceId);// this sets the workflow xaml & instanceid

                unitOfWork.Commit(); 

            }
        }

All we do here is load the Order & workflow from their respective repositories and then start the workflow and assign the workflow to the order. All pretty simple stuff.

Resuming the Workflow Instance

Resuming the instance is as easy as loading the Order and then calling the OrderApprovalWorkflowHost passing the persisted Xaml & InstanceId that was saved when starting the workflow.

        public void SubmitApproval(int orderId, int userId, bool isApproved)
        {
            var order = orderRepository.Get(orderId);

            orderApprovalWorkflowHost
                .Resume(order.WorkflowInstanceId, order.WorkflowXaml, userId, isApproved);
        }

 

Conclusion

The approach I’ve outlined in this post has some major benefits:

  • Ability to change the workflow definition without code deployment
  • Ability to change workflow definition without invalidating persisted instances
  • Provides a simple way to view active instances

When you want to change the logic flow in a persisted instance the simplest approach is either to force the workflow to end and then restart it or to just rollback any state changes the workflow has done and simply restart a new instance then delete the old instance. This needs to be carefully thought out though especially if any of the activities send emails.

Well that’s all I’ve got for this post, I’m really keen to hear any feedback and as always I’d be happy to answer any questions.

Comments

  1. Thanks a lot !!! You saved me at 5am trying to resume a workflow with xaml definition stored in the db.

    ReplyDelete
  2. Gr8 post ,declarative and effective in deep sense

    ReplyDelete

Post a Comment

Popular posts from this blog

Freeing Disk Space on C:\ Windows Server 2008

I just spent the last little while trying to clear space on our servers in order to install .NET 4.5. Decided to post so my future self can find the information when I next have to do this. I performed all the usual tasks: Deleting any files/folders from C:\windows\temp and C:\Users\%UserName%\AppData\Local\TempDelete all EventViewer logs Save to another Disk if you want to keep themRemove any unused programs, e.g. FirefoxRemove anything in C:\inetpub\logsRemove any file/folders C:\Windows\System32\LogFilesRemove any file/folders from C:\Users\%UserName%\DownloadsRemove any file/folders able to be removed from C:\Users\%UserName%\DesktopRemove any file/folders able to be removed from C:\Users\%UserName%\My DocumentsStop Windows Update service and remove all files/folders from C:\Windows\SoftwareDistributionDeleting an Event Logs Run COMPCLN.exe Move the Virtual Memory file to another disk However this wasn’t enough & I found the most space was cleared by using the Disk Cleanup to…

CPF Contribution Rates for new Singapore Permanent Residents (SPR’s)

Recently my wife and I applied and got approved for Singapore Permanent Residency. After completing the formalities the most significant immediate change is the contribution to CPF which is Singapore’s mandatory social security savings scheme requiring contributions from employers and employees. CPF contributions start from the date you obtain SPR status, which is the date of the entry permit.   Being a relentless budgeter I needed to know exactly how much I and my employer would have to contribute so that I could adjust my budget accordingly as the employee contributions get deducted from the monthly salary. After doing some research I discovered that there is a “graduated” approach to CPF contributions for new SPR’s where the contributions gradually increase in the first and second year and then upon reaching the third year are at the full amount. Note: There is an option for employers to contribute the full amount for year 1 and year 2 and the employee can use the graduated rate, b…

Consuming the SSRS ReportExecutionService from a .NET Client

I’ve just finished writing a nice wrapper which internally calls the SSRS ReportExecutionService to generate reports.
Whilst it was fairly simple to implement there has been some major changes between 2005 and 2008 and the majority of online and documentation is based on the 2005 implementation. The most important change is that the Report Server and Report Manager are no longer hosted in IIS which will be a welcomed change to Sys Admins but makes the security model and hosting model vastly different. So far I’ve yet to figure out how to allow Anonymous Access, if anyone knows how to do this leave a comment and it will be most appreciated. Getting StartedTo get started you’ll want to add a service reference to http://localhost/ReportServer_SQL2008/ReportExecution2005.asmx where ReportServer_SQL2008 is the name you configure in the Reporting Services Configuration Manager. The Web Application files are located in C:\Program Files\Microsoft SQL Server\MSRS10.SQL2008\Reporting Servic…