Approval Workflows are Missing From SharePoint 2010

Issue: Approval Workflows are Missing From SharePoint 2010

After migrating our SharePoint 2007 farm to SharePoint 2010 using database attachment approach, the "Approval" workflows are missing in the list of workflows when user clicks "Add a workflow" in the list or library. User can only see "Disposition Approval" and "Three-state" workflows.

Stage 1: Locate the issue

At first, I thought it is the issue with the AddWrkfl.aspx web page. After digging into the source code of AddWrkfl.aspx which is in the Microsoft.SharePoint.ApplicationPages.dll, it doesn't seems to be the issue. The page just loads what ever workflows available for the list.
 
The next suspect would be the feature activation. I tried to reactivate all the features at site collection level with name of "workflow" in it. There were no errors. It seems everything works fine. I turned on the verbose logging for the SharePoint 2010 farm and found something in the log.

The most interesting log that may be related to the error would be

Ensuring module folder _catalogs/wfpub/Approval - SharePoint 2010
Instantiating file "ReviewApproval_1033.xoml" from module "Approval - SharePoint 2010".
Failed to instantiate file "ReviewApproval_1033.xoml" from module "Approval - SharePoint 2010": The specified list does not exist.
Failed to instantiate file "ReviewApproval_1033.xoml" from module "Approval - SharePoint 2010": The specified list does not exist.
 

To see the "_catalogs" folders, I used SharePoint Designer 2010 to open the web site. Under the "_catalogs" folder, there are a few SharePoint list and there was a folder "wfpub" with a few sub folders with name such as "Approval - SharePoint 2010".
 
I also created another new site collection and see the "Approval" workflows are working fine in the new site collection. I used the SharePoint Designer 2010 to open the newly created site collection and notice that the "wfpub" is not a folder under "_catalogs" but a SharePoint list. It also has files under its sub folders.
 
Now I suspected that the "The specified list does not exist" indicated that the List "wfpub" was missing, but I didn't know how to re-create it.

Stage 2: Call Microsoft Support

Now I decided to call the Microsoft Support to solve the issue. Microsoft Support gave me an instruction to change the xml of workflows

change any <AssociationCategories>None</AssociationCategories> to <AssociationCategories></AssociationCategories>
 
So if do a search, you will get  

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\FEATURES>findstr /s "AssociationCategories" *

ReviewWorkflows\ReviewApproval.xml:            <AssociationCategories>None</AssociationCategories>
ReviewWorkflows\ReviewFeedback.xml:            <AssociationCategories>None</AssociationCategories>
SignaturesWorkflow\Signatures.xml:            <AssociationCategories>None</AssociationCategories>  

change all these files and then do a iisreset. Make sure "SharePoint 2007 Workflows" feature are activated. After this change, the "Approval" and a few other workflows start to show up in the "Add a workflow" page. I think these are the SharePoint 2007 workflows. I still didn't see the "Approval - SharePoint 2010" workflow.

After spending a few hours on the phone with Microsoft Support, we decided to continue the next day. I also gave them my thought on the corrupted "wfpub" list.
 

Stage 3: Focus on "wfpub" list

Based on the clues I had so far, I decided to focus my investigation on the corrupted "wfpub" list. I use SPD to delete the "wfpub" folder, and reactivated the "Workflows" feature. A "wfpub" folder was created but not a list.  

The "ReviewApproval_1033.xoml" was from feature folder "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\FEATURES\ReviewWorkflowsSPD1033". The "workflowfiles.xml" has
 
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">

    <!-- ReviewApproval -->

    <Module

        Name="Approval - SharePoint 2010"

        Url="_catalogs/wfpub/Approval - SharePoint 2010"

        RootWebOnly="TRUE" >

        <File Url="ReviewApproval_1033.xoml" Type="GhostableInLibrary" />

        <File Url="ReviewApproval_1033.xoml.rules" Type="GhostableInLibrary" />


Obviously the feature needs to have a valid url "Url="_catalogs/wfpub/Approval - SharePoint 2010" to work.

So why a folder "wfpub" was created instead of a list when the "Workflows" feature was activated?


After pondering on this question for a while, I realized that there must be a process that try to create "wfpub" list during the feature activation but it fails. When it comes to provision the module files, another process created a folder "wfpub" instead.
 
However the workflow provisioning process only works with a SharePoint list instead of a folder. The process that creates the "wfpub" must be in the feature activation event receiver.
 

<?xml version="1.0" encoding="utf-8"?>

<!-- _lcid="1033" _version="14.0.4758" _dal="1" -->

<!-- _LocalBinding -->

<!-- Copyright (c) Microsoft Corporation. All rights reserved. -->

<Feature  Id="3BC0C1E1-B7D5-4e82-AFD7-9F7E59B60409"

          Title="Routing Workflows - SharePoint 2010 (en-US)"

          Description="This feature provides Routing Workflows for a language (en-US)"

          Hidden="TRUE"

          Scope="Site"

          Version="14.0.0.0"

          AlwaysForceInstall="TRUE"

          ReceiverAssembly="Microsoft.Office.Workflow.Feature, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"

          ReceiverClass="Microsoft.Office.Workflow.Feature.WorkflowFeatureReceiver"

          xmlns="http://schemas.microsoft.com/sharepoint/">
 

I decided to use reflector to look into the source code of "Microsoft.Office.Workflow.Feature", but I couldn't locate the assembly in the file system. So I copied it from c:\windows\assembly and loaded it into reflector. In the reflector, there is a "WorkflowFeatureReceiver" class and noticeable there was a method called "EnsureRootPublicLibrary".

private static SPDocumentLibrary EnsureRootPublicLibrary(SPWeb web)

{
    if (web == null)
    {
        throw new ArgumentNullException("web");
    }
    try
    {
        return (web.Lists["wfpub"] as SPDocumentLibrary);
    }

    catch (ArgumentException)
    {
        web.Lists.Add("wfpub", "wfpub", "_catalogs/wfpub", null, 0x7a, "100");
        return (web.Lists["wfpub"] as SPDocumentLibrary);
    }
}


Bingo! This was exactly what I was looking for. This method is used by FeatureActivated as

SPFeatureProperty property3 = properties.Definition.Properties["EnsureRootPublicLibrary"];
        ...           
}

            else if (property3 != null)           
{
                EnsureRootPublicLibrary(web);           
}

This indicated the feature with property "EnsureRootPublicLibrary" would be responsible for creating the "wfpub" list. Searching the features, I had

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\FEATURES>findstr /s "EnsureRootPublicLibrary" *

OffWFCommon\Feature.xml:        <Property Key="EnsureRootPublicLibrary" Value="true" />

OffWFCommon\Feature.xml:            <CustomUpgradeAction Name="EnsureRootPublicLibrary" />
 
It is the "OffWFCommon" feature! This is a hidden feature of SharePoint.

Solution: Recreate the "wfpub" list.


First use SPD to delete the "wfpub" folder first.


Solution 1 is to recreated the "wfpub" list manually using code like

    class Program

    {
        static void Main(string[] args)

        {
            string rootUrl = args[0];
            using (SPSite site = new SPSite(rootUrl))
            {
                using (SPWeb web = site.RootWeb)
                {
                    Console.WriteLine("Web is " + web.Url);
                    web.Lists.Add("wfpub", "wfpub", "_catalogs/wfpub", null, 0x7a, "100");
                    Console.WriteLine("wfpub is created.");
                }
            } 
        }
    } 

Solution 2 is to reactivated the "OffWFCommon" from command line
 

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\BIN>STSADM.EXE -o activatefeature -name offwfcommon -url http://sptest-sea -force

Operation completed successfully.

With either solution, the "wfpub" list will be created. After this, reactivate the "Workflows" feature. The SharePoint 2010 workflows will show up in the "Add a workflow" list now.

Conclusion: The "wfpub" list was not created during upgrade.

The end of the story is that somehow during the upgrade from SharePoint 2007 to SharePoint 2010, the "wfpub" list was not created. The "OffWFCommon" is a hidden feature that responsible for creating this list. Since "OffWFCommon" is activated during the upgrade or by default. The "wfpub" list never gets a chance to be recreated if it is missing.

The "Workflows" feature activation will create a conflicting "wfpub" folder under "_catalogs" if the "wfpub" list is not there. To solve the issue, the "wfpub" folder must be deleted first and then use code or reactivate the "OffWFCommon" feature to recreate the "wfpub" list.