Friday, July 17, 2009

Tips for using SPWeb.ProcessBatchData

There’re quite some samples about how to use the ProcessBatchData method of the SPWeb class on the internet but it turns out that the full capabilities of the method, i.e. the input XML that it uses are not that well documented. So here’re several examples that I managed to get to work while working on my SPListItem editor tool:

  • Create a folder in a SharePoint list/library:

<?xml version="1.0" encoding="utf-8"?>
<ows:Batch OnError="Continue">
  <Method ID="Test">
    <SetList Scope="Request">82d62a9a-55ba-49c8-a9b8-68ec965a5931</SetList>
    <SetVar Name="Cmd">Save</SetVar>
    <SetVar Name="ID">New</SetVar>
    <SetVar Name="Type">1</SetVar>
    <SetVar Name="owsfileref">/sites/1/docs/folder1</SetVar>
  </Method>
</ows:Batch>

The critical line here is the setting of the Type parameter – it is actually an alias of the system FSObjType field and without it the call won’t work.

This way you can create subfolders too, e.g. using:

    <SetVar Name="owsfileref">/sites/1/docs/folder1/sub1</SetVar>

The only requirement is that the parent folder exists, if it doesn’t – the call will fail.

  • Rename a file or a folder in a SharePoint list/library:

<?xml version="1.0" encoding="UTF-8"?>
<ows:Batch OnError="Continue"><Method ID="Test">
  <SetList Scope="Request">82d62a9a-55ba-49c8-a9b8-68ec965a5931</SetList>
  <SetVar Name="Cmd">Save</SetVar>
  <SetVar Name="ID">28</SetVar>
  <SetVar Name="owsfileref">/sites/1/docs/fld2</SetVar>
  <SetVar Name="owsnewfileref">fld1</SetVar>
</Method></ows:Batch>

So you need both the server relative URL of the file or folder object in the owsfileref parameter and the ID of the associated list item here. The new name is provided in the owsnewfileref parameter. One remark here – you can provide the new name without the extension part – the extension will be added automatically for files. In case you provide an extension (which differs from the original extension of the file) it will be ignored.

  • Using the owshiddenversion field:

Using this field is very important especially when you have versions enabled or mandatory check out for editing in document libraries. In order that you can use it, you first need to retrieve the list items having set the ViewFields property of the SPQuery object to contain the owshiddenversion field. And you need to provide the same value of the owshiddenversion field in the ProcessBatchData XML as you retrieved originally with the list item. A sample XML would look like:

<?xml version="1.0" encoding="UTF-8"?>
<ows:Batch OnError="Continue">
  <Method ID="M0">
    <SetList>82d62a9a-55ba-49c8-a9b8-68ec965a5931</SetList>
    <SetVar Name="Cmd">Save</SetVar>
    <SetVar Name="ID">23</SetVar>
    <SetVar Name="owsfileref">/sites/1/docs/codes.txt</SetVar>
    <SetVar Name="owshiddenversion">93</SetVar>
    <SetVar Name="urn:schemas-microsoft-com:office:office#Title">some title</SetVar>
  </Method>
</ows:Batch>

Basically this field is auto-incremented with every update of the list item. SharePoint checks the value of the field and compares it to the value of the current version of the list item. So if you have it in the update XML and another user has updated the item before you, you will receive the standard error message of saving conflict with a concurrent user. And in case the file in a document library that you try to update is not checked out, you’ll get again a standard error message. In both cases if you don’t include the owshiddenversion field in the update XML the update will actually succeed and you will end up with either overwriting the changes of another user or breaking the concurrence constraints that are normally enforced with the check-out mechanism.

And if you intend to do several updates in a row on a certain list item you should either re-fetch the item after each update to have the updated value of the owshiddenversion field for the next update (this is what actually happens when you update an item with SPListItem.Update) or be more economical and increment the value by one yourself.

  • Change the moderation status of a list item with the moderate command

OK, so it was known for some time that next to the Save command there is an undocumented Moderate command for updating the list item moderation status (setting an item to approved, rejected, draft, etc) using the ProcessBatchData method. Here is a sample XML:

<?xml version="1.0" encoding="UTF-8"?>
<ows:Batch OnError="Continue">
  <Method ID="M0">
    <SetList>82d62a9a-55ba-49c8-a9b8-68ec965a5931</SetList>
    <SetVar Name="Cmd">Moderate</SetVar>
    <SetVar Name="ID">26</SetVar>
    <SetVar Name="owsfileref">/sites/1/docs/a.txt</SetVar>
    <SetVar Name="owshiddenmodstatus">3</SetVar>
    <SetVar Name="urn:schemas-microsoft-com:office:office#_ModerationStatus">0</SetVar>
    <SetVar Name="owsitemlevel">2</SetVar>
    <SetVar Name="_Level">2</SetVar>
    <SetVar Name="owshiddenversion">12</SetVar>
  </Method>
</ows:Batch>

Note the usage of two more “hidden” fields here – the owshiddenmodstatus and owsitemlevel – they should contain the values of the original values of the _ModerationStatus and _Level fields when the list item was retrieved. The urn:schemas-microsoft-com:office:office#_ModerationStatus parameter should contain the new value of the moderation status. Actually these two are required only for document libraries – for lists you can provide just the urn:schemas-microsoft-com:office:office#_ModerationStatus parameter. Using this syntax you can quickly approve items in batches, and a clear difference with the Save command is that the Moderate command works on not checked out items (this holds for document libraries) and actually fails when the item is checked out.

Several tips for “hacking” the ProcessBatchData XML syntax

A good starting point is to check the standard SharePoint “Lists" web service – especially the UpdateListItems method. It uses a similar XML syntax for updating list items, and basically it’s easier to get things working with it – no fancy internal parameter names are needed. And the thing is that the web method internally uses SPWeb.ProcessBatchData and the input XML is being “translated” to the ProcessBatchData form. A nice way to check the work of the Lists.UpdateListItems is to open a list in datasheet view and check the requests sent to the web service with a web debugging proxy as fiddler. This way you’ll have real time XML samples – after all it’s obvious that the Lists.UpdateListItems was designed for use primarily by the standard list datasheet view.

As for the “translation” of the Lists.UpdateListItems XML to the ProcessBatchData form you can reflect the code of the standard SharePoint STSSOAP.DLL assembly – it’s located in the _app_bin subfolder of your SharePoint web applications. The “translation” is implemented in the ConstructCaml method of the Microsoft.SharePoint.SoapServer.ListDataImpl class. I actually got this method working in a small WinForm application, here is the sample code:

        private void TestConstructCaml()
        {
            System.Web.Services.WebService svc = new System.Web.Services.WebService();
            CreateHttpContext();
            test.ListDataImplProxy p = test.ListDataImplProxy.CreateUnderlyingInstance(svc);

            string listName = "docs";
            string xml = "<Batch OnError=\"Continue\"><Method ID=\"M0\" Cmd=\"Moderate\"><Field Name=\"ID\">22</Field><Field Name=\"FileRef\">/sites/1/docs/zterminal.csv</Field><Field Name=\"_ModerationStatus\">0</Field><Field Name=\"_Level\">2</Field><Field Name=\"owshiddenversion\">45</Field></Method></Batch>";

            string s = p.ConstructCaml(listName, xml);
            Console.WriteLine(s);

        }

        private void CreateHttpContext()
        {
            HttpContext.Current = null;
            string url = "http://racoon-vpc-hg:2909/sites/1";
            SPSite site = new SPSite(url);
            SPWeb web = site.OpenWeb();

            HttpRequest r = new HttpRequest("default.aspx", url, "");
            HttpResponse rs = new HttpResponse(null);
            HttpContext.Current = new HttpContext(r, rs);

            HttpContext.Current.Items["HttpHandlerSPWeb"] = web;
            HttpContext.Current.Items["HttpHandlerSPSite"] = site;

        }

With this code I was able to check the translated ProcessBatchData XML from a Lists.UpdateListItems input XML. The ListDataImplProxy class is a reflection proxy class which calls the methods of the ListDataImpl class using reflection. It was generated with my reflection proxy generating utility, which can be downloaded from here: http://stefan-stanev-sharepoint-blog.blogspot.com/2009/05/how-to-access-non-public-class-members.html

20 comments:

  1. I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.
    Thank You
    SharePoint Site Branding

    ReplyDelete
  2. How would you apply this SPWeb.ProcessBatchData approach to generating publishing pages that need content approval? creating these items in a batch would leave them in a 'draft' state, and it would appear that check-in and approval would require a loop, defeating the batch purpose... any thoughts?

    ReplyDelete
  3. Hi Alex,
    indeed this is the case. The ProcessBatchData method works fine for batch updates, and batch moderation status changes, but you cannot use it for batch check in-s (actually I don't know of a way to achieve batch check-in-s /using a single method call/ in SharePoint). The only alternative that I can think of for creating publishing pages in batches is to use a Module feature, which of course has its own drawbacks. Several things here - there is a way to provision publishing pages with a Module feature without it being necessary to have the source aspx files for the pages in the feature - the trick is with using the "SetupPath" attribute in the Module element (check my posting on how to provision publishing pages in sandbox solutions for more details). The approach with the Module feature can also be used without having a feature at all /no folders in the TEMPLATE/FEATURES folder/ ... but this can be only achieved with reflection, which is generally not the right way to do things.

    ReplyDelete
  4. How do we create an item inside a folder? I tried many ways if anyone could help that would be great.

    ReplyDelete
  5. Hi Anonymous,
    you need to have this SetVar element:
    <SetVar Name="RootFolder">/sites/mysite/MyList/MyFolder</SetVar>
    The "Name" attribute should contain "RootFolder" and the value should contain the server relative (not the site or site collection relative) URL of the target folder in the list.

    ReplyDelete
  6. Hi Stefan,

    how we can use processbatchdata to insert bulk records into sharepoint document library

    Regards,
    Chiru

    ReplyDelete
  7. Hi Anonymous,
    unfortunately you can't do this with the ProcessBatchData method. The reason for this is that in a document library you can't create a list item without having a document/file associated with it. So, basically you first need to upload one or more files in the document library which as far as I know is not possible to do in batches. Even the "multiple upload" feature available in the SharePoint UI uploads the selected files opening a separate WEB DAV request for each file.

    Greets
    Stefan

    ReplyDelete
  8. Hi Stefan,

    Is it possible to use ProcessBatchData to create multiple document sets in a document library?

    Thanks,
    MJ.

    ReplyDelete
    Replies
    1. Hi MJ,

      I was actually able to create a document set using the ProcessBatchData method, but it seems that if you create a document set in this manner it is not fully initialized. The thing is that when you open the document set in the SharePoint UI (i.e. when you see its home page), you see this warning message at the top of the page: "Content types that are available to this Document Set have been added or removed. Update the Document Set.". The "update" sentence is a link and when you click it, the document set looks ready for use.
      And this is how I created the document set - I actually used my list item editor tool (http://splistitemeditor.codeplex.com/) - it uses internally the ProcessBatchData method. So I first created a regular folder in a test document library and then simply changed its "ContentType" field to "DocumentSet" and its HTML_x0020_File_x0020_Type field to "SharePoint.DocumentSet".

      Greetings
      Stefan

      Delete
  9. Hi Stefan,
    Thanks for your time in providing such a good article.
    I would like to get a suggestion from you, I have a document library and if I want to copy(insert) these items(with files and metadata) into another document library, is it possible with ProcessBatchData?
    I really appreciate your time in suggesting a good solution for me.

    Thanks,
    DA

    ReplyDelete
  10. Hi DA,
    unfortunately not, the ProcessBatchData method can't do the job in your case - you can copy the metadata with it, but not the file objects themselves. Something which comes close to achieving the latter thing is the SPListItem.Copy method (which is internally used by the SP Copy web service). Note that the Copy method creates a parent-child relation between the original file and the copy (represented by the SPFile.SourceFile property of the copy SPFile instance)

    Greetings
    Stefan

    ReplyDelete
  11. Hi Stefan,

    Can we use ProceBatchData to delete splist items even if the items are checkout by other user? Is this possible?

    ReplyDelete
    Replies
    1. Hi Joseph,
      No, it's not possible to delete items which are checked out with the ProcessBatchData method. You can use the the SPFile.UndoCheckOut method to override the check out of the file first and then you will be able to delete it.

      Greetings
      Stefan

      Delete
  12. Hi Stefan,

    How do I move a list item to a different folder inside the same list? Tried to set a new value for the RootFolder, but the items stays inside the old folder... (Other fields are updated correctly though)

    Thanks
    Ted

    ReplyDelete
  13. Hi Stephen,

    using ProceBatchData, can we reserve Modified and ModifiedBy values like systemupdate.

    Thanks,
    Srikanth

    ReplyDelete
    Replies
    1. Did u get any response for this? if yes please let me know how can i reserve the modified value

      Delete
  14. Thanks a lot man, this blog saved my day

    ReplyDelete
  15. HI Stefan! Thank you !
    p.s. I see one error in the first line of "Create a folder in a SharePoint list/library" code snippet. encoding="utf-8", UTF-8 must be in UPPER CASE -:)

    ReplyDelete
  16. Hi Stefan - from following this & your other blog posts on ProcessBatchData, I've seen that a number of the old RPC methods (http://msdn.microsoft.com/en-us/library/office/ms480784.aspx) can be called through ProcessBatchData, using an identically formed request as the RPC approach and work. Great insight!

    Where I haven't had any luck with it though is the methods relating to fields; specifically "DELETEFIELD", "NEWFIELD" and "UPDATEFIELD". Have you had any luck leveraging these methods, or similar functionality, through ProcessBatchData?

    ReplyDelete
    Replies
    1. Disregard - for any future readers, the issue I encountered was with the encoding of certain characters within the MSDN examples. When I changed my "FieldXML" to instead contain a CDATA element with standard XML inside, the FIELD methods all worked.

      Delete