Wednesday, September 9, 2009

SPWeb ProcessBatchData – DisplayPost method

The DisplayPost WSS RPC method is sorts of analogue of the SPWeb.ProcessBatchData method in that you can use it to execute batches of the other RPC methods. It can be invoked using an HTML Post request against _vti_bin/owssvr.dll – in fact this is the testing setup for calling RPC methods as prescribed in MSDN – check here. This article describes how to create a small HTML page that sets several HTML form inputs, most notably a PostBody parameter that is intended to contain an XML with ows:Batch root and Method elements for various RPC methods.

So, at first glance using this method with ProcessBatchData doesn’t seem very reasonable – after all why would you want to add another level of indirection and call batches through the DisplayPost method instead of directly with ProcessBatchData. But still there’re two usages of the DisplayPost method that may be useful and I will describe them briefly.

The first usage is almost identical to the Display method (check my previous posting on that) – you specify the XMLDATA SetVar parameter and the method returns the list data in XML format:

<ows:Batch OnError="Continue">

  <Method ID="0">

    <SetList>702f059d-71f2-4f78-a41a-48978d381948</SetList>

    <SetVar Name="View">{CE0FFB35-F6A9-4F57-B06C-374B2AA4571B}</SetVar>

    <SetVar Name="XMLDATA">TRUE</SetVar>

    <SetVar Name="Cmd">DisplayPost</SetVar>

  </Method>

</ows:Batch>

Note that the PostBody parameter is not used in this case. The View SetVar parameter is optional – if omitted the default view of the list is used. Similarly to the Display method SetVar parameters like SortField, SortDir, FilterField1, FilterValue1, FilterField2, FilterValue2, RootFolder can be used for simple filtering and sorting. The Query SetVar parameter however doesn’t work with DisplayPost.

And the second usage of the DisplayPost method which is much more interesting:

<ows:Batch OnError="Continue">

  <Method ID="0">

    <SetVar Name="Cmd">DisplayPost</SetVar>

    <SetVar Name="PostBody">

      &lt;ows:XML&gt;

        &lt;SetList&gt;702f059d-71f2-4f78-a41a-48978d381948&lt;/SetList&gt;

        &lt;View&gt;

          &lt;ViewFields&gt;&lt;FieldRef Name='ID' /&gt;&lt;/ViewFields&gt;

          &lt;ViewBody&gt;&lt;Column Name='ID'/&gt;&lt;HTML&gt;,&lt;/HTML&gt;&lt;/ViewBody&gt;

        &lt;/View&gt;

      &lt;/ows:XML&gt;

    </SetVar>

  </Method>

</ows:Batch>

And this is the unescaped XML fragment passed to the PostBody SetVar parameter:

<ows:XML>

  <SetList>702f059d-71f2-4f78-a41a-48978d381948</SetList>

  <View>

    <ViewFields><FieldRef Name='ID' /></ViewFields>

    <ViewBody><Column Name='ID'/><HTML>,</HTML></ViewBody>

  </View>

</ows:XML>

So, in the outer method definition we have just the method type – Cmd parameter and the PostBody one. The actual stuff is the XML contained in the PostBody SetVar parameter. As you see it contains an <ows:XML> root element – this is an ancient element from the times of the STS used for rendering. So instead of the familiar <ows:Batch> element we see that there is (still) support for other STS CAML “container” elements. At this point you may ask yourself – can the <ows:XML> element be put directly into the batch string of the ProcessBatchData. Unfortunately this doesn’t work, so the indirection of the DisplayPost method is required here. And let’s have a look at the result that we have when executing this batch:

<Results>

  <Result ID="0" Code="0">

    1,2,3,391,392,444,445,446,447,448,449,450,451,452,453,454,

  </Result>

</Results>

So, what I actually did was defining a custom list view (using standard View CAML) within the <ows:XML> element and managed to retrieve some list data with it. The View element placed in an <ows:XML> effectively forces the rendering of the view definition provided in it. You can also provide a fully blown view definition copied from a SharePoint list schema file and the method will return HTML that you see normally in ListView web parts. And you can construct a view definition with custom ViewBody, ViewFields and Query elements that can be used either for rendering purposes or for data retrieval. For data retrieval you will create perhaps a smaller definition with Query part and ViewBody enumerating the fields probably using some unique separator for which you know that it’s not contained in some of the values. For rendering purposes you can specify also custom ViewHeader, ViewFooter and ViewEmpty elements. If you want to render a standard view, you can provide an empty View element with just a Name attribute like this:

<View Name="{CE0FFB35-F6A9-4F57-B06C-374B2AA4571B}" />

Note the upper cased, curly braced view ID in the Name attribute.

And the conclusion about this method is – well … if you have a knack for CAML (may be gone soon – beware of SP 2010) you can use it for both rendering and data retrieving purposes. For the former – you can start with a standard view definition and introduce minor changes without the need of creating custom list schemas (meaning creating custom list templates). For the latter you can take advantage of the fact that you can get many result sets with one call using batches (either placing several DisplayPost methods in the ProcessBatchData batch or several View elements inside the <ows:XML> element in the PostBody) and that the result data will be just as small in size as you specify for its constructing in the ViewBody element as opposed to the produced XML-s or SPListItemCollection’s data if you use the trivial methods for list item data retrieval.

Sunday, September 6, 2009

SPWeb ProcessBatchData – Display method

Basically all methods of the deprecated WSS RPC Protocol (MSDN) can be invoked via the SPWeb’s ProcessBatchData method. Since there’s little to no documentation regarding most of the RPC methods as to their usage directly with ProcessBatchData we can suspect that their support may be dropped in the new SharePoint 2010 (we still have to see how much of the CAML will be thrown out of it, and we’re talking about some really ancient stuff which’s been around since the times of the SharePoint Team Services). Most of the RPC methods were designed to serve specific tasks in the old STS but with the developing of the SharePoint object model most of these could now be much easier carried out with the latter. But … the thing is that even now in SharePoint 2007 the RPC methods are left intact and the ProcessBatchData is advertised as a faster method for inserting, updating and deleting (at least for that, but this may be true for other usages as well) of multiple list items than the standard SPListItem object model implementation.

So, let’s have a look at the Display method. A sample XML for calling it may look like this:

<?xml version="1.0" encoding="UTF-8"?>

<ows:Batch OnError="Continue">

  <Method ID="0">

    <SetList Scope="Request">702f059d-71f2-4f78-a41a-48978d381948</SetList>

    <SetVar Name="Cmd">Display</SetVar>

    <SetVar Name="XMLDATA">TRUE</SetVar>

    <SetVar Name="View">{FF964526-BBFA-4500-A43C-F4B7785E5F71}</SetVar>

  </Method>

</ows:Batch>

The SetList element contains the SPList ID, the Cmd SetVar – the method’s name. The XMLDATA SetVar parameter is actually mandatory but even if its value is FALSE the output of the method will be in XML format, not an HTML presentation of the list data. The View SetVar parameter is not mandatory – if it is not present, the default view of the list will be used, but the GUID format is important here – it should be with curly braces and uppercase. The result XML from the ProcessBatchData method will look something like this:

<Results>

  <Result ID="0" Code="0">

    <xml xmlns:s='uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882'

         xmlns:dt='uuid:C2F41010-65B3-11d1-A29F-00AA00C14882'

         xmlns:rs='urn:schemas-microsoft-com:rowset'

         xmlns:z='#RowsetSchema'>

      <s:Schema id='RowsetSchema'>

          <!-- schema data removed for brevity -->

      </s:Schema>

      <rs:data>

        <z:row ows_LinkTitle='some item' ows_fdate='2009-06-08 04:15:00' ows_flookup='1;#first' ows__UIVersionString='46.0' ows__ModerationStatus='2' ows_Editor='1;#Stefan Stanev' ows__Level='2' ows_ID='1' ows_owshiddenversion='104' ows_UniqueId='1;#{D2D82ED2-CBE7-4A56-84EC-3502FB8C2611}' ows_FSObjType='1;#0' ows_Created_x0020_Date='1;#2009-06-07 11:03:04' ows_Created='2009-06-07 11:03:04' ows_FileLeafRef='1;#1_.000' ows_FileRef='1;#sites/1/Lists/somelist/1_.000' />

        <z:row ows_LinkTitle='another item' ows_flookup='4;#one;more' ows__UIVersionString='22.0' ows__ModerationStatus='2' ows_Editor='1;#Stefan Stanev' ows__Level='2' ows_ID='2' ows_owshiddenversion='56' ows_UniqueId='2;#{9C1D1419-742A-41DC-85E1-1B1E7A50A2C8}' ows_FSObjType='2;#0' ows_Created_x0020_Date='2;#2009-06-07 11:03:27' ows_Created='2009-06-07 11:03:27' ows_FileLeafRef='2;#2_.000' ows_FileRef='2;#sites/1/Lists/somelist/2_.000' />

      </rs:data>

    </xml>

  </Result>

</Results>

The XML schema of the result (the part within the corresponding Result element) is actually identical to the XML schema of the result of the GetListItems method of the standard Lists web service (though the former is generated in the COM owssvr.dll library and the latter in the Microsoft.SharePoint.dll assembly with managed code – in the getter of the SPListItemCollection.Xml property). The result XML contains the data of all fields in the specified view plus several system fields and the returned items are sorted and filtered in accordance with the view’s query settings.

Several additional SetVar parameters can be used for simple filtering and sorting of the returned result set – basically these are the same that appear as query parameters when you sort or filter list views in the SharePoint UI – e.g. SortField, SortDir, FilterField1, FilterValue1, FilterField2, FilterValue2, RootFolder:

    <SetVar Name="SortField">Title</SetVar>

    <SetVar Name="SortDir">Desc</SetVar>

    <SetVar Name="FilterField1">Title</SetVar>

    <SetVar Name="FilterValue1">another</SetVar>

    <SetVar Name="RootFolder">/sites/1/Lists/somelist/fldr1/f1</SetVar>

If you use the RootFolder parameter with * as value you will get all items in the list recursively.

There is yet another optional SetVar parameter that can be used with the Display method – the Query parameter. Despite its name, you cannot specify a view CAML query in it, but just a list of field names, separated with spaces – basically with it you can specify the view fields that will appear in the result (the extra system fields will appear too). An unpleasant side effect of using it is that it effectively overrides the specified view’s query settings – so you end up with an unfiltered and unsorted result set – the effect of the SortField, FilterField1, etc SetVar-s (if present) is not affected though. A sample usage of the Query parameter (the asterisk – * – value can be used here as well):

    <SetVar Name="Query">ID Title</SetVar>

So, the conclusion about the Display ProcessBatchData method is that it is yet another way to retrieve SharePoint list item data, though it is not as flexible as the other mechanisms for list item data fetching – at least in respect to the query options that can be used for filtering and sorting the result set. One advantage though may be that like the other ProcessBatchData methods it can be executed in batches – so with one call of the ProcessBatchData method you will be able to retrieve the data from several lists.

Saturday, August 29, 2009

Two small SPListItem extension methods

So, it is about two small extension methods which do basically the same job – getting a specific field value from a SPListItem instance. Since LINQ to SharePoint is still not quite popular (let’s see if SharePoint 2010 will change that) the SPListItem indexer is the usual way to get the data from a SharePoint list item. It does the job but its main disadvantage is that it operates with objects which means no type safety, cumbersome code, type casts, additional checks, etc.

These two extension methods are generics methods as well – so in the generics parameter you basically specify the return type of the method. And why two – it’s simple – because of the big dichotomy in .NET types – reference and value types. The first method works with reference field value types, the second one with value types (check out the where clause in the methods’ declarations). And the latter’s return type is not actually the generics parameter type but its Nullable counterpart – the SPListItem’s indexer is always expected to return null-s, isn’t it?

    1     public static class ListItemHelper

    2     {

    3         public static T GetValue<T>(this SPListItem item, string fieldName) where T : class

    4         {

    5             object o = item[fieldName];

    6             if (o == null || !(o is T)) return null;

    7             return (T)o;

    8         }

    9 

   10         public static Nullable<T> GetValue2<T>(this SPListItem item, string fieldName) where T : struct

   11         {

   12             object o = item[fieldName];

   13             if (o == null || !(o is T)) return null;

   14             return (Nullable<T>)(T)o;

   15         }

   16     }

And here’s a small sample of how to use the methods:

    1     SPListItem it = list.Items[0];

    2     string title = it.GetValue<string>("Title");

    3     DateTime? created = it.GetValue2<DateTime>("Created");

Overloads of the methods which expect the GUID SPField ID-s can also be created.

Thursday, August 6, 2009

How to display all versions in a SharePoint list view page

This is a small article about several undocumented query parameters that can be used on SharePoint list view pages to extract and display additional list data, e.g. the previous versions of the displayed list items.
The parameter that forces the display of all versions of the list items is “IncludeVersions”, it can be used like this:

http://someserver/sites/1/docs/Forms/AllItems.aspx?IncludeVersions=TRUE

A big note here – when you click the context menu commands for the list items they will be applied on the latest version of the item, and for document libraries always the latest version of the document will be opened. So the net value here is only the possibility to see the differences between the values of the displayed fields in the different item versions.
[UPDATE: check this posting for a possible work-around]
Another parameter is the “RootFolder” one, especially when the value that is provided for it is the asterisk character – then it forces a flat view of the list items, analogous to the “RecursiveAll” view scope option:

http://someserver/sites/1/docs/Forms/AllItems.aspx?IncludeVersions=TRUE&RootFolder=*

And here is how using the “IncludeVersions” parameter in conjunction with the well known “FilterFieldN” and “FilerValueN” parameters you can display all latest approved versions of your list (as if you are a user with reader rights who can’t see the pending or draft versions, but sees the latest approved versions of the items that are now pending):

http://someserver/sites/1/docs/Forms/AllItems.aspx?IncludeVersions=TRUE&RootFolder=*&FilterField1=_ModerationStatus&FilterValue1=0&FilterField2=_IsCurrentVersion&FilterValue2=1

Note the fields used to filter all versions – the _ModerationStatus and _IsCurrentVersion, the value 0 for the moderation status field corresponds to the approval status “Approved”.

Monday, July 27, 2009

New version of the SPListItem editor released

Here is a small video with the new features:

The latest release can be found here: http://splistitemeditor.codeplex.com/

Friday, July 17, 2009

Tips for using SPWeb.ProcessBatchData

There’re quite some samples about how to use the ProcessBatchData method of the SPWeb class on the internet but it turns out that the full capabilities of the method, i.e. the input XML that it uses are not that well documented. So here’re several examples that I managed to get to work while working on my SPListItem editor tool:

  • Create a folder in a SharePoint list/library:

<?xml version="1.0" encoding="utf-8"?>
<ows:Batch OnError="Continue">
  <Method ID="Test">
    <SetList Scope="Request">82d62a9a-55ba-49c8-a9b8-68ec965a5931</SetList>
    <SetVar Name="Cmd">Save</SetVar>
    <SetVar Name="ID">New</SetVar>
    <SetVar Name="Type">1</SetVar>
    <SetVar Name="owsfileref">/sites/1/docs/folder1</SetVar>
  </Method>
</ows:Batch>

The critical line here is the setting of the Type parameter – it is actually an alias of the system FSObjType field and without it the call won’t work.

This way you can create subfolders too, e.g. using:

    <SetVar Name="owsfileref">/sites/1/docs/folder1/sub1</SetVar>

The only requirement is that the parent folder exists, if it doesn’t – the call will fail.

  • Rename a file or a folder in a SharePoint list/library:

<?xml version="1.0" encoding="UTF-8"?>
<ows:Batch OnError="Continue"><Method ID="Test">
  <SetList Scope="Request">82d62a9a-55ba-49c8-a9b8-68ec965a5931</SetList>
  <SetVar Name="Cmd">Save</SetVar>
  <SetVar Name="ID">28</SetVar>
  <SetVar Name="owsfileref">/sites/1/docs/fld2</SetVar>
  <SetVar Name="owsnewfileref">fld1</SetVar>
</Method></ows:Batch>

So you need both the server relative URL of the file or folder object in the owsfileref parameter and the ID of the associated list item here. The new name is provided in the owsnewfileref parameter. One remark here – you can provide the new name without the extension part – the extension will be added automatically for files. In case you provide an extension (which differs from the original extension of the file) it will be ignored.

  • Using the owshiddenversion field:

Using this field is very important especially when you have versions enabled or mandatory check out for editing in document libraries. In order that you can use it, you first need to retrieve the list items having set the ViewFields property of the SPQuery object to contain the owshiddenversion field. And you need to provide the same value of the owshiddenversion field in the ProcessBatchData XML as you retrieved originally with the list item. A sample XML would look like:

<?xml version="1.0" encoding="UTF-8"?>
<ows:Batch OnError="Continue">
  <Method ID="M0">
    <SetList>82d62a9a-55ba-49c8-a9b8-68ec965a5931</SetList>
    <SetVar Name="Cmd">Save</SetVar>
    <SetVar Name="ID">23</SetVar>
    <SetVar Name="owsfileref">/sites/1/docs/codes.txt</SetVar>
    <SetVar Name="owshiddenversion">93</SetVar>
    <SetVar Name="urn:schemas-microsoft-com:office:office#Title">some title</SetVar>
  </Method>
</ows:Batch>

Basically this field is auto-incremented with every update of the list item. SharePoint checks the value of the field and compares it to the value of the current version of the list item. So if you have it in the update XML and another user has updated the item before you, you will receive the standard error message of saving conflict with a concurrent user. And in case the file in a document library that you try to update is not checked out, you’ll get again a standard error message. In both cases if you don’t include the owshiddenversion field in the update XML the update will actually succeed and you will end up with either overwriting the changes of another user or breaking the concurrence constraints that are normally enforced with the check-out mechanism.

And if you intend to do several updates in a row on a certain list item you should either re-fetch the item after each update to have the updated value of the owshiddenversion field for the next update (this is what actually happens when you update an item with SPListItem.Update) or be more economical and increment the value by one yourself.

  • Change the moderation status of a list item with the moderate command

OK, so it was known for some time that next to the Save command there is an undocumented Moderate command for updating the list item moderation status (setting an item to approved, rejected, draft, etc) using the ProcessBatchData method. Here is a sample XML:

<?xml version="1.0" encoding="UTF-8"?>
<ows:Batch OnError="Continue">
  <Method ID="M0">
    <SetList>82d62a9a-55ba-49c8-a9b8-68ec965a5931</SetList>
    <SetVar Name="Cmd">Moderate</SetVar>
    <SetVar Name="ID">26</SetVar>
    <SetVar Name="owsfileref">/sites/1/docs/a.txt</SetVar>
    <SetVar Name="owshiddenmodstatus">3</SetVar>
    <SetVar Name="urn:schemas-microsoft-com:office:office#_ModerationStatus">0</SetVar>
    <SetVar Name="owsitemlevel">2</SetVar>
    <SetVar Name="_Level">2</SetVar>
    <SetVar Name="owshiddenversion">12</SetVar>
  </Method>
</ows:Batch>

Note the usage of two more “hidden” fields here – the owshiddenmodstatus and owsitemlevel – they should contain the values of the original values of the _ModerationStatus and _Level fields when the list item was retrieved. The urn:schemas-microsoft-com:office:office#_ModerationStatus parameter should contain the new value of the moderation status. Actually these two are required only for document libraries – for lists you can provide just the urn:schemas-microsoft-com:office:office#_ModerationStatus parameter. Using this syntax you can quickly approve items in batches, and a clear difference with the Save command is that the Moderate command works on not checked out items (this holds for document libraries) and actually fails when the item is checked out.

Several tips for “hacking” the ProcessBatchData XML syntax

A good starting point is to check the standard SharePoint “Lists" web service – especially the UpdateListItems method. It uses a similar XML syntax for updating list items, and basically it’s easier to get things working with it – no fancy internal parameter names are needed. And the thing is that the web method internally uses SPWeb.ProcessBatchData and the input XML is being “translated” to the ProcessBatchData form. A nice way to check the work of the Lists.UpdateListItems is to open a list in datasheet view and check the requests sent to the web service with a web debugging proxy as fiddler. This way you’ll have real time XML samples – after all it’s obvious that the Lists.UpdateListItems was designed for use primarily by the standard list datasheet view.

As for the “translation” of the Lists.UpdateListItems XML to the ProcessBatchData form you can reflect the code of the standard SharePoint STSSOAP.DLL assembly – it’s located in the _app_bin subfolder of your SharePoint web applications. The “translation” is implemented in the ConstructCaml method of the Microsoft.SharePoint.SoapServer.ListDataImpl class. I actually got this method working in a small WinForm application, here is the sample code:

        private void TestConstructCaml()
        {
            System.Web.Services.WebService svc = new System.Web.Services.WebService();
            CreateHttpContext();
            test.ListDataImplProxy p = test.ListDataImplProxy.CreateUnderlyingInstance(svc);

            string listName = "docs";
            string xml = "<Batch OnError=\"Continue\"><Method ID=\"M0\" Cmd=\"Moderate\"><Field Name=\"ID\">22</Field><Field Name=\"FileRef\">/sites/1/docs/zterminal.csv</Field><Field Name=\"_ModerationStatus\">0</Field><Field Name=\"_Level\">2</Field><Field Name=\"owshiddenversion\">45</Field></Method></Batch>";

            string s = p.ConstructCaml(listName, xml);
            Console.WriteLine(s);

        }

        private void CreateHttpContext()
        {
            HttpContext.Current = null;
            string url = "http://racoon-vpc-hg:2909/sites/1";
            SPSite site = new SPSite(url);
            SPWeb web = site.OpenWeb();

            HttpRequest r = new HttpRequest("default.aspx", url, "");
            HttpResponse rs = new HttpResponse(null);
            HttpContext.Current = new HttpContext(r, rs);

            HttpContext.Current.Items["HttpHandlerSPWeb"] = web;
            HttpContext.Current.Items["HttpHandlerSPSite"] = site;

        }

With this code I was able to check the translated ProcessBatchData XML from a Lists.UpdateListItems input XML. The ListDataImplProxy class is a reflection proxy class which calls the methods of the ListDataImpl class using reflection. It was generated with my reflection proxy generating utility, which can be downloaded from here: http://stefan-stanev-sharepoint-blog.blogspot.com/2009/05/how-to-access-non-public-class-members.html

Sunday, July 5, 2009

SPQuery Scope and ModerationType ViewAttributes

Basically there is some sporadic info here and there about these on the internet, for instance that the “Scope" attribute can take the values of “Recursive” and “RecursiveAll”.

So to wrap it up:

The possible values of the “Scope” attribute are:

  • Default
  • FilesOnly
  • Recursive
  • RecursiveAll

The possible values of the “ModerationType” attribute are:

  • HideUnapproved
  • Contributor
  • Moderator

It turns out that these two are analogous to the Scope and ModerationType properties of the SPView class, which are briefly explained here and here.