Friday, April 30, 2010

Updating read-only fields with the Lists web service

This is a known limitation (from a user’s perspective that is, obviously Microsoft had a good reason to design it this way) – you cannot edit fields like the Created, Modified, Created By, Modified By, etc using the standard SharePoint Lists web service and its UpdateListItems method (Updated 2011-1-20 – the Created By (Author) and Modified By(Editor) fields can be updated in this manner only for non-library SharePoint lists). On the other hand using the object model with SPListItem.UpdateOverwriteVersion or SPWeb.ProcessBatchData you can easily modify the values of read-only fields. And back to the Lists web service – I recently found a small work-around to this issue – the idea is quite simple – if the problem is with read-only fields why just not change the ReadOnly attribute in the field’s schema and check to see whether this will work. I tested the schema changing and it turned out that the UpdateListItems method was now able to change the values of the modified fields. The next question is whether it is possible to make this field schema change with a web service – the answer is again yes – the Lists web service has the UpdateList method which allows among other things to modify the schema of the fields in the list. So, combining the two methods it is possible to first call UpdateList and change the fields whose values should be modified to non read-only, then call UpdateListItems and update the values in the list items and finally call UpdateList again to have the read-only attribute set back to the original value (check the sample code snippet below). Note here that setting the ReadOnly schema attribute back to TRUE is necessary otherwise you will see some side effects like the fields appearing in the new and edit forms of the list, the Modified field will stop reflecting the last modified date, etc. Another important note – if the field is sealed (its “Sealed” attribute is set to TRUE) – the Lists.UpdateList cannot update its schema so the whole approach is unusable in this case (unless you set the “Sealed” attribute to false with the object model or a tool which further complicates the issue).

And now several words about the disadvantages of this work-around (a little attempt to discourage you from using it) – let me start with the schema changes – it is obviously not natural to change the schema of the list (or its children) to get an update of its list items right, another thing is that these schema changes and reverts will become a real mess if you try to run several updates simultaneously. So my advice would be that this approach should be used only in a one-time batch updates and/or when there is really no alternative to using the standard SharePoint web services (no option for using the object model or to create and deploy a custom web service).

And here is a small sample code snippet that demonstrates this work-around:

        public void Test()

        {

            string webUrl = "http://myserver";

            string listName = "docs";

            Lists.ListsSoapClient listsClient = this.GetListsClient(webUrl);

 

            // 1st a call to Lists.GetList - we need the list's version - it is returned in the Version attribute

            XElement listData = XElement.Parse(listsClient.GetList(listName).OuterXml);

            string listID = listData.Attribute("ID").Value;

            string version = listData.Attribute("Version").Value;

            // in the updateFields parameter of Lists.UpdateList the full schema of the fields should be provided

            string updateFields = @"<Fields>

   <Method ID='1'>

      <Field ID='{28cf69c5-fa48-462a-b5cd-27b6f9d2bd5f}' ColName='tp_Modified' RowOrdinal='0' ReadOnly='FALSE' Type='DateTime' Name='Modified' DisplayName='Modified' StorageTZ='TRUE' SourceID='http://schemas.microsoft.com/sharepoint/v3' StaticName='Modified' FromBaseType='TRUE' Version='4' ShowInNewForm='FALSE' ShowInEditForm='FALSE' />

   </Method>

   <Method ID='2'>

      <Field ID='{8c06beca-0777-48f7-91c7-6da68bc07b69}' ColName='tp_Created' RowOrdinal='0' ReadOnly='FALSE' Type='DateTime' Name='Created' DisplayName='Created' StorageTZ='TRUE' SourceID='http://schemas.microsoft.com/sharepoint/v3' StaticName='Created' FromBaseType='TRUE' Version='4' ShowInNewForm='FALSE' ShowInEditForm='FALSE' />

   </Method>

</Fields>";

            XmlDocument doc = new XmlDocument();

            doc.LoadXml(updateFields);

            // Lists.UpdateList: set fields to not read-only

            XElement result = XElement.Parse(listsClient.UpdateList(listID, null, null, doc.DocumentElement, null, version).OuterXml);

            // get updated version from the result XML - for the second call of Lists.UpdateList

            version = result.Elements().Where(el => el.Name.LocalName == "ListProperties").First().Attribute("Version").Value;

 

            // prepare the XML for the list item update

            string updateDates = @"<Batch OnError='Continue'>

  <Method ID='M0' Cmd='Update'>

    <Field Name='ID'>1</Field>

    <Field Name='FileRef'>/docs/zt.txt</Field>

    <Field Name='Modified'>2010-04-04T22:17:00Z</Field>

    <Field Name='Created'>2010-01-01T00:05:00Z</Field>

  </Method>

</Batch>";

 

            doc.LoadXml(updateDates);

            // Lists.UpdateListItems: update Created & Modified

            result = XElement.Parse(listsClient.UpdateListItems(listID, doc.DocumentElement).OuterXml);

 

            // revert the fields' schema

            updateFields = updateFields.Replace("ReadOnly='FALSE'", "ReadOnly='TRUE'");

            doc.LoadXml(updateFields);

            // Lists.UpdateList: set fields back to read-only

            result = XElement.Parse(listsClient.UpdateList(listID, null, null, doc.DocumentElement, null, version).OuterXml);

        }

The code demonstrates how to update the Created and Modified fields of a document library item (check the comments inside the code for more details). Note that this is a sample quality code and you shouldn’t use it directly without serious improvements – for example using a try-finally block with the second call to the Lists.UpdateList method placed in the “finally” block; adding an extra call to Lists.GetList to get a fresh value of the “Version” attribute of the list, just before the second list update, etc.

58 comments:

  1. You make my day. Thank you for your great and nice work.

    John

    ReplyDelete
  2. i can't make this work.. plz tell me what namespaces needed.. i'm tired trying it

    ReplyDelete
  3. Hi anonymous,
    probably you have problems with the "Lists.ListsSoapClient" reference - it is just a service reference (WCF) to the SharePoint Lists web service. You can check the source code of this tool - http://splistitemeditor.codeplex.com/SourceControl/changeset/view/42195# and more specifically the wschelper.cs file.

    ReplyDelete
  4. Hi Stefan, I currently have two windows services and both has function calls to the UpdateListItems of the Sharepoint site by having two separate instances of the web service. I noticed that the update made by one of the windows service wasn't reflected in the site. I'm looking at the possibility that both windows service made the call simultaneously. If such occurs, will sharepoint accept just one call and what would happen to the other?

    ReplyDelete
  5. Hi anonymous,

    I know about this problem - I mentioned it as a big disadvantage of this approach in the posting (you can see the paragraph starting with "And now several words about the disadvantages of this work-around").
    So what I can suggest is to set the fields once to not-read-only without setting them back to read-only after the import of the items is complete. The problem with the fields being not-read-only will be that the Modified field will stop to reflect the last update date of the item, but if the items/documents are not being used while you import new items that shouldn't be a problem. Then the final reverting of fields' schema can be postponed for a later moment - if this will be applicable in your case.

    ReplyDelete
  6. Hi Stefan,
    I have a little problem with this approach. It works pretty well on one site but I am getting 401: Unauthorized on other site. The error message is returned in the moment when I am calling UpdateList method with CAML containing all information needed to set readonly attribute to false. Don't you know, where the problem could be? I have full control permissions for both sites (working/nonworking) and lists. Maybe the problem could be in Sealed attribute, but I could'nt see or find it. Thanks

    ReplyDelete
  7. Hi Frantisek,
    can you provide some more information - is the site under the same site collection as the first one or is it in another site collection, or even another web application or farm. About checking and setting the Sealed or ReadOnly attributes of the Created or Modified fields you can use the SharePoint manager tool (http://spm.codeplex.com/) - expanding the object hierarchy tree you can locate your containing list and your fields and edit their schema either directly (through the SchemaXml) property or modifying the corresponding properties of the SPField class. After you set these with the tool you can remove the calls to the Lists.UpdateList method and keep the Lists.UpdateListItems call only - it is important to know whether you have some general permission issues or it is only the UpdateList functionality.

    ReplyDelete
  8. Hi Stefan,
    thanks for your reply. I don't know, what you exactly mean with the same site collection but probably yes. Both of them resides under the same parent site on the same Sharepoint server. I've found that the site, where presented method fails, was previously migrated from SP2.0 to SP3.0 (the other one was completely built on SP3.0), so maybe it can be a problem (due different schema or something like thal, I have no idea). As I mentioned before, I have completely the same permissions for both sites and both lists (Full permissions). Thanks for the link to that tool, but I am afraid I will not be able to use it, because I have no adninistrative access to the sharepoint server.

    ReplyDelete
  9. Hi Frantisek,
    Have you tested the code without the calls to the Lists.UpdateList method (it won't be able to update the read-only fields but the update of the other fields should be OK). The fact that the site was migrated from the older version of SharePoint may have something to do with your problem - maybe the schema of the Fields is somewhat different. Btw - you can check the schema of the fields using the Lists.GetList method and compare the fields from the two lists.

    ReplyDelete
  10. Hi Stefan,
    I tried the code without Lists.UpdateList (I've breakpoint set on it and so I know that all previous code work and I have a field schema in XML readed from the server). I tried to update the field with exactly the same information, what I readed from the server but with no success. I didn't tried to update any other field, but I can try it. You are asking about the field schema difference. Yes, there is a difference but I don't think it can have any major impact. There are few more attributes in the schema of the site built directly on SP3.0. Common attributes are: ID, ColName, RowOrdinal, ReadOnly, Type, List, Name, DisplayName, SourceID, StaticName, FormBasedType (difference is only in the ID Field values). Additional attributes for SP3.0 which are not present in the migrated site are: Version(41), Required(false), Group(""), ShowField(""). I compared schemes of lists as well and it can be maybe more interesting for you. All attributes names are the same for both of them. Differences are only in the attribute values (first value in the brackts is from imported schema and second from original SP3.0 - only interesting fields included, ID and URL like attributes are not included): FeatureID(GUID,GUID), ServerTemplate(100,103), Version(21, 1675), Flags(536875017, 612372616), ItemCount(4, 6), ReadSecurity(1,2), WriteSecurity(1,2), HasUniqueScopes(false, true), EnableVersioning(false, true), Author(id, id).
    Thanks for your help

    ReplyDelete
  11. Hi again Stefan, I am sorry I interuupted you, finally I got it work... I am idiot, I had a bad list GUID set (good in constant, but I've not used it :D) and because there are two lists with similar names in the sharepoint site, I overlooked it in the result XML returned by the server... Your suggestion to try different field was good :), because I've fount there are completely other data thani in the list I am trying to modify. It works perfectly ofcourse I am sorry again. Thanks for great article.

    ReplyDelete
  12. Hi Stefan, I tried your code with to change the modified by field, but it didn't work it is always set to the system account instead of the logged in user

    ReplyDelete
  13. Hi Anonymous,
    can you try set the ReadOnly attribute of the "Modified by" (it's internal name is 'Editor') manually with a tool - e.g. the SharePoint Manager utility (http://spm.codeplex.com/) and then try to update its value with the List item editor tool (http://splistitemeditor.codeplex.com/), using the web services mode. Do you just try to update the "Modified by" value or you also change the moderation status of the item (setting it to approved)?

    ReplyDelete
  14. Hi, Thanks for your post. I am able to update Created time and modified time, but not able to update Created by (Author) and Modified by (Editor) fields after setting them read only false. Is it possible ?

    ReplyDelete
  15. Hi Anonymous,
    I rechecked and I saw that the Author and Editor fields indeed cannot be updated in this way for document libraries. This works only for non-library lists. I am going to update the posting with this important bit.

    ReplyDelete
  16. Hi Steffan,
    I tried your code for a library list and it worked fine! The only difference is that i had to keep in a list the name and the id of each file so i can retrieve the id by giving the filename and use it in the batch!

    Hope this helps!

    ReplyDelete
  17. Hi Everyone,
    Any luck with the library lists?

    Thanks.

    ReplyDelete
  18. Should this method work with wss 2.0? It doesn't change modified or created fields. Other fields can be changed though.

    ReplyDelete
  19. Hi Anonymous,
    Honestly, I have no clue about that. I haven't worked with SharePoint 2003 and wss 2.0 for quite some time now.

    ReplyDelete
  20. Hi Stefan,
    is it possible update CreatedBy and ModifiedBy Fields direct with Web Service, withouth have to set readonly properties of both fields to false?.
    Is walkaround valid for old sharepoint 2003 Web Service=
    Thankx a lot ;)

    ReplyDelete
  21. Hi Anonymous,
    It doesn't work for sure with the Lists web service without having the fields set to not readonly beforehand. It doesn't work in SharePoint 2010 using the new RESTful ListData.svc either.
    Unfortunately I have no idea whether this workaround works in SharePoint 2003 (I haven't worked with it for 3-4 years now).

    ReplyDelete
  22. God Bless You!! You saved my day!!!

    ReplyDelete
  23. Hi Stefan,

    I tried to use above code for Created by and Modified by fields but it's not working. i want to update above fields using web service. please guide me how to update modified by and created by field

    ReplyDelete
  24. Hi Anonymous,

    a quick question - are you trying to update the "created by" and "modified by" fields in a document library or in a non-library list. Note that for a document library this approach won't work as I mentioned in the beginning of the posting (see the sentence in red).

    Greets
    Stefan

    ReplyDelete
  25. Hi stefan,

    I am trying update above fileds from document library.Please tell me any another way can do it.

    ReplyDelete
  26. Hi Anonymous,

    Unfortunately it is not possible to achieve this using the standard SharePoint Lists web service (this is by design). You will have to go for a custom solution here - either with a custom web service or with a non-web utility which uses the SharePoint object model and runs on one of the servers in the SharePoint farm.

    Greets
    Stefan

    ReplyDelete
    Replies
    1. Hi Stefan Stanev
      Please, Could you give me a link detail from msdn?
      Thanks/VinhPham

      Delete
  27. Can we update the list schema xml property which is read only through Client Object Model??..
    I need to change the readSecurty and WriteSecurity attributes of SchamaXML properties of list ..
    Thanks In Advanced

    ReplyDelete
  28. Hi Sangram,
    No, I don't think that it is possible to change the schema xml property with the Client Object Model. It may be possible to change the ReadSecurity and WriteSecurity properties using the Client.svc web service though (this is the web service which the Client Object Model calls internally).
    You can check the exact calls to this service using an HTTP monitoring utility (like Fiddler 2) while traversing the hierarchy of the Client Object model with my small utility - SharePoint 2010 explorer (it is available on my blog).

    Greetings
    Stefan

    ReplyDelete
  29. Thanks Stefen, I could not see any parameter passing in ProcessQuery in Client.svc at fiddler..is there any other way to do it?

    ReplyDelete
    Replies
    1. Hi Sangram,

      I tried it myself and it didn't work with the Clients service. I used this XML:

      <Request AddExpandoFieldTypeSuffix="true" SchemaVersion="14.0.0.0" LibraryVersion="14.0.4762.1000" ApplicationName=".NET Library" xmlns="http://schemas.microsoft.com/sharepoint/clientquery/2009">
      <Actions>
      <SetProperty Id="195" ObjectPathId="116" Name="ReadSecurity">
      <Parameter Type="Int32">2</Parameter>
      </SetProperty>
      <Method Name="Update" Id="196" ObjectPathId="116" Version="18" />
      </Actions>
      <ObjectPaths>
      <Identity Id="116" Name="740c6a0b-85e2-48a0-a494-e0f1759d4aa7:web:9ea1eaf4-2116-4c56-93b3-d5f08c3ee805:list:6f1a9fa7-e41a-4a32-953d-8ad7ac31fd38" />
      </ObjectPaths>
      </Request>

      It gives an error that the field doesn't exist (although it gets returned when you request all the properties of the list). The old SharePoint 2007 style Lists.asmx with its UpdateList method also can't be used for these properties and I can't think of other standard SharePoint web services that may work for this. If you have a strict requirement that this should get done using a web service you should probably consider writing a simple custom one - this in case that it will be possible to deploy custom services to your target server.

      Greetings
      Stefan

      Delete
  30. Thanks Stefen for your valuble reply. I am trying this for Office 365 so i could not deploy web service on the target server, also tried to use SharePoint designer but there also I couldnt find the option to do these settings.

    ReplyDelete
  31. Hi Sangram,
    a quick question - is the "Item-level Permissions" bit available in the "List Settings/Advance Settings" page in Office 365 (it is there in standard SP 2010 at least for non-library lists)? If yes I suppose that it will be possible to automate the POST action of the page.
    Let me know what you think.

    Greets
    Stefan

    ReplyDelete
  32. Hi Stefen,
    Yes, It is availbale on o365 like SP2010 for non library lists. what is mean by POST action the page??

    ReplyDelete
  33. Hi Stefen,
    One more question I have today, I am trying to move listitem from source to target site..
    I have problem to moving lookup fields item because as they are refering source site Listitem ID(which from List I refered for lookup fields at source site)which is obviously not same on target site(if ID's same on target list which i refering for target lookupfield then I am fine, but ID generate auto so it's not same )..Can you help me on that??

    ReplyDelete
  34. Hi Sangram,
    I meant a POST HTTP request - in the browser you submit the HTML form of the page and the browser issues an HTTP POST request to the server. Basically you should be able to simulate this POST request with code using the HttpWebRequest class. For that you will need to check the exact POST parameters that the page sends to the server and take care of the "__REQUESTDIGEST" parameter (available in a hidden field in the HTML form) which has different value on every page load, meaning that before emulating the POST request you will have to do a GET request to load the page so that you can retrieve the value of this parameter. Not a very straight-forward job but should be doable.
    Regarding your second question - as I see it, you will have to have an extra field in the target list of the referenced items (the list that is pointed to by the lookup field). This field should serve as a secondary unique identifier which should not change when you move these items between the source and target list. If you have a document library - it should be easy - you can use the file name or the file path relative to the library's root folder. If you have a non-library list you should add a custom field (e.g. source_id or something similar) which you should populate with the values of the source items' ID field. When you have this auxiliary field to identify the relation between the source and destination reference items you will be able to adjust the lookup field values in the list which points to these items.

    Greetings
    Stefan

    ReplyDelete
    Replies
    1. Thanks Stefen, as a field look up problem worked for me.but, honestly i dont want to create new custom field for the reference ID of the lookup list..Is there no API availble in COM to identified the same.I di not try for POST request things as i have to tried it for permissions..

      Delete
  35. Hi Stefen,

    one more problem :), I created lookup Field on List as "Create Column" which not associated to any content type..I want to create of source list Field(lookup) with target site in same list name and same refering list for look up..i trried with create field as Field.SchemaXML and replace the reference List ID from Source to target list ID.. field is creating but not refering that target list name in field setting..for that I could not move further..could you please help in that??

    ReplyDelete
  36. Hi Sangram,

    Can you please send all your questions not related to this blog posting to my email - code@stefan-stanev.com - I will be happy to help you.

    Greetings
    Stefan

    ReplyDelete
  37. This comment has been removed by the author.

    ReplyDelete
  38. Nice trick, worked great!

    I do have one question, I have been using this (modified a bit) to hide fields in certain forms. It works great for number and text fields but I can not figure out how to get it to work for calculated fields. I download the lists schema and find the calculated field that I would like to hide in the display form and set the ShowInDisplayForm="FALSE" (also tried setting it to 0) but it seems to just pass by that change. Any Ideas?

    ReplyDelete
  39. Hi Bill,

    you can also try with setting the "Hidden" attribute to FALSE - it will have basically the same effect - the calculated field will not appear in the display form.

    Greets
    Stefan

    ReplyDelete
    Replies
    1. sorry, I meant setting the Hidden attribute to TRUE

      Delete
  40. Hi Stefan,

    Your post says it is not achievable in Libraries. Can you please advise, how to achieve for Libraries? I am having a hard time, getting this to work for libraries. Any help would be highly appreciated.

    ReplyDelete
  41. Hi Ven,

    Unfortunately it is not possible to achieve this for document libraries using the Lists web service.If you are using SP 2010 or Office 365 you can check with the client object model, but as far as I could see it is not possible with it either. I checked for a third option - the Copy.asmx web service, but again it doesn't work. As far as I can see you will need to create a custom solution for this - a custom tool or a custom web service which use the server object model.

    Greetings
    Stefan

    ReplyDelete
  42. Thanks, stefan, you solve my big problem!

    ReplyDelete
  43. Thank you very much Stefan...

    ReplyDelete
  44. Hi Stefan,

    In sharepoint survey we have column "completed" if we have pageseperator.This field i want to update it using webservice.i follwed same procedure wat u have given in this post ..But it didnt work.I updated that "Completed" field to readable.when i try to survey response ,it is adding one more question instead of updating "Completed" column..Please give me some suggessions.

    ReplyDelete
  45. Hi Gomathi,

    it is not possible to update the "Completed" field because it is of type Computed and you can't update computed fields. The "Completed" field simply displays the value of the internal hidden _Level field as either "yes" or "no", and as far as I could see for surveys the value of the _Level field is always 1 which gets displayed as Yes in the "Completed" field.

    Greetings
    Stefan

    ReplyDelete
    Replies
    1. Hi Stefan,

      Thanks a lot. This "_Level" field am getting in response ad 255 if the survey is saved (not completed). Is there any other way to implementent save scenario using webservice..?

      Thanks
      Gomathi S

      Delete
  46. how could i get these values {28cf69c5-fa48-462a-b5cd-27b6f9d2bd5f}...

    ReplyDelete
  47. Hi Stefan,
    Is there any option to stop version change Against this modification date change?
    Thanks in advance

    ReplyDelete
  48. Hi Stefan

    I hoping your code can help me. I am wanting to copy the version number of a document into another field. The only way I've been able to do this so far is to run a workflow which causes the version number to increase as I need to check out the document to update the list. If this code can do the trick how do I use the code and where do I put it???

    Thanks in advance

    Mike

    ReplyDelete
  49. CSOM can change the Author field in a doc lib but versioning needs to be temporarily turned off. Also it will update the most recent version's Author field. So in order to set the Author field you would have to either start from scratch or do some other strange gymnastics to promote the oldest and do that up the line then delete all that duplication of versions.
    Anyway, so just use some code and turn off versioning, issue some code like item["Author"] = User.Id + ";#" + User.Id; item.Update(); ctx.ExecuteQuery(); and you should see the change. Can't believe nobody has ironed this out after all these years.. Can't believe I was banging my head over this for a week just to have to find this out for myself.

    ReplyDelete
  50. Thank you Anonymous,
    You make my day :D.

    ReplyDelete