Iterating SharePoint sites using a search query

This is a pretty simple query that can be used to iterate the sites and sub-sites under some URL and get the sites’ title, description and URL:

SELECT url, contentclass, title, description FROM scope() WHERE (contentclass='STS_Web' OR contentclass='STS_Site') and site='http://myserver/mysite'

The contentclass managed field is used to filter the “site” search items and the site managed field – to specify a starting URL the sites below which we want to retrieve. Normally the STS_Web and STS_Site items cause issues since they “hide” the welcome pages of the corresponding sites but in this case they come quite handy allowing the retrieval of some metadata (title and description) of the sites.

Update content types and site columns with feature receiver using the standard manifest file

The usual way of creating content types and site columns is by using an element manifest file with the standard Field and ContentType elements in a feature. The creation is quite easy but when it comes to updating you see some really hard times – first Microsoft strongly discourages the modifying of the original manifest files and second even if you do that the changes you made won’t get propagated to the target lists. Quite unpleasant, isn’t it. So, to address this problem I created a feature receiver that can read a manifest file and apply all changes to the existing artifacts propagating the changes to the site’s lists (actually this receiver is an enhanced version of my “common actions” receiver from my previous posting).

First – one big note – the updating receiver is still a beta quality code and should be used carefully – there is a command line utility in the receiver’s solution that allows starting the update routine directly and also supports a “test run” mode – so you may consider using this first and check the verbose notifications that it outputs as to which changes it is about to apply (more on this later).

Another important note – do not modify the element manifest file of the original feature that created the site columns and content types – use a separate file, a copy of the original one with the desired modifications. This is because before the artifact is updated with the object model its schema is read from the manifest file in the file system (you can check that easily – the schema is saved to the content database after the artifact gets its update method called). This also means that you can modify this way “ghosted” site columns and content types (MSDN recommends strongly against that) but not such that were already created or modified with the object model. The solution described here handles both cases.

And several words on what it actually updates and how exactly it applies the updates – 1st starting with the latter – only standard object model calls are used – the Update methods of the SPField and SPContentType classes are used with the option of pushing down the changes to the corresponding list fields and inheriting content types respectively. So the idea is not just to update the site columns and content types but to propagate these changes to the lists where these are actually used. The SPField objects are updated with setting the field schema from the manifest file to the SchemaXml property of the field – one note here – all attributes missing in the manifest schema that are present in the existing field schema are copied from the latter – so this is actually a merge of the two field definitions rather than a total overwriting from the schema in the manifest file (I wanted to keep safe here for cases like lookup fields which have attributes that are normally set after the field is created). So, in cases when the existing site columns are modified from the UI or from code after the site is created the updating with the receiver may result in a merge of the field definition rather than a definition that is identical to the one in a newly created site. Note – only existing site columns are being modified, if a column from the manifest file is not found in the site, it is skipped.

And about content types – two elements are updated here – the FieldRefs and the XmlDocuments. For FieldRefs – you can add, remove, change the attributes of the FieldRef-s of the content type as well as reorder them. All attributes for which public settable properties in the SPFieldLink class exist are set (exceptions being – ShowInNewForm and ShowInEditForm – no properties for them). XmlDocuments are also added, updated and deleted according to the changes in the content type schema. Something interesting (and maybe peculiar) here – the XmlDocuments are also inherited from the base content types – unless a XmlDocument with the same namespace exists in the derived content type which overrides the parent’s definition. This leads to some unexpected results especially with the list item event receiver XmlDocument – because of this inheritance mechanism you may end up with one and the same event receiver’s method being called more than once on a list item modification event (but that’s another story). Another “intrinsic” behavior with the event receiver XmlDocument – when you add receivers with it they are added to the lists to which the content type is bound, but if you remove the XmlDocument from the schema definition – these remain in the destination lists, though if you modify the receivers XmlDocument – the old receivers are removed and the new ones are added (basically this is what we have with the object model). I also tested modifications of the FormTemplates and FormUrls XmlDocument elements and they seem to work just fine. Note about content type updates – only existing site content types are being modified, if a content type from the manifest file is not found in the site, it is skipped.

Here is a feature.xml from a sample feature that uses the receiver:

<?xml version="1.0" encoding="utf-8"?>

<Feature

    Title="ZCommonTest"

    Description="ZCommonTest"

    Id="7A0F010D-7993-4bcd-9127-7B15FEFFC0FF"

    Scope="Web"

    Hidden="TRUE"

    DefaultResourceFile="core"   

    ReceiverAssembly="Stefan.Sharepoint.Util, Version=1.0.0.0, Culture=neutral, PublicKeyToken=338fd4db0d2cb397"

    ReceiverClass="Stefan.Sharepoint.Util.FeatureReceivers.CommonActionsReceivers"

    xmlns="http://schemas.microsoft.com/sharepoint/">

    <ElementManifests>

    </ElementManifests>

  <Properties>

    <Property Key="CTsFieldsUpdatePath" Value="fields.xml,ctypes.xml" />

  </Properties>

</Feature>

So basically, you need to set the appropriate ReceiverAssembly and ReceiverClass attributes and the custom Property element with Key=”CTsFieldsUpdatePath” – the value of the property element can contain one or many (comma separated) manifest files’ paths – the feature folder relative path actually.

In the receiver’s solution there is also a console utility project – the tool has a command for direct invocation of the update functionality, here is a sample call:

UtilTest.exe -o updatecontenttypesandfields -weburl http://racoon-vpc-hg:2909/sites/4 -manifestpath "C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\FEATURES\ZCommonTest\ctypes.xml" -verbose -log z.log

Most of the arguments don’t need any clarification as to their usage. It is a good idea to use the –verbose and/or -log parameters – they will output useful information and details about the changes that are applied. There is yet another parameter that you may consider using: –testrun – when applied you see just the output with the changes that would have been applied in normal mode without actually calling the update methods for the fields and content types (you won’t see the correct change notifications for some content types if you have modifications in both base and derived content types though for obvious reasons).

You may download the solution from here (it is the same one as in my previous posting – you can check the installation notes in it too).

Common actions feature receiver

We all know about the standard SharePoint feature elements – Module, Field, ContentType, ContentTypeBinding, ListInstance, etc. Still in many cases there are tasks we want to achieve for which we need to use feature receivers and use custom code to do one thing or another. Many of these tasks are quite common and recurring so it is obviously wise to have some XML configuration that can be reused in the various features that we have and handled by a common feature receiver. The most natural thing would be to extend somehow the standard elements though it’s not possible to have both standard and custom elements in one and the same manifest file. Actually we can’t place non-standard elements in an element manifest file but we can have another configuration file (or files) that the feature or the feature receiver knows about and can process its elements when the feature gets activated. And this is what I started developing (I say starting because I intend to extend it with other features in the future) – a custom XML configuration format and a custom feature receiver that handles it.

So, first a word about what “common actions” I have put in this feature receiver – the first one is quite versatile – it sets various settings to SharePoint lists – it allows to set most of the properties of the SPList class, to add site columns and content types to the list, to order and hide the content types, to create views in the list, etc. The second one is rather simple – it sets up the lookup site columns that you have – something that can’t be achieved with the standard Field element.

One step back now – about the “common action” that sets the various SPList settings – I guess that you have already asked yourself why have such an element since we have the standard ListTemplate element and can create custom list definition features with all schema configurations for a custom list. There are several reasons for that (at least these are the things that made be stop using custom list templates): the first one is that the list template is applicable only for lists that are about to be created – you can’t use them to change anything on existing lists, second – they are rather uneconomical – if you wish to extend a list based on a standard list template with a couple of fields and one or two list views you need to create a separate list definition. Imagine what happens when you need 20 or so different lists (I was quite disappointed when I found out that the standard ListInstance element allows adding data to the list but not changing its schema).

About the available for downloading solution – besides the receiver’s code it contains also an XSD file used for validation of the “common actions” XML – it is actually an extension of the standard WSS schema definitions (which are somewhat incomplete actually) – the list view and field schema definitions are actually reused from the standard definitions – you will see shortly. One other big advantage of having an XML schema definition is that you can use the Visual Studio’s editor with all goodies like IntelliSense, inline validation, etc – there is a VS configuration XML file in the project that if put in the appropriate VS installation folder enables that (see below in the installation notes). There is also a console tool project in the solution – this tool can be used to generate a feature containing the list settings for one, several or all lists from an existing site – something like a reverse engineering utility which can speed up greatly the development process. The feature that is generated contains a standard elements manifest file with ListInstance elements for the specified lists – so basically this feature when applied to a new site will create the lists from the source “snapshot” and will add the various customizations that were applied to them.

And here are the XMLs of a sample feature that uses the common action provider:

<?xml version="1.0" encoding="utf-8"?>

<Feature Title="my feature" Description="my feature" Id="699b4ad2-c77e-440c-a488-b569d71e3467" Scope="Web" Hidden="TRUE" DefaultResourceFile="core"

         ReceiverAssembly="Stefan.Sharepoint.Util, Version=1.0.0.0, Culture=neutral, PublicKeyToken=338fd4db0d2cb397"

         ReceiverClass="Stefan.Sharepoint.Util.FeatureReceivers.CommonActionsReceivers" xmlns="http://schemas.microsoft.com/sharepoint/">

  <ElementManifests>

    <ElementManifest Location="elements.xml" />

  </ElementManifests>

  <Properties>

    <Property Key="CommonActionsPath" Value="commonactions.xml" />

  </Properties>

</Feature>

this is the sample feature.xml file – the ReveiverAssembly and ReceiverClass attributes of the Feature element are set accordingly and there is a Property element with Key attribute set to “CommonActionsPath”, whose value contains the relative path within the feature folder of the “common actions” configuration XML.

And this is a sample “common actions” configuration XML:

<?xml version="1.0" encoding="utf-8"?>

<CommonActions xmlns="http://schemas.microsoft.com/sharepoint/" >

 

  <ListSettings ListUrl="Lists/Action">

    <ListProperties Title="Actions" EnableVersioning="TRUE" EnableModeration="TRUE" />

    <Fields>

      <FieldRef ID="282d1240-b8d5-4862-b9ff-940e170cc7b2" Name="ActionID" />

      <FieldRef ID="6810bdee-f664-4a9f-b58c-39329fb8036e" Name="ActionExpireDate" />

      <FieldRef ID="3964fd24-5908-40cd-9822-9271b2e5dad7" Name="ActionImage" />

      <FieldRef ID="a16c3c60-5782-4b70-a965-722ccf8cb32d" Name="ActionFlashFile" />

    </Fields>

    <View DisplayName="All Items" Type="HTML" DefaultView="TRUE">

      <Query>

        <OrderBy>

          <FieldRef Name="ActionExpireDate" Ascending="FALSE" />

        </OrderBy>

      </Query>

      <ViewFields>

        <FieldRef Name="Edit" />

        <FieldRef Name="LinkTitle" />

        <FieldRef Name="ActionID" />

        <FieldRef Name="ActionExpireDate" />

      </ViewFields>

      <RowLimit Paged="TRUE">100</RowLimit>

    </View>

  </ListSettings>

</CommonActions>

You see a CommonActions root element containing one ListSettings element – you can have many of these. Most of the attributes and elements a self explanatory and the FieldRef and View elements are actually identical to the standard WSS field and view definitions (I mentioned that the standard SharePoint XSDs are reused here).

So let me briefly explain the various XML elements and attributes that can be used within the ListSettings elements. Basically all sub-elements are optional, so you can use just a subset of these to apply the appropriate settings. You can use the ListUrl attribute in the ListSettings element to specify the SharePoint list that you want to modify. There is another alternative to specify the list to be customized – with it you can actually specify many lists – this is very handy if you want to apply the same settings to multiple lists, here is the syntax – you use a sub-element of the ListSettings element:

    <ListSelector All="FALSE">

      <ListUrls>

        <add ListUrl="Lists/mylist"/>

      </ListUrls>

      <ListTemplates>

        <add ListTemplate="100"/>

      </ListTemplates>

    </ListSelector>

You can specify either list URLs or list templates, besides add elements remove elements can also be specified to finely tune the set of lists that you want to work with.

The ListProperties sub-element can contain the following attributes:

<ListProperties Title="Actions" Description="" EnableVersioning="FALSE" EnableModeration="FALSE" EnableMinorVersions="FALSE" Direction="none" MajorVersionLimit="0" MajorWithMinorVersionsLimit="0" AnonymousPermMask64="0" AnonymousPermMask="0" AllowDeletion="TRUE" EnableDeployWithDependentList="TRUE" EnableDeployingList="TRUE" AllowMultiResponses="FALSE" EnableAttachments="TRUE" EnableFolderCreation="FALSE" ForceCheckout="FALSE" DraftVersionVisibility="Reader" DefaultItemOpen="PreferClient" Hidden="FALSE" ContentTypesEnabled="TRUE" MultipleDataList="FALSE" Ordered="FALSE" RequestAccessEnabled="TRUE" ShowUser="TRUE" AllowEveryoneViewItems="FALSE" ReadSecurity="1" WriteSecurity="1" OnQuickLaunch="TRUE" EnableAssignToEmail="FALSE" EnableSyndication="TRUE" IrmEnabled="FALSE" IrmExpire="FALSE" IrmReject="FALSE" EnableSchemaCaching="FALSE" NoCrawl="FALSE" SendToLocationName="" SendToLocationUrl="" />

The attributes bear the same names as the corresponding SPList class properties. Note here that the different types of lists doesn’t support the setting of some of the properties to values different from the default ones – and if you try to do that the properties’ setters will throw an exception.

The Fields sub-element can contain both Field and FieldRef elements:

    <Fields>

      <Field ID="{282D1240-B8D5-4862-B9FF-940E170CC7B2}" Name="ActionID" SourceID="http://schemas.microsoft.com/sharepoint/v3" StaticName="ActionID" Group="my group" Type="Text" DisplayName="Action id" Required="FALSE" MaxLength="255" />

      <FieldRef ID="6810bdee-f664-4a9f-b58c-39329fb8036e" Name="ActionExpireDate" />

    </Fields>

The Field element contains a standard WSS field definition and can be used to create a field in the list with the specified inline field schema. The FieldRef element can be used to add existing site columns to the list – only the ID attribute is used to locate the field in the site columns, the Name attribute is optional and its use is just to give a visual hint about the field’s name. Note here – if the list already exists and the field is already created in it the feature receiver takes no action and skips the field.

There are two sub-elements for applying content type settings to the list:

    <ContentTypeRefs>

      <ContentTypeRef Name="my content type"/>

    </ContentTypeRefs>

    <ContentTypeOrder>

      <ContentTypeRef Name="my content type"/>

      <ContentTypeRef Name="Item"/>

    </ContentTypeOrder>

The ContentTypeRefs element can be used to add existing content types to the list – basically the same as you can achieve with the standard ContentTypeBinding element – the difference here is that you specify the content type’s Name not ID. I added this doublet of the standard functionality in case it is used with the multiple list select option of the ListSettings element – if you want to add one or more content types to several lists simultaneously. Note here – if the content type is already added to the list – the feature receiver just skips it. The second sub-element – ContentTypeOrder – allows you to set the order of the content types as they appear in the “New” menu item of the SharePoint list.

The View sub-element (you can have more than one) contains a standard WSS View definition – you will quickly recognize the standard Query, ViewFields and RowLimit elements within it. Mandatory attributes in the View element are the DisplayName and Type ones. The view definitions are matched to the existing views in the list using the view’s display name. If a view with that name doesn’t exist it is created, otherwise the view definition is applied to the existing view.

After the brief description of the ListSettings “common action” I will describe the lookup field “common action” with one or two lines only – first – here’s a sample usage:

<LookupFieldSettings ID="efd8d109-4f6f-4828-ac32-d9f2e11f8669" Name="MyField" LookupListUrl="Lists/MyList" ShowField="Title" />

The field is located in the site columns using the field ID, the Name attribute is just a visual hint. The LookupListUrl attribute specifies the site relative URL of the lookup list and the ShowField attribute – the lookup source field in the lookup list. If the lookup field is already set up (its LookupList property is set) the receiver will skip it.

Using the command line tool to generate features from existing sites

This is a sample usage of the tool that generates a feature with standard elements manifest file containing ListInstance elements for several lists specified in the command line arguments and a common actions configuration file with settings for these lists:

UtilTest.exe -o exportlistfeature -weburl http://racoon-vpc-hg:2909 -listurl Lists/mylist,Lists/mylist2 -outputpath "c:\temp\myfeature" -exportfields -exportcontenttypes  -exportcontenttypeorder -dontusesitecolumns -featuretitle "my feature"

The syntax is pretty similar to the one of the standard stsadm utility – the parameters starting with a dash are argument names and the ones following each argument name are argument values (they are optional). These arguments are mandatory - -o, –weburl, –listurl, –outputpath and –featuretitle – their names are explanatory for their usage. The –listurl argument can contain one list URL only or several separated with commas, it can also contain the * (asterisk) character for all lists (without the hidden ones) in the site. The other arguments are optional: -exportfields –exportcontenttypes and –exportcontenttypeorder specify whether the corresponding sub-elements in the ListSettings element should be exported and the –dontusesitecolumns argument specifies whether the custom list columns  should be exported with inline schemas or with FieldRef elements if identical site columns exist.

There is one other command of the export utility – the one that exports a feature for setting up the lookup fields for the specified site (actually for the site columns of the site collection whose lookup lists are in the specified site):

UtilTest.exe -o exportlookupfeature -weburl http://racoon-vpc-hg:2909 -outputpath "c:\temp\lookupfeature" -featuretitle "lookup feature"

Installation notes

• the Stefan.Sharepoint.Util should be placed in the GAC (this is only if you use the receiver class from it otherwise you can copy the code to some assembly of yours)
• the contents of the 12 folder of the solution should be copied to your SharePoint’s 12 folder. It actually contains one XSD file only - stefan.common.xsd which should be copied to the SharePoint’s 12\TEMPLATE\XML folder.
• the file stefan.xml from the root of the solution should be copied to the Xml\Schemas subfolder of your Visual Studio’s installation folder (for VS 2008 it’s normally C:\Program Files\Microsoft Visual Studio 9.0\Xml\Schemas). After you copy the file there and restart your Visual Studio you will have the IntelliSense support for the “common actions” XML format.
• the command line utility UtilTest.exe uses the Stefan.Sharepoint.Util assembly so you need to have the two files together if you want to use it (unless the Stefan.Sharepoint.Util assembly is in the GAC of the machine).

You can download the common actions receiver code from here.