Web.config modifications and copying files to the web application’s physical path

Yes, I know that we have the standard SharePoint SPWebConfigModification class and also the stsadm CopyAppBinContent command. But … the problem is that unfortunately there’re cases when they just can’t do the job the right way causing quite some frustration. When I thought about these problems I came to the conclusion that spending a couple of hours and with some two or three hundred lines of code I could achieve the same thing – and this is the solution that I came up with (maybe one in a row of many already). It’s pretty simple and does both things in one piece of functionality (a feature with a custom feature receiver that is) – it can simultaneously apply changes to the web.config files on all servers in a farm and also can copy files to the physical location of the selected SharePoint web application (not just resource files).

So, let’s get right to it:

<?xml version="1.0" encoding="utf-8"?>

<Feature Id="EA0DD85A-F215-4ffb-893C-7EFFAA09FDFF"

     Version="1.0.0.0"

     Hidden="FALSE"

     Scope="WebApplication"

     Title="Web application config and path settings feature"

     Description=""

     ReceiverAssembly="Stefan.SharePoint, Version=1.0.0.0, Culture=neutral, PublicKeyToken=6ced17f280b33956"

     ReceiverClass="Stefan.SharePoint.FeatureReceivers.WebAppPathReceiver"

     xmlns=http://schemas.microsoft.com/sharepoint/>

  <ElementManifests></ElementManifests>

   <Properties>

    <Property Key="ConfigChangePath" Value="webconfig.xml"/>

    <Property Key="WebAppSourcePath" Value="80"/>

    <Property Key="UrlZone" Value="Default"/>

  </Properties>

</Feature>

This is a sample feature.xml file of the feature that modifies the web.config and copies (actually xcopies) files to the web application’s physical folder. You can see the custom feature receiver class specified in the feature definition – the receiver checks the properties of the feature definition and starts a custom timer job passing the values of these properties to it. It is the custom timer job that runs on all servers in the SharePoint farm that implements the aforementioned functionality.

And here is a short description of the feature’s custom properties:

• ConfigChangePath – specifies the path (relative to the feature definition path) of the xml file with the changes that should be applied to the web.config of the target web application (I’ll describe its schema in a little while) – the property is optional.
• WebAppSourcePath - specifies the path (relative to the feature definition path) of a folder whose contents (files and sub-folders) will be copied recursively to the physical folder of the target web application – the property is optional.
• UrlZone – specifies the URL zone of the target web application – you may need this property only if you have an extended web application. The property is optional and if you don’t specify a value the Default URL zone will be used. You can specify more than one URL zones in the property like Default|Custom|Internet (using the ‘|’ character as separator). Then the web.config modifications and the web application files will be copied to all IIS web sites’ physical folders of the extended web application.

Note that the Scope attribute of the feature is set to WebApplication. This is actually not a requirement and the scope can be set to Site or Web as well – the feature receiver can handle all scenarios.

The image above shows the folder structure and the files of the sample feature. You can see that the 80 sub-folder of the feature folder contains an App_GlobalResources sub-folder with one resource file in it.

And a sample web modification XML file looks like:

<?xml version="1.0" encoding="utf-8" ?>

<ConfigChanges>

  <Namespaces>

    <NamespaceAlias alias="a" url="http://someaddress.com" />

  </Namespaces>

  <ConfigItem ParentElementXPath="/configuration/appSettings" ElementXPath="add[@key='somekey']" CreateOption="Add" >

    <![CDATA[<add key="somekey" value="somevalue1" />]]>

  </ConfigItem>

  <ConfigItem ParentElementXPath="/configuration/appSettings" ElementXPath="add[@key='somekey2']" CreateOption="AddOrReplace" >

    <![CDATA[<add key="somekey2" value="somevalue2" />]]>

  </ConfigItem>

</ConfigChanges>

The web.config modification XML comes with a custom XSD – you can use it in Visual Studio for intelli-sense and validation support and the XSD file should also be deployed to the 12\TEMPLATE\XML folder of the SharePoint installation since the timer job uses it to validate the input XML file.

The elements of the custom web.config modification XML are mostly self-explanatory. You will notice that the modifications that can be applied to the web.config work only on XML elements (not XML attributes) and the parent element of the element(s) to be inserted should exist in the web.config file.

The ParentElementXPath attribute of the ConfigItem element contains the XPath that will be used to locate the parent element of the element that should be added, updated or deleted – as I already mentioned this element should exist in the web.config. The ElementXPath attribute contains the XPath that will be used to locate/identify the element to be created/modified – note here that the XPath for it is relative to the parent element. The third attribute of the ConfigItem element is the CreateOption one – it has three possible values:

• Add – this will create a new element, if the element already exists it won’t get overwritten
• AddOrReplace – this will create a new element or overwrite the element if it exists
• Remove – this will delete the element if it already exists

As you see the functionality provided is pretty basic and simple but as I said before the code is really small and straight-forward and can be quite easily modified or extended.

The custom solution can be downloaded from here.

Content type features revisited

In one of my previous postings I discussed a custom solution that updates feature-created content types pushing down the changes to the destination lists. Since we already have the public beta of SharePoint 2010 I was curios how this problem may be handled in the new SharePoint. What I noticed first was the new Overwrite attribute that is introduced in the content type element definition and I expected some really good news about it, but after some further investigation the conclusions are a bit mixed. So, before going back to the new stuff about content types in SharePoint 2010 (which is not that much actually) let me make a step back and make a deep dive into the content type definitions intricacies so that we can see all possible issues about them, the problem with the propagation (pushing down) of changes to the destination lists and the changes introduced in SharePoint 2010. As I mentioned already the changes in 2010 are not that big and the main focus of this article will not be about them. So, let me start with some terminology – it is not officially sanctioned but it will help in my further explanations if we have strict definitions:

Some terminology

First thing – the type of content types depending on their scope:

1. site content type – this is merely a schema definition with no direct relation to the data – you see these in the content type gallery of the site
2. list content type – this is a content type in a list – its schema has direct representation in the list fields

Second – the type of site content types depending on the method of creating it:

1. feature content type – this a site content type created with a feature that has a ContentType element
2. object model (OM) content type – this is a site content type created using the SharePoint object model – namely the SPContentType and SPContentTypeCollection classes

Third – two groups of feature content types depending on the source of their schema:

1. ghosted content type – its schema is read directly from the elements XML file in the feature that created the content type
2. unghosted content type – the schema is read from the content database – a ghosted content type can get unghosted if something in its schema gets changed using the SharePoint OM.

phantom field links – these are elements in the FieldLinks collections of a list content type that do not have corresponding list fields (i.e. they are not provisioned) – I will explain later how you can end up with such.

Content type inheritance

And several words about the content type inheritance – this is something essential in respect to pushing down site content type updates to list content types. I won’t get into details here about how content type inheritance can be defined in the feature elements file using the ID attribute of the ContentType element – you can check the corresponding SharePoint SDK documentation on that.

First off – inheritance may be not even the correct word for this type of relation, at least if you compare it to class inheritance in programming languages. So, for content types – depending on the scope of the content type there’re some restrictions: only site content types can be inherited (this means that you can’t have a list content type inheriting another list content type). And list content types although bearing the same names as the site content types used to create them are actually inheritors of these site content types – so basically all rules of “inheritance” apply for them too. The inheritance relation for content types is actually rather loose. The most obvious case to see the content type “inheritance” in action is when you create a new child content type (using a feature or the object model alike) – the result is basically that the whole schema definition of the parent content type is copied to the child content type (for the feature creation case – the schema specified for the child type is then merged with the schema inherited from the parent). When it comes to updates it becomes a little bit stranger – you have two modes here – propagating updates – meaning that the changes in the parent content type will be applied to its descendants and non-propagating changes – which won’t be applied to the descendants. What’s more – every element in the schema of the child content type is modifiable including the inherited parts which means that they are nothing more than a separate and independent from the parent’s definition copy in the child content type schema. You can imagine that using non-propagating updates and modifying both the parent and child content types these can get totally different. So much for inheritance here.

And about the propagating updates: these are actually more like the publisher-subscriber relation in a database replication model rather than class-like inheritance. The main rule here is that only immediate changes are propagated to the descendant content types – what do I mean here with immediate changes – these are changes that you make with the object model to a SPContentType object after you get it from the parent web and before you call its Update method with the pushing down flag set. This means that no previous changes made to the parent content type with non-propagating updates will ever get propagated afterwards. A small example here: let’s consider two content types – A with field links a, b and c which is a parent of content type B with the same field links. If we remove field a from content type A and add field d to it and update it without the propagation option the fields of content type B will remain again a, b and c. If we then remove field b and add field e to A and update with pushing the changes, the fields of B will be a, c and e (in A we’ll have – c, d and e). So you see that the propagation doesn’t work en masse – you can’t force the exact copying of the parent content type to the child one – and this is not only when you’ve made modifications to the child content type, but also when you’ve made modifications to the parent with non-propagating updates. So in the case of the above example if you want to force the removing of field a and the adding of d in B you should use this rather awkward work-around – add again a and remove d from A with non-propagating update and then remove a and add again d with a propagating update. Actually with this kind of “trick” you can remove any field in the descendant content types even such that were never present in A (meaning such that the descendants never “inherited” from A when they were created).

Another example of the “looseness” of the content type “inheritance” is that there’re cases in which you can get the parent content type deleted and this won’t affect the child content type the least – when you check the Parent property of the child type you will see the closest existing ancestor of the type after the deleted former parent (if none more derived exists you will see the standard Item content type).

So what’s the verdict – the pushing down updates of site content types (they are applicable only for site content types since list content types cannot be inherited) though a rather good feature can’t ensure in all cases correct (in the sense of getting the exact fields or subset of fields in the descendants) propagation of the changes in the parent content type to its descendants. Even if you use the brute force approach of first deleting all fields in the parent content type and then add them again there will be cases when the child content type will have different fields in its schema. And actually this holds for scenarios when you have the creating content type feature unmodified and using another feature or tool to modify the content types and propagate the changes. When it comes for updating (reapplying) the creating feature it may get even worse – I will now explain why:

Activating the content type feature

This is actually about getting the content type creating feature activated repeatedly with the force flag set. In SharePoint 2010 there is the new Overwrite attribute for the ContentType feature element which somewhat mitigates some of these issues and I will describe it later. Here I will explain the common issues of reactivating a content type feature with modified content type definitions on both SharePoint 2007 and SharePoint 2010. So, let me quickly mention a big warning about this in MSDN – the recommendation is (was?) not to update the original content type definition files and use new ones if necessary (I guess this was the reason to introduce the Overwrite attribute in the first place).

Ok, so what happens to the ghosted site content types when you modify the creating feature and activate it again using the force parameter – well, they simply get updated (the same actually happens if you just modify the elements file and check the definitions after recycling the application pool – the difference with the feature reactivation is that with the latter the new content type definitions get also provisioned on the site). As you may expect it – the unghosted site content types (i.e. feature content types that were modified consequently with the object model) do not get modified in this case. And what happens to the list content types inheriting the feature site content types – at first glance – nothing – they seem exactly the same as before the update. But when you check the SPContentType object and specifically its FieldLinks collection you see some strange things – all new fields that were added to the content type definitions also appear here – and this is no matter whether the parent site content type is ghosted or unghosted. I called these field links in the terminology section – phantom field links. The problem with them is that they exist in the list content type definition but there are no matching list fields – on the surface nothing seems wrong – at least in the SharePoint UI. But when you try to add this field to the content type (and consequently to the list) – then nothing happens and it can be quite frustrating if you don’t know the reason for that – the system just detects that the field link is already in the content type definition (though you don’t see it in the UI since it’s not provisioned) and refuses to add it again. To resolve the problem you will need to add the field to the list directly or perform some complex equilibristic with adding/removing fields in the parent site content type and propagate the changes to the descendant.

The case with the phantom field links is quite worrying – you have your list content types (no matter whether created from ghosted or unghosted site content types or whether they themselves were further modified) and a simple change to the original creating feature manifest file adds changes to their schema – as to why the SharePoint designers introduced this “merging” schema logic is beyond my knowledge. Now the MSDN recommendation against modifying the feature element manifest seems quite justified (the Overwrite attribute quite nicely fixes the phantom field links issue and the problem with updating at least site content types – see below).

Deactivating the content type feature

I think there is some interesting information about the deactivation of the content type features: first you have two options here – to deactivate with and without the force parameter – yes, there is a difference here. When you don’t specify the force attribute then only the content types without inheritors (including inheriting list content types) get deleted. And when you specify the force parameter – all feature site content types get deleted. And what about the inheriting list content types – they remain intact, just their Parent property returns a higher ancestor or ultimately the standard Item content type. So, as I mentioned in the “inheritance” paragraph, the inheritance concept for content types is pretty loose. Actually when you activate the feature again and the site content types are created once again the parent relation is “fixed” to point to the original parent. And what happens with the phantom field links – well, it seems that they are quite persistent and won’t disappear unless you uninstall the creating feature from the farm – quite nasty stuff. As you see deactivating with the force parameter and activating the content type again you can get your site content types updated (actually overwritten since you will lose all you modifications to the unghosted site content types) – so at least this is some alternative to the Overwrite attribute for SharePoint 2007.

The Overwrite ContentType element attribute in SharePoint 2010

With all the aforementioned problems with feature site content types you probably may want to have a facility to create a content type using the object model so that it never relies on element manifest files in the file system. Well, this is what the Overwrite attribute actually does. The content type created with the Overwrite attribute can still be regarded as a feature content type since it will get removed when the feature is deactivated but in all other respects behaves as an OM content type (this is also the case with content types created with a sandboxed solution). Just to mention here that the SPContentType class in SharePoint 2010 now has a public constructor in which you can specify an exact SPContentTypeId – in SharePoint 2007 the content type id of new content types created with the object model is auto-generated (this in case you are interested in creating your own content types programmatically).

So, the new attribute allows you to update feature site content types without deactivating the creating feature and no matter whether the target content types were unghosted or not. Actually when you create a content type with the Overwrite attribute the content type will never be ghosted in the first place but this may be the case if initially the content type is created without the Overwrite attribute and you add it at some later moment when you want to update the feature. And another piece of good news here – the issue with the phantom field links in the inheriting list content types is also fixed – they are gone not only for content types initially created with the Overwrite attribute but for ones updated consequently with that attribute.

The conclusion about the new Overwrite attribute in SharePoint 2010 is that it does a great job for what it was intended – updating (overwriting) the site content types. It would have been a good idea if there were an option to have these updates propagated to the inheriting list content types but I think Microsoft had a good reason not to include that.

The problem with the update propagation actually remains and the updating of the site content types alone makes it even a little bit harder. This is because you end up in a situation with the the site content type and the inheriting list content type being different (with the site content type independently modified with the updated feature) so the inheritance propagation with its immediate changes mechanism will be hardly useful to get the job done. One possible but ugly solution that I already briefly mentioned is with deleting all field links of the site content type and then creating them again (possibly in a feature receiver). Since this will fail to handle removed fields in the modified content type definition (they will remain in the inheriting types) a design rule may be applied to all possible updates – to not delete fields from the content type definition but to mark them with the Hidden attribute.

And several words about my content type updates propagation solution – it will still be applicable for SharePoint 2010 but will still have all complexities of a custom solution for a group of issues for which we still won’t have a complete out-of-the-box solution.

SharePoint Search and HTML Meta tags

Using the standard HTML meta tags in your pages (including SharePoint ones) – e.g. Title, Keywords, Description, Robots, etc is quite common. Apart from these you can use custom meta tags (specifying custom values in the name attribute) to define (or extend) your custom meta data for the page. These can be crawled by search engines including the SharePoint search engine and can be used for various filtering and sorting purposes when using the search functionality. Normally in SharePoint the list item (publishing pages are list items) meta data is stored in the list items’ fields but there are cases in which storing meta data in HTML meta tags may have some advantages.

First off – HTML meta tags can be created relatively easy – you can place a custom control in your master page (pages) which outputs the meta tags in the HTML of your pages, an even better solution here is to use a delegate control. So, basically with a single control you ensure that all your pages contain your set of custom meta tags. A particularly good scenario for the meta tags created in this way is when you need them for meta properties whose values can be easily generated or calculated based on other meta data (like list fields) or certain criteria. For example these can be the URLs of the pages or the type of the parent web, etc. In the case of such calculable meta properties you can imagine the overhead of creating auxiliary list fields and maintaining them in all your page libraries compared to the neat approach of the custom web control that generates all meta data in a single piece of code.

A recent example of using meta tags with SharePoint search that I had was for a solution where I needed to display the rollup image of every page in the search results. The problem with the publishing rollup image (actually with the publishing image field type) is that it stores its value as HTML markup (a single HTML img element with several attributes) and the SharePoint crawler simply ignores all HTML markup. The result is that you have your image field containing the source of the image file but the search service just can’t get it for you. A possible solution that I have seen for this problem is to create an additional field that will be populated with just the path of the image file using a list item event receiver for updating it when the rollup image field gets changed. The thing is that implementing and maintaining this is not so easy and not that economically looking.

Here is a small code snippet to demonstrate a simple way to generate the meta tag HTML for the standard rollup image source attribute:

        protected override void OnLoad(EventArgs e)

        {

            base.OnLoad(e);

 

            string image = GetItemRollupImage();

            this.litMeta.Text = string.Format("<meta name=\"myrollupimage\" content=\"{0}\" />", Server.HtmlEncode(image));

        }

 

        protected string GetItemRollupImage()

        {

            if (SPContext.Current.ListItem != null)

            {

                ImageFieldValue imgValue = SPContext.Current.ListItem[FieldId.RollupImage] as ImageFieldValue;

                if (imgValue != null) return imgValue.ImageUrl;

            }

            return string.Empty;

        }

The crawl properties that the SharePoint search crawler creates for the custom HTML meta tags are in the Web crawled properties category – they are with the same name as the name of the custom meta tags – just uppercase. These can be created with code too – the ID of the propset for the meta tag crawled properties is d1b5d3f0-c0b3-11cf-9a92-00a0c908dbf1 (this is actually the ID of the meta tag Web crawled properties subcategory). The last step before you can use the meta tag crawled properties is to create managed properties mapped to them.

Iterating SharePoint sites using a search query

This is a pretty simple query that can be used to iterate the sites and sub-sites under some URL and get the sites’ title, description and URL:

SELECT url, contentclass, title, description FROM scope() WHERE (contentclass='STS_Web' OR contentclass='STS_Site') and site='http://myserver/mysite'

The contentclass managed field is used to filter the “site” search items and the site managed field – to specify a starting URL the sites below which we want to retrieve. Normally the STS_Web and STS_Site items cause issues since they “hide” the welcome pages of the corresponding sites but in this case they come quite handy allowing the retrieval of some metadata (title and description) of the sites.

Update content types and site columns with feature receiver using the standard manifest file

The usual way of creating content types and site columns is by using an element manifest file with the standard Field and ContentType elements in a feature. The creation is quite easy but when it comes to updating you see some really hard times – first Microsoft strongly discourages the modifying of the original manifest files and second even if you do that the changes you made won’t get propagated to the target lists. Quite unpleasant, isn’t it. So, to address this problem I created a feature receiver that can read a manifest file and apply all changes to the existing artifacts propagating the changes to the site’s lists (actually this receiver is an enhanced version of my “common actions” receiver from my previous posting).

First – one big note – the updating receiver is still a beta quality code and should be used carefully – there is a command line utility in the receiver’s solution that allows starting the update routine directly and also supports a “test run” mode – so you may consider using this first and check the verbose notifications that it outputs as to which changes it is about to apply (more on this later).

Another important note – do not modify the element manifest file of the original feature that created the site columns and content types – use a separate file, a copy of the original one with the desired modifications. This is because before the artifact is updated with the object model its schema is read from the manifest file in the file system (you can check that easily – the schema is saved to the content database after the artifact gets its update method called). This also means that you can modify this way “ghosted” site columns and content types (MSDN recommends strongly against that) but not such that were already created or modified with the object model. The solution described here handles both cases.

And several words on what it actually updates and how exactly it applies the updates – 1st starting with the latter – only standard object model calls are used – the Update methods of the SPField and SPContentType classes are used with the option of pushing down the changes to the corresponding list fields and inheriting content types respectively. So the idea is not just to update the site columns and content types but to propagate these changes to the lists where these are actually used. The SPField objects are updated with setting the field schema from the manifest file to the SchemaXml property of the field – one note here – all attributes missing in the manifest schema that are present in the existing field schema are copied from the latter – so this is actually a merge of the two field definitions rather than a total overwriting from the schema in the manifest file (I wanted to keep safe here for cases like lookup fields which have attributes that are normally set after the field is created). So, in cases when the existing site columns are modified from the UI or from code after the site is created the updating with the receiver may result in a merge of the field definition rather than a definition that is identical to the one in a newly created site. Note – only existing site columns are being modified, if a column from the manifest file is not found in the site, it is skipped.

And about content types – two elements are updated here – the FieldRefs and the XmlDocuments. For FieldRefs – you can add, remove, change the attributes of the FieldRef-s of the content type as well as reorder them. All attributes for which public settable properties in the SPFieldLink class exist are set (exceptions being – ShowInNewForm and ShowInEditForm – no properties for them). XmlDocuments are also added, updated and deleted according to the changes in the content type schema. Something interesting (and maybe peculiar) here – the XmlDocuments are also inherited from the base content types – unless a XmlDocument with the same namespace exists in the derived content type which overrides the parent’s definition. This leads to some unexpected results especially with the list item event receiver XmlDocument – because of this inheritance mechanism you may end up with one and the same event receiver’s method being called more than once on a list item modification event (but that’s another story). Another “intrinsic” behavior with the event receiver XmlDocument – when you add receivers with it they are added to the lists to which the content type is bound, but if you remove the XmlDocument from the schema definition – these remain in the destination lists, though if you modify the receivers XmlDocument – the old receivers are removed and the new ones are added (basically this is what we have with the object model). I also tested modifications of the FormTemplates and FormUrls XmlDocument elements and they seem to work just fine. Note about content type updates – only existing site content types are being modified, if a content type from the manifest file is not found in the site, it is skipped.

Here is a feature.xml from a sample feature that uses the receiver:

<?xml version="1.0" encoding="utf-8"?>

<Feature

    Title="ZCommonTest"

    Description="ZCommonTest"

    Id="7A0F010D-7993-4bcd-9127-7B15FEFFC0FF"

    Scope="Web"

    Hidden="TRUE"

    DefaultResourceFile="core"   

    ReceiverAssembly="Stefan.Sharepoint.Util, Version=1.0.0.0, Culture=neutral, PublicKeyToken=338fd4db0d2cb397"

    ReceiverClass="Stefan.Sharepoint.Util.FeatureReceivers.CommonActionsReceivers"

    xmlns="http://schemas.microsoft.com/sharepoint/">

    <ElementManifests>

    </ElementManifests>

  <Properties>

    <Property Key="CTsFieldsUpdatePath" Value="fields.xml,ctypes.xml" />

  </Properties>

</Feature>

So basically, you need to set the appropriate ReceiverAssembly and ReceiverClass attributes and the custom Property element with Key=”CTsFieldsUpdatePath” – the value of the property element can contain one or many (comma separated) manifest files’ paths – the feature folder relative path actually.

In the receiver’s solution there is also a console utility project – the tool has a command for direct invocation of the update functionality, here is a sample call:

UtilTest.exe -o updatecontenttypesandfields -weburl http://racoon-vpc-hg:2909/sites/4 -manifestpath "C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\FEATURES\ZCommonTest\ctypes.xml" -verbose -log z.log

Most of the arguments don’t need any clarification as to their usage. It is a good idea to use the –verbose and/or -log parameters – they will output useful information and details about the changes that are applied. There is yet another parameter that you may consider using: –testrun – when applied you see just the output with the changes that would have been applied in normal mode without actually calling the update methods for the fields and content types (you won’t see the correct change notifications for some content types if you have modifications in both base and derived content types though for obvious reasons).

You may download the solution from here (it is the same one as in my previous posting – you can check the installation notes in it too).

Common actions feature receiver

We all know about the standard SharePoint feature elements – Module, Field, ContentType, ContentTypeBinding, ListInstance, etc. Still in many cases there are tasks we want to achieve for which we need to use feature receivers and use custom code to do one thing or another. Many of these tasks are quite common and recurring so it is obviously wise to have some XML configuration that can be reused in the various features that we have and handled by a common feature receiver. The most natural thing would be to extend somehow the standard elements though it’s not possible to have both standard and custom elements in one and the same manifest file. Actually we can’t place non-standard elements in an element manifest file but we can have another configuration file (or files) that the feature or the feature receiver knows about and can process its elements when the feature gets activated. And this is what I started developing (I say starting because I intend to extend it with other features in the future) – a custom XML configuration format and a custom feature receiver that handles it.

So, first a word about what “common actions” I have put in this feature receiver – the first one is quite versatile – it sets various settings to SharePoint lists – it allows to set most of the properties of the SPList class, to add site columns and content types to the list, to order and hide the content types, to create views in the list, etc. The second one is rather simple – it sets up the lookup site columns that you have – something that can’t be achieved with the standard Field element.

One step back now – about the “common action” that sets the various SPList settings – I guess that you have already asked yourself why have such an element since we have the standard ListTemplate element and can create custom list definition features with all schema configurations for a custom list. There are several reasons for that (at least these are the things that made be stop using custom list templates): the first one is that the list template is applicable only for lists that are about to be created – you can’t use them to change anything on existing lists, second – they are rather uneconomical – if you wish to extend a list based on a standard list template with a couple of fields and one or two list views you need to create a separate list definition. Imagine what happens when you need 20 or so different lists (I was quite disappointed when I found out that the standard ListInstance element allows adding data to the list but not changing its schema).

About the available for downloading solution – besides the receiver’s code it contains also an XSD file used for validation of the “common actions” XML – it is actually an extension of the standard WSS schema definitions (which are somewhat incomplete actually) – the list view and field schema definitions are actually reused from the standard definitions – you will see shortly. One other big advantage of having an XML schema definition is that you can use the Visual Studio’s editor with all goodies like IntelliSense, inline validation, etc – there is a VS configuration XML file in the project that if put in the appropriate VS installation folder enables that (see below in the installation notes). There is also a console tool project in the solution – this tool can be used to generate a feature containing the list settings for one, several or all lists from an existing site – something like a reverse engineering utility which can speed up greatly the development process. The feature that is generated contains a standard elements manifest file with ListInstance elements for the specified lists – so basically this feature when applied to a new site will create the lists from the source “snapshot” and will add the various customizations that were applied to them.

And here are the XMLs of a sample feature that uses the common action provider:

<?xml version="1.0" encoding="utf-8"?>

<Feature Title="my feature" Description="my feature" Id="699b4ad2-c77e-440c-a488-b569d71e3467" Scope="Web" Hidden="TRUE" DefaultResourceFile="core"

         ReceiverAssembly="Stefan.Sharepoint.Util, Version=1.0.0.0, Culture=neutral, PublicKeyToken=338fd4db0d2cb397"

         ReceiverClass="Stefan.Sharepoint.Util.FeatureReceivers.CommonActionsReceivers" xmlns="http://schemas.microsoft.com/sharepoint/">

  <ElementManifests>

    <ElementManifest Location="elements.xml" />

  </ElementManifests>

  <Properties>

    <Property Key="CommonActionsPath" Value="commonactions.xml" />

  </Properties>

</Feature>

this is the sample feature.xml file – the ReveiverAssembly and ReceiverClass attributes of the Feature element are set accordingly and there is a Property element with Key attribute set to “CommonActionsPath”, whose value contains the relative path within the feature folder of the “common actions” configuration XML.

And this is a sample “common actions” configuration XML:

<?xml version="1.0" encoding="utf-8"?>

<CommonActions xmlns="http://schemas.microsoft.com/sharepoint/" >

 

  <ListSettings ListUrl="Lists/Action">

    <ListProperties Title="Actions" EnableVersioning="TRUE" EnableModeration="TRUE" />

    <Fields>

      <FieldRef ID="282d1240-b8d5-4862-b9ff-940e170cc7b2" Name="ActionID" />

      <FieldRef ID="6810bdee-f664-4a9f-b58c-39329fb8036e" Name="ActionExpireDate" />

      <FieldRef ID="3964fd24-5908-40cd-9822-9271b2e5dad7" Name="ActionImage" />

      <FieldRef ID="a16c3c60-5782-4b70-a965-722ccf8cb32d" Name="ActionFlashFile" />

    </Fields>

    <View DisplayName="All Items" Type="HTML" DefaultView="TRUE">

      <Query>

        <OrderBy>

          <FieldRef Name="ActionExpireDate" Ascending="FALSE" />

        </OrderBy>

      </Query>

      <ViewFields>

        <FieldRef Name="Edit" />

        <FieldRef Name="LinkTitle" />

        <FieldRef Name="ActionID" />

        <FieldRef Name="ActionExpireDate" />

      </ViewFields>

      <RowLimit Paged="TRUE">100</RowLimit>

    </View>

  </ListSettings>

</CommonActions>

You see a CommonActions root element containing one ListSettings element – you can have many of these. Most of the attributes and elements a self explanatory and the FieldRef and View elements are actually identical to the standard WSS field and view definitions (I mentioned that the standard SharePoint XSDs are reused here).

So let me briefly explain the various XML elements and attributes that can be used within the ListSettings elements. Basically all sub-elements are optional, so you can use just a subset of these to apply the appropriate settings. You can use the ListUrl attribute in the ListSettings element to specify the SharePoint list that you want to modify. There is another alternative to specify the list to be customized – with it you can actually specify many lists – this is very handy if you want to apply the same settings to multiple lists, here is the syntax – you use a sub-element of the ListSettings element:

    <ListSelector All="FALSE">

      <ListUrls>

        <add ListUrl="Lists/mylist"/>

      </ListUrls>

      <ListTemplates>

        <add ListTemplate="100"/>

      </ListTemplates>

    </ListSelector>

You can specify either list URLs or list templates, besides add elements remove elements can also be specified to finely tune the set of lists that you want to work with.

The ListProperties sub-element can contain the following attributes:

<ListProperties Title="Actions" Description="" EnableVersioning="FALSE" EnableModeration="FALSE" EnableMinorVersions="FALSE" Direction="none" MajorVersionLimit="0" MajorWithMinorVersionsLimit="0" AnonymousPermMask64="0" AnonymousPermMask="0" AllowDeletion="TRUE" EnableDeployWithDependentList="TRUE" EnableDeployingList="TRUE" AllowMultiResponses="FALSE" EnableAttachments="TRUE" EnableFolderCreation="FALSE" ForceCheckout="FALSE" DraftVersionVisibility="Reader" DefaultItemOpen="PreferClient" Hidden="FALSE" ContentTypesEnabled="TRUE" MultipleDataList="FALSE" Ordered="FALSE" RequestAccessEnabled="TRUE" ShowUser="TRUE" AllowEveryoneViewItems="FALSE" ReadSecurity="1" WriteSecurity="1" OnQuickLaunch="TRUE" EnableAssignToEmail="FALSE" EnableSyndication="TRUE" IrmEnabled="FALSE" IrmExpire="FALSE" IrmReject="FALSE" EnableSchemaCaching="FALSE" NoCrawl="FALSE" SendToLocationName="" SendToLocationUrl="" />

The attributes bear the same names as the corresponding SPList class properties. Note here that the different types of lists doesn’t support the setting of some of the properties to values different from the default ones – and if you try to do that the properties’ setters will throw an exception.

The Fields sub-element can contain both Field and FieldRef elements:

    <Fields>

      <Field ID="{282D1240-B8D5-4862-B9FF-940E170CC7B2}" Name="ActionID" SourceID="http://schemas.microsoft.com/sharepoint/v3" StaticName="ActionID" Group="my group" Type="Text" DisplayName="Action id" Required="FALSE" MaxLength="255" />

      <FieldRef ID="6810bdee-f664-4a9f-b58c-39329fb8036e" Name="ActionExpireDate" />

    </Fields>

The Field element contains a standard WSS field definition and can be used to create a field in the list with the specified inline field schema. The FieldRef element can be used to add existing site columns to the list – only the ID attribute is used to locate the field in the site columns, the Name attribute is optional and its use is just to give a visual hint about the field’s name. Note here – if the list already exists and the field is already created in it the feature receiver takes no action and skips the field.

There are two sub-elements for applying content type settings to the list:

    <ContentTypeRefs>

      <ContentTypeRef Name="my content type"/>

    </ContentTypeRefs>

    <ContentTypeOrder>

      <ContentTypeRef Name="my content type"/>

      <ContentTypeRef Name="Item"/>

    </ContentTypeOrder>

The ContentTypeRefs element can be used to add existing content types to the list – basically the same as you can achieve with the standard ContentTypeBinding element – the difference here is that you specify the content type’s Name not ID. I added this doublet of the standard functionality in case it is used with the multiple list select option of the ListSettings element – if you want to add one or more content types to several lists simultaneously. Note here – if the content type is already added to the list – the feature receiver just skips it. The second sub-element – ContentTypeOrder – allows you to set the order of the content types as they appear in the “New” menu item of the SharePoint list.

The View sub-element (you can have more than one) contains a standard WSS View definition – you will quickly recognize the standard Query, ViewFields and RowLimit elements within it. Mandatory attributes in the View element are the DisplayName and Type ones. The view definitions are matched to the existing views in the list using the view’s display name. If a view with that name doesn’t exist it is created, otherwise the view definition is applied to the existing view.

After the brief description of the ListSettings “common action” I will describe the lookup field “common action” with one or two lines only – first – here’s a sample usage:

<LookupFieldSettings ID="efd8d109-4f6f-4828-ac32-d9f2e11f8669" Name="MyField" LookupListUrl="Lists/MyList" ShowField="Title" />

The field is located in the site columns using the field ID, the Name attribute is just a visual hint. The LookupListUrl attribute specifies the site relative URL of the lookup list and the ShowField attribute – the lookup source field in the lookup list. If the lookup field is already set up (its LookupList property is set) the receiver will skip it.

Using the command line tool to generate features from existing sites

This is a sample usage of the tool that generates a feature with standard elements manifest file containing ListInstance elements for several lists specified in the command line arguments and a common actions configuration file with settings for these lists:

UtilTest.exe -o exportlistfeature -weburl http://racoon-vpc-hg:2909 -listurl Lists/mylist,Lists/mylist2 -outputpath "c:\temp\myfeature" -exportfields -exportcontenttypes  -exportcontenttypeorder -dontusesitecolumns -featuretitle "my feature"

The syntax is pretty similar to the one of the standard stsadm utility – the parameters starting with a dash are argument names and the ones following each argument name are argument values (they are optional). These arguments are mandatory - -o, –weburl, –listurl, –outputpath and –featuretitle – their names are explanatory for their usage. The –listurl argument can contain one list URL only or several separated with commas, it can also contain the * (asterisk) character for all lists (without the hidden ones) in the site. The other arguments are optional: -exportfields –exportcontenttypes and –exportcontenttypeorder specify whether the corresponding sub-elements in the ListSettings element should be exported and the –dontusesitecolumns argument specifies whether the custom list columns  should be exported with inline schemas or with FieldRef elements if identical site columns exist.

There is one other command of the export utility – the one that exports a feature for setting up the lookup fields for the specified site (actually for the site columns of the site collection whose lookup lists are in the specified site):

UtilTest.exe -o exportlookupfeature -weburl http://racoon-vpc-hg:2909 -outputpath "c:\temp\lookupfeature" -featuretitle "lookup feature"

Installation notes

• the Stefan.Sharepoint.Util should be placed in the GAC (this is only if you use the receiver class from it otherwise you can copy the code to some assembly of yours)
• the contents of the 12 folder of the solution should be copied to your SharePoint’s 12 folder. It actually contains one XSD file only - stefan.common.xsd which should be copied to the SharePoint’s 12\TEMPLATE\XML folder.
• the file stefan.xml from the root of the solution should be copied to the Xml\Schemas subfolder of your Visual Studio’s installation folder (for VS 2008 it’s normally C:\Program Files\Microsoft Visual Studio 9.0\Xml\Schemas). After you copy the file there and restart your Visual Studio you will have the IntelliSense support for the “common actions” XML format.
• the command line utility UtilTest.exe uses the Stefan.Sharepoint.Util assembly so you need to have the two files together if you want to use it (unless the Stefan.Sharepoint.Util assembly is in the GAC of the machine).

You can download the common actions receiver code from here.

ListView control powered by ProcessBatchData

In my last posting I demonstrated how it is possible to render list view HTML using SPWeb.ProcessBatchData with the DisplayPost method – I strongly recommend that you check it first before continuing with this one. While I tested the method I created a small web control that renders list view HTML much like the standard ListView web part or the SPView.RenderAsHtml method, … with several differences though. Basically this is experimental stuff, more like a proof of concept which should be thoroughly tested before used for more serious purposes.

The control is really simple and the code is merely two hundred or so lines – many important production features like appropriate exception handling, parameters and variables checks and logging are just missing. The public interface offers just four public properties which you can use to set up the control – and these should be set before the OnPreRender event:

public Guid ListID { get; set; }

public Guid ViewID { get; set; }

public SPWeb Web { get; set; }

public string ViewHtmlSchema { get; set; }

So, you need to set the ListID and the ViewID properties with the ID-s of the SPList and SPView that you want to render – note that the control uses just the ID-s, so if you have them cached you can pass them directly without opening the corresponding SPList and SPView instances – the control doesn’t use the SharePoint object model to open the view and list objects either – it just puts the ID-s in the ProcessBatchData batch string. For the Web property you should provide a SPWeb instance for the SharePoint site containing the source list – the ProcessBatchData method will be called on it. If you don’t set this property the current web will be used, but if it is not the web containing the list to be rendered, the rendering will fail. The ViewHtmlSchema property is optional in that if you skip it the ViewID property will be used to select the view of the list to be rendered. If you choose to use it however it will override the ViewID property and you will need to pass a valid View CAML definition to it – this is what you basically see in a View element in a schema.xml file of a list template or in the SPView.HtmlSchemaXml property. Several common scenarios of customized View schema that I can think of are for example a standard View schema with dynamically modified Query element for achieving custom filtering or sorting based on certain conditions or a standard View schema with slight modifications of the ViewBody, ViewHeader, ViewFooter and ViewEmpty elements to achieve different look and feel in the rendering. On the other hand you can construct a CAML View definition for some really customized rendering that doesn’t look anything like the standard ListView table like HTML presentation.

And now, let’s have a look at some portions of the code:

        private const string _displayPostXml = @"<?xml version=""1.0"" encoding=""UTF-8""?>

<ows:Batch OnError=""Continue"">

<Method ID=""0"">

  <SetVar Name=""Cmd"">DisplayPost</SetVar>

    {0}

  <SetVar Name=""PostBody"">{1}</SetVar>

</Method>

</ows:Batch>";

 

        private const string _postBody = @"<ows:XML>

    <SetList Scope=""Request"">{0}</SetList>

    {1}

</ows:XML>";

these two constant strings are used as templates for constructing the batch XML for the DisplayPost method. And this is the actual method that does this job:

        protected string GetViewHtml()

        {

            if (this.CurrentWeb == null) return string.Empty;

            bool allowUpdates = this.CurrentWeb.AllowUnsafeUpdates;

            try

            {

                // this should be set - otherwise the call to ProcessBatchData will throw

                this.CurrentWeb.AllowUnsafeUpdates = true;

 

                // format the saved in the ViewState query parameters as SetVar-s

                string parms = this.FormatParams();

 

                // if the ViewHtmlSchema property is not set format an empty View element with Name attribute

                string viewSchema = !string.IsNullOrEmpty(this.ViewHtmlSchema) ? this.ViewHtmlSchema : string.Format(@"<View Name=""{0}"" />", this.ViewID.ToString("B").ToUpper());

 

                // format the post body XML

                string postBody = string.Format(_postBody, this.ListID, viewSchema);

                // format the DisplayPost method XML

                string methodXml = string.Format(_displayPostXml, parms, EscapeForXml(postBody));

                // call the ProcessBatchData method

                string result = this.CurrentWeb.ProcessBatchData(methodXml);

 

                // obviously not the fastest way to get the result

                XmlDocument doc = new XmlDocument();

                doc.LoadXml(result);

                XmlElement el = doc.DocumentElement.SelectSingleNode("./Result") as XmlElement;

                return el == null ? string.Empty : el.InnerText;

            }

            finally

            {

                this.CurrentWeb.AllowUnsafeUpdates = allowUpdates;

            }

        }

As you see this method uses the two XML templates and the values of the control’s public properties from above to construct the DisplayPost batch XML. You can see the difference in the two modes of operation of the Control – when using just the ViewID property without specifying a ViewHtmlSchema – this results in a batch string with just an empty View element in it, and when you specify a View CAML definition in the ViewHtmlSchema property – then the full schema is inserted into the PostBody parameter of the DisplayPost method. An important note here: if you use a custom view definition and want to use the standard ViewHeader that displays context menus in the header cells with sorting and filtering options the root View element of the definition should have a Name attribute containing the ID of an existing view of the source list – otherwise the rendering of the context menus will fail.

The GetViewHtml method of the control is called normally from the control’s Render method, though there is one other usage of it that I will explain a little later. Let's first see another important piece of code in the control – the setting of the URL query parameters as SetVar parameters in the DisplayPost method XML body. You know that when you filer and sort a standard ListView web part clicking its header cells or the associated context menus certain parameters appear in the current page URL’s query part – parameters like View, RootFolder, SortField, SortDir, FilterField1, FilterValue1, FilterField2, FilterValue2. So the thing is that if we want the rendered view to be fully interactive these parameters should be somehow added to the XML of the DisplayPost method. And well, this turns out to be easy – using CAML SetVar parameters with the same names and values just does the trick. This is the method that saves the URL query parameters to the control’s ViewState:

        protected void GetSetVarParamsFromQuery()

        {

            // don't save the query params on event callback

            if (this.Page != null && this.Page.IsCallback) return;

 

            string viewParam = HttpContext.Current.Request.QueryString["View"];

 

            if (!string.IsNullOrEmpty(viewParam))

            {

                try

                {

                    Guid viewIDParam = new Guid(viewParam);

                    // check the value of the View query param - if it's not our viewID - don't save the query params - they should be used for another ListView

                    if (!this.ViewID.Equals(viewIDParam)) return;

                }

                // catch the Guid constructor exception

                catch { return; }

            }

            // else - if a view query param is not present - just proceed with saving the query params to the ViewState

 

            if (this.SetVarParams == null) this.SetVarParams = new NameValueCollection();

            else this.SetVarParams.Clear();

 

            // save all query params to the ViewState

            foreach (string par in HttpContext.Current.Request.QueryString.AllKeys)

            {

                this.SetVarParams[par] = HttpContext.Current.Request.QueryString[par];

            }

        }

you may ask why I save the URL query parameters to the ViewState and not use them directly in the DisplayPost XML – the answer is simple – these are not always guaranteed to be present in the page’s URL – especially in cases when you have two or more ListView web parts or ListView controls on the page – when you filter or sort one of the ListView-s it puts the parameters for its own filtering and sorting in the query string and in the meantime the other controls should be able to preserve their state. The ListView for which the query parameters should be applied is determined by the View query parameter containing the source view’s ID for that control (this is why the standard ListView web part creates always a hidden view which is a copy of the view that you select to be displayed in it – this guarantees that view that it renders is always unique and not used by another ListView). And the above method does exactly the same – it checks the View query parameter and saves the other query parameters only if it matches its ViewID property.

The method that formats the saved URL query parameters as SetVar CAML parameters is this one:

        protected string FormatParams()

        {

            NameValueCollection parms;

            if (this.SetVarParams == null) parms = new NameValueCollection();

            else parms = new NameValueCollection(this.SetVarParams);

 

            // add the group SetVar-s - the groupArgument will be set when the GetViewHtml method is called from the client callback handler

            if (!string.IsNullOrEmpty(this.groupArgument))

            {

                parms["GroupString"] = this.groupArgument;

                parms["ClientCallback"] = "1";

            }

 

            // this SetVar should be set for the group client java script to work

            if (!string.IsNullOrEmpty(this.ID)) parms["WebPartID"] = this.ID;

            StringBuilder sb = new StringBuilder();

            // output the SetVar-s

            foreach (string par in parms.AllKeys)

            {

                sb.AppendFormat("<SetVar Name=\"{0}\">{1}</SetVar>\r\n", EscapeForXml(par), EscapeForXml(parms[par]));

            }

            return sb.ToString();

        }

The interesting thing to note here is that several extra SetVar parameters can be optionally added here – these are used when the ListView uses grouping and dynamically loads the items for an expanded group. The control handles this by implementing the ICallbackEventHandler interface for ajax-like loading of data without page reloads – this is how the standard ListView web part does this as well. And as I mentioned before this is the second usage of the GetViewHtml control’s method – this time for rendering just the part that renders the items below the expanded group. To implement this the control renders some auxiliary java script snippets that the java script from the standard ViewHeader view element calls so that it can interact with the rendering control. Note also that the control’s ID property is used here so if you want to use the grouping functionality you should set it explicitly.

You can download the full code from here.