Visual sandbox web parts … possible after all

So, there’s been somewhat of a commotion about this thing recently so I decided to give it a try. Let me mention several things first – I’m still wondering whether the sandbox solutions will be that popular and widely used to start with and whether the ability to create user controls for web parts is that important and effort-saving (well, it’s been a practice for me to use user controls with web parts but that’s not so for most of my colleagues), but despite this temperate pessimism of mine I decided to create a small POC that demonstrates how this can be implemented.

So let me start with several words about why the TemplateControl.LoadControl method is not working in a sandbox solution. The reason is quite simple – your code in a sandbox solution actually executes in a console application – the SPUCWorkerProcess.exe. So basically what you have is that a console application has created an HttpContext and a Page instance, in which your web part class is also instantiated – it is actually the only control (in some cases you can have a couple more) on that page. And in this mock-like asp.net environment you don’t have some crucial asp.net pieces as the HostingEnvironment and the VirtualPathProvider – without these two the LoadControl method simply won’t work (I may explain the sandbox and SharePoint user code intrinsic workings in another posting). And it is not that you don’t have access to the file system, even if you had, the LoadControl method still wouldn’t work because there wouldn’t be a VirtualPathProvider to provide the file to the asp.net compiler. The bad news is that the same holds for the TemplateControl.ParseControl method too.

And to the idea behind the implementation: I first saw a codeplex project - SharePoint Developer Tools for Visual Studio 2010 – it actually had a custom VS 2010 project template for a visual sandbox web part … but it didn’t work – it didn’t compile due to one error in this line:

Controls.Add(new ASP.SandboxedVisualWebPart1_ascx());

I immediately understood what the idea behind this was: a class name like this one - ASP.somecontrol_ascx - gives away a class generated by the asp.net runtime or by the aspnet_compiler.exe tool from an ascx file. So, aspx and ascx files get eventually compiled to real code Page and Control classes in temporary assemblies that the asp.net runtime creates if not existing and loads when you navigate the aspx page or use the ascx control. These assemblies can be created also using the aspnet_compiler.exe utility that comes with the .NET framework, which is used primarily to precompile your web sites when you deploy them. But you can also use this tool to compile any ascx (and aspx) file that you have – in our case the tool can be used to compile the user controls of the sandbox web parts. So, the tool will spit out one or several assemblies that contain the compiled classes of the ascx files but there are two issues with circular references:

• you cannot add a reference to this assembly in your web part assembly because the former itself has a reference to your assembly
• the second circular reference is one of class usage – you first need to have your assembly compiled before the ascx assemblies can be created (since they need the code behind classes from your assembly), but you wouldn’t be able to compile the web part assembly if it uses the ascx compiled classes (which could be created only after your assembly is compiled) – what a paradox.

And now, here’re the steps that I used to solve the above mentioned issues – note that all these are executed as a post build step of the web part project (the actual implementation is with a PowerShell script file):

1. first step is to copy all ascx files that you have in the project to a temporary folder and use the aspnet_compiler.exe tool to create assembly (assemblies) containing the compiled ascx classes
2. second step (this solves the first circular reference issue) is to use the ILMerge tool (it’s a free Microsoft tool that can be downloaded from here – note that it doesn’t come with the .NET installation) to merge the compiled ascx assembly (assemblies) with the web part assembly – this is what the ILMerge tool does – merging assemblies (great thing). And now you will have the ascx compiled classes right into your web part assembly.
3. third step comes to the second circular reference issue – as I mentioned these steps are executed as a post build step which means that you have the web part assembly already compiled – but how did that happen if you need to reference the compiled ascx class. The answer is simple – you don’t reference it at all or at least not as a class – here is how this is possible: with a standard visual web part you use the user control via a member variable whose type is actually the code behind class of the user control and the only “reference” to the user control is the LoadControl call in the CreateChildControls method of the web part where the control is created and its instance gets cast to the code behind class member variable. And we have the same here, the only difference is the control instantiation in the CreateChildControls method and this is the key line of code for that:

_userCtrl = (UserControl1)Activator.CreateInstance(Type.GetType("ASP.usercontrol1_ascx"));

As you see – a little bit of reflection (will this pass the security checks in the sandbox - yes) and the ascx compiled class instance is created providing only its class name as a string (which is easily deducible from the name of the ascx file).

You can download the sample project which demonstrates this technique (or rather work-around) from here. And I want to make a big NOTE here that this is only a POC, the implementation is a bit sloppy and probably won’t work in all cases and/or in more complex scenarios. Also in the PowerShell script that implements the post-build step there’re several hard-coded values for file locations and names which may break the whole thing. So, having said that let me outline shortly the steps that you can use to create a simple sandbox solution that uses user controls in web parts:

• Open Visual Studio 2010 as administrator
• Create an Empty SharePoint Project (selecting the “Deploy as sandbox solution” option) – e.g. SandboxWebPart
• Add a new Web Part item to the project (e.g. WebPart1)
• Add a new User Control item to the project (e.g. UserControl1)
• Drag the newly create ascx item (UserControl1.ascx) from below the ControlTemplates/SandboxWebPart folder and drop it below the web part’s node – WebPart1
• Delete the ControlTemplates mapped folder (you won’t be able to build the package with it)
• This is optional – open the UserControl1.ascx.cs and UserControl1.ascx.designer.cs and change the namespace to be the namespace that you have in the web part’s code file. If you change the namespace in these two files you will have to change the Inherits attribute in the Control directive in the UserControl1.ascx file so that it reflects the new full name of the code-behind class.
• This is important – in the UserControl1.ascx in the Control directive at the top change the AutoEventWireup to false – otherwise you will receive security errors at runtime (sorry, you will need to hook your events explicitly).
• From the properties pane of the UserControl1.ascx – set the Deployment Type property to NoDeployment (remember you cannot deploy ascx files in a sandbox solution)
• Add some code to your web part so that it can use the user control – first create a member variable with the user control’s code-behind class type:

protected UserControl1 _userCtrl;

• Add this code to the web part’s CreateChildControls override method:

_userCtrl = (UserControl1)Activator.CreateInstance(Type.GetType("ASP.usercontrol1_ascx"));
this.Controls.Add(_userCtrl);

• Add the postbuild.ps1 file from the sample project into the project root folder
• Open the project properties page and in the Build Events section put this in the “Post-build event command line” text box:

PowerShell -command "set-executionpolicy -ExecutionPolicy bypass"
PowerShell -command "$(ProjectDir)postbuild.ps1" '$(SolutionDir)' '$(ProjectDir)'  '$(TargetDir)' '$(TargetFileName)' '$(ConfigurationName)'

• Big NOTE – here – remember that you’re running Visual Studio as administrator and you’re first changing the execution policy setting of PowerShell and then you will run a PowerShell script which may be a huge security threat – make sure that you check carefully the code in the postbuild.ps1, so that you know what it does and how it does it. Another note – instead of using the script as a post-build step you can run it standalone, after you make a normal build of the project – in this case you will have to provide all command line arguments that it expects.
• Make sure that you have installed the ILMerge tool (see above) to this location - C:\Program Files (x86)\Microsoft\ILMerge (it is hard-coded in the ps script)
• Build the solution – in the output window you should see something like this:

------ Rebuild All started: Project: SandboxWebPart, Configuration: Debug Any CPU ------
  SandboxWebPart -> c:\Projects\SandboxWebPart\SandboxWebPart\bin\Debug\SandboxWebPart.dll
  asxc compilation started...
  Compiling ascx files:
  C:\Projects\SandboxWebPart\SandboxWebPart\ascxtmp\UserControl1.ascx
  Utility to precompile an ASP.NET application
  Copyright (C) Microsoft Corporation. All rights reserved.
  Merging assemblies:
  C:\aa618f13-0fa9-4dc8-b6ee-714f24d847f2\bin\App_Web_wagenclm.dll
  C:\aa618f13-0fa9-4dc8-b6ee-714f24d847f2\bin\SandboxWebPart.dll
========== Rebuild All: 1 succeeded, 0 failed, 0 skipped ==========

• You can now run deploy and check your web part with a user control in it (hope this works for you).

SharePoint 2010 – content type feature upgrading

This is something that I had overlooked in my lengthy posting on SharePoint content types from a couple of months ago. It is about the new UpgradeActions element that you can have in the feature.xml file now and more precisely about the AddContentTypeField element inside it:

<UpgradeActions>

  <VersionRange BeginVersion ="1.0.0.0" EndVersion="2.0.0.0" />

  <AddContentTypeField ContentTypeId="0x01006DB0361318574E1CAF601F96E79717BB" FieldId="6DB03613-1857-4E1C-AF60-1F96E79717B4" PushDown="TRUE"/>

</UpgradeActions>

So, with this small XML snippet that you can add to your content type feature you will be able to upgrade your sample (already existing) site content type by adding an extra field to its schema and also (more importantly) the change in the schema will be pushed to the inheriting list content types. So far so good – so we already have at least some support out-of-the box for upgrading site content types (if we don’t count the Overwrite attribute that I covered extensively in the “lengthy” posting). But … on second thought it doesn’t look that promising and actually leaves quite some issues open – several that I can think of are that you can only add fields to the content type but cannot remove fields; also the fields will be added after all existing fields in the content type, i.e. you can’t control the field order and not to mention that you can’t change anything in the XmlDocuments section of the content type schema definition. Another thing that bothers me is that these upgrades will look quite “patchy” – you will have your initial content type schema definition in a standard SharePoint item elements file which will be extended by some separate definitions in another file – in this case the feature.xml one. But enough criticism, who knows - we can have something better till the time of the RTM version of SharePoint 2010.

SharePoint 2010 – how to add web parts to rich html fields with code

It’s actually pretty simple – check out the small code snippet below that adds an xslt list view web part and a content editor web part to the PublishingPageContent field of a publishing page:

[Update 2010-12-24: the XLV web part provisioned with the code below will have an issue when you use folders in the target list – check the first two comments after the posting for a quick fix]

        private void TestAddRichContentWebParts(SPFile file)

        {

            // NOTE: we assume that the SPFile was checked out

            // get the limited wp manager for that SPFile

            SPLimitedWebPartManager mngr = file.GetLimitedWebPartManager(PersonalizationScope.Shared);

 

            // create a sample XLV

            XsltListViewWebPart xlv = new XsltListViewWebPart();

            // get a target list

            SPList list = file.ParentFolder.ParentWeb.Lists["Pages"];

            // set the ListName property with the capitalized list ID w/o the curly braces

            xlv.ListName = list.ID.ToString("B").ToUpper();

            // use the schema of the default view

            xlv.ViewGuid = list.DefaultView.ID.ToString("B").ToUpper();

            // add the web part to the special wpz zone

            mngr.AddWebPart(xlv, "wpz", 0);

            // store the web part's internal ID

            Guid xlvID = mngr.GetStorageKey(xlv);

 

            // create a sample content editor wp

            ContentEditorWebPart cwp = new ContentEditorWebPart();

            cwp.Title = "My editor";

            // add the web part to the special wpz zone

            mngr.AddWebPart(cwp, "wpz", 0);

            // store the web part's internal ID

            Guid cwpID = mngr.GetStorageKey(cwp);

 

            // get the list item for that SPFile

            SPListItem item = file.Item;

            // update the PublishingPageContent field value

            item["PublishingPageContent"] = "some content<br/>" + this.GetEmbeddedWPString(xlvID) + "<br/>some other content" + this.GetEmbeddedWPString(cwpID);

            // update the item

            item.SystemUpdate(false);

            // the SPFile instance can be checked in, published, approved afterwards

        }

 

        private string GetEmbeddedWPString(Guid wpID)

        {

            const string form = @"<div class=""ms-rtestate-read ms-rte-wpbox"">

  <div class=""ms-rtestate-notify ms-rtegenerate-notify ms-rtestate-read {0}"" id=""div_{0}""></div>

  <div id=""vid_{0}"" style=""display:none""></div>

</div>";

            // set the web part's ID as part of the ID-s of tho div elements

            return string.Format(form, wpID);

        }

The adding of the web parts involves two steps:

• adding the web parts to the page with the limited web part manager – this is what you do with web parts in web part zones as well
• the second and extra step is to set the HTML content of the rich HTML field that will contain the web parts – you need to add HTML “anchors” in it for the web parts – these will be the places in the content where the web parts will get rendered. The “anchors” contain several HTML div elements with ID attributes containing the internal guid IDs of the web parts.

And several notes on the sample code:

• the sample TestAddRichContentWebParts method is provided with a SPFile instance for a publishing or wiki page file. You can get the page file with SPWeb.GetFile and should check it out if it is a publishing page (the default settings of a Pages library require check out) before calling the TestAddRichContentWebParts method.
• notice that in the call of GetLimitedWebPartManager.AddWebPart you should pass “wpz” for the zoneId parameter. This special zone id name (an obvious acronym of “web part zone”) is required otherwise the web parts will end up in one of the real web part zones on the page.
• in the sample the “PublishingPageContent” field of the page is updated – you can update any HTML field (and I suppose any HTML containing/rendering field) on a publishing or wiki page in the same manner. The GetEmbeddedWPString method generates the HTML “anchor” for the web part that you should embed in the HTML contents of the rich HTML field – this will be the place in the content where the web part will appear.

ListViewWebPart & SPView – two sides of the same coin

So, the idea behind this metaphor is that these two SharePoint object model classes are actually two representations of one and the same internal piece of data (or entity). Basically this means that when you create one of the objects you automatically get the other one, when you delete one of them, the other one also gets deleted and when you change the properties of one of them you see visual or other changes in the other. So, list views in SharePoint always come with a pair of representing objects – a ListViewWebPart and SPView – and since behind them is a single entity you can’t have one of the objects in the pair replaced with another SPView or LVWP. In SharePoint 2010 with the introduction of the LVWP inheritor – the XSLT list view web part (XLV) you have the same pairing but with the XLV instead of the LVWP on the web part side.

So, to clarify the above mentioned let me present several examples from the SPView and LVWP perspectives –first, what do we have from the SPView class perspective:

• When you create a new SPView (from code or the UI alike) you get a web part page (in the list root folder or in the Forms subfolder of the list) containing a LVWP on it that displays the target list’s data. The deleting of the SPView deletes the web part list view page together with the LVWP on it. So far, it sounds rather mundane, doesn’t it?
• But the more interesting part is when you have the list view web part page and delete the LVWP part on it using the SharePoint UI (you may do that with code too) – the result is that the SPView object that has created the containing web part view page is also deleted – that’s probably something you haven’t expected to happen (be careful to not delete the default view this way). This way you will end up with an empty view (actually ex-view) web part page and no SPView – you can inspect the Views collection property of the containing SPList to make sure that the corresponding SPView is really gone.

And what do we have from the LVWP perspective:

• When you add a LVWP to a publishing or simple web part page with the UI or code you will get a new SPView in the parent list’s Views collection. The new SPView will have its Hidden property set to true and its DisplayName will be empty. The Guid ID property of the SPView will be the same as the ViewGuid property of the LVWP but also as the ID property itself of the LVWP (save that the ID of the LVWP has the “g_” prefix).
• When you change the Query property or the ViewFields of this hidden SPView you will see that the changes will become immediately visible in the LVWP. Note that if you have the LVWP on a publishing page that requires check-out you will need to have the page checked out first otherwise you will receive an error when you call the SPView.Update method (I will show a code snippet below how this can be achieved).
• When you delete the LVWP its hidden SPView also gets deleted. It’s the same the other way around – if you delete the hidden SPView, the LVWP will get deleted (you don’t need to have the page checked out in this case – remember that pretty much the same happens if you delete the SPList itself – then all LVWP based on it get deleted)
• Another note here – the standard UI for editing the properties of the LVWP is a little bit misleading – you see a drop-down from which you can select one of the existing visible SPViews for the LVWP – you may think that this way you associate the SPView with the LVWP but this is not the case – what happens actually is that the schema of the visible SPView gets copied to the LVWP’s own hidden SPView. You can see that this is the case quite easily – if you modify the visible view that you have “associated” with the LVWP you’ll see that the changes won’t be applied to the LVWP.
• It’s pretty much the same when you add a LVWP to a page with code: in this case you need to create a new instance of the ListViewWebPart class and set its ListName property and either the ViewGuid or ListViewXml properties. When you set the former, you need to provide the guid ID of an existing SPView – you will notice however that after the LVWP is already added to the page the ViewGuid property now has the ID of its internal SPView not the original one. If you use the ListViewXml you should provide the view schema yourself (to cut a long story short you can use for example the SchemaXml property of the SPView returned by SPList.GetUncustomizedViewByBaseViewId) – then you will have full control over the various parts of the view schema – query, view fields, etc.

So let me now show you several examples of how you can update the LVWP’s hidden SPView with the SharePoint UI and with code and to briefly outline the cases when you may need to do that (just to quickly announce here that there is some good news for SharePoint 2010 about that).

Starting with SharePoint 2007 – from a user interface perspective you can modify the LVWP’s SPView’s schema by getting the web part in edit mode and clicking the “Edit the current view” link in the part’s properties pane – this will open the standard “modify view” application page. Another option here is to select another existing view from the “Selected View” drop-down.

A case in which you don’t have control over certain aspects of the part’s hidden SPView’s schema is when you provision LVWPs with Module features. Basically all you can specify in this case is a “BaseViewID” (which will define the presentation part) but nothing for the view’s query, view fields, etc. So this issue can be resolved only with code – there’re two options here – to either add the LVWPs to the pages with code (manipulating the ListViewXml property) or have the part’s hidden SPView modified after the web parts is already created (with the UI or provisioned with a Module feature). The latter is a little bit tricky, here is a code snippet that can achieve that:

      public void ProcessCommonAction(SPWeb rootWeb, SPWeb web, XElement element)

        {

            string pageUrl = element.Attribute("PageUrl").Value;

            string listUrl = element.Attribute("ListUrl").Value;

 

            SPFile file = web.GetFile(pageUrl);

            if (!file.Exists) return;

 

            Guid listID = file.ParentFolder.ParentListId;

            SPList library = null;

 

            bool inLibrary = !listID.Equals(Guid.Empty);

            if (inLibrary)

            {

                library = web.Lists[listID];

                if (library.ForceCheckout && file.Level != SPFileLevel.Checkout) file.CheckOut();

            }

 

            try

            {

                // it is important that the SPView object is retrieved after the checkout of the file

                SPList listViewList = web.GetList(web.ServerRelativeUrl.TrimEnd('/') + "/" + listUrl);

                SPView view = listViewList.Views.Cast<SPView>().FirstOrDefault(v => v.Url.EndsWith(pageUrl, StringComparison.OrdinalIgnoreCase));

                if (view != null)

                {

                    ViewHelper.AddOrUpdateView(listViewList, view, element.Element(XHelper.WSSNs + "View"));

                }

            }

            finally

            {

                if (inLibrary)

                {

                    if (library.ForceCheckout) file.CheckIn("");

                    if (library.EnableMinorVersions) file.Publish("");

                    if (library.EnableModeration) file.Approve("");

                }

            }

        }

Notice that the page containing the LVWP may need to be checked out before the SPView instance can get updated. Also note that the SPView object should be retrieved only after the page is checked out.

The SharePoint 2010 story

Several pieces of good news here starting with the new XLV web part which comes with numerous improvements over the SharePoint 2007’s LVWP (there is an on-going series on that on the MS SharePoint team blog). One of the main improvements here relevant to this posting is that the presentation settings were moved from the SPView class to the web part itself – the Xsl and XslLink properties of the XLV instead of the cumbersome CAML holding presentation properties in SPView. This means that you can get the query schema copied from existing views with the UI but can provide your custom rendering in the web part itself (remember the problem with free form views in the announcement list in SharePoint 2007).

Another “goodie” in SharePoint 2010 is that you can directly use the standard view editing page (_layouts/ViewEdit.aspx) to modify the XLV’s hidden SPView – you have the button in the ribbon (I like the ribbon already). So you don’t need to do the cumbersome stuff as in SharePoint 2007 – getting the properties pane of the web part opened, etc., but you can just select the XLV (the blue border around the part should appear) so that the “ListTools” tab appears in the ribbon, and then from the “List” sub-tab select “Modify View” – see the image (on a publishing page you will need to have the page checked out otherwise you’ll receive an error when you try to update the view in the view editing page):

Notice that the the standard view editing page opens but with the text field for the display name and the check-box for default view hidden (this is a hidden SPView after all), also the delete button is missing, but if you are insistent you can delete the SPView with code (which will of course delete the XLV too):