WinForm tool with DataGridView for SharePoint list item updating

This is a small tool that can preview, update, delete and insert list items in SharePoint lists. If you've had problems with the standard datasheet list view, you can check the video below - this tool overcomes some of the limitations of the datasheet view and allows for easy manipulation of SharePoint list items:




The tool uses a DataGridView control and fills it with the data of a selected SharePoint list.
The data is retrieved as DataTable. Three ways to get a DataTable with SharePoint list data are used - these can be changed using the drop-down control in the toolbar of the tool:

• SPWeb.GetSiteData(SPSiteDataQuery) - this method returns a DataTable, all columns of which are with type System.String.
  
• SPList.GetItems(SPQuery) - to get a SPListItemCollection and then SPListItemCollection.GetDataTable() - the DataTable returned in this case has typed columns for some of the list fields with certain primitive value types - e.g. System.Int32, System.DataTime and System.Double. One drawback here - the lookup-s are unpleasantly trimmed - they contain only the textual part, not the lookup item ID.
  
• SPList.GetItems(SPQuery) - to get a SPListItemCollection and then custom method generating a DataTable from the SPListItem-s in the collection - similar to the standard SPListItemCollection.GetDataTable() but with certain modifications - the column for the yes/no field type is typed - System.Boolean, the lookup values are not trimmed, etc (this is the default way of retrieving list data in the tool).

The updated items are saved using SPWeb.ProcessBatchData(string) - if there're errors during the saving you will see the standard red exclamation mark icons in the left-hand margin of the DataGridView next to the items whose update failed.
The copy-paste functionality allows adding multiple new lines at once. The "copy" logic uses the standard copy functionality of the DataGridView and this has several disadvantages. One is that when drop-downs are used for lookup columns (the combo for lookups button is checked) - the "copy" action gets the display value of the column (the lookup text), not the ID, which then cannot be pasted normally - some nasty error message boxes appear, so don't try this at home.
Note also that not all columns that you can choose from the list field picker are updateable - you will get an error message if you try to update such.
The UI is still not that user friendly as one can wish for, but improvements will follow in the future.

How to serialize generic dictionaries and other xml "unserializable" types with the XmlSerializer

Imagine you have this type of class:

public class test2
{
 public string S { get; set; }
 public Type T { get; set; }
 public Dictionary<string,string> dict = new Dictionary<string,string>() { { "a", "b" }, { "c", "d" } };
}

and you want to serialize it to an XML like this:

<?xml version="1.0" encoding="utf-16"?>
<test2  S="some string" T="System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
 <Dictionary Key="a" Value="b" />
 <Dictionary Key="c" Value="d" />
</test2>

Then you may want to check this class which greatly facilitates the process.
OK, the task at hand doesn't seem very hard and can be achieved in at least two ways:

• one is to define a doublet property for every property or field whose type is not XML serializable. The role of the doublet property will be to convert the value of the original property/field to some serializable type (the setter of this property will do the reverse conversion) - the idea here is simple - the original property/field will be marked with the [XmlIgnore] attribute and its value will be serialized via the doublet property. The main disadvantage of this approach is obvious - you will spend much time creating these doublet properties, your class will probably double in size, you'll have to introduce quite some extra members just for the sake of XML serialization.
• the other approach is to make your class implement the IXmlSerializable interface and write your own serialization and deserialization logic - this one is even harder than the previous - basically what you want the XmlSerializer to do for you, you do it yourself.
  

And now let's have a look at my solution - here is the small class at the top with the extra stuff needed to serialize it the way shown above:

public class test2 : IXmlSerializable
{
[XmlAttribute]
public string S { get; set; }
[XmlAttribute]
public Type T { get; set; }
[XmlElement(ElementName = "Dictionary")]
public Dictionary<string, string> dict = new Dictionary<string, string>() { { "a", "b" }, { "c", "d" } };

#region IXmlSerializable Members
public System.Xml.Schema.XmlSchema GetSchema() { return null; }
public void ReadXml(System.Xml.XmlReader reader)
{
Stefan.Xml.XmlSerializer.ReadXmlDeserialize(this, reader, _proxyData);
}
public void WriteXml(System.Xml.XmlWriter writer)
{
Stefan.Xml.XmlSerializer.WriteXmlSerialize (this, writer, _proxyData);
}
public static readonly Stefan.Xml.XmlSerializer.XmlProxyData _proxyData = new Stefan.Xml.XmlSerializer.XmlProxyData(typeof(test2))
{
TypeMappings = new List<Stefan.Xml.XmlSerializer.XmlTypeMappingBase>()
{
 new Stefan.Xml.XmlSerializer.XmlTypeMapping<Type, string> ()
 {
     GetterMethod = t => t == null ? null : t.AssemblyQualifiedName,
     SetterMethod = s => s == null ? null : Type.GetType(s)
 },

 new Stefan.Xml.XmlSerializer.XmlTypeMapping<Dictionary<string, string>, DictPair[]> ()
 {
     GetterMethod = t => t == null ? null : t.Select(kp => new DictPair(){ Key = kp.Key, Value = kp.Value}).ToArray(),
     SetterMethod = s => { if (s == null) return null; var d = new Dictionary<string, string>(); foreach (var el in s) { d[el.Key] = el.Value; } return d; }
 }
}
};
#endregion
}

So what do we have:

• the original properties decorated with standard XmlXXX attributes (you can use them to your liking), no extra properties added
  
• the class implements the IXmlSerializable interface, but in the ReadXml and WriteXml methods you see just a single call to two methods of the Stefan.Xml.XmlSerializer utility class - this class does the trick and serializes and deserializes for you
  
• and the most important ingredient - the Stefan.Xml.XmlSerializer.XmlProxyData class which you provide as parameter to the Stefan.Xml.XmlSerializer methods. With it you can specify mappings between types and provide method delegates which convert to and from the two mapped types of each mapping. What is the idea here - if you have properties of type which is not XML serializable you can define a mapping for this type to a type which is serializable - e.g. between System.Type and System.String; then you need to provide two method delegates - one which converts from System.Type to System.String and the other - from System.String to System.Type. And when you provide this type mapping to the Stefan.Xml.XmlSerializer class it will treat every property of the mapped type in your class as if it had the substitute type.
  

So I guess you start to get the whole idea behind this serialization approach - it is somewhat of a combination between the two trivial approaches mentioned above but eliminates most of the drawbacks in them and helps automating the process.
Let's have a look now at the Stefan.Xml.XmlSerializer.XmlProxyData class and see how the type mappings can be defined. We have in it this property: List <XmlTypeMappingBase> TypeMappings. XmlTypeMappingBase is an abstract class which is inherited by this generics definition: public class XmlTypeMapping<TProxied, TProxy>, which has these two members that you need to actually set: Func<TProxy, TProxied> SetterMethod and Func<TProxied, TProxy> GetterMethod. These are none other than the conversion methods for the two types in the type mapping. The two type parameters in the XmlTypeMapping<TProxied, TProxy> definition are actually the two types you want to map. TProxied in our example will be System.Type and TProxy will be System.String.
In the sample class above besides the System.Type to System.String mapping, you can see one more mapping - it is between Dictionary<string,string> and DictPair[], the latter is a small class, defined as:

public class DictPair
{
 [XmlAttribute]
 public string Key { get; set; }
 [XmlAttribute]
 public string Value { get; set; }
}

Actually it is a good idea to have such auxiliary classes and the definitions of the XmlTypeMapping<TProxied, TProxy>-s or of the whole List<> of type mappings in a separate utility class, so that they can be reused by the various classes you want to serialize.

How does this work

And several words about how this thing works - so I guess it is more or less clear that Stefan.Xml.XmlSerializer uses internally the standard .NET XmlSerializer. There are yet some other similarities between the two XmlSerializer-s. The standard XmlSerializer generates dynamic classes (serializers for the specific types you serialize which inherit the XmlSerializer class) in dynamic assemblies. Well, this is basically what Stefan.Xml.XmlSerializer does - it creates a dynamic type that has properties matching all public properties and fields of the serialized type - for the properties whose type is in the type mapping list the matching properties are from the substitute type and their getters and setters actually call the conversion methods that you provide in the type mapping definitions. This dynamic proxy type holds an instance of the original type and the getters and setters of its properties get and set the corresponding members of the original type. Also the XmlXXX custom attributes set for the properties of the original type are set for the properties of the proxy type as well. So when you serialize calling Stefan.Xml.XmlSerializer, an instance of this proxy type is created and its internal member is initialized with the instance that you want to serialize - then this proxy instance is serialized by the standard XmlSerializer providing converted properties for all specified types. The deserialization works the opposite - the standard XmlSerializer creates an instance of the proxy type from the XML stream which initializes its internal instance of the original type, then the properties of this instance are copied to the original instance that you provide to the Stefan.Xml.XmlSerializer.ReadXmlDeserialize method.

Reflection proxy class generator - or how to access non-public members and use internal types the faster way

This is a small utility that generates reflection proxy classes - i.e. classes that contain methods and properties matching all methods and properties (public and non-public) of specified .NET classes and calling the latter with reflection. So if you have the following .NET class:

class MyClass{
private void SomeMethod(int i, string s){}
}

the generated reflection proxy class will look something like:

public class MyClassProxy{
public void SomeMethod(int i, string s) { /* reflection call onto the underlying type's method */ }
}

So if you want to make use of some non-visible methods or do something with non-public types with reflection (which is not generally advisable but can be the last resort in many cases) and you are tired of using the awkward MethodInfo-s, PropertyInfo-s and the like, you can try the automatically generated proxy classes which expose hidden members and types and make them readily accessible.

Inspiration and implementation concerns

I should admit here that the idea for creating the proxy generator came to me while I was reflecting the code of the SharePoint solution generator (quite nice stuff one can see in it). What I noticed there was that there were 30 or so proxy classes that call on various methods and properties of the standard classes in the SharePoint object model. I am not sure whether the guys from Microsoft created these by hand or somehow automatically, but it was clear to me that there is lots of stuff that can be accessed only via reflection - since Microsoft themselves create and use reflection proxy classes (at least in the SharePoint domain). Basically the paradigm behind my proxy classes differs quite significantly from theirs at least in respect to the inner implementation and workings. One main concern of mine was that I wanted all the code to be located in the proxy class itself without dependancies to other custom classes e.g. helper classes, base proxy class, etc. The reason for that is transparent - it allows for very easy use of the generated proxies - you just copy the generated .cs file and don't depend on external custom assemblies or other code. And about the class generation implementation itself - I used the System.CodeDom API here - so in the path of the standard recommendations for doing this kind of job providing some good level of flexibility and (at least theoretically) .NET language independance. So no nasty string concatenations for code generation - but you can check the code to find out more.

How to generate

You should first download the generator's solution and build it. It contains two projects: Stefan.CodeDomGen - a class library with one main class - ProxyGenerator - the generator itself; and console - a small console application that calls the ProxyGenerator in the class library.
The console application expects one command line parameter - this is the path to the configuration XML file. The configuration XML file looks like:

<?xml version="1.0" encoding="utf-8" ?>
<project ProxyNamespace="test" ProxyCodeFile="test.cs">
 <assembly Name="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c">
   <class Name="Microsoft.SharePoint.SPWeb" ProxyName="" />
   <class Name="Microsoft.SharePoint.Library.SPRequest" ProxyName="" />
 </assembly>
</project>

Most elements and attributes in it are self explanatory. In the project element - the ProxyNamespace attribute specifies the name of the namespace which will contain the generated proxy classes and the ProxyCodeFile attribute specifies the path of the output code file that will be generated by the tool. The assembly elements denote the assemblies that contain the classes to be proxied and the class elements - the classes to be proxied - note - you can specify public and non-public types here. The ProxyName attribute specifies the name of the proxy class that will be generated - if you leave this attribute blank, the proxy class name will look like this - [Original class name]Proxy - i.e. - the name of the original class with "Proxy" appended to it. And ... that's basically it - once you have the XML file set with all classes that you want to be proxied, just run the console application providing the path to the XML as a parameter and check the generated .cs file after the tool completes the generation.

How to use the proxies

You first need to take the generated .cs file and add it to the project in which you'll make use of the proxies. The project won't compile at first but don't get startled - you need to add references to all assemblies that contain the various types used throughout the proxy classes - in the case of the sample XML above you'll need to add references to the following assemblies - System.Configuration, Microsoft.SharePoint, Microsoft.SharePoint.Library, Microsoft.Web.Design.Server. If you wonder how to add references to the latter two assemblies here's how I do it (if you already know how skip the following sentence) - I locate the assembly I need in a command prompt window using the dir xxxx* /s command from within the c:\windows\assembly folder - because the latter is a special windows folder windows explorer shows a special view of its contents and hides the sub-folder structure that contains the assembly files. So once you've found the exact location of the assembly you just copy it to some temporary folder from which you will be able to reference it using the Visual Studio's Add Reference/Browse command.
And now you are ready to use the proxy classes. But first let me quickly list the extra members that the proxy classes have next to the members matching the methods and properties of the corresponding proxied classes.

- UnderlyingInstance property - each proxy instance contains an instance of the proxied class so that it can call methods on it - obviously - if this one is not set, the proxy won't work. Note here - if the proxied class is public, the type of the property will be the real type of the proxied class, otherwise its type will be just object.
- FromUnderlyingInstance method - static method that returns a proxy instance - it has one parameter which accepts an instance of the proxied type - the type of the parameter is the same as the type of the UnderlyingInstance property. This is how you can get a proxy from an existing instance of the proxied class. It is actually a shortcut to this - MyClassProxy proxy = new MyClassProxy () { UnderlyingInstance = someInstance };. And if you wonder why there is no (or there is still no) implicit conversion operator for the proxied type you can check the comments in the generator code ;)
- CreateUnderlyingInstance static method - actually this may have several overloads - their signatures match the signatures of the various instance constructors of the proxied class - and they return an instance of the proxy class. So they provide the means to create instances of the proxy class which initialize the UnderlyingInstance property with creating a new instance of the proxied class using one of its constructors (internally via a ConstructorInfo). So if you don't have an instance of the proxied class and want to create one, using CreateUnderlyingInstance is the way to do it.

And last but not least - an important note about the types of the parameters and return types of the generated proxy methods - there're two rules here:
1 - if the type has a matching proxy type it is substituted with the latter
2 - if the type is not a public one and has no proxy type it is substituted with the object type (for obvious reasons).

So basically the implications of the above are that the proxied types are in a sense hidden by the proxy types and you'll use and manipulate proxies in parameters and return types rather than instances of the proxied classes. Still the interchange between the proxy class and the proxied class is easily realized with the use of the UnderlyingInstance property and the FromUnderlyingInstance method.

What next

Here's a short list of improvements that can be added to the proxy generator:
1 - I intentionally didn't add support for member fields - no proxy wrappers are generated for them but in some scenarios it may be handy to have such.
2 - Now, method and property wrappers are added for all methods and properties in the proxied classes, but this results in huge proxy class size - may be the "en masse" approach can be changed (or complemented) with some selective approach, so that only specific members get proxied - these can be specified in the configuration XML.
3 - Now, proxies can be generated only for classes - but probably it will be reasonable that support for enumerations and probably delegates and interfaces is added (these will be useful primarily for non-public types).