An Introduction to Linq to Xml (XLinq)

Fabio Vazquez, Microsoft Fabio Vazquez (Redmond, WA) is a Software Engineer for the Office Online Team at Microsoft. He is a former Microsoft Most Valuable Professional (MVP) in C# and VFP and a frequent speaker at software development conferences in Brazil, including the Microsoft PDC (Professional Developers Conference) and Tech-Ed. Fabio has two books published and has a great passion for agile development practices.

Microsoft is working on a group of related technologies that will hopefully change the way we think when programming with .NET languages, especially C# and VB.NET. The new versions of these languages (C# 3.0 and VB 9) bring together some great compiler improvements that enable the so called Linq (Language Integrated Query) features. As the name suggests, Linq brings query constructions to these languages and will allow developers to think on the collections of their programs as sets of values that can be queried, as they do with database tables in SQL language. Any .Net type that implements the IEnumerable interface is eligible to be the source of a query using Linq constructs. Ordinary collections, Xml documents or even objects representing database entities can be queried using high level language constructs. Microsoft is focusing its efforts on enabling a great developer experience when it comes to using the Linq features against Xml documents and Database tables. The project focusing on Linq over Xml documents is called Linq to Xml. There is another project focusing on providing run-time infrastructure for managing relational data as objects, known as Linq to Sql. In this article we will focus on Linq to Xml, trying to show how working with Xml documents will become easier and more natural when Linq to Xml become available. Along the way, we will touch some of the important enabling features that are being incorporated into the next versions of C# and VB, which are the real power force behind all the Linq strategy.

How to get Linq

In order to work with Linq, we need to download Linq Preview. At the time of this writing, the latest version of Linq Preview available for download is the May 2006 CTP (Community Technology Preview). The first step then is to get this package from the following link:
http://www.microsoft.com/downloads/details.aspx?familyid=1e902c21-340c-4d13-9f04-70eb5e3dceea&displaylang;=en

After the setup, new versions of the C# and VB compilers will be installed on your machine. These versions, as the package name suggests, are a “preview” of what is to come on the next generation of these compilers from Microsoft. It is not recommended that you use these compiler versions to build binaries that will be used on a production environment. Use them primarily as study subject.

Linq Preview allows you to alter the Visual Studio 2005 configurations so that it gives support to the new version of the C# compiler and IntelliSense. In order to enable these features, go to the “bin” folder under the Linq Preview installation directory and run the “Install C# IDE Support.Vbs” file. This integration is completely reversible and can be undone by executing the file “Uninstall C# IDE Support.Vbs”, located in the same folder.

NOTE: There is a bug in this CTP that causes commands to disappear from Visual Studio, including some commands that control smart tag display. It would be extremely annoying if it was not possible to overcome this, but it is! You can find the steps to get rid of this bug on the following MSDN Forum link:
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=419975&SiteID;=1

From now on, every time you open a project or solution on Visual Studio 2005, a message box will be presented prompting that an unsupported version of the respective compiler will be used and that some features may not work as expected:

Figure 1: Message box warning that we are using an unsupported version of the compiler.

With this integration with Visual Studio being installed, we can have access to templates for additional Linq specific projects:

Figure 2: Linq project templates in Visual Studio after the installation of the CTP.

Linq to Xml and all language integrated query features are still being designed and will continue evolving until its final release at the “Orcas” timeframe. Orcas is the codename for the next version of Visual Studio, which is expected to ship in 2007, after Windows Vista and Office 2007 have been shipped. Therefore, the information presented in this article is subject to change as Linq features are developed and the community feedback is collected by Microsoft.

NOTE: For those that have already begun to study Linq and the underling new features that are going to be incorporated into C# 3.0 and VB 9 compilers, I would like to clarify that this article will not go into details about language features like Static classes, Extension Methods, Lambda Expressions, Anonymous Types and Object Initalizers. Even though these features are the basis for the excellent Linq functionalities, this article will try to adopt a more pragmatic approach, discussing the topics relevant to the manipulation of Xml documents and the major differences between the Linq model and the classic W3C DOM API implemented by the classes on the System.Xml namespace in .Net.

A brief introduction to Linq

Before going deep into the Linq to Xml API and queries, let’s see a brief introduction to the Integrated Query concepts. Consider the following code that uses the new Integrated Query features to query over an array of integers:

int[] numbers = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };

IEnumerable selected = from number in numbers
                            select number;

foreach (int number in selected)
{
    Console.WriteLine(number);
}

Here we used a SQL like syntax to select all the numbers in the array. The construct in the form “from...in...select” is the real alien thing here. This is the new integrated query syntax that is being incorporated into C# and VB compilers and will hopefully take these .Net languages closer to some functional languages. Notice that the result of the query is returned as an IEnumerable (IEnumerable in the example), making it possible for us to iterate through this structure using a foreach loop. In this example we are not actually doing anything useful with the query, since we are selecting all the same elements present in the original array. Let’s go ahead and select only the even numbers:

IEnumerable selected = from number in numbers
                            where number % 2 == 0
                            select number;

Now this is becoming interesting! As we can see, the “where” clause can receive any valid C# statement. This statement is evaluated for each member in the collection being iterated over (in our example, all number in the numbers collection).

If we wanted to order the result of the query in descending order, we might have used the orderby clause in the query, like this: IEnumerable selected = from number in numbers orderby number descending select number;

Now, let’s see an example using an array of strings:

string[] names = { "Paul", "John", "George", "Ringo"};

var namesLength = from name in names
                  select name.Length;

foreach (int len in namesLength)
{
    Console.WriteLine(len);
}

The query in this code selects the length in characters of each string in the array. It uses the Length property of the string class, since we are querying over an array of strings. If you play close attention to the code above, you will see that there is this strange “var” keyword. This is a new feature from C# 3.0 that allows us to omit the type of any local variable we use in our code. The type will be inferred by the compiler and cannot be changed along the execution of the program. So, this will not mean any revolutionary concept shift in the language and it also doesn’t have any parallel with the “variant” types existing in other languages. It is just a means of leaving to the compiler the task of specifying the type of the object, but it doesn’t take from the programmer the responsibility of knowing the exact type of each variable she is working on. The exception to this rule is when we are working with anonymous types, which are also a new feature of C# 3.0 and VB 9. Anonymous types will not be covered in this article, but there are excellent references about them on the Internet – please, see the References section at the end of this article for more information. Having said that, it is correct to say that these two lines of code are functionally equivalent:

var number = 5;
int number = 5;

NOTE: VB 9 will also have a feature equivalent to the C# “var”, but it will use the old VB programmers’ friend Dim keyword, as this:

Dim number = 5

Which is the same as:

Dim number As Integer = 5

In order to clarity the various possibilities that we have with Language Integrated Query, let’s see some miscellaneous use cases that will hopefully give a task of what can be accomplished with Linq:

To get all directories whose name start with the letter “T” that are children of the root folder:

DirectoryInfo rootDir = new DirectoryInfo("c:\\");

var dirsStartingWithT = from dir in rootDir.GetDirectories()
                        where dir.Name.ToUpper().StartsWith("T")      
                        select dir;

foreach (DirectoryInfo dir in dirsStartingWithT)
{
    Console.WriteLine(dir.FullName);
}

To get all processes that are eating more than 10 MB of memory:

Process [] processesRunning = Process.GetProcesses();

var heavyProcesses = from process in Process.GetProcesses()
                     where process.WorkingSet / 1024 / 1024 > 10
                     select process;

foreach (Process process in heavyProcesses)
{
    Console.WriteLine
        (
            "Memory use of Process {0}: {1} MB", 
            process.ProcessName,
            process.WorkingSet / 1024 / 1024
        );
}

To get, via reflection, all public methods from a given type:

Type targetType = typeof(DateTime);

var constructors = from method in targetType.GetMethods()
                   where method.IsPublic
                   orderby method.Name
                   select method;

foreach (var method in constructors)
{
    Console.WriteLine(method);
}

If you stop to think about how those pieces of code would be written without Linq resources, you may start do notice the huge difference in paradigm that Linq will bring for .Net developers along the time, as they start feeling comfortable with this new syntax and style.

The Linq to Xml object model

The Linq to Xml API is very simple. Few classes play the main role in this model, where we could highlight XNode, XElement, XDocument and XAttribute. If you take out the “X” letter from the beginning of the class name, you have a hint about what each class represents in the Xml DOM. All these classes are defined in the System.Xml.XLinq namespace in the assembly System.Xml.XLinq.Dll.

Figure 3: The Linq to Xml class model. All these classes are from the System.Xml.XLinq namespace.

Writing Xml Documents

When we use the classical DOM model, the task of creating a new document from scratch or composing Xml fragments is made in an imperative way. This means that we have to instruct the API into doing what we want. But the problem is that the API does not converge to the document structure, so we have to pay more attention to the API usage than to the real Xml structure.

Consider that we want to create this simple Xml document:

  
    Charles Darwin
  

If we use the classical DOM model through the System.Xml namespace in a C# program, we can create this document this way:

XmlDocument doc = new XmlDocument();

XmlElement nameElement = doc.CreateElement("Name");
nameElement.InnerText = "Charles Darwin";

XmlElement customerElement = doc.CreateElement("Customer");
customerElement.SetAttribute("Id", "1");
customerElement.AppendChild(nameElement);

XmlElement customersElement = doc.CreateElement("Customers");
customersElement.AppendChild(customerElement);

XmlDeclaration xmlDeclaration = doc.CreateXmlDeclaration("1.0", Encoding.UTF8.HeaderName, String.Empty);
XmlComment xmlComment = doc.CreateComment("Sample Xml Document");

doc.AppendChild(customersElement);
doc.InsertBefore(xmlComment, customersElement);
doc.InsertBefore(xmlDeclaration, xmlComment);

On the other hand, by using the new Linq to Xml classes available in the System.Xml.XLinq namespace, we can write something like the following:

XDocument doc = 
    new XDocument(
        new XDeclaration("1.0", Encoding.UTF8.HeaderName, String.Empty),
        new XComment("Sample Xml Document"),
        new XElement("Customers", 
            new XElement("Customer",
                new XAttribute("Id", "1"),
                    new XElement("Name", "Charles Darwin")
            )
        )
    );

As we can notice, the Linq to Xml syntax is much more intuitive and directed towards the structure of the final document. This makes it easier to use and maintain this code, since it describes more clearly its intention. This style of composing Xml document in Linq to Xml is known as Functional Construction.

Another difference between the W3C model and the Linq to Xml one is the fact that with Linq, we are decoupled from the XDocument class when we create documents. We can use the XElement class if we don’t need to include nodes such as processing instructions and Xml declarations. The following code creates an Xml document without using the XDocument class:

XElement doc = 
    new XElement(
        new XElement("Customers",
            new XElement("Customer",
                new XAttribute("Id", "1"),
                    new XElement("Name", "Charles Darwin")
            )
        )
    );

If we had to write a functional equivalent code using the W3C API, we would have to use the factory methods in System.Xml.XmlDocument class in order to construct the nodes for the elements and attributes.

VB and Xml Literals

As in many other parts of Linq to Xml support, VB takes a more productive approach than C#. One of the differences VB 9 will have over C# 3.0 when it refers to the construction of Xml documents is the use of Xml Literals. To understand this concept better, take a look at the following VB 9 code that creates exactly the same Xml document as the C# code created before:

Figure 4: Xml Literals in VB 9.0 editor.

Notice that we can simply copy and paste the text of and Xml document and pass if to a variable of type XDocument. The string will be resolved by the VB compiler as a series of calls to the constructor of the appropriate class. The IDE has still some features that make the developer experience even better, such as expand/collapse and viewing of hidden text:

Figure 5: Enhanced editor in VB 9.0 supports expand/collapse and hidden text viewing.

Pretty Printing

One task that is unjustifiable hard to perform with the DOM API in .Net is pretty printing the Xml document loaded into an XmlDocument object. The following code shows how this can be done considering that we want to print to the console:

XmlDocument doc = new XmlDocument();
doc.LoadXml(_sampleXml);

XmlTextWriter tw = new XmlTextWriter(Console.Out);
tw.Formatting = Formatting.Indented;
tw.Indentation = 3;

doc.Save(tw);

XDocument doc = XDocument.Parse(_sampleXml);
Console.WriteLine(doc.ToString());

NOTE: The method XDocument.Parse() is functionally equivalent to the XmlDocument.LoadXml() method in System.Xml. It parses the string content passed in as parameter and loads it into the XDocument object.

Searching for Content

In the System.Xml API we have the great support to XPath at our disposal, which makes the task of searching content into Xml a very powerful and flexible activity. Despite the fact that XPath is an extraordinary tool for Xml developers, working with it is not the most obvious thing one can learn. Linq to Xml takes the notion of Xml Axes from XPath and provides some methods that can be used to navigate through the Xml document hierarchy. The examples below show how some of these methods can be used. Please consider that the following document was loaded into a string variable named “_xmlDoc”, as this:

string _xmlDoc = 
    @"
        
        
            Charles Darwin
        
        
            Isaac Newton
        
      ";

To get all the elements with a specific name that are children of a given parent element, we can use the Element(XName) method, which is defined on the XContainer class and is so inherited by the XElement class:

XElement doc = XElement.Parse(_xmlDoc);

IEnumerable elements = doc.Elements("Customer");
Console.WriteLine("Number of elements: {0}", elements.Count());

foreach (XElement element in elements)
{
    Console.WriteLine(element);
}

This code is getting all “Customer” elements that are children of the document element (Customers). It generates the following output:

Number of elements: 2

  Charles Darwin


  Isaac Newton

The Elements() method returns a collection of XElement objects (IEnumerable). This means that we can iterate over each XElement in this collection with a foreach loop, for example.

Notice that the ToString() method of each XElement is being implicitly called by the Console.WriteLine() method, producing a pretty printed output of the element to the console.

To get all children nodes of an element, we can use something like this:

XElement doc = XElement.Parse(_xmlDoc);

IEnumerable nodes = doc.Nodes();
Console.WriteLine("Number of nodes: {0}", nodes.Count());

foreach (XNode node in nodes)
{
    Console.WriteLine(node);
}

This will produce the following output:

Number of nodes: 3


  Charles Darwin


  Isaac Newton

This time 3 nodes were returned. That is because the comment node was also retrieved by the Nodes() method, as would had happened with any other node that might exist in the document.

There is a method named Element() on the XElement class that can be used to get a specific element by searching for its name:

XElement doc = XElement.Parse(_xmlDoc);
XElement firstCustomerElement = doc.Element("Customer");
Console.WriteLine(firstCustomerElement);

  Charles Darwin

XElement doc = XElement.Parse(_xmlDoc);

IEnumerable names = doc.Elements("Customer").Elements("Name");

Console.WriteLine("Number of Name elements: {0}", names.Count());
foreach (XElement name in names)
{
    Console.WriteLine(name);
}

Number of Name elements: 2
Charles Darwin
Isaac Newton

XElement doc = XElement.Parse(_xmlDoc);

IEnumerable ids = doc.Elements("Customer").Attributes("Id");

Console.WriteLine("Number of Id attributes: {0}", ids.Count());
foreach (XAttribute id in ids)
{
    Console.WriteLine
        (
            "Attribute {0} form {1}: {2}", 
            id.Name, 
            id.Parent.Element("Name").Value, 
            id.Value
        );
}

Number of Id attributes: 2
Attribute Id form Charles Darwin: 1
Attribute Id form Isaac Newton: 2

Dim elements As IEnumerable(Of XElement) = doc.Elements("Customer")

Dim elements As IEnumerable(Of XElement) = doc.

Dim ids As IEnumerable(Of XAttribute) = doc..@Id

Dim darwin As IEnumerable(Of XElement) = doc.(0).
Dim isaac As IEnumerable(Of XElement) = doc.(1).

Dim names As IEnumerable(Of XElement) = doc...

Dim names As IEnumerable(Of XElement) = doc..

  
    
      33
      2.5000
      14
      0
    
    
      40
      18.4000
      2
      0
    
    
      62
      49.3000
      10
      0
    
    
      64
      33.2500
      6
      0
    
  
  
    
      39
      18.0000
      10
      0
    
    
      75
      7.7500
      20
      0
    
    
      77
      13.0000
      18
      0
    
  
  
    
      62
      49.3000
      3
      0
    
    
      70
      15.0000
      6
      0
    
  

XElement doc = XElement.Parse(_sampleXml);

IEnumerable orderIds = from anOrder in doc.Descendants("Order")
                                   select anOrder.Attribute("Id");

foreach (XAttribute id in orderIds)
    Console.WriteLine(id);

Id="10574"
Id="10577"
Id="10822"

XElement doc = XElement.Parse(_sampleXml);

IEnumerable orderIds = from anOrder in doc.Descendants("Order")
                               select anOrder.Attribute("Id").Value;

foreach (string id in orderIds)
    Console.WriteLine(id);

10574
10577
10822

XElement doc = XElement.Parse(_sampleXml);

IEnumerable orders = 
    from anItem in doc.Elements("Order").Elements("Item")
    where (int) anItem.Element("ProductID") == 62
    select anItem.Parent;

foreach (var order in orders)
{
    Console.WriteLine(order.Attribute("Id"));
}

XElement doc = XElement.Parse(_sampleXml);

var orders = from anOrder in doc.Elements("Order")
             where (DateTime) anOrder.Attribute("Date") > new DateTime(1998, 01, 01)
             select anOrder;

foreach (var order in orders)
{
    Console.WriteLine
        (
            "Order {0} put at {1}", 
            order.Attribute("Id").Value, 
            ((DateTime) order.Attribute("Date")).ToShortDateString()
        );
}

Order 10822 put at 8/1/1998

  
    4
  
  
    3
  
  
    2
  

XElement doc = XElement.Parse(_sampleXml);

XElement ordersSummary =
    new XElement("OrdersSummary",
        from anOrder in doc.Elements("Order")
        select 
            new XElement("Order",
                new XAttribute("Id", anOrder.Attribute("Id").Value),
                new XElement("ItemsOrdered",
                    anOrder.Elements("Item").Count()
                )
            )
    );

Console.WriteLine(ordersSummary);

public void TransformOrdersXmlToHtml()
{
    XElement doc = XElement.Parse(_sampleXml);

    XElement html = 
        new XElement("html",
            new XElement("head",
                new XElement("title", "Transformation Sample"),
                GetCssStyleElement()
            ),
            GetHtmlBodyContent(doc)
        );

    html.Save("\\Orders.Htm");
}


private object GetCssStyleElement()
{
    return 
        new XElement("style",
            "h2    { background-color:Navy; color:white; margin-bottom:5pt; padding:3pt }",
            "body  { background-color:#cccccc; font-family:Arial; }",
            "table { background-color:White; border:solid 1pt black; }",
            "td    { width:25%; border-bottom: solid 1pt whitesmoke; }"
        );
}

private XElement GetHtmlBodyContent(XElement doc)
{
    return 
        new XElement("body",
            from anOrder in doc.Elements("Order")
            select GetOrderDiv(anOrder)
        );  
}


private XElement GetOrderDiv(XElement orderElement)
{
    return 
        new XElement("div",
            new XElement("h2", "Order: " + orderElement.Attribute("Id").Value),
            new XElement("table",
                new XAttribute("width", "100%"),
                new XElement("tr", 
                    new XElement("th", new XAttribute("align", "center"), "Product Id"),
                    new XElement("th", new XAttribute("align", "right"), "Unit Price"),
                    new XElement("th", new XAttribute("align", "right"), "Quantity"),
                    new XElement("th", new XAttribute("align", "right"), "Total")
                ),
                from item in orderElement.Elements("Item")
                select 
                new XElement("tr", 
                    new XElement("td", new XAttribute("align", "center"), item.Element("ProductID").Value),
                    new XElement("td", new XAttribute("align", "right"), item.Element("UnitPrice").Value),
                    new XElement("td", new XAttribute("align", "right"), item.Element("Quantity").Value),
                    new XElement("td", new XAttribute("align", "right"), 
                     ((double) item.Element("UnitPrice") * (double) item.Element("Quantity")).ToString("n2"))
                )
            )
        );
}