Various Ways to Get Distinct Values from a List<T> using LINQ

Introduction

This article talks about the various scenarios regarding filtering distinct values from a List<T>. One practical example is if you have a list of products and wanted to get the distinct values from the list. To make it more clear, let’s take an example.

Using the Code

Consider that we have this model below that houses the following properties:

public class Product  
{
        public int ProductID { get; set; }
        public string Make { get; set; }
        public string Model { get; set; }
}

Now let’s create a method that would create a list of Products. For example:

private List<Product> GetProducts() {  
            List<Product> products = new List<Product>();
            products.Add(new Product { ProductID = 1, Make = "Samsung", Model = "Galaxy S3" });
            products.Add(new Product { ProductID = 2, Make = "Samsung", Model = "Galaxy S4" });
            products.Add(new Product { ProductID = 3, Make = "Samsung", Model = "Galaxy S5" });
            products.Add(new Product { ProductID = 4, Make = "Apple", Model = "iPhone 5" });
            products.Add(new Product { ProductID = 5, Make = "Apple", Model = "iPhone 6" });
            products.Add(new Product { ProductID = 6, Make = "Apple", Model = "iPhone 6" });
            products.Add(new Product { ProductID = 7, Make = "HTC", Model = "Sensation" });
            products.Add(new Product { ProductID = 8, Make = "HTC", Model = "Desire" });
            products.Add(new Product { ProductID = 9, Make = "HTC", Model = "Desire" });
            products.Add(new Product { ProductID = 10, Make = "Nokia", Model = "Lumia 735" });
            products.Add(new Product { ProductID = 11, Make = "Nokia", Model = "Lumia 930" });
            products.Add(new Product { ProductID = 12, Make = "Nokia", Model = "Lumia 930" });
            products.Add(new Product { ProductID = 13, Make = "Sony", Model = "Xperia Z3" });

            return products;
}

The method above returns a list of Products by adding some dummy data to the List just for the simplicity of this demo. In a real scenario, you may want to query your database and load the result to your model. Now let’s bind the Products data in GridView.

protected void Page_Load(object sender, EventArgs e) {  
            if (!IsPostBack) {
                GridView1.DataSource = GetProducts();
                GridView1.DataBind();
            }
}

Running the code will give you the following output below:

If you notice, there are few items above that contain the same values or commonly called as "duplicate" values. Now let’s try to get the distinct row values from the list using the LINQ Distinct function. The code now would look like this:

if (!IsPostBack) {  
                GridView1.DataSource = GetProducts().Distinct();
                GridView1.DataBind();
}

Unfortunately, running the code will still give you the same output. This means that the Distinct LINQ function doesn’t work at all. I was surprised and my first reaction was like…

(source: http://homepage.ntlworld.com/)

What??? Really???

Yes, it doesn’t work as expected! This is because the Distinct method uses the Default equality comparer to compare values under the hood. And since we are dealing with reference type object then the Distinct will threat the values as unique even if the property values are the same.

So How Are We Going To Deal With This?

There are few possible ways to accomplish this and these are:

Option 1: Using a combination of LINQ GroupBy and Select operators:

if (!IsPostBack) {  
       GridView1.DataSource = GetProducts()
                              .GroupBy(o => new { o.Make, o.Model })
                              .Select(o => o.FirstOrDefault());
       GridView1.DataBind();
}

The code above uses the LINQ GroupBy function to group the list by Make and Model and then use the LINQ Select function to get the first result of the grouped data. The figure below produced the result.

Output

Option 2: Using a combination of LINQ Select and Distinct operators:

if (!IsPostBack) {  
       GridView1.DataSource = GetProducts()
                              .Select(o => new { o.Make, o.Model })
                              .Distinct();
       GridView1.DataBind();
}

The approach above creates a collection of an anonymous types. Doing a Distinct on the anonymous types will automatically override the Equals and GetHashCode to compare each property.

Output

Option 3: Using the IEqualityCompare interface:

class ProductComparer : IEqualityComparer<Product>  
{
            public bool Equals(Product x, Product y) {
                if (Object.ReferenceEquals(x, y)) return true;

                if (Object.ReferenceEquals(x, null) || Object.ReferenceEquals(y, null))
                    return false;

                return x.Make == y.Make && x.Model == y.Model;
            }
            public int GetHashCode(Product product) {
                if (Object.ReferenceEquals(product, null)) return 0;
                int hashProductName = product.Make == null ? 0 : product.Make.GetHashCode();
                int hashProductCode = product.Model.GetHashCode();
                return hashProductName ^ hashProductCode;
            }
}

The Distinct operator has an overload method that lets you pass an instance of IEqualityComparer. So for this approach, we created a class “ProductComparer” that implements the IEqualityCompaper. Here’s the code to use it:

if (!IsPostBack) {  
       GridView1.DataSource = GetProducts()
                              .Distinct(new ProductComparer());
       GridView1.DataBind();
}

The approach above is my preferred option because it allows me to implement my own GetHashCode and Equals methods for comparing custom types. Also getting into a habit of making interfaces makes your code more reusable and readable.

Output

As you observe, the duplicate values are now gone. Now here's another scenario. What if we want to get the distinct values for a certain field in the list? For example, get the distinct “Make” values such as Samsung, Apple, HTC, Nokia and Sony and then populate the result to a DropDownList control for filtering purposes. I was hoping that the Distinct function has an overload that can compare values based on a property or field like GetProducts().Distinct(o => o.PropertyToCompare) but then again, it doesn't seem to have that overload. So I came up with the following workarounds.

Option 1: Using GroupBy and Select operators

if (!IsPostBack) {  
        DropDownList1.DataSource = GetProducts()
                                   .GroupBy(o => o.Make)
                                   .Select(o => o.FirstOrDefault());
        DropDownList1.DataTextField = "Make";
        DropDownList1.DataValueField = "Make";
        DropDownList1.DataBind();
}

The code above returns a collection of Make by grouping it.

Option 2: Using Select and Distinct operators

if (!IsPostBack) {  
         DropDownList1.DataSource = GetProducts()
                                    .Select(o => new { Make = o.Make } )
                                    .Distinct();
         DropDownList1.DataTextField = "Make";
         DropDownList1.DataValueField = "Make";
         DropDownList1.DataBind();
}

The code above creates a collection of Make and then calls the LINQ Distinct to filter out duplicate values.

Output

Running the code for both options above will give this output below:

That’s it! I hope someone finds this post useful. ;)

Summary

In this article, we've learned the various ways to filter a collection of List by providing examples. In the first scenario, I would really recommend option 3 using IEqualityCompare because it allows you to implement your own compararer for comparing custom types. It also makes your code more reusable and manageable.

Points of Interest

I haven't done any speed benchmarking in each approach in terms of querying large result set. It would be worth looking at which approach is faster.

History

• 7^th June, 2016: Initial version