Consuming Google (Reader) with .NET: Part 2 - Google Reader using .NET

• Download source - 77.33 KB

Introduction

This article starts where the previous one (Consuming Google (Reader) with .NET: Part 1 - Authentication) stopped.
This is an implementation of Google Reader in .NET.

Before getting started, it's important to know that there is a site with plenty of information on the Google Reader API: http://code.google.com/p/pyrfeed/wiki/GoogleReaderAPI. But what we'll also be doing is using Fiddler to reverse engineer the Google Reader API.

Reverse Engineering

When you visit Google Reader via your browser, you're basically always sending GET and POST requests. Using Fiddler, you can then view the content of those requests.

Let's say we want to subscribe to http://sandrinodimattia.net/Blog. When you're logged in to Google Reader, you can press the Add Subscription button, enter the URL and press Add. If you do this while Fiddler is open, you'll see the following:

After pressing the Add button, you can see that the URL /reader/api/0/subscription/quickadd was visited and 2 fields were posted (quickadd and T). And for each available action in Google Reader, you can use Fiddler to view the URL and the post fields that are hidden underneath.

If you take a closer look at the screenshot, you see a field called T, this is a token. It identifies your session, but expires quickly. That's why you'll see that our code requests a new token for each new request.

Adding POST to GoogleSession

In the last article, we created the GoogleSession class. This class helps us in getting data from Google. But now that we have to make POST requests, we'll also need to send data to Google. That's why we'll add the following method to our GoogleSession class:

/// <summary>
/// Send a post request to Google.
/// </summary>
/// <param name="url"></param>
/// <param name="postFields"></param>
/// <returns></returns>
public void PostRequest(string url, params GoogleParameter[] postFields)
{ 
    // Format the parameters.
    string formattedParameters = string.Empty;
    foreach (var par in postFields.Where(o => o != null))
        formattedParameters += string.Format("{0}={1}&", par.Name, par.Value);
    formattedParameters = formattedParameters.TrimEnd('&');
    
    // Append a token.
    formattedParameters += String.Format("&{0}={1}", "T", GetToken());
     
    // Get the current post data and encode.
    ASCIIEncoding ascii = new ASCIIEncoding();
    byte[] encodedPostData = ascii.GetBytes(
        String.Format(formattedParameters));
        
    // Prepare request.
    HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
    request.Method = "POST";
    request.ContentType = "application/x-www-form-urlencoded";
    request.ContentLength = encodedPostData.Length;
     
    // Add the authentication header. 
    request.Headers.Add("Authorization", "GoogleLogin auth=" + auth);
    
    // Write parameters to the request.
    using (Stream newStream = request.GetRequestStream())
        newStream.Write(encodedPostData, 0, encodedPostData.Length);
        
    // Get the response and validate.
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    if (response.StatusCode != HttpStatusCode.OK)
        throw new LoginFailedException(
            response.StatusCode, response.StatusDescription);
}    

This function uses a URL and GoogleParameters to build a POST request. And the token we talked about is also automatically included in the post fields. Thanks to this function, we'll be able to send POST requests to Google Reader with ease.

The GoogleService Base Class

Before really getting started, we'll just go and create a base class we might want to re-use when implementing other Google services. It encapsulates the authentication mechanism and the GoogleSession class.

public abstract class GoogleService : IDisposable
{ 
    /// <summary>
    /// Current google session.
    /// </summary>
    protected GoogleSession session;
    
    /// <summary>
    /// Creating this class will automatically try to log in and create a session.
    /// That way for each service we create we don't need to worry 
    /// about the implementation of authentication and session.
    /// </summary>
    /// <param name="service"></param>
    /// <param name="username"></param>
    /// <param name="password"></param>
    /// <param name="source"></param>
    protected GoogleService
	(string service, string username, string password, string source)
    {
        // Get the Auth token.
        string auth = ClientLogin.GetAuthToken(service, username, password, source);
        // Create a new session using this token.
        this.session = new GoogleSession(auth);
    }
    
    /// <summary>
    /// Clean up the session.
    /// </summary>
    public void Dispose()
    {
        if (session != null)
            session.Dispose();
    }
}

Item Types

When reading from Google Reader, you'll be working with 2 different types of data. Pure XML and syndication items. Syndication items (SyndicationItem in .NET) are actually items that come from a feed. For example, if you visit this site you'll get a list of your read items in the shape of an atom feed: http://www.google.com/reader/atom/user/-/state/com.google/read. Each item in this feed is a SyndicationItem.

For both types, we'll be using some basic base classes:

public abstract class GoogleSyndicationItem
{ 
    /// <summary>
    /// Initialize the item.
    /// </summary>
    /// <param name="item"></param>
    internal GoogleSyndicationItem(SyndicationItem item)
    {
        if (item != null)
        {
            LoadItem(item);
        }
    }
    
    /// <summary>
    /// Load the item (to be implemented by inheriting classes).
    /// </summary>
    /// <param name="item"></param>
    protected abstract void LoadItem(SyndicationItem item);
    
    /// <summary>
    /// Get the text from a TextSyndicationContent.
    /// </summary>
    /// <param name="content"></param>
    /// <returns></returns>
    public string GetTextSyndicationContent(SyndicationContent content)
    {
        TextSyndicationContent txt = content as TextSyndicationContent;
        if (txt != null)
            return txt.Text;
        else
            return "";
    }
}

public abstract class GoogleXmlItem : SyndicationItem
{
    /// <summary>
    /// Initialize the item.
    /// </summary>
    /// <param name="item"></param>
    internal GoogleXmlItem(XElement item)
    {
        if (item != null)
        {
            LoadItem(item);
        }
    }
    
    /// <summary>
    /// Load the item (to be implemented by inheriting classes).
    /// </summary>
    /// <param name="item"></param>
    protected abstract void LoadItem(XElement item);
    
    /// <summary>
    /// Get a list of descendants.
    /// </summary>
    /// <param name="item"></param>
    /// <param name="descendant"></param>
    /// <param name="attribute"></param>
    /// <param name="attributeValue"></param>
    /// <returns></returns>
    protected IEnumerable<XElement> GetDescendants
	(XElement item, string descendant, string attribute, string attributeValue)
    {
        return item.Descendants(descendant).Where(o => o.Attribute(attribute) != null && 
		o.Attribute(attribute).Value == attributeValue);
    }
    
    /// <summary>
    /// Get a descendant.
    /// </summary>
    /// <param name="item"></param>
    /// <param name="descendant"></param>
    /// <param name="attribute"></param>
    /// <param name="attributeValue"></param>
    /// <returns></returns>
    protected XElement GetDescendant(XElement item, string descendant, 
	string attribute, string attributeValue)
    {
        return GetDescendants(item, descendant, attribute, attributeValue).First();
    }
    
    /// <summary>
    /// Get the value of a descendant.
    /// </summary>
    /// <param name="item"></param>
    /// <param name="descendant"></param>
    /// <param name="attribute"></param>
    /// <param name="attributeValue"></param>
    /// <returns></returns>
    protected string GetDescendantValue
	(XElement item, string descendant, string attribute, string attributeValue)
    {
        var desc = GetDescendant(item, descendant, attribute, attributeValue);
        if (desc != null)
            return desc.Value;
        else
            return "";
    }
}   

And for almost every type of data in Google Reader, I've also created a class:

• Feed (the URL to a feed with the title of that feed)
• ReaderItem (you could say this is an article, a blog post, ...)
• State (this is the state of an item in Google Reader, like read, starred, ...)
• Subscription (a subscription in Google reader is a feed you subscribed to)

Here is the implementation for subscription:

public class Subscription : GoogleXmlItem
{ 
    /// <summary>
    /// Id of the subscription.
    /// </summary>
    public string Id { get; set; }
    
    /// <summary>
    /// Title of the feed.
    /// </summary>
    public string Title { get; set; }
    
    /// <summary>
    /// URL to the subscription.
    /// </summary>
    public string Url { get; set; }
    
    /// <summary>
    /// List of categories.
    /// </summary>
    public List<string> Categories { get; set; }
    
    /// <summary>
    /// Initialize the subscription.
    /// </summary>
    /// <param name="item"></param>
    internal Subscription(XElement item)
        : base(item)
    {
    }
    
    /// <summary>
    /// Load the subscription item.
    /// </summary>
    /// <param name="item"></param>
    protected override void LoadItem(XElement item)
    {
        // Initialize categories list.
        Categories = new List<string>();
        
        // Regular fields.
        Id = GetDescendantValue(item, "string", "name", "id");
        Title = GetDescendantValue(item, "string", "name", "title");
        
        // Parse the URL.
        if (Id.Contains('/'))
            Url = Id.Substring(Id.IndexOf('/') + 1, Id.Length - Id.IndexOf('/') - 1);
            
        // Get the categories.
        var catList = GetDescendant(item, "list", "name", "categories");
        if (catList != null && catList.HasElements)
        {
            var categories = GetDescendants(item, "string", "name", "label");
            Categories.AddRange(categories.Select(o => o.Value));
        }
    }
} 

URLs Everywhere

Like mentioned before, the Google Reader API is based on URLs and GET/POST requests. To organise this, we've also got a few classes regarding URLs:

• ReaderUrl: A single class containing all the required URLs and paths
• ReaderCommand: Enum representing common tasks (like adding a subscription)
• ReaderCommandFormatter: Class containing extension methods for ReaderCommand to convert these enum values to actual Google Reader URLs

public static class ReaderUrl
{
    /// <summary>
    /// Base url for Atom services.
    /// </summary>
    public const string AtomUrl = "https://www.google.com/reader/atom/";
    
    /// <summary>
    /// Base url for API actions.
    /// </summary>
    public const string ApiUrl = "https://www.google.com/reader/api/0/";
    
    /// <summary>
    /// Feed url to be combined with the desired feed.
    /// </summary>
    public const string FeedUrl = AtomUrl + "feed/";
    
    /// <summary>
    /// State path.
    /// </summary>
    public const string StatePath = "user/-/state/com.google/";
    
    /// <summary>
    /// State url to be combined with desired state. For example: starred
    /// </summary>
    public const string StateUrl = AtomUrl + StatePath;
    
    /// <summary>
    /// Label path.
    /// </summary>
    public const string LabelPath = "user/-/label/";
    
    /// <summary>
    /// Label url to be combined with the desired label.
    /// </summary>
    public const string LabelUrl = AtomUrl + LabelPath;
}

public enum ReaderCommand
{
    SubscriptionAdd,
    SubscriptionEdit,
    SubscriptionList,
    TagAdd,
    TagEdit,
    TagList,
    TagRename,
    TagDelete
}

public static class ReaderCommandFormatter
{
    /// <summary>
    /// Get the full url for a command.
    /// </summary>
    /// <param name="comm"></param>
    /// <returns></returns>
    public static string GetFullUrl(this ReaderCommand comm)
    {
        switch (comm)
        {
            case ReaderCommand.SubscriptionAdd:
                return GetFullApiUrl("subscription/quickadd");
            case ReaderCommand.SubscriptionEdit:
                return GetFullApiUrl("subscription/edit");
            case ReaderCommand.SubscriptionList:
                return GetFullApiUrl("subscription/list");
            case ReaderCommand.TagAdd:
                return GetFullApiUrl("edit-tag");
            case ReaderCommand.TagEdit:
                return GetFullApiUrl("edit-tag");
            case ReaderCommand.TagList:
                return GetFullApiUrl("tag/list");
            case ReaderCommand.TagRename:
                return GetFullApiUrl("rename-tag");
            case ReaderCommand.TagDelete:
                return GetFullApiUrl("disable-tag");
            default:
                return "";
        }
    }
    
    /// <summary>
    /// Get the full api url.
    /// </summary>
    /// <param name="append"></param>
    /// <returns></returns>
    private static string GetFullApiUrl(string append)
    {
        return String.Format("{0}{1}", ReaderUrl.ApiUrl, append);
    }
}

And Finally... ReaderService

Finally there's the implementation of the most common tasks in Google Reader:

public class ReaderService : GoogleService
{
    /// <summary>
    /// Current username.
    /// </summary>
    private string username;
    
    /// <summary>
    /// Initialize the Google reader.
    /// </summary>
    /// <param name="username"></param>
    /// <param name="password"></param>
    /// <param name="source"></param>
    public ReaderService(string username, string password, string source)
        : base("reader", username, password, source)
    {
        this.username = username;            
    }
    
    #region Feed
    /// <summary>
    /// Get the contents of a feed.
    /// </summary>
    /// <param name="feedUrl">
    /// Must be exact URL of the feed, 
    /// ex: http://sandrinodimattia.net/blog/syndication.axd
    /// </param>
    /// <param name="limit"></param>
    /// <returns></returns>
    public IEnumerable<ReaderItem> GetFeedContent(string feedUrl, int limit)
    {
        try
        {
            return GetItemsFromFeed(String.Format("{0}{1}", 
		ReaderUrl.FeedUrl, System.Uri.EscapeDataString(feedUrl)), limit);
        }
        catch (WebException wex)
        {
            HttpWebResponse rsp = wex.Response as HttpWebResponse;
            if (rsp != null && rsp.StatusCode == HttpStatusCode.NotFound)
                throw new FeedNotFoundException(feedUrl);
            else
                throw;
        }
    }
    #endregion
    #region Subscription
    /// <summary>
    /// Subscribe to a feed.
    /// </summary>
    /// <param name="feed"></param>
    public void AddSubscription(string feed)
    {
        PostRequest(ReaderCommand.SubscriptionAdd, 
		new GoogleParameter("quickadd",  feed));
    }
    
    /// <summary>
    /// Tag a subscription (remove it).
    /// </summary>
    /// <param name="feed"></param>
    /// <param name="folder"></param>
    public void TagSubscription(string feed, string folder)
    {
        PostRequest(ReaderCommand.SubscriptionEdit,
            new GoogleParameter("a", ReaderUrl.LabelPath + folder), 
            new GoogleParameter("s", "feed/" + feed), 
            new GoogleParameter("ac", "edit"));
    }
    
    /// <summary>
    /// Get a list of subscriptions.
    /// </summary>
    /// <returns></returns>
    public List<Subscription> GetSubscriptions()
    {
        // Get the XML for subscriptions.
        string xml = session.GetSource(ReaderCommand.SubscriptionList.GetFullUrl());
        // Get a list of subscriptions.
        return XElement.Parse(xml).Elements
	("list").Elements("object").Select(o => new Subscription(o)).ToList();
    }
    #endregion
    #region Tags
    /// <summary>
    /// Add tags to an item.
    /// </summary>
    /// <param name="feed"></param>
    /// <param name="folder"></param>
    public void AddTags(ReaderItem item, params string[] tags)
    {
        // Calculate the amount of parameters required.
        int arraySize = tags.Length + item.Tags.Count + 2;
        
        // Set all parameters.
        GoogleParameter[] parameters = new GoogleParameter[arraySize];
        parameters[0] = new GoogleParameter("s", "feed/" + item.Feed.Url);
        parameters[1] = new GoogleParameter("i", item.Id);
        
        int nextParam = 2;
        
        // Add parameters.
        for (int i = 0; i < item.Tags.Count; i++)
            parameters[nextParam++] = new GoogleParameter
		("a", ReaderUrl.LabelPath + item.Tags[i]);
        for (int i = 0; i < tags.Length; i++)
            parameters[nextParam++] = new GoogleParameter
		("a", ReaderUrl.LabelPath + tags[i]);
            
        // Send request.
        PostRequest(ReaderCommand.TagAdd, parameters);
    }
    
    /// <summary>
    /// Rename a tag.
    /// </summary>
    /// <param name="tag"></param>
    /// <param name="newName"></param>
    public void RenameTag(string tag, string newName)
    {
        PostRequest(ReaderCommand.TagRename,
            new GoogleParameter("s", ReaderUrl.LabelPath + tag),
            new GoogleParameter("t", tag),
            new GoogleParameter("dest", ReaderUrl.LabelPath + newName));
    }
    
    /// <summary>
    /// Remove tag (for all items).
    /// </summary>
    /// <param name="tag"></param>
    public void RemoveTag(string tag)
    {
        PostRequest(ReaderCommand.TagDelete,
            new GoogleParameter("s", ReaderUrl.LabelPath + tag),
            new GoogleParameter("t", tag));
    }
    
    /// <summary>
    /// Remove tag from a single item.
    /// </summary>
    /// <param name="item"></param>
    /// <param name="tag"></param>
    public void RemoveTag(ReaderItem item, string tag)
    {
        PostRequest(ReaderCommand.TagEdit,
            new GoogleParameter("r", ReaderUrl.LabelPath + tag),
            new GoogleParameter("s", "feed/" + item.Feed.Url),
            new GoogleParameter("i", item.Id));
    }
    
    /// <summary>
    /// Get a list of tags.
    /// </summary>
    /// <returns></returns>
    public List<string> GetTags()
    {            
        string xml = session.GetSource(ReaderCommand.TagList.GetFullUrl());
        
        // Get the list of tags.
        var tagElements = from t in XElement.Parse(xml).Elements
				("list").Descendants("string")
                          where t.Attribute("name").Value == "id"
                          where t.Value.Contains("/label/")
                          select t;
                          
        // Create a list.
        List<string> tags = new List<string>();
        foreach (XElement element in tagElements)
        {
            string tag = element.Value.Substring(element.Value.LastIndexOf('/') + 1, 
                element.Value.Length - element.Value.LastIndexOf('/') - 1);
            tags.Add(tag);
        }
        
        // Done.
        return tags;
    }
    
    /// <summary>
    /// Get all items for a tag.
    /// </summary>
    /// <param name="tag"></param>
    /// <param name="limit"></param>
    /// <returns></returns>
    public IEnumerable<ReaderItem> GetTagItems(string tag, int limit)
    {
        return GetItemsFromFeed(String.Format("{0}{1}", 
	ReaderUrl.LabelPath, System.Uri.EscapeDataString(tag)), limit);
    }
    #endregion
    #region State
    /// <summary>
    /// Add state for an item.
    /// </summary>
    /// <param name="item"></param>
    /// <param name="state"></param>
    public void AddState(ReaderItem item, State state)
    {
        PostRequest(ReaderCommand.TagEdit,
            new GoogleParameter("a", 
		ReaderUrl.StatePath + StateFormatter.ToString(state)),
            new GoogleParameter("i", item.Id),
            new GoogleParameter("s", "feed/" + item.Feed.Url));
    }
    
    /// <summary>
    /// Remove a state from an item.
    /// </summary>
    /// <param name="item"></param>
    /// <param name="state"></param>
    public void RemoveState(ReaderItem item, State state)
    {
        PostRequest(ReaderCommand.TagEdit,
            new GoogleParameter("r", 
		ReaderUrl.StatePath + StateFormatter.ToString(state)),
            new GoogleParameter("i", item.Id),
            new GoogleParameter("s", "feed/" + item.Feed.Url));
    }
    
    /// <summary>
    /// Get the content of a state. 
    /// For example: Get all starred items.
    /// </summary>
    /// <param name="state"></param>
    /// <param name="limit"></param>
    /// <returns></returns>
    public IEnumerable<ReaderItem> GetStateItems(State state, int limit)
    {
        return GetItemsFromFeed(String.Format("{0}{1}", 
		ReaderUrl.StateUrl, StateFormatter.ToString(state)), limit); 
    }
    #endregion
    
    /// <summary>
    /// Post a request using a reader command.
    /// </summary>
    /// <param name="command"></param>
    /// <param name="postFields"></param>
    private void PostRequest(ReaderCommand command, params GoogleParameter[] postFields)
    {
        session.PostRequest(ReaderCommandFormatter.GetFullUrl(command), postFields);
    }
    
    /// <summary>
    /// Get items from a feed and convert them to a GoogleReaderItem.
    /// </summary>
    /// <param name="url"></param>
    /// <param name="limit"></param>
    /// <returns></returns>
    private IEnumerable<ReaderItem> GetItemsFromFeed(string url, int limit)
    {
        SyndicationFeed feed = session.GetFeed(url, 
		new GoogleParameter("n", limit.ToString()));
        return feed.Items.Select<SyndicationItem, ReaderItem>(o => new ReaderItem(o));
    }
}

And as you can see, the ReaderService does a few things:

• Subscriptions (list, add)
• Tags (add, rename, delete, ...)
• States (add, remove, list)
• Feed (list contents)

And it actually re-uses a few of the things we talked about:

• GoogleSession to send post requests, get feeds, ...
• ReaderCommand, ReaderCommandFormatter, ReaderUrl to do all the URL related stuff
• GoogleParameter to set POST fields (fields we can find using Fiddler)

Putting It All Together

The console application was also updated with our ReaderService:

class Program
{
    static void Main(string[] args)
    {
        // Empty line.
        Console.WriteLine("");
        
        // Get username.
        Console.Write(" Enter your Google username: ");
        string username = Console.ReadLine();
        
        // Get password.
        Console.Write(" Enter your password: ");
        string password = Console.ReadLine();
        
        // Query.
        using (ReaderService rdr = new ReaderService
		(username, password, "Sandworks.Google.App"))
        {
            // Display.
            Console.WriteLine("");
            Console.WriteLine(" Last 5 articles from Sandrino's Blog: ");
            foreach (ReaderItem item in rdr.GetFeedContent
		("http://sandrinodimattia.net/blog/syndication.axd?format=rss", 5))
            {
                Console.WriteLine("  - " + item.Author + ": " + item.Title);
            }
        }
        
        // Pause.
        Console.ReadLine();
    }
}

There you go, now you've got everything you need to get started with Google Reader in .NET. The following article will be about creating a basic WPF application to have a simple desktop version of Google Reader.

Enjoy...

History

• 12^th July, 2010: Initial post