Spring.NET Framework Social Extensions

Spring Framework Social ExtensionsThe Spring.NET Application Framework provides a series of social extensions for Twitter, Dropbox and LinkedIn. The focus of this post will be using the Twitter extensions. At the time of this writing the Spring.Social.Twitter Nuget Package is at 2.0.0.M1. The goal is to create a windows service to poll twitter for a users time-line and all followers and their time-lines. Please keep in mind the intent of the post is not to show best practices or build a complete end to end application but simply to play with Spring.Social.Twitter and RavenDB. I know there are applications out there already that do this but I wanted to build something. Recently, the itch returned, after a bit I found Spring.NET Framework Social Extensions and decided to give it a go. I did look at Linq2Twitter a while back but thought the authentication needed work and never really found the time to go further. It does appear to have been updated recently, will have to check it out again. That said, time to start building a simple console app to demonstrate how to

  • Authenticate with Twitter REST API using Spring.NET Application Framework Social Extensions.
  • Use SlowCheetah to do App.config transformations.
  • Setup RavenDb Embedded for data storage.
  • Leverage Spring.NET Application Framework Social Twitter Extensions to simply and quickly pull tweets in chunks of 200 and persisting the data in RavenDB.

If all goes as planned, subsequent post(s) will delve into refactoring the console application to do more (i.e. dealing with max_id and since_id for twitter polling ), eventually porting to a windows service.

The assumption is the reader is familiar with Visual Studio 2012, has a basic understanding of Twitter, has a Twitter account or will create one, and has a basic understanding of RavenDb. Don’t worry about how to setup or configure RavenDb for Embedded or IIS mode, this will be covered to some extent in the post.

Here is the complete source. A demo branch is the working branch, where all changes will be made prior to integrating into the master. Outlined below are the change-sets the blog attempts to walk through, starting with an initial demo shell and ending with persisting tweets to RavenDB.

Change-sets:

Assumptions:

  • Familiar with VS 2012
  • Familiar Nuget Package Manager Console
  • Familiar with Twitter
  • Own a Twitter Account or can create a twitter account
  • Using Locally Hosted RavenDB in IIS or Embedded mode

Requirements:

  • VisualStudio 2012
  • NuGet
  • Spring.NET Social Twitter Extensions 2.0.0-M1
  • RavenDb 2.0.2230
  • Visual Studio Extension SlowCheetah – XML Transforms
  • Twitter Account
  • Twitter Application Tokens ( will generate )

Basic Setup:

SlowCheetah VS Extension Setup

  • Open VS 2012.
  • Navigate to Tools-> Extensions and Updates
  • Click Online
  • Search for SlowCheetah
  • Click Download button
  • Click Install button
  • This will require a VS 2012 restart.

SlowCheetah VS Extension SetupSlowCheetah VS Extension Setup

Use Package Manager Console to Add References

If you have not already done so, please pull start demo shell to get started. Open the Package Manager Console in VS to install the nuget packages mentioned above and type the Intall-Package commands below.

  • Spring.Social.Twitter nuget package, Install-Package Spring.Social.Twitter -Pre
  • RavenDB.Embedded nuget package, Install-Package RavenDB.Embedded -version 2.0.2230

Setup App.Config Transforms

  1. Open VS 2012
  2. Install Visual Studio Extension SlowCheetah – XML Transforms
  3. Requires a VS 2012 restart
  4. Open SocialTrailsDemo
  5. Right click on App.config
  6. Select “Add Transform” menu item to create a Debug and Release transform for the App.config file.
  7. Add the appSettings keys from code block 2 to the app.config.
  8. Add the twitter keys necessary for authentication to appSettings code-block for app.debug.config and app.release.config to code block 2.
  9. Add the code block 3 to App.Debug.config and App.Release.config files.
  10. Add or update .gitignore file to ignore app.debug.config and app.release.config as seen in code block 1.
1. app.config 
1
2

    # User Config Files
      App.*.config
2. app.config 
1
2
3
4
5
6
7
8

  <appSettings>
    <!-- Fill in your consumer key and secret here to make the OAuth sample work. -->
    <!-- Twitter sign-up: https://dev.twitter.com/ -->
    <add key="twitterConsumerKey" value=""/>
    <add key="twitterConsumerSecret" value=""/>
    <add key="twitterOAuthAccessToken" value=""/>
    <add key="twitterOAuthAccessTokenSecret" value=""/>
  </appSettings>
3. app.debug.config and app.release.config appSettings section 
1
2
3
4
5
6

<appSettings>
  <add key="twitterConsumerKey" value="AddKey" xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
  <add key="twitterConsumerSecret" value="AddSecret" xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
  <add key="twitterOAuthAccessToken" value="AddToken" xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
  <add key="twitterOAuthAccessTokenSecret" value="AddTokenSecret" xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
</appSettings>

Authenticate with Twitter API

Spring.NET Social Twitter offers integration with Twitter’s REST API through the ITwitter interface and its implementation, TwitterTemplate. Creating an instance of TwitterTemplate involves invoking its constructor, passing in the application’s OAuth credentials and an access token/secret pair authorizing the application to act on a user’s behalf.

git change-set

4. TwitterTemplate OAuth Example 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

//
using System;
using System.Configuration;
//
using Spring.Social.Twitter.Api;
using Spring.Social.Twitter.Api.Impl;

namespace SocialTrailsDemo
{
    class Program
    {
        static void Main(string[] args)
        {
            Run();
        }

        static private void Run()
        {
            string consumerKey = ConfigurationManager.AppSettings["twitterConsumerKey"]; // The application's consumer key
            string consumerSecret = ConfigurationManager.AppSettings["twitterConsumerSecret"]; // The application's consumer secret
            string accessToken = ConfigurationManager.AppSettings["twitterOAuthAccessToken"]; // The access token granted after OAuth authorization
            string accessTokenSecret = ConfigurationManager.AppSettings["twitterOAuthAccessTokenSecret"]; // The access token secret granted after OAuth authorization

            ITwitter twitter = new TwitterTemplate(consumerKey, consumerSecret, accessToken, accessTokenSecret);

            Console.WriteLine("Authenticated against Twitter Rest API using Spring.Net Application Framework Social Twitter Extensions.");
            Console.ReadKey();
        }
    }
}

Setup RavenDB Embedded Database

In a previous post, RavenDB Embedded Configuration proper configuration of RavenDB Embedded is discuss in more detail. A couple quick points on configuration listed below.

  • RunInMemory=true
  • UseEmbeddedHttpServer=true
  • NonAdminHttp.EnsureCanListenToWhenInNonAdminContext(9090)

git change-set

5. Setup RaveDB Embedded Database 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

//
using System;
using System.Configuration;
//
using Raven.Client.Embedded;
using Raven.Database.Server;
//
using Spring.Social.Twitter.Api;
using Spring.Social.Twitter.Api.Impl;

namespace SocialTrailsDemo
{
    class Program
    {
        static void Main(string[] args)
        {
            Run();
        }

        static private void Run()
        {
            string consumerKey = ConfigurationManager.AppSettings["twitterConsumerKey"]; // The application's consumer key
            string consumerSecret = ConfigurationManager.AppSettings["twitterConsumerSecret"]; // The application's consumer secret
            string accessToken = ConfigurationManager.AppSettings["twitterOAuthAccessToken"]; // The access token granted after OAuth authorization
            string accessTokenSecret = ConfigurationManager.AppSettings["twitterOAuthAccessTokenSecret"]; // The access token secret granted after OAuth authorization

            ITwitter twitter = new TwitterTemplate(consumerKey, consumerSecret, accessToken, accessTokenSecret);

            using (var store = new EmbeddableDocumentStore
            {
                DataDirectory = "~/App_Data/Database",
                RunInMemory = true,
                UseEmbeddedHttpServer = true,
            })
            {
                NonAdminHttp.EnsureCanListenToWhenInNonAdminContext(9090);
                store.Initialize();
                store.HttpServer.SystemConfiguration.AllowLocalAccessWithoutAuthorization = true;

                Console.Write("RavenDb Embedded Document Store Initialized Successfully.");

                Console.ReadKey();
            }
        }
    }
}

Pull Twitter Time-line Data Asynchronously

At this point just pull a set number of tweets from a twitter handle ( will refactor later to use max_id and since_id ). Two new methods are created, PerformInitialTimelineLoad and GetUserTimeLine to load a time-line using Spring.Social.Twitter Nuget Package asynchronously. ITwitter provides a series of asynchronous operations against a twitter account. In this example, the goal is to pull a time-line based on a twitter handle in chunks of 200, adding the results to a IList, TimelineOperations.GetUserTimelineAsync(psScreenName, 200). it is important to note Twitter limits the amount of data pulled back in a single request and the number of requests are time-boxed, more details. At this point the the list is simply output to the console. In the next code block the data will be persisted to RavenDB.

git change-set

6. Pull Twitter Timeline 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

//
using System;
using System.Collections.Generic;
using System.Configuration;
//
using Raven.Abstractions.Extensions;
using Raven.Client;
using Raven.Client.Embedded;
using Raven.Database.Server;
//
using Spring.Social.Twitter.Api;
using Spring.Social.Twitter.Api.Impl;

namespace SocialTrailsDemo
{
    class Program
    {
        static void Main(string[] args)
        {
            Run();
        }

        static private void Run()
        {
            string consumerKey = ConfigurationManager.AppSettings["twitterConsumerKey"]; // The application's consumer key
            string consumerSecret = ConfigurationManager.AppSettings["twitterConsumerSecret"]; // The application's consumer secret
            string accessToken = ConfigurationManager.AppSettings["twitterOAuthAccessToken"]; // The access token granted after OAuth authorization
            string accessTokenSecret = ConfigurationManager.AppSettings["twitterOAuthAccessTokenSecret"]; // The access token secret granted after OAuth authorization

            ITwitter twitter = new TwitterTemplate(consumerKey, consumerSecret, accessToken, accessTokenSecret);

            using (var store = new EmbeddableDocumentStore
            {
                DataDirectory = "~/App_Data/Database",
                RunInMemory = true,
                UseEmbeddedHttpServer = true,
            })
            {
                NonAdminHttp.EnsureCanListenToWhenInNonAdminContext(9090);
                store.Initialize();
                store.HttpServer.SystemConfiguration.AllowLocalAccessWithoutAuthorization = true;

                Console.Write("RavenDb Embedded Document Store Initialized Successfully.");

                PerformInitialTimelineLoad(store, twitter, "frozenbytes");

                Console.ReadKey();
            }
        }

        private static void PerformInitialTimelineLoad(IDocumentStore store, ITwitter ctx, string psScreenName)
        {
            IList<Tweet> tweets = GetUserTimeLine(ctx, psScreenName);

            foreach (var tweet in tweets)
                Console.WriteLine("{0} - Message -> {1}", tweet.User.ScreenName, tweet.Text);

        }

        private static IList<Tweet> GetUserTimeLine(ITwitter pxTwitterCtx, string psScreenName)
        {
            ulong statusesCount = 1400;
            int i = Convert.ToInt32(statusesCount);

            IList<Tweet> twits = new List<Tweet>();

            while (i > 0)
            {
                // sinceId and maxId
                var task = pxTwitterCtx.TimelineOperations.GetUserTimelineAsync(psScreenName, 200);
                twits.AddRange(task.Result);
                i = i - 200;
            }

            return twits;
        }

    }
}

Persist to RavenDB

Persisting Spring Tweet objects to RavenDb is simple once RavenDb is configured. Open a DocumentSession with a using statement to auto close the session once operations are complete. The logic in the using statement loops through the IList<Tweet> tweet list passing each Tweet object to the DocumentSession Store method then calling SavesChanges on the DocumentSession. RavenDb handles serializing the Tweet POCO, including nested objects to JSON before persisting the RavenDb Document Store. RavenDb supports batch processing but we are not using it here. Outlined below is the git diff of the changes necessary to persist to RavenDB and the final code.

git change-set

Persist Timeline to RavenDb DiffPersist Timeline to RavenDb Diff

7. Persist Twitter Timeline to RavenDB 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

private static void PerformInitialTimelineLoad(IDocumentStore store, ITwitter ctx, string psScreenName)
{
    using (IDocumentSession s = store.OpenSession())
    {
        IList<Tweet> tweets = GetUserTimeLine(ctx, psScreenName);

        foreach (var tweet in tweets)
        {
            Console.WriteLine("{0} - Message -> {1}", tweet.User.ScreenName, tweet.Text);
            s.Store(tweet);
        }

        s.SaveChanges();
    }
}

View Data in RavenDb Studio

Reviewing the DocumentStore in RavenDb Studio is quick and simple. Since the current configuration is an Embedded, In Memory version of RavenDb running on port 9090 to view RavenDb Studio you must be running the console application. Then open Chrome, navigate to url http://localhost:9090 to see RavenDb Studio.

RavenDb Studio SocialTrails Collection of DocumentsRavenDb Studio SocialTrails Collection of Documents

RavenDb Studio SocialTrails Tweet View 1RavenDb Studio SocialTrails Tweet View 1

RavenDb Studio SocialTrails Tweet View 1RavenDb Studio SocialTrails Tweet View 1

References

Comments