The Spring.NET Application Framework provides a series of social extensions for Twitter, Dropbox and LinkedIn. The focus of this post will be using the Twitter extensions. At the time of this writing the Spring.Social.Twitter Nuget Package is at 2.0.0.M1. The goal is to create a windows service to poll twitter for a users time-line and all followers and their time-lines. Please keep in mind the intent of the post is not to show best practices or build a complete end to end application but simply to play with Spring.Social.Twitter and RavenDB. I know there are applications out there already that do this but I wanted to build something. Recently, the itch returned, after a bit I found Spring.NET Framework Social Extensions and decided to give it a go. I did look at Linq2Twitter a while back but thought the authentication needed work and never really found the time to go further. It does appear to have been updated recently, will have to check it out again. That said, time to start building a simple console app to demonstrate how to
- Authenticate with Twitter REST API using Spring.NET Application Framework Social Extensions.
- Use SlowCheetah to do App.config transformations.
- Setup RavenDb Embedded for data storage.
- Leverage Spring.NET Application Framework Social Twitter Extensions to simply and quickly pull tweets in chunks of 200 and persisting the data in RavenDB.
If all goes as planned, subsequent post(s) will delve into refactoring the console application to do more (i.e. dealing with max_id and since_id for twitter polling ), eventually porting to a windows service.
The assumption is the reader is familiar with Visual Studio 2012, has a basic understanding of Twitter, has a Twitter account or will create one, and has a basic understanding of RavenDb. Don’t worry about how to setup or configure RavenDb for Embedded or IIS mode, this will be covered to some extent in the post.
Here is the complete source. A
demo branch is the working branch, where all changes will be made prior to integrating into the master. Outlined below are the change-sets the blog attempts to walk through, starting with an initial demo shell and ending with persisting tweets to RavenDB.
- Start with Demo Shell
- Authenticate with Twitter Api
- Setup Embedded RavenDb
- Pull Twitter Timeline Data Asynschronously
- Persist to RavenDB
- Familiar with VS 2012
- Familiar Nuget Package Manager Console
- Familiar with Twitter
- Own a Twitter Account or can create a twitter account
- Using Locally Hosted RavenDB in IIS or Embedded mode
- VisualStudio 2012
- Spring.NET Social Twitter Extensions 2.0.0-M1
- RavenDb 2.0.2230
- Visual Studio Extension SlowCheetah – XML Transforms
- Twitter Account
- Twitter Application Tokens ( will generate )
- Install Visual Studio Extension –> SlowCheetah – XML Transforms. This will require a VS 2012 restart.
- Create a simple console application called SocialTrailsDemo using Visual Studio 2012.
- Add a couple of references
- Generate Twitter Application Tokens
- Setup App.config Transforms using SlowCheetah – XML Transforms
- Authenticate with Twitter API using OAuth
- Setup RavenDB Embedded Database
- Pull Twitter Time-line Data Asynchronously
- Persist to RavenDB
- View Data in RavenDb Studio
SlowCheetah VS Extension Setup
- Open VS 2012.
- Navigate to
Tools-> Extensions and Updates
- Search for
- This will require a VS 2012 restart.
Use Package Manager Console to Add References
If you have not already done so, please pull start demo shell to get started. Open the Package Manager Console in VS to install the nuget packages mentioned above and type the
Intall-Package commands below.
- Spring.Social.Twitter nuget package,
Install-Package Spring.Social.Twitter -Pre
- RavenDB.Embedded nuget package,
Install-Package RavenDB.Embedded -version 2.0.2230
Setup App.Config Transforms
- Open VS 2012
- Install Visual Studio Extension SlowCheetah – XML Transforms
- Requires a VS 2012 restart
- Open SocialTrailsDemo
- Right click on App.config
- Select “Add Transform” menu item to create a Debug and Release transform for the App.config file.
- Add the appSettings keys from
code block 2to the app.config.
- Add the twitter keys necessary for authentication to appSettings code-block for app.debug.config and app.release.config to
code block 2.
- Add the
code block 3to App.Debug.config and App.Release.config files.
- Add or update .gitignore file to ignore app.debug.config and app.release.config as seen in
code block 1.
1 2 3 4 5 6 7 8
1 2 3 4 5 6
Authenticate with Twitter API
Spring.NET Social Twitter offers integration with Twitter’s REST API through the ITwitter interface and its implementation, TwitterTemplate. Creating an instance of TwitterTemplate involves invoking its constructor, passing in the application’s OAuth credentials and an access token/secret pair authorizing the application to act on a user’s behalf.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Setup RavenDB Embedded Database
In a previous post, RavenDB Embedded Configuration proper configuration of RavenDB Embedded is discuss in more detail. A couple quick points on configuration listed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46
Pull Twitter Time-line Data Asynchronously
At this point just pull a set number of tweets from a twitter handle ( will refactor later to use max_id and since_id ). Two new methods are created,
GetUserTimeLine to load a time-line using Spring.Social.Twitter Nuget Package asynchronously.
ITwitter provides a series of asynchronous operations against a twitter account. In this example, the goal is to pull a time-line based on a twitter handle in chunks of 200, adding the results to a IList
TimelineOperations.GetUserTimelineAsync(psScreenName, 200). it is important to note Twitter limits the amount of data pulled back in a single request and the number of requests are time-boxed, more details. At this point the the list is simply output to the console. In the next code block the data will be persisted to RavenDB.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
Persist to RavenDB
Persisting Spring Tweet objects to RavenDb is simple once RavenDb is configured. Open a DocumentSession with a using statement to auto close the session once operations are complete. The logic in the using statement loops through the
IList<Tweet> tweet list passing each Tweet object to the DocumentSession Store method then calling SavesChanges on the DocumentSession. RavenDb handles serializing the Tweet POCO, including nested objects to JSON before persisting the RavenDb Document Store. RavenDb supports batch processing but we are not using it here. Outlined below is the git diff of the changes necessary to persist to RavenDB and the final code.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
View Data in RavenDb Studio
Reviewing the DocumentStore in RavenDb Studio is quick and simple. Since the current configuration is an
Embedded, In Memory version of RavenDb running on
port 9090 to view RavenDb Studio you must be running the console application. Then open Chrome, navigate to url
http://localhost:9090 to see RavenDb Studio.