May 2013

Where Does My Tweet Go?

Measuring the spreading of information through Twitter

The Filter Bubble challenge

Every day, we follow more people and more brands on social networks, yet none of us have time to keep pace with the resulting constant flow of information. On Twitter, this flow of information has two characteristics: it has an extreme velocity (500M tweets are created everyday), and it contains a lot of noise (about 75% of tweets are never re-tweeted). Just like Google, and their beloved PageRank that organize the web by prioritizing content according to its relevance, we thought the world would benefit from a filter that would sort the signal and the noise. Our challenge was therefore to determine a way to evaluate what is signal and what is noise, and sort tweets according to the value of the information they carry.

Example of WDMTG screens

Introducing SpreadRank

We created an algorithm, the SpreadRank designed to compute a score for each tweet. The beauty of the SpreadRank algorithm is that it is organically linked with the natural visualization of information propagation throughout the network: the higher the SpreadRank, the more stretched out the spreading graph is. By visualizing the SpreadRank of each tweet we enable users to see understand how far a specific message has spread and what made it spread so far.

Macbook displaying WDMTG

Browsing tweets

Where Does My Tweet Go (WDMTG) is the web application that gives access to the power of the Spreadrank. WDMTG allows browsing, exploration and analysis of information propagation through Twitter via a beautiful user interface. Key features include:

Keywords definition

WDMTG allows user to define a list of keywords, hashtags or Twitter accounts they want to listen to.

Example of keywords

Real-time listening

Focus on Spreadrank feature

WDMTG will then parse in realtime the Twitter firehose and spot every tweet coming from these accounts and/or containing these keywords or hashtags, and gets re-tweeted. Applying the SpreadRank algorithm on these tweets will score each one of them according to the value of the information they carry.

SpreadRank visualization

As message spreading is a concept that is hard to apprehend, visualization is a key facet of WDMTG. Several visualization options are available:

  • 2D - This provide a top view of the propagation graph. Mouse dragging the graph makes it easily switch from 2D to 3D state.
  • 3D - This provides a perspective view of the graph. Camera orientation and zoom level can be easily changed.
  • Timeline - This view shows the retweets horizontally, sorted by time of retweet.
  • Tree - This view shows spreading as a tree, highlighting biggest influencers and spreading levels.

These multiple visualization options allow easy understanding of SpreadRank score for a specific tweet. It also reveals hidden insights: number of RTs, number of propagation layers reached, Twitter accounts who were key in the information propagation (influencers), etc.

Analytics and activation

WDMTG is a highly versatile tool, adding to powerful listening capabilities and great dataviz analytics and activations capabilities.

WDMTG analytics tooling allow users to have access global metrics on a given topics, such as reach and levels, or to have demographics on people interested in a particular key word, hashtag or account. Analytic tooling also provide a ranking of most influent Twitter accounts per topic.

Key features of WDMTG include is the possibility to download the list of most influent Twitter accounts as a CSV format. This feature closes the gap between simple listening and reporting and activation and allows multiple usage, particularly :

  • Direct contact
  • Influencers partnership
  • Advertising on influencers followers
  • Look-alike targeting

Behind the scenes

Technology: the backend engine is written in Scala and uses Akka and Storm to process high-volume data from Twitter to generate graphs in real-time. The frontend uses modern HTML features such as Web Workers, the History API, and WebGL, using the fantastic Three.js library.

WebGL, HTML5 and Three.js logos Learn more about this project