Facebook and Twitter API

Overview

This code works by creating request using the Facebook/Twitter App’s keys and tokens, and screen name. These keys and tokens are required for oAuth 2.0 authentication protocols that Facebook and Twitter use.  If the parameters are valid, the portlet creates and sends a request to the Twitter Search API and the Facebook Graph API Explorer using the Apache Commons framework and receives a corresponding JSON document for each with the requested posts.

Technologies used

  • Twitter API
  • Facebook Graph API
  • Java Libraries
  • Apache Commons Http Client Library
  • Javax Servlet Library

Facebook and Twitter HTTP Requests

Both the Facebook and Twitter APIs work using HTTP requests. Simply put, is the protocol that is used by web servers and browsers to communicate. HTTP is based on a request and a response. The portlet uses it's configuration fields and creates requests to both Facebook and Twitter. This requests are send using Query Strings

Query Strings

The query string format is an internet standard for HTTP requests. It is the part of a URL which assigns values to specified parameters. Only the basics will be seen in order to have basic understand of the requests that are sent to the APIs. Take for example, a similar request to the one used for the Twitter API:

https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=ONTO2050&count=16

When this request is sent, it's indicating that it's intended for the user_timeline endpoint (the URL of that specific service) and that the response will be a JSON object. The question mark is used as a separator, and is not part of the query string. After the question mark, what follows its the list of the Query String Parameters that the API will receive. Some guidelines must be followed when using Query Strings:

  • The URL should always have a single question mark in it "?", which marks the start of the parameter's list.
  • URL Query String parameters should be separated by the ampersand "&"
  • Each URL Query String parameters is made of two elements: the query string parameter name and its value, and these are separated by the equals sign "="

For example, in the previous request we have a query string with 2 parameters: screen_name with the value "ONTO2050", and count with the value "16".

Twitter API

Twitter Timeline Endpoints

The Twitter API has several endpoints that return a timeline of Tweet data, the endpoint used in this portlet is the GET statuses / user_timeline, which returns a collection of the most recent Tweets posted by the indicated by the screen_name or user_id parameters in a JSON object. For more advanced documentation, follow this link.

The structure of the request used for this portlet it's the following:

https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=ONTO2050&count=16&exclude_replies=true&tweet_mode=extended

Parameters used

Name Description Example
screen_name The screen name of the user for whom to return results. ONTO2050
count Specifies the number of Tweets to try and retrieve, up to a maximum of 200 per distinct request.  16
exclude_replies This parameter will prevent replies from appearing in the returned timeline.  true
tweet_mode Extra-parameter that asks to render Tweets that contain more than 140 characters extended

JSON response structure

The next code shows the structure of the JSON that it's received. It's a JSON Array that contains a JSON Object for each number of the Tweets that were requested, and it returns many attributes that are not required. This is a simplified version that only shows the attributes that are used on the portlet:

[
	//JSON Object of a non-retweeted post
    {
        "created_at": "Tue Oct 08 20:21:09 +0000 2019",
		"id_str": "1181665857350582272",
        "full_text": "We are in Elgin for the last suburban Small Plans, Big Ideas forum event, thank you @gailbrdnlibrary for hosting us today. https://t.co/9EDzKLd6fy",
        . . .
        "user": {
            "screen_name": "ONTO2050",
            . . .
            },
            . . .
        },
        "favorite_count": 5,
        . . .
    },

	//JSON Object of a retweeted post
    {
        "created_at": "Mon Oct 07 16:34:47 +0000 2019",
        "entities": {
            "user_mentions": [
                {
                    "screen_name": "MyUrbanDreams",
                    . . .
                },
                . . .
            ],
			. . .
        },
        "retweeted_status": {
            "id_str": "1180622007483932672",
            "full_text": "@janegrover and I spent the morning in Lake County with high school students of #congressmanbradschneider's STEM program. We shared some of @onto2050's regional work and had discussions about climate change.",
            },
			. . .
        },
        "favorite_count": 0,
		. . .
    },
    {
		. . .
    }
]

There are some important differences in the type of posts that are received, and a post can be either a tweet posted by the account or a retweet of a tweet made by another account. Depending on the type of post, the logic for getting the attributes changes.  As you can see, the key attributes are:

Attributes Description
retweeted_status First attribute to be checked. If it exists, the post is a retweet
created_at Date when the post was created or retweeted by the account
full_text The text of the tweet. If it is a retweet, it must be obtained from the JSON object retweeted_status, otherwise it will be truncated
screen_name The screen name of the account that posted the tweet. If it's a retweet, it's inside the JSON Array user_mentions inside entities, and it's always the first element of the array. Otherwise, it's in the JSON Object user
id_str Post id used to created the link to the post.  If it is a retweet, it's located inside the JSON Object retweeted_status
favorite_count Number of "Favorites" the post has

Creating the request in the portlet

To create the request in the portlet, the HTTP Apache Commons Library is used. Using these library, we are able to send a request using a GET method (GET is used to request data from a specified resource). However, another step it's required: making an Authorization Request Header.

Authorization Request Header

The Authorization Request Header contains the credentials to authenticate a user agent with a server, and the Twitter API requires this in order to grant permission to use the API. Basically, the specific header Twitter requires its compromised by the Twitter App Keys/Tokens and other parameters.

String authorizationHeaderValue = "OAuth oauth_consumer_key="some token", oauth_nonce="1570727464118", oauth_signature="JHTvha%2BSS5F26nzjZG4DWSPakWk%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1570727464", oauth_token="some token", oauth_version="1.0"

The code above represents and example of an Authorization Header Twitter uses. This header it's build with the next values:

Header parameter Description
oauth_consumer_key The Twitter Consumer Key, identifies which application is making the request
ouath_nonce Unique token the portlet generates for each unique request. Twitter will use this value to determine whether a request has been submitted multiple times
oauth_signature Value generated by running all of the other request parameters and two secret values (Twitter Consumer Secret and Twitter Access Token Secret) through a signing algorithm
oauth_signature_method The signature method algorithm used by Twitter is HMAC-SHA1. This value should be used for any authorized request sent to Twitter’s API
oauth_timestamp Indicates when the request was created. This value should be the number of seconds since the Unix epoch at the point the request is generated, easily generated by Java
oauth_token The Twitter Access Token, represents a user’s permission to share access to their account. 
oauth_version Should always be 1.0 for any request sent to the Twitter API.

The methods and logic used to generate some of these parameters will not be described here, and they can be analyzed from the source code if desired. For more detailed documentation regarding the Twitter header, follow this link.

Sending the request and getting a response 

Now that we have our Authorization Request Header, we can now create the request and get the response. For this, we will use the HTTP Apache Commons GetMethod and HTTPClient classes. The next code represents the process:

GetMethod getMethod = new GetMethod("https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=ONTO2050&count=16&exclude_replies=true&tweet_mode=extended");
getMethod.addRequestHeader("Authorization", authorizationHeaderValue);
HttpClient cli = new HttpClient();
cli.executeMethod(getMethod);
String response = getMethod.getResponseBodyAsString();
JSONArray jsonResponseArray = JSONFactoryUtil.createJSONArray(response);

The steps performed were as follows:

  1. Create a new GetMethod using the URL Query String as a parameter.
  2. Call the method addRequestHeader, and give it two parameters: the "Authorization" string which specifies the type of header, and the string of the Authorization Request Header we created.
  3. Create a new HttpClient that will make the request, and then call the method executeMethod that takes as parameter our GetMethod. The GetMethod object will now receive the response.
  4. Call the method getResponseBodyAsString that gives the response from Twitter as String response.
  5. Convert the String response to the JSONArray that portlet will now process using the method JSONFactoryUtil.createJSONArray.

Facebook Graph API

Facebook Posts Endpoints

Similar to Twitter, the Facebook API has several endpoints. The endpoint used in this portlet is the GET id / posts, which returns the public posts that the Facebook page with the specified id made. The structure of the request used for this portlet it's the following:

The structure of the request used for this portlet it's the following:

https://graph.facebook.com/id/posts?fields=likes.limit(0).summary(true),message,permalink_url,created_time&limit=16&date_format=r&access_token=some_token

The id at the beginning of the URL it's the Facebook page ID. Will only work if the Facebook app has "Public Access Permissions", otherwise the id must be changed to 'https://graph.facebook.com/me/posts? . . .', since it will only have access the its own posts. For more information about Facebook permissions follow this guide

Parameters used

Unlike the Twitter API, Facebook allows to specify the fields we require to reduce the payload and avoid unnecessary fields. This is done with the parameter fields.

Name Description Example
fields The fields of data required for the posts, separated by a comma. Will be explained below likes.limit(0).summary(true),message
limit Number of posts to be returned 16
date_format Code that specifies the date format, accepts only PHP date formats r
access_token The Facebook Access Token of the app "some_random_token"

Next, we will see each of the attributes that can be used for the fields parameter:

Attribute Description
likes.limit(0).summary Number of likes the Facebook post has
message The message of the Facebook post
permalink_url The external link to the Facebook post
created_time The creation date of the post with the date_format specified

JSON response structure

The next code shows the structure of the JSON that it's received. It's a JSON Object that contains a JSON Array of the requested publications. This JSON it's much more short and simple.

{
    "data": [
        {
            "message": "Some Facebook publication",
            "permalink_url": "https://www.facebook.com/2464045940331772/posts/2258990760837292",
            "created_time": "Thu, 30 May 2019 18:48:09 +0000",
            "id": "2464045940331772_2258990760837292",
            "likes": {
                "summary": {
                    "total_count": 70,
                    . . .
                }
            }
        },
        . . .
    ],
    . . .
}

The key attributes of the JSON are the same ones that the field parameter specifies in the request.

Creating the request in the portlet

To create the request in the portlet, the HTTP Apache Commons Library is also used, done similarly to Twitter by sending a request using a GET method. However, one key difference it's that these type of Facebook requests require no Authorization Header, since that is usually done bya Facebook login, which the portlet does not require. Instead, it only sends the access token as part pf the URL Query String as means of authorization.

Sending the request and getting a response 

Since we require no header, we can proceed to the request, very similarly to the Twitter API. The next code represents the process:

GetMethod getMethod = new GetMethod("https://graph.facebook.com/me/posts?fields=likes.limit(0).summary(true),message,permalink_url,created_time&limit=16&date_format=r&access_token=some_token");
HttpClient cli = new HttpClient();
cli.executeMethod(getMethod);
String response = getMethod.getResponseBodyAsString();
JSONObject jsonResponseObject = JSONFactoryUtil.createJSONObject(response);
JSONArray jsonResponseArray = jsonResponseObject.getJSONArray("data");

The process is basically the same, but with two key differences:

  1. We do not add call method addRequestHeader since there is no request header.
  2. Twitter receives a JSON Array, but Facebook receives a a JSON Object with the JSON Array called "data".

After getting the requests

After this process it's completed, and the portlet has now the JSON Arrays it received from both Twitter and Facebook, it will parse them and create the custom SocialMediaPost classes. That process will not be documented here, but the code will be shared for individual analysis. This code only works to illustrate the code logic and auxiliary methods, and it will most likely not compile as a standalone as it is.

SocialSliderPortlet.java