Node-RED Hosting in the Cloud with a Twitter Feed Analyzer Sample

Due to applications and components in global IoT (The Internet of Things) ecosystem getting more and more interrelated, it becomes essential to be capable of wiring them together to achieve automation. Advanced connectivity of systems allows to eliminate repetitive programming tasks and therefore - to free time for research and development.
One of the tools to make different systems interact with each other is Node-RED  - a simple programming tool for binding online services, hardware devices, and APIs in an interactive and visually-appealing fashion. It provides a browser-based editor and a wide set of nodes to build and deploy various functional flows.
Within this article, we’ll take a look at a solution we’ve implemented for a quick Node-RED delivery and will deploy a sample flow of Twitter streaming data analyzer to store and alert upon posts on any theme you are interested in.

A Simple Way to Run Node-RED Instance Within a Cloud

At Jelastic, the ready-to-go Node-RED development environment is provided as a single-click installation package. It is available within Jelastic JPS Collection on GitHub alongside with other preconfigured solutions and could be installed on any Platform of 4.9.5 Jelastic version and higher.  
All the required resources and settings for running Node-RED on Jelastic are declared within the package manifest.jps file, where the main points to consider are:

• nodered/node-red-docker Docker image with the latest tag is used as a solution base
• created Node-RED instance is assigned 16 dynamic cloudlets, which enable it to scale vertically up to 2 GiB of RAM and 6.4 GHz of CPU
• automatically attached endpoint allows to access Node-RED admin panel through a randomly allocated port at Shared Load Balancer (eliminating the necessity to purchase Public IP address)

So, let’s bring it to life - log in to your Jelastic Cloud account and proceed with the following steps.  
1. Within the main dashboard page, click Import at the top and switch to the URL tab. Here, insert link to the corresponding manifest.jps file:
https://github.com/jelastic-jps/node-red/blob/master/manifest.jps Click Import to continue.
2. In the opened installation dialog box:

• type Environment name to be used as its internal hostname
• optionally, specify environment Display Name (so-called alias, for being shown within dashboard and SSH terminal)
• select an environment Region (if multiple ones are available)

Click Install and wait a bit to be shown a notification message on this process completion. Click Open in browser to access Node-RED development environment and start creating your first project (so-called Flow). Further, we’ll learn how to interact with this wizard through a hands-on example of Twitter content tracker - building such a useful and practical tool will also help you to get started with Node-RED as quick as possible.

Deploying Twitter Feed Analyzer as Node-RED Flow

So, the application we are going to build is intended to search through Twitter feed upon a specified criteria (keyword, hashtag, and/or username), conduct sentiment analysis of observed posts and process them accordingly. Depending on the message character, the taken action should be:

• upon detecting negative content - to alert via email message
• for positive and neutral results - to store a mention within a dedicated file inside a container, for each type of characteristic separately

For analysis, we’ll use #Cloud, #CloudComputing, and #PaaS hashtags as a base for the corresponding hosting area investigation. Obviously, you are free to specify any other search parameters upon your requirements - e.g. your company name.
The whole application structure will look like below:Here, the following component types (nodes) are involved:

• Twitter - used to authenticate with your Twitter account and specify search keywords
• Sentiment - designed to analyze posts and return a corresponding value (negative, positive or neutral)
• Switch - intended to evaluate sentiment scores and take a corresponding action according to the defined tone
• two File Nodes - provided to store positive and negative messages
• Email - allows receiving instant alert messages with negative feedback

Altogether, such a flow enables you to quickly react to negative feedback and, simultaneously, to learn about user experience, best practices, news, and events within neutral and positive posts. This way, building such a tool will help you to get relevant information in time and stay tuned to what’s happening in the market.
So, below we’ll consider the parameters that should be adjusted for each of these node types and consequently implement the described above workflow.

Twitter Node

Within the Node-RED flow editor, expand the social section and drag&drop the twitter icon to place it on the sheet. This will be the very base of our application.Double-click the picked node and specify the following information in the opened dialog box:

• Twitter ID - click on the pencil icon to Authorise app with your Twitter account login and password; after this operation successfully passes, select Add to finish
• Search - choose an area for conducting search - for example, all public tweets
• for - specify a search term (keyword, hashtag, and/or username); multiple entries сould be separated with spaces (for the bot to look for their joint mention) or commas without spaces (to detect separate terms)
• Name - optionally, give some custom name to the node (or leave the default one)
  Select Done above to proceed.

Tip: For more comprehensive feedback analysis, append an additional debug node to the main twitter one so that fetched posts could be reviewed directly through the editor.As a result, records of all semantic types will be available within the sidebar debug tab, being stored within its default msg.payload log.

Sentiment Node

Now, expand the analysis section and choose a sentiment node - it will analyze found posts and return a corresponding sentiment value to determine whether a tweet is negative, positive or neutral.This component requires no additional adjustments unless you’d like to change its name (otherwise, double-click this block, specify the preferred denomination and click Done).

Switch Node

Now let’s add another switch element from the function section. It is intended to evaluate the received sentiment score and take a corresponding action according to the defined tone.

To adjust the switch node configs, double-click it and provide the following details in the opened dialog box:

• Name - type custom node name (or leave the default one)
• Property - specify the msg.sentiment.score value and +add the following rules to determine which sentimental value should be considered as neutral, positive or negative respectively:
  • == 0
  • > 0
  • < 0 

Select checking all rules at the box bottom and click Done.

File Nodes

Now, we are going to set up logging of the observed neutral and complimentary messages. But before the corresponding node insertion, you need to create file(s) for such mentions to be stored in - thus, open Config manager for your Node-RED container and enter the desired destination directory.
Here, select New file within options menu, type a name for it (e.g. neutral) and press Enter. Repeat these actions to create another file for positive tweets.

Then, expand storage wizard section and drag the comprised  file node to the sheet. Double-click this block to adjust the appropriate parameters:

• Filename - specify a full path to a destination storing file
• Action - select append to file to add new tweets to the end of a file. Optionally, tick  the following additional options are available for this settings:
  • Add newline (\n) to each payload? - for a new line to be added to every message (can be turned off if required)
  • Create directory if it doesn't exist? - for a new directory to be created if it doesn't exist
• Name - enter index number of the required switch node condition so that tweets with the corresponding semantic value will be written to this file (e.g. 1 for neutral posts in this example)

Click Done to confirm. Repeat these operations to configure one more file node that will process and save positive tweets.

Email Node

The last required email component could be located under the social section.Place it within the work area and double-click to provide the following details:

• To - type an email address the alerts on negative tweets should be sent to
• Server & Port - leave the default values (smtp.gmail.com host and 465 port number correspondingly)
• Userid & Password - provide credentials to an email account from which the notifications will be sent
• Name - type index number of the corresponding switch node condition according to the post type you’d like to receive the alerts on (in our case, semantic range for negative tweets was defined 3rd)

When you are finished, click Done.

Checking Results

So, your self-made Twitter bot has started to work, tracking the current posts and processing the information you’ve asked about. Subsequently, results of this analysis could be checked as follows:

• for neutral/positive mentions - click Config for the Node-RED instance within your Jelastic dashboard and navigate to the appropriate file; its content can be reviewed just through the embedded file manager:
  

• for negative tweets - just track new incomings within your email box
  

That’s it! The Node-RED flow is now up and running to analyze streaming data from Twitter. Try it yourself by getting an account at one of Cloud Hosting Platforms.