Jigsaw Security runs a big data platform that was purpose built to provide situational awareness for cyber security, military and business intelligence. One of the goals of building the system was to create a platform that could show how often things were occurring which in turn can tell you the risk exposure of your business or to indicate patterns that in the case of military use could potentially safe war fighters from injury or death.
Over the last month, Jigsaw Security has begun testing of a new analytic module that can detect when the tone of news articles is negative or when comments are positive in nature. Below is an example of a stream of normal news articles.
As you can see there are several topics being monitored. A business in financial industries may want to monitor for keywords such as stock market or some similar topic, financial news, financial sentiment and other areas of interest. In this case we are monitoring several keywords which are listed below:
Democrats
Mueller Report
President/Collusion
President (in general)
Replublican
Terrorist Activity
These are some of the topics that Jigsaw Security data scientist had already entered in searches previously so we just went with this as an example of what news is being generated. There is nothing particular interesting about this data set until we start looking at it in different ways.
Sentiment
One of the things we always want to understand as data scientist is sentiment. We want to know what the general public thinks about a particular news article or how something is being handled. In our analytic model we decided to look for words that were positive and also negative in describing a situation. Below are a few positive terms and some negative ones:
Positive Terms Example
supportive
awesome
exceeding expectations
winning
Negative Terms Example
penalty
legal action
lawsuit
losses
In reality our word definitions for positive and negative sentiment are very large and also are based on how the words are used. For example to determine a positive outcome our criteria would probably be something like "had a great time" or "excelled at college" or maybe "winning shot won the game". These large data sets are used to pick out positive and negative sentiment based on how an article is written. The next thing we have to do then is look and see exactly what has been going on in near real time or over a particular time period. When we started looking at some various dates and time we can start to see large swings in data. In the search below we looked for President as our keyword and we found a day where there was a huge spike in activity.
What we know over time is what is normal for a particular term or the manner in which terms are being used in articles. What these spikes can tell us is that something has changed, some condition has triggered or something has happened. The same method can be used just on simple volume of new news stories. When big news happens, the number of articles will surely increase to report significant events. This in and of itself is a good indicator of a change in normal news reporting volumes and when something major occurs.
Remember our sentiment discussion above? Well when we find a spike in activity or an incident or change in what is "normal" for the new cycle, it is best to take a look and see if the stories being written during an event are positive or negative.
In the example above we can see by looking at our sentiment algorithm that the incident above is being reported in a positive manner and is being reported as a positive news story. In the case above the news stories being reported was Merck and Pfizer cooperating in the medical space to extend kidney cancer survival rates. This spike in activity shows that the sentiment on that story was widely reported and covered by many media organizations. Below is a excerpt of the stories being read at that time.
As you can see there is business value in looking at how a story is perceived and you can make educated analytic guesses prior to publishing your own story based on how things are worded or how the story is written.
We just wanted to take some time today and show how intelligence platforms can be used to look at data in ways that can help you make informed decisions in your business, whatever it is!
About: The images and information presented in this post are from the Jigsaw Analytic Platform that allows Jigsaw Security customers to make use of Big Data solutions in their own use cases. For more information on our data solutions, visit the Portfolio section of our website to see how we can help your business to leverage the Jigsaw Analytic Platform. The platform can do more than just stop hackers, it can help you make highly informed decisions based on historical data, perception, sentiment and predictive analytics. Jigsaw Security is a managed security provider and integrator of big data solutions.