Technological progress during the twentieth century was driven by the needs of National Security and war – just look at the aircraft from the ‘30s and those from the ‘50s. Advances in medicine, mathematics, engineering, electronics, and computing can all be traced back to the massive investments made by national governments during the second world war, designed to give their country the edge in the conflict.
I’m not saying that the national security issues today are the same as in 1940, but I do believe that the interest being shown by the FBI and Department for Homeland Security in using social media, and the funds they put into research and development, could lead to breakthroughs and a change in the way in which social media is viewed by governments, law enforcement agencies, corporations, and individuals.
According to a post on WebProNews, the FBI are looking for a system with prediction built in, so that they and other law enforcement agencies can react to events before they get picked up by news agencies, and so that they can predict and plan their requirements to some degree. The really interesting thing to me is that they want to combine social media streams with other streams of data, in order to increase the value of the prediction.
Nearly a year ago, the UK Technology Strategy Board ran a competition, Have I got views for You? in their Small Business Research initiative, funded by Instinct (Innovative Science and Technology in Counter Terrorism). The team which became Churnbar came together to enter this competition, and looking back through the project documents, it’s almost as though the FBI have lifted their requirement directly from the TSB competition.
The combination of social media data with other data to make predictions, whether for law enforcement, national security, or business planning will bring the public voice directly into the planning process, and allow for near-real time forecasting of events, in a similar way to Lampos & Christianini’s work on “nowcasting” flu outbreaks from analysis of Twitter posts (see the Flu Detector).
A considerable amount of work is required to develop these systems, so the funding provided by Instinct in the UK and the FBI and DHS in the USA should be applauded as a means to an end. Naturally, considerations of privacy, and human rights need to be taken into account, as with the use of any new communications technology by the authorities, but the benefits of this work should spread far beyond law enforcement.