The future of crowdsourcing: Integrating humans with machines

In early February, the Intelligence Advanced Research Projects Activity (IARPA) announced a multi-year effort to develop and test collaboration methods to improve reasoning. The Crowdsourcing Evidence, Argumentation, Thinking, and Evaluation (“CREATE”) program intends to improve analysts’ and decision makers’ understanding of the information that supports their conclusions.

{mosads}But does IARPA’s initiative represent current and future crowdsourcing trends, especially given the long R&D cycle of government organizations?

Crowdsourcing isn’t new. The US Department of Defense (DoD) and the intelligence community (IC) have been using the technique for decades:

• Following the disappearance of USS Scorpion in 1968, the Navy established a large and varied group of experts who successfully located the submarine, after more traditional measures had failed.
• In the early 2000s, the CIA ran a series of prediction-market initiatives called Future Map, in which experts (in phase one) and the general crowd (in phase two), voted on the chances of certain geopolitical occurrences unfolding in a stock-market-style competition.
• Several years later, IARPA sponsored a large project to improve the prediction of future events, which in turn yielded a larger project called the Good Judgment Project, led by political psychologist Phillip Tetlcok. This project researched, among other things, the ability of groups to get better results than individuals when trying to predict future events.

In 2016, acting director of the GSA Technology Transformation Service’s Innovation Portfolio Kelly Olson emphasized the need for continued crowdsourcing in government. CREATE exemplifies crowdsourcing’s growing popularity with the DoD and the IC, and their understanding of its potential in solving complex problems.  

The crowdsourcing industry as a whole is still booming. IBISWorld research estimates that the market for US crowdsourced service providers grew at a 45.5% CAGR during 2011–2016, putting it at about $6.5 billion for 2016.

Yet despite the growing popularity of crowdsourcing for ideation, research, analysis, and prediction, the Silicon Valley crowdsourcing scene seems to be losing the hype it once enjoyed. Users can choose from nearly 3,000 platforms acquire information and information tools, innovative and creative ideas, funding, and human or specialized labor. The industry appears to have matured, and entrepreneurs are now looking for novel ways to take crowdsourcing to the next level.

While crowdsourcing is indeed an excellent way to solve problems, the industry and its clients are now more than ever aware of the limitations: issues with quality control, the need for significant resources to manage crowds, lack of participant commitment, confidentiality concerns, the too-common dominance of “loud” participants over real experts, and the challenge of generating high resolution knowledge that general crowds (even of experts) can’t produce simply because they don’t know the client well enough.

Many companies and startups are now developing their crowdsourcing offerings to account for these problems by covering more areas of implementations such as higher education or governance, diversifying crowds or making the crowds more specific (e.g., a crowd of life scientists), and creating more efficient incentive mechanisms to encourage participation.

What’s next? How can crowdsourcing progress to version 2.0? The future of crowdsourcing relies on its ability to integrate the human component (crowds) with advanced technological capabilities, most prominently artificial intelligence (AI) and big data. The next big thing for crowdsourcing will be a better synergy between sophisticated information technology and the human judgment that is, at least for now, irreplaceable.

That future is already here:

• CrowdFlower’s AI allows businesses to perform tasks with algorithms and machine learning, then bring in human judgment if they’re not quite as confident in their technology – and the human input makes the algorithms smarter.
• The Artificial Intelligence for Disaster Response (AIDR) combines crowdsourcing with real-time machine learning for disaster response.
• British WireWax uses AI and human input to automatically detect countless features in videos.
• Republic Systems allows clients to utilize the best open innovation platforms by monitoring the popularity of over 3,000 existing crowdsourcing platforms.

These and many other examples depict a bright future for crowdsourcing. They also illustrate a transition to an aggregation of insights generated by a constellation of analysts empowered by advanced technology and vice-versa – what I have termed “big knowledge.”

Though the integration of AI, machine learning, big data, and crowdsourcing still has a way to go before it can be totally useful and reliable, that day seems to be fast approaching, especially considering that human-machine technologies are only in their infancy. Big knowledge has the potential to rekindle the hype, and the hope, of crowdsourcing.

Shay Hershkovitz, Ph.D., is chief strategy officer at Wikistrat, Inc. and a political science professor at Tel Aviv University specializing in intelligence studies. He is also a former IDF intelligence officer whose book, “Aman Comes To Light,” deals with the history of the Israeli intelligence community.

The views expressed by contributors are their own and are not the views of The Hill.