The alchemy of big data in government

For nine luminous centuries, the Library of Alexandria was a grand beacon of innovation. The ancient Egyptians understood the power of collected and analyzed data (containing over 700,000 manuscripts), which helped elevate Egypt to a world power. The Library of Alexandria stood as antiquity’s shining light, playing a central role in the development of medical thought and practice.

{mosads}The potential for today’s Alexandria can be found in the expansive amount of readily publically available data. The possibilities of this data leading a digital transformation are endless. According to Eric Schmidt, CEO of Google, we produce more data every other day than we did from the inception of early civilization until the year 2003, combined. Therefore, the organizing, managing and analyzing of data are all more important than ever.

Data are everywhere, flowing from the sensor networks that surround us and at the root of our transactional activities. What, why and how we make choices in our lives are reflected in, and can be discerned through, the collection, organization and taxonomy of data.

The U.S. government maintains one of largest repository of documents in the world. Millions of supporting documents are compiled and stored every year by a multitude of government agencies, which have a responsibility to preserve, secure and retrieve vital information when needed. While paper documents are still very much routine for government operations, the goal has been to increasingly move from paper to electronic images.

The means and processes of transactions can vary within financial systems or on the global digital marketplace. The information we can gain from analyzing these transactions can be very important to any business. When the extracted data are systematically combined with multi-layered analytics, a forensic and predictive meaning is created that can be transformed into actionable insights in reporting systems.

Data analytics can be of great value to both the public and private sectors. In government, transactional activities can be found throughout all agencies. The processing and disbursement of payments occurs every day to millions of beneficiaries through Social Security, veterans benefits, workman’s compensation, federal retirement, utilities and insurance needs resulting from natural disasters.

In 2012, the Obama administration announced the Big Data Research and Development Initiative. Six federal departments and agencies were selected and provided with more than $200 million in new commitments that “together, promise to greatly improve the tools and techniques needed to access, organize, and glean discoveries from huge volumes of digital data.” The purpose of the initiative was to improve government’s ability to extract insights from various data streams and make better decisions in support of national security objectives, scientific discovery or to help drive economic growth. Two years later, the government has embraced data analytics as an integral aspect of operational function in the federal space.

The Big Data Research and Development Initiative does formulate a working paradigm for a better qualification of risk management. In various government programs, including financial services, transportation, human resources, Medicaid, government subsidies and expense management, the resulting analytics can be used, to detect fraud, waste and abuse by examining habits and trends derived from transactions.

Another industry that is effectively using and analyzing data to make informed, educated decisions is transportation. Government agencies are using data to gain information on scheduling mass transit arrival times to provide updates to citizens and data gained from cameras, sensors and geo-tracking with analytics can “see” whether a particular spot is occupied or not, and transmit that information to a device within the car, such as GPS or the driver’s mobile phone, to actually guide the driver to the closest available space. Predictive analytics can also be used to enable transit operators and managers to make decisions rapidly for near real-time adjustments of vehicles or services. For example, analytics can be used to plan roadways based on projected population growth, anticipate the impact of temporarily closing a road, build a new station or change a route. Agencies can also anticipate rider reaction to a fare increase to determine potential attrition, or to identify what would happen if a specific route was cancelled or adjusted by 15 minutes.

Additionally, data can provide informed insights in the healthcare system. By analyzing information gained from data, statistics can be used to curtail unnecessary spending and enhance visibility by measuring outcomes and value. This insight helps track quality of care and serves as care management software to make sense of, and direct patient care from, all perspectives.

This type of analysis on the agency level can save taxpayers money, therefore strengthening public trust and increasing efficiency by providing timely and needed aid. Advances in automation and transactional data modeling can also directly impact customer service, especially in contact centers, by allowing for more rapid and accurate agency decision-making.

In the critical and specialized area of government response to disasters, cultivating data from spending patterns on products, medicines and services during a hurricane can provide real-time, vital information to local law enforcement, Federal Emergency Management Agency (FEMA) and the Coast Guard regarding logistics gaps by location. Lessons learned and insights gained through data retrieval can lead to better protocols, preparation and the ability to predict trends in the face of future catastrophic events.

In the private sector, information mined from transactions can be used for demographical analysis and to calculate consumer purchasing habits, credit risks and predict consumer trends. Financial institutions can use predictive algorithms to create the best financial management options from market and transactional data. Combined with social media analytics, optimizing economic forecasting has become a new data analytic art.

In both the commercial and government arenas, the field of data analytics is relatively new and has room for growth. The world’s data production has been estimated to double every two years and the ability to store, prioritize, analyze and share data is a key to research and development (R&D) investment.

A major focus of R&D investment is how to take high-speed data streams of both ‘structured data” (residing in a predetermined field) and “unstructured data” (not organized in a pre-defined manner). Eighty percent of data is unstructured. That means specialized optic technologies, software algorithms and innovative processes are necessary to de-clutter data and allow for distillation and sophisticated assessment. The goal of this type of technology is to develop a deployable, fully automated, real-time, secure way to collect and analyze complex streams of data.

The future of applied data analytics looms bright and the data sets of disparate information are seemingly endless. Technological R&D advances such as “machine thinking,” which will allow connected devices on the “Internet of Things” to talk to and learn from each other, will contribute immensely to the use of data analytics. These smart devices will also recognize who we are from biometric authentications, as well as augmenting our expertise. Data analytics is an exciting new frontier providing insights into the future conduct of our world. It is a science deeply rooted in our daily communications and transactions.

The splendid ambition that was the Library of Alexandria can be captured again. The alchemy of data to knowledge to learning can, and will, digitally transform our nation.

Logsdon is the senior director of public advocacy for CompTIA. In this role, he runs the association’s New and Emerging Technologies Committee (focused on the policy surrounding social, mobile, big data/data analytics, cloud, Internet of Things and smart cities). He was also the staff lead for CompTIA’s federally focused technology convergence commission, which examined the impact on the public sector when social, mobile, analytics and cloud converge. Follow him on Twitter @DJLSmartData. Brooks serves as the vice president for government relations and marketing for Sutherland Government Solutions. He is also vice chairman of the CompTIA New and Emerging Technologies Committee. Brooks served at the Department of Homeland Security as the first director of legislative affairs for the Science and Technology Directorate. He also spent six years on Capitol Hill as a senior advisor to the late Sen. Arlen Specter (Pa.). Follow him on Twitter @ChuckDBrooks and on LinkedIn.