The widespread adoption of computers, smartphones, tablets and other devices connected via robust, high-bandwidth networks is leading to enormous amounts of data being generated and stored by companies large and small. With so much information at our disposal, computer scientists are developing techniques to help us process and make sense of all this new Big Data.
You can get a better sense of where big data is going when you review the major players and gain insight into which industries might prosper and which seem doomed to fail as big data penetrates into more aspects of our society.
What Is Big Data?
You may have heard the term “big data” being bandied about and have at least a notion of what it means. Clearly, it has to do with large amounts of information. But what is big data exactly?
Big data refers to enormous sets of data that people capture and store in large servers. The information is meant to be searched, sliced and diced as sophisticated software analyzes it for patterns, clues to behavior and insights previously unattainable.
Examples of big data include recordings of every earthquake to hit a particular country or region, automated speech-to-text from every phone call made by a particular person, group or institution and every mention of a product that occurred in social media during a narrow time span.
Data science professionals set up their databases in two main types: Structured and unstructured. Structured data sets are built when you already know the parameters you want to measure and keep track of, such as vital statistics on each person in a particular organization or in a population of patients who all have the same ailment. Unstructured data is not organized, so you need powerful computers to quickly run through all the information as you process analytics.
Who Are the Primary Players in Big Data?
If you’re new to big data, a bit of an introduction to the primary players is in order. According to a recent report from TechBullion (using the latest available figures for their big data efforts), major companies doing important work in big data include these top 10 firms:
- IBM $1.37 billion
- Oracle $745 million
- Hewlett Packard Enterprise $680 million
- Palantir $672 million
- Splunk $668 million
- Accenture $480 million
- SAS Institute $480 million
- Dell $480 million
- Teradata $432 million
- Microsoft $396 million
Big Data Disruption and Industry Progress
It’s useful to consider the progress being made by companies working in big data and how big data is disrupting the status quo now.
In the food industry, big data is already leading to big changes. Disruption is occurring as companies “start with the vastly increased supply of information everywhere from the plant genome to water management, fertilization, climate, soil, machinery, and crop protection systems,” noted AgFund News.
“Add the expanding ways to get and use data both in both farming practices and advances in crop genetics,” and you can change the way we grow food and what crops we will focus on. For example, big data lets scientists develop new seed traits and public health researchers detect and track food borne illnesses. Farmers use big data to make better decisions about farm management, aiming to use analytics to guide them in precision farming.
Airbnb is another example of big data disrupting an industry. The team at Airbnb developed a smartphone app that lets people rent their home to strangers through an online platform.
Users of the service provide massive amounts of feedback, which is gold for Airbnb to data mine. “The insight gained from this feedback enables Airbnb to ensure they concentrate efforts on signing up landlords in popular destinations at peak times, and structure pricing so that the use of their global network of properties is optimized,” noted Cloud Computing News.
In retail, another industry that processes enormous amounts of information, there is plenty of progress. According to Computerworld, big data is now letting retailers “create comprehensive supplier profiles, including data from external sources – such as Dun & Bradstreet – for financial, risk or performance metrics and provide risk managers with real-time analytics dashboards.” This real-time ability to see information affecting supply chains represents an advance that can be applied to multiple industries, so its disruptive effects will change more than retail.
Which Industries Will Prosper and Which May Die Off ?
Some industries are more suited to benefiting from the possibilities of big data than others. Consider the following sectors, which analysts predict will do well thanks to big data:
Banking is poised to take off with big data. “Not only does big data assist in simplifying how a bank’s system filters required consumer information, but it also contributes to the prompt handling of large and small scale banking problems before they are revealed to clients,” noted a report from Innovation Enterprise. “Big data also helps the industry keep track of contracts, keep a clear record of credit card limits assigned per customer, and ensure no client is exceeding their limit unjustly or that a fraud is being committed.”
Healthcare is still an art as much as a science, but healthcare practitioners rely on hard numbers, from statistics covering a sub-population of patients to the lab results used to make a diagnosis. Medical practices, clinics, hospitals and other facilities routinely store massive amounts of information on each of their patients in electronic health records.
Applying analytics in big data operations to healthcare information can help physicians achieve such worthy goals as making better decisions about treating patients and predicting whether a patient will fare better with one medication or another, based on demographics and other salient information.
The insurance industry is based on actuarial tables and other intense collections of information. Applying analytics to insurance big data would enable the industry to provide better customer service in terms of matching customers with the correct policy for their specific needs.
Who is purchasing what and for how much, and how long do certain types of properties take to sell compared to others? How do ZIP codes, census information and other databases contribute to the success of real estate agents? With big data and data modeling software, realtors can make more accurate predictions about how property will move and what price ranges the market will truly bear.
Retailers need to know who their customers are and what is motivating them to buy or not buy certain items. When they collect massive amounts of purchase pattern data (such as from credit cards and customer loyalty cards) along with the tidal wave of data being generated across social media platforms, these businesses can truly put their fingers on the pulse of their customers. You can make better predictions about what types of items will sell best when and where.
Combined with GPS tracking data, retailers can use big data to determine where a customer is in real-time, such as near a brick and mortar store that wants to offer shoppers a special offer when they get closer to the establishment.
Big Data Might Spell the Demise of Some Industries
Much as some industries can be expected to prosper with big data, the rise of big data might spell the slow decline if not complete demise of others.
Just as the industrial revolution and automation caused some industries to peter out while others went on the ascent, we can expect the same dynamics to occur as big data continues to grow in influence.
For example, the legal profession and journalism may take a hit. the Economist reports that with big data, “E-discovery software can search mountains of legal documents much more quickly than human clerks or paralegals can. Some forms of journalism, such as writing market reports and sports summaries, are also being automated.”
What to Expect from Big Data in 2018
Companies of all sizes are predicted to move their big data from local servers to a cloud computing environment, noted a report from Tech Republic. One reason for the move is financial, with CFOs looking to cut down on IT expenses.
Moving to the cloud means you will no longer have to waste money on predicting how much server space and bandwidth you need for your changing population of employees. The cloud computing service provider will have its own dedicated IT team and will take care of server upgrades for you.
Organizations will also likely start working with “dark data,” a term that applies to paper documents not yet scanned into digital format, such as printouts of contracts and memos, photos and videos. Companies that digitize this as part of their big data analytics efforts can use the older dark data to get a much better historical view of themselves, detect patterns (such as in sales or customer loyalty) heretofore undiscovered.
One of the biggest things you should expect from big data this year will be the increased use of artificial intelligence to sort through and make more sense out of the massive information being gathered and generated year after year. “For the business users this would mean better access to actionable intelligence, and elimination of routine tasks that can be delegated to the bots,” noted Inside Big Data. “For users who want to stay relevant in the new economy, this would allow them transform their roles in to knowledge workers that focus on tasks that can still only be done based on the general intelligence.”
We will provide more insight into big data with fresh articles on a regular basis. Please check back for updates. We welcome feedback and look forward to your comments!