The definition of big data refers to groups of data that are so large and unwieldy that regular database management tools have difficulty capturing, storing, sharing and managing the information.
An example of big data is the results from the use of Facebook by its over 800 million active users.
Data sets that are too large to be processed easily by standard software.
The massive amounts of data collected over time that are difficult to analyze and handle using common database management tools. The data are analyzed for marketing trends in business as well as in the fields of manufacturing, medicine and science. The types of data include business transactions, email messages, photos, surveillance videos, activity logs and unstructured text from blogs and social media, as well as the huge amounts of data that can be collected from sensors of all varieties (see machine-generated data).Big Data Storage and Big Data AnalyticsThe storage industry is continuously challenged as big data increases exponentially. While the physical storage can be enhanced with more terabyte drive arrays, the software infrastructure must be flexible enough to quickly and economically accommodate ever greater volumes of transactions and queries. See Hadoop, Spark and cloud computing.The analytical challenge is deriving meaningful information from data in petabyte and exabyte volumes. Big data analytics breaks down the data sets into smaller chunks for efficient processing and employs parallel computing to derive intelligence for effective decision-making (see MPP).Big Data vs. Market ResearchAnalyzing big data may be able to forecast trends and people's habits without the kind of explanation one would derive from traditional market research. The big data patterns show the next probable behavior of a person or market without a logical explanation as to why. Right, wrong or indifferent, big data has been one of the hottest buzzwords of the 2010s. See Petabyte Age and Google Zeitgeist.
Origin of big-data
- In 2000, economist Francis X. Diebold published the first version of a paper titled “Big Data Dynamic Factor Models for Macroeconomic Measurement and Forecasting." After being interviewed on his use of the term "big data" by NYTimes.com blogger Steve Lohr, Dieblold undertook his own investigation, in which he concluded: “The term Big Data, which spans computer science and statistics/econometrics, probably originated in the lunch-table conversations at Silicon Graphics in the mid-1990s, in which John Mashey figured prominently."
- The Wikipedia article cites several sources from 2009 having "big data" in the title, which is when the term seems to have caught on. The same two words can be attested in the 1980s and 1990s, but not in the current sense of the term.