AI Definitions: Big Data

Big Data - Data that’s too big to fit on a single server. Typically, it is unstructured and fast-moving. In contrast, small datafits on a single server, is already in structured form (rows and columns), and changes relatively infrequently. If you are working in Excel, you are doing small data. Two NASA researchers (Michael Cox and David Ellsworth) first wrote in a 1997 paper that when there’s too much information to fit into memory or local hard disks, “We call this the problem of big data.” Many companies wind up with big data, not because they need it, they just haven’t bothered to delete it. Thus, big data is sometimes defined as “when the cost of keeping data around is less than the cost of figuring out what to throw away.”    

Big Data looks to collect and manage large amounts of varied data to serve large-scale web applications and vast sensor networks. Meanwhile, data science looks to create models that capture the underlying patterns of complex systems and codify those models into working applications. Although big data and data science both offer the potential to produce value from data, the fundamental difference between them can be summarized in one statement: collecting does not mean discovering. Big data collects. Data science discovers.  

More AI definitions here.