Designing the Future: Using Big Data in the Enterprise

By Mark Melillo & Thomas Palmieri

Imagine if you could run your business using relevant, real-time data based on customer interactions. Every decision would be backed by information gathered by your business or those you benchmark against, and then carefully examined. That is the promise of Big Data. It’s the type of number crunching that helps predict the future by analyzing the past, for businesses, individuals and communities.

However, while collecting and storing that information has become easier than ever, becoming a data-driven enterprise is not easy task. It takes time, effort, great skills and dedicated employees to create a data-driven culture where using data and analytics to drive decision-making becomes a regular part of the daily business routine — from the shop floor up to the executive suite. In today’s business world, data analytics can be performed at speeds unimaginable just a short time ago. That speed is a key characteristic in building a data-driven business strategy.

Every computer and every sensor in a building or in a community is capturing data. The challenge is to take all those sources and analyze their data. In the old days, it could take a month to program how to extract a specific data source and create something intelligent from the information captured. Today, tools such as Splunk and others, that information can be immediately made available to the enterprise.

Every data source has an application that can decipher it. We can obtain that information very, very quickly. We've gotten very good at putting together pattern-matching algorithms, which, in a sea of data, can identify trends or single out anomalies. Machine learning assists in this task in two fundamental ways: supervised and unsupervised.

"Supervised" means we spend considerable time teaching the machine in order for its understanding of the world, so to speak, to improve. "Unsupervised" means we leave the machine-learning algorithms to themselves to uncover what it believes are anomalous events that it thinks represent normalcy. We use both methods all of the time, and they're now being focused on operational elements of IT infrastructure so that fewer humans need to be involved in deciding whether something is wrong and worthy of someone's attention.

Designing machines this way can help predict the future. Increasingly, that's been employed by various businesses. For example, in retail much of the guesswork involved in stocking shelves or responding to trends has been eliminated because point of sale systems (POS) connect with sensors embedded in products — and sometimes even the store shelves — to generate data that helps stores predict demand and order accordingly.

However, machines can also help warn us that a wanted criminal has entered a stadium or predict serious issues, such as an impending natural disaster or that a vehicle is about to have a critical problem. We've had the secret desire for machines to help us predict what's in store for us, prepare us for impending doom or sort through the realities of life and tap us on the shoulder with a polite, “Excuse me, but you're about to run into a problem, and here's what you should do to prevent it.”


We've spent decades trying to create machines that have the ability to learn from the world around us to understand what normal conditions are — just so we can identify when something has gone awry or is about to skid off the tracks. That's what will allow us to act to prevent it. In the future, machines might even to take the action themselves to avoid and remediate the problem for us. Machine learning and artificial intelligence are today's answers to that secret desire and predictive analytics is one effective tool to make those desires a reality.

Want to discuss how to incorporate Big Data & IoT in your organization?

Contact a Melillo Consultant Today!

Comments are closed