Managing Big Data: Assembling Your Big Data Toolkit & Team

By Dan Sytsma, President, Melillo Consulting

Major corporations have more data than ever at their disposal. How they analyze that information to make informed business decisions requires skill and specific tools. A combination of strong technical skills with a strong toolkit is necessary to understand IT requirements and to be able to blend them with business requirements. Those are the talented individuals – either in-house or external consultants – who can step back and ask the proper questions to really understand what the business challenges are, and how to solve them.

Read Part One of this Post: Managing Big Data: The Challenge of Creating Business Value

Some companies might prefer to train their employees to work with big data. The challenge that comes with that strategy is that often, everyone is already too busy with the day-to-day activity of keeping their operations up and running, as well as with the swelling backlog of projects already on the books. It is difficult for employees to find time to invest in the new skills that are required, especially in the area of Splunk, a popular analytics tool that we employ.

Many time-strapped companies find that ultimately they can’t make that kind of an investment, so they look for an organization such as ours, which has provided many big data implementations. It’s quite likely we’ve created similar solutions for customers of their size and complexity. We can bring that knowledge and those best practices to help a customer with the knowledge transfer needed to ensure strategies are implemented efficiently and correctly.

Another inherent challenge when creating a Big Data analytics team is staff attrition. Every company faces retainment issues. Once an associate is trained, he or she instantly becomes more marketable. It becomes increasingly difficult to keep top performers satisfied when the competition starts dangling incentives at the employees you’ve trained. In data analytics, the competition for talent is immense: According to the MIT Sloan Management Review, 40 percent of companies surveyed are struggling to find and retain the data analytics talent.

All of that has compounded the current shortage of big data talent, including higher-level data scientists, who really understand quantitative analysis and data. Those people exist, but they're also at a premium. There aren’t enough in the industry right now, but we have a whole new generation\who have grown up with data being omnipresent. Millennials are contemporary with instantaneous analysis and with the instant gratification of asking questions. They also tend to have excellent communication skills, so they can translate back to customers what the data really means. The way that they view data and filter it to make better decisions is going to make an impact in the future.

Customers should assess what they want to learn from a Big Data strategy, really getting into the roots of problems, determining where they're coming from and then identifying the deliverables from a Big Data standpoint.

For example, we're working with a big online radio organization in New York, which is experiencing a huge challenge with its data. It has millions of subscribers and a significant amount of data from a customer support issue. Yet, it has no way to harness all that data to learn what the trends are or what problems are recurring. Learning those answers will help the company offer a better customer experience and higher levels of customer satisfaction, which, hopefully, will result in increased product loyalty.

At Melillo, we focus on the assessment methodology. We don't go in with any perceived convictions and take the time to listen to the customers, understand what their challenges are with Big Data, what they're trying to solve and where those sources of data originate. We then create a methodology to capture that information, solve the business issues and collaborate with the customer to define the deliverables for the project. Often, part of the issue is that the datasets are so large and complex that it’s impossible to process using traditional tools.

Customers tend at first to look at the traditional tool sets that they’ve already invested in -- but quickly look for other, more robust, tools that are available in the marketplace. Some are adamant about going to open source because that gives them the best flexibility and the most options: They can do a lot of good customization and have more control. Others are moving toward tools that are becoming the standards in the marketplace, such as Splunk.

The intersection of the business and an IT organization coming together is critical to manage big data in the enterprise. The excitement level is high around big data and tools such as Splunk that really help customers solve issues. It's a great time to be in these environments with customers and users of this technology: They really know that they're making a difference.

Want to discuss how to incorporate Big Data in your organization?

Contact a Melillo Consultant Today!

Comments are closed