For all kinds of organizations, whether large multi-national enterprises or small businesses, developing a big data strategy is a difficult and time-consuming exercise. In fact, big data projects can take up to 18 months to finish. While a few within an organization may be very well aware of what Big Data is and what the possibilities of Big Data are, not everyone else, including the decision-makers, are aware of this. Developing a proof of concept (PoC) is a right approach to begin with and develop a business case. This can help organizations to answer questions like where to start, departments to be involved, functional areas to be addressed, and what will be the return on the required investment. All of these aspects should be involved in your big data business case.
Buying Business Intelligence (BI) and Analytics solutions have modified dramatically within the past few years with the advent of emerging technologies and the ever increasing sources of data. Historically, vendors didn’t supply a Proof-Of-Concept (POC) stage throughout the buying process and if a vendor did provide one, it’d typically take months to line up and costed a fortune to launch.
Once the context is set, it is important to create the right objectives of the Proof of Concept. A Proof of Concept is not just about financial objectives, but also about the learning experience for the organization. Big Data requires a different way of working and a different culture. It not only involves new hardware and software to use but access to real-time insights into what’s happening within an organization. Over-confidence can backfire when the Proof of Concept does not achieve its set objectives and you cannot move ahead with more Big Data projects. For the buyer, the misalignment of interests often results in a disappointing PoC and a high cost. To avoid this, here are some tips.
How to conduct a successful Big Data Proof of Concept–
- Use your own data – A PoC built on sample data does nothing. A lot of vendors try to reduce the number of data sources or the volume of data for a PoC. However, all organizations have their unique demands and challenges. For a PoC to succeed, using own real data is the best way to prepare a business case and chalk the plan ahead.
- Involve Big Data distribution vendor at the POC conceptualization stage – Cloudera, DataStax, Hortonworks, IBM, MapR, Google – whatever distribution of Hadoop / Big Data you are choosing for the POC, it is a must to involve these companies from the conceptualization stage
- Pretty ain’t everything– Visualizing data is important but most PoCs tend to focus on a few static but beautifully built dashboards. More often than not, these are sample dashboards which can be cooked easily by a vendor using a multitude of visualization software available. The real challenge is to have self-service, easily configurable customized dashboards on your own data so that one does not spend a lot of time and money later on this aspect.
- Define governance – Big Data projects can turn out to be one of the most multi-dimensional endeavors in the organization. There will be a constant need for business users to comment on whether additional investment in Big Data is resulting in multifold increase in decision making capability. The list can go on and on. All these decisions need to be orchestrated from an organizational perspective by a committee of senior executives. Having a ‘Big Data governance council’ is a must.
- Proving value– For your POC to succeed in six weeks, you need to translate expectations of success into clear metrics for success by listing qualitative and quantitative measures
- Scalability– The PoC should be able to address future requirements with minimal effort and not just some old reporting need. BI requirements tend to be highly dynamic because businesses change all the time and business users are continually refining and adjusting their requirements. Today’s reporting needs will look very different in a year from now, and today’s analysis will likely be relevant for only a short period of time before becoming obsolete.
- Involve corporate IT– In order to reduce lead time and dependencies on corporate IT, most business users dream of an analytics solution that can be a self-service tool without the need for intervention from IT. However, it is still highly recommended that you consult your organization’s IT professionals regarding topics with which they are more familiar: scalability, integration cycles and so forth. They can help to achieve a better architecture with their inherent know-how of company’s IT landscape and pain areas.
What Components Should You Consider in Your Big Data PoC?
- Big Data Storage and Processing
- Real-Time Ingestion
- Data Virtualization and Federation
- BI, Reporting and Visualization
- Analytics
- ETL / ELT – Data Integration
- Data Discovery and Exploration
- Data Governance
CloudMoyo has delivered successful Business Intelligence & Analytics projects for its clients across multiple industries such as healthcare, transportation, pharma, retail. A lot of this success can be attributed to a thorough assessment of client landscape followed by a proof of concept on real live client data. Most of these clients were able to pursue their big data projects after a successful PoC. With its expertise in deploying cloud based analytical solutions, CloudMoyo is the right partner for you to engage for your Big Data proof of concept. Book your 5-day Azure Assessment workshop now!
