IG has an on-premises Hadoop system to support its MIS and Data Science capabilities. However, with the changing landscape of technology and tools required by Data Science, there was an urgent need to revisit the Big Data capabilities of the organization, to check the pulse of the existing estate and prepare the runaway for the multi-million pound Data Science projects waiting to get started. This is where Rajib kicked off the need to clearly define the Big Data requirement of IG and propose the choice of a platform which will shape and support the Data Science capabilities of the organization.

This session is an attempt to share the journey which involved technical people from various backgrounds like Enterprise Architecture, Data Engineering, Data Science, Cloud Architecture, Platform Engineering, Big Data & Data Warehouse Expertise, multiple External Consultants and even business stakeholders. The journey took almost a year with serious challenges from various stakeholders, but the engagement ensured that the final outcome should be as inclusive as possible and with enough rationale to support the final suggestion.

Rajib believes this journey cannot be unique to IG and many other organizations might be embarking on the same journey. Your organization might well be one such.