Experimentation is an inalienable part of any Ecommerce company's effort to maximize the conversion on their website. There is a constant need for improvement of the website in terms of the type of design, pricing and messaging that resonates well with incoming website visitors. To find the better version of the website on a regular basis, requires comparison of multiple website designs simultaneously and evaluating how each version performed against the cohort of incoming website visitors. But while performing this experiment, a substantial portion of the visitors are shown a relatively inferior version of the website which hits the conversion numbers resulting in a loss of revenue. The amount of time it takes to conclude an experiment is directly proportional to the number of daily website visitors, which is why Small and Medium Businesses (SMBs), especially those having low traffic on their websites are skeptical about performing experiments because the cost of experimentation is too high. Large companies such as Amazon, Flipkart, Walmart etc., conclude their experiments quickly as they have a huge volume of daily traffic. Thus the SMBs do not iterate and improve their websites at the rate of large companies. Therefore the need for a better experimentation framework that works well with a low traffic set up and ensures less revenue loss while the experiment is in progress.

In this talk, Tuhin and Abir talk about the pros and cons of existing processes and how the Multi-armed bandit algorithm can be used to solve this problem. The speakers will also showcase an open source python library they developed called pybandit, which provides an out of the box solution to implement such an experimentation processes.