Using YoY/MoM conversion rate goals as targets can backfire

A common exercise product teams do at the end of each year is goal setting and revision. We often see conversion rate goals / objectives being set like: Increase the conversion rate from 6.7% to 7.4% When goals measure absolute…

Read More
How traffic is allocated in a partitioned ramp experiment with spillover protection

Partitioned ramps: A surprising feature of using hash functions for A/B split tests

Whilst improving Mojito’s PRNG & devising an ITP2.X workaround last year we introduced a modular splitting tool in Mojito that lets users split traffic with hash functions. We’re amazed by the features that hash functions enable in split testing such…

Read More
You can publish commits to specific environments, like staging and production. Bitbucket shows what is deployed at any given time.

Deploying A/B split tests through Bitbucket Pipelines CI

Running A/B tests or experiments on the web requires injecting lots of JS and CSS into your web app to change the look and feel of the page. Reckless deployments of this code can (and sometimes does) break web applications….

Read More

Presentation: How to avoid 5 testing pitfalls & run trustworthy experiments (Web Analytics Wednesday Melbourne, 2019-11-06)

Earlier this month, I gave a talk to the local Web Analytics Wednesday group in Melbourne on running A/B tests and trustworthy experimentation. It features some of our biggest mistakes in split testing and the simple methods we take to…

Read More Documentation for the Mojito split testing framework launched

It’s been a couple of months since we announced that we’d open-sourced Mojito… and at long last you can now find all the documentation for our split testing framework in one central resource! Here, you’ll find: Detailed documentation & API…

Read More

Why we build experiments in an IDE rather than a WYSIWYG

Experiments built entirely within SaaS platforms’ web interfaces often take longer and require unnecessary busy-work. This article explores the reasons we would rather build experiments in an IDE and how Mojito supports this approach, so you don’t have to touch…

Read More

Why an A/B testing tool should form an experiments layer over your site

There’s a reason tag managers are now the de facto for tag deployment. Before tag managers, you’d embed tags directly into your application. It could take weeks or months to deploy them inside large, monolithic apps… Meanwhile, you’d be shifting…

Read More
The Mojito logo.

Introducing Mojito: Mint Metrics’ open-source split testing tool

Update: We have just launched our documentation site for Mojito here. We’re excited to open source Mojito – the experimentation stack we’ve used to run well over 500 experiments for Mint Metrics’ clients. It’s a fully source-controlled experimentation stack for…

Read More

Track Optimizely, VWO & Mojito tests into Google Optimize

You’ve probably audited your Google Analytics setup and validated the data roughly matches data in your CRM etc (bonus points if you perform this QA process regularly). But who regularly audits the data quality of Optimizely / VWO /…

Read More

Why purpose-built analytics tools beat Optimizely / VWO’s A/B test tracking

We typically find that relying just on Optimizely, VWO or’s A/B test tracking has numerous hidden costs: Restrictive analytics capabilities Worse site performance Increases your compliance obligations & compromises your data sovereignty In our experience Analytics tools like GA…

Read More