9 posts tagged with "mojito"

Whilst improving Mojito's PRNG & devising an ITP2.X workaround last year we introduced a modular splitting tool in Mojito that lets users split traffic with hash functions. We're amazed by the features that hash functions enable in split testing such as:

• Deterministic results: Users will always be bucketed the same way for a given input - regardless of when or where you're making the decision from (e.g. client-side/server-side/web/app)
• Sufficiently random: Random numbers are uniformly distributed despite near-identical inputs

But hiding in plain sight was a novel ramping process that Lukas Vermeer, Booking.com's Director of Experimentation, pointed out to us. We'd not encountered it before. But now that we were using hash functions, it was possible...

Running A/B tests or experiments on the web requires injecting lots of JS and CSS into your web app to change the look and feel of the page. Reckless deployments of this code can (and sometimes does) break web applications. And when it does break, it results in failed experiments and terrible user experiences.

An example of a split testing container JS file with KBs of minified, experimental code.

But could we make experiment deployment safer and more reliable through a CI workflow? We think it's possible through how we use Bitbucket Pipelines.

The Mojito split testing framework's docs are built upon Facebook's Docosaurus.

It's been a couple of months since we announced that we'd open-sourced Mojito... and at long last you can now find all the documentation for our split testing framework in one central resource! Here, you'll find:

• Detailed documentation & API references
• Quick-start / Getting started guide (coming soon)
• Showcasing users and unique use-cases of the split testing framework (also yet to come)
• Mojito news and announcements

Issues, help & contributing#

The website code is open-source, too. So if you get stuck or want to help us improve the documentation, drop an issue on the repo and hopefully we can help you out. The site is built using Facebook's Docosaurus and allows adding articles through simple markdown - It's an easy way to build comprehensive documentation.

Anyway, we hope you find this resource useful in getting started with Mojito.

Visit Mojito.mx.

Experiments built entirely within SaaS platforms’ web interfaces often take longer and require unnecessary busy-work. This article explores the reasons we would rather build experiments in an IDE and how Mojito supports this approach, so you don’t have to touch a web interface.

There's a reason tag managers are now the de facto for tag deployment.

Before tag managers, you'd embed tags directly into your application. It could take weeks or months to deploy them inside large, monolithic apps... Meanwhile, you'd be shifting precious developer time off high-value projects. And the practice of tagging the app just added further bloat/technical-debt to your heavy codebase.

...and then tag managers became popular.

Image credit: Blastam Analytics

Now, independent of the web application code, tags could be setup, QA'd and deployed before your coffee went cold. This led to an explosion in data collection and marketing efficiency.

This efficiency is critical in the fast-paced world of experimentation...

Update: We have just launched our documentation site for Mojito here.

We're excited to open source Mojito - the experimentation stack we've used to run well over 500 experiments for Mint Metrics' clients.

It's a fully source-controlled experimentation stack for building, launching and analysing experiments from your favourite IDEs.

A better way to run experiments...#

We typically find that relying just on Optimizely, VWO or Convert.com's A/B test tracking has hidden costs:

• Restrictive analytics capabilities
• Worse site performance
• Increases your compliance obligations & compromises your data sovereignty

In our experience Analytics tools like GA and Snowplow are more trustworthy and full-featured. And, at Mint Metrics, all experiments get tracked into both GA & Snowplow for clients. We no longer use or trust SaaS testing tools' built-in trackers.

Here's how purpose-built analytics tools lifts your split testing game...

Remember the good old days of JS errors? (Image credit)

Building large, complex experiments introduces new logic, new code and sometimes new bugs. But most A/B testing tools don't perform error tracking or handling for you. So when you launch your experiment and it tanks...

...did your awesome new idea just not work? Or did bugs torpedo your idea?

Client-side A/B testing tools get criticised for loading huge chunks of JS synchronously in the head (rightfully so). Despite the speed impact, these tools deliver far more value through the experiments they deliver. And luckily, we can help manage the problem in a few ways.

Here are ways we manage container weight at Mint Metrics when managing a client-side A/B testing tool for our clients: