Customer Highlight: Building Lean Analytics at MobFox

Treasure Data helps an ad tech startup stay ahead of the game with a skeleton crew.

In business, how a company manages its data can make the difference between success or failure. Nowhere is this more true than ad tech, where data practically is the business. On an ad server, every bid, impression, or error that occurs must be captured and reported to each stakeholder. Accounting must know the final bid price on each impression, in order to reconcile accounts, bill advertisers, and pay publishers. Publishers need need to know which ads were shown alongside their content, to which viewers. Advertisers need reports on the context, audience, and performance of their campaigns. All of this requires capturing and processing massive volumes of data, in near real time.

Navigating Rapid Growth

In late 2012, a lean engineering team at MobFox struggled to keep up as their mobile ad network blew past 5 billion events per day in less than 2 years. MobFox’s focus was on giving mobile publishers the highest degree of control over their ads, which put data in an incredibly central role. Creating rich reports and advanced control panels was, in fact, their primary differentiator.

This turned out to be hugely successful, leaving the tiny team scrambling to scale their infrastructure against this massive growth. They quickly hit the limit of their sharded MySQL approach, encountering numerous issues:

• Difficulty sizing servers for high peak-to-average write loads
• Huge, unwieldy connection pools to a central DB server
• High volume write loads causing buffer overruns during garbage collection

Previous Analytics Architecture

Most ad tech firms would have thrown a full data engineering team at the problem, managing complex NoSQL databases and manual ETL pipelines. But with only 2 full time engineers for the entire product, MobFox was in uncharted waters. They needed a completely managed solution with future-proof flexibility, which is exactly what they found in Treasure Data.

A Complete Solution

Within just two weeks, they were able simplify their entire architecture with Treasure Data’s full suite of managed products. Treasure Agent immediately solved all of their data collection issues, replacing the connection pool nightmare with a simple, lossless solution. Treasure Data’s schema-flexible storage let them to move away from managing MySQL altogether and allowed them to effortlessly stay on top of variable data without constantly updating ETL scripts. Treasure Data’s cloud compute platform let them create robust processing routines to clean and enrich their raw event logs in preparation for reporting. Finally, Treasure Data’s output connectors let them set up pipelines pushing processed data to all areas of their business.

New Analytics Architecture With Treasure Data

1 Trillion Events and Beyond

MobFox has been so happy with their Treasure Data solution that they’ve felt no need to change it, even as their platform has grown to over 1 Trillion events per month. Today, every part of the business relies on Treasure Data to get the data they need without hassling their engineering team. Through a web-based SQL console, customer support reps can investigate glitches or other irregular events when responding to customer complaints. The Accounting and Management teams do ad-hoc investigation of business metrics, and can easily push new insights to their BI tool for syndication. Even the engineering team enjoys the flexibility Treasure Data provides, allowing them to easily collect new data for optimization or A/B testing without corrupting the production pipeline.

Treasure Data gives MobFox an analytics program rivaling some of its largest competitors with a fraction of the headcount, and the agility they need to stay ahead in a rapidly changing industry.