Enterprise Customer Data Platform - Treasure Data Blog - Part 56

Treasure Data: Who We Are

Behind the TD Cloud Data Warehouse is a team of brilliant hackers and experienced software professionals. Two of our engineers have designed and built large-scale distributed systems that power one of the world’s largest e-commerce......

Log Everything as JSON. Make Your Life Easier

The Story of an Engineer. Here is an anecdote. I am sure some of you have had a similar experience. (more…)...

January 17, 2012

Enabling Facebook’s Log Infrastructure with Fluentd

Facebook uses Scribe as its core log aggregation service. The description of Github reads, “Scribe is a server for aggregating log data streamed in real time from a large number of servers.”.....

December 4, 2011

Real-Time Log Collection with Fluentd and MongoDB

For those of you who do not know what MongoDB is, it is an open-source, document-oriented database developed at 10gen, Inc. It is schema-free and uses a JSON-like format to manage semi-structured data......

November 23, 2011

Fluentd: The Missing Log Collector Software

The fundamental problem with logs is that they are usually stored in files although they are best represented as streams (by Adam Wiggins, CTO at Heroku). Traditionally, they have been dumped into text-based files and collected by rsync in......

November 21, 2011

MessagePack: The Missing Serializer

MessagePack, the efficient, blazing, fast serializer is the core of our technology. The best way to describe MessagePack is “JSON on steroids”. It supports an almost identical set of data types as JSON —Nil, Boolean, Integer, Float, String, Array, and Associative Array— but runs much faste...

56