Treasure Data Blog - Part 56

January 17, 2012

Enabling Facebook’s Log Infrastructure with Fluentd

Facebook uses Scribe as its core log aggregation service. The description of Github reads, “Scribe is a server for aggregating log data streamed in real time from a large number of servers.”.....

December 4, 2011

Real-Time Log Collection with Fluentd and MongoDB

For those of you who do not know what MongoDB is, it is an open-source, document-oriented database developed at 10gen, Inc. It is schema-free and uses a JSON-like format to manage semi-structured data......

November 23, 2011

Fluentd: The Missing Log Collector Software

The fundamental problem with logs is that they are usually stored in files although they are best represented as streams (by Adam Wiggins, CTO at Heroku). Traditionally, they have been dumped into text-based files and collected by rsync in......

November 21, 2011

MessagePack: The Missing Serializer

MessagePack, the efficient, blazing, fast serializer is the core of our technology. The best way to describe MessagePack is “JSON on steroids”. It supports an almost identical set of data types as JSON —Nil, Boolean, Integer, Float, String, Array, and Associative Array— but runs much faste...

56

Forrester Wave Real-Time
Interaction Management See full report