Fluentd Meetup (and Docker Integration) !



We are happy to announce that we will be hosting our Fluentd meet up! Feel free to join us, today, to attend our presentation titled  Fluentd and Docker Infrastructure. We will talk about Fluentd, it internally, use cases and the new native integration of the Fluentd driver inside Docker made by Treasure Data.

We’ll have lots of food and drinks, you can register in the link below. Don’t miss out!

Fluentd, Docker and All That: Logging Infrastructure 2015 Edition

Thursday, Jul 23, 2015, 6:30 PM

Nest GSV
425 Broadway Street Redwood City, CA

29 Data Lovers Attending

As software becomes less monolithic and more service-oriented, log collection becomes a real problem. How can we stop hacking together brittle log parsing scripts and start building a unified logging layer?Fluentd is a data collector written in Ruby to solve this problem. Unlike other log management tools that are designed for a single backend sys…

Check out this Meetup →


Fluentd v0.12.14 has been released!


Fluentd is a high performance data collector that allows you to implement an unified logging layer. Treasure Data is happy to announce the availability of release v0.12.14. The most relevant changes are:

  • Configuration: Log unused section configuration as a warning level.
  • Configuration: Add ‘@’ prefix to the log_level this allow to keep the log_level for backward compatibility.
  • Parser: Added a new  time_key option for the RegexpParser.

Also in this release an important fix was to merge in the out_forward plugin preventing the UDP heartbeat bug. This was possible through the addition of the new dns_round_robin option. Users of the latest td-agent version are not affected by this problem.

For more technical details about this release, we invite you to visit the official Fluentd Blog post:


If you are looking for a scalable analytics backend for Fluentd, give Treasure Data a try. You can request a demo or sign up for a free trial.

Treasure Data Python Client; MySQL, PostgreSQL, and Jira connectors; and more


Treasure Data has made Python the glue for your end-to-end data analytics pipeline. Now it’s easier than ever to link your data science toolkit directly to a robust and infinitely scalable data storage and analytics pipeline in the cloud and be up and running in minutes!

Maybe you’re brand new to data science, are a manager looking for a fast cycle time, or are an experienced old hand at Data Science.  In addition to connecting directly to staples like Jupyter and Pandas, you can now pipe in data directly from MySQL, Postgre SQL and Jira, as well as engage Treasure Data Agent directly from Python.  Whatever your skill level, you can leave the analytics infrastructure to us while you focus on your data! (more…)

Open Source: Treasure Data Sponsoring FISL16


It’s well known that Treasure Data promotes open source. We have contributed to numerous open source projects and created some of our own.  Our service is built atop open source components relevant to our core business. One of our most relevant solutions is Fluentd, an open source data collector that allows you to unify your logging layer with structured logs. Fluentd has reliability built-in, a pluggable architecture, and is optimized for high performance. (more…)

Data Engineering 101: Integrating MongoDB and Treasure Data


When trying out different database or data storage backends, the choice of which to use – and for which part of your architecture – can be a tradeoff: Do I want excellent latency control, or do I need literally effortless – and potentially infinite – scalability?   Do I prefer to work with a sharding and implementation strategy in order to run lightning fast on bare metal, or can a tolerate a few extra milliseconds of latency to enjoy the benefits of automatic scaling overhead in the cloud?  Am I building a one-off app that works well as a standalone, or do I need to scale massively from the get go?  Do I prefer to ramp up my engineers to a powerful new query paradigm, leveraging Mongo’s impressive and helpful community,  or am I comfortable enough with the knowledge transfer from SQL to its emerging and popular big data counterparts such as Presto and Hive? (more…)

Data Science 101: Interactive Analysis with Jupyter, Pandas and Treasure Data


In case you were wondering, the next time you overhear a data scientist talking excitedly about “Pandas on Jupyter”, s/he’s not citing the latest 2-bit sci-fi from the orthographically challenged!

Treasure Data gives you a cloud-based analytics infrastructure accessible via SQL.  Our interactive engines like Presto give you the power to crunch billions of records with ease.   As a data scientist, you’ll still need to learn how to write basic SQL queries, as well as how to use any external tool you choose – like Excel or Tableau – for visualization.