Archive for February, 2010

Ralf: S3 logfile downloader and merger

Today we released a new gem called Ralf, it stands for Retreive Amazon LogFiles

Download the gem at or browse the source at



  • An S3 account (duh)
  • Enable logging on S3 (use cyberduck for example)


r = => '/my/config.yaml', :date => '2010-02-01')

Or run it in one go: => '/my/config.yaml', :date => '2010-02-01')


:config   a YAML config file, if none given it tries to open /etc/ralf.yaml or ~/.ralf.yaml
:date     the date to parse

:aws_access_key_id      (required in config)
:aws_secret_access_key  (required in config)
:out_path               (required in config)
:out_prefix             (optional, defaults to 's3_combined') Prefix the output file

You can ommit a configuration file when you supply the required parameters :aws_access_key_id_, :aws_secret_access_key and :out_path
They take precedence over the config file


  • There is no logrotation as we know it like on regular unix machines

More info: S3 Server Access Logging


  • CLI executable
  • Grouping/combinig on week/month


This plugin is created for for processing the logfiles generated by S3.

Authors: Leon Berenschot and K.J. Wierenga