Small, simple and secure way of sending Docker container logs to logentries

Deprecation notice

Heka project has been marked deprecated, which makes these instructions obsolete.

tl;dr;

Send all container logs on the host to logentries log with the token :

$ docker run -d -v /var/lib/docker:/var/lib/docker:ro tray/heka-logentries:latest

For the curious, head over to our GitHub repo and check the content.

How Stuff Works

To keep things relatively small we decided to use progrium/busybox image as our base. This gave us a starting size of just 5MB. We could use the official busybox image which is half the size, but in the end it wouldn't make much of a difference.

Heka adds another 37MB when extracted, but it's well worth it. We were already using it for a similar task and based on the positive experience decided to stick with it.

After configuring Heka to watch container logs and send data to logentries using TCP with TLS we were almost done.

Sending logs to logentries requires user to create a log which is defined by a token. This token must be present in every log line sent, otherwise logentries will reject it.

Deconstruction of the JSON logs generated by Docker and inserting the token is done by a simple script written in Lua, which Heka supports out of the box. Using built in JSON decoder, all we had to do was:

local ok, payload = pcall(cjson.decode, read_message("Payload")) if not ok then return -1 end

inject_message(token .. " " .. payload.time .. " " .. read_message("Hostname") .. " " .. payload.log)

On top of that, there's a small shell script (sed wrapper actually) set as an entry point that replaces a token placeholder in Heka's configuration with the one provided by user.

And that's it!

Drawbacks

Decision to use progrium/busybox as a base image wasn't the smartest one due to the scary notice about it probably being deprecated in the near future.

Heka does produce a small amount of logs when starting and those do end up in logentries, but it's only a couple of lines so not too bad.

Feedback

We would love to hear your thoughts. Please leave a comment here, on the Docker registry or GitHub.

Cheers!