Every application writes logs. And while we don’t need to think about logs in every single context, like on our desktop machines, we should absolutely pay attention to logs on servers. Why? Log files give us insight into the lifecycle of our applications, help us fix problems, and enable us to collect critical business metrics.
So log files are important. Central log management is being adopted in more and more organizations these days, and the benefits are significant. So why don’t we love our application logs more? Why is application logging still such a painful topic?
It’s probably because of two main points.
First, it’s still very difficult to get good structured data out of the myriads of different log formats. We have to reach for black magic like regular expressions and abstractions like Grok to be able to extract data from plain text. This is painful!
Second, because applications only write their logs to the local file system, we have to run log collectors on every machine in order to ship the data to our central log management system. Think about the inefficiency!
Something must change. We should embrace structured logging and write our applications to support it.
At this year’s Open Source Monitoring Conference, I’m talking about Graylog Collector, a tool that can help you to collect logs from legacy applications. I will also talk about how you can improve your applications to support structured logging.
The Open Source Monitoring Conference is a great place to exchange ideas, educate yourself and to get an overview of the current monitoring space. Last year was the first time I attended the conference, and I had a really great time due to awesome speakers and attendees as well as the outstanding organization by Netways.
I look forward to seeing you at my talk. There’s also a hackathon this year after the main conference to actually get something done. Hope to see you there as well!
The Author Bernd Ahlers
Bernd Ahlers works for Graylog, Inc. since 2014 to develop the Graylog open source log management platform. Before that he spent some years on developing backend systems and implementing infrastructure automation. Bernd is a member of the Icinga Packages & Tools Team. He pretends to be a climber in his free time.