This lesson's interactive features are locked, please to keep using them

Log Rotation

Click to hide video

When logging to a file, it's important to rotate it occasionally. Otherwise, it grows forever and eventually fills your disk. Log rotation needs to:

Stop writing to the current log file.
Rename the existing file, e.g. from linko.log to linko.log.0.
Create a new log file and start writing to it instead.

We typically keep a set number of old log files, renaming them all in sequence. For example:

linko.log is the active file, currently being written to
linko.log.0 is the previous log file
linko.log.1 is an older one
linko.log.2 is even older
linko.log.99 is the oldest log file (assuming we keep 100 log files)

It's a bit trickier than it sounds. We need to ensure that:

The rotation happens atomically (as one uninterrupted operation), so that we don't lose any log entries during the rotation process.
Old log files are compressed to preserve disk space.
Rotation takes place on a regular schedule (perhaps daily), or when the log file reaches a certain size.

Lumberjack

Fortunately, Nate Finch created gopkg.in/natefinch/lumberjack.v2, which handles this complexity across operating systems. We just need a few changes to log initialization:

logger := &lumberjack.Logger{
	Filename:   logFile,
	MaxSize:    1,
	MaxAge:     28,
	MaxBackups: 10,
	LocalTime:  false,
	Compress:   true,
}
handlers = append(handlers, slog.NewJSONHandler(logger, &slog.HandlerOptions{
	ReplaceAttr: replaceAttr,
}))

It will use gzip compression to compress old log files, and will rotate the logs when they reach 1 megabyte in size.

You Won't Usually Rotate Logs in Your Application Code

Log rotation is usually not handled by application servers themselves. Servers focus on business logic. Rotation is often handled by the infrastructure running the app, such as:

A container orchestrator (like Kubernetes)
A cloud provider's managed service (like AWS, GCP, or Azure)
A third party logging service (like Loggly, Datadog, or Sentry)

We're gonna make it our application's responsibility in this course simply for the learning experience!

Assignment

Switch file logging to a rotating writer.

Add the gopkg.in/natefinch/lumberjack.v2 package to your module.
Replace the bufio.NewWriter / os.OpenFile setup in your initializeLogger function with a lumberjack.Logger as shown above. Use the configuration from the example.
Update the close function to call logger.Close() on the lumberjack logger instead of flushing and closing the file manually.
Start your server with LINKO_LOG_FILE=linko.access.log set.

Copy this script into the top level of your project as spamhomepage.sh and run it to generate enough logs to trigger rotation:

#!/usr/bin/env bash

set -euo pipefail
for i in {1..5000}; do
  curl -sS "http://localhost:8899" > /dev/null
  if (( i % 100 == 0 )); then
    echo "Completed $i requests"
  fi
done

After running the script, you should have:

An active linko.access.log file
At least one rotated linko.access*.gz file

With your server running, run and submit the CLI tests from the root of the Linko repo.