Log Tailing and Rotation

Log tailing reads newly appended content from a file. A restart checkpoint must identify the file generation as well as a position. A path such as app.log can refer to a different file after rotation, and a byte offset is not the same as a count of characters in UTF-8.

Rename-and-create rotation moves the old name and creates a new file. The application may still write through its old open handle, so a collector needs a policy for draining that generation. Copy-and-truncate can keep the inode while shortening the file and can lose writes during copying. Test the actual policy, including collector downtime and partial final lines.

Saving a read position before the record becomes recoverable can lose it after a crash. Saving progress after delivery may replay it. Coordinate checkpoints with recoverable input or queue state and retry-safe output. File identity rules are tool-specific; inode reuse, changed devices, and deleted rotated files are not solved by storing a path and offset alone.

Reference: Log and telemetry collection lab.