Instruction 05-logs-and-journalctl: Reading

Linux troubleshooting depends on evidence, and the system journal is one of the most important evidence sources on a systemd-based host. systemd-journald receives records from several sources, including service standard output and standard error, native journal clients, kernel messages, and compatible logging interfaces. It stores more than a line of text: a record can also contain timestamps, process identifiers, user identifiers, executable paths, systemd unit names, boot identifiers, transport information, and priority. journalctl is the query tool for this structured data.

Running journalctl without filters may produce far more information than needed. A good investigation narrows scope deliberately. Start with the relevant boot by using -b. Restrict the source with -u followed by a unit name. Restrict time with --since and --until. Restrict severity with -p. For example, -p warning includes warning and all more severe priorities, not only records labeled warning. The standard numeric priority order is 0 emergency, 1 alert, 2 critical, 3 error, 4 warning, 5 notice, 6 informational, and 7 debug. Lower numbers represent greater severity.

Time context matters. A failure seen today may belong to the current boot, while a startup failure from yesterday may be stored under an earlier boot. journalctl --list-boots shows known boots when the required data exists. The current boot can be selected with -b 0 or simply -b. Previous boots can commonly be addressed with negative offsets such as -b -1, but previous records will not be available if the host only keeps a volatile journal or if retention has removed them.

Output format should match the task. Human-readable formats such as short-iso make chronological review convenient. Verbose output exposes all fields associated with each selected record. JSON output is useful for scripts because each record is represented as structured data rather than an ambiguous display line. Exported records can contain hostnames, usernames, process arguments, internal addresses, authentication events, application data, or secrets accidentally written by software. Treat exported files as sensitive evidence and use restrictive permissions.

The journal is not automatically proof of root cause. A warning near a failure may be unrelated, an application may omit useful details, and clocks can be inaccurate. Correlate timestamps, units, process identifiers, boot IDs, and surrounding events. Begin with a narrow hypothesis, broaden the time window if necessary, and record the exact query used. Avoid jumping directly to destructive maintenance operations. Reading, filtering, and exporting evidence should come before retention changes or journal cleanup.

Access differs among distributions. An unprivileged user may see only records associated with that user, while administrative access may expose the system journal. An empty result can therefore mean that no records matched, that old records were not retained, or that the current account cannot read them. Distinguish these possibilities before concluding that a service produced no logs.