journalctl Explained: How to Read Linux Service Logs and Find Errors

A service has failed, but systemctl status only shows a short summary. The next useful question is not “Which restart command should I try?” It is “What did the service report before it failed?” On a systemd-based Linux server, journalctl is an essential tool for answering that question.

This lesson follows managing services with systemctl. We will build a small, repeatable investigation around one service, one boot, and a relevant time window.

🎯 What You’ll Learn

• How the journal differs from a plain text log file.
• How to filter by unit, boot, and time.
• How to follow new messages while reproducing a problem.
• Why missing logs do not prove that nothing happened.
• How to turn a failure message into a focused next check.

📚 The Journal, journald, and journalctl

Keep the three names separate: systemd-journald collects events, the journal stores them with metadata, and journalctl queries them. A record can include a timestamp, priority, process identity, unit association, and message. Filtering on these fields is more precise than searching every file for a word that looks like an error.

Many service messages reach the journal through standard output or standard error. Applications may also write directly to their own files, send logs elsewhere, or use a container logging system. A quiet journal does not prove a quiet application.

You can read entries your account is allowed to access. The examples below use sudo for the system journal; use it only when your account is authorized. On a system with appropriate journal access, it may be unnecessary.

🔍 Start with One Service and One Boot

systemctl status example.service --no-pager
sudo journalctl -u example.service -b -n 50 --no-pager

-u selects the unit, -b limits the query to the current boot, and -n 50 selects recent entries. --no-pager prints the result directly. Without it, press q to leave the pager.

First identify the exact service name using systemctl list-unit-files --type=service. A familiar application name is not always its unit name. If the output is empty, verify the unit, time range, and access before drawing conclusions.

🕒 Narrow the Time Window

Suppose an application began failing shortly after a configuration change. Search around that moment rather than collecting days of unrelated messages:

sudo journalctl -u example.service --since "-30 min" --no-pager

For a bounded example window, use both endpoints. These dates are illustrative; replace them with the incident time on your server:

sudo journalctl -u example.service \
  --since "2026-09-04 14:00:00" \
  --until "2026-09-04 14:15:00" --no-pager

Check timezone assumptions when comparing browser errors, monitoring alerts, and server timestamps. Two reports can describe the same event using different clocks. Record the date and timezone in your incident notes.

🔁 Look at a Previous Boot

sudo journalctl --list-boots
sudo journalctl -b -1 -u example.service --no-pager

The first command shows retained boots. The second selects the preceding boot relative to the latest retained boot in the normal local journal. Older events are available only if they were stored and have not been removed by retention.

Journal storage can be volatile under /run/log/journal or persistent under /var/log/journal. With Storage=auto, persistent storage depends on the persistent journal directory being present. Distribution configuration can change the defaults. See journald storage and retention settings.

If a reboot erased volatile logs, a query cannot reconstruct them. Check whether central logging, application files, or monitoring captured the missing period. Decide retention requirements before the next incident, rather than changing logging policy during a read-only investigation.

📡 Follow New Events

sudo journalctl -u example.service -f

Keep this running in one terminal while making a harmless test request from another. Press Ctrl+C to stop following; that exits the viewer, not the service.

For example, if your own test website returns an error, note the time, make a single request, and look for a corresponding entry. Do not generate high-volume traffic just to force messages. If nothing appears, the web server may write request or error logs to its own configured files.

🚦 Filter by Priority, Then Restore Context

sudo journalctl -u example.service -b -p warning --no-pager

This includes warning and more severe priorities. It is useful for triage, but lower-priority messages may explain the sequence leading to failure. Revisit the surrounding time window without the priority filter before deciding on a cause. Exact filter behavior is documented in the journalctl manual.

🧩 Read the Cause Before the Final Failure

Imagine this simplified, illustrative sequence:

app: cannot open configuration file: Permission denied
app: initialization failed
systemd: example.service: Failed with result 'exit-code'

The last line reports the outcome. The first line provides a concrete lead. Investigate the named file, its parent-directory permissions, and the account running the service. Do not respond by making files world-writable: that changes security without establishing the required access.

These are investigation prompts, not universal diagnoses. Confirm the explanation with another relevant check before changing anything. For network-related failures, revisit TCP and UDP ports and DNS resolution.

🧪 Mini Lab: Build an Evidence Trail

1. Choose an installed service and read its status.
2. Read its latest 50 journal entries from the current boot.
3. Choose one message and identify its time, unit, and reported action.
4. Expand the surrounding time window to understand what came before it.
5. Write down one hypothesis and one check that could confirm or reject it.

You do not need to break a service to practice. Normal startup messages are enough to learn the sequence. If you use a lab VM with an intentional failure, preserve the original message before fixing it.

⚠️ Common Mistakes

Reading only the final red line. Look earlier for the operation that failed.

Searching all history without a time boundary. An old failure can distract from a new, unrelated incident.

Assuming no entries means no problem. Check access, retention, filters, and the application’s logging destination.

Sharing raw logs publicly. Logs may contain tokens, user information, request parameters, and internal addresses. Share a minimal relevant excerpt with secrets removed.

Deleting logs to fix the service. Removing evidence rarely fixes the application’s underlying problem.

✅ Knowledge Check

1. What do -u and -b select?
2. Why might previous-boot entries be unavailable?
3. Does Ctrl+C during journalctl -f stop the service?
4. Why look before “Failed with result”?

Review the learning path: Linux services → systemd units → systemctl commands → journalctl.