You have a program that should start automatically, keep running without an open terminal, restart after an unexpected failure, and produce logs you can inspect. Starting it with nohup, screen, or a shell background operator may keep a process alive temporarily, but it does not create a properly managed Linux service.
On a systemd-based distribution, the normal solution is a service unit. A unit file tells systemd what to start, which account should run it, where it should run, how failures should be handled, and whether the service can be attached to the normal boot process.
This guide continues the Servers Academy learning path: Linux services and daemons → systemd units and targets → systemctl commands → journalctl logs.
Create a file ending in .service under /etc/systemd/system/, define its [Unit], [Service], and [Install] sections, verify it with systemd-analyze verify, and run systemctl daemon-reload. Start it with systemctl start, inspect it with systemctl status and journalctl, and enable it only after the service works correctly.
- 🎯 What You’ll Learn
- 🧩 What a Service Unit Actually Defines
- 📄 A Minimal but Responsible Example
- 1. Create the Program
- 2. Create a Dedicated Service Account
- 3. Write the Unit File
- 🔍 Understanding the Most Important Directives
- ExecStart
- WorkingDirectory
- Environment
- Restart
- 4. Verify the Unit Before Loading It
- 5. Reload systemd and Start the Service
- 6. Verify Restart Behavior
- 7. Enable the Service at Boot
- 🌐 Does the Service Need the Network?
- 🛠️ How to Change the Service Later
- 🧪 Troubleshooting Checklist
- ⚠️ Common Mistakes
- 🧹 Remove the Lab Service
- ✅ Knowledge Check
🎯 What You’ll Learn
- What each section of a service unit controls.
- Why
ExecStartshould use an absolute executable path. - How to run a service under a dedicated, unprivileged account.
- How restart policies behave.
- How to validate, start, test, enable, and troubleshoot a new unit.
🧩 What a Service Unit Actually Defines
A service unit is configuration for the systemd service manager. It is not the program itself. The file describes how systemd should manage one workload.
| Section | Purpose | Typical settings |
|---|---|---|
[Unit] | Description and relationships with other units | Description=, After=, Wants= |
[Service] | How the workload runs | Type=, User=, WorkingDirectory=, ExecStart=, Restart= |
[Install] | How enablement attaches the unit to another unit | WantedBy= |
The common location for an administrator-created system service is:
/etc/systemd/system/example.service
Package-provided units normally live in distribution-managed directories such as /usr/lib/systemd/system/ or /lib/systemd/system/. Do not edit vendor files directly. Package updates can replace them. Use a local unit under /etc/systemd/system/ for your own service, or a drop-in override when customizing an installed service.
📄 A Minimal but Responsible Example
The lab below creates a small timestamp service named academy-clock.service. It writes one message every 30 seconds to standard output, which systemd normally connects to the journal.
Use a disposable VM or lab server. The commands create a script, a system account, and a system service. Do not reuse the names if they already exist. Check first:
getent passwd academy-clock
systemctl status academy-clock.service --no-pager
A “not found” result is expected before the lab. If either object already exists, choose different names or inspect the existing configuration instead of overwriting it.
1. Create the Program
Create a simple executable script:
sudo tee /usr/local/bin/academy-clock >/dev/null <<'EOF'
#!/bin/sh
while true; do
printf 'academy-clock: alive at %s\n' "$(date --iso-8601=seconds)"
sleep 30
done
EOF
sudo chmod 0755 /usr/local/bin/academy-clock
Test the program before involving systemd:
/usr/local/bin/academy-clock
Wait for one message, then press Ctrl+C. If the program cannot run interactively, a service unit will not repair it.
2. Create a Dedicated Service Account
A background service rarely needs unrestricted root privileges. Create a system account without a home directory or interactive login:
sudo useradd --system \
--no-create-home \
--shell /usr/sbin/nologin \
academy-clock
The location of nologin varies. Check it first with command -v nologin and use the returned absolute path. Some distributions also provide higher-level account-management tools.
Confirm that the service account can execute the program:
sudo -u academy-clock /usr/local/bin/academy-clock
Stop it with Ctrl+C after the first message.
3. Write the Unit File
sudo tee /etc/systemd/system/academy-clock.service >/dev/null <<'EOF'
[Unit]
Description=Servers Academy clock service
[Service]
Type=simple
User=academy-clock
Group=academy-clock
ExecStart=/usr/local/bin/academy-clock
Restart=on-failure
RestartSec=5s
NoNewPrivileges=true
PrivateTmp=true
[Install]
WantedBy=multi-user.target
EOF
This unit is intentionally small. Each line has a specific job:
Description=gives administrators a readable explanation.Type=simpletells systemd to treat theExecStartprocess as the main service process. This is also the default when no other type-specific settings imply a different type.User=andGroup=remove unnecessary root privileges.ExecStart=uses an absolute path to the executable.Restart=on-failurerequests a restart after an unexpected failure, but not after an ordinary clean stop.RestartSec=5savoids an immediate tight restart loop.NoNewPrivileges=trueprevents the service and its children from gaining privileges through execution.PrivateTmp=truegives the service a private view of temporary directories.WantedBy=multi-user.targetsupplies the enablement relationship used bysystemctl enable.
A real application may need a working directory, environment file, writable state directory, network readiness, capabilities, or different shutdown behavior. Begin with the program’s documented requirements. Every permission and dependency should have a reason.
🔍 Understanding the Most Important Directives
ExecStart
ExecStart= defines the command systemd starts. It is not interpreted like a normal interactive shell command line. Shell operators such as pipes, output redirection, wildcard expansion, and && are not automatically available.
If shell behavior is genuinely required, call a shell explicitly:
ExecStart=/bin/sh -c 'exec /absolute/path/program | /absolute/path/filter'
For most services, a wrapper script is easier to test and maintain. Do not add & to the command. With Type=simple, the main process should remain in the foreground so systemd can supervise it.
WorkingDirectory
Use WorkingDirectory= when the application expects to start from a particular directory:
WorkingDirectory=/opt/my-application
Use an absolute path and ensure the service account can traverse the parent directories. A working directory does not grant permission to read or write files.
Environment
Services do not inherit the same interactive environment as your shell. If a program works in your terminal but fails as a service, compare its user, working directory, executable path, and required environment.
Environment=APP_MODE=production
EnvironmentFile=/etc/my-application/environment
Do not place passwords, API tokens, or private keys directly in a world-readable unit file. Unit configuration and environment values can be visible through administrative interfaces. Use the application’s supported secret-management mechanism and restrict access appropriately.
Restart
A restart policy improves recovery from some failures; it does not make a broken service healthy. A bad path, invalid configuration, or missing dependency can cause repeated failures until systemd applies its start-rate limit.
| Setting | General behavior |
|---|---|
Restart=no | Do not automatically restart; this is the default. |
Restart=on-failure | Restart after unsuccessful termination and selected failure conditions. |
Restart=always | Restart after clean or failed termination, except when the stop is caused by an equivalent systemd stop operation. |
For long-running services, on-failure is often a useful starting point. One-shot jobs have different semantics and should not be designed by copying a daemon unit unchanged. Consult the systemd.service documentation for the exact restart matrix and service types.
4. Verify the Unit Before Loading It
sudo systemd-analyze verify /etc/systemd/system/academy-clock.service
No output commonly means that the verifier found no diagnostics. A successful verification cannot prove that the application will work: files may be missing at runtime, permissions may be wrong, or external dependencies may be unavailable.
You can also inspect security-related properties:
systemd-analyze security academy-clock.service
Treat the score and recommendations as review aids. Stronger sandboxing settings can break an application that legitimately needs access to specific resources. Add restrictions deliberately and test them.
5. Reload systemd and Start the Service
After creating or changing a unit file, tell the manager to reread unit definitions:
sudo systemctl daemon-reload
This does not restart the service. Start the new unit separately:
sudo systemctl start academy-clock.service
Check the manager’s view:
systemctl status academy-clock.service --no-pager
systemctl is-active academy-clock.service
Then read the messages produced by the program:
sudo journalctl -u academy-clock.service -b -n 20 --no-pager
You should see timestamp messages from the current boot. If the unit failed, preserve the error before editing or restarting it.
6. Verify Restart Behavior
The safest basic test is to inspect the configured policy:
systemctl show academy-clock.service \
-p Restart -p RestartUSec -p NRestarts
On an isolated lab machine, you can test an unexpected process failure by identifying the main PID and sending it a normal termination signal:
systemctl show academy-clock.service -p MainPID
sudo kill -TERM MAIN_PID
Replace MAIN_PID with the numeric value you just inspected. Do this only for this disposable lab service. After at least five seconds, verify the new process and journal:
systemctl status academy-clock.service --no-pager
systemctl show academy-clock.service -p MainPID -p NRestarts
sudo journalctl -u academy-clock.service -b -n 30 --no-pager
A manual systemctl stop is different from an unexpected process failure. It should leave this service stopped instead of fighting the administrator’s request.
7. Enable the Service at Boot
Enable the unit only after starting and testing it successfully:
sudo systemctl enable academy-clock.service
systemctl is-enabled academy-clock.service
Enablement normally creates symbolic links based on the [Install] section. It does not mean the service is healthy, and it does not necessarily start an inactive service. The combined command systemctl enable --now is useful when you intentionally want both actions, but keeping them separate during initial testing makes each step easier to verify.
🌐 Does the Service Need the Network?
Do not add network dependencies merely because the application eventually uses a network connection. Many network clients can start early and retry later.
After=network.target provides ordering relative to the network-management stack; it does not guarantee internet access or a reachable remote API. A service that truly must wait for configured network connectivity may use:
[Unit]
Wants=network-online.target
After=network-online.target
This works only when the appropriate wait-online service is enabled and correctly reflects the system’s network manager. It can also slow startup. Even then, “online” does not prove that DNS, a remote database, or an external API is reachable. Applications still need timeouts and retry logic.
🛠️ How to Change the Service Later
For your own unit under /etc/systemd/system/, edit the file, verify it, reload manager configuration, and restart only when the change requires it:
sudo systemd-analyze verify /etc/systemd/system/academy-clock.service
sudo systemctl daemon-reload
sudo systemctl restart academy-clock.service
systemctl status academy-clock.service --no-pager
sudo journalctl -u academy-clock.service -b -n 30 --no-pager
daemon-reload rereads unit files. reload academy-clock.service would ask the application itself to reload its own configuration, but our example program does not implement that feature. These actions are not interchangeable.
For a package-provided service, create a drop-in instead of modifying the vendor file:
sudo systemctl edit example.service
Inspect the effective result with:
systemctl cat example.service
🧪 Troubleshooting Checklist
- Verify syntax:
systemd-analyze verify. - Confirm the file was loaded:
systemctl cat academy-clock.service. - Read the full status:
systemctl status. - Read the journal: filter by unit and current boot.
- Test as the service user: reproduce the command with the same account and paths.
- Inspect permissions: check every required file and parent directory.
- Confirm the environment: do not assume the service receives your shell’s variables or PATH.
- Check the application itself: a running process does not prove that its port, API, or job works.
⚠️ Common Mistakes
Using a relative executable path. Use an absolute path in ExecStart and verify it exists.
Running everything as root. Give the service only the identity and access it needs.
Putting & after the command. Let the main process remain in the foreground so systemd can track it.
Assuming shell syntax works in ExecStart. Pipes, redirection, and variable expansion require explicit handling.
Forgetting daemon-reload. Editing a file on disk does not automatically update the manager’s loaded definition.
Enabling before testing. A broken enabled service becomes a broken startup task.
Using Restart=always to hide a failure. A restart loop creates noise and load while leaving the cause unresolved.
Treating network.target as internet readiness. Unit ordering and external service availability are different questions.
🧹 Remove the Lab Service
When the practice exercise is complete, stop and disable the unit before removing its files:
sudo systemctl disable --now academy-clock.service
sudo rm /etc/systemd/system/academy-clock.service
sudo systemctl daemon-reload
sudo rm /usr/local/bin/academy-clock
sudo userdel academy-clock
These commands permanently remove the lab unit, script, and account. Run them only for the exact objects created in this tutorial. Do not paste the cleanup sequence into a server where those names belonged to pre-existing resources.
✅ Knowledge Check
- Why should a long-running
Type=simpleprogram remain in the foreground? - What is the difference between
daemon-reloadand restarting a service? - Why should
ExecStartuse an absolute path? - Does
WantedBy=multi-user.targetenable the service by itself? - Why can an active service still be unhealthy?
🎓 Check Your Answers
1. systemd can supervise the main process directly. 2. daemon-reload rereads unit definitions; restart stops and starts the workload. 3. A service has a controlled execution environment and should not depend on an interactive shell’s PATH. 4. No; systemctl enable creates the enablement links. 5. systemd can confirm process lifecycle state without testing every application-level function.
A reliable service unit begins with a program that already works, runs it with the least access it needs, expresses only real dependencies, and makes its failures observable. Validate the unit, start it manually, inspect its logs, verify the application, and only then enable it for boot.
Continue learning: the next useful topic is systemd timers—how to schedule repeatable server jobs without relying on a permanently running loop.







