Debugging a Broken Docker Compose Stack

docker compose ps showed every container as "running." The app itself was unreachable — connection refused, no error page, nothing. The instinct was to restart the stack and see if it fixed itself. Restarting first would have wiped out the one piece of evidence that actually explained what was wrong.

The triage order, before touching anything else

1. docker compose ps — confirm the actual state and restart count. A container listed as "running" can still be failing its own healthcheck; a high restart count means Compose has already tried to fix this for you and failed.
2. docker compose logs --tail=100 <service> — read what the container actually said before it died, without scrolling through its entire history.
3. The exit code — docker compose ps -a or docker inspect <container> --format='{{.State.ExitCode}}' narrows the failure to a category before reading a single further line.
4. The healthcheck status, if the service defines one — docker inspect --format='{{json .State.Health}}' <container> shows the last several check results, including the actual command output.

Doing this in order, before restarting anything, takes under two minutes and almost always narrows the problem to one of a handful of categories — a config error, a resource limit, or a dependency that isn't ready yet.

logs -f vs logs --tail: two different jobs

docker compose logs -f <service> follows output live — the right tool while a failure is actively happening, watching a service boot in real time. docker compose logs --tail=100 <service> (or any N) pulls the last N lines of history without following, which is what a service that already crashed and restarted actually needs: the crash already happened, so there's nothing left to follow, only history to read. Reaching for -f on an already-dead container just shows silence; reaching for --tail on a live boot means missing whatever comes next.

What the exit code actually narrows down

Exit code	Meaning	Typical cause
0	Clean exit	The process finished on its own — expected for one-shot jobs, unexpected for a long-running service
1	Generic application error	A config error, missing environment variable, or an unhandled exception in the app itself
137	SIGKILL (128 + 9)	Almost always the kernel's OOM killer, or Docker forcibly stopping an unresponsive container
143	SIGTERM (128 + 128)	The process was asked to stop gracefully — normal on a docker compose stop or down, unexpected mid-run

docker inspect <container> --format='{{.State.OOMKilled}}' answers the 137 question directly — true means the kernel killed the process for using too much memory, which is a resource-limit problem, not a bug to debug in the app's code.

Reading a healthcheck block that's failing

A service with a healthcheck: block reports one of three states: starting (still inside start_period, failures don't count yet), healthy, or unhealthy (the check command itself has failed enough times in a row). docker inspect --format='{{json .State.Health}}' <container> returns the last five results, including the actual stdout of the check command — often the fastest way to discover the healthcheck command is testing the wrong port or path, not that the app itself is actually broken.

Crash loop or slow first boot? The restart count is the tell

A database container on this homelab sat in restarting for what looked exactly like a crash loop — until checking docker compose ps a minute later showed the restart count hadn't moved since the first check. That was the actual signal: a genuine crash loop keeps incrementing its restart count every few seconds as Compose tries and fails repeatedly, while a slow first boot restarts once (or not at all) and then just takes a while inside that single attempt. The database in question was running an initial schema migration on first launch — a one-time, several-minute delay that looks identical to a crash loop for the first ten seconds and nothing like one a minute later.

The fix in both directions is different: a real crash loop means the underlying error needs fixing before the container will ever stay up, whether that's a networking issue covered in the networking guide or a bad restart policy interacting badly with a real failure, as covered in the restart policies guide. A slow first boot just needs patience, and a start_period on the healthcheck long enough to not flag it as unhealthy before it's actually ready.

Decision table

Symptom	Check first	Likely cause
Container listed "running" but app unreachable	healthcheck status via docker inspect	Healthcheck testing the wrong port/path, or app not actually listening yet
Restart count climbing every few seconds	logs --tail for the crash reason, then the exit code	Genuine crash loop — config error, missing dependency, or exit 137 (OOM)
Restart count flat, container "restarting" for minutes	Whether this is a first-ever start	Slow first boot — migration, index build, or large dataset load
Exit code 137	docker inspect for OOMKilled: true	Memory limit too low for the workload, not necessarily an app bug
Exit code 143 outside a deliberate stop/down	Whether anything else issued a stop	Something external sent SIGTERM — check for an orchestration script or a health-based auto-restart

The bottom line

Read before you restart. docker compose ps for state, logs --tail for history, the exit code for the category, and the healthcheck's own output if one is defined — in that order, every time, regardless of which of the seven services in a stack is the one that broke. Once a stack is stable again, tearing it down and back up cleanly is its own decision covered in down vs stop vs kill.

Frequently asked questions

What does exit code 137 mean in Docker?

128 + 9 — the process received SIGKILL. It almost always means the kernel's OOM killer terminated the container for using too much memory; confirm with docker inspect <container> --format='{{.State.OOMKilled}}' before assuming it's an application bug.

How do I check why a Docker Compose container keeps restarting?

Check docker compose ps for the restart count first — if it's climbing every few seconds, read docker compose logs --tail=100 <service> for the actual error before the next restart clears it, then check the exit code to narrow the category.

What is the difference between docker compose logs and docker logs?

docker compose logs <service> reads by the service name defined in the compose file and can show output from every service in the stack at once; docker logs <container> needs the actual container name or ID. For a Compose-managed stack, docker compose logs is almost always the more convenient of the two.

How do I know if a Docker healthcheck is failing?

docker inspect --format='{{json .State.Health}}' <container> shows the current status (starting, healthy, or unhealthy) and the last several check results, including the actual output of the check command — read that output before assuming the application itself is broken.

Why does my container show as running but the app doesn't respond?

Check the healthcheck status first if one is defined — a container can be in the running state while its healthcheck reports unhealthy, often because the check is testing the wrong port or path, or the app hasn't finished starting up yet.