coredumpctl: Finding a Binary Crash

What is coredumpctl and how it works

When a binary crashes with SEGV, the kernel can save a core dump — a snapshot of process memory at the moment of the crash. In systemd-based distributions, coredumpctl handles collecting, storing, and searching these dumps. It’s a wrapper around systemd-coredump, which stores dumps in /var/lib/systemd/coredump/ and indexes metadata through journald.

Installation is trivial for most distributions:

# Debian/Ubuntu
sudo apt install systemd-coredump

# RHEL/Fedora
sudo dnf install systemd-coredump

After installation, verify that kernel.core_pattern points to a pipe into systemd-coredump:

cat /proc/sys/kernel/core_pattern
# expected: |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e

If it shows core.%e.%p, dumps are written to files and coredumpctl won’t see them.

Listing dumps: coredumpctl list

The base command prints all saved crashes:

coredumpctl list

Output contains columns: MESSAGE ID, TIMESTAMP, PID, UID, GID, COMM, EXE, COREFILE.

Key flags for list:

Crash details: coredumpctl info

For a single dump, info gives everything needed before pulling the file:

coredumpctl info 1234

Where 1234 is the process PID or number from list. Output includes:

• the signal that caused the crash (SIGSEGV, SIGABRT, etc.)
• timestamp
• executable path
• core file size
• MESSAGE ID — correlation with journald

# Example output (key lines)
         PID: 1234 (myapp)
     UID:GID: 1000:1000
      Signal: 11 (SIGSEGV)
    Timestamp: Mon 2025-01-06 14:23:01 UTC
     Command Line: /opt/myapp/bin/myapp --config prod.yaml
     Executable: /opt/myapp/bin/myapp
       Core File: /var/lib/systemd/coredump/myapp.1234.abc123.core

Extracting the core file: coredumpctl dump

The extracted file can be piped directly into gdb or saved to disk:

# Save to current directory
coredumpctl dump 1234 --output=myapp.core

# Pipe directly into gdb
coredumpctl dump 1234 -o - | gdb /opt/myapp/bin/myapp -

Filtering and searching by binary, PID, time

In production, dumps accumulate by the dozens, and searching by PID is impractical. coredumpctl supports multiple filters at once:

# All crashes of a specific binary
coredumpctl list --exe /opt/myapp/bin/myapp

# Crashes in the last hour
coredumpctl list --since "1 hour ago"

# By specific user and binary
coredumpctl list --exe /usr/bin/python3 --uid 1000

# Only SIGSEGV
coredumpctl list --signal 11

For scripts and automation, JSON output is convenient:

coredumpctl list --json=short | jq '.[] | select(.exe == "/opt/myapp/bin/myapp") | .pid'

Practical examples for debugging crashes

A typical debugging cycle looks like this:

# 1. Find the latest crash of myapp
coredumpctl list --exe /opt/myapp/bin/myapp -n 1

# 2. Check details — what signal and where
coredumpctl info <PID>

# 3. Pull the core and launch gdb
coredumpctl dump <PID> --output=/tmp/myapp.core
gdb /opt/myapp/bin/myapp /tmp/myapp.core

# 4. In gdb:
(gdb) bt full
(gdb) info registers
(gdb) x/16i $pc

If dumps don’t appear, check coredumpctl list and journalctl -u systemd-coredump. A common cause: LimitCORE in the systemd unit is set to 0, or kernel.core_pattern isn’t configured as a pipe.

[Service]
LimitCORE=infinity