Decompress a ClickHouse Binary (Cross-Architecture)

ClickHouse release and CI clickhouse binaries are self-extracting: a small
decompressor stub, followed by the zstd-compressed real ELF and a trailer.

The normal way to decompress is to run the binary once: it extracts the inner
ELF in place and re-execs it. That works on the binary's own architecture,
and also on a foreign architecture if qemu user-mode emulation for it is
installed (e.g. qemu-aarch64 to run an aarch64 binary on x86). When qemu for
the target is not available, you cannot run the binary at all and must extract the
payload offline.

This skill extracts the inner ELF without executing anything, on any host.

When to use

• You downloaded a CI/release clickhouse and gdb/lldb shows no real symbols
  (it only sees the decompressor stub's tiny symbol table).
• The binary's architecture differs from the host (cannot self-extract by running).
• You need the inner ELF to load a core dump (see ci/decrypt-cores.md for the
  matching core-dump decryption procedure).

Format

See utils/self-extracting-executable/types.h:

[ decompressor ELF ]
[ compressed file blobs ]
[ FileData[] ]                 # one per packed file, each followed by its name
[ MetaData (16 bytes) @ EOF ]

MetaData { uint64 number_of_files; uint64 start_of_files_data; }
FileData { uint64 start, end, name_length, uncompressed_size, umask; bool exec; }

MetaData sits at the very end of the file. start_of_files_data points at the
FileData array; each 48-byte FileData is followed by the file name. The
compressed bytes for a file are input[start:end] (zstd, possibly multi-frame).
The packed clickhouse ELF is the entry with exec = true.

Steps

1. Download the binary from the build job for the exact commit, for example:

   curl -s "https://clickhouse-builds.s3.amazonaws.com/PRs/<pr>/<sha>/pr/build_<arch>_<sanitizer>/clickhouse" -o clickhouse.sfx

   The path segment after <sha> is the normalized workflow name — pr for PR
   builds, masterci for master/reference builds (REFs/master/<sha>/masterci/...).
   Find the precise URL in the build job's artifact_report_build_*.json, or via
   .claude/tools/fetch_ci_report.js "<pr-url>". Download in the foreground (a
   killed/resumed curl can append garbage past EOF and break the trailer; verify
   the size matches Content-Length).

2. Extract the inner ELF:

   python3 .claude/skills/decompress-binary/extract_self_extracting.py clickhouse.sfx clickhouse.elf

3. Verify it is the right build and has symbols:

   file clickhouse.elf                                          # ELF ..., not stripped, with debug_info
   llvm-objdump -s -j .note.gnu.build-id clickhouse.elf | tail  # must match the core's build id

   The build id must equal the one in the crash report / core. A mismatched
   binary yields unusable backtraces.

4. Use it with the core dump:

   gdb clickhouse.elf core.<pid>      # or: lldb clickhouse.elf -c core.<pid>

   gdb and lldb read foreign-architecture cores fine for backtraces and memory
   inspection (you are not executing the target).

Notes

• A truncated or corrupted download is the most common failure: if the script
  reports an implausible number_of_files, re-download cleanly and check the size.
• The inner ELF is large (several GB for sanitizer builds, unstripped). Make sure
  there is enough disk.
• Shortcut when you can run the binary: if the host matches the binary's
  architecture, or qemu user-mode emulation for it is installed, just run
  ./clickhouse once to self-extract in place. This skill is for the case where
  neither is possible.