Exercises — Cross-compilation — toolchains, sysroot

The general shape is <arch>-<vendor>-<os>-<abi/libc>. This triplet only has three dash-separated tokens, so one field is dropped:

• arm → arch: 32-bit ARM CPU.
• (vendor missing) → the vendor field (unknown/pc/none) is decorative and is routinely omitted. Tools treat a 3-token triplet as arch-os-abi.
• linux → os: runs under a Linux kernel (so printf, malloc, syscalls all exist).
• gnueabihf → abi/libc: gnu (glibc) + eabi (embedded ABI) + hf (hard-float — floats passed in FPU registers).

See ABI and calling conventions for why the hf matters at the register level.

• build = x86-64 laptop (where the compiler binary itself was assembled — same as host here).
• host = x86-64 laptop (where gcc executes).
• target = Raspberry Pi ARM (where the output runs).

Because build = host ≠ target, this is an ordinary cross compile (not a Canadian cross).

The sysroot rule: every absolute default path $p$ that begins at root / becomes $S+p$ (relocate the root, don't replace the logic — exactly like chroot). ${\text{includes}}^{\prime }=\{\text{/opt/pi/usr/include},\text{/opt/pi/usr/local/include}\}$ ${\text{libs}}^{\prime }=\{\text{/opt/pi/lib},\text{/opt/pi/usr/lib}\}$ Note there is no double slash: /opt/pi + /usr/include = /opt/pi/usr/include.

• (a) source → assembly: arm-linux-gnueabihf-gcc -S (the compiler proper).
• (b) assembly → object: arm-linux-gnueabihf-as (the assembler).
• (c) objects + libs → executable: arm-linux-gnueabihf-ld (the linker).

In practice gcc orchestrates all three, but each stage is a distinct prefixed tool. The object files are in ELF object file format.

Expect: ELF 32-bit LSB executable, ARM, EABI5. The prefix's arch (arm) determines the emitted ISA. If it printed x86-64, the prefix did not take effect — you accidentally invoked the native gcc (wrong binary in $PATH). Fix by giving the full prefixed path or checking which arm-linux-gnueabihf-gcc.

Mechanism: the native gcc has host include paths (/usr/include) baked in at its build time. The cross gcc's baked-in defaults point at a sysroot that is empty or unspecified — it deliberately refuses to fall back to the host's /usr/include, because those are x86 headers. So no stdio.h is visible. Fix: --sysroot=/opt/pi (or use a toolchain that bundles a populated sysroot). This makes <sysroot>/usr/include/stdio.h the file it finds.

configure compares the --host and --build triplets. If they differ, it concludes: "the binaries I produce (for host) will not run on the build machine." Therefore any test of the form "compile a tiny program, run it, read its output" is impossible — the ARM test binary cannot execute on x86. Disabled feature: run-time probes such as "does this system's malloc(0) return non-null?" or "endianness by executing a test" — configure switches to cached / assumed answers (or requires you to pre-seed a config.cache). See Build systems CMake and autotools.

The linker matches by symbol name, not by calling convention. float add(float,float) has the same mangled name in both ABIs, so the symbol resolves — link succeeds. At run time the mismatch bites: soft-float code passes the float in an integer register (r0), hard-float code reads it from an FPU register (s0). One side writes r0, the other reads s0 → garbage / trap. This is an ABI mismatch that no linker check catches. Fix: keep the entire triplet — including float ABI — identical across your code, libc, and every dependency.

aarch64-none-elf-gcc -nostdlib -ffreestanding \
    -T link.ld -o firmware.elf start.s main.c

• -nostdlib: the triplet's none means no libc and no crt0 startup exist — don't try to link them, or you get undefined-symbol errors for things that were never provided.
• -ffreestanding: tells the compiler this is not a hosted environment — main is not special, printf/malloc are not assumed, and it won't insert calls to library helpers.
• -T link.ld: with no OS loader, you decide where code and data sit in physical memory; the linker script places sections at the chip's real addresses. This ties into Static vs dynamic linking — bare metal is fully static, there is no dynamic loader on the chip.

Primary include dir is $S/\text{usr/include}$ = /opt/pi/usr/include. Append the requested relative header sys/types.h: $\text{/opt/pi/usr/include}+\text{/}+\text{sys/types.h}=\text{/opt/pi/usr/include/sys/types.h}$ Single slash between every component — no //. That is the exact file open() receives.

• build = x86-64 CI server (where the compiler is assembled).
• host = ARM build-farm node (where the compiler will execute).
• target = MIPS router (what the compiler's output runs on).

All three differ (x86 ≠ arm ≠ mips) → this is a Canadian cross. An ordinary cross has build = host ≠ target; here even build ≠ host, so you need a cross toolchain to build the cross compiler — two toolchains deep.

Minimum contents (all target/ARM builds):

• /opt/pi/usr/include/openssl/*.h and /opt/pi/usr/include/stdio.h (headers → compiler knows prototypes/struct layouts).
• /opt/pi/usr/lib/libssl.so, /opt/pi/usr/lib/libcrypto.so, /opt/pi/lib/libc.so.6 (libraries → linker resolves symbols against target code).
• Startup files /opt/pi/usr/lib/crt1.o, crti.o, crtn.o.

The fatal-but-buildable mistake: placing host x86 libssl.so inside the sysroot. Symbol names match, so it links, but the shared object holds x86 instructions. On the Pi the dynamic loader tries to map ARM ELF that references an x86 .so → the loader either rejects the ELF class or the app segfaults on the first OpenSSL call. Every file under the sysroot must be target-ISA.

What makes build ≠ host ≠ host a Canadian cross? ::: All three roles differ, so you need a cross toolchain even to build the cross compiler. Why does a soft/hard-float mismatch link but crash? ::: Linker matches symbol names only; float register conventions differ at run time. What does --host ≠ --build tell configure? ::: That it is a cross build, so disable run-a-test-binary feature probes. Sysroot S=/opt/pi, header usr/include: full path of sys/types.h? ::: /opt/pi/usr/include/sys/types.h