Worked examples — Disassembly — objdump, reading assembly

Prerequisites we lean on: Calling Conventions (System V x86-64), Stack Frames & The Stack Pointer, Compiler Optimization Levels, and the parent Disassembly.

The scenario matrix

Every case this topic can throw at you, and the example that covers it:

Read top to bottom; each cell builds on the last.

C1 — Signed comparison, positive branch

Steps

1. test edi, edi computes edi & edi. Why this step? ANDing a number with itself changes nothing but does set the flags — cheaper than cmp edi, 0. The sign flag (SF) ends up = the top bit of a, the zero flag (ZF) = 1 if a == 0.
2. jle 0xb means "jump if less-than-or-equal (signed)". Why this step? Source says do the return 1 if a > 0; assembly inverts it and jumps away to the return 0 block when a <= 0. Inversion is the normal pattern (parent's if example).
3. For a = 5: 5 <= 0 is false → no jump → mov eax, 1. Result 1.
4. For a = 0: 0 <= 0 is true → jump to 0xb → mov eax, 0. Result 0.
5. For a = -3: -3 <= 0 true → jump → 0.

Verify: the C function returns a > 0 ? 1 : 0. For {5, 0, -3} that is {1, 0, 0} — matches steps 3-5. ✔

C2 — Negative operand, < branch

Steps

1. Decode 0xfffffffc. Why this step? In 32-bit two's complement, the top bit is the sign. 0xfffffffc = 4294967292 if unsigned, but as a signed int it is $0xfffffffc-2^{32}=4294967292-4294967296=-4$. The compiler wrote -4 this way.
2. cmp [rbp-0x4], -4 computes x - (-4) = x + 4 and sets flags. Why this step? cmp never stores a result; it only feeds the flags to the next jump (parent's rule).
3. jge 0xf = "jump if greater-or-equal (signed)". Source wants y=1 when x < -4; assembly jumps to the else (y=2) when x >= -4.
4. For x = -10: is -10 >= -4? No. So we do not jump → fall through to y = 1.

Verify: -10 < -4 is true, so y = 1 — matches step 4. And the immediate: -4 stored as unsigned is $2^{32}-4=4294967292=\text{0xfffffffc}$. ✔

C3 — The unsigned trap

Steps

1. Same bit pattern, two readings. Why this step? 0xffffffff is -1 as a signed int, but 4294967295 as unsigned. The bits are identical; only the compiler's chosen instruction decides how they're compared.
2. setg uses signed semantics. So f asks: is -1 > 3? No → al = 0.
3. seta uses unsigned semantics. So g asks: is 4294967295 > 3? Yes → al = 1.
4. Why does this matter for reading asm? The mnemonic tells you the C type. g/a-family = unsigned, l/ge-family = signed. You can recover signedness that the raw bytes alone hide.

Verify: signed -1 > 3 = 0, unsigned 4294967295 > 3 = 1. ✔

C4 — Degenerate input: the empty function

Steps

1. Even doing nothing, -O0 still emits the prologue (push rbp; mov rbp, rsp) and epilogue (pop rbp; ret). Why this step? -O0 is mechanical: it sets up a frame for locals it might need, then tears it down. See Stack Frames & The Stack Pointer.
2. The lone 90 is a nop (no-operation). Why this step? It marks the (empty) function body and can aid alignment. It changes no state.
3. Net effect on registers: rbp pushed then popped → unchanged; rsp back where it began.

Verify: count the bytes: 55 (1) + 48 89 e5 (3) + 90 (1) + 5d (1) + c3 (1) = 7 bytes, and the last address is 6 for a 1-byte ret, so the function spans 0x0..0x6 inclusive = 7 bytes. ✔ Register net change = zero. ✔

C5 — A loop that counts down (back-edge jump)

Steps

1. Initialise eax=0 (sum), edx=3 (i). Why this step? The compiler keeps both in registers at this level — no stack spill needed for a tight loop.
2. add eax, edx then sub edx, 0x1. Why the reordered order? The compiler ran the body first, decrement after, so the loop can end on a single jne testing the sub's flags — one branch per iteration.
3. jne 0xa = "jump if not equal" — it reads the zero flag left by sub. As long as i didn't hit 0, we loop. Why this step? Counting down to zero is cheaper: the CPU sets ZF for free on sub, so no separate cmp is needed.
4. What the loop looks like (described inline, in case the figure isn't loaded): the four instructions add → sub → jne sit in a straight vertical column, but jne at address 0xf targets address 0xa — an address earlier than itself. That upward jump is the back-edge (drawn as the thick red curved arrow on the right of the figure): it loops control from the bottom of the body back to add eax, edx at the top. When i finally reaches 0 the zero flag is set, jne is not taken, and control falls straight through to ret — leaving the loop.
5. Trace: sum=0,i=3 → add→sum=3, sub→i=2, back-edge taken. → add→sum=5,i=1, taken. → add→sum=6,i=0, not taken → fall through → ret.

Verify: $3+2+1=6$. Final eax = 6. ✔

C6 — Decode the raw bytes by hand (variable length)

Steps

1. First byte 48 is a REX prefix. Why this step? In the range 0x40–0x4f, 48 means "this operation uses 64-bit operands" (the parent noted 48 = 64-bit). It is a prefix, not a standalone op — the real opcode follows.
2. Next byte 83 = opcode for "arithmetic with an 8-bit immediate" on a 32/64-bit register. Why this step? 0x83 is the compact form used when the constant fits in one signed byte.
3. c0 is the ModR/M byte selecting the operation (the /0 field = ADD) and the register (rax). 05 is the immediate = 5.
4. Assemble the meaning: add rax, 0x5, total length 4 bytes. Why this matters: you cannot chop x86 into fixed 4-byte words like RISC — see CISC vs RISC. You must walk prefix → opcode → ModR/M → immediate.

Verify: The instruction length here is prefix(1) + opcode(1) + modrm(1) + imm8(1) = 4 bytes, and 0x05 decodes to decimal 5. ✔

C7 — Limiting case: the optimizer deletes it all

Steps

1. Everything is constant. Why this step? The optimizer does constant folding: it evaluates 2*3+4 = 10, then 10*10 = 100 at compile time. See Compiler Optimization Levels.
2. 0x64 is that folded constant. Why this step? $0x64=6\cdot 16+4=100$. The entire function collapses to "load 100, return".
3. This is the limiting behaviour of optimization: when a result depends on nothing at runtime, the runtime work disappears entirely.

Verify: $2\cdot 3+4=10$, $10\cdot 10=100$, and $0x64=100$. ✔

C8 — Real-world: crash address → source line

Steps

1. The bug is division by zero. Why this step? idiv esi with esi = 0 triggers a #DE (divide error) CPU exception — a hardware trap, not a memory segfault, but the OS surfaces it to your process.
2. cdq before idiv. Why this step? idiv divides the 64-bit value in edx:eax. cdq copies the sign bit of eax across all of edx so a signed divide is correct.
3. The faulting instruction is at 0x401133 (idiv). Why is the crash line printed as 0x40113a? Precisely: on a #DE fault the CPU pushes the faulting rip itself (0x401133) onto the kernel stack — the saved instruction pointer points at the idiv, not past it. But your program's own signal/crash reporter didn't print that raw rip; it printed the symbol boundary it resolved the address into. 0x40113a here is the start of the next symbol — functions are commonly padded to a 16-byte boundary, so the linker left gap bytes between divide (ending at 0x401135) and the next function at 0x40113a. A naive reporter that rounds "which function am I nearest to" upward, or that reads a stale return address off the stack, lands on that padded boundary.
4. The reliable method: never trust a rounded address. Find the largest instruction address ≤ the reported address that lies inside a real function. 0x401133 (idiv) is the last real instruction ≤ 0x40113a inside divide, so that is the culprit. Confirm the live rip in Debugging with GDB rather than trusting the printed line.

Verify: idiv is at 0x401133; the padding gap up to the next symbol is $0x40113a-0x401135=5$ bytes (after ret), and $10/0$ is undefined → divide error. ✔

C9 — Exam twist: same brackets, two meanings

Steps

1. [rdi+rsi] is an address expression: 100 + 4 = 104. Why this step? Square brackets in Intel syntax mean "compute this address".
2. mov eax, [rdi+rsi] (A) dereferences: go to address 104, load the value stored there. Why this step? mov with a bracket source reads memory. So eax = 77.
3. lea eax, [rdi+rsi] (B) does not touch memory — it stores the address itself. Why this step? lea = "load effective address"; it hands you 104 as a number. The optimizer abuses this as a free adder (parent's -O2 add example). So eax = 104.
4. The trap: identical brackets, opposite behaviour — one reads RAM, one computes a number.

Verify: (A) loads memory[104] = 77. (B) computes 100 + 4 = 104. ✔

C10 — Tail call: a jmp where you expected call

Steps

1. add edi, 0x1 prepares the argument. Why this step? edi holds arg 1 (System V convention). helper(a+1) needs a+1 in edi, so the compiler increments it in place.
2. The last instruction is jmp helper, not call helper. Why this step? This is a tail call: wrapper's only remaining act is to return whatever helper returns. Instead of call helper (which pushes a return address) then ret, the compiler reuses wrapper's own return address — it jmps straight into helper. When helper runs its ret, it returns directly to wrapper's caller, skipping wrapper entirely.
3. Why no ret here? There's nothing left to do after helper returns, so wrapper has no epilogue at all. The stack frame count stays flat — this is the essence of tail-call optimization.
4. Reading rule: a jmp to another function's label at the end of a function is not a loop and not a bug — it is a tail call. The return value flows through untouched.

Verify: wrapper(a) returns helper(a+1). For a = 7, edi becomes 8, and the result is whatever helper(8) yields — wrapper adds nothing on top. So the observable result equals helper(8), i.e. the +1 is the only transformation. ✔

Connections

• 5.3.17 Disassembly — objdump, reading assembly (Hinglish) — the parent topic, read alongside.
• Calling Conventions (System V x86-64) — why args land in edi/esi and results in eax.
• Stack Frames & The Stack Pointer — the prologue/epilogue in C4 and the tail call in C10.
• Compiler Optimization Levels — constant folding in C7, lea reuse in C9, tail calls in C10.
• Debugging with GDB — confirm the live rip in the C8 crash.
• CISC vs RISC — why C6's bytes are variable length.