The Stack and Calling Conventions

assembly1-7 built one simple, self-invented LC-3 calling convention. cpu8bit1-12 used one straightforward subroutine convention per chip. This chapter is where that simplicity ends: x86-64 doesn't have a calling convention — it has (at least) two real, competing, mutually incompatible standards, and getting the wrong one is a genuine way to crash a real program.

The Stack — RSP and RBP

RSP is a full 64-bit stack pointer, free to point anywhere — as fully general as the Z80's own SP from cpu8bit1-7, with no 6502-style page lock. PUSH/POP are native instructions, operating on 64-bit values by default in long mode.

RBP plays a role neither LC-3 nor the 6502/Z80 ever formally needed: a stable frame pointer. As a function runs, RSP itself keeps moving — every push and pop shifts it. RBP is set once, at the start of a function, and held fixed for its entire duration, giving a stable reference point for that function's own parameters and local variables regardless of how RSP moves around them. It's conceptually the same idea as the 6502/Z80's own convention of dedicating a register to a specific job (cpu8bit1-3's own R6-as-stack-pointer precedent from assembly1-7) — just applied to a genuinely new problem, deep local-variable frames, that this arc's simpler subroutines never had.

Worth an honest, brief note: modern compiled code often skips the frame pointer entirely ("frame pointer omission"), computing local-variable offsets directly from RSP instead, as a performance optimization. RBP is a real, common convention, not a hardware requirement.

Register-Based Argument Passing — A Genuine Departure

Neither assembly1-7's LC-3 subroutines nor cpu8bit1-12's own capstone formally passed "arguments" at all — a value was just already sitting in whatever register the subroutine happened to expect, by ad-hoc agreement between caller and callee. Real x86-64 calling conventions formalize this completely: the first several arguments to a function are passed directly in specific, standardized registers, not via the stack — the stack is reserved for overflow arguments beyond that fixed count, and for local storage.

Two Competing Standards

This is the real complexity: which registers hold which arguments depends entirely on which operating system the code targets.

A concrete example — calling a hypothetical function with three integer arguments:

; System V AMD64 ABI (Linux/macOS)
MOV RDI, 10   ; 1st argument
MOV RSI, 20   ; 2nd argument
MOV RDX, 30   ; 3rd argument
CALL some_function

; Microsoft x64 (Windows) — same call, different registers
MOV RCX, 10   ; 1st argument
MOV RDX, 20   ; 2nd argument
MOV R8, 30    ; 3rd argument
SUB RSP, 32   ; reserve the required 32-byte shadow space first
CALL some_function

One more real, shared rule worth naming: in both conventions, RBX, RBP, and R12–R15 are callee-saved — a called function must preserve their values before returning, exactly the register-preservation discipline assembly1-7 first established for R7 and cpu8bit1-12's own capstone reinforced by deliberately choosing C over B.

Why Real Programs Must Actually Care

In every prior course, "the calling convention" was something the programmer invented for their own self-contained program — nothing outside that program ever needed to agree with it. x86-64 calling conventions are a real contract: honoring them correctly is what lets hand-written assembly interoperate with the operating system, system libraries, and code compiled by an entirely different compiler. Get it wrong, and the failure isn't a logic bug in your own code — it's a silent violation of an agreement code outside your control was relying on, often surfacing as a crash or corrupted data far from the actual mistake.

Hands-On Exercises