Memory-mapped files — Coding notes

WHY does this exist?

Three concrete WHYs you should be able to say out loud:

Avoid double-copying — share the kernel page cache instead of copying into a private buffer.
Random access becomes free — jumping to byte 5,000,000 is just ptr[5000000], no lseek.
Lazy loading (demand paging) — a 4 GB file mapped costs almost nothing until you touch a page; only the pages you access are loaded.

WHAT is it, precisely?

Definition

mmap (POSIX)

void *mmap(void *addr, size_t length, int prot,
           int flags, int fd, off_t offset);

Maps length bytes of file fd starting at offset into memory, returning a pointer addr.

prot: PROT_READ, PROT_WRITE, PROT_EXEC
flags: MAP_SHARED (writes go back to file + other processes see them) vs MAP_PRIVATE (copy-on-write, writes stay private)

HOW does it actually work? (derive the mechanism)

We derive the behaviour from one fact: virtual memory already maps virtual pages to physical frames via the page table, filling them on a page fault. Memory-mapping just changes what backs a page.

Step 1 — the call sets up bookkeeping, not data. mmap creates a Virtual Memory Area (VMA): "virtual range $[V, V+L)$ corresponds to file region $[off, off+L)$ ." Why this step? Setting page-table entries to "not present" lets the OS defer all real work — nothing is read yet.

Step 2 — first touch causes a page fault. You read ptr[k]. The CPU finds the page-table entry marked not present → page fault → trap into kernel. Why? This is the hook that lets the OS notice "you want data that isn't loaded."

Step 3 — the fault handler loads from file. The handler computes which file page corresponds to that virtual address, reads it from disk into a physical frame (often it's already in the page cache → zero disk I/O), and installs the page-table entry. The faulting instruction is re-executed and now succeeds. Why re-execute? So the program is oblivious — it just sees a normal memory access that "happened to be slow once."

Step 4 — writes mark the page dirty. With MAP_SHARED, writing sets the dirty bit. Why? So the OS knows this page differs from disk and must be written back later (on msync, munmap, or when reclaimed).

The mapping arithmetic you must be able to reproduce: $\text{virtual addr } V_a \;\Rightarrow\; \text{file offset} = off + (V_a - V_{\text{base}})$ $\text{page number} = \left\lfloor \frac{V_a - V_{\text{base}}}{P} \right\rfloor, \quad P = \text{page size (typ. } 4096\text{ B)}$

Why this step? Because the OS aligns everything to page boundaries; understanding this explains why offset must be a multiple of $P$ .

Worked examples

Worked example Example 3 — Counting characters without

read()

int fd = open("data.txt", O_RDONLY);
struct stat st; fstat(fd, &st);
char *p = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
long count = 0;
for (off_t i = 0; i < st.st_size; i++)
    if (p[i] == 'a') count++;
munmap(p, st.st_size);

Why each step? fstat gives the size so we map exactly the file; MAP_PRIVATE because we only read; the loop treats the file as a plain array — no syscalls inside the loop, that's the win.

Forecast-then-Verify

Recall Forecast: process A maps a file

MAP_SHARED and writes 'X' at byte 0. Process B has the same file mapped MAP_SHARED. Does B see 'X'? What if B used MAP_PRIVATE? MAP_SHARED: Yes — both map the same page-cache frame, so B sees 'X' instantly (no disk round-trip needed). MAP_PRIVATE: No — on B's first write it gets a copy-on-write private copy; A's writes (and B's) are not visible across each other. (For reads of pages B never wrote, B may still see A's data until COW triggers — implementation-dependent, so don't rely on it.)

Common mistakes (steel-manned)

The 80/20 core

Recall Feynman: explain to a 12-year-old

Imagine a giant book in the library. Normally to read it you photocopy every page you want and carry the copies home — slow and wasteful. Memory-mapping is like getting magic glasses: you put them on and the book just appears on your desk. But the trick is the pages are blank until you actually look at one — the moment your eyes land on a page, it instantly fills in (that's the "page fault"). If you write a note on a page with the "shared" glasses, your friend wearing the same shared glasses sees your note too, because you're both looking at the same page, not copies.

Connections

Virtual Memory — mmap is built entirely on its page-table + page-fault machinery.
Demand Paging — the lazy-loading mechanism mmap reuses.
Page Cache — what MAP_SHARED pages actually point at.
Copy-on-Write — how MAP_PRIVATE and fork() defer copies.
File Descriptors and read-write syscalls — the alternative mmap replaces.
Shared Memory (IPC) — MAP_SHARED of the same file is an IPC channel.
SIGBUS and SIGSEGV — faults you hit when mapping is misused.

What does mmap make a file appear as?

A region of the process's virtual address space, accessed via pointers instead of read/write syscalls.

Why is mmap faster than read() for repeated access?

It avoids copying from the kernel page cache into a private user buffer — the process points directly at the page cache.

When are file pages actually loaded under mmap?

Lazily, on first access, via a page fault (demand paging) — not at mmap() time.

Difference between MAP_SHARED and MAP_PRIVATE?

SHARED: writes go to the file and are visible to other mappers. PRIVATE: copy-on-write, writes stay local and never hit the file.

What constraint applies to the offset argument of mmap?

It must be a multiple of the system page size (e.g. 4096), else EINVAL.

How many page faults to sequentially read a 10000-byte file (4096 B pages)?

ceil(10000/4096) = 3 major faults.

What does writing to a MAP_SHARED page do immediately, and when does it reach disk?

It updates the page cache and sets the dirty bit; disk write-back is deferred to msync/munmap/reclaim/flush daemon.

Which call forces dirty mapped pages to disk now?

msync(addr, length, MS_SYNC).

What signal do you get accessing whole pages beyond the file's end?

SIGBUS.

File offset for virtual address Va in a mapping with base Vb and file offset off?

off + (Va - Vb).

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, normally jab tum file padhte ho toh read() syscall karte ho, jo data ko disk se kernel ke page cache mein laata hai, aur phir ek extra copy tumhare user buffer mein daalta hai. Yeh dusri copy time aur memory waste karti hai. Memory-mapped file ka idea simple hai: OS ko bolo "is file ko meri memory ka hissa bana do". Iske baad file padhna matlab sirf ek pointer ptr[i] dereference karna, aur likhna matlab ptr[i] = x — koi syscall loop mein nahi, koi extra copy nahi.

Sabse important baat: mmap karte hi poori file RAM mein load nahi hoti. OS sirf bookkeeping karta hai (VMA banata hai) ki "yeh virtual range is file ki hai". Jab tum koi byte pehli baar touch karte ho, tab page fault hota hai, aur OS us page ko disk se laata hai (ya page cache mein already hai toh free mein). Isiliye tum 4 GB ki file map kar sakte ho 2 GB RAM ke saath — sirf touched pages memory leti hain. Yeh hi demand paging hai.

MAP_SHARED aur MAP_PRIVATE ka farq yaad rakhna exam aur interview dono ke liye crucial hai. SHARED matlab tumhari writes file mein wapas jaati hain aur dusre processes ko bhi dikhti hain — yeh IPC (inter-process communication) ka tareeka ban jaata hai. PRIVATE matlab copy-on-write — tum jaise hi likhte ho, ek private copy ban jaati hai, file aur dusre logon ko kuch nahi dikhta.

Ek common galti: log sochte hain SHARED mein write turant disk pe jaata hai. Nahi — write sirf page cache update karta hai aur dirty bit set karta hai. Disk pe baad mein jaata hai (msync, munmap, ya flush daemon). Agar abhi durability chahiye toh msync call karo. Mantra yaad rakho: Map, Touch, Fault, Fill, Dirty.

Test yourself — Operating Systems

Connections