OS architecture overview

  • Programs run inside an operating system (Linux, Windows, macOS) which provides services (I/O, memory management, isolation) and shapes their behavior at runtime

User mode vs. kernel mode

  • User mode (EL0) is where applications execute with restricted privileges—no direct hardware or MMU access
  • Kernel mode (EL1) has full access to hardware, memory management, and system resources

Processes

  • Each process gets a unique virtual address space, enforced by the MMU and page tables
  • Assigned a process identifier (PID); you can list process trees (ps axfj) or view dynamic stats with htop/atop

System calls

  • Interaction between user mode and kernel via syscalls: on ARMv8-A, use the SVC (supervisor call) instruction to trap to EL1
  • Kernel decodes the syscall number/arguments, executes it, then returns to user mode

Objects and handles

  • The kernel represents resources (files, sockets, memory) as objects; user‐mode programs get handles to refer to them
  • For example, open() returns a file handle; close() notifies the kernel to release it

Threads

  • A new process starts with one thread; multithreading via APIs (e.g., pthread_create) adds more within the same address space
  • Threads share code/data but have independent registers (PC, SP, flags, local variables) and exit when the last thread finishes

Process memory management

  • Virtual addresses are translated by the MMU (programmed via page tables) into physical addresses
  • You can inspect a process’s memory map via /proc/<pid>/maps or /proc/self/maps

Memory pages

  • Memory regions are page‐aligned (4 KB standard on Linux; can be 16 KB or 64 KB on ARMv8-A; huge pages also possible)

Memory protections

  • Each region has Read/Write/Execute (RWX) flags, shown by letters (rwx, with - for absent perms)
  • Access Permission (AP) bits control unprivileged vs. privileged access

Anonymous and memory-mapped memory

  • Anonymous memory: zero-filled pages for heap allocation, obtained via brk or mmap MAP_ANONYMOUS and subdivided by the runtime allocator
  • Memory-mapped files: demand-loaded from disk, share unmodified pages between processes; writes propagate to disk unless MAP_PRIVATE is used

Address space layout randomization (ASLR)

  • Randomizes base load addresses of binaries, libraries, and data to hinder memory-corruption exploits
  • Entropy varies by OS; can be disabled via /proc/sys/kernel/randomize_va_space or within GDB (set disable-randomization)

Stack implementations

  • Four variants: Full Ascending, Full Descending, Empty Ascending, Empty Descending

    • Full: SP points to last pushed item; Empty: SP points to next free slot
    • Ascending: stack grows to higher addresses; Descending: to lower addresses

Shared memory

  • Regions intentionally mapped into multiple processes, sharing the same physical pages; marked “s” in /proc/self/maps
  • Writes by one process are immediately visible to others; also used for TrustZone communication between Secure and Normal worlds