Rainbow-OS: Moving a Game Out of the Kernel

For most of its life, every „app“ in Rainbow-OS — the editor, the BASIC
interpreter, the C compiler, the games — was really part of the kernel.
They were compiled straight into kernel.elf and called kernel
functions directly. Convenient, but wrong: a game has no business living inside
the kernel. This update takes the first real step toward fixing that. Space
Invaders is now a standalone ring-3 program, loaded from disk
with run invaders.bin — not compiled into the kernel at all.

Space Invaders running as a ring-3 user program in Rainbow-OS
Space Invaders running as a ring-3 user program in Rainbow-OS

Why it was hard

Rainbow-OS could already run ring-3 programs (the little C programs from its
own compiler). But a game needs things those tiny programs never did: it has to
draw to the screen, read which keys are held down right now, and pace
itself against a clock. In ring 3 it can do none of that directly — drawing
to video memory, touching the timer, reading the keyboard hardware are all
privileged. The only door from userland into the kernel is the syscall, and there
were only eight of them (putchar, printf, exit, and a few odds and ends). A game
needs a real API.

A syscall API for games

So the kernel grew a small set of new syscalls — the contract between a
program and the machine:

  • blit — the program draws into its own 480 KB
    back buffer (fast, in its own memory) and asks the kernel to copy it to the
    screen once per frame. The kernel validates the buffer is really the program’s
    before touching it.
  • keydown — is this key currently held?
  • ticks / yield — read the timer, and sleep
    until the next interrupt (a ring-3 program can’t hlt itself).
  • getfont — borrow the ROM font to draw its own text.

All the game logic and rendering happen in userland; the kernel only does the
handful of privileged things, behind a validated boundary.

Build and run path for a ring-3 user program
Build and run path for a ring-3 user program

A tiny SDK

To build a C program into a flat ring-3 binary you need a little runtime: a
startup stub that zeroes memory and calls main, assembly stubs that
turn each syscall into an int 0x80, a one-line memset,
and a linker script that places everything at the program’s load address. That’s
the new apps/lib — a minimal SDK. A build script runs the
cross-compiler over it and drops the resulting invaders.bin into the
disk image. The game’s source moved out of kernel/ entirely, and the
kernel got a few kilobytes smaller.

Three bugs, three lessons

Porting the game surfaced exactly the kind of bug that only appears once code
runs unprivileged — and each one was a small lesson in how the hardware
actually works.

The frame-pacing deadlock. The game sleeps between frames with
yield, which runs hlt in the kernel. But the syscall
gate disables interrupts on entry — so the CPU halted with no way
to ever wake up. The fix is one instruction: enable interrupts before halting.

The invalid-opcode loop. The first build ran main,
then froze with the screen still showing the boot console. The serial log gave it
away: Unhandled interrupt: 0x06, over and over. The compiler, left to
its own devices, had optimised some conditional logic into a cmov
instruction — which didn’t exist until the Pentium Pro. On the emulated 486
it’s an illegal opcode, and with no handler the CPU just retried it forever.
Telling the compiler to target a 486, like the kernel does, fixed it instantly.

The uninitialised memory. A flat binary has no zeroed memory
handed to it — the loader copies only the bytes on disk. The startup stub
has to zero the program’s static data itself, all 480 KB of back buffer
included, before main runs.

The payoff

The result looks identical to the in-kernel version — the same marching
fleet, bunkers and bombs at 800×600 — but it’s a genuinely separate,
unprivileged program now. It can crash without taking the system down, it reaches
the kernel only through the syscall door, and its source no longer pollutes the
kernel tree. The big subsystems (the editor, BASIC, the compiler) are still
built-in and will take more work to extract — they need file-I/O syscalls
and the host toolchain — but the road out of the kernel now exists, and the
first traveller has made the trip.

Nach oben scrollen