Lazy allocating sbcl, part II

After working on this in my spare time for a long time, I have something that almost works. The lazy sbcl 0.8.20.5 patches will with the following customize-target-features.lisp: :sb-lazy-linkage-table-space, :sb-lazy-dynamic-space, :sb-lazy-read-only-space, :sb-lazy-static-space, :sb-linkage-table-space. They change the specific allocations for that space so the memory is only allocated when it is needed. The net effect is:

$ diff -u /proc/13546/status /proc/13598/status
--- /proc/13546/status  2005-03-18 07:14:16.085383120 +0100
+++ /proc/13598/status  2005-03-18 07:14:16.085383120 +0100
@@ -1,18 +1,18 @@
 Name:  sbcl
 State: T (stopped)
 SleepAVG:      88%
-Tgid:  13546
-Pid:   13546
+Tgid:  13598
+Pid:   13598
 PPid:  8211
 TracerPid:     0
 Uid:   1000    1000    1000    1000
 Gid:   1000    1000    1000    1000
 FDSize:        256
 Groups:        4 6 20 24 27 29 44 109 111 114 1000
-VmSize:          886848 kB
+VmSize:           33624 kB
 VmLck:        0 kB
-VmRSS:     9160 kB
-VmData:          860116 kB
+VmRSS:     8804 kB
+VmData:            6900 kB
 VmStk:        8 kB
 VmExe:      104 kB
 VmLib:     1416 kB
@@ -22,7 +22,7 @@
 ShdPnd:        0000000000000000
 SigBlk:        0000000000000000
 SigIgn:        0000000000000000
-SigCgt:        0000000b000034fe
+SigCgt:        0000000f000034fe
 CapInh:        0000000000000000
 CapPrm:        0000000000000000
 CapEff:        0000000000000000

There are 2 gotchas:

• The patch is at the wrong place (linux-os.c) because there is no abstraction layer for 'get memory' and 'handle memory fault'.
• There is voodoo going on with the dynamic space: if I just allocate one page on every fault the system gets strange errors like 'file is not there' when renaming. If I allocate 10 pages on every fault (like in the patch) it works. This seems to be that sometimes the system needs the memory without first touching it. With the cmucl lazy patches this was due to system calls pointing to unallocated memory, but this does not seems the case here. I should investigate this further.

The good news is that the system build itself a dozen times without problems, and passes all the tests I've thrown it.