Any program that you write, any application that you run, the chances that it internally has some sort of dynamic memory allocation are extremely high. And the chances that the programmer who wrote that application handled it themselves are extremely, extremely low.

Modern programming languages like Python, JavaScript, even Golang handle memory allocation and freeing themselves. Python is famous for its arena model, others have their own way of handling memory. But at the lowest rungs of any language's runtime, one thing that's common amongst all languages is that they interact with the machine's memory. Some aspects of memory are already known while a program or an app is being written. Size of a variable, whatever you assign it. That's the minority part of it. The majority is what you don't know.

You don't know the size of a request that's coming into your app while you're writing it. Come to think of it, these aren't things that anyone thinks of while writing their app. Now, with the advent of LLM generated code, the thought that goes into the "arcane" concept of dynamic memory allocation is just so minuscule.

But this is what runs almost everything we've ever interacted with.

Now, C has a bunch of allocators, out of which the most famous one, which you, reader, obviously know, has two functions called malloc and free (malloc has cousins like calloc and realloc, but we aren't going to be talking about them right now). The allocator gets a bunch of malloc and free calls and has to respond to each of them immediately.

Fundamental requirements that an allocator must satisfy:

  1. Handling arbitrary request sequences.
  2. Making immediate responses to requests.
  3. Using only heap memory.
  4. Aligning blocks (ensuring that you offer a standardized block of memory that's capable of handling any type of data object).
  5. Not modifying any allocated blocks.

The meat of this article is around point 4, about how malloc/free works, and how macOS (M3, 64-bit ARM) and Ubuntu (64-bit, Dockerized) handle memory allocation and fragmentation differently.

Any practical allocator needs some data structure that allows it to distinguish block boundaries and to distinguish between allocated and free blocks. Most allocators embed this information in the blocks themselves.

(Generic) anatomy of a block

Every block has three sections: a header, a payload and padding.

[ header ][ payload ............ ][ padding ]
          ^
          malloc returns this

Block sizes are multiples of 8 or 16. The header has information such as the block size, an allocated flag and other, OS-dependent information (we'll see how Ubuntu adds more to the header and how macOS handles this). The payload has whatever you want to put in the memory block. And padding is whatever space remains between the end of your payload and the end of the block.

The test

Everything from here on runs on two setups:

  • macOS: an M3 MacBook on macOS 26.6. I stopped a test program inside malloc with lldb, and the frame it landed in was xzm_malloc_zone_malloc_type_malloc in libsystem_malloc. xzm is xzone malloc, Apple's newer allocator, and it's what serves ordinary malloc calls there now.
  • Linux: Ubuntu 26.04.1 LTS (glibc 2.43) in a Docker container on the same machine, so it's aarch64.

A program that allocates three 5-byte blocks, writes "hello" and "world" into the first two, and then prints the bytes.

macOS:

a=0x1009fdb00  b=0x1009fdb10  c=0x1009fdb20   malloc_size(a)=16

0x1009fdb00   68 65 6c 6c   'h' 'e' 'l' 'l'    <- a starts here
0x1009fdb04   6f 00 00 00   'o' + padding
0x1009fdb08   00 00 00 00   padding
0x1009fdb0c   00 00 00 00   padding
0x1009fdb10   77 6f 72 6c   'w' 'o' 'r' 'l'    <- b starts here
0x1009fdb14   64 00 00 00   'd' + padding
0x1009fdb18   00 00 00 00   padding
0x1009fdb1c   00 00 00 00   padding

The blocks are exactly 16 bytes apart and the bytes go payload, padding, payload, padding. There's no header anywhere. Block b starts on the byte right after a's padding ends.

glibc:

a=0xaaaad46ed010  b=0xaaaad46ed030  c=0xaaaad46ed050   (32 apart)

0xaaaad46ed000: 00 00 00 00 00 00 00 00    prev_size (unused, see below)
0xaaaad46ed008: 21 00 00 00 00 00 00 00    a's size field = 0x21
0xaaaad46ed010: 68 65 6c 6c 6f 00 00 00    "hello" + padding   <- a
0xaaaad46ed018: 00 00 00 00 00 00 00 00    padding
0xaaaad46ed020: 00 00 00 00 00 00 00 00    padding / b's prev_size
0xaaaad46ed028: 21 00 00 00 00 00 00 00    b's size field = 0x21
0xaaaad46ed030: 77 6f 72 6c 64 00 00 00    "world" + padding   <- b
0xaaaad46ed038: 00 00 00 00 00 00 00 00    padding
0xaaaad46ed040: 00 00 00 00 00 00 00 00    padding / c's prev_size
0xaaaad46ed048: 21 00 00 00 00 00 00 00    c's size field = 0x21

Finding

macOS has NO header at all. The blocks sit exactly 16 bytes apart. "hello" starts on the very first byte of its block, 5 bytes of payload and 11 bytes of padding. Then comes "world". xzone groups blocks of the same size into runs and keeps its bookkeeping in a separate metadata region. This hides the allocator's internals from an attacker in case of a buffer overflow attack.

Ubuntu does it the old-fashioned way. Right before every payload there's an 8-byte header. Out of a 32-byte block, that's 8 bytes that are already gone. In my test, the header contained 0x21. 0x20 is 32, the block size, i.e., a block is 32 bytes in size (the extra 1 is a flag, more on that below). The remaining 24 bytes held 5 bytes for "hello" ("world" in the next block) and 19 bytes of padding.

┌─────────┬───────────────────┬───────────────────┐
│         │       macOS       │      Ubuntu       │
├─────────┼───────────────────┼───────────────────┤
│ Blocks  │ 2 × 16 = 32 bytes │ 2 × 32 = 64 bytes │
├─────────┼───────────────────┼───────────────────┤
│ Header  │ 0                 │ 2 × 8 = 16 bytes  │
├─────────┼───────────────────┼───────────────────┤
│ Payload │ 2 × 5 = 10 bytes  │ 2 × 5 = 10 bytes  │
├─────────┼───────────────────┼───────────────────┤
│ Padding │ 2 × 11 = 22 bytes │ 2 × 19 = 38 bytes │
└─────────┴───────────────────┴───────────────────┘

The same 5-byte malloc calls use twice as much memory on Ubuntu as on macOS.

Inside glibc's 8-byte header

bit:  63 ........ 48 47 ............ 4   3   2   1   0
      [  always 0  ][   chunk size    ][0 ][A ][M ][P ]
  • Bit 0, P (PREV_INUSE): is the previous chunk allocated?
  • Bit 1, M (IS_MMAPPED): did this chunk come from its own mmap instead of the heap?
  • Bit 2, A (NON_MAIN_ARENA): does it belong to one of the per-thread arenas rather than the main heap?
  • Bit 3 is always 0, because sizes are multiples of 16 and nobody has claimed it.
  • Bits 48 to 63 are always 0 in practice. A user process on 64-bit Linux gets 2⁴⁷ bytes of address space on x86-64 and usually 2⁴⁸ on arm64, so no chunk can be big enough to need them.

To me, the most interesting one is bit 0. This single bit makes coalescing two free memory chunks so much simpler than on macOS.

Fragmentation

The "padding" that we've seen the allocator add is also called internal fragmentation. You asked for 5 bytes, you got a 16-byte block on macOS (11 wasted) or a 32-byte block on Ubuntu (19 wasted).

Then, we have external fragmentation. External fragmentation is what happens when there are enough free bytes in total to fulfill a malloc request, but not in one contiguous block. So even though we have the memory, we can't use it to serve the request.

heap:  [used 16][free 16][used 16][free 16][used 16][free 16]
free bytes: 48
malloc(32): no hole is 32 bytes, so the heap has to grow

How each OS deals with it

Ubuntu merges. When you free a chunk, glibc looks at both neighbours. First it checks bit 0 of its own header. If the P bit is clear, the previous chunk is free, so glibc uses prev_size to find it and merges the two. Then it checks the next chunk, which needs a look at the header of the chunk 2 chunks over.

Consider the following example:

Say you free B, and C comes right after it:

[ A ][ B ][ C ][ D ]
       ^ freeing this
  1. glibc reads B's size from B's header and jumps forward by that much. That lands on C.
  2. C's header gives C's size, but nothing in it says whether C is free.
  3. glibc jumps forward by C's size and lands on D.
  4. It checks D's P bit, which answers "is the chunk before me (C) in use?"
    • P = 0: C is free, so B and C merge into one chunk.
    • P = 1: C is in use, so there's no forward merge.

macOS doesn't merge at all. Blocks of one size live together in a run, and a freed slot only ever serves its own size class. Nothing gets glued into a bigger block. The upside is that malloc never has to search for a fit, and tiny objects sit packed next to each other with no header.

I allocated 100,000 blocks of 200 bytes, then a small guard block after them. Then I freed either every other block (checkerboard) or all of them (contiguous). Then I allocated 50,000 blocks of 400 bytes and looked at how much the heap grew in that last step.

┌───────────────┬──────────────┬────────────┐
│               │ checkerboard │ contiguous │
├───────────────┼──────────────┼────────────┤
│ Ubuntu        │ +20,480 KB   │ +0 KB      │
├───────────────┼──────────────┼────────────┤
│ macOS         │ +20,480 KB   │ +20,480 KB │
└───────────────┴──────────────┴────────────┘

Checkerboard is a tie. glibc has nothing to merge, and neither will put a 400-byte request in a smaller slot. In the contiguous case, glibc merged the 100,000 freed chunks into one huge region, and the 400-byte blocks went there. This resulted in no heap growth. macOS, on the other hand, grew exactly the way it did in the checkerboard case.

macOS favours the heap growing (which can lead to more usage of swap than you'd see in a stock Ubuntu machine).

A single C program running on two separate allocators on a single machine. But the way the allocators work is completely different.

So, while macOS favours speed, security and small objects, Ubuntu favours reuse.