allocate: thread-local allocations.

[iskamag]

No description provided.

[froggey]

Due to cross-cpu migration I don't think this is SMP-safe at the moment. Each cpu has it's own tlab bump/limit, but there's no synchronization here ensuring the correct limit is compared against if a migration occurs between the limit load and bump update.

0. %do-allocate-from-general-area called
1. Load fs:tlab-limit
2. Migration occurs. This switches cpu and changes fs, makes the loaded limit stale
3. Fetch/inc fs:tlab-base
4. Compare new bump pointer against the stale limit (this is effectively another cpu's limit)

Same issue on arm64.

Possible solutions:

1. put tlab in the thread object instead of the cpu. means we have more tlabs and memory pressure increases but then they are truly thread-local. keeping the allocation meters accurate might be a pain here. the advantage is that no synchronization is needed at all on the hot path and a thread's thread object is inherently stable.
2. disable interrupts on the hot path? ugly, scary. disabling interrupts is dangerous. Very tricky when also needing to write arbitrary heap memory
3. keep the tlab on the cpu, but go back to using atomics and load the cpu struct at the start of %do-allocate-from-general-area, this way it's consistent the whole way through
4. something else? secret fourth thing?

Aside from this, I really like the approach. It's surprisingly non-invasive

[froggey]

arm64 build completes successfully with my changes btw

[iskamag]

TSX would've fixed this.

Anyway, I think option 1 is better. Since Claimore would be thread-local...

[froggey]

changes look good, but we still have cpu-local accounting (bytes-consed, cons-allocation-count, cons-fast-path-hits, general-allocation-count, general-fast-path-hits), are they worth keeping/do they have the same race condition as allocation did?

[iskamag]

technically there's a race but on x86 they should add up to the same number with a tiny skew, while ARM would lose counts. However there is no real data corruption.

Whether to keep those counters is up to you.

[iskamag]

i.e x86 could have 7+3 allocs instead of 6+4

[froggey]

I don't think the GC relies on these counters, so happy to merge this. Maybe we can revisit accuracy at a later time