274 lines
10 KiB
Markdown
274 lines
10 KiB
Markdown
# Entity Removal & Deletion Lifecycle (Brainstorm & Rework)
|
|
|
|
> [!WARNING]
|
|
> **Status: Needs Rework**
|
|
> The entity deletion and removal logic below was extracted as-is from [`Game/Plans/02_Entity_Allocation_And_Lifecycle.md`](file:///w:/Classified/Juliet/Game/Plans/02_Entity_Allocation_And_Lifecycle.md) for future brainstorming.
|
|
> The original swap-and-pop approach required complex mutual back-pointer fixups across both base and derived memory buffers.
|
|
|
|
---
|
|
|
|
## 1. Brainstorming Notes & Alternative Approaches
|
|
|
|
The original design relied on `VectorArena::RemoveAtFast` (swap-and-pop) for both `Entities` and `by_type[kind].arena`. While this maintained dense contiguous memory, swapping arbitrary elements in memory invalidated pointers in both directions, requiring runtime pointer patching (`derived->Base = movedEntity; base->Derived = movedComponent`).
|
|
|
|
### Ideas for Simpler & More Robust Alternatives:
|
|
|
|
1. **Active Flag / Tombstone (`is_active` bool)**:
|
|
- Keep deleted entities in place rather than moving or swapping them.
|
|
- Simply toggle `entity->is_active = false;`.
|
|
- Iterators and simulation loops skip inactive entities (`if (!entity->is_active) continue;`).
|
|
- Pointers to entities and derived components remain stable for their entire lifetime.
|
|
|
|
2. **Intrusive Free List for Slot Recycling**:
|
|
- Instead of shifting memory on deletion, vacant slots are linked into an intrusive free list (`first_free_index`).
|
|
- When an entity is destroyed:
|
|
- `entity->is_active = false;`
|
|
- Overwrite unused slot memory with `next_free_index`.
|
|
- When allocating a new entity:
|
|
- Check if `first_free_index != indexMax`.
|
|
- Pop from the free list in $O(1)$; otherwise push back to the end of the arena.
|
|
- Zero pointer invalidation, zero memmove/memcpy overhead during deletion.
|
|
|
|
3. **Stable Generational Handles / Slot Map**:
|
|
- If external systems need references to entities that might be deleted, combine the free list with a generational counter to detect stale lookups safely.
|
|
|
|
4. **Deferred Compaction / Garbage Collection**:
|
|
- If contiguous packing is strictly necessary for SIMD/cache optimization, defer compaction to a designated level load or scene transition rather than doing it per-frame on individual deletes.
|
|
|
|
---
|
|
|
|
## 2. Extracted Original Removal Logic (As-Is from Plan 2)
|
|
|
|
### 2.1 In-Memory Removal vs. Immediate Disk Deletion Hazards
|
|
In game development, deleting an entity in the editor or during gameplay must **never synchronously invoke disk deletion**:
|
|
1. **Frame Rate Stutters:** Blocking on synchronous OS filesystem APIs (`DeleteFileA`) introduces multisecond frame freezes.
|
|
2. **Transactional Safety:** If the editor crashes or the user exits without saving, disk modifications cannot be rolled back.
|
|
3. **Undo/Redo Support:** An editor action stack must allow recovering deleted entities before changes are permanently committed to disk.
|
|
|
|
Therefore, Juliet enforces a strict separation:
|
|
- **Immediate in-memory destruction:** Releases the entity from active simulation and registers its identifier in `World::PendingDeletions`.
|
|
- **Deferred disk deletion:** Executed strictly during explicit `SaveWorld` operations.
|
|
|
|
### 2.2 Fast In-Memory Removal (`RemoveAtFast`) & Mutual Pointer Fixup
|
|
`VectorArena::RemoveAtFast` utilizes swap-and-pop: the element at the target index is replaced by the last element in the vector, and `Count` is decremented.
|
|
|
|
```cpp
|
|
void RemoveAtFast(index_t index)
|
|
{
|
|
Assert(Arena);
|
|
Assert(index < Count);
|
|
Assert(Count > 0);
|
|
|
|
Type* elementAdr = DataFirst + index;
|
|
|
|
if (DataLast != elementAdr)
|
|
{
|
|
Swap(DataLast, elementAdr);
|
|
}
|
|
|
|
--DataLast;
|
|
--Count;
|
|
}
|
|
```
|
|
|
|
#### The Pointer Invalidation Problem:
|
|
When `Entity A` (at `index`) is swapped with `Entity Z` (at `DataLast`), the physical address of `Entity Z` changes from `DataLast` to `elementAdr`.
|
|
If `Entity Z` has a derived struct `derivedZ`, `derivedZ->Base` previously pointed to `DataLast`. After `RemoveAtFast`, `derivedZ->Base` points to garbage or the freed slot!
|
|
|
|
#### The Pointer Fixup Protocol:
|
|
To preserve the mutual back-pointer invariant, `DestroyEntity` explicitly fixes up the swapped entity's derived back-pointer:
|
|
|
|
```cpp
|
|
void DestroyEntity(EntityManager& manager, EntityID id)
|
|
{
|
|
Entity* baseArray = manager.Entities.DataPtr();
|
|
size_t count = manager.Entities.Size();
|
|
|
|
size_t targetIndex = indexMax;
|
|
for (size_t i = 0; i < count; ++i)
|
|
{
|
|
if (baseArray[i].ID == id)
|
|
{
|
|
targetIndex = i;
|
|
break;
|
|
}
|
|
}
|
|
|
|
if (targetIndex == indexMax)
|
|
{
|
|
return;
|
|
}
|
|
|
|
Entity* targetEntity = &baseArray[targetIndex];
|
|
Class* classPtr = targetEntity->derived_kind;
|
|
Assert(classPtr != nullptr);
|
|
|
|
// 1. Remove derived component from typed array via swap-and-pop
|
|
RemoveDerivedComponent(manager, classPtr, targetEntity->derived);
|
|
|
|
// 2. Remove base entity via swap-and-pop in VectorArena
|
|
bool wasLast = (targetIndex == count - 1);
|
|
manager.Entities.RemoveAtFast(targetIndex);
|
|
|
|
// 3. Pointer fixup: If an element was swapped into targetIndex, fix its back-pointer!
|
|
if (!wasLast && targetIndex < manager.Entities.Size())
|
|
{
|
|
Entity* movedEntity = &manager.Entities[targetIndex];
|
|
auto* derivedTemp = reinterpret_cast<entity_template*>(movedEntity->derived);
|
|
Assert(derivedTemp != nullptr);
|
|
derivedTemp->base = movedEntity;
|
|
}
|
|
}
|
|
```
|
|
|
|
### 2.3 O(1) Component Removal in `typed_entity_array` via Swap-and-Pop
|
|
To keep derived components packed contiguously for SIMD/cache iteration:
|
|
1. Locate the component's index within `by_type[kind].arena`. Because components have uniform stride `classPtr->size_of`:
|
|
$$\text{componentIndex} = \frac{\text{reinterpret\_cast<uint8*>(derivedPtr)} - \text{reinterpret\_cast<uint8*>(typedArray.array)}}{\text{classPtr->size\_of}}$$
|
|
2. If the component is not the last one in the typed arena:
|
|
- Copy the last component into the slot occupied by the deleted component.
|
|
- Update the moved component's `Base->Derived` pointer to point to its new slot.
|
|
3. Decrement `typedArray.count`.
|
|
4. Pop the arena allocation if it was the top of the stack, or decrement count to mark slot reclamation.
|
|
|
|
```cpp
|
|
void RemoveDerivedComponent(EntityManager& manager, Class* classPtr, DerivedType derivedPtr)
|
|
{
|
|
Assert(classPtr != nullptr);
|
|
Assert(derivedPtr != nullptr);
|
|
|
|
typed_entity_array& typedArray = manager.by_type[classPtr->kind];
|
|
Assert(typedArray.count > 0);
|
|
Assert(typedArray.array != nullptr);
|
|
|
|
size_t stride = classPtr->size_of;
|
|
auto* targetByte = reinterpret_cast<uint8*>(derivedPtr);
|
|
auto* firstByte = reinterpret_cast<uint8*>(typedArray.array);
|
|
|
|
size_t componentIndex = static_cast<size_t>(targetByte - firstByte) / stride;
|
|
Assert(componentIndex < typedArray.count);
|
|
|
|
size_t lastIndex = typedArray.count - 1;
|
|
if (componentIndex != lastIndex)
|
|
{
|
|
uint8* lastByte = firstByte + (lastIndex * stride);
|
|
|
|
// Copy last component data into target slot
|
|
MemCopy(targetByte, lastByte, stride);
|
|
|
|
// Fixup the base pointer of the moved component
|
|
auto* movedDerived = reinterpret_cast<entity_template*>(targetByte);
|
|
Assert(movedDerived->base != nullptr);
|
|
movedDerived->base->derived = targetByte;
|
|
}
|
|
|
|
typedArray.count -= 1;
|
|
if (typedArray.count == 0)
|
|
{
|
|
typedArray.array = nullptr;
|
|
}
|
|
}
|
|
```
|
|
|
|
### 2.4 Tracking Deletions in `World::PendingDeletions`
|
|
In `World.h`, the `World` struct is extended with a pending deletions container:
|
|
|
|
```cpp
|
|
struct World
|
|
{
|
|
Arena* WorldArena = nullptr;
|
|
EntityManager* EntityManager = nullptr;
|
|
VectorArena<EntityID, 1024> PendingDeletions;
|
|
};
|
|
```
|
|
|
|
When an entity is deleted in the world:
|
|
```cpp
|
|
void RemoveWorldEntity(World& world, EntityID id)
|
|
{
|
|
Assert(world.EntityManager != nullptr);
|
|
|
|
// Record pending disk deletion
|
|
world.PendingDeletions.PushBack(id);
|
|
|
|
// Destroy in memory immediately
|
|
DestroyEntity(*world.EntityManager, id);
|
|
}
|
|
```
|
|
|
|
### 2.5 Disk File Cleanup during `SaveWorld`
|
|
During `SaveWorld`, before saving modified entities, the engine iterates over `world.PendingDeletions` and removes their associated `.jasset` files:
|
|
|
|
```cpp
|
|
void ProcessPendingDeletions(World& world, NonNullPtr<Arena> scratchArena)
|
|
{
|
|
for (size_t i = 0; i < world.PendingDeletions.Size(); ++i)
|
|
{
|
|
EntityID id = world.PendingDeletions[i];
|
|
|
|
// Format relative asset path: Assets/Entities/{ID}.jasset
|
|
char filenameBuffer[64];
|
|
juliet_snprintf(filenameBuffer, sizeof(filenameBuffer), "Entities/%llu.jasset", id);
|
|
|
|
String assetPath = GetAssetPath(scratchArena, WrapString(filenameBuffer));
|
|
|
|
if (PlatformDeleteFile(assetPath))
|
|
{
|
|
Log(LogLevel::Message, LogCategory::Game, "Deleted entity asset: %s", CStr(assetPath));
|
|
}
|
|
}
|
|
|
|
world.PendingDeletions.Clear();
|
|
}
|
|
```
|
|
|
|
### 2.6 Associated Swap-and-Pop Unit Test
|
|
```cpp
|
|
void TestSwapAndPopPointerFixup()
|
|
{
|
|
Log(LogLevel::Message, LogCategory::Game, "[UnitTest] Running TestSwapAndPopPointerFixup...");
|
|
|
|
TempArena tempArena = scratch_begin(nullptr, 0);
|
|
|
|
World testWorld{};
|
|
InitWorld(&testWorld, tempArena.Arena);
|
|
InitEntityManager(&testWorld);
|
|
EntityManager& manager = *testWorld.EntityManager;
|
|
|
|
// Allocate 3 entities: E0, E1, E2
|
|
Inert* e0 = MakeEntity<Inert>(manager, 1.0f, 0.0f, 0.0f);
|
|
Inert* e1 = MakeEntity<Inert>(manager, 2.0f, 0.0f, 0.0f);
|
|
Inert* e2 = MakeEntity<Inert>(manager, 3.0f, 0.0f, 0.0f);
|
|
|
|
EntityID id0 = e0->base->ID;
|
|
EntityID id1 = e1->base->ID;
|
|
EntityID id2 = e2->base->ID;
|
|
|
|
Assert(manager.Entities.Size() == 3);
|
|
|
|
// Delete middle entity E1 (triggers swap with E2)
|
|
DestroyEntity(manager, id1);
|
|
|
|
Assert(manager.Entities.Size() == 2);
|
|
|
|
// Verify E2's mutual back-pointers are still completely intact
|
|
Entity* remaining0 = &manager.Entities[0];
|
|
Entity* remaining1 = &manager.Entities[1];
|
|
|
|
Assert(remaining0->ID == id0);
|
|
Assert(remaining1->ID == id2);
|
|
|
|
auto* derived0 = reinterpret_cast<entity_template*>(remaining0->derived);
|
|
auto* derived1 = reinterpret_cast<entity_template*>(remaining1->derived);
|
|
|
|
Assert(derived0->base == remaining0);
|
|
Assert(derived1->base == remaining1);
|
|
|
|
ShutdownEntityManager();
|
|
ShutdownWorld(&testWorld);
|
|
|
|
scratch_end(tempArena);
|
|
Log(LogLevel::Message, LogCategory::Game, "[UnitTest] PASSED: TestSwapAndPopPointerFixup");
|
|
}
|
|
```
|