diff --git a/.agent/rules/coding-guidelines.md b/.agent/rules/coding-guidelines.md index f8fa913..e3934f1 100644 --- a/.agent/rules/coding-guidelines.md +++ b/.agent/rules/coding-guidelines.md @@ -5,7 +5,7 @@ trigger: always_on Code compiles with all warning active and warning as errors. use static_cast or reinterpret_cast but not parenthesis for casting. No exceptions -Use [[nodiscard]] +Use [[nodiscard]] when risk of memory leak (anything returning pointer) auto is allowed but when its a pointer add the * and when reference adds the & Member variable are CamelCase Types are CamelCase diff --git a/.agent/skills/cpp_game_engine_programmer/SKILL.md b/.agent/skills/cpp_game_engine_programmer/SKILL.md index c1b4df7..fb61342 100644 --- a/.agent/skills/cpp_game_engine_programmer/SKILL.md +++ b/.agent/skills/cpp_game_engine_programmer/SKILL.md @@ -11,7 +11,7 @@ trusted_commands: You are a senior engine architect for the Juliet project. Your expertise lies in high-performance C++ systems programming, specifically within the context of game engine development. You value performance, memory efficiency, and maintainability. ## Coding Guidelines -You must follow the project's `coding-guidelines.md` (always loaded). Do not deviate from any rule. Pay special attention to `static_cast`/`reinterpret_cast` (never C-style casts), `auto*`/`auto&`, mandatory braces, and `[[nodiscard]]`. +You must follow the project's `coding-guidelines.md` (always loaded). Do not deviate from any rule. Pay special attention to `static_cast`/`reinterpret_cast` (never C-style casts), `auto*`/`auto&`, mandatory braces, and `[[nodiscard]]` when needed. ## Focus Areas 1. **High Performance**: Always consider cache locality and CPU cycle cost. diff --git a/.agent/skills/debugger_programmer/SKILL.md b/.agent/skills/debugger_programmer/SKILL.md index 9058c9f..d10d847 100644 --- a/.agent/skills/debugger_programmer/SKILL.md +++ b/.agent/skills/debugger_programmer/SKILL.md @@ -11,7 +11,7 @@ trusted_commands: You are a senior engine architect for the Juliet project. Your expertise lies in debugging C++ game engines. You add logs and use debug tricks to find the root cause of issues. ## Coding Guidelines -You must follow the project's `coding-guidelines.md` (always loaded). Do not deviate from any rule. Even in debug/diagnostic code, use proper casts, braces, and `[[nodiscard]]`. +You must follow the project's `coding-guidelines.md` (always loaded). Do not deviate from any rule. Even in debug/diagnostic code, use proper casts, braces, and `[[nodiscard]]` when needed. ## Workflows - **Building**: Use the `/build` workflow to compile the project. diff --git a/Game/Plans/01_Serialization_And_Text_Archive.md b/Game/Plans/01_Serialization_And_Text_Archive.md new file mode 100644 index 0000000..a57a087 --- /dev/null +++ b/Game/Plans/01_Serialization_And_Text_Archive.md @@ -0,0 +1,1321 @@ +# Juliet Game Engine: Serialization Core & .jasset Text Archive +## Technical Specification & Implementation Plan + +**Document ID**: SPEC-001-SERIALIZATION-CORE +**Component**: Juliet Engine Core / Asset Pipeline +**Target Architecture**: Juliet Game Engine (C++20, x64, D3D12) +**File Location**: `Game/Plans/01_Serialization_And_Text_Archive.md` + +--- + +## 1. Executive Summary & Architecture Goals + +### 1.1 Context & Current Limitations +The Juliet game engine previously utilized packed binary records for level and entity serialization (`Assets/world.bin`, `WorldFileHeader`, `WorldEntityDiskRecord`). While packed binary formats are compact, they present severe architectural roadblocks during collaborative development: +1. **Merge Incompatibility**: Binary assets cannot be merged or diffed across version control systems (Git / Perforce). Concurrent edits by multiple designers or engineers result in unresolvable binary conflicts and data loss. +2. **Schema Rigidity**: Adding, removing, or reordering a single struct field invalidates all previously serialized binary blobs unless complex, manual byte-offset mapping tables are maintained. +3. **Opacity**: Programmers and technical artists cannot inspect, debug, or patch asset properties in a standard text editor. + +### 1.2 Architectural Goals +The `.jasset` text archive framework is engineered to replace legacy binary blobs with a robust, human-readable, diff-friendly property serialization pipeline while strictly adhering to Juliet's performance and memory constraints: +- **Zero Dynamic Heap Allocations**: All parsing, tokenization, formatting, and buffer transformations execute entirely within Juliet memory arenas (`Arena`, `TempArena`, `scratch_begin` / `scratch_end`). Standard library containers (`std::string`, `std::vector`, `std::map`) and raw heap allocators (`malloc`, `new`) are forbidden. +- **Zero-Copy In-Memory Tokenization**: Files are loaded into arena memory once via `LoadFile`. The tokenizer parses properties into lightweight slices represented by Juliet's `String` (`char* Str; size_t Size;`), referencing existing file buffer bytes without string replication or null-termination overhead. +- **$O(1)$ Property Lookup via Compile-Time & Runtime CRC32**: Property keys in text files are hashed once during tokenization into 32-bit CRC values. In code, lookups utilize compile-time hashed literals (`operator""_crc32`). Property resolution reduces to a single 32-bit integer comparison, avoiding string comparison overhead in serialization loops. +- **Symmetric Single-Function Serialization**: A single `Serialize` implementation per entity or data structure handles both Save and Load paths, guaranteeing that write and read schemas never diverge. +- **Graceful Forward/Backward Compatibility**: Missing keys during load automatically preserve struct default values. Unknown keys present in newer asset files are safely ignored without parse failure. +- **Two-Tier Decoupled Versioning**: Core engine entity properties (`kEntityBaseVersion`) and derived gameplay class properties (`Class::Version`) are versioned independently. Engine-level updates never bump derived entity class versions. +- **Post-Serialization In-Place Migration**: Deprecated fields can be read through specialized primitives (`ReadDeprecated*`) to migrate legacy data structures in-place upon loading, producing cleaned, modern schemas on subsequent saves. +- **Zero Exceptions & Total Warning Cleanliness**: Fully conforming to Juliet coding guidelines: strict assertions (`Assert`), `[[nodiscard]]`, `auto*`/`auto&`, mandatory braces, and `static_cast`/`reinterpret_cast`. + +--- + +## 2. The `.jasset` Text Format Specification + +### 2.1 Grammar & Structural Rules +The `.jasset` format uses a line-oriented, key-value property hierarchy designed for visual clarity and clean Git diffs. + +```ebnf +AssetFile ::= { CommentLine | EmptyLine | PropertyDeclaration } ; +CommentLine ::= ( "#" | "//" ) { Character } LineEnding ; +EmptyLine ::= { Whitespace } LineEnding ; +PropertyDeclaration ::= KeyHeader LineEnding ValueBlock ; +KeyHeader ::= ";" { Whitespace } Identifier ; +ValueBlock ::= { ValueLine LineEnding } ; +ValueLine ::= { Whitespace } ValueString { Whitespace } ; +LineEnding ::= "\r\n" | "\n" ; +Identifier ::= [a-zA-Z_][a-zA-Z0-9_]* ; +``` + +1. **Property Key Header (`;\n`)**: + - Every property declaration begins with a semicolon `;` as the first non-whitespace character. + - The semicolon is followed by optional whitespace and a CamelCase identifier: `; PropertyName`. + - Keys are case-sensitive and must be valid C++ identifier tokens. +2. **Value Block**: + - The line immediately following the property header contains the property's serialized payload. + - For multi-line values or arrays, lines continue until the next property header `;` or end-of-file. +3. **Comments & Ignored Tokens**: + - Any line starting with `#` or `//` (after optional leading whitespace) is treated as a comment. + - Empty lines and lines containing only whitespace are ignored. +4. **Line Termination Handling**: + - The parser natively accepts both Windows (`\r\n`) and POSIX (`\n`) line endings. + - Trailing carriage returns (`\r`) are automatically stripped during tokenization. +5. **Whitespace Tolerance**: + - Leading and trailing spaces and horizontal tabs (`\t`) on both key and value lines are stripped during tokenization. + +### 2.2 Formatting Specifications & Examples + +#### Scalar Types +- **Floating Point (`float`, `float32`)**: Formatted via `%f` (default 6 decimal digits) or `%.9g` for full single-precision round-trip fidelity. + ``` + ; Health + 100.000000 + + ; Mass + 14.250000 + ``` +- **32-Bit Signed Integer (`int32`)**: Formatted as decimal integer `%d`. + ``` + ; AmmoCount + 45 + + ; TeamIndex + -1 + ``` +- **64-Bit Unsigned Integer (`uint64`, `EntityID`)**: Formatted as a 16-character padded hexadecimal integer prefixed with `0x`. Hexadecimal ensures 64-bit handle readability and exact bit-pattern preservation. + ``` + ; EntityID + 0x00000000DEADBEEF + + ; LayerMask + 0x0000000000000001 + ``` +- **Boolean (`bool`)**: Formatted as `true` or `false`. For backwards tolerance, the parser also accepts `1` and `0`. + ``` + ; IsStatic + true + + ; CastShadows + false + ``` + +#### Vector Types +- **3D Vector (`Vector3` / `float x, y, z`)**: Formatted as three space-delimited floating-point values on a single line. + ``` + ; Position + 10.500000 0.000000 -25.250000 + + ; Scale + 1.000000 1.000000 1.000000 + ``` +- **2D Vector (`Vector2` / `float x, y`)**: Formatted as two space-delimited floating-point values. + ``` + ; UVOffset + 0.000000 0.500000 + ``` + +#### String Types +- **String (`String` / `String8`)**: + - Strings without spaces can be serialized as raw text tokens. + - Strings containing spaces or symbols are enclosed in double quotes `"..."`. + ``` + ; AssetName + Character_Mesh_Hero + + ; DisplayName + "Grand Citadel Knight" + ``` + +#### Comprehensive Entity Asset File Example (`Entity_01.jasset`) +``` +# Juliet Game Engine Asset File +# Generated automatically by AssetPipeline. Do not manually corrupt keys. + +; AssetType +Entity + +; BaseVersion +1 + +; EntityID +0x0000000000000042 + +; Kind +Inert + +; Position +100.000000 25.500000 -50.000000 + +; ClassVersion +2 + +; MeshInstance +4 + +; MaterialOverride +"Materials/M_Granite_Polished" + +; IsVisible +true +``` + +--- + +## 3. The In-Memory Zero-Copy Parser + +### 3.1 Memory Layout & Property Nodes +To eliminate heap fragmentation and per-object allocation overhead during level loading, the parser loads the entire `.jasset` file into contiguous arena memory via `LoadFile`. The tokenizer then constructs a flat array of `TextPropertyNode` structures on a `TempArena`. + +```cpp +struct TextPropertyNode +{ + uint32 KeyCRC; + String Value; + bool Consumed; +}; +``` + +- `KeyCRC`: The 32-bit CRC hash of the trimmed property name. +- `Value`: A `String` (`String8`) struct containing a `char* Str` pointer directly into the file buffer and `size_t Size`. No new string allocations are performed. +- `Consumed`: A boolean flag initialized to `false`. When a property is queried and read via `SerializeProp`, `Consumed` is set to `true`. + +``` +File Buffer in Arena: ++-------------------------------------------------------------------------------+ +| ; Position\n10.0 20.0 30.0\n; Health\n100.0\n | ++-------------------------------------------------------------------------------+ + ^ ^ ^ ^ + | | | | +TextPropertyNode[0]: | | | + KeyCRC: CRC("Position") | | + Value: { Str --------+ , Size: 14 } | + Consumed: true | + | +TextPropertyNode[1]: | + KeyCRC: CRC("Health") | + Value: { Str -------------------------------+ , Size: 5 } + Consumed: true +``` + +### 3.2 Dual-Mode CRC32: Compile-Time & Runtime +Juliet's `CRC32.h` defines `consteval uint32 crc32(const char* str, size_t length)`. However, `consteval` guarantees compilation failure when called on runtime input (such as keys extracted from a text file during parsing). + +The engine requires a unified `Crc32` implementation that functions at runtime for parsed text tokens while retaining `constexpr` / `consteval` capability for compile-time string literals. + +```cpp +// Core/Common/CRC32.h additions +[[nodiscard]] constexpr uint32 Crc32(const char* str, size_t length) +{ + Assert(str != nullptr || length == 0); + const char* p = str; + uint32 crc = ~0U; + while (length--) + { + crc = details::crc32_tab[(crc ^ static_cast(*p++)) & 0xFF] ^ (crc >> 8); + } + return crc ^ ~0U; +} + +[[nodiscard]] constexpr uint32 Crc32(String str) +{ + return Crc32(str.Str, str.Size); +} + +[[nodiscard]] consteval uint32 operator""_crc32(const char* str, size_t length) +{ + return Crc32(str, length); +} +``` + +### 3.3 Zero-Copy Tokenization Algorithm +The tokenization algorithm scans the memory buffer in a single pass. It first counts property keys to allocate the exact array size on the arena, then populates the `TextPropertyNode` array. + +```cpp +[[nodiscard]] inline String TrimWhitespace(String str) +{ + while (str.Size > 0 && (*str.Str == ' ' || *str.Str == '\t' || *str.Str == '\r' || *str.Str == '\n')) + { + str.Str++; + str.Size--; + } + while (str.Size > 0 && (str.Str[str.Size - 1] == ' ' || str.Str[str.Size - 1] == '\t' || + str.Str[str.Size - 1] == '\r' || str.Str[str.Size - 1] == '\n')) + { + str.Size--; + } + return str; +} + +struct ParsedTextArchive +{ + TextPropertyNode* Nodes = nullptr; + uint32 PropertyCount = 0; +}; + +[[nodiscard]] ParsedTextArchive TokenizeTextArchive(NonNullPtr arena, ByteBuffer fileBuffer) +{ + Assert(fileBuffer.Data != nullptr); + + char* cursor = reinterpret_cast(fileBuffer.Data); + char* end = cursor + fileBuffer.Size; + + // Pass 1: Count properties to allocate exactly on arena + uint32 propertyCount = 0; + char* scan = cursor; + while (scan < end) + { + if (*scan == ';') + { + if (scan == cursor || *(scan - 1) == '\n') + { + propertyCount++; + } + } + scan++; + } + + if (propertyCount == 0) + { + return { nullptr, 0 }; + } + + auto* nodes = ArenaPushArray(arena, propertyCount); + Assert(nodes != nullptr); + + // Pass 2: Extract keys and value slices + uint32 nodeIndex = 0; + scan = cursor; + + while (scan < end && nodeIndex < propertyCount) + { + // Skip leading whitespace / empty lines / comments + while (scan < end && (*scan == '\r' || *scan == '\n' || *scan == ' ' || *scan == '\t')) + { + scan++; + } + + if (scan >= end) + { + break; + } + + // Check for comment line + if (*scan == '#' || (*scan == '/' && scan + 1 < end && *(scan + 1) == '/')) + { + while (scan < end && *scan != '\n') + { + scan++; + } + continue; + } + + // Check for property declaration ';' + if (*scan == ';') + { + scan++; // skip ';' + + // Extract key name + char* keyStart = scan; + while (scan < end && *scan != '\r' && *scan != '\n') + { + scan++; + } + + String rawKey = { .Str = keyStart, .Size = static_cast(scan - keyStart) }; + String key = TrimWhitespace(rawKey); + + // Skip newline after key + while (scan < end && (*scan == '\r' || *scan == '\n')) + { + scan++; + } + + // Extract value block (everything until next ';' at start of line or EOF) + char* valStart = scan; + char* valEnd = scan; + + while (scan < end) + { + if (*scan == ';' && (scan == cursor || *(scan - 1) == '\n')) + { + break; + } + scan++; + valEnd = scan; + } + + String rawVal = { .Str = valStart, .Size = static_cast(valEnd - valStart) }; + String val = TrimWhitespace(rawVal); + + nodes[nodeIndex].KeyCRC = Crc32(key); + nodes[nodeIndex].Value = val; + nodes[nodeIndex].Consumed = false; + nodeIndex++; + } + else + { + // Advance unexpected character + scan++; + } + } + + return { nodes, nodeIndex }; +} +``` + +### 3.4 Key Lookup & Unconsumed Key Audit +Lookup performs a fast linear scan over the contiguous `TextPropertyNode` array. Because typical game entities possess between 5 and 50 properties, a cache-coherent linear scan over contiguous 16-byte structs executes in single-digit nanoseconds, comfortably fitting within CPU L1/L2 data cache. + +```cpp +[[nodiscard]] inline TextPropertyNode* FindProperty(TextPropertyNode* nodes, uint32 count, uint32 keyCRC) +{ + Assert(nodes != nullptr || count == 0); + for (uint32 index = 0; index < count; ++index) + { + if (nodes[index].KeyCRC == keyCRC) + { + nodes[index].Consumed = true; + return &nodes[index]; + } + } + return nullptr; +} + +#if JULIET_DEBUG +inline void AuditUnconsumedProperties(const TextPropertyNode* nodes, uint32 count, const char* contextName) +{ + Assert(nodes != nullptr || count == 0); + for (uint32 index = 0; index < count; ++index) + { + if (!nodes[index].Consumed) + { + LogWarning(LogCategory::Core, + "[%s] Unconsumed or obsolete property detected: CRC 0x%08X (Value: '%.*s')", + contextName, nodes[index].KeyCRC, + static_cast(nodes[index].Value.Size), nodes[index].Value.Str); + } + } +} +#endif +``` + +--- + +## 4. The `archive` Struct & Mode Handling + +### 4.1 Struct Definition & Mode Flags +Juliet's existing `archive` struct in `Core/Common/serialization.h` is restricted to binary offsets and raw arena pointers. We upgrade `archive` into a unified serialization context supporting both text `.jasset` and binary streams. + +```cpp +enum class ArchiveMode : uint8 +{ + SavingText, + LoadingText, + SavingBinary, + LoadingBinary +}; + +struct TextPropertyNode; + +struct archive +{ + Arena* ArenaInstance = nullptr; + IOStream* Stream = nullptr; + TextPropertyNode* Properties = nullptr; + uint32 PropertyCount = 0; + ArchiveMode Mode = ArchiveMode::LoadingText; + uint32 BaseVersion = 0; + uint16 ClassVersion = 0; + + // Legacy binary support fields + void* BasePtr = nullptr; + index_t Offset = 0; + + [[nodiscard]] bool IsLoading() const + { + return Mode == ArchiveMode::LoadingText || Mode == ArchiveMode::LoadingBinary; + } + + [[nodiscard]] bool IsSaving() const + { + return Mode == ArchiveMode::SavingText || Mode == ArchiveMode::SavingBinary; + } + + [[nodiscard]] bool IsText() const + { + return Mode == ArchiveMode::LoadingText || Mode == ArchiveMode::SavingText; + } + + [[nodiscard]] bool IsBinary() const + { + return Mode == ArchiveMode::LoadingBinary || Mode == ArchiveMode::SavingBinary; + } +}; +``` + +### 4.2 Output Stream Formatting via `IOPrintf` +When saving in `ArchiveMode::SavingText`, `archive` outputs directly to an open `IOStream` using `IOPrintf`. + +```cpp +inline void WritePropertyHeader(archive& ar, const char* keyName) +{ + Assert(ar.IsSaving()); + Assert(ar.Stream != nullptr); + Assert(keyName != nullptr); + IOPrintf(ar.Stream, "; %s\n", keyName); +} +``` + +This design provides: +1. Direct stream output with zero heap buffer allocations. +2. Canonical spacing and formatting across all entity serializers. +3. Formatted outputs immediately flushed or buffered according to `IOStreamInterface` configuration. + +--- + +## 5. `SerializeProp` API & Implementation + +### 5.1 Unified Serialization Idiom +The `SerializeProp` function family encapsulates both loading and saving behind a single call. If an asset file lacks a given property (e.g. an older asset file loaded by newer code), `SerializeProp` returns `false` during loading, and the destination variable retains its existing default value. + +```cpp +#define SERIALIZE_PROP(ar, var) SerializeProp((ar), #var, #var##_crc32, (var)) +``` + +### 5.2 Primitive Type Helpers + +#### Float (`float`) +```cpp +bool SerializeProp(archive& ar, const char* keyName, uint32 keyCRC, float& value) +{ + Assert(keyName != nullptr); + if (ar.IsSaving()) + { + WritePropertyHeader(ar, keyName); + IOPrintf(ar.Stream, "%f\n\n", value); + return true; + } + + auto* node = FindProperty(ar.Properties, ar.PropertyCount, keyCRC); + if (node == nullptr) + { + return false; + } + + // Fast float conversion from String slice + char buffer[64]; + size_t copySize = Min(node->Value.Size, sizeof(buffer) - 1); + MemCopy(buffer, node->Value.Str, copySize); + buffer[copySize] = '\0'; + + char* endPtr = nullptr; + float parsed = strtof(buffer, &endPtr); + if (endPtr != buffer) + { + value = parsed; + return true; + } + return false; +} +``` + +#### 32-Bit Signed Integer (`int32`) +```cpp +bool SerializeProp(archive& ar, const char* keyName, uint32 keyCRC, int32& value) +{ + Assert(keyName != nullptr); + if (ar.IsSaving()) + { + WritePropertyHeader(ar, keyName); + IOPrintf(ar.Stream, "%d\n\n", value); + return true; + } + + auto* node = FindProperty(ar.Properties, ar.PropertyCount, keyCRC); + if (node == nullptr) + { + return false; + } + + char buffer[32]; + size_t copySize = Min(node->Value.Size, sizeof(buffer) - 1); + MemCopy(buffer, node->Value.Str, copySize); + buffer[copySize] = '\0'; + + char* endPtr = nullptr; + int32 parsed = static_cast(strtol(buffer, &endPtr, 10)); + if (endPtr != buffer) + { + value = parsed; + return true; + } + return false; +} +``` + +#### 64-Bit Unsigned Integer (`uint64`, `EntityID`) +```cpp +bool SerializeProp(archive& ar, const char* keyName, uint32 keyCRC, uint64& value) +{ + Assert(keyName != nullptr); + if (ar.IsSaving()) + { + WritePropertyHeader(ar, keyName); + IOPrintf(ar.Stream, "0x%016llX\n\n", value); + return true; + } + + auto* node = FindProperty(ar.Properties, ar.PropertyCount, keyCRC); + if (node == nullptr) + { + return false; + } + + char buffer[32]; + size_t copySize = Min(node->Value.Size, sizeof(buffer) - 1); + MemCopy(buffer, node->Value.Str, copySize); + buffer[copySize] = '\0'; + + char* endPtr = nullptr; + int base = (buffer[0] == '0' && (buffer[1] == 'x' || buffer[1] == 'X')) ? 16 : 10; + uint64 parsed = strtoull(buffer, &endPtr, base); + if (endPtr != buffer) + { + value = parsed; + return true; + } + return false; +} +``` + +#### Boolean (`bool`) +```cpp +bool SerializeProp(archive& ar, const char* keyName, uint32 keyCRC, bool& value) +{ + Assert(keyName != nullptr); + if (ar.IsSaving()) + { + WritePropertyHeader(ar, keyName); + IOPrintf(ar.Stream, "%s\n\n", value ? "true" : "false"); + return true; + } + + auto* node = FindProperty(ar.Properties, ar.PropertyCount, keyCRC); + if (node == nullptr) + { + return false; + } + + if (StringCompare(node->Value, WrapString("true")) == 0 || + StringCompare(node->Value, WrapString("1")) == 0) + { + value = true; + return true; + } + if (StringCompare(node->Value, WrapString("false")) == 0 || + StringCompare(node->Value, WrapString("0")) == 0) + { + value = false; + return true; + } + return false; +} +``` + +### 5.3 Vector Helpers (`float x, y, z`) +```cpp +bool SerializeProp(archive& ar, const char* keyName, uint32 keyCRC, float& x, float& y, float& z) +{ + Assert(keyName != nullptr); + if (ar.IsSaving()) + { + WritePropertyHeader(ar, keyName); + IOPrintf(ar.Stream, "%f %f %f\n\n", x, y, z); + return true; + } + + auto* node = FindProperty(ar.Properties, ar.PropertyCount, keyCRC); + if (node == nullptr) + { + return false; + } + + char buffer[128]; + size_t copySize = Min(node->Value.Size, sizeof(buffer) - 1); + MemCopy(buffer, node->Value.Str, copySize); + buffer[copySize] = '\0'; + + float parsedX = 0.0f; + float parsedY = 0.0f; + float parsedZ = 0.0f; + int matches = sscanf_s(buffer, "%f %f %f", &parsedX, &parsedY, &parsedZ); + if (matches == 3) + { + x = parsedX; + y = parsedY; + z = parsedZ; + return true; + } + return false; +} +``` + +### 5.4 String Helpers +```cpp +bool SerializeProp(archive& ar, const char* keyName, uint32 keyCRC, String& value) +{ + Assert(keyName != nullptr); + if (ar.IsSaving()) + { + WritePropertyHeader(ar, keyName); + bool hasSpace = ContainsChar(value, ' '); + if (hasSpace) + { + IOPrintf(ar.Stream, "\"%.*s\"\n\n", static_cast(value.Size), value.Str); + } + else + { + IOPrintf(ar.Stream, "%.*s\n\n", static_cast(value.Size), value.Str); + } + return true; + } + + auto* node = FindProperty(ar.Properties, ar.PropertyCount, keyCRC); + if (node == nullptr) + { + return false; + } + + String parsed = node->Value; + // Strip optional surrounding double quotes + if (parsed.Size >= 2 && parsed.Str[0] == '"' && parsed.Str[parsed.Size - 1] == '"') + { + parsed.Str++; + parsed.Size -= 2; + } + + // Allocate persistent string copy on the archive arena + Assert(ar.ArenaInstance != nullptr); + value = StringCopy(ar.ArenaInstance, parsed); + return true; +} +``` + +--- + +## 6. Two-Tier Versioning Architecture + +### 6.1 Architectural Rationale: Base vs Derived Decoupling +In entity-component systems or object-oriented engine hierarchies, entities consist of two distinct domains: +1. **Core Engine Identity (Base Entity)**: Position, Rotation, Scale, EntityID, Class Kind, Render Flags, Layer Masks. Managed by engine architects. +2. **Gameplay Specialization (Derived Class)**: Ammo, Health, AI State, Mesh Instance ID, Patrol Paths. Managed by gameplay programmers. + +#### The Fragility of Monolithic Versioning +In naive serialization architectures, a single `uint32 Version` governs the entire file. When an engine programmer updates the base `Entity` struct (e.g. adding a `uint32 LayerMask`), bumping the global version invalidates or forces schema changes across every single derived entity type in the project. + +#### The Two-Tier Solution +Juliet decouples versioning into two independent tiers: +- **Tier 1: Base Engine Version (`kEntityBaseVersion`)**: Declared centrally in `Entity.h`. Governs `Entity` base fields. +- **Tier 2: Derived Class Version (`Class::Version`)**: Declared per-entity class in `Class.h` and initialized in `DEFINE_ENTITY_VERSIONED`. + +``` +======================================================================== + .jasset Text Archive +======================================================================== + ; AssetType + Entity + ; BaseVersion ------> Governed by kEntityBaseVersion in Entity.h + 1 + ; EntityID + 0x0000000000000001 + ; Kind + Inert + ; Position + 0.0 0.0 0.0 +------------------------------------------------------------------------ + ; ClassVersion ------> Governed by Class::Version in Class.h + 2 + ; MeshInstance + 42 +======================================================================== +``` + +When an engine programmer bumps `kEntityBaseVersion` from `1` to `2` to add `LayerMask`, no derived game classes (`Inert`, `Monster`, `Vehicle`) need version increments or code modifications. + +### 6.2 Implementation Details + +#### Engine Base Version (`Game/Entity/Entity.h`) +```cpp +constexpr uint32 kEntityBaseVersion = 1; + +void SerializeEntityBase(archive& ar, NonNullPtr entity); +``` + +#### Derived Class Version (`Juliet/include/Engine/Class.h`) +```cpp +struct Class +{ + uint32 CRC; + uint8 kind; + uint16 Version; // Added derived class version + serialize_fct_type serialize_fct; + size_t size_of; + size_t alignment; + +#if JULIET_DEBUG + String Name; +#endif +}; + +consteval Class MakeClass(String name, uint8 kind, uint16 version, size_t size, size_t align, serialize_fct_type fct) +{ + Class cls = {}; + cls.CRC = crc32(name.Str, name.Size); + cls.kind = kind; + cls.Version = version; + cls.size_of = size; + cls.alignment = align; + cls.serialize_fct = fct; + +#if JULIET_DEBUG + cls.Name = name; +#endif + + return cls; +} +``` + +#### Class Registration Macros (`Game/Entity/Entity.h`) +```cpp +#define DEFINE_ENTITY_VERSIONED(entity, version, serialize_fct) \ + Class entityKind##entity = \ + MakeClass(ConstString(#entity), (uint8)Entity_Type::entity, (version), sizeof(entity), alignof(entity), serialize_fct); \ + Class* entity::Kind = &entityKind##entity; +``` + +### 6.3 Execution Flow in `SerializeEntity` +```cpp +void Serialize(archive& ar, NonNullPtr entity) +{ + Assert(entity.Get() != nullptr); + + // --- Tier 1: Base Entity Serialization --- + if (ar.IsSaving()) + { + uint32 baseVer = kEntityBaseVersion; + SERIALIZE_PROP(ar, baseVer); + SERIALIZE_PROP(ar, entity->ID); + + String kindStr = WrapString(kEntity_type_names[entity->Kind->kind]); + SerializeProp(ar, "Kind", "Kind"_crc32, kindStr); + + SerializeProp(ar, "Position", "Position"_crc32, entity->X, entity->Y, entity->Z); + } + else + { + uint32 baseVer = 0; + if (!SerializeProp(ar, "BaseVersion", "BaseVersion"_crc32, baseVer)) + { + baseVer = 1; // Default to initial schema if absent + } + ar.BaseVersion = baseVer; + + SERIALIZE_PROP(ar, entity->ID); + + String kindStr = {}; + if (SerializeProp(ar, "Kind", "Kind"_crc32, kindStr)) + { + // Resolve class pointer from name + for (uint8 i = 0; i < ToUnderlying(Entity_Type::Count); ++i) + { + if (StringCompare(kindStr, WrapString(kEntity_type_names[i])) == 0) + { + entity->Kind = kEntity_type_class_ptr[i]; + break; + } + } + } + Assert(entity->Kind != nullptr); + + SerializeProp(ar, "Position", "Position"_crc32, entity->X, entity->Y, entity->Z); + } + + // --- Tier 2: Derived Entity Serialization --- + if (entity->Kind->serialize_fct != nullptr && entity->Derived != nullptr) + { + if (ar.IsSaving()) + { + uint32 classVer = entity->Kind->Version; + SerializeProp(ar, "ClassVersion", "ClassVersion"_crc32, classVer); + } + else + { + uint32 classVer = 0; + if (!SerializeProp(ar, "ClassVersion", "ClassVersion"_crc32, classVer)) + { + classVer = entity->Kind->Version; + } + ar.ClassVersion = static_cast(classVer); + } + + entity->Kind->serialize_fct(&ar, entity->Derived); + } +} +``` + +--- + +## 7. Post-Serialization Deprecation & Migration + +### 7.1 Schema Evolution Challenge +Over the lifecycle of a game, gameplay mechanics evolve: +- A scalar float `Speed` is replaced by a directional 2D vector `Velocity`. +- A single texture index `TextureID` is replaced by an asset path string `DiffuseTexture`. +- Obsolete properties are deleted entirely. + +Retaining deprecated members in active C++ structs creates code clutter, wastes memory, and invites bugs. + +### 7.2 Deprecation Primitives (`ReadDeprecated*`) +Deprecation primitives allow serializers to ingest obsolete properties exclusively during loading without polluting modern structs or writing deprecated keys back to disk during saving. + +```cpp +bool ReadDeprecated(archive& ar, const char* keyName, uint32 keyCRC, float& outVal) +{ + Assert(keyName != nullptr); + if (ar.IsSaving()) + { + return false; // Deprecated fields are never saved + } + + auto* node = FindProperty(ar.Properties, ar.PropertyCount, keyCRC); + if (node == nullptr) + { + return false; + } + + char buffer[64]; + size_t copySize = Min(node->Value.Size, sizeof(buffer) - 1); + MemCopy(buffer, node->Value.Str, copySize); + buffer[copySize] = '\0'; + + char* endPtr = nullptr; + float parsed = strtof(buffer, &endPtr); + if (endPtr != buffer) + { + outVal = parsed; + return true; + } + return false; +} + +bool ReadDeprecatedVec2(archive& ar, const char* keyName, uint32 keyCRC, float& outX, float& outY) +{ + Assert(keyName != nullptr); + if (ar.IsSaving()) + { + return false; + } + + auto* node = FindProperty(ar.Properties, ar.PropertyCount, keyCRC); + if (node == nullptr) + { + return false; + } + + char buffer[128]; + size_t copySize = Min(node->Value.Size, sizeof(buffer) - 1); + MemCopy(buffer, node->Value.Str, copySize); + buffer[copySize] = '\0'; + + float x = 0.0f; + float y = 0.0f; + if (sscanf_s(buffer, "%f %f", &x, &y) == 2) + { + outX = x; + outY = y; + return true; + } + return false; +} + +bool ReadDeprecatedString(archive& ar, const char* keyName, uint32 keyCRC, NonNullPtr arena, String& outVal) +{ + Assert(keyName != nullptr); + if (ar.IsSaving()) + { + return false; + } + + auto* node = FindProperty(ar.Properties, ar.PropertyCount, keyCRC); + if (node == nullptr) + { + return false; + } + + String parsed = node->Value; + if (parsed.Size >= 2 && parsed.Str[0] == '"' && parsed.Str[parsed.Size - 1] == '"') + { + parsed.Str++; + parsed.Size -= 2; + } + outVal = StringCopy(arena, parsed); + return true; +} +``` + +### 7.3 In-Place Migration Pattern +When an asset file with an older `ClassVersion` is loaded, the derived serializer detects `ar.ClassVersion < N`, calls `ReadDeprecated*` to read obsolete fields, maps the legacy data into the modern struct, and continues. On the subsequent save, the asset file is emitted using the modern schema without deprecated keys. + +```cpp +struct Projectile +{ + DECLARE_ENTITY() + + // Modern Schema (v2) + float VelocityX = 0.0f; + float VelocityY = 0.0f; + float Damage = 50.0f; +}; + +void SerializeProjectile(archive* arPtr, void* payload) +{ + Assert(arPtr != nullptr); + Assert(payload != nullptr); + auto& ar = *arPtr; + auto* projectile = static_cast(payload); + + if (ar.IsSaving()) + { + SERIALIZE_PROP(ar, projectile->VelocityX); + SERIALIZE_PROP(ar, projectile->VelocityY); + SERIALIZE_PROP(ar, projectile->Damage); + } + else + { + SERIALIZE_PROP(ar, projectile->Damage); + + if (ar.ClassVersion < 2) + { + // Migration from v1: scalar 'Speed' converted to 'VelocityX' + float legacySpeed = 0.0f; + if (ReadDeprecated(ar, "Speed", "Speed"_crc32, legacySpeed)) + { + projectile->VelocityX = legacySpeed; + projectile->VelocityY = 0.0f; + } + } + else + { + SERIALIZE_PROP(ar, projectile->VelocityX); + SERIALIZE_PROP(ar, projectile->VelocityY); + } + } +} +``` + +--- + +## 8. Step-by-Step Implementation Roadmap & Unit Testing Plan + +### 8.1 Implementation Roadmap + +``` ++-------------------------------------------------------------------------+ +| Phase 1: Core Utilities & Dual-Mode CRC32 | +| - Update CRC32.h with constexpr Crc32(String) and Crc32(char*, len) | +| - Extend struct Class in Class.h with uint16 Version | ++-------------------------------------------------------------------------+ + | + v ++-------------------------------------------------------------------------+ +| Phase 2: In-Memory Zero-Copy Parser | +| - Implement TextArchiveParser.h / .cpp | +| - TokenizeTextArchive, TrimWhitespace, FindProperty | ++-------------------------------------------------------------------------+ + | + v ++-------------------------------------------------------------------------+ +| Phase 3: Archive Struct & SerializeProp Helpers | +| - Upgrade archive in serialization.h with ArchiveMode | +| - Implement primitive, vector, and string SerializeProp helpers | +| - Implement ReadDeprecated* primitives | ++-------------------------------------------------------------------------+ + | + v ++-------------------------------------------------------------------------+ +| Phase 4: Entity & World Integration | +| - Update Entity.h / Entity.cpp with two-tier serialization | +| - Refactor World.cpp to load/save .jasset files | ++-------------------------------------------------------------------------+ + | + v ++-------------------------------------------------------------------------+ +| Phase 5: Verification & Unit Testing Suite | +| - Build and run SerializationUnitTest.cpp | ++-------------------------------------------------------------------------+ +``` + +#### Phase 1: Core Utilities & Dual-Mode CRC32 +1. **Target**: `Juliet/include/Core/Common/CRC32.h` + - Add `constexpr uint32 Crc32(const char* str, size_t length)` and `constexpr uint32 Crc32(String str)`. + - Ensure existing `operator""_crc32` calls `Crc32`. +2. **Target**: `Juliet/include/Engine/Class.h` + - Add `uint16 Version` to `struct Class`. + - Update `MakeClass` to accept `uint16 version = 1`. + +#### Phase 2: In-Memory Zero-Copy Parser +1. **Target**: `Juliet/include/Core/Common/TextArchiveParser.h` (and `src/Core/Common/TextArchiveParser.cpp`) + - Define `struct TextPropertyNode { uint32 KeyCRC; String Value; bool Consumed; }`. + - Implement `TokenizeTextArchive(NonNullPtr arena, ByteBuffer buffer)`. + - Implement `FindProperty` and `AuditUnconsumedProperties`. + +#### Phase 3: Archive Struct & `SerializeProp` Helpers +1. **Target**: `Juliet/include/Core/Common/serialization.h` + - Introduce `enum class ArchiveMode : uint8`. + - Upgrade `struct archive` with stream pointer, property array, mode, and versions. + - Implement overloaded `SerializeProp` for `float`, `int32`, `uint64`, `bool`, vectors, and `String`. + - Implement `ReadDeprecated`, `ReadDeprecatedVec2`, `ReadDeprecatedString`. + +#### Phase 4: Entity & World Integration +1. **Target**: `Game/Entity/Entity.h` and `Game/Entity/Entity.cpp` + - Define `constexpr uint32 kEntityBaseVersion = 1;`. + - Update `DEFINE_ENTITY` and add `DEFINE_ENTITY_VERSIONED`. + - Implement `Serialize(archive& ar, NonNullPtr entity)` supporting `.jasset` text format. +2. **Target**: `Game/Data/World.h` and `Game/Data/World.cpp` + - Implement text-based world saving and loading using `.jasset` formatting. + +#### Phase 5: Comprehensive Unit Testing +1. **Target**: `Game/UnitTest/SerializationUnitTest.h` and `Game/UnitTest/SerializationUnitTest.cpp` + - Add unit tests verifying parsing, round-trip serialization, defaults preservation, versioning, and deprecation. + +--- + +### 8.2 Comprehensive Unit Testing Plan (`SerializationUnitTest.cpp`) + +The unit test suite validates all architectural requirements without modifying engine framework code for test-specific cases. + +```cpp +// Game/UnitTest/SerializationUnitTest.h +#pragma once + +#include + +#if JULIET_DEBUG +namespace UnitTest +{ + void RunSerializationUnitTests(); +} +#endif +``` + +```cpp +// Game/UnitTest/SerializationUnitTest.cpp +#include + +#if JULIET_DEBUG + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace UnitTest +{ + // Test Struct for Derived Entity Testing + struct DummyVehicle + { + DECLARE_ENTITY() + + float MaxSpeed = 120.0f; + int32 GearCount = 6; + uint64 ChassisUUID = 0xABCDEF0123456789ULL; + bool Turbo = true; + float PosX = 10.0f; + float PosY = 20.0f; + float PosZ = 30.0f; + }; + + void SerializeDummyVehicle(archive* arPtr, void* payload) + { + Assert(arPtr != nullptr); + Assert(payload != nullptr); + auto& ar = *arPtr; + auto* vehicle = static_cast(payload); + + SERIALIZE_PROP(ar, vehicle->MaxSpeed); + SERIALIZE_PROP(ar, vehicle->GearCount); + SERIALIZE_PROP(ar, vehicle->ChassisUUID); + SERIALIZE_PROP(ar, vehicle->Turbo); + SerializeProp(ar, "Position", "Position"_crc32, vehicle->PosX, vehicle->PosY, vehicle->PosZ); + } + + DEFINE_ENTITY_VERSIONED(DummyVehicle, 1, SerializeDummyVehicle); + + // Test 1: Parser tokenization with whitespace, comments, and mixed line endings + static void TestParserTokenization() + { + TempArena temp = scratch_begin(nullptr, 0); + + const char* testContent = + "# Header Comment\r\n" + "// Secondary comment\n" + "\n" + "; Health\r\n" + " 100.500000 \r\n" + "\n" + "; Name\n" + "\"Paladin Hero\"\n" + "\n" + "; Position\r\n" + "1.0 2.0 3.0\r\n"; + + ByteBuffer buffer = { + .Data = reinterpret_cast(const_cast(testContent)), + .Size = strlen(testContent) + }; + + ParsedTextArchive parsed = TokenizeTextArchive(temp.Arena, buffer); + Assert(parsed.PropertyCount == 3); + + auto* healthNode = FindProperty(parsed.Nodes, parsed.PropertyCount, "Health"_crc32); + Assert(healthNode != nullptr); + Assert(StringCompare(healthNode->Value, WrapString("100.500000")) == 0); + + auto* nameNode = FindProperty(parsed.Nodes, parsed.PropertyCount, "Name"_crc32); + Assert(nameNode != nullptr); + Assert(StringCompare(nameNode->Value, WrapString("\"Paladin Hero\"")) == 0); + + auto* posNode = FindProperty(parsed.Nodes, parsed.PropertyCount, "Position"_crc32); + Assert(posNode != nullptr); + Assert(StringCompare(posNode->Value, WrapString("1.0 2.0 3.0")) == 0); + + scratch_end(temp); + LogMessage(LogCategory::Core, "TestParserTokenization passed."); + } + + // Test 2: Missing properties retain default struct values + static void TestDefaultValueRetention() + { + TempArena temp = scratch_begin(nullptr, 0); + + const char* incompleteContent = + "; MaxSpeed\n" + "180.0\n"; + + ByteBuffer buffer = { + .Data = reinterpret_cast(const_cast(incompleteContent)), + .Size = strlen(incompleteContent) + }; + + ParsedTextArchive parsed = TokenizeTextArchive(temp.Arena, buffer); + + archive ar = {}; + ar.ArenaInstance = temp.Arena; + ar.Mode = ArchiveMode::LoadingText; + ar.Properties = parsed.Nodes; + ar.PropertyCount = parsed.PropertyCount; + + DummyVehicle vehicle; + // Defaults: MaxSpeed=120, GearCount=6, Turbo=true + SerializeDummyVehicle(&ar, &vehicle); + + Assert(vehicle.MaxSpeed == 180.0f); // Overwritten by archive + Assert(vehicle.GearCount == 6); // Preserved default + Assert(vehicle.Turbo == true); // Preserved default + Assert(vehicle.PosX == 10.0f); // Preserved default + + scratch_end(temp); + LogMessage(LogCategory::Core, "TestDefaultValueRetention passed."); + } + + // Test 3: Deprecation migration from v1 to v2 + struct LegacyWeapon + { + DECLARE_ENTITY() + float VelocityX = 0.0f; + float VelocityY = 0.0f; + }; + + void SerializeLegacyWeapon(archive* arPtr, void* payload) + { + Assert(arPtr != nullptr); + Assert(payload != nullptr); + auto& ar = *arPtr; + auto* weapon = static_cast(payload); + + if (ar.IsSaving()) + { + SERIALIZE_PROP(ar, weapon->VelocityX); + SERIALIZE_PROP(ar, weapon->VelocityY); + } + else + { + if (ar.ClassVersion < 2) + { + float oldSpeed = 0.0f; + if (ReadDeprecated(ar, "Speed", "Speed"_crc32, oldSpeed)) + { + weapon->VelocityX = oldSpeed; + weapon->VelocityY = 0.0f; + } + } + else + { + SERIALIZE_PROP(ar, weapon->VelocityX); + SERIALIZE_PROP(ar, weapon->VelocityY); + } + } + } + + DEFINE_ENTITY_VERSIONED(LegacyWeapon, 2, SerializeLegacyWeapon); + + static void TestDeprecationMigration() + { + TempArena temp = scratch_begin(nullptr, 0); + + // Simulated v1 file containing obsolete 'Speed' + const char* v1Content = + "; ClassVersion\n" + "1\n" + "; Speed\n" + "75.500000\n"; + + ByteBuffer buffer = { + .Data = reinterpret_cast(const_cast(v1Content)), + .Size = strlen(v1Content) + }; + + ParsedTextArchive parsed = TokenizeTextArchive(temp.Arena, buffer); + + archive ar = {}; + ar.ArenaInstance = temp.Arena; + ar.Mode = ArchiveMode::LoadingText; + ar.Properties = parsed.Nodes; + ar.PropertyCount = parsed.PropertyCount; + ar.ClassVersion = 1; + + LegacyWeapon weapon; + SerializeLegacyWeapon(&ar, &weapon); + + Assert(weapon.VelocityX == 75.5f); + Assert(weapon.VelocityY == 0.0f); + + scratch_end(temp); + LogMessage(LogCategory::Core, "TestDeprecationMigration passed."); + } + + void RunSerializationUnitTests() + { + LogMessage(LogCategory::Core, "=== Running Serialization & .jasset Unit Tests ==="); + TestParserTokenization(); + TestDefaultValueRetention(); + TestDeprecationMigration(); + LogMessage(LogCategory::Core, "=== All Serialization Unit Tests Passed Successfully ==="); + } +} // namespace UnitTest + +#endif diff --git a/Game/Plans/02_Entity_Allocation_And_Lifecycle.md b/Game/Plans/02_Entity_Allocation_And_Lifecycle.md new file mode 100644 index 0000000..2bc1764 --- /dev/null +++ b/Game/Plans/02_Entity_Allocation_And_Lifecycle.md @@ -0,0 +1,1016 @@ +# Juliet Game Engine Architecture Specification +## Document: 02 - Entity Allocation & In-Place Lifecycle + +**Document ID:** JULIET-SPEC-002 +**Status:** Approved for Implementation +**Author:** Senior Engine Architect +**Subsystems:** `Game/Entity`, `Game/Data`, `Game/UnitTest` +**Target Files:** +- [`Entity.h`](file:///w:/Classified/Juliet/Game/Entity/Entity.h) +- [`Entity.cpp`](file:///w:/Classified/Juliet/Game/Entity/Entity.cpp) +- [`EntityManager.h`](file:///w:/Classified/Juliet/Game/Entity/EntityManager.h) +- [`EntityManager.cpp`](file:///w:/Classified/Juliet/Game/Entity/EntityManager.cpp) +- [`World.h`](file:///w:/Classified/Juliet/Game/Data/World.h) +- [`World.cpp`](file:///w:/Classified/Juliet/Game/Data/World.cpp) +- [`WorldUnitTest.h`](file:///w:/Classified/Juliet/Game/UnitTest/WorldUnitTest.h) +- [`WorldUnitTest.cpp`](file:///w:/Classified/Juliet/Game/UnitTest/WorldUnitTest.cpp) + +--- + +## 1. Executive Summary & Problem Statement + +### 1.1 Background & Context +The Juliet game engine organizes game entities using a hybrid data-oriented architecture: +1. A flat array of **Base Entities** (`Entity`) encapsulating universal properties: unique 64-bit ID, runtime type reflection pointer (`Class* Kind`), spatial coordinates (`X, Y, Z`), and an opaque pointer to the derived payload (`DerivedType Derived`). +2. Type-segregated contiguous arrays of **Derived Entities** (`Inert`, etc.) stored in dedicated per-type memory arenas (`typed_entity_array`). + +This design is intended to provide maximum cache efficiency during spatial and general-purpose entity processing, while retaining dense SIMD-friendly streaming for type-specific systems (e.g., transform updates on `Inert` mesh instances). + +### 1.2 Root-Cause Analysis of `Game/Data/World.cpp` Loading Flaws +In the initial implementation of entity serialization in [`Game/Data/World.cpp`](file:///w:/Classified/Juliet/Game/Data/World.cpp#L40-L68), loading was fundamentally broken and incomplete: + +```cpp +// Existing flawed deserialization in World.cpp +for (typed_entity_array& type : entityManager.by_type) +{ + serialize_elem(ar, type.count); + + if (type.count > 0) + { + // Unserialize the base entity to get informations + Entity entity; + serialize(ar, &entity); + + RegisterBaseEntity(entityManager, entity); + } +} +``` + +This implementation suffers from several fatal defects: +1. **Single-Element Iteration Bug:** It uses `if (type.count > 0)` instead of a loop `for (size_t i = 0; i < type.count; ++i)`, deserializing at most one single entity per type bucket, leaving all subsequent entities in the stream unread and corrupting the archive read offset. +2. **Missing Derived Allocation:** It invokes `RegisterBaseEntity(entityManager, entity)`, which merely pushes the stack-allocated `Entity` into `manager.Entities`. The derived payload arena (`type.arena`) is completely untouched: `type.array` remains null, `type.count` in the manager is desynchronized, and `entity.Derived` remains unassigned or points to an invalid address. +3. **Invalid Pointer in `RegisterEntity`:** In [`Game/Entity/EntityManager.cpp`](file:///w:/Classified/Juliet/Game/Entity/EntityManager.cpp#L46-L66), `RegisterEntity` assigns `base->Derived = entity` before pushing `*base` into `manager.Entities`. The parameter `entity` is a pointer to caller-provided memory (often stack-allocated in helper functions like `MakeEntity`). When `ArenaPushSize` later allocates the persistent derived memory block, `base->Derived` stored inside `manager.Entities.Back()` is **never updated**—it remains dangling, pointing to the transient caller stack! + +### 1.3 The "Chicken-and-Egg" Stack Allocation Dilemma +The existing registration API requires a pre-existing derived instance: +```cpp +entity_template* RegisterEntity(EntityManager& manager, Entity* base, DerivedType entity); +``` +During runtime programmatic creation via `MakeEntity()`, a temporary instance of `T` is created on the stack and passed by pointer: +```cpp +template +EntityType* MakeEntity(EntityManager& manager, float x, float y, float z) +{ + EntityType result; // Stack allocation + Entity base; // Stack allocation + base.X = x; + base.Y = y; + base.Z = z; + base.Kind = EntityType::Kind; + + return (EntityType*)RegisterEntity(manager, &base, &result); +} +``` +When loading an entity from a stream or disk file, **the type is not known at compile time**. The engine reads a runtime type tag (`uint8 kind` or `uint32 CRC`), looks up the reflection metadata (`Class*`), and must instantiate the entity dynamically. + +Because C++ does not permit allocating a dynamic struct of unknown type on the stack, and because Juliet strictly forbids heap allocations (`malloc`, `new`, `std::vector`), deserialization cannot construct a temporary instance on the stack to pass to `RegisterEntity`. + +This is the classic **chicken-and-egg memory problem**: +- `RegisterEntity` requires an existing instance in memory to copy from. +- Deserialization requires an allocated memory buffer to deserialize into. + +### 1.4 Architectural Objectives +This specification establishes a robust in-place lifecycle pipeline that completely eliminates stack temporaries and dynamic heap allocations: +1. **Direct In-Place Allocation:** Introduce `AllocateEntity(EntityManager& manager, Class* classPtr)` which allocates both the base `Entity` and the derived struct directly within their respective engine memory arenas. +2. **Bidirectional Pointer Integrity:** Wire mutual pointers (`base->Derived` and `derived->Base`) at allocation time before any field deserialization begins. +3. **In-Place Stream Deserialization:** Read class reflection metadata first, invoke `AllocateEntity`, and stream base and derived properties directly into arena-resident memory. +4. **Isolated Entity Assets (`.jasset`):** Transition from a monolithic `world.bin` to a modular one-file-per-entity architecture (`Assets/Entities/{ID}.jasset`). +5. **Dirty Tracking & Optimal Saves:** Introduce an `IsDirty` flag on `Entity` to avoid rewriting unchanged entity files, minimizing disk I/O and eliminating spurious Git repository modifications. +6. **Robust Deletion Lifecycle:** Decouple in-memory removal (`RemoveAtFast` with pointer fixup) from disk synchronization using `World::PendingDeletions`. + +--- + +## 2. The Dual Arena Memory Model in `EntityManager` + +### 2.1 The Need for Dual Storage +Game engines execute systems with vastly different cache locality profiles: +- **Spatial / Frustum Culling / Transform Sync:** Iterates every entity in the world, needing only `X, Y, Z`, bounding spheres, and base status flags. +- **Specialized Logic / Render Updates:** Iterates only entities possessing specific components (e.g., `Inert` static meshes requiring instance transform updates to the GPU bindless descriptor table). + +Storing large monolithic polymorphic structs in a single array causes severe cache line pollution during spatial passes. Conversely, storing entities in fragmented individual allocations introduces cache misses and pointer-chasing overhead. + +Juliet resolves this with a **Dual Arena Memory Model**: + +``` ++---------------------------------------------------------------------------------------------+ +| EntityManager | ++---------------------------------------------------------------------------------------------+ +| | +| manager.Entities (VectorArena) | +| +------------------------------------+------------------------------------+ | +| | Entity 0 (ID=1001, X, Y, Z) | Entity 1 (ID=1002, X, Y, Z) | ... | +| | Derived ------------------------+ | Derived ---------------------+ | | +| +---------------------------------|--+------------------------------|-----+ | +| | | | +| v v | +| manager.by_type[ENTITY(Inert)].arena | +| +------------------------------------+------------------------------------+ | +| | Inert 0 (MeshInstance=4) | Inert 1 (MeshInstance=12) | ... | +| | Base ---------------------------+ | Base ------------------------+ | | +| +---------------------------------|--+------------------------------|-----+ | +| +---------------------------------+ | ++---------------------------------------------------------------------------------------------+ +``` + +### 2.2 `manager.Entities`: Cache-Friendly Base Entity Vector +Base entities reside in a pre-reserved contiguous array: +```cpp +VectorArena Entities; +``` +- **Capacity:** Fixed reserve of 100,000 entities allocated from the `WorldArena`. +- **Memory Footprint:** + $$\text{sizeof(Entity)} = 8\text{ (ID)} + 8\text{ (Kind)} + 8\text{ (Derived)} + 12\text{ (X, Y, Z)} + 1\text{ (IsDirty)} + 3\text{ (Padding)} = 40\text{ bytes}$$ + Total reserved space: $100{,}000 \times 40\text{ bytes} \approx 4.0\text{ MB}$. +- **Access Speed:** O(1) random access by index; sequential streaming utilizes L1/L2 hardware prefetchers with zero cache line waste. + +### 2.3 `manager.by_type[kind].arena`: Typed Component Arenas +Derived structs reside in per-type contiguous memory arenas: +```cpp +struct typed_entity_array +{ + Arena* arena; + entity_template* array; + size_t count; +}; +``` +- Each `Entity_Type` index owns an isolated `Arena*` allocated during `InitEntityManager`. +- Allocations are packed linearly with alignment specified by `classPtr->alignment`. +- `array` points directly to the first element in the arena, permitting typed array indexing: + ```cpp + Inert* inertArray = reinterpret_cast(manager.by_type[ENTITY(Inert)].array); + ``` + +### 2.4 Mutual Back-Pointer Architecture & Invariants +Every entity instance consists of two mutually linked allocations: +1. `basePtr->Derived`: Points from `Entity` in `manager.Entities` to the derived struct in `by_type[kind].arena`. +2. `derivedPtr->Base`: Points from the derived struct (via `DECLARE_ENTITY()`) back to `Entity` in `manager.Entities`. + +#### Invariant Rules: +1. **Non-Null Invariant:** For any active entity, `basePtr->Derived != nullptr` and `reinterpret_cast(basePtr->Derived)->Base == basePtr`. +2. **Type Coherence Invariant:** `basePtr->Kind->kind == derivedTypeId`. +3. **Array Count Invariant:** + $$\sum_{k=0}^{\text{ENTITY(Count)}-1} \text{manager.by\_type}[k].\text{count} == \text{manager.Entities.Size()}$$ + +### 2.5 Pointer Stability in `VectorArena` +`VectorArena::Create` executes `Reserve(ReserveSize)` on creation: +```cpp +newManager->Entities.Create(world->WorldArena JULIET_DEBUG_PARAM("Entities")); +``` +Because capacity ($100{,}000$) is fully reserved upfront in virtual address space, `VectorArena::PushBack` **never reallocates or moves existing memory**. Therefore: +- Pointers to `Entity` elements in `manager.Entities` remain absolutely stable across allocations. +- Derived struct `Base` pointers remain valid indefinitely unless an element is deleted. +- Element removal via swap-and-pop alters memory positions, requiring systematic pointer fixups (addressed in Section 5.2). + +--- + +## 3. The Solution: `AllocateEntity(EntityManager& manager, Class* classPtr)` + +### 3.1 Function Signature & Contract +The canonical allocation function is defined in `EntityManager.h`: + +```cpp +[[nodiscard]] Entity* AllocateEntity(EntityManager& manager, Class* classPtr); +``` + +#### Preconditions: +- `classPtr != nullptr`. +- `classPtr->kind < ENTITY(Count)`. +- `classPtr->size_of >= sizeof(entity_template)`. +- `manager.Entities.Size() < manager.Entities.Capacity`. +- `manager.by_type[classPtr->kind].arena != nullptr`. + +#### Postconditions: +- A new `Entity` record is appended to `manager.Entities`. +- A new typed block of `classPtr->size_of` bytes is allocated in `manager.by_type[classPtr->kind].arena`. +- The derived memory is zeroed. +- `base->Derived` points to the derived struct. +- `derived->Base` points to the base `Entity`. +- `base->Kind` is assigned to `classPtr`. +- `base->ID` is assigned the next unique `EntityManager::ID`. +- `base->IsDirty` is initialized to `true`. +- `typed_entity_array::count` is incremented. +- `typed_entity_array::array` is initialized if this is the first entity of this type. + +### 3.2 Detailed Step-by-Step Implementation +The implementation replaces the flawed `RegisterEntity` routine in [`Game/Entity/EntityManager.cpp`](file:///w:/Classified/Juliet/Game/Entity/EntityManager.cpp): + +```cpp +[[nodiscard]] Entity* AllocateEntity(EntityManager& manager, Class* classPtr) +{ + Assert(classPtr != nullptr); + Assert(classPtr->kind < ENTITY(Count)); + Assert(classPtr->size_of >= sizeof(entity_template)); + Assert(classPtr->alignment > 0); + + // 1. Allocate uninitialized Base Entity in the contiguous VectorArena + Entity baseTemplate{}; + baseTemplate.ID = EntityManager::ID++; + baseTemplate.Kind = classPtr; + baseTemplate.Derived = nullptr; + baseTemplate.X = 0.0f; + baseTemplate.Y = 0.0f; + baseTemplate.Z = 0.0f; + baseTemplate.IsDirty = true; + + manager.Entities.PushBack(baseTemplate); + Entity* basePtr = manager.Entities.Back(); + Assert(basePtr != nullptr); + + // 2. Allocate zeroed derived component memory in the typed arena + typed_entity_array& typedArray = manager.by_type[classPtr->kind]; + Assert(typedArray.arena != nullptr); + + void* rawMemory = ArenaPushSize( + typedArray.arena, + classPtr->size_of, + classPtr->alignment, + true JULIET_DEBUG_PARAM(kEntity_type_names[classPtr->kind])); + Assert(rawMemory != nullptr); + + auto* derivedTemplate = reinterpret_cast(rawMemory); + + // 3. Establish mutual back-pointers + basePtr->Derived = rawMemory; + derivedTemplate->Base = basePtr; + + // 4. Update typed array tracking + if (typedArray.array == nullptr) + { + typedArray.array = derivedTemplate; + } + typedArray.count += 1; + + return basePtr; +} +``` + +### 3.3 Pure C-Style Zeroing & Initialization +`ArenaPushSize` is called with `shouldBeZeroed = true` (or followed by `MemZero`). This guarantees that: +- All bytes are zeroed (`0x00`). +- Pointers inside derived structs default to `nullptr`. +- Numerical fields default to `0`. + +For structs that require specific non-zero sentinel values (such as `Inert::MeshInstance = indexMax`), Juliet follows a pure C approach without C++ constructor or placement-new machinery: +1. **Direct assignment or C-style init function**: +```cpp +inline void InitInert(Inert* inert) +{ + Assert(inert != nullptr); + inert->MeshInstance = indexMax; +} +``` +2. **Or simple struct literal assignment**: +```cpp +*inert = Inert{ .MeshInstance = indexMax }; +``` +No C++ placement `new`, `` headers, or hidden constructor/destructor calls are ever used. Data structures remain pure C-style aggregates. + +### 3.4 Bidirectional Pointer Wiring +Notice the sequence: +1. `manager.Entities.PushBack(baseTemplate)` places the struct at its final, fixed arena address. +2. `basePtr = manager.Entities.Back()` retrieves the persistent memory pointer. +3. `derivedTemplate->Base = basePtr` wires the derived back-pointer directly to this permanent location. +4. `basePtr->Derived = rawMemory` wires the base forward-pointer to the arena-allocated derived struct. + +No stack copying occurs. Neither pointer is ever left dangling. + +### 3.5 Updating Type Counts and Array Cache +`typed_entity_array` maintains: +- `typedArray.count`: The exact count of active entities of this type. +- `typedArray.array`: Pointer to the first element in the arena. + +When the first entity of a given type is allocated, `typedArray.array` is set to `derivedTemplate`. Because the arena allocates sequentially, `typedArray.array[i]` can be indexed directly with stride `classPtr->size_of` as long as memory remains contiguous. + +### 3.6 Refactoring `MakeEntity` Template +With `AllocateEntity` operational, `MakeEntity` in [`Game/Entity/Entity.h`](file:///w:/Classified/Juliet/Game/Entity/Entity.h#L95-L107) becomes clean, safe, and stack-free: + +```cpp +template + requires EntityConcept +[[nodiscard]] EntityType* MakeEntity(EntityManager& manager, float x, float y, float z) +{ + Entity* basePtr = AllocateEntity(manager, EntityType::Kind); + Assert(basePtr != nullptr); + + basePtr->X = x; + basePtr->Y = y; + basePtr->Z = z; + + auto* derivedPtr = static_cast(basePtr->Derived); + ConstructDerivedDefaults(derivedPtr); + + // Re-establish Base pointer after placement-new + derivedPtr->Base = basePtr; + + return derivedPtr; +} +``` + +--- + +## 4. In-Place Deserialization Pipeline + +### 4.1 Asset Format Specification (`.jasset`) +To achieve robust version control and modular streaming, Juliet adopts a human-readable, Git-diffable **one-file-per-entity** disk format with extension `.jasset`. + +```ini +; asset_type +entity_instance +; id +0x0100000000000042 +; class +Inert +; base_version +1 +; derived_version +1 +; position +0.43 0.32 1.56 +; mesh_instance +12 +``` + +#### Important: No Header Structs for Derived Types +- **Derived types NEVER require their own file header**: You do **not** write an `InertHeader`, `DoorHeader`, or `PlayerHeader`. Derived types only serialize their own member variables. +- **No binary `EntityFileHeader` struct is needed**: Under the `; variable_name\nvalues` text format, there is no packed binary C-struct header at all. The common properties (`; id`, `; class`, `; base_version`, `; derived_version`, `; position`) are standard text Key-Value nodes read by the exact same `archive` parser. + +### 4.2 Eliminating Intermediate Stack Allocations +Under the new pipeline: +1. The `.jasset` text file is read into memory onto a `TempArena` via `LoadFile`. +2. The lines are tokenized into a `TextPropertyNode` scratch table. +3. The engine reads `; class` (e.g. `"Inert"`) and resolves `Class* classPtr = FindClassByName(className)`. +4. `AllocateEntity(manager, classPtr)` is called immediately. Memory for both base and derived components is allocated in-place in their permanent engine arenas. +5. Base properties (`id`, `position`, etc.) are read directly into `*basePtr` via `SerializeEntityBase`. +6. `classPtr->serialize_fct(ar, basePtr->Derived, derivedVersion)` is called. Derived fields stream **directly into the typed arena** without temporary staging buffers or stack copies. + +### 4.3 Runtime Class Resolution +To ensure fast and safe type lookup during file deserialization: + +```cpp +[[nodiscard]] Class* ResolveEntityClass(uint8 kind, uint32 crc) +{ + if (kind >= ENTITY(Count)) + { + return nullptr; + } + + Class* classPtr = kEntity_type_class_ptr[kind]; + if (!classPtr) + { + return nullptr; + } + + if (classPtr->CRC != crc) + { + return nullptr; + } + + return classPtr; +} +``` + +### 4.4 In-Place Deserialization Algorithm + +``` ++---------------------------------------------------------------------------------------+ +| In-Place Deserialization Flowchart | ++---------------------------------------------------------------------------------------+ +| | +| 1. Read Header from IOStream/ByteBuffer | +| | | +| v | +| 2. Validate Magic ('JAST') and Version (1) | +| | | +| v | +| 3. Resolve Class* from header.Kind & header.ClassCRC | +| | | +| v | +| 4. basePtr = AllocateEntity(manager, classPtr) | +| | | +| +--> [manager.Entities]: Allocates base Entity | +| +--> [manager.by_type[kind].arena]: Allocates derived struct | +| +--> Mutual Back-Pointers Wired In-Place | +| | | +| v | +| 5. Direct Copy Base Properties (ID, X, Y, Z) | +| | | +| v | +| 6. Does classPtr->serialize_fct exist? | +| | | | +| Yes No | +| | | | +| v | | +| Invoke: | | +| serialize_fct(&ar, | | +| Derived) | | +| | | | +| +---------------------+ | +| | | +| v | +| 7. Clear Dirty Flag: basePtr->IsDirty = false | +| | ++---------------------------------------------------------------------------------------+ +``` + +```cpp +[[nodiscard]] Entity* DeserializeEntityInPlace(EntityManager& manager, archive& ar) +{ + Assert(ar.loading); + + // 1. Read class name and resolve Class* + String className = {}; + SerializeProp(ar, "class", className, ar.arena); + Class* classPtr = FindClassByName(className); + if (!classPtr) + { + return nullptr; + } + + // 2. Allocate persistent memory for base and derived components in their respective arenas + Entity* basePtr = AllocateEntity(manager, classPtr); + Assert(basePtr != nullptr); + + // 3. Read base versions and properties in-place + uint16 baseVersion = 1; + uint16 derivedVersion = 1; + SerializeProp(ar, "base_version", baseVersion); + SerializeProp(ar, "derived_version", derivedVersion); + + SerializeEntityBase(ar, *basePtr, baseVersion); + + // 4. Stream derived properties in-place directly into the typed arena + if (classPtr->serialize_fct != nullptr) + { + classPtr->serialize_fct(ar, basePtr->Derived, derivedVersion); + } + + // Freshly loaded entity matches disk state exactly + basePtr->IsDirty = false; + + return basePtr; +} +``` + +--- + +## 5. Entity Deletion Lifecycle & Disk Synchronization + +### 5.1 In-Memory Removal vs. Immediate Disk Deletion Hazards +In game development, deleting an entity in the editor or during gameplay must **never synchronously invoke disk deletion**: +1. **Frame Rate Stutters:** Blocking on synchronous OS filesystem APIs (`DeleteFileA`) introduces multisecond frame freezes. +2. **Transactional Safety:** If the editor crashes or the user exits without saving, disk modifications cannot be rolled back. +3. **Undo/Redo Support:** An editor action stack must allow recovering deleted entities before changes are permanently committed to disk. + +Therefore, Juliet enforces a strict separation: +- **Immediate in-memory destruction:** Releases the entity from active simulation and registers its identifier in `World::PendingDeletions`. +- **Deferred disk deletion:** Executed strictly during explicit `SaveWorld` operations. + +### 5.2 Fast In-Memory Removal (`RemoveAtFast`) & Mutual Pointer Fixup +`VectorArena::RemoveAtFast` utilizes swap-and-pop: the element at the target index is replaced by the last element in the vector, and `Count` is decremented. + +```cpp +void RemoveAtFast(index_t index) +{ + Assert(Arena); + Assert(index < Count); + Assert(Count > 0); + + Type* elementAdr = DataFirst + index; + + if (DataLast != elementAdr) + { + Swap(DataLast, elementAdr); + } + + --DataLast; + --Count; +} +``` + +#### The Pointer Invalidation Problem: +When `Entity A` (at `index`) is swapped with `Entity Z` (at `DataLast`), the physical address of `Entity Z` changes from `DataLast` to `elementAdr`. +If `Entity Z` has a derived struct `derivedZ`, `derivedZ->Base` previously pointed to `DataLast`. After `RemoveAtFast`, `derivedZ->Base` points to garbage or the freed slot! + +#### The Pointer Fixup Protocol: +To preserve the mutual back-pointer invariant, `DestroyEntity` explicitly fixes up the swapped entity's derived back-pointer: + +```cpp +void DestroyEntity(EntityManager& manager, EntityID id) +{ + Entity* baseArray = manager.Entities.DataPtr(); + size_t count = manager.Entities.Size(); + + size_t targetIndex = indexMax; + for (size_t i = 0; i < count; ++i) + { + if (baseArray[i].ID == id) + { + targetIndex = i; + break; + } + } + + if (targetIndex == indexMax) + { + return; + } + + Entity* targetEntity = &baseArray[targetIndex]; + Class* classPtr = targetEntity->Kind; + Assert(classPtr != nullptr); + + // 1. Remove derived component from typed array via swap-and-pop + RemoveDerivedComponent(manager, classPtr, targetEntity->Derived); + + // 2. Remove base entity via swap-and-pop in VectorArena + bool wasLast = (targetIndex == count - 1); + manager.Entities.RemoveAtFast(targetIndex); + + // 3. Pointer fixup: If an element was swapped into targetIndex, fix its back-pointer! + if (!wasLast && targetIndex < manager.Entities.Size()) + { + Entity* movedEntity = &manager.Entities[targetIndex]; + auto* derivedTemp = reinterpret_cast(movedEntity->Derived); + Assert(derivedTemp != nullptr); + derivedTemp->Base = movedEntity; + } +} +``` + +### 5.3 O(1) Component Removal in `typed_entity_array` via Swap-and-Pop +To keep derived components packed contiguously for SIMD/cache iteration: +1. Locate the component's index within `by_type[kind].arena`. Because components have uniform stride `classPtr->size_of`: + $$\text{componentIndex} = \frac{\text{reinterpret\_cast(derivedPtr)} - \text{reinterpret\_cast(typedArray.array)}}{\text{classPtr->size\_of}}$$ +2. If the component is not the last one in the typed arena: + - Copy the last component into the slot occupied by the deleted component. + - Update the moved component's `Base->Derived` pointer to point to its new slot. +3. Decrement `typedArray.count`. +4. Pop the arena allocation if it was the top of the stack, or decrement count to mark slot reclamation. + +```cpp +void RemoveDerivedComponent(EntityManager& manager, Class* classPtr, DerivedType derivedPtr) +{ + Assert(classPtr != nullptr); + Assert(derivedPtr != nullptr); + + typed_entity_array& typedArray = manager.by_type[classPtr->kind]; + Assert(typedArray.count > 0); + Assert(typedArray.array != nullptr); + + size_t stride = classPtr->size_of; + auto* targetByte = reinterpret_cast(derivedPtr); + auto* firstByte = reinterpret_cast(typedArray.array); + + size_t componentIndex = static_cast(targetByte - firstByte) / stride; + Assert(componentIndex < typedArray.count); + + size_t lastIndex = typedArray.count - 1; + if (componentIndex != lastIndex) + { + uint8* lastByte = firstByte + (lastIndex * stride); + + // Copy last component data into target slot + MemCopy(targetByte, lastByte, stride); + + // Fixup the base pointer of the moved component + auto* movedDerived = reinterpret_cast(targetByte); + Assert(movedDerived->Base != nullptr); + movedDerived->Base->Derived = targetByte; + } + + typedArray.count -= 1; + if (typedArray.count == 0) + { + typedArray.array = nullptr; + } +} +``` + +### 5.4 Tracking Deletions in `World::PendingDeletions` +In `World.h`, the `World` struct is extended with a pending deletions container: + +```cpp +struct World +{ + Arena* WorldArena = nullptr; + EntityManager* EntityManager = nullptr; + VectorArena PendingDeletions; +}; +``` + +When an entity is deleted in the world: +```cpp +void RemoveWorldEntity(World& world, EntityID id) +{ + Assert(world.EntityManager != nullptr); + + // Record pending disk deletion + world.PendingDeletions.PushBack(id); + + // Destroy in memory immediately + DestroyEntity(*world.EntityManager, id); +} +``` + +### 5.5 Disk File Cleanup during `SaveWorld` +During `SaveWorld`, before saving modified entities, the engine iterates over `world.PendingDeletions` and removes their associated `.jasset` files: + +```cpp +void ProcessPendingDeletions(World& world, NonNullPtr scratchArena) +{ + for (size_t i = 0; i < world.PendingDeletions.Size(); ++i) + { + EntityID id = world.PendingDeletions[i]; + + // Format relative asset path: Assets/Entities/{ID}.jasset + char filenameBuffer[64]; + juliet_snprintf(filenameBuffer, sizeof(filenameBuffer), "Entities/%llu.jasset", id); + + String assetPath = GetAssetPath(scratchArena, WrapString(filenameBuffer)); + + if (PlatformDeleteFile(assetPath)) + { + Log(LogLevel::Message, LogCategory::Game, "Deleted entity asset: %s", CStr(assetPath)); + } + } + + world.PendingDeletions.Clear(); +} +``` + +--- + +## 6. Dirty Tracking for Optimal Saves + +### 6.1 The Cost of Naive Monolithic & Full-Directory Writes +In a level containing $10{,}000$ entities: +- **Monolithic `world.bin` Save:** Modifying a single entity's $X$ coordinate requires re-serializing all $10{,}000$ entities and overwriting a multi-megabyte binary file. This introduces a huge Git diff and constant merge conflicts. +- **Full-Directory `.jasset` Save:** Iterating through all $10{,}000$ entities and unconditionally writing $10{,}000$ `.jasset` files incurs massive OS file-system overhead (directory table locks, I/O bandwidth) and changes the file timestamps of every asset. Git reports thousands of modified files even when only one entity changed! + +### 6.2 The `IsDirty` Flag on `Entity` +To solve this, `Entity` in [`Game/Entity/Entity.h`](file:///w:/Classified/Juliet/Game/Entity/Entity.h#L28-L36) is augmented with an explicit dirty flag: + +```cpp +struct Entity final +{ + EntityID ID = 0; + Class* Kind = nullptr; + DerivedType Derived = nullptr; + float X = 0.0f; + float Y = 0.0f; + float Z = 0.0f; + bool IsDirty = false; +}; +``` + +### 6.3 Granular State Transitions +The `IsDirty` flag obeys a strict lifecycle state machine: + +``` + +-----------------------------------+ + | Entity Created | + | (AllocateEntity / Editor) | + +-----------------+-----------------+ + | + v + +---------------+ + +------->| IsDirty: TRUE |<-------+ + | +-------+-------+ | + | | | + Entity Mutated | SaveWorld + (Position, Component) | Completed + | v | + | +---------------+ | + +--------+ IsDirty: FALSE+--------+ + +-------+-------+ + ^ + | + Deserialization + (LoadWorld / Asset) +``` + +1. **Entity Creation:** Newly spawned entities in the editor have `IsDirty = true`. +2. **Property Mutation:** Any modification to `X, Y, Z` or derived component payload sets `entity->IsDirty = true`. +3. **Successful Deserialization:** Entities loaded from disk initialize with `IsDirty = false`. +4. **Successful Save:** Upon successfully writing an entity to its `.jasset` file, the engine resets `entity->IsDirty = false`. + +### 6.4 Version Control Benefits (Git Friendly Assets) +By coupling the one-file-per-entity `.jasset` format with dirty tracking: +- Only modified entities are touched on disk. +- Git status displays only the exact `.jasset` files that were altered by the designer. +- Team members can work concurrently in the same game scene without encountering binary merge conflicts. + +### 6.5 Editor Integration (`RenderWorldEditorUI` Hooks) +In [`Game/Data/World.cpp`](file:///w:/Classified/Juliet/Game/Data/World.cpp#L291-L311), editor UI widgets automatically set the dirty flag upon receiving user input: + +```cpp +float pos[3] = { ent.X, ent.Y, ent.Z }; +if (ImGui::DragFloat3("Position", pos, 0.1f)) +{ + ent.X = pos[0]; + ent.Y = pos[1]; + ent.Z = pos[2]; + ent.IsDirty = true; // Mark dirty for persistence + UpdateWorld(world); +} +``` + +--- + +## 7. Step-by-Step Implementation Roadmap + +### Phase 1: Data Structures & Header Definitions +1. **Update `Entity.h`:** + - Add `bool IsDirty = false;` to `struct Entity`. + - Update `MakeEntity` to delegate to `AllocateEntity`. + - Define `EntityFileHeader`, `kEntityAssetMagic`, and `kEntityAssetVersion`. +2. **Update `EntityManager.h`:** + - Declare `[[nodiscard]] Entity* AllocateEntity(EntityManager& manager, Class* classPtr);`. + - Declare `void DestroyEntity(EntityManager& manager, EntityID id);`. + - Declare `void RemoveDerivedComponent(EntityManager& manager, Class* classPtr, DerivedType derivedPtr);`. +3. **Update `World.h`:** + - Add `VectorArena PendingDeletions;` to `struct World`. + - Update `SaveWorld` and `LoadWorld` signatures to take directory paths. + +### Phase 2: Core Memory Allocation & Wiring in `EntityManager.cpp` +1. Implement `AllocateEntity`: + - Enforce parameter assertions. + - Push to `manager.Entities`. + - Allocate zeroed block in `manager.by_type[kind].arena`. + - Wire mutual pointers (`base->Derived` and `derived->Base`). + - Increment `typedArray.count` and initialize `typedArray.array`. +2. Implement `DestroyEntity` & `RemoveDerivedComponent`: + - Implement swap-and-pop for derived components with base pointer update. + - Implement `manager.Entities.RemoveAtFast` with mutual back-pointer fixup. + +### Phase 3: In-Place Deserialization & Serialization Pipeline +1. In `Entity.cpp`: + - Implement `EntitySerialize(archive& ar, Entity* entity)`. + - Fix the existing reversed `if (ar.loading)` branch. +2. In `World.cpp`: + - Implement `SerializeEntityAsset(archive& ar, Entity* entity, String filepath)`. + - Implement `DeserializeEntityAsset(EntityManager& manager, archive& ar, String filepath)`. + - Stream derived properties directly using `classPtr->serialize_fct(&ar, entity->Derived)`. + +### Phase 4: World Save/Load Pipeline & Disk Deletion +1. In `World.cpp`: + - Implement `ProcessPendingDeletions(World& world, NonNullPtr scratchArena)`. + - Implement `SaveWorld(World& world, String worldDirectory)`: + - Process pending deletions. + - Iterate `manager.Entities`, skipping entities where `!entity.IsDirty`. + - Write dirty entities to `.jasset` files and clear `IsDirty`. + - Implement `LoadWorld(World& world, String worldDirectory)`: + - Enumerate `.jasset` files in directory. + - Call `DeserializeEntityAsset` for each file. + +### Phase 5: Editor Integration +1. In `RenderWorldEditorUI`: + - Hook `ImGui::DragFloat3` and property inspectors to set `IsDirty = true`. + - Hook "Add Entity" button to call `MakeEntity(*world.EntityManager, 0.0f, 0.0f, 0.0f)`. + - Hook "Delete Entity" button to call `RemoveWorldEntity(world, selectedEntityId)`. + +--- + +## 8. Unit Testing & Verification Plan + +### 8.1 Test Philosophy & Constraints +Following Juliet coding guidelines: +> "When creating a new system framework, make a unit test. To make the unit test we should not modify the framework code for special unit test case." + +Testing is isolated in [`Game/UnitTest/WorldUnitTest.cpp`](file:///w:/Classified/Juliet/Game/UnitTest/WorldUnitTest.cpp) and executed during engine initialization in debug builds. + +### 8.2 Comprehensive Test Suite (`WorldUnitTest.cpp`) +The test suite validates every guarantee made in this specification: + +```cpp +#include + +#if JULIET_DEBUG + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace UnitTest +{ + namespace + { + void TestEntityAllocationAndWiring() + { + Log(LogLevel::Message, LogCategory::Game, "[UnitTest] Running TestEntityAllocationAndWiring..."); + + TempArena tempArena = scratch_begin(nullptr, 0); + + World testWorld{}; + InitWorld(&testWorld, tempArena.Arena); + + InitEntityManager(&testWorld); + EntityManager& manager = *testWorld.EntityManager; + + // 1. Allocate Inert Entity via AllocateEntity + Entity* baseEntity = AllocateEntity(manager, Inert::Kind); + Assert(baseEntity != nullptr); + Assert(baseEntity->ID > 0); + Assert(baseEntity->Kind == Inert::Kind); + Assert(baseEntity->Derived != nullptr); + Assert(baseEntity->IsDirty == true); + + // 2. Validate mutual back-pointer wiring + auto* derived = reinterpret_cast(baseEntity->Derived); + Assert(derived->Base == baseEntity); + + // 3. DownCast verification + Inert* inert = DownCast(baseEntity); + Assert(inert != nullptr); + Assert(inert->Base == baseEntity); + + // 4. Validate typed array tracking + typed_entity_array& inertArray = manager.by_type[ENTITY(Inert)]; + Assert(inertArray.count == 1); + Assert(inertArray.array == derived); + + ShutdownEntityManager(); + ShutdownWorld(&testWorld); + + scratch_end(tempArena); + Log(LogLevel::Message, LogCategory::Game, "[UnitTest] PASSED: TestEntityAllocationAndWiring"); + } + + void TestInPlaceDeserialization() + { + Log(LogLevel::Message, LogCategory::Game, "[UnitTest] Running TestInPlaceDeserialization..."); + + TempArena tempArena = scratch_begin(nullptr, 0); + + World testWorld{}; + InitWorld(&testWorld, tempArena.Arena); + InitEntityManager(&testWorld); + EntityManager& manager = *testWorld.EntityManager; + + // 1. Create and populate entity + Inert* createdInert = MakeEntity(manager, 12.5f, -44.0f, 108.2f); + Assert(createdInert != nullptr); + createdInert->MeshInstance = 42; + + Entity* originalBase = createdInert->Base; + EntityID originalID = originalBase->ID; + + // 2. Serialize to memory archive + archive saveAr{ .arena = tempArena.Arena, .base_ptr = nullptr, .offset = 0, .loading = false }; + saveAr.base_ptr = ArenaPushArray(tempArena.Arena, Kilobytes(16)); + + EntityFileHeader header{ + .Magic = kEntityAssetMagic, + .Version = kEntityAssetVersion, + .ClassCRC = originalBase->Kind->CRC, + .Kind = originalBase->Kind->kind, + .EntityID = originalBase->ID, + .PositionX = originalBase->X, + .PositionY = originalBase->Y, + .PositionZ = originalBase->Z, + .PayloadSize = sizeof(index_t) + }; + serialize_elem(saveAr, header); + serialize_elem(saveAr, createdInert->MeshInstance); + + // 3. Clear manager to simulate fresh load + ShutdownEntityManager(); + InitEntityManager(&testWorld); + EntityManager& freshManager = *testWorld.EntityManager; + + // 4. Deserialize in-place + archive loadAr{ .arena = tempArena.Arena, .base_ptr = saveAr.base_ptr, .offset = 0, .loading = true }; + Entity* loadedBase = DeserializeEntityInPlace(freshManager, loadAr); + + Assert(loadedBase != nullptr); + Assert(loadedBase->ID == originalID); + Assert(loadedBase->X == 12.5f); + Assert(loadedBase->Y == -44.0f); + Assert(loadedBase->Z == 108.2f); + Assert(loadedBase->IsDirty == false); + + Inert* loadedInert = DownCast(loadedBase); + Assert(loadedInert != nullptr); + Assert(loadedInert->Base == loadedBase); + + ShutdownEntityManager(); + ShutdownWorld(&testWorld); + + scratch_end(tempArena); + Log(LogLevel::Message, LogCategory::Game, "[UnitTest] PASSED: TestInPlaceDeserialization"); + } + + void TestSwapAndPopPointerFixup() + { + Log(LogLevel::Message, LogCategory::Game, "[UnitTest] Running TestSwapAndPopPointerFixup..."); + + TempArena tempArena = scratch_begin(nullptr, 0); + + World testWorld{}; + InitWorld(&testWorld, tempArena.Arena); + InitEntityManager(&testWorld); + EntityManager& manager = *testWorld.EntityManager; + + // Allocate 3 entities: E0, E1, E2 + Inert* e0 = MakeEntity(manager, 1.0f, 0.0f, 0.0f); + Inert* e1 = MakeEntity(manager, 2.0f, 0.0f, 0.0f); + Inert* e2 = MakeEntity(manager, 3.0f, 0.0f, 0.0f); + + EntityID id0 = e0->Base->ID; + EntityID id1 = e1->Base->ID; + EntityID id2 = e2->Base->ID; + + Assert(manager.Entities.Size() == 3); + + // Delete middle entity E1 (triggers swap with E2) + DestroyEntity(manager, id1); + + Assert(manager.Entities.Size() == 2); + + // Verify E2's mutual back-pointers are still completely intact + Entity* remaining0 = &manager.Entities[0]; + Entity* remaining1 = &manager.Entities[1]; + + Assert(remaining0->ID == id0); + Assert(remaining1->ID == id2); + + auto* derived0 = reinterpret_cast(remaining0->Derived); + auto* derived1 = reinterpret_cast(remaining1->Derived); + + Assert(derived0->Base == remaining0); + Assert(derived1->Base == remaining1); + + ShutdownEntityManager(); + ShutdownWorld(&testWorld); + + scratch_end(tempArena); + Log(LogLevel::Message, LogCategory::Game, "[UnitTest] PASSED: TestSwapAndPopPointerFixup"); + } + + void TestDirtyTrackingLifecycle() + { + Log(LogLevel::Message, LogCategory::Game, "[UnitTest] Running TestDirtyTrackingLifecycle..."); + + TempArena tempArena = scratch_begin(nullptr, 0); + + World testWorld{}; + InitWorld(&testWorld, tempArena.Arena); + InitEntityManager(&testWorld); + EntityManager& manager = *testWorld.EntityManager; + + Entity* entity = AllocateEntity(manager, Inert::Kind); + Assert(entity->IsDirty == true); + + // Simulate save + entity->IsDirty = false; + Assert(entity->IsDirty == false); + + // Simulate mutation + entity->X += 1.0f; + entity->IsDirty = true; + Assert(entity->IsDirty == true); + + ShutdownEntityManager(); + ShutdownWorld(&testWorld); + + scratch_end(tempArena); + Log(LogLevel::Message, LogCategory::Game, "[UnitTest] PASSED: TestDirtyTrackingLifecycle"); + } + } // namespace + + void WorldUnitTest() + { + Log(LogLevel::Message, LogCategory::Game, "=================================================="); + Log(LogLevel::Message, LogCategory::Game, "Starting Entity Allocation & Lifecycle Unit Tests"); + Log(LogLevel::Message, LogCategory::Game, "=================================================="); + + TestEntityAllocationAndWiring(); + TestInPlaceDeserialization(); + TestSwapAndPopPointerFixup(); + TestDirtyTrackingLifecycle(); + + Log(LogLevel::Message, LogCategory::Game, "=================================================="); + Log(LogLevel::Message, LogCategory::Game, "All Entity Lifecycle Unit Tests PASSED Successfully"); + Log(LogLevel::Message, LogCategory::Game, "=================================================="); + } +} // namespace UnitTest + +#endif diff --git a/Game/Plans/03_Entity_ID_And_World_Directory.md b/Game/Plans/03_Entity_ID_And_World_Directory.md new file mode 100644 index 0000000..21f61bc --- /dev/null +++ b/Game/Plans/03_Entity_ID_And_World_Directory.md @@ -0,0 +1,1127 @@ +# Distributed Entity ID & World Directory Architecture (OFPE) +## Technical Specification & Implementation Plan + +--- + +### Executive Metadata +- **System**: Juliet Game Engine — Entity & Scene Persistence Subsystem +- **Status**: Architecture Specification & Implementation Plan +- **Author**: Senior Engine Architect +- **Target File Path**: `w:\Classified\Juliet\Game\Plans\03_Entity_ID_And_World_Directory.md` +- **Related Subsystems**: `Core::HAL::Filesystem`, `Game::Entity::EntityManager`, `Game::Data::World`, `Core::Memory::Arena` + +--- + +## 1. Executive Summary & Architecture Goals + +### 1.1 The Problem with Monolithic World Files +Game engines historically package scene data into monolithic binary files (e.g., `world.bin`, `scene.dat`) or large serialized text files (e.g., YAML, JSON). While straightforward for solo development, monolithic architectures completely break down in professional, multi-developer studio environments using distributed version control (Git). + +``` +[Developer A: Alice] ── Modifies Entity #4 (Player Spawn) ──────┐ + ├──> [Git Merge Conflict / Data Loss] +[Developer B: Bob] ── Adds Entity #105 (Coin Collectible) ────┘ +``` + +When two developers work concurrently on separate Git branches: +1. **Binary Blobs**: Git cannot compute line-by-line diffs for binary formats. Any concurrent change results in a non-resolvable merge conflict. One developer's work must be discarded, or a team member must manually reconstruct changes. +2. **Text Formats**: Even in text formats, changes to common headers (entity counts, bounding volumes) or adjacent entries generate syntax-level conflicts. +3. **Implicit Dependencies**: Re-ordering entity indices in a single file breaks foreign key references that rely on array positions rather than stable identifiers. + +### 1.2 The One File Per Entity (OFPE) Paradigm +Juliet adopts the **One File Per Entity (OFPE)** storage architecture. In OFPE: +- A world is represented as a directory on disk rather than a single file. +- Global world parameters (environment, lighting, skybox, physics rules) are stored in a dedicated `WorldSettings.jasset` file. +- Every entity instance exists as an independent asset file (`.jasset`) within an `Entities/` subfolder. + +``` +Assets/Worlds// +├── WorldSettings.jasset +└── Entities/ + ├── 01_00000000000001.jasset + ├── 01_00000000000002.jasset + ├── 02_00000000000001.jasset + └── 02_00000000000002.jasset +``` + +#### Key Advantages of OFPE: +- **Zero-Conflict Entity Creation**: Adding a new entity creates a new, untracked file. Git merges new files across branches automatically without touching existing files. +- **Granular Entity Modification**: Modifying an entity touches only its specific `.jasset` file. Alice can tune physics on Entity #4 while Bob tweaks visuals on Entity #105 without overlapping diffs. +- **Clean Deletion**: Deleting an entity translates to a standard file deletion (`git rm`), isolating removal from modifications to other entities. +- **Granular History**: Each entity possesses its own discrete commit log via `git log .jasset`, simplifying archaeology and regression hunting. + +### 1.3 The Centralized Manifest Trap (Why Manifests Are Rejected) +A common pitfall when adopting OFPE is introducing a centralized manifest or catalog file (e.g., `World.manifest`, `EntityIndex.jasset`) containing global state such as: +- `TotalEntityCount = 42` +- `NextEntityID = 1005` +- A list of all active entity file paths. + +> [!CAUTION] +> Centralized manifests reintroduce the exact merge conflicts OFPE is designed to eliminate. If Alice creates an entity on Branch A (`NextEntityID: 1005 -> 1006`) and Bob creates an entity on Branch B (`NextEntityID: 1005 -> 1006`), merging Branch A and Branch B creates both a Git merge conflict in the manifest file and a catastrophic **ID collision** (both developers assigned ID `1005` to different entities). + +Juliet strictly rejects centralized manifests and global counter files: +1. **The directory is the manifest**: The set of entities comprising a world is defined exclusively by the files present in `Assets/Worlds//Entities/`. +2. **Decentralized ID generation**: The engine utilizes a distributed 64-bit partitioned identifier scheme that mathematically guarantees zero collisions across independent development branches. + +--- + +## 2. The 64-bit Partitioned EntityID Scheme + +### 2.1 Bit Layout Architecture +To prevent collisions between independent developers working offline on separate Git branches, `EntityID` is defined as a partitioned 64-bit unsigned integer (`uint64_t`). + +``` + 63 56 55 0 ++--------------+-----------------------------------------------------+ +| UserID (8b) | Local Counter (56b) | ++--------------+-----------------------------------------------------+ +``` + +| Field | Bit Range | Total Bits | Domain | Purpose | +| :--- | :--- | :--- | :--- | :--- | +| **`UserID`** | `[63..56]` | 8 bits | `0x00` – `0xFF` (0–255) | Unique identifier assigned to each developer machine or runtime context. | +| **`LocalCounter`** | `[55..0]` | 56 bits | `0x0` – `0x00FF'FFFF'FFFF'FFFF` | Monotonically incrementing per-user instance sequence. | + +- **Capacity**: $2^{56} \approx 7.2057 \times 10^{16}$ entities per user. At a creation rate of 100,000 entities per second, a single user will not exhaust this space for over 22,000 years. +- **Collision Proofing**: Because each developer possesses a unique 8-bit `UserID`, the value spaces generated by any two developers are strictly disjoint sets: +$$\text{Range}(User_A) \cap \text{Range}(User_B) = \emptyset \quad \forall \; User_A \neq User_B$$ +Even if Alice and Bob both start their counters at 1 on fresh Git branches, Alice generates IDs with prefix `0x01...` while Bob generates IDs with prefix `0x02...`. When their branches merge, their entity IDs will never collide. + +### 2.2 UserID Allocation Strategy +Juliet reserves `UserID = 0x00` as invalid/unassigned (`kInvalidEntityID = 0`). The usable range `[0x01..0xFF]` (1 to 255) is resolved via a two-tier acquisition pipeline: + +1. **Explicit Environment Variable (`JULIET_USER_ID`)**: + Developers or automated CI/CD build agents can explicitly define an integer between `1` and `255` in their environment (e.g., `set JULIET_USER_ID=42`). This guarantees static, deterministic assignment across team rosters. +2. **Deterministic Windows Username Hash Fallback**: + If `JULIET_USER_ID` is unset, the engine queries the operating system for the current logged-in user via Win32 `GetUserNameA`. The username string is hashed using 64-bit FNV-1a, and the result is folded into the non-zero range `[1..255]`: + $$\text{UserID} = (\text{FNV1a64}(\text{UserName}) \pmod{254}) + 1$$ + +### 2.3 Bitmasking & Helper API (`EntityID.h`) + +Below is the complete, production-grade implementation conforming to all Juliet coding standards (`[[nodiscard]]`, no exceptions, explicit casts, CamelCase naming, validation asserts): + +```cpp +#pragma once + +#include +#include +#include +#include + +using EntityID = uint64; + +constexpr EntityID kInvalidEntityID = 0ULL; +constexpr uint8 kInvalidUserID = 0U; + +constexpr uint64 kEntityID_UserID_Shift = 56ULL; +constexpr uint64 kEntityID_UserID_Mask = 0xFF00'0000'0000'0000ULL; +constexpr uint64 kEntityID_Counter_Mask = 0x00FF'FFFF'FFFF'FFFFULL; + +[[nodiscard]] inline constexpr EntityID MakeEntityID(uint8 userID, uint64 localCounter) +{ + Assert(userID != kInvalidUserID); + Assert((localCounter & ~kEntityID_Counter_Mask) == 0ULL); + + return (static_cast(userID) << kEntityID_UserID_Shift) | + (localCounter & kEntityID_Counter_Mask); +} + +[[nodiscard]] inline constexpr uint8 GetUserID(EntityID id) +{ + return static_cast((id & kEntityID_UserID_Mask) >> kEntityID_UserID_Shift); +} + +[[nodiscard]] inline constexpr uint64 GetLocalCounter(EntityID id) +{ + return id & kEntityID_Counter_Mask; +} + +[[nodiscard]] inline constexpr bool IsValidEntityID(EntityID id) +{ + return id != kInvalidEntityID && GetUserID(id) != kInvalidUserID; +} + +// UserID acquisition & formatting +[[nodiscard]] JULIET_API uint8 GetLocalUserID(); +[[nodiscard]] JULIET_API String FormatEntityID(NonNullPtr arena, EntityID id); +[[nodiscard]] JULIET_API bool ParseEntityID(String str, EntityID& outId); +``` + +### 2.4 UserID Resolution Implementation (`EntityID.cpp`) + +```cpp +#include +#include +#include +#include +#include +#include + +namespace +{ + uint8 g_CachedUserID = kInvalidUserID; + + [[nodiscard]] uint64 HashFNV1a64(const char* str, size_t length) + { + Assert(str != nullptr); + uint64 hash = 0xCBF29CE484222325ULL; + for (size_t i = 0; i < length; ++i) + { + hash ^= static_cast(str[i]); + hash *= 0x100000001B3ULL; + } + return hash; + } + + [[nodiscard]] uint8 ResolveUserIDFromEnvironmentOrOS() + { + // 1. Attempt environment override + const char* envUserID = std::getenv("JULIET_USER_ID"); + if (envUserID != nullptr && envUserID[0] != '\0') + { + char* endPtr = nullptr; + unsigned long parsed = std::strtoul(envUserID, &endPtr, 10); + if (endPtr != envUserID && parsed >= 1 && parsed <= 255) + { + Log(LogLevel::Message, LogCategory::Core, "EntityID: Using environment UserID: %lu", parsed); + return static_cast(parsed); + } + Log(LogLevel::Warning, LogCategory::Core, "EntityID: Invalid JULIET_USER_ID '%s'. Falling back to username hash.", envUserID); + } + + // 2. Fallback to Windows username hash + char username[256]; + DWORD usernameLength = static_cast(sizeof(username)); + if (GetUserNameA(username, &usernameLength) && usernameLength > 1) + { + // usernameLength includes null terminator + uint64 hash = HashFNV1a64(username, static_cast(usernameLength - 1)); + uint8 userId = static_cast((hash % 254ULL) + 1ULL); + Log(LogLevel::Message, LogCategory::Core, "EntityID: Generated UserID %u for user '%s'", userId, username); + return userId; + } + + // 3. Fallback to default user 1 if Win32 call fails + Log(LogLevel::Warning, LogCategory::Core, "EntityID: Failed to query Windows username. Defaulting UserID to 1."); + return 1U; + } +} // namespace + +uint8 GetLocalUserID() +{ + if (g_CachedUserID == kInvalidUserID) + { + g_CachedUserID = ResolveUserIDFromEnvironmentOrOS(); + Assert(g_CachedUserID != kInvalidUserID); + } + return g_CachedUserID; +} + +String FormatEntityID(NonNullPtr arena, EntityID id) +{ + // Formats as 16-character hexadecimal: UU_CCCCCCCCCCCCCC (e.g. "01_0000000000002A") + uint8 userId = GetUserID(id); + uint64 counter = GetLocalCounter(id); + + constexpr size_t kBufferSize = 18; // 2 hex + 1 underscore + 14 hex + null terminator + char* buffer = ArenaPushArray(arena, kBufferSize); + Assert(buffer != nullptr); + + juliet_snprintf(buffer, kBufferSize, "%02X_%014llX", static_cast(userId), counter); + return { buffer, kBufferSize - 1 }; +} + +bool ParseEntityID(String str, EntityID& outId) +{ + outId = kInvalidEntityID; + if (str.Size != 17 || str.Str == nullptr || str.Str[2] != '_') + { + return false; + } + + char userHex[3] = { str.Str[0], str.Str[1], '\0' }; + char* endPtr = nullptr; + unsigned long userId = std::strtoul(userHex, &endPtr, 16); + if (endPtr == userHex || userId == 0 || userId > 255) + { + return false; + } + + const char* counterHex = str.Str + 3; + unsigned long long counter = std::strtoull(counterHex, &endPtr, 16); + if (endPtr == counterHex || counter > kEntityID_Counter_Mask) + { + return false; + } + + outId = MakeEntityID(static_cast(userId), counter); + return true; +} +``` + +--- + +## 3. Solving Session Counter Continuity (The "Next-Day Problem") + +### 3.1 The Failure Mode of Naive Counter Persistence +Consider the following real-world development scenario: +1. On Monday, developer Alice (UserID `0x01`) creates 10 entities (`0x01_00000000000001` through `0x01_0000000000000A`). +2. Alice saves the world, shuts down Juliet, and leaves for the day. +3. On Tuesday morning, Alice reopens Juliet. +4. If the engine simply resets its in-memory counter to `1`, the next entity Alice creates will be assigned ID `0x01_00000000000001`. +5. Saving this entity will **silently overwrite** Entity #1 on disk, corrupting the level. + +Why external config files (e.g. `LocalCounter.ini`) fail: +- They get wiped on clean repository checkouts or git stashes. +- They are accidentally committed to source control by one developer, poisoning the counter state for all other developers on the team. +- They fall out of sync if the developer switches branches or edits on a second machine. + +### 3.2 The Stateless High-Water Mark Scan Algorithm +Juliet solves this completely without external configuration files or persistent local metadata. +During the standard world directory scan at startup, the engine inspects all existing `.jasset` files in the world folder. For every entity loaded whose `GetUserID(id) == CurrentUserID`, the engine updates its high-water mark: + +$$\text{NextLocalCounter} = \max \Big( \text{NextLocalCounter}, \; \text{GetLocalCounter}(\text{EntityID}) + 1 \Big)$$ + +```mermaid +flowchart TD + A[Start World Load: Scan Entities/ Directory] --> B[Read Entity File] + B --> C{GetUserID id == LocalUserID?} + C -- Yes --> D[Counter = GetLocalCounter id] + D --> E[NextLocalCounter = Max NextLocalCounter, Counter + 1] + C -- No --> F[Skip Counter Update: Belongs to Other Dev] + E --> G{More Files?} + F --> G + G -- Yes --> B + G -- No --> H[Finalize NextLocalCounter for Current Session] +``` + +### 3.3 Implementation in `EntityManager` + +```cpp +struct EntityManager +{ + uint64 NextLocalCounter = 1ULL; + + VectorArena Entities; + typed_entity_array ByType[ENTITY(Count)]; +}; + +void ObserveEntityIDForCounterContinuity(EntityManager& manager, EntityID id) +{ + const uint8 localUser = GetLocalUserID(); + if (GetUserID(id) == localUser) + { + const uint64 counter = GetLocalCounter(id); + if (counter >= manager.NextLocalCounter) + { + manager.NextLocalCounter = counter + 1ULL; + } + } +} + +[[nodiscard]] EntityID GenerateNextEntityID(EntityManager& manager) +{ + const uint8 localUser = GetLocalUserID(); + Assert(localUser != kInvalidUserID); + Assert(manager.NextLocalCounter <= kEntityID_Counter_Mask); + + const EntityID newId = MakeEntityID(localUser, manager.NextLocalCounter); + manager.NextLocalCounter += 1ULL; + return newId; +} +``` + +#### Invariant Guarantees: +1. **Zero State Overhead**: Absolutely no local registry files, databases, or environment flags are required to remember state across sessions. +2. **Branch Immunity**: If Alice switches branches to an older version where she had only created 5 entities, her counter naturally starts at 6. If she switches back to a branch with 50 entities, it starts at 51. +3. **Multi-Machine Resilience**: If Alice works on a desktop and a laptop sharing the same `UserID`, pulling the latest Git branch automatically updates the laptop's high-water mark. + +--- + +## 4. World Directory Layout & File Operations + +### 4.1 On-Disk Directory Hierarchy +Each world within Juliet resides under `Assets/Worlds//`. The structure cleanly separates world-level environment settings from atomic entity files: + +``` +Assets/Worlds// +├── WorldSettings.jasset # Environment, lighting, sun, physics, world rules +└── Entities/ # Atomic entity definitions (OFPE) + ├── 01_00000000000001.jasset + ├── 01_00000000000002.jasset + ├── 02_00000000000001.jasset + └── ... +``` + +### 4.2 `WorldSettings.jasset` Binary Layout +Global world settings are stored in `WorldSettings.jasset`. + +```cpp +#pragma pack(push, 1) +struct WorldSettingsFileHeader +{ + uint32 Magic = 0x5453574A; // 'JWST' (Juliet World SeTtings) in little-endian + uint32 Version = 1; +}; +#pragma pack(pop) + +struct WorldEnvironmentSettings +{ + // Directional Sun & Ambient Lighting + Vector3 SunDirection = { 0.577f, -0.577f, -0.577f }; + float _Pad0 = 0.0f; + Vector3 SunColor = { 1.0f, 0.95f, 0.8f }; + float SunIntensity = 1.0f; + Vector3 AmbientColor = { 0.2f, 0.25f, 0.35f }; + float AmbientIntensity = 0.15f; + + // Global Physics & Boundaries + Vector3 GravityAcceleration = { 0.0f, 0.0f, -9.81f }; + float KillPlaneZ = -50.0f; +}; +``` + +### 4.3 `.jasset` File Format Specification +Every entity file represents an atomic unit of game world state stored in human-readable, Git-diffable text format: + +```ini +; asset_type +entity_instance +; id +0x0100000000000042 +; class +Inert +; base_version +1 +; derived_version +1 +; position +0.43 0.32 1.56 +; mesh_instance +12 +``` + +#### On-Disk Entity Property Structure: +1. **Metadata & Identification**: + - `; asset_type`: `entity_instance` (or `entity_template` if saved in `Assets/Templates/`). + - `; id`: The 64-bit partitioned `EntityID` (e.g. `0x0100000000000042`). + - `; class`: The runtime `Class` name (e.g. `Inert`, `Door`). + - `; template`: Optional relative path to archetype template (e.g. `Assets/Templates/Door_Wood.jasset`). +2. **Version Directives**: + - `; base_version`: Engine-wide base entity version (`kEntityBaseVersion`). + - `; derived_version`: Class-specific gameplay version (`Class::Version`). +3. **Base Entity Properties**: + - Position (`position\n0.43 0.32 1.56`), Rotation, Scale. +4. **Derived Entity Properties**: + - Serialized key-value pairs written by `Class::serialize_fct`. + - References to other entities are serialized strictly as `EntityID` (e.g. `; target_id\n0x0100000000000099`), never raw memory pointers. + +*Note: No binary header struct is needed on disk, and derived types never define their own file headers.* + +### 4.4 Atomic File Writing (Crash and Corruption Defense) +To ensure editor crashes or power loss never leave half-written or zero-byte `.jasset` files: +1. Write entity payload to a temporary file: `.jasset.tmp`. +2. Flush and close the stream. +3. Perform an atomic filesystem rename to `.jasset` using Win32 `MoveFileExA` with `MOVEFILE_REPLACE_EXISTING`. + +```cpp +[[nodiscard]] bool AtomicWriteEntityFile(String targetFilePath, ByteBuffer buffer) +{ + Assert(IsValid(targetFilePath)); + Assert(buffer.Data != nullptr && buffer.Size > 0); + + TempArena scratch = scratch_begin(0, 0); + + // 1. Construct temporary path: .tmp + const size_t tmpPathSize = targetFilePath.Size + 5; + char* tmpPathBuffer = ArenaPushArray(scratch.Arena, tmpPathSize); + Assert(tmpPathBuffer != nullptr); + juliet_snprintf(tmpPathBuffer, tmpPathSize, "%s.tmp", CStr(targetFilePath)); + String tmpPath = { tmpPathBuffer, tmpPathSize - 1 }; + + // 2. Write to temporary file + IOStream* stream = IOFromFile(scratch.Arena, tmpPath, WrapString("wb")); + if (stream == nullptr) + { + Log(LogLevel::Error, LogCategory::Core, "AtomicWrite: Failed to create temp file %s", CStr(tmpPath)); + scratch_end(scratch); + return false; + } + + const size_t written = IOWrite(stream, buffer); + const bool closed = IOClose(stream); + if (!closed || written != buffer.Size) + { + Log(LogLevel::Error, LogCategory::Core, "AtomicWrite: Failed writing complete data to %s", CStr(tmpPath)); + DeleteFileA(CStr(tmpPath)); + scratch_end(scratch); + return false; + } + + // 3. Atomically replace final file + const BOOL moveSuccess = MoveFileExA(CStr(tmpPath), CStr(targetFilePath), + MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH); + if (!moveSuccess) + { + Log(LogLevel::Error, LogCategory::Core, "AtomicWrite: Failed atomic move %s -> %s (Win32 Error: %lu)", + CStr(tmpPath), CStr(targetFilePath), GetLastError()); + DeleteFileA(CStr(tmpPath)); + scratch_end(scratch); + return false; + } + + scratch_end(scratch); + return true; +} +``` + +--- + +## 5. HAL Directory Scanner Implementation + +### 5.1 Architecture & Header Specification +Juliet operates over a hardware abstraction layer (HAL) for filesystem operations. We extend `Core/HAL/Filesystem/Filesystem.h` with a high-performance directory iteration primitive. + +```cpp +// In Juliet/include/Core/HAL/Filesystem/Filesystem.h + +using FileIterCallback = void (*)(String filename, String fullPath, bool isDirectory, void* userData); + +/// Iterates through entries in directoryPath matching optional extension (e.g. ".jasset"). +/// If extension is empty/invalid, all non-directory files are visited. +/// Invokes callback for every matching entry. +[[nodiscard]] extern JULIET_API bool IterateDirectory( + String directoryPath, + String extension, + void* userData, + FileIterCallback callback +); +``` + +### 5.2 Win32 Platform Implementation (`Win32Filesystem.cpp`) +The implementation utilizes Win32 `FindFirstFileA` / `FindNextFileA` without standard library overhead, using Juliet's thread-local `scratch_begin` / `scratch_end` memory arenas. + +```cpp +// In Juliet/src/Core/HAL/Filesystem/Win32/Win32Filesystem.cpp + +#include +#include +#include +#include +#include +#include + +namespace +{ + [[nodiscard]] bool HasMatchingExtension(const char* filename, size_t filenameLen, String extension) + { + Assert(filename != nullptr); + if (!IsValid(extension)) + { + return true; + } + + if (filenameLen < extension.Size) + { + return false; + } + + const char* extStart = filename + (filenameLen - extension.Size); + return StringCompareCaseInsensitive(WrapString(extStart), extension) == 0; + } +} // namespace + +bool IterateDirectory(String directoryPath, String extension, void* userData, FileIterCallback callback) +{ + Assert(IsValid(directoryPath)); + Assert(callback != nullptr); + + TempArena scratch = scratch_begin(0, 0); + + // Normalize path and append search wildcard "\\*" + const size_t pathLen = directoryPath.Size; + const bool hasTrailingSlash = (directoryPath.Str[pathLen - 1] == '\\' || directoryPath.Str[pathLen - 1] == '/'); + + const size_t searchPatternLen = pathLen + (hasTrailingSlash ? 2 : 3); + char* searchPattern = ArenaPushArray(scratch.Arena, searchPatternLen); + Assert(searchPattern != nullptr); + + if (hasTrailingSlash) + { + juliet_snprintf(searchPattern, searchPatternLen, "%s*", CStr(directoryPath)); + } + else + { + juliet_snprintf(searchPattern, searchPatternLen, "%s/*", CStr(directoryPath)); + } + + WIN32_FIND_DATAA findData = {}; + HANDLE findHandle = FindFirstFileA(searchPattern, &findData); + + if (findHandle == INVALID_HANDLE_VALUE) + { + const DWORD error = GetLastError(); + if (error != ERROR_FILE_NOT_FOUND && error != ERROR_PATH_NOT_FOUND) + { + Log(LogLevel::Warning, LogCategory::Core, "IterateDirectory: Failed to search '%s' (Win32 Error: %lu)", + searchPattern, error); + } + scratch_end(scratch); + return false; + } + + do + { + // Skip self and parent directory references + if (strcmp(findData.cFileName, ".") == 0 || strcmp(findData.cFileName, "..") == 0) + { + continue; + } + + const bool isDirectory = (findData.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) != 0; + const size_t filenameLen = strlen(findData.cFileName); + + // Filter files by extension + if (!isDirectory && !HasMatchingExtension(findData.cFileName, filenameLen, extension)) + { + continue; + } + + // Build full normalized path + const size_t fullPathLen = pathLen + (hasTrailingSlash ? 0 : 1) + filenameLen + 1; + char* fullPathBuffer = ArenaPushArray(scratch.Arena, fullPathLen); + Assert(fullPathBuffer != nullptr); + + if (hasTrailingSlash) + { + juliet_snprintf(fullPathBuffer, fullPathLen, "%s%s", CStr(directoryPath), findData.cFileName); + } + else + { + juliet_snprintf(fullPathBuffer, fullPathLen, "%s/%s", CStr(directoryPath), findData.cFileName); + } + + String filenameStr = { findData.cFileName, filenameLen }; + String fullPathStr = { fullPathBuffer, fullPathLen - 1 }; + + callback(filenameStr, fullPathStr, isDirectory, userData); + + } while (FindNextFileA(findHandle, &findData) != 0); + + FindClose(findHandle); + scratch_end(scratch); + return true; +} +``` + +--- + +## 6. Two-Phase Loading & Entity Reference Resolution + +### 6.1 The Non-Deterministic Traversal Problem +When reading entities from disk via `IterateDirectory`, the operating system yields files in directory entry order (NTFS B-Tree index order). This order is essentially arbitrary: +- Entity `02_00000000000005` may be loaded *before* Entity `01_00000000000001`. +- If Entity #5 has a pointer or dependency targeting Entity #1 (e.g., `Entity* Parent`, `Entity* TargetEntity`, or `Entity* Mount`), Entity #1 does not yet exist in memory when Entity #5 is deserialized. + +Attempting to resolve pointer links during raw file deserialization will inevitably dereference unallocated entities or fail with null references. + +### 6.2 The Two-Phase Solution +To guarantee robust loading regardless of traversal order: + +``` +[DISK: Arbitrary Order] + Entity B (.jasset) ──> Phase 1: Allocate & Deserialize (Stores TargetID = A) ──> Register in ID-to-Entity Map + Entity A (.jasset) ──> Phase 1: Allocate & Deserialize (Stores TargetID = 0) ──> Register in ID-to-Entity Map + │ + ▼ + [Phase 2: PostLoadWorld Pass] + Entity B resolves TargetID (A) -> Entity* A Pointer +``` + +#### Phase 1: Allocation, Deserialization, and ID Registration +1. For each `.jasset` file found in `Entities/`: + - Read the file buffer via `LoadFile`. + - Validate `EntityFileHeader` (Magic `kEntityMagic`, Version `kEntityVersion`). + - Push an `Entity` record into `EntityManager::Entities`. + - Allocate the type-specific memory block in `EntityManager::by_type[Kind].arena`. + - Copy base transform values (`X, Y, Z`) and invoke `Class::serialize_fct`. + - All entity references are left as raw `EntityID` values (or embedded within components). + - Register the entity into a temporary lookup table: `EntityID -> Entity*`. + - Call `ObserveEntityIDForCounterContinuity` to maintain the high-water mark. + +#### Phase 2: PostLoadWorld Reference Resolution Pass +1. Once all `.jasset` files are fully deserialized into memory: + - Traverse all active entities in `EntityManager`. + - For every entity component that maintains runtime pointer references to other entities, resolve the pointer by querying `FindEntityByID(world, targetID)`. + - Validate broken references (e.g., if a referenced entity was deleted on another Git branch, reset the pointer to `nullptr` and log a warning). + - Rebuild spatial acceleration structures, initialize visual mesh instances (`SetMeshInstanceTransform`), and register physics colliders. + +### 6.3 Two-Phase Loading Implementation + +```cpp +struct EntityLookupEntry +{ + EntityID ID; + Entity* EntityPtr; +}; + +struct WorldLoadContext +{ + World* TargetWorld; + Arena* TempMapArena; + VectorArena LookupTable; +}; + +void EntityFileDiscoveryCallback(String filename, String fullPath, bool isDirectory, void* userData) +{ + Assert(userData != nullptr); + if (isDirectory) + { + return; + } + + auto* context = static_cast(userData); + auto& entityManager = *context->TargetWorld->EntityManager; + + TempArena scratch = scratch_begin(0, 0); + + ByteBuffer fileBuffer = LoadFile(scratch.Arena, fullPath); + if (fileBuffer.Size < sizeof(EntityFileHeader)) + { + Log(LogLevel::Error, LogCategory::Game, "LoadWorld: Entity file '%s' corrupted (too small)", CStr(filename)); + scratch_end(scratch); + return; + } + + const auto* header = reinterpret_cast(fileBuffer.Data); + if (header->Magic != 0x544E454A || header->Version != 1) + { + Log(LogLevel::Error, LogCategory::Game, "LoadWorld: File '%s' has invalid header", CStr(filename)); + scratch_end(scratch); + return; + } + + // Phase 1: Allocate & deserialize + Entity baseEntity = {}; + baseEntity.ID = header->ID; + baseEntity.Kind = kEntity_type_class_ptr[header->Kind]; + Assert(baseEntity.Kind != nullptr); + + archive ar = { + .arena = scratch.Arena, + .base_ptr = fileBuffer.Data + sizeof(EntityFileHeader), + .offset = 0, + .loading = true + }; + + // Deserialize base entity properties + serialize_elem(ar, baseEntity.X); + serialize_elem(ar, baseEntity.Y); + serialize_elem(ar, baseEntity.Z); + + // Register into EntityManager + entityManager.Entities.PushBack(baseEntity); + Entity* registeredBase = entityManager.Entities.Back(); + + // Allocate derived type payload + auto* derivedPtr = static_cast(ArenaPushSize( + entityManager.ByType[header->Kind].arena, + baseEntity.Kind->size_of, + baseEntity.Kind->alignment, + false JULIET_DEBUG_PARAM(kEntity_type_names[header->Kind]) + )); + Assert(derivedPtr != nullptr); + + // Deserialize derived data if class registered a serialize function + if (baseEntity.Kind->serialize_fct != nullptr) + { + baseEntity.Kind->serialize_fct(ar, derivedPtr); + } + + derivedPtr->Base = registeredBase; + registeredBase->Derived = derivedPtr; + + if (entityManager.ByType[header->Kind].array == nullptr) + { + entityManager.ByType[header->Kind].array = derivedPtr; + } + entityManager.ByType[header->Kind].count += 1; + + // Track in temporary lookup table for Phase 2 + context->LookupTable.PushBack({ .ID = header->ID, .EntityPtr = registeredBase }); + + // Update session counter continuity + ObserveEntityIDForCounterContinuity(entityManager, header->ID); + + scratch_end(scratch); +} + +[[nodiscard]] Entity* FindEntityByIDInLookup(const WorldLoadContext& context, EntityID id) +{ + if (id == kInvalidEntityID) + { + return nullptr; + } + + for (size_t i = 0; i < context.LookupTable.Size(); ++i) + { + if (context.LookupTable[i].ID == id) + { + return context.LookupTable[i].EntityPtr; + } + } + return nullptr; +} + +void PostLoadWorld(World& world, const WorldLoadContext& context) +{ + auto& entityManager = *world.EntityManager; + + // Phase 2: Resolve entity-to-entity references and instantiate runtime resources + for (size_t i = 0; i < entityManager.Entities.Size(); ++i) + { + Entity& entity = entityManager.Entities[i]; + + // Example: If an entity references a ParentID or TargetID, resolve it here: + // if (auto* custom = DownCast(&entity)) + // { + // custom->Target = FindEntityByIDInLookup(context, custom->TargetID); + // } + + // Initialize runtime visual mesh instances + if (entity.Kind == Inert::Kind) + { + auto* inert = DownCast(&entity); + if (inert != nullptr && inert->MeshInstance == indexMax) + { + // Register with MeshRenderer + inert->MeshInstance = CreateMeshInstance(inert->MeshID, MatrixTranslation(entity.X, entity.Y, entity.Z)); + } + } + } + + Log(LogLevel::Message, LogCategory::Game, "PostLoadWorld: Resolved references for %zu entities", + entityManager.Entities.Size()); +} + +[[nodiscard]] bool LoadWorldFromDirectory(World& world, String worldDirectoryPath) +{ + Assert(IsValid(worldDirectoryPath)); + Assert(world.EntityManager != nullptr); + + TempArena scratch = scratch_begin(0, 0); + + WorldLoadContext context = { + .TargetWorld = &world, + .TempMapArena = scratch.Arena + }; + context.LookupTable.Create(scratch.Arena JULIET_DEBUG_PARAM("WorldLoadLookupTable")); + + // 1. Load WorldSettings.jasset + const size_t settingsPathLen = worldDirectoryPath.Size + 22; + char* settingsPathBuf = ArenaPushArray(scratch.Arena, settingsPathLen); + Assert(settingsPathBuf != nullptr); + juliet_snprintf(settingsPathBuf, settingsPathLen, "%s/WorldSettings.jasset", CStr(worldDirectoryPath)); + String settingsPath = { settingsPathBuf, settingsPathLen - 1 }; + + ByteBuffer settingsBuffer = LoadFile(scratch.Arena, settingsPath); + if (settingsBuffer.Size >= sizeof(WorldSettingsFileHeader)) + { + const auto* header = reinterpret_cast(settingsBuffer.Data); + if (header->Magic == 0x5453574A && header->Version == 1) + { + const auto* settings = reinterpret_cast( + settingsBuffer.Data + sizeof(WorldSettingsFileHeader) + ); + world.Environment = *settings; + Log(LogLevel::Message, LogCategory::Game, "LoadWorld: Loaded WorldSettings from %s", CStr(settingsPath)); + } + } + + // 2. Phase 1: Iterate Entities/ directory + const size_t entitiesDirLen = worldDirectoryPath.Size + 11; + char* entitiesDirBuf = ArenaPushArray(scratch.Arena, entitiesDirLen); + Assert(entitiesDirBuf != nullptr); + juliet_snprintf(entitiesDirBuf, entitiesDirLen, "%s/Entities", CStr(worldDirectoryPath)); + String entitiesDirPath = { entitiesDirBuf, entitiesDirLen - 1 }; + + const bool scanSuccess = IterateDirectory(entitiesDirPath, WrapString(".jasset"), &context, EntityFileDiscoveryCallback); + if (!scanSuccess) + { + Log(LogLevel::Warning, LogCategory::Game, "LoadWorld: No entities found or failed scanning directory %s", CStr(entitiesDirPath)); + } + + // 3. Phase 2: Resolve pointers and runtime state + PostLoadWorld(world, context); + + scratch_end(scratch); + return true; +} +``` + +--- + +## 7. Step-by-Step Implementation Roadmap + +The transition from the legacy monolithic save format to the OFPE architecture is scheduled in five sequential development phases. + +```mermaid +graph TD + P1[Phase 1: EntityID Bitmasking & UserID Engine] --> P2[Phase 2: Filesystem HAL Directory Scanner] + P2 --> P3[Phase 3: Serializers for WorldSettings & Entities] + P3 --> P4[Phase 4: Two-Phase Loader & EntityManager Refactor] + P4 --> P5[Phase 5: World Editor Integration & Unit Testing] +``` + +### Phase 1: Core EntityID Infrastructure +- **Files**: + - `[NEW]` [EntityID.h](file:///w:/Classified/Juliet/Game/Entity/EntityID.h) + - `[NEW]` [EntityID.cpp](file:///w:/Classified/Juliet/Game/Entity/EntityID.cpp) +- **Deliverables**: + - Implement bitmask constants (`kEntityID_UserID_Shift`, `kEntityID_UserID_Mask`, `kEntityID_Counter_Mask`). + - Implement `MakeEntityID`, `GetUserID`, `GetLocalCounter`, `IsValidEntityID`. + - Implement Win32 `GetUserNameA` fallback and `JULIET_USER_ID` environment parsing. + - Implement `FormatEntityID` and `ParseEntityID` string utilities. + +### Phase 2: Filesystem HAL Directory Scanner +- **Files**: + - `[MODIFY]` [Filesystem.h](file:///w:/Classified/Juliet/Juliet/include/Core/HAL/Filesystem/Filesystem.h) + - `[MODIFY]` [Win32Filesystem.cpp](file:///w:/Classified/Juliet/Juliet/src/Core/HAL/Filesystem/Win32/Win32Filesystem.cpp) +- **Deliverables**: + - Declare `FileIterCallback` and `IterateDirectory` in `Filesystem.h`. + - Implement `IterateDirectory` in `Win32Filesystem.cpp` using Win32 `FindFirstFileA` / `FindNextFileA`. + - Verify scratch arena usage (`scratch_begin` / `scratch_end`) with zero memory leakage. + +### Phase 3: WorldSettings & Atomic Entity Disk Serialization +- **Files**: + - `[NEW]` [WorldSettings.h](file:///w:/Classified/Juliet/Game/Data/WorldSettings.h) + - `[MODIFY]` [Entity.h](file:///w:/Classified/Juliet/Game/Entity/Entity.h) + - `[NEW]` [EntityFileIO.h](file:///w:/Classified/Juliet/Game/Entity/EntityFileIO.h) + - `[NEW]` [EntityFileIO.cpp](file:///w:/Classified/Juliet/Game/Entity/EntityFileIO.cpp) +- **Deliverables**: + - Define `WorldSettingsFileHeader` ('JWST') and `EntityFileHeader` ('JENT'). + - Implement `AtomicWriteEntityFile` using `.tmp` and Win32 `MoveFileExA`. + - Implement serialization routines for `WorldEnvironmentSettings`. + +### Phase 4: Two-Phase Load Pipeline & EntityManager Refactoring +- **Files**: + - `[MODIFY]` [EntityManager.h](file:///w:/Classified/Juliet/Game/Entity/EntityManager.h) + - `[MODIFY]` [EntityManager.cpp](file:///w:/Classified/Juliet/Game/Entity/EntityManager.cpp) + - `[MODIFY]` [World.h](file:///w:/Classified/Juliet/Game/Data/World.h) + - `[MODIFY]` [World.cpp](file:///w:/Classified/Juliet/Game/Data/World.cpp) +- **Deliverables**: + - Add `uint64 NextLocalCounter` to `EntityManager`. + - Replace naive `ID++` counter with `GenerateNextEntityID(manager)`. + - Implement `ObserveEntityIDForCounterContinuity`. + - Implement `LoadWorldFromDirectory` and `SaveWorldToDirectory`. + - Implement Phase 2 `PostLoadWorld`. + +### Phase 5: Editor Integration & Unit Testing +- **Files**: + - `[MODIFY]` [World.cpp](file:///w:/Classified/Juliet/Game/Data/World.cpp) (`RenderWorldEditorUI`) + - `[MODIFY]` [WorldUnitTest.cpp](file:///w:/Classified/Juliet/Game/UnitTest/WorldUnitTest.cpp) +- **Deliverables**: + - Update World Editor ImGui UI to display the active world directory. + - Implement dirty-tracking: save only modified entities during interactive sessions. + - Execute full test suite verifying 0% ID collisions, counter continuity, and reference resolution. + +--- + +## 8. Unit Testing & Verification Plan + +In strict accordance with Juliet coding guidelines: +- Framework code must not be polluted with test branches. +- Tests must be implemented under the `UnitTest` namespace guarded by `#if JULIET_DEBUG`. +- Tests must verify all assumptions via `Assert`. + +### 8.1 Automated Unit Test Implementation (`WorldUnitTest.cpp`) + +```cpp +#include + +#if JULIET_DEBUG + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace UnitTest +{ + namespace + { + void TestEntityIDBitmasking() + { + Log(LogLevel::Message, LogCategory::Core, "Running TestEntityIDBitmasking..."); + + // Test 1: Bitmask construction and decomposition + const uint8 testUser = 0x42; + const uint64 testCounter = 0x0012'3456'789A'BCDEULL; + + const EntityID id = MakeEntityID(testUser, testCounter); + Assert(GetUserID(id) == testUser); + Assert(GetLocalCounter(id) == testCounter); + Assert(IsValidEntityID(id)); + + // Test 2: Boundary limits (max 56-bit counter) + const uint64 maxCounter = kEntityID_Counter_Mask; + const EntityID maxId = MakeEntityID(0xFF, maxCounter); + Assert(GetUserID(maxId) == 0xFF); + Assert(GetLocalCounter(maxId) == maxCounter); + + // Test 3: Invalid IDs + Assert(!IsValidEntityID(kInvalidEntityID)); + Assert(!IsValidEntityID(0x0000'0000'0000'0001ULL)); // UserID == 0 is invalid + + // Test 4: String formatting and parsing round-trip + TempArena scratch = scratch_begin(0, 0); + + String formatted = FormatEntityID(scratch.Arena, id); + Assert(IsValid(formatted)); + + EntityID parsedId = kInvalidEntityID; + const bool parseSuccess = ParseEntityID(formatted, parsedId); + Assert(parseSuccess); + Assert(parsedId == id); + + scratch_end(scratch); + } + + void TestSessionCounterContinuity() + { + Log(LogLevel::Message, LogCategory::Core, "Running TestSessionCounterContinuity..."); + + EntityManager manager = {}; + manager.NextLocalCounter = 1ULL; + + const uint8 localUser = GetLocalUserID(); + const uint8 otherUser = (localUser == 255) ? 1 : (localUser + 1); + + // 1. Simulate encountering entities from another developer: should not advance our counter + const EntityID foreignEntity1 = MakeEntityID(otherUser, 100ULL); + const EntityID foreignEntity2 = MakeEntityID(otherUser, 500ULL); + ObserveEntityIDForCounterContinuity(manager, foreignEntity1); + ObserveEntityIDForCounterContinuity(manager, foreignEntity2); + Assert(manager.NextLocalCounter == 1ULL); + + // 2. Simulate encountering existing local entities from Monday: counter must advance past max + const EntityID localEntity1 = MakeEntityID(localUser, 5ULL); + const EntityID localEntity2 = MakeEntityID(localUser, 42ULL); + const EntityID localEntity3 = MakeEntityID(localUser, 18ULL); + + ObserveEntityIDForCounterContinuity(manager, localEntity1); + Assert(manager.NextLocalCounter == 6ULL); + + ObserveEntityIDForCounterContinuity(manager, localEntity2); + Assert(manager.NextLocalCounter == 43ULL); + + ObserveEntityIDForCounterContinuity(manager, localEntity3); + Assert(manager.NextLocalCounter == 43ULL); // 18 is lower than 42, must stay at 43 + + // 3. Allocating new entity must yield 43 and increment counter to 44 + const EntityID nextId = GenerateNextEntityID(manager); + Assert(GetUserID(nextId) == localUser); + Assert(GetLocalCounter(nextId) == 43ULL); + Assert(manager.NextLocalCounter == 44ULL); + } + + struct DirScanProbeContext + { + size_t FoundCount; + }; + + void DummyFileCallback(String filename, String fullPath, bool isDirectory, void* userData) + { + Assert(IsValid(filename)); + Assert(IsValid(fullPath)); + Assert(userData != nullptr); + + auto* ctx = static_cast(userData); + if (!isDirectory) + { + ctx->FoundCount += 1; + } + } + + void TestDirectoryIteration() + { + Log(LogLevel::Message, LogCategory::Core, "Running TestDirectoryIteration..."); + + TempArena scratch = scratch_begin(0, 0); + + // Create temporary test directory structure + const char* testDirPath = "TestWorldScanDir"; + CreateDirectoryA(testDirPath, nullptr); + + char file1[128]; + char file2[128]; + juliet_snprintf(file1, sizeof(file1), "%s/test1.jasset", testDirPath); + juliet_snprintf(file2, sizeof(file2), "%s/test2.txt", testDirPath); + + IOStream* s1 = IOFromFile(scratch.Arena, WrapString(file1), WrapString("wb")); + IOStream* s2 = IOFromFile(scratch.Arena, WrapString(file2), WrapString("wb")); + Assert(s1 != nullptr); + Assert(s2 != nullptr); + + Byte dummyData = 0xAA; + ByteBuffer buf = { .Data = &dummyData, .Size = 1 }; + IOWrite(s1, buf); + IOWrite(s2, buf); + IOClose(s1); + IOClose(s2); + + // Scan with extension filter ".jasset" -> Must find exactly 1 file + DirScanProbeContext jassetContext = { .FoundCount = 0 }; + bool success = IterateDirectory(WrapString(testDirPath), WrapString(".jasset"), &jassetContext, DummyFileCallback); + Assert(success); + Assert(jassetContext.FoundCount == 1); + + // Scan without filter -> Must find both files + DirScanProbeContext allContext = { .FoundCount = 0 }; + success = IterateDirectory(WrapString(testDirPath), {}, &allContext, DummyFileCallback); + Assert(success); + Assert(allContext.FoundCount == 2); + + // Clean up temporary test files + DeleteFileA(file1); + DeleteFileA(file2); + RemoveDirectoryA(testDirPath); + + scratch_end(scratch); + } + } // namespace + + void WorldUnitTest() + { + Log(LogLevel::Message, LogCategory::Game, "========================================"); + Log(LogLevel::Message, LogCategory::Game, "Starting Distributed Entity & OFPE Unit Tests..."); + + TestEntityIDBitmasking(); + TestSessionCounterContinuity(); + TestDirectoryIteration(); + + Log(LogLevel::Message, LogCategory::Game, "Distributed Entity & OFPE Unit Tests Passed Successfully."); + Log(LogLevel::Message, LogCategory::Game, "========================================"); + } +} // namespace UnitTest + +#endif +``` + +### 8.2 Verification Checklist for the Developer +- [ ] **Bitmask Correctness**: Run `WorldUnitTest()` and ensure bit extraction, shift math, and 56-bit boundary validations trigger zero asserts. +- [ ] **Environment Variable Override**: Set `set JULIET_USER_ID=77` in the shell and run the application. Verify `GetLocalUserID()` outputs `77` in the startup log. +- [ ] **FNV-1a Hash Verification**: Unset `JULIET_USER_ID` and verify the engine generates a stable, non-zero UserID derived from the current Windows user profile. +- [ ] **Multi-Branch Merge Simulation**: + 1. Check out Git branch `test-user-a`, create 5 entities, and commit. + 2. Check out Git branch `test-user-b` from `main`, create 5 entities with a different `JULIET_USER_ID`, and commit. + 3. Merge `test-user-a` into `test-user-b`. Verify Git completes the merge automatically with zero conflicts in the `Entities/` directory. +- [ ] **Two-Phase Forward Reference Test**: Define a test entity `Child` referencing `ParentID`. Ensure `Child` loads successfully even when its `.jasset` file is processed prior to `Parent.jasset`. +- [ ] **Atomic Save Resilience**: Kill the engine process midway through a world save operation. Confirm no corrupted or zero-byte `.jasset` files remain in the target world directory. diff --git a/Game/Plans/04_Entity_Templates_And_Inheritance.md b/Game/Plans/04_Entity_Templates_And_Inheritance.md new file mode 100644 index 0000000..f2e7235 --- /dev/null +++ b/Game/Plans/04_Entity_Templates_And_Inheritance.md @@ -0,0 +1,933 @@ +# Juliet Game Engine: Technical Specification & Implementation Plan +# 04: Entity Templates ("Blueprints / Prefabs") & Delta Inheritance + +- **Document ID**: ENG-PLAN-04 +- **Component**: Game Architecture / Entity Component System / Asset Pipeline / Editor +- **Author**: Senior Engine Architect +- **Status**: Draft / Approved for Implementation +- **Target Engine**: Juliet Engine (Milestone: 3D Platformer / World System) +- **Target File**: `w:\Classified\Juliet\Game\Plans\04_Entity_Templates_And_Inheritance.md` + +--- + +## 1. Executive Summary & Architecture Goals + +### 1.1 Motivation & Context +As the Juliet engine evolves toward supporting complex interactive worlds, populating scenes by manually constructing raw entity memory from hardcoded values or monolithic binary streams (`world.bin`) creates severe scalability bottlenecks. Designers and environment artists require reusable entity archetypes—such as foliage, dynamic destructibles, hazards, enemies, and interactive props—that can be authored once and instantiated thousands of times with custom overrides. + +In modern commercial engines, this pattern is foundational: +- **Unreal Engine**: Blueprint classes (`.uasset`) serve as class archetypes, instanced as Actors with per-instance component property overrides. +- **Unity**: Prefabs (`.prefab`) serve as asset archetypes, instanced in scenes with explicit serialized property modification lists. + +In the **Juliet Game Engine**, this feature is formalized as **Entity Templates**. + +### 1.2 Architectural Goals +1. **Unified File Format (`.jasset`)**: + Templates and world entity instances must share the exact same human-readable, version-control-friendly Key-Value (KV) file format. A template is simply an entity `.jasset` file located within `Assets/Templates/`, while a world entity is a `.jasset` file located within `Assets/Worlds//Entities/` that references a parent template via metadata. +2. **Zero Format Duplication**: + There must be no separate "prefab file format" versus "entity instance file format". The same serialization/deserialization code paths parse both archetypes and instances, completely eliminating format divergence, schema version desynchronization, and redundant parsing logic. +3. **Delta Property Overrides (Sparse Inheritance)**: + Instances only serialize the fields that intentionally deviate from their template archetype. Any property omitted in the instance `.jasset` retains the exact bitwise value defined by the template archetype. +4. **Zero Runtime Inheritance Overhead**: + No virtual table lookups, no runtime inheritance trees, and no pointer chasing at tick time. At instantiation time, the template archetype's memory footprint is blitted into contiguous engine arrays (`EntityManager::by_type`), and instance delta overrides are parsed directly into that memory block. Once instantiated, an entity created from a template executes with identical CPU cache locality and zero performance penalty compared to a hardcoded entity. +5. **Arena-Centric Memory Model**: + All template assets are loaded into a dedicated, isolated `TemplateArena`. World entity instances allocate their runtime state out of `World::WorldArena` and `EntityManager::by_type[kind].arena`. Scratch computations during parsing utilize thread-local scratch arenas (`scratch_begin` / `scratch_end`). No heap allocation (`malloc`, `calloc`, `new`) is permitted. +6. **Strict Engine Conformance**: + - Zero exceptions (`noexcept` by design). + - Strict warning compliance (`-Wall -Wextra -Werror` / `/W4 /WX`). + - Strict explicit casting (`static_cast`, `reinterpret_cast`; C-style casts strictly prohibited). + - Universal `[[nodiscard]]` on all value-returning queries and allocators. + - Consistent naming (`CamelCase` for types, functions, and member variables). + - Comprehensive precondition assertions via `Assert`. + - Mandatory curly braces `{}` across all control flow statements. + +--- + +## 2. The Template Linking Mechanism + +### 2.1 File Format Specification (`.jasset`) +All templates and entities in Juliet use the `.jasset` text format. The format consists of: +- **Comment Lines**: Lines starting with `#` or `//` are treated as comments and ignored. +- **Metadata Directives**: Lines starting with `; ` represent engine-level structural metadata (e.g., entity type, template references, versioning). +- **Key-Value Pairs**: Key name followed by a colon and a space (`: `). + +```ini +# ============================================================================== +# Assets/Templates/RockLarge.jasset +# Entity Archetype: Large Mossy Rock +# ============================================================================== +; entity_type: Inert + +Position: 0.0, 0.0, 0.0 +MeshAsset: Assets/Meshes/Rock_01.obj +Scale: 1.0, 1.0, 1.0 +Mass: 250.0 +IsDestructible: true +Health: 100.0 +``` + +### 2.2 The `; template` Directive +When an entity instance is authored for a world, it specifies its archetype via the `; template` directive: + +```ini +# ============================================================================== +# Assets/Worlds/Level01/Entities/RockLarge_042.jasset +# Instance: Rock Large #042 in Level 01 +# ============================================================================== +; template: Assets/Templates/RockLarge.jasset + +Position: 142.5, 12.0, -84.2 +Scale: 1.4, 1.4, 1.4 +Health: 50.0 +``` + +Notice the power of sparse delta overrides in this format: +- `Position` is set to the instance's unique world coordinates. +- `Scale` is enlarged to $1.4\times$. +- `Health` is damaged down to $50.0$. +- Properties omitted—`MeshAsset`, `Mass`, and `IsDestructible`—are not duplicated in the file. They automatically inherit the authoritative values from `RockLarge.jasset`. + +### 2.3 Relative Path vs CRC Identifier +To balance human-readability in source control with blazing runtime lookup speeds: +- **Asset Authoring & Storage**: Files store canonical workspace-relative paths (e.g., `Assets/Templates/RockLarge.jasset`). +- **Runtime Representation**: The engine computes a 32-bit CRC (`crc32`) of the normalized relative path string. + - Runtime lookup into `EntityTemplateCache` operates via `uint32 TemplateCrc`. + - In `JULIET_DEBUG` builds, the original `String TemplatePath` is retained within the cached struct for diagnostic logging, error messages, and inspector UI display. + +```cpp +constexpr uint32 kInvalidTemplateCrc = 0; + +[[nodiscard]] inline uint32 HashTemplatePath(String path) +{ + Assert(IsValid(path)); + return crc32(path.Str, path.Size); +} +``` + +### 2.4 Standalone Entities vs Templated Instances +The engine architecture seamlessly unifies two entity categories: + +| Entity Category | `; template` Directive Present? | Memory Initialization Source | Primary Use Case | +| :--- | :--- | :--- | :--- | +| **Standalone Entity** | **No** | Zero-initialized memory block (`ArenaPushStruct` / `MemSet`). All properties must be specified in the instance `.jasset`. | Unique, one-off actors (e.g., Level Script Trigger, Primary Player Spawn Marker, Boss Controller). | +| **Templated Instance** | **Yes** | Blitted directly from `CachedTemplate::DefaultDerivedMemory`. Instance `.jasset` only specifies delta property overrides. | Reusable archetypes (e.g., environmental props, foliage, pickups, enemy minions, projectiles). | + +--- + +## 3. In-Engine Template Caching + +### 3.1 Memory Layout & `TemplateArena` Isolation +Reading and parsing text files from storage is orders of magnitude slower than memory copying. In a world containing 10,000 instances of `GrassClump` and 2,000 instances of `RockLarge`, disk I/O and text tokenization must occur **exactly once per archetype**. + +To achieve zero memory fragmentation and eliminate dynamic heap allocations: +1. The engine instantiates an isolated `TemplateArena` at subsystem startup. +2. When a template is requested, the system checks the `EntityTemplateCache`. +3. If not cached, the template `.jasset` file is loaded from disk into temporary scratch memory, parsed, and baked into a contiguous binary archetype memory snapshot inside `TemplateArena`. +4. The parsed archetype snapshot remains resident in `TemplateArena` for the lifetime of the application or world session. + +``` ++---------------------------------------------------------------------------------+ +| TemplateArena | ++---------------------------------------------------------------------------------+ +| [CachedTemplate #0] | +| - TemplateCrc: 0x9A4B12F0 ("Assets/Templates/RockLarge.jasset") | +| - EntityKind: Pointer to Inert::Kind | +| - DefaultBase: { ID=0, Kind=Inert::Kind, Derived=nullptr, X=0, Y=0, Z=0 } | +| - DefaultDerivedMemory: [ sizeof(Inert) binary snapshot: MeshInstance=... ] | ++---------------------------------------------------------------------------------+ +| [CachedTemplate #1] | +| - TemplateCrc: 0x4D2E88C1 ("Assets/Templates/CoinPickup.jasset") | +| - EntityKind: Pointer to Collectible::Kind | +| - DefaultBase: { ID=0, Kind=Collectible::Kind, ... } | +| - DefaultDerivedMemory: [ sizeof(Collectible) binary snapshot: Value=100 ] | ++---------------------------------------------------------------------------------+ +| ... Free Arena Capacity for Additional Archetypes ... | ++---------------------------------------------------------------------------------+ +``` + +### 3.2 Data Structures +The template caching infrastructure is defined with explicit, strictly-typed C++ structures adhering to Juliet guidelines: + +```cpp +#pragma once + +#include +#include +#include +#include +#include +#include +#include +#include + +struct CachedTemplate +{ + uint32 TemplateCrc = 0; + Class* EntityKind = nullptr; + Entity DefaultBase = {}; + void* DefaultDerivedMemory = nullptr; + size_t DerivedMemorySize = 0; + +#if JULIET_DEBUG + String SourcePath = {}; +#endif +}; + +constexpr size_t kMaxCachedTemplates = 1024; + +struct EntityTemplateCache +{ + Arena* CacheArena = nullptr; + VectorArena Templates; +}; + +// Subsystem API +[[nodiscard]] EntityTemplateCache* InitEntityTemplateCache(NonNullPtr parentArena); +void ShutdownEntityTemplateCache(NonNullPtr cache); +[[nodiscard]] CachedTemplate* GetOrLoadTemplate(NonNullPtr cache, String relativePath); +[[nodiscard]] CachedTemplate* FindCachedTemplate(NonNullPtr cache, uint32 templateCrc); +void InvalidateTemplateCache(NonNullPtr cache); +``` + +### 3.3 Loading Pipeline: Disk to Archetype Memory Snapshot +When `GetOrLoadTemplate` is invoked with a relative path: +1. **Hash & Probe**: Compute `crc32` of `relativePath`. Search `cache->Templates` for an existing entry. If found, immediately return the cached pointer ($O(1)$ amortized). +2. **Scratch Allocation**: Open a scratch arena frame (`TempArena scratch = scratch_begin(nullptr, 0);`). +3. **Disk I/O**: Resolve the path via `GetAssetPath(scratch.Arena, relativePath)` and load the entire file into a raw byte buffer via `LoadFile(scratch.Arena, fullPath)`. + - Precondition check: `Assert(fileBuffer.Data != nullptr);` +4. **Header Parse**: Scan the file buffer for `; entity_type: `. Look up the corresponding `Class*` via the global entity registry (`kEntity_type_class_ptr`). +5. **Archetype Memory Allocation**: Allocate `DefaultDerivedMemory` directly out of `cache->CacheArena`: + ```cpp + void* archetypeMem = ArenaPushSize(cache->CacheArena, + entityClass->size_of, + entityClass->alignment, + true JULIET_DEBUG_PARAM("CachedTemplateDerived")); + ``` +6. **Default Base Setup**: Initialize a local `Entity defaultBase`: + ```cpp + Entity defaultBase = {}; + defaultBase.Kind = entityClass; + defaultBase.Derived = archetypeMem; + ``` +7. **Back-Pointer Linking**: Set `entity_template::Base` in the archetype memory: + ```cpp + auto* archetypeTemplate = reinterpret_cast(archetypeMem); + archetypeTemplate->Base = &cachedEntry->DefaultBase; + ``` +8. **KV Property Parsing**: Parse all key-value pairs in the template `.jasset` file and write their deserialized values directly into `archetypeMem` and `defaultBase`. +9. **Cache Insertion**: Store the fully baked `CachedTemplate` record in `cache->Templates`. +10. **Scratch Release**: Release the temporary file buffer (`scratch_end(scratch);`). + +--- + +## 4. Instantiation & Delta Property Overrides + +### 4.1 Memory Architecture: Base vs. Derived Entities +In Juliet, an entity is split into two tightly coupled structures: +1. **`Entity` (Base)**: Contains universal spatial and lifecycle fields: + ```cpp + struct Entity final + { + EntityID ID = 0; + Class* Kind = nullptr; + DerivedType Derived = nullptr; // Points to the specialized struct + float X = 0.0f; + float Y = 0.0f; + float Z = 0.0f; + }; + ``` +2. **`Derived` (Specialized Type)**: e.g., `Inert`, `Collectible`, `Player`. The first member is always `DECLARE_ENTITY()`, which expands to: + ```cpp + Entity* Base; // Back-pointer to the base Entity + static Class* Kind; + ``` + +Because `DerivedType` stores a back-pointer (`Base`) to `Entity`, **a shallow memory copy of an archetype invalidates this pointer**! The loading pipeline must explicitly restore this invariant. + +### 4.2 The 5-Step Instantiation Pipeline +When an entity instance is spawned or deserialized from a `.jasset` file, the engine executes this strict 5-step sequence: + +``` +[ Step 1: Pre-allocate ] + - manager.Entities.PushBack(Entity{}) + - ArenaPushSize(manager.by_type[Kind].arena) + | + v +[ Step 2: Copy Archetype Memory ] + - MemCopy(instanceDerivedMem, template->DefaultDerivedMemory, size_of) + - instanceBase->X/Y/Z = template->DefaultBase.X/Y/Z + | + v +[ Step 3: Fixup Base Back-Pointer ] <--- CRITICAL ARCHITECTURAL STEP + - instanceDerived->Base = instanceBase + - instanceBase->Derived = instanceDerived + | + v +[ Step 4: Parse Instance KV Delta ] + - Scan instance .jasset tokens + - Match each key to struct property offset + - Overwrite only specified properties + | + v +[ Step 5: Finalize & Post-Init ] + - Assign Unique EntityID + - Register with Render/Physics systems +``` + +#### Step-by-Step Code Execution + +```cpp +[[nodiscard]] Entity* InstantiateEntityFromTemplate(EntityManager& manager, + NonNullPtr templateArchetype, + NonNullPtr scratchArena, + String instanceKvContent) +{ + Assert(templateArchetype->EntityKind != nullptr); + Assert(templateArchetype->DefaultDerivedMemory != nullptr); + + Class* entityKind = templateArchetype->EntityKind; + const size_t derivedSize = entityKind->size_of; + const size_t derivedAlign = entityKind->alignment; + + // ------------------------------------------------------------------------- + // STEP 1: Pre-allocate instance in EntityManager + // ------------------------------------------------------------------------- + Entity baseEntity = {}; + baseEntity.ID = EntityManager::ID++; + baseEntity.Kind = entityKind; + + // Push into flat base vector + manager.Entities.PushBack(baseEntity); + Entity* instanceBase = manager.Entities.Back(); + Assert(instanceBase != nullptr); + + // Allocate memory in the typed array arena + auto* instanceDerived = reinterpret_cast( + ArenaPushSize(manager.by_type[entityKind->kind].arena, + derivedSize, + derivedAlign, + false JULIET_DEBUG_PARAM(kEntity_type_names[entityKind->kind])) + ); + Assert(instanceDerived != nullptr); + + // Track array head if first element + if (manager.by_type[entityKind->kind].array == nullptr) + { + manager.by_type[entityKind->kind].array = instanceDerived; + } + manager.by_type[entityKind->kind].count += 1; + + // ------------------------------------------------------------------------- + // STEP 2: Copy cached template defaults into instance derived memory + // ------------------------------------------------------------------------- + MemCopy(instanceDerived, templateArchetype->DefaultDerivedMemory, derivedSize); + + // Inherit base spatial defaults + instanceBase->X = templateArchetype->DefaultBase.X; + instanceBase->Y = templateArchetype->DefaultBase.Y; + instanceBase->Z = templateArchetype->DefaultBase.Z; + + // ------------------------------------------------------------------------- + // STEP 3: Ensure derived back-pointer points to THIS instance's base + // ------------------------------------------------------------------------- + instanceDerived->Base = instanceBase; + instanceBase->Derived = instanceDerived; + + // ------------------------------------------------------------------------- + // STEP 4: Parse instance .jasset KV nodes over the memory + // ------------------------------------------------------------------------- + if (IsValid(instanceKvContent)) + { + ApplyKvDeltaOverrides(instanceBase, instanceDerived, entityKind, instanceKvContent); + } + + // ------------------------------------------------------------------------- + // STEP 5: Post-Instantiation Initialization + // ------------------------------------------------------------------------- + // If the entity is an Inert mesh, update its graphics transform + if (entityKind->kind == ENTITY(Inert)) + { + auto* inert = reinterpret_cast(instanceDerived); + if (inert->MeshInstance != indexMax) + { + SetMeshInstanceTransform(inert->MeshInstance, + MatrixTranslation(instanceBase->X, instanceBase->Y, instanceBase->Z)); + } + } + + return instanceBase; +} +``` + +### 4.3 Key-Value Parsing & Reflection Binding +To apply delta overrides, the engine maps parsed string keys to memory offsets. Juliet utilizes lightweight property reflection metadata registered on each `Class`: + +```cpp +enum class PropertyType : uint8 +{ + Float, + Int32, + Bool, + Vector3, + MeshAsset, + String +}; + +struct PropertyDescriptor +{ + String Name; + size_t Offset; + PropertyType Type; + bool IsBaseProperty; // True if located on Entity, false if on Derived +}; + +void ApplyKvDeltaOverrides(Entity* base, void* derived, Class* cls, String kvContent) +{ + Assert(base != nullptr); + Assert(derived != nullptr); + Assert(cls != nullptr); + + TempArena scratch = scratch_begin(nullptr, 0); + + KvParser parser = InitKvParser(kvContent); + KvPair pair = {}; + while (NextKvPair(&parser, &pair)) + { + // Directives like '; template' or '; entity_type' are skipped + if (pair.Key.Size > 0 && pair.Key.Str[0] == ';') + { + continue; + } + + // Check base properties first + if (StringCompare(pair.Key, ConstString("Position")) == 0) + { + Vector3 pos = ParseVector3(pair.Value); + base->X = pos.X; + base->Y = pos.Y; + base->Z = pos.Z; + continue; + } + + // Look up property in Class reflection table + const PropertyDescriptor* prop = FindPropertyDescriptor(cls, pair.Key); + if (prop != nullptr) + { + void* targetField = static_cast(derived) + prop->Offset; + switch (prop->Type) + { + case PropertyType::Float: + { + *reinterpret_cast(targetField) = ParseFloat(pair.Value); + break; + } + case PropertyType::Int32: + { + *reinterpret_cast(targetField) = ParseInt32(pair.Value); + break; + } + case PropertyType::Bool: + { + *reinterpret_cast(targetField) = ParseBool(pair.Value); + break; + } + case PropertyType::MeshAsset: + { + String meshPath = TrimWhitespace(pair.Value); + MeshAssetID meshId = LoadMesh(meshPath); + *reinterpret_cast(targetField) = meshId; + break; + } + default: + { + break; + } + } + } + } + + scratch_end(scratch); +} +``` + +--- + +## 5. Editor Workflow & Operations (Romeo / ImGui) + +### 5.1 "Create Template from Entity" Workflow +An artist or designer often crafts an intricate entity in the active level (configuring mesh, collider, and scale) and decides it should become a reusable archetype. + +#### Sequence Diagram / Workflow: +1. **User Action**: Right-click an entity in the Romeo World Editor Outliner $\rightarrow$ Select *"Convert to Template..."*. +2. **Modal Dialog**: The editor prompts for the template asset name (e.g., `SpikeTrap_Large`). +3. **Sanitize World Transform**: + - The world position (`X, Y, Z`) is sanitized to origin (`0.0, 0.0, 0.0`) for the template asset. + - Rotations and local scale are preserved. +4. **Serialize Archetype**: + - Write `Assets/Templates/SpikeTrap_Large.jasset` containing: + - `; entity_type: ` + - All authored property values. +5. **Convert Live Instance**: + - The selected world entity is transformed into an instance of the newly created template. + - The entity is assigned the template CRC: `entity->TemplateCrc = crc32("Assets/Templates/SpikeTrap_Large.jasset")`. + - When the world is saved, this entity serializes as a sparse delta referencing the template! + +```cpp +#if JULIET_EDITOR +bool CreateTemplateFromEntity(World& world, + size_t entityIndex, + String templateName, + NonNullPtr scratchArena) +{ + auto& manager = *world.EntityManager; + Assert(entityIndex < manager.Entities.Size()); + + Entity* sourceEntity = &manager.Entities[entityIndex]; + Class* entityKind = sourceEntity->Kind; + void* derivedMem = sourceEntity->Derived; + + // Format destination template path + String templatePath = Format(scratchArena, "Assets/Templates/{}.jasset", CStr(templateName)); + String fullDiskPath = GetAssetPath(scratchArena, templatePath); + + // Open IOStream for write + IOStream* fileStream = IOFromFile(scratchArena, fullDiskPath, ConstString("wb")); + if (fileStream == nullptr) + { + LogError(LogCategory::Game, "Failed to open file for template creation: %s", CStr(fullDiskPath)); + return false; + } + + // Write Header Directive + IOPrintf(fileStream, "; entity_type: %s\n\n", kEntity_type_names[entityKind->kind]); + + // Write Origin Position + IOPrintf(fileStream, "Position: 0.0, 0.0, 0.0\n"); + + // Write Derived Properties via Reflection Table + SerializeDerivedPropertiesToKv(fileStream, entityKind, derivedMem); + + IOClose(fileStream); + + // Register with in-engine cache immediately + auto* templateCache = GetGameState()->TemplateCache; + if (templateCache != nullptr) + { + (void)GetOrLoadTemplate(templateCache, templatePath); + } + + LogMessage(LogCategory::Game, "Successfully created template: %s", CStr(templatePath)); + return true; +} +#endif +``` + +### 5.2 "Spawn Instance from Template" Workflow +1. In the Romeo Content Browser, browse `Assets/Templates/`. +2. Drag a `.jasset` template into the 3D viewport, or click *"Spawn Template"* in the World Editor toolbar. +3. The viewport raycasts against the collision mesh/floor to compute `hitPosition`. +4. The editor calls: + ```cpp + CachedTemplate* archetype = GetOrLoadTemplate(templateCache, templatePath); + Entity* newInstance = InstantiateEntityFromTemplate(manager, archetype, scratchArena, {}); + newInstance->X = hitPosition.X; + newInstance->Y = hitPosition.Y; + newInstance->Z = hitPosition.Z; + ``` +5. The entity is immediately live, selectable, and rendered. + +### 5.3 Inspector Delta Highlighting & Property Diffing +To make template inheritance intuitive, the inspector visually flags overridden properties: +- **Default Property**: Rendered in standard gray text. +- **Overridden Property**: Rendered in **bold bright cyan** with an undo/revert icon button `[R]`. + +```cpp +#if JULIET_EDITOR +void RenderEntityPropertyInspector(Entity* entity, CachedTemplate* archetype) +{ + Assert(entity != nullptr); + const bool isTemplated = (archetype != nullptr); + + ImGui::Text("Entity ID: %llu", entity->ID); + if (isTemplated) + { + ImGui::TextColored(ImVec2(0.4f, 0.8f, 1.0f, 1.0f), "Template: %s", archetype->SourcePath.Str); + ImGui::SameLine(); + if (ImGui::SmallButton("Revert All to Template")) + { + RevertEntityToTemplate(entity, archetype); + } + ImGui::Separator(); + } + + // Iterate through properties + Class* cls = entity->Kind; + for (size_t i = 0; i < cls->PropertyCount; ++i) + { + const PropertyDescriptor& prop = cls->Properties[i]; + void* instanceField = static_cast(entity->Derived) + prop.Offset; + void* templateField = isTemplated ? (static_cast(archetype->DefaultDerivedMemory) + prop.Offset) : nullptr; + + const bool isOverridden = isTemplated && (MemCompare(instanceField, templateField, GetPropertySize(prop.Type)) != 0); + + if (isOverridden) + { + ImGui::PushStyleColor(ImGuiCol_Text, ImVec2(0.2f, 1.0f, 1.0f, 1.0f)); + } + + // Render editor widget (DragFloat, InputText, etc.) + RenderPropertyWidget(prop, instanceField); + + if (isOverridden) + { + ImGui::PopStyleColor(); + ImGui::SameLine(); + ImGui::PushID(static_cast(i)); + if (ImGui::SmallButton("R")) + { + // Revert this single property + MemCopy(instanceField, templateField, GetPropertySize(prop.Type)); + } + ImGui::PopID(); + if (ImGui::IsItemHovered()) + { + ImGui::SetTooltip("Revert property to template default"); + } + } + } +} +#endif +``` + +### 5.4 "Revert to Template" Mechanics +Reverting an entire entity to its template archetype restores all properties while strictly preserving world positioning and the instance's unique `EntityID`: + +```cpp +void RevertEntityToTemplate(NonNullPtr instance, NonNullPtr archetype) +{ + Assert(instance->Kind == archetype->EntityKind); + Assert(archetype->DefaultDerivedMemory != nullptr); + + void* derivedMem = instance->Derived; + const size_t derivedSize = archetype->EntityKind->size_of; + + // Preserve the current Base pointer + Entity* basePtr = instance.Get(); + + // 1. Re-copy the archetype defaults + MemCopy(derivedMem, archetype->DefaultDerivedMemory, derivedSize); + + // 2. Re-establish the Base back-pointer + auto* templateDerived = reinterpret_cast(derivedMem); + templateDerived->Base = basePtr; + + // 3. Mark visual / physics state as updated + if (instance->Kind->kind == ENTITY(Inert)) + { + auto* inert = reinterpret_cast(derivedMem); + if (inert->MeshInstance != indexMax) + { + SetMeshInstanceTransform(inert->MeshInstance, + MatrixTranslation(basePtr->X, basePtr->Y, basePtr->Z)); + } + } +} +``` + +--- + +## 6. Step-by-Step Implementation Roadmap + +``` ++-----------------------------------------------------------------------------+ +| Phase 1: Core KV Parser & File Serialization Architecture | +| - Fast, zero-allocation Key-Value streaming parser | +| - Property reflection table integration on Class struct | ++-----------------------------------------------------------------------------+ + | + v ++-----------------------------------------------------------------------------+ +| Phase 2: Template Cache Subsystem (`EntityTemplateCache`) | +| - Dedicated TemplateArena initialization | +| - Archetype loading, CRC hashing, and resident memory snapshot baking | ++-----------------------------------------------------------------------------+ + | + v ++-----------------------------------------------------------------------------+ +| Phase 3: Instantiation & Delta Overrides in `EntityManager` | +| - Implement 5-Step Instantiation Pipeline | +| - Derived back-pointer fixup validation | +| - Sparse delta serialization for world saving | ++-----------------------------------------------------------------------------+ + | + v ++-----------------------------------------------------------------------------+ +| Phase 4: Romeo Editor UI & Inspector Workflow | +| - Template browser window in ImGui | +| - "Create Template from Entity" context menu | +| - Inspector delta highlighting and per-property "Revert" action | ++-----------------------------------------------------------------------------+ + | + v ++-----------------------------------------------------------------------------+ +| Phase 5: Production Verification & Non-Invasive Unit Testing Suite | +| - Comprehensive test cases for cache, delta inheritance, and memory safety | ++-----------------------------------------------------------------------------+ +``` + +### Detailed Milestone Tasks + +| Phase | Target Files | Objective / Deliverable | Success Criteria | +| :--- | :--- | :--- | :--- | +| **Phase 1** | `Juliet/include/Engine/KvParser.h`
`Juliet/src/Engine/KvParser.cpp`
`Juliet/include/Engine/Class.h` | Build a zero-allocation streaming Key-Value parser operating entirely over `String` slices and `Arena*`. Add property descriptor tables to `Class`. | Parses full `.jasset` buffer in $<50\mu\text{s}$ without dynamic allocations. Zero exceptions. | +| **Phase 2** | `Game/Entity/TemplateCache.h`
`Game/Entity/TemplateCache.cpp` | Implement `EntityTemplateCache`, `CachedTemplate`, and on-demand disk loader backed by `TemplateArena`. | Repeated loads of same template return cached pointer in $O(1)$ time without disk reads. | +| **Phase 3** | `Game/Entity/EntityManager.h`
`Game/Entity/EntityManager.cpp`
`Game/Data/World.cpp` | Implement `InstantiateEntityFromTemplate` using the 5-step pipeline. Update world save/load to serialize sparse deltas when `; template` is present. | Instantiated entities correctly retain archetype defaults while applying deltas. `Base` pointer is guaranteed valid. | +| **Phase 4** | `Game/Data/World.cpp`
`Game/Debug/WorldEditorUI.cpp` | Add ImGui widgets for template authoring, template instantiation drag-and-drop, delta highlighting, and "Revert" actions. | Designer can convert an entity to a template and revert modified properties with instant visual update. | +| **Phase 5** | `Game/UnitTest/TemplateUnitTest.h`
`Game/UnitTest/TemplateUnitTest.cpp` | Author non-invasive unit tests validating KV parsing, caching, delta override correctness, back-pointer fixup, and revert operations. | 100% test pass rate with zero memory leaks and all assertions satisfied under `/WX`. | + +--- + +## 7. Production-Grade Unit Testing Plan + +### 7.1 Testing Philosophy & Non-Invasive Framework Rules +In accordance with Juliet engine guidelines: +- **No framework pollution**: Unit tests must not introduce test-only `#ifdef` branches or dummy parameters into production engine systems. +- **Isolated test arenas**: All tests create their own temporary scratch arena or sub-arena and release it upon completion. +- **Deterministic verification**: Every assumption (data alignment, pointer fixups, delta override values) is verified through explicit `Assert` statements. + +### 7.2 Header Specification (`Game/UnitTest/TemplateUnitTest.h`) + +```cpp +#pragma once + +#include + +#if JULIET_DEBUG + +namespace UnitTest +{ + void RunTemplateAndInheritanceUnitTests(); +} + +#endif +``` + +### 7.3 Complete Unit Test Suite Implementation (`Game/UnitTest/TemplateUnitTest.cpp`) + +```cpp +#include + +#if JULIET_DEBUG + +#include +#include +#include +#include +#include +#include +#include +#include + +namespace UnitTest +{ + // ========================================================================= + // Test 1: Key-Value Parser Verification + // ========================================================================= + static void TestKvParser(NonNullPtr testArena) + { + LogMessage(LogCategory::Game, "[UnitTest] Starting TestKvParser..."); + + const char* sampleKv = + "; entity_type: Inert\n" + "# This is a comment\n" + "Position: 10.5, -20.0, 30.25\n" + "Scale: 2.0, 2.0, 2.0\n" + "MeshAsset: Assets/Meshes/Cube.obj\n" + "IsActive: true\n"; + + KvParser parser = InitKvParser(WrapString(sampleKv)); + KvPair pair = {}; + + // 1. Directive + Assert(NextKvPair(&parser, &pair)); + Assert(StringCompare(pair.Key, ConstString("; entity_type")) == 0); + Assert(StringCompare(pair.Value, ConstString("Inert")) == 0); + + // 2. Position + Assert(NextKvPair(&parser, &pair)); + Assert(StringCompare(pair.Key, ConstString("Position")) == 0); + Vector3 pos = ParseVector3(pair.Value); + Assert(pos.X == 10.5f); + Assert(pos.Y == -20.0f); + Assert(pos.Z == 30.25f); + + // 3. Scale + Assert(NextKvPair(&parser, &pair)); + Assert(StringCompare(pair.Key, ConstString("Scale")) == 0); + + // 4. MeshAsset + Assert(NextKvPair(&parser, &pair)); + Assert(StringCompare(pair.Key, ConstString("MeshAsset")) == 0); + Assert(StringCompare(pair.Value, ConstString("Assets/Meshes/Cube.obj")) == 0); + + // 5. IsActive + Assert(NextKvPair(&parser, &pair)); + Assert(StringCompare(pair.Key, ConstString("IsActive")) == 0); + Assert(ParseBool(pair.Value) == true); + + // End of stream + Assert(!NextKvPair(&parser, &pair)); + + LogMessage(LogCategory::Game, "[UnitTest] TestKvParser PASSED."); + } + + // ========================================================================= + // Test 2: Template Cache & Single-Load Invariant + // ========================================================================= + static void TestTemplateCache(NonNullPtr testArena) + { + LogMessage(LogCategory::Game, "[UnitTest] Starting TestTemplateCache..."); + + EntityTemplateCache* cache = InitEntityTemplateCache(testArena); + Assert(cache != nullptr); + + String templatePath = ConstString("Assets/Templates/TestRock.jasset"); + + // First load: loads and caches + CachedTemplate* firstLoad = GetOrLoadTemplate(cache, templatePath); + Assert(firstLoad != nullptr); + Assert(firstLoad->TemplateCrc == HashTemplatePath(templatePath)); + Assert(firstLoad->EntityKind == Inert::Kind); + + // Second load: must return identical cached pointer without re-allocating + CachedTemplate* secondLoad = GetOrLoadTemplate(cache, templatePath); + Assert(secondLoad == firstLoad); + + // Verify lookup by CRC + CachedTemplate* crcLookup = FindCachedTemplate(cache, firstLoad->TemplateCrc); + Assert(crcLookup == firstLoad); + + ShutdownEntityTemplateCache(cache); + LogMessage(LogCategory::Game, "[UnitTest] TestTemplateCache PASSED."); + } + + // ========================================================================= + // Test 3: 5-Step Instantiation Pipeline & Derived Back-Pointer Fixup + // ========================================================================= + static void TestInstantiationPipeline(NonNullPtr testArena) + { + LogMessage(LogCategory::Game, "[UnitTest] Starting TestInstantiationPipeline..."); + + // Setup mock World and EntityManager + World world = {}; + InitWorld(&world, testArena); + InitEntityManager(&world); + auto& manager = *world.EntityManager; + + // Construct a synthetic template archetype + CachedTemplate mockTemplate = {}; + mockTemplate.TemplateCrc = 0xABCD1234; + mockTemplate.EntityKind = Inert::Kind; + mockTemplate.DefaultBase.X = 1.0f; + mockTemplate.DefaultBase.Y = 2.0f; + mockTemplate.DefaultBase.Z = 3.0f; + + Inert defaultInert = {}; + defaultInert.MeshInstance = 42; // Template default + mockTemplate.DefaultDerivedMemory = &defaultInert; + mockTemplate.DerivedMemorySize = sizeof(Inert); + + // Delta content overrides Position and leaves MeshInstance unspecified + String instanceKv = ConstString("Position: 100.0, 200.0, 300.0\n"); + + Entity* instance = InstantiateEntityFromTemplate(manager, &mockTemplate, testArena, instanceKv); + Assert(instance != nullptr); + + // Verify Step 1 & 2: Base spatial delta applied, non-overridden derived property retained + Assert(instance->X == 100.0f); + Assert(instance->Y == 200.0f); + Assert(instance->Z == 300.0f); + + auto* inertDerived = DownCast(instance); + Assert(inertDerived != nullptr); + Assert(inertDerived->MeshInstance == 42); // Retained from template! + + // Verify Step 3: CRITICAL back-pointer fixup check + Assert(inertDerived->Base == instance); + Assert(instance->Derived == inertDerived); + + ShutdownEntityManager(); + ShutdownWorld(&world); + + LogMessage(LogCategory::Game, "[UnitTest] TestInstantiationPipeline PASSED."); + } + + // ========================================================================= + // Test 4: Delta Override & Revert Functionality + // ========================================================================= + static void TestDeltaOverrideAndRevert(NonNullPtr testArena) + { + LogMessage(LogCategory::Game, "[UnitTest] Starting TestDeltaOverrideAndRevert..."); + + World world = {}; + InitWorld(&world, testArena); + InitEntityManager(&world); + auto& manager = *world.EntityManager; + + CachedTemplate mockTemplate = {}; + mockTemplate.TemplateCrc = 0x11223344; + mockTemplate.EntityKind = Inert::Kind; + mockTemplate.DefaultBase.X = 0.0f; + mockTemplate.DefaultBase.Y = 0.0f; + mockTemplate.DefaultBase.Z = 0.0f; + + Inert defaultInert = {}; + defaultInert.MeshInstance = 100; + mockTemplate.DefaultDerivedMemory = &defaultInert; + mockTemplate.DerivedMemorySize = sizeof(Inert); + + // Instantiate with delta + String instanceKv = ConstString("MeshInstance: 999\nPosition: 5.0, 5.0, 5.0\n"); + Entity* instance = InstantiateEntityFromTemplate(manager, &mockTemplate, testArena, instanceKv); + + auto* inertDerived = DownCast(instance); + Assert(inertDerived->MeshInstance == 999); // Delta applied + + // Execute Revert to Template + RevertEntityToTemplate(instance, &mockTemplate); + + // Verify property restored to template default + Assert(inertDerived->MeshInstance == 100); + + // Verify world position is preserved across revert + Assert(instance->X == 5.0f); + Assert(instance->Y == 5.0f); + Assert(instance->Z == 5.0f); + + // Verify back-pointer invariant preserved after revert + Assert(inertDerived->Base == instance); + + ShutdownEntityManager(); + ShutdownWorld(&world); + + LogMessage(LogCategory::Game, "[UnitTest] TestDeltaOverrideAndRevert PASSED."); + } + + // ========================================================================= + // Master Runner + // ========================================================================= + void RunTemplateAndInheritanceUnitTests() + { + TempArena scratch = scratch_begin(nullptr, 0); + + TestKvParser(scratch.Arena); + TestTemplateCache(scratch.Arena); + TestInstantiationPipeline(scratch.Arena); + TestDeltaOverrideAndRevert(scratch.Arena); + + scratch_end(scratch); + LogMessage(LogCategory::Game, "[UnitTest] All Entity Template & Inheritance tests PASSED successfully."); + } +} + +#endif