AMWdev.amoosed.com For developers

FOR DEVELOPERS / TECHNICAL REFERENCE

Native ABI roadmap toward 1.0

The native C ABI roadmap, package parity gates, and work remaining before a stable 1.0 release.

Illustrated protocol guide → · Technical reference →

The 0.28 GameReader C++ API is a compile-time interface. C++ standard library types, exceptions, virtual tables, and compiler runtime choices cross that interface, so it is not a binary compatibility promise. The experimental AmoosedWebGameReaderC shared library proves a separate C boundary with a plain-C caller. Its amw_preview_* symbols may change before 1.0. The detailed 1.0 C ABI design proposal defines the versioning, ownership, cancellation, and error decisions still to be implemented. The proposed 1.x evolution checker now has tests for additive minor releases and breaking edits. It reads candidate declarations from a future public C header; it does not turn this preview into a stable ABI. The first 1.0 release still needs its actual header, binary, and immutable baseline before the checker becomes a package gate. The native compiler has a separate owner-side proposed C header for static-site build and offline verification. It has C/C++ compile checks but no implementation, binary, or stability promise. Its amwc1_ namespace and freeze gate are independent of the general Reader's amw1_ work. The separate amwc_preview_* revision-8 library exercises local site verification and hosted, member, and external-index builds through a plain-C caller. Its header and binary are owner-side. Bundle and WAV work now observes the cancel flag within their long loops, with a threaded WAV test on both OSes. A C build fixture now cancels hosted, member, and external editions after observing site staging and checks cleanup on Windows and Linux. Late external credential publication is now covered by a deterministic internal C++ fixture: an observer cancels immediately after the private directory is published, and neither destination nor staging directory survives on Windows or Linux. A revision-8 C build-progress callback lets a caller cancel at that same checkpoint. Its direct C API fixture verifies site, credentials, and stages are removed on Windows and Linux. The remaining build status taxonomy, clean-package consumers, and broader command parity remain before a compiler 1.0 release or a complete authoring SDK. The same threaded fixture creates a competing site destination after staging starts. Hosted, member, and external builds return OUTPUT_EXISTS, retain the other process's marker, remove their stage, and publish no external credentials on Windows and Linux. An external build also faces a competing private credential destination after site staging starts. It returns OUTPUT_EXISTS, preserves that marker, removes its stages, and publishes no site on both hosts. Revision 5 reports existing output, malformed configuration, missing input, verified-site failure, and output I/O through structured categories. Plain-C fixtures exercise existing output, malformed configuration, missing input, and a regular file blocking the output parent on Windows and Linux. Site and private credential destination collisions are also covered. Staged-site verification failure is now covered by a C fixture that damages a carrier after staging, gets VERIFICATION_FAILED, and sees no site or stage on Windows and Linux. The wider failure matrix remains before the categories can freeze. Revision 6 adds a query for the compiled compiler capabilities and zeroes its caller-owned outputs on invalid arguments. Its bits are preview-only; the amwc1_ assignments remain a proposal until the 1.0 freeze gate. Revision 6 introduced a checked client contract snapshot for the Windows calling convention, opaque handle, numeric statuses, capability bits, and function declarations. Developer-site and native parity builds reject a changed contract at the same revision. Binary export and plain-C behavior gates remain separate; this is not an immutable 1.0 baseline. Revision 7 adds a build-progress callback, two phase constants, and a capability bit. Its client snapshot captures the callback declaration as well as the existing signatures and numeric values. Revision 8 adds a SITE_STAGED notification before verification and a new checked snapshot. This phase says bytes are ready to inspect, not yet trusted. The revision-8 compiler C fixture now confirms that build and verify calls consume a non-null operation even when a required output or error buffer is missing. The preview implementation follows that documented single-use rule on Windows and Linux; the 1.0 compiler contract still needs its own package gate.

Current prototype

include/AmoosedWeb/PreviewC.h exports catalogue parsing and enumeration, plus Windows and Linux hosted/external index opening, full resource metadata enumeration, member edition ID query, game session refresh and item fetch, and verified resource download for byte-addressable carriers, Windows PNG low-bit carriers, AWF1/AWF2 sparse-frame carriers, and WAV tone carriers. Callers pass bytes with explicit lengths, receive opaque catalogue, site, or game handles, query fixed-width integer values, and copy strings into caller-owned buffers. No C++ object or allocation crosses the boundary. The DLL catches parser and download exceptions and reports a status and optional bounded error text. A site handle keeps the verified index and HTTP origin together; its cancel entry point may be called from another thread. The caller must wait for in-flight operations to finish before releasing a handle. Revision 7 adds site_prepare plus hosted/external site_load calls: prepare performs no network I/O, so the caller can share the handle and cancel an active index load. It returns a distinct CANCELLED status for cancelled index loading, download, game refresh, and game fetch. The older convenience open calls remain but cannot cancel until they return a handle. Delayed local fixtures exercise in-flight .NET index-opening and download cancellation; the download leaves neither a final file nor a staging file. tests/PreviewCTests.c compiles as C and exercises parse failure, ownership, revision, item enumeration, size queries, and a too-small buffer. The developer-site/examples/dotnet-abi/ program exercises the same DLL through .NET 10 P/Invoke, including UTF-8 copying and error handling. The CMake tests/PreviewSiteTests.c and the .NET example also open local hosted and external sites, check resource metadata, and download a verified entry resource. The CMake package exports AmoosedWeb::GameReaderC separately from the C++ target. The opt-in Unreal Win64 subsystem calls revision 14 through the same prebuilt DLL. An editor automation test refreshes the hosted catalogue and compares a fetched item's output bytes with its source file. This proves that path on a local fixture, not packaged-game or other-platform support. A separate source-free Unreal preview ZIP passes a clean editor and shipping consumer build and an editor module-load check. It is a local review package, not a promise of ABI stability or a tested packaged game. On Windows, npm run test:game-abi rebuilds and installs the package, runs the plain-C test, and runs the .NET example against the installed DLL.

This is still a preview ABI, not a complete replacement for the published Reader. Revision 9 includes a Windows game session handle that stages hosted or external catalogue refreshes, retains the previous verified snapshot on failure, and fetches a catalogue item through the same verified C++ core. Preview revision 12 added a 64 MiB caller-owned memory read with a verified-index size query, short-buffer result, private plaintext staging, and an operation variant for cancellation. Windows and Linux plain-C fixtures compare exact bytes with file output and check pre-dispatch and in-flight cancellation. The in-flight fixtures assert that cancellation reports zero output length, leaves the entire caller buffer untouched, and permits a successful read on the same site handle afterward. Linux also checks that a damaged carrier leaves the caller buffer unchanged. Windows and Linux plain-C fixtures now open a checksummed index advertising a resource one byte above the 64 MiB memory limit; both the size query and operation read return UNSUPPORTED, report zero output length, and leave caller memory unchanged. The locally packaged .NET preview also lists that oversized entry and raises its numeric and named UNSUPPORTED result on Windows and Linux without allocating the resource. Other language-package parity and the wider package matrix remain 1.0 gates. The C boundary exposes the verified member edition ID but does not parse member credential files or provide structured progress callbacks. Windows and Linux decoders now cover the website carrier modes, but other platforms still need adapters. The .NET Reader and Unreal plugin must retain their current behavior until parity and target-platform tests prove a replacement. The owner-side SDK packaging script now builds a local, DLL-only Windows x64 NuGet review package and verifies installation with an isolated consumer. That package is not in the public release or a claim of 1.0 ABI stability. Revision 8 adds a distinct ALREADY_EXISTS result when a download target exists before staging or appears before publish. Other output errors retain OUTPUT_ERROR; key, verification, and network failures still shared a broad read status at that revision. A delayed carrier fixture creates a destination while the C download is in flight and verifies the new status, preserved bytes, and staging cleanup. Revision 9 passes typed Reader failure categories through hosted and external index loading, game refresh, resource download, and game item fetch. It reports bad format, failed authentication, failed verification, transport failure, and unsupported decoding separately from diagnostic text. Some provider failures and malformed host responses still share broad categories; the numeric values remain preview-only until the 1.0 failure matrix is tested across platforms. Part hashing, complete-resource hash finalization, and hosted or external-index digest checks now report an internal provider failure separately from a digest mismatch. Focused Windows and Linux fixtures inject both outcomes; resource failures leave output unpublished. Hosted-index responses with an unsafe origin, invalid footer or payload range, changed ETag, or incomplete whole-body fallback now report Verification instead of BadFormat. The focused fixture covers those five response paths on Windows and Linux. Other provider and host responses still need classification and wrapper parity before 1.0. The built-in Windows and OpenSSL crypto hosts now expose a classified AES-GCM open result to the C++ Reader. A rejected authentication tag remains Authentication; a provider setup or processing failure becomes Internal for external-index and member-part reads. Windows and Linux vector tests check the rejected-tag result, and mock-host fixtures inject both categories through the index and resource readers. Existing custom C++ crypto hosts retain the older boolean method and its conservative authentication classification. This does not change the preview C signatures or numeric status values.

.NET package replacement gate

The portable C++ core and full preview C ABI now build on Ubuntu 22.04. An OpenSSL-backed non-Windows crypto host passes fixed SHA-256, AES-256-CTR, and authenticated AES-256-GCM vectors there. A libcurl host verifies bounded range, suffix, whole-body, redirect, and cancellation behavior on a local server. Together they open a generated hosted index and recover exact byte-addressable, PNG low-bit, sparse-frame, and WAV tone public resources through a POSIX output adapter and libpng-backed media decoder, including existing-target, output-race, symlinked-parent, and abort cleanup cases. Run npm run test:game-linux for that gate. A plain-C Linux fixture opens a hosted index, downloads exact bytes, rejects overwrite, and exercises pre-dispatch cancellation. It also opens an externally keyed game starter, recovers a public resource, rejects member data with an absent or incorrect key, and recovers exact member bytes with the matching key. The ELF exports exactly the 43 symbols declared in the public header. A source-free Linux x64 review ZIP now passes archive inventory and digest checks, an extracted CMake consumer build, and exact-byte generated-site download. Wider parity fixtures and target-platform checks remain before publication. The output directory is caller-trusted; arbitrary directory rename rights during publication are outside the current guarantee.

The Linux plain-C fixture also cancels a download after the server starts a slow carrier response, verifies that no final or staging file remains, and then downloads the resource through the same site handle. A finished operation clears its transport cancellation state so the next serialized operation is usable. This loopback fixture does not replace target-host cancellation tests. The Linux plain-C gate also changes a byte in a hosted resource part while keeping the verified index intact. The preview ABI must return the typed VERIFICATION_FAILED status, clear the byte count, and leave neither a final file nor a staging file. This covers one corrupt byte carrier; the wider failure-category matrix and real HTTPS hosts remain 1.0 gates. The portable Reader core now returns Cancelled when cancellation arrives during a failing media decode, instead of retaining the decoder's Verification category. A deterministic mock-host fixture checks the two status paths on Windows and Linux; timely interruption of every real media decoder remains unproven.

The published AmoosedReader exposes ListAsync, DownloadAsync, and DownloadAllAsync, supports hosted and external indexes, accepts a separate member key file, preserves an explicit executable-path constructor, and uses CancellationToken. A smaller native-backed implementation must preserve those public behaviors, including the constructor contract or a documented major-version migration. It must copy key bytes from files into bounded native calls without placing them in process arguments, zero temporary key buffers, serialize operations on a site handle, map cancellation to site_cancel, and dispose the handle only after active calls complete. DownloadAllAsync needs the same path validation, directory containment, no-overwrite semantics, and per-resource results as the CLI path. AmoosedNativePreviewReader now exercises those operations side by side on Windows using a separately installed preview DLL. Its local fixtures cover hosted and external indexes, all Windows media codecs, member credential edition matching, cancellation before start, full pull, and no overwrite. A delayed-carrier fixture confirms in-flight cancellation through both the raw C ABI and high-level .NET wrapper leaves no final or staging file. Hostile filesystem races and production target hosts still need testing. A production NuGet package must include the matching native library for each claimed runtime identifier; a platform-limited preview must not silently replace the cross-platform explicit-CLI path.

The npm run test:game-http gate with AMW_TEST_READER_PARITY=1 requires a matching locally built AmoosedWeb.Native NuGet package. It extracts that package's CLI and compares the published CLI-backed .NET facade with the C++ interop facade on hosted, external, member, PNG low-bit, AWF1/AWF2, and WAV fixtures. Resource metadata, downloaded bytes, and a full hosted pull match on the local fixtures. Failure categories, cancellation behavior relative to the CLI facade, hostile filesystem races, and production hosts still need parity testing before changing the default constructor or dropping the large CLI dependency. tools/package-game-sdks.ps1 now requires a matching local owner release, WSL Ubuntu (or AMW_WSL_DISTRO) for the Linux build, and .NET 10. It packages both Windows x64 and Linux x64 C libraries in one local NuGet review artifact. It runs the CLI parity gate plus clean packaged Reader consumers on both platforms before it reports a review SDK ready. The Linux self-contained .NET consumer lists and downloads exact hosted, externally indexed, and member resources. It rejects missing and incorrect member keys without publishing output. The packaged Linux .NET consumer also recovers exact PNG low-bit, sparse-frame, and WAV tone resources. Its clean-package gate checks that a damaged hosted carrier raises AmoosedNativePreviewException with numeric status 11 and named VerificationFailed, without leaving output or a staging file. The Windows clean NuGet consumer now checks the same typed status for both file and memory reads after WAV data, AWF2 payload, and AWF2 marker damage. The packaged CLI and source-level native facade remain the cross-language references for those local fixtures. The same flags remain available for a standalone test run. The parity fixture also checks that existing output is preserved and that malformed, wrong-edition, and wrong-key member credentials fail in the same IOException family. The native preview gives malformed local files a BadFormat status and edition/key mismatches an AuthenticationFailed status. The parity gate also mutates WAV data and both the AWF2 payload and marker, then checks that the packaged CLI and native .NET Readers reject each carrier without publishing output. Exact marker comparison matters: decoding a binary marker with Node.js ascii masked a flipped high bit and was corrected. These local fixtures do not prove behavior on production hosts.

An opt-in AmoosedWeb.Reader.NativePreview .NET 10 review package now links the matching AmoosedWeb.GameReader.Native binary package. It contains a separate namespace, public Reader facade, and result records, but no amw.exe or embedded JavaScript runtime. Clean Windows and Linux consumers verify the exact ABI revision and packaged library. With AMW_TEST_PACKAGED_NATIVE_READER=1 and AMW_GAME_SDK_NUGET_FEED set to the local review package directory, npm run test:game-http also restores the package from an isolated cache and checks exact hosted, member, external, PNG low-bit, sparse-frame, and WAV fixture bytes through its public .NET facade. This does not replace the published AmoosedWeb.Reader; parity and platform coverage remain the migration gate. Its API signatures are documented publicly at dev.amoosed.com/reader-reference.html#preview. Version 0.28.13-preview of the .NET facade exposes the C status number through AmoosedNativePreviewException.StatusCode and a named status enum, preserving the numeric result for unknown future codes. The underlying C DLL remains 0.28.11-preview at ABI revision 14. The preview now reports per-binary capabilities, which the native .NET and Unreal consumers check before using Windows site or game features. Cancellation keeps its .NET exception type. The standalone 0.28.13-preview .NET facade also exposes ReadBytesAsync for resources up to 64 MiB. Its clean Windows and Linux consumers compare returned bytes against hosted, externally indexed, member, and media fixtures; they reject wrong keys and damaged carriers, and the delayed-carrier fixture cancels an in-flight memory read without returning bytes. At that review point the published CLI-backed AmoosedWeb.Reader API remained unchanged. The later local Windows release build selects the C/C++ Reader for the same default API and retains the explicit CLI constructor. Its clean NuGet consumer checks an actual generated-site read; this is not yet a registry release or evidence of complete target-host parity. The same preview facade now exposes synchronous DecodeStandaloneWav through the revision-14 C audio function. A clean Windows consumer and a self-contained Linux consumer compare exact bytes and reject wrong keys and damaged WAVs. It is separate from website resource reads and is not a stable 1.0 operation.

Revision 10 adds a single-use operation handle prepared before worker dispatch. Its cancel entry point is safe to call from another thread; the worker owns the handle until it returns. Operation variants cover hosted and external index load, resource download, game refresh, and item fetch. A cancelled operation does not poison the site or game transport for the next serialized operation. The previous site/game cancellation calls remain for preview compatibility. The 0.28.11-preview C boundary validates strict UTF-8 and rejects embedded NUL in length-delimited origin, game ID/version, resource, item ID, and output-path text before passing it to platform adapters. Credential and member-key bytes remain binary. Malformed direct text returns INVALID_ARGUMENT; malformed catalogue JSON returns BAD_FORMAT in revision 13. Revision 13 also reports NOT_FOUND, INVALID_STATE, and STALE_REVISION for the corresponding verified-index, handle-lifetime, and game rollback outcomes.

Rules proposed for a 1.0 ABI

  1. Freeze a versioned C header and exported symbol set only after the full general Reader operations pass parity tests. Prefix 1.x Reader symbols with amw1_; keep the preview namespace separate. Game and engine calls remain optional preview layers until their own parity gate passes. Give a future native compiler ABI a separate amwc1_ namespace and version.
  2. Use explicit uint32_t/uint64_t widths and opaque handles. Do not expose STL, C++ exceptions, compiler-specific classes, or ownership of memory allocated by the opposite side. Specify the calling convention on Windows.
  3. Make every returned string and byte array a caller-owned copy. Provide a length query and an exact buffer-too-small result. Never return pointers into a handle that can be invalidated by a refresh. The 1.0 resource read will support a bounded 64 MiB caller-owned memory buffer in addition to transactional file output. Its size query uses the verified index; the read stages plaintext privately and changes caller memory only after complete verification. Larger resources use file output.
  4. Freeze numeric status codes, define unknown-code behavior, and return diagnostic text in an optional bounded caller buffer. Never include keys or protected content in diagnostics.
  5. Add new operations without changing existing signatures or struct layout. Callback tables, if needed for HTTP, crypto, cancellation, and output, must carry a struct_size and version; ignore known trailing fields and reject a table smaller than the required prefix. Define thread and callback lifetime rules before accepting callbacks.
  6. Keep the AMW wire-format version, package version, and ABI major/minor separate. A newer library may read old AMW media while preserving the same ABI. An incompatible ABI gets a new major symbol namespace and can coexist.
  7. Test the actual shared Reader library in C and C# consumers. Test Unreal before advertising a stable game or engine capability. On Windows, test x64 DLL loading with a clean install; then build and test each other claimed platform. tools/package-game-sdks.ps1 now checks that the packaged DLL is PE32+ x64 and exports exactly the declarations in PreviewC.h. Repeat this check for each future platform binary and freeze the 1.0 symbol inventory as a compatibility gate.

1.0 release gate

The general Reader gate is hosted and external index open, resource metadata, public and member reads and downloads, all carrier codecs claimed by each platform package, cancellation, transactional output, and precise error states. Run the same fixtures through Node, C ABI, and C# with valid, corrupt, truncated, wrong-key, redirected, and stale-carrier cases. Revision 14 has typed status paths, but their wrapper coverage and exact stable values remain part of the 1.0 freeze. Game catalogue refresh, item fetch, and Unreal require their own parity gate before stable game capability claims; they do not delay a fully proven general Reader 1.0. Ship only public headers, library binaries, import files, runtime dependencies, samples, and versioned documentation for each supported platform. The public SDK terms and the protocol license permit use without a fee when both user and product remain below EUR 1 trillion in annual revenue; if either reaches the threshold, an enterprise agreement is required. The standard price absent a different negotiated price is EUR 133,700,000 per year per product. These terms cover AmoosedWeb code and protocol implementations. Implementation source belongs in a separately encrypted member edition issued to paying licensees under their agreement. The preview draft license still requires owner review before public SDK publication.