|
PineForge unreleased
Deterministic PineScript v6 backtest runtime — C ABI reference
|
1.0.0 was released on 2026-09-30: engine v1.0.0 with pineforge-codegen 1.0.0. 1.0.1 followed on 2026-10-02: engine v1.0.1, which changes only documentation, with pineforge-codegen 1.0.1. 1.1.0 followed on 2026-10-04: engine v1.1.0, which adds six functions to the C ABI (the checked settings calls of <pineforge/pineforge.h>, which a strategy generated by codegen 1.1.0 exports), with pineforge-codegen 1.1.0. 1.2.0 followed on 2026-10-05: engine v1.2.0, which adds the compiled execution-capabilities extension to the C ABI (PF_CAPABILITIES_API_VERSION and the two calls of <pineforge/pineforge.h> that a strategy generated by codegen 1.2.0 defines), with pineforge-codegen 1.2.0. 1.3.0 followed on 2026-10-06: engine v1.3.0, whose <pineforge/pineforge.h> is 1.2.0's and whose native runner admits the confirmed-bar shapes that a strategy generated by codegen 1.3.0 proves through its confirmed-bar receipt, with pineforge-codegen 1.3.0. From 1.0.0 the engine's version number is a semantic version over the surfaces this page lists, and over nothing else: a PATCH release fixes behaviour, a MINOR release adds to a surface, and only a MAJOR release removes or changes one. Each rule below names the checker or the CTest row that holds it on this tree; where no checker holds part of a rule, the page says so. The codegen states its half in its own docs/PUBLIC_CONTRACT.md (pineforge-codegen); the two releases pair as the last section below says.
1.4.0 followed on 2026-10-08: engine v1.4.0 with pineforge-codegen 1.4.0. It adds two runtime failure getters and the feature probe PINEFORGE_HAS_RUN_FAILURE_CODES_V1, without changing C layouts or the then-current PF_ABI_VERSION 4, which was unchanged in 1.4.0. It also changes report provenance types and adds resolution/failure fields. The repository's CHANGELOG.md, under 1.4.0 — 2026-10-08, inventories the state-hash additions and report migration.
1.5.0 followed on 2026-10-12: engine v1.5.0 with pineforge-codegen 1.5.0. It adds recorded outputs — doubles in named per-bar slots, run-level doubles and events, through the nine pf_outputs runtime exports of <pineforge/pineforge.h>, recorded only when a caller turns them on — and gives each run's report to the call that began it. The repository's CHANGELOG.md, under 1.5.0 — 2026-10-12, inventories both and the test inputs' move to a private repository.
1.6.0 followed on 2026-10-17: engine v1.6.0 with pineforge-codegen 1.6.0. It extends typed failures across source value admission (strategy_value_invalid), derived execution and settlement arithmetic (engine_command_value_invalid), non-finite canonical trade or summary doubles (report_result_invalid) and post-boundary serialization faults (report_post_execution_failed), without changing C layouts or the previous PF_ABI_VERSION 4. The repository's CHANGELOG.md, under 1.6.0 — 2026-10-17, inventories these changes and the documentation, test, CI and benchmark updates since 1.5.0.
1.6.1 followed on 2026-10-18: engine v1.6.1 with pineforge-codegen 1.6.1. It declares selected-window contract revision 1.3-parity (["1.3-parity"] under capabilities.selected_window_contract_revisions when the runtime supports selected windows and [] otherwise), which emits the four typed codes (strategy_value_invalid, engine_command_value_invalid, report_result_invalid and report_post_execution_failed), with parity with the 1.4.0 reference on tested cases, and no completeness claim (1.6.0 declared 1.3 without a completeness proof). It also records per-stage startedAt and seconds in the preflight summary and adds noindex to raw Doxygen source listings and omits them from sitemap.xml, without changing C layouts or the previous PF_ABI_VERSION 4. The repository's CHANGELOG.md, under 1.6.1 — 2026-10-18, inventories these changes and the CI, documentation and test updates since 1.6.0. Use the same-version 1.6.1 pair, with regeneration and relinking as below.
From 1.6.2, the image declares selected-window contract revision 1.3 again (["1.3"] under capabilities.selected_window_contract_revisions when the runtime supports selected windows and [] otherwise), now with the completeness proof: the section-4 producers and manifest are present and value ownership is complete, and it emits the four typed codes (strategy_value_invalid, engine_command_value_invalid, report_result_invalid and report_post_execution_failed). A build declaring 1.3 pairs only with a complete receipt; 1.3-parity remains the parity build's literal, and a tuple holding both is invalid (the published 1.5.0 image declared 1.3-parity, earlier images carried no revision member; the published 1.6.0 image declared 1.3 without a completeness proof; and engine 1.6.1, released without a hub image, declares 1.3-parity). The repository's CHANGELOG.md, under 1.7.0, inventories this change.
The public surface and stability rules for these revisions are in Selected windows and walk-forward use.
1.8.0 requires rebuilding native C++ objects for native_order_v8 and relinking strategy libraries against the matching headers and archive. PF_ABI_VERSION is 5: the loader refuses a library stamped 4 at its ABI mismatch check before calling strategy entry points. This is an explicit exception to the earlier promise of one ABI number throughout 1.x; the caller-allocated C report and trade layouts remain unchanged.
X.Y.Z, or X.Y.Z-rc.N (N from 1) for a release candidate; its tag is vX.Y.Z or vX.Y.Z-rc.N. release.yml computes it with scripts/release_version.py, which refuses any other spelling, a version that does not sort above VERSION in semver order (X.Y.Z-rc.N below X.Y.Z) and a tag that exists (scripts/test_release_version.py, a ci_preflight stage).prerelease: true: .github/workflows/release.yml reads both from the script's prerelease output, which scripts/test_release_version.py holds. The workflow-lint stage of ci_preflight lints the workflow's syntax, expressions and shell, not its behaviour; a dry_run dispatch rehearses it without publishing.project(VERSION), the package's PineForge_VERSION, PINEFORGE_VERSION_MAJOR / _MINOR / _PATCH, PINEFORGE_VERSION_STRING and pf_version_get() carry the numeric X.Y.Z. The package's PineForge_VERSION_FULL, PINEFORGE_VERSION_FULL, pf_version_string(), the tag and the release tarball's name carry the full version, -rc.N included (ABI stability, "Version macros"). test_cmake_version_source holds the resolution from VERSION and from a tag; the installed-package smoke test scripts/ci_verify.py runs (cmake/smoke_consumer) holds that the package, the installed header and the library name one full version, VERSION exactly; and release.yml verifies every staged tarball with scripts/release_version.py check-install before it packs it.The C ABI is <pineforge/pineforge.h> and the native host API <pineforge/native_c_api.h> it includes: the PF_API functions, the pf_* structs and enumerations, and the PF_* constants.
PF_ABI_VERSION is 5 from 1.8.0. Earlier 1.x releases used ABI 4. It versions module compatibility as well as the layouts a caller allocates or strides over (pf_report_t, pf_trade_t, the metrics structs), and a consumer checks pf_abi_version() == PF_ABI_VERSION before it runs a strategy. test_c_abi checks that equality, test_live_run_status pins the value 5, and test_run_strategy_range_end pins the Python harness's EXPECTED_PF_ABI to it. The public Unix test_strategy_module_abi_epoch row requires the typed refusal of a stamp-4 module before any strategy entry point runs, and acceptance of a module stamped with the current header value.PF_API function is removed or renamed in 1.x; new ones are added. scripts/check_c_abi_runtime.py (a source guard of every ci_verify profile and of ci_preflight) pins the exact name sets EXPECTED_RUNTIME and EXPECTED_NATIVE_C_API and the declaration counts, so an unrecorded addition, removal or rename fails. It reads names only: a signature change that keeps its name is caught only where a test calls the function, the frozen v1 closure below among them.struct_size, version) grows only by an appended tail, and the runtime keeps accepting every length it published (each *_SIZE constant). The static_asserts in src/c_abi.cpp pin the size and field offsets of the structs the header shares with the runtime — pf_bar_t, pf_trade_tick_t, pf_trade_t, pf_report_t, pf_security_diag_t, pf_trace_entry_t — to their internal C++ twins, and pf_equity_stats_t / pf_metrics_t to literal offsets and sizes; src/native_c_host.cpp pins the published lengths of the native structs that grew by tails; test_native_c_api_frozen_header compiles the frozen PF_NATIVE_API_VERSION 1 header closure (the private fixture set native_c_api/v1) and runs it against the current runtime; test_c_abi and test_native_c_api_c99 check the equity offsets from C. These pins are against the internal twins and the frozen v1 closure; no checker compares pf_report_t or pf_trade_t with a 1.0.0 baseline.scripts/check_native_c_api_surface.py holds every C enumeration of native_c_api.h and pineforge.h equal, value for value, to its kernel twin (one named exclusion) or declares it C-only with a reason, and src/c_abi.cpp's static_asserts pin the pf_magnifier_distribution_t values. No checker compares a value with the previous release.The selected-window path is PUBLIC for walk-forward use through the docker/run_json.py command line or the C headers <pineforge/selected_window.h> and <pineforge/selected_window_plan.h>. A walk-forward driver such as the HPO native layer may submit successive non-overlapping windows over one chart, for example [T0, T1) followed by [T1, T2). Each window is one independent request. The CLI creates a fresh strategy state for its request in _make_state (docker/run_json.py:5012) inside _selected_run (docker/run_json.py:4917).
These are the selected-window options declared by the CLI:
| Option | Contract | Declaration |
|---|---|---|
--report-policy selected-window/v1 | Opt into the selected-window report policy. | add_argument (docker/run_json.py:5457) |
--window-start-ms | Start T in calendar-aligned Unix milliseconds. | add_argument (docker/run_json.py:5460) |
--window-end-ms | End E in calendar-aligned Unix milliseconds; T < E. | add_argument (docker/run_json.py:5462) |
--preroll-bars | Requested warm-up N in script bars, from 0 through 5000. | add_argument (docker/run_json.py:5464) |
--fed-start-ms | Frozen start F of the supplied primary feed; F <= T. | add_argument (docker/run_json.py:5466) |
--capabilities-json | With --so LIB, report runtime support without reading OHLCV or running a strategy. | add_argument (docker/run_json.py:5468) |
--validate-window-only | Validate and plan the request; return report_window with applied: false, without executing a strategy. | add_argument (docker/run_json.py:5471) |
--run-phase-fd | Inherited AF_UNIX SOCK_STREAM descriptor N >= 3 for phase transport v1. | add_argument (docker/run_json.py:5474) |
--request-feed-inventory | Inventory bound to the strategy library, used for selected-window admission. | add_argument (docker/run_json.py:5477) |
--report-shape | This build offers full/v1 for selected-window requests. | add_argument (docker/run_json.py:5480) |
--curve-point-budget | Refused by this build, which does not offer a compact shape. | add_argument (docker/run_json.py:5482) |
--results-digest | Refused by this build, which does not offer a results digest. | add_argument (docker/run_json.py:5485) |
An execution request supplies the policy, all four window arguments and explicit --input-tf and --script-tf values, as checked by validate_request (docker/selected_window_report.py:200), together with the strategy library, chart and request-feed inventory. The selected interval is [T, E); preroll precedes T, as the configuration header documents.
The runner defaults to no limit on symbol count, total extra-feed count or request-feed inventory bytes. --max-symbol-feeds N applies N to both counts; --max-inventory-bytes N bounds the inventory reader. Unset or 0 means unlimited; a positive integer sets the limit, and an empty, negative or non-integer value is a usage error (exit 2). --tv-compat selects defaults of 256 symbols/feeds and 8 MiB inventory from TV_COMPAT_LIMITS in docker/run_json.py. Explicit parameters, including 0, always win. The inventory binder accepts --max-inventory-bytes and --tv-compat with the same semantics. The entrypoint forwards PINEFORGE_MAX_SYMBOL_FEEDS, PINEFORGE_MAX_INVENTORY_BYTES and PINEFORGE_TV_COMPAT to the relevant tools; the inventory setting reaches both binder and runner.
Before relying on the CLI path, probe the library with docker/run_json.py --so LIB --capabilities-json. The probe is _capabilities_main (docker/run_json.py:4900); its support decision is _runtime_supports_selected_window (docker/run_json.py:4871), which checks the selected-window ABI, execution observer, phase transport and planner. The capability record is built by capabilities_record (docker/selected_window_report.py:756). Require capabilities.selected_window_wire_version == 1, capabilities.report_policy == "selected-window/v1", full/v1 in capabilities.report_shapes, and a capabilities.selected_window_contract_revisions literal that the driver understands. This tree declares ["1.3"] on a supported runtime and [] otherwise; see the capability table for revision history. The revision is declared in the probe, not in each report. The probe also sets selected_window_bars_bin_version (docker/run_json.py:4913) to 0: selected-window requests require a CSV chart. General binary-feed support does not imply selected-window binary-chart support.
The public declarations include the descriptors and return contracts in the two headers:
| Header | Function and declaration | Purpose |
|---|---|---|
include/pineforge/selected_window.h | pf_selected_window_version (include/pineforge/selected_window.h:61) | Selected-window ABI version, currently 1. |
include/pineforge/selected_window.h | strategy_set_selected_window_v1 (include/pineforge/selected_window.h:84) | Copy pending bounds, or clear selection with a null descriptor, for the next admitted generation. |
include/pineforge/selected_window.h | strategy_selected_window_counts_v1 (include/pineforge/selected_window.h:99) | Read counts belonging to the latest attempt's successful selected Results generation. |
include/pineforge/selected_window_plan.h | pf_selected_plan_version (include/pineforge/selected_window_plan.h:114) | Planner bridge version, currently 1. |
include/pineforge/selected_window_plan.h | pf_plan_selected_primary_v1 (include/pineforge/selected_window_plan.h:131) | Plan one request over primary rows, without loading or running a strategy. |
For direct C use, check the relevant version getter returns 1 and initialize each descriptor's struct_size and version exactly as its header requires. The planner bridge is a separate shared library from the caller's pinned build; it is not the strategy library. The C ABI versions do not declare a CLI contract revision; consumers of the CLI must use its capability record.
The configuration setter and counts getter return 0 on success and negative values for the refusals documented in the header. A refused setter leaves the configuration unchanged; a failed counts read leaves valid output storage unchanged. The planner bridge instead returns 0 when it writes a whole result, including a planner refusal: inspect status, where 0 is a successful plan. A negative bridge return is a bridge failure and leaves valid output storage unchanged.
The policy and report shape follow the report schema, including its revision 3.1 clarification. The report carries the window bounds and coverage, feed identity, and requested, available, used and trimmed preroll with its timestamps and shortfall, assembled by build_report_window (docker/selected_window_report.py:460). These warm-up bars precede the window and are counted separately. The lookback declaration is unknown in this build, so the requested preroll is not a proof of sufficient warm-up. The same report_window appears at the top level, in applied_runtime and in fingerprint.provenance.runtime, held by check_report_placements (docker/selected_window_report.py:707). The selected curve has one anchor at T with initial capital and zero open profit, then one point per window script bar, held by check_selected_curve (docker/selected_window_report.py:609).
An executed success is a complete report with report_window.applied: true and harness exit status 0. A validation-only plan has applied: false and does not establish execution success. Request or execution failures have a nonzero harness status and the typed code / args failure line described in Run-failure codes. If stdout cannot take a whole report, the harness returns 1 with the reason on stderr and no second JSON document, as enforced by _write_report (docker/run_json.py:4823).
Functions and flags are append-only. No published function or flag above is removed or renamed in 1.x; new ones are added. The C layout rules above apply to these descriptors. A contract revision changes only by a new revision literal in capabilities.selected_window_contract_revisions, rather than by changing the meaning of an existing literal. The version rule applies here: a PATCH release fixes behaviour, a MINOR release adds to a surface, and only a MAJOR release removes or changes one. scripts/check_c_abi_runtime.py pins the runtime export names; review holds the flag and revision stability promise across releases.
The pure helper modules docker/selected_window_*.py and siblings such as docker/run_phase_transport.py, docker/run_execution_observer.py and docker/request_feed_inventory.py are implementation details, even where cited here as evidence. Their Python imports and call signatures are not a public API. Code under src/ not reachable through the published headers or CLI is outside this contract. A consumer must go through the CLI or the headers.
The run-failure vocabulary is docker/run_failure_codes.json: the codes strategy_get_last_error_code answers, their classes and retryable flags, and the arguments strategy_get_last_error_args answers (Run-failure codes).
| Code | Optional argument | Closed values |
|---|---|---|
engine_invariant | reason (vocab) | order_key_hash_collision |
The reason identifies refusal of two live order keys whose 64-bit hashes collide. Other invariant failures keep {}, and the English is unchanged.
deprecated with the codes that replace it. A code keeps its class, its retryable flag, its since and each argument's kind; a closed list of values keeps every value, and an optional argument stays optional. A new code, value, optional argument or English template is an addition. scripts/check_run_failure_codes.py (the source-guard-run-failure-codes stage of ci_preflight and of every ci_verify profile) holds the catalog against the catalog of the newest release tag reachable from HEAD, and the generated RunFailureCode enum and registry against the catalog.run_failure_codes-<tag>.json and run_failure_codes_diff-<tag>.json are release assets. release.yml stamps the diff with scripts/gen_run_failure_catalog_diff.py --stamp, which refuses a removed code, a diff that does not start where the release notes do (or at a release candidate after it), a release that adds or changes codes but is not a minor or major release, and a code whose since is not the release; the source-guard-run-failure-diff stage holds the checked-in diff to the catalog byte for byte.The native C++ API is the kernel's host surface: <pineforge/native_host.hpp>, <pineforge/native_run_spec.hpp>, <pineforge/native_fx_curve.hpp>, <pineforge/native_order.hpp>, <pineforge/native_calendar.hpp>, <pineforge/native_calc_point.hpp> (new in 1.5.0), <pineforge/market_driver.hpp>, <pineforge/order_action.hpp>, <pineforge/execution.hpp>, the header-only <pineforge/native_toolkit.hpp> and <pineforge/native_module.hpp> (Native engine).
examples/native, 16 C++ and 2 C hosts) build and run as the example_* CTest rows of the release and kernel profiles; scripts/check_native_include_independence.py (the native-include-independence stage of the release, native and kernel profiles) compiles every native root header and every example against the installed headers with the Pine source layer removed; scripts/check_native_c_api_surface.py fails when a public NativeStrategyHost member appears or disappears without its row in native_c_api.h's COVERAGE block; scripts/check_native_cpp_versions.py pins every native type to its versioned inline namespace and the member order of the structs it names. Each of them holds the current tree against itself: no checker yet compiles a host written against 1.0.0's headers against a later release, so across releases review holds the promise, until a frozen 1.0.0 native header closure joins these rows the way the private fixture set native_c_api/v1 does for the C API.engine.hpp states the rule where the script ABI epoch opens.engine_script_run_v19 for every 1.x release. It is the inline namespace of BacktestEngine, NativeStrategyHost and the execution consumer, and PINEFORGE_HAS_NATIVE_STRATEGY_HOST_V19 announces it. scripts/check_native_cpp_versions.py and scripts/check_aggregate_cpp_versions.py pin the one live epoch; the CTest rows test_script_cpp_abi, test_settlement_cpp_abi and test_aggregate_cpp_versions_runtime link callers built against the frozen v18 headers (host-fc7aad6) and against the live ones, and require the mismatched pairs to fail on the epoch-qualified BacktestEngine::broker_state_hash symbol (scripts/ci_verify.py requires their prepared receipts, so none of them skips there).strategy_broker_state_hash, the per-bar pf_report_t::broker_state_hash rows), the stream fingerprint (strategy_stream_state_hash) and the native continuation hash, is the epoch's. A minor release may change a recipe only by adding a hashed field, with the domain tags unchanged and only when the added field changes no trade or report; its CHANGELOG discloses the addition under State hashes, because state hashes from earlier releases of the epoch will not match. A recipe change that removes, renames or re-encodes a hashed field is a new epoch and a major release. Behaviour changes, fixes that change values, trades or reports while the recipe stays the same, are not recipe changes: they follow the normal release and parity rules. The first uses of the addition rule are #315 and #316, in 1.1.0. Values from builds before 1.0.0 are not comparable. scripts/check_aggregate_cpp_versions.py and scripts/check_broker_state_hash_coverage.py pin the domain tags pineforge-broker-state/v19 and pineforge-source-adapter/v4 and the stream fingerprint's version 19; witness rows pin values — test_native_host_hash_extension, test_native_lean_path, test_native_match_hash_witness, test_native_continuation_view, test_adapter_quiet_bar, test_publication_witness among them. Two limits: the native continuation hash folds the resolved timezone's resource digest, so a tzdata release that rewrites a zone moves the value of a run in that zone (test_native_report_truth states why it pins no raw continuation value), and no row pins a strategy_stream_state_hash value directly.The C surface is a documented subset of the native C++ API in 1.0, not its twin. What a C host cannot reach is the table The 1.0 C boundary in Native engine (section Driving the kernel from C): every C++ capability the 1.0 C surface does not expose, with its reason, the C route where there is one, and the checker row that pins it – a [--] row of native_c_api.h's COVERAGE block, a named ENUM_TWINS exclusion, or a C_V1_EXCLUSIONS row of scripts/check_native_c_api_surface.py. The checker fails when a public NativeStrategyHost member has no COVERAGE row, when a C enumeration is neither twinned to its kernel enumeration nor declared C-only, when a C_V1_EXCLUSIONS row's C++ declaration goes or its C spelling appears, and when that table cites a C_V1_EXCLUSIONS row once too few or too many. Four of its rows were planned for 1.1.0; none of 1.1.0, 1.2.0 and 1.3.0 includes them: a non-mutating execution preview, the origin and label of an applied event, a closed-trade entry-comment accessor and a replace-options word. Nothing in 1.0 promises C and C++ parity beyond the fields and calls the C surface declares.
X.Y.Z, prerelease included: engine v1.0.0-rc.1 with codegen 1.0.0-rc.1 (PyPI spells it 1.0.0rc1), engine v1.0.0 with codegen 1.0.0, engine v1.0.1 with codegen 1.0.1, engine v1.1.0 with codegen 1.1.0, engine v1.2.0 with codegen 1.2.0, engine v1.3.0 with codegen 1.3.0. Any other pair is unsupported: a release candidate with its final release, two minors, or two builds that agree only on PF_ABI_VERSION. Generated strategy code compiles against the engine's internal C++ headers, which the C ABI does not cover and the script ABI checks above version (ABI stability, "What's *not* guaranteed"). Regenerate the C++ and relink the strategy library on every pair change.release.yml dispatches engine-release to pineforge-release with client_payload {version: "vX.Y.Z[-rc.N]", prerelease: true|false, run_id}; codegen dispatches its own version. The hub, on the 1.0 line, builds and publishes an image only for an engine and a codegen of the same version, and publishes a prerelease pair under its exact version, never a stable or latest tag; that check lives in the pineforge-release repository, not on this tree.Not covered by the version number:
engine.hpp, ta.hpp, the Pine source layer under include/pineforge/source/ and include/pineforge/compat/), which the pairing rule covers instead;PF_API, and the text of log lines;scripts/, tutorial/ and benchmarks/.The JSON report keys of docker/run_json.py are the release image's schema (Report schema); scripts/test_report_schema_keys.py pins the metrics.equity keys, which kept sharpe_tv / sortino_tv when the C fields' pre-1.0 spellings were removed. What 1.0 changes for a 0.x user is in the repository's CHANGELOG.md.