PTWM
ptwm_core

ptwm_core::tensor_record_compact

Tensor and plane records of the compact container (\x89PTWC).

ptwm_core::tensor_record_compact

Tensor and plane records of the compact container (\x89PTWC).

rsmod

ptwm_core::tensor_record_compact

crates/ptwm-core/src/tensor_record_compact.rs:1
mod tensor_record_compact

Tensor and plane records of the compact container (`\x89PTWC`).

A compact record holds the same [`TensorRecord`] as a legacy record, with varints instead of fixed-width fields, without the name (the container index stores it), and without fields the reader can derive. All varints are canonical unsigned LEB128 ([`crate::varint`]); integers are little-endian. ```text TENSOR RECORD tflags u8 TF_* bits; bits 6-7 must be 0 [chain_id] varint registry index, < registry count (no TF_INLINE_CHAIN) dtype_code varint at most u16::MAX input_format u8 orig_size varint [shape] varint ndim (<= 64), then varint per dimension (TF_SHAPE) [dtype_name] varint len, then UTF-8 bytes (TF_DTYPE_NAME) [raw_meta] varint len, then bytes (TF_RAW_META) [payload_hash] u64 (TF_PAYLOAD_HASH) [inline chain] varint et_len, compact Extension Table of the chain's ops in first-use node order, varint chain_len, CW2 chain (TF_INLINE_CHAIN) [deps] varint count >= 1, then per dependency: ref_kind u8, ref_flags u8 (bit 0: hash), varint ref_len, bytes, [hash 32 B] (TF_DEPS) n_planes varint at most 255 plane × n_planes PLANE pflags u8 PF_* bits, state source in bits 2-3; bits 6-7 must be 0 role u8 PlaneRole codec_ref varint Extension Table index of a plane codec [state_version] u8 (PF_STATE_EXTRA) [state_info] varint at most u16::MAX (PF_STATE_EXTRA) [inline_len] varint inline state length (state source Inline) payload_len varint [row_len] varint 1..=u32::MAX (PF_LAYOUT_ROWS) [crc32] u32 (PF_CRC32) [chunks] varint n (<= payload_len), then n × (varint offset, varint decoded_size), each at most u32::MAX (PF_CHUNKS) [external] ref_kind u8, ref_flags u8 (bit 0: hash), varint ref_len, bytes, state_format_version u8, [hash 32 B] (state source External) [inline state] bytes payload bytes ``` **Metadata.** `TF_SHAPE` stands for the CBOR shape map of [`encode_shape_metadata`] with the canonical dtype name of `dtype_code`, or with the stored name under `TF_DTYPE_NAME` (which must then differ from the canonical name). Other metadata is stored as `TF_RAW_META`. The writer uses `TF_SHAPE` exactly when it reproduces the record's metadata bytes, so the reader rejects `TF_RAW_META` bytes that `TF_SHAPE` could store. **Chains.** A registry chain may have a bound Source (see [`crate::chain::wire_compact::write_chain_compact_deferred`]); the record then needs `TF_SHAPE` and a shape and dtype that bind ([`bound_source_params`]). An inline chain binds its Source to the record the same way as the `\x89PTWS` frame does. **Derived fields.** The in-memory `chain_ref` (`0x8000` for an inline chain), `payload_len`, and each plane's `codec_id` (from the Extension Table entry its `codec_ref` names) are not stored. The writer rejects a record whose values disagree with what the reader derives, so each record has one encoding and parsing then writing gives the same bytes. The in-memory `flags` byte is not stored either. Like the legacy writer ([`crate::tensor_record::write_tensor_record`]), the compact writer ignores it and the reader recomputes it from `payload_hash`, `dependencies` and `tensor_metadata`. A record whose `flags` disagree with those fields (for example a legacy record parsed with the dependency bit set and no dependencies) therefore reads back with the derived flags.
rsconst

ptwm_core::tensor_record_compact::MAX_NDIM

crates/ptwm-core/src/tensor_record_compact.rs:116
pub const MAX_NDIM

Largest dimension count a compact record stores. Shape metadata with more dimensions is stored as `TF_RAW_META`.

rsconst

ptwm_core::tensor_record_compact::MIN_RECORD_LEN

crates/ptwm-core/src/tensor_record_compact.rs:121
pub const MIN_RECORD_LEN

Smallest compact record: tflags, dtype_code, input_format, orig_size, n_planes would be 5; the index requires at least 3.

rsfn

ptwm_core::tensor_record_compact::parse_compact_record

crates/ptwm-core/src/tensor_record_compact.rs:497
pub fn parse_compact_record(buf: &[u8], ctx: &RecordContext<'_>) -> Result<TensorRecord, PtwmCoreError>

Parse a compact record that fills `buf` exactly. The returned record has an empty `name`; the container index supplies it.

rsconst

ptwm_core::tensor_record_compact::PF_CHUNKS

crates/ptwm-core/src/tensor_record_compact.rs:106
pub const PF_CHUNKS
rsconst

ptwm_core::tensor_record_compact::PF_CRC32

crates/ptwm-core/src/tensor_record_compact.rs:105
pub const PF_CRC32
rsconst

ptwm_core::tensor_record_compact::PF_LAYOUT_ROWS

crates/ptwm-core/src/tensor_record_compact.rs:110
pub const PF_LAYOUT_ROWS
rsconst

ptwm_core::tensor_record_compact::PF_STATE_EXTRA

crates/ptwm-core/src/tensor_record_compact.rs:111
pub const PF_STATE_EXTRA
rsconst

ptwm_core::tensor_record_compact::PF_STATE_MASK

crates/ptwm-core/src/tensor_record_compact.rs:109
pub const PF_STATE_MASK
rsconst

ptwm_core::tensor_record_compact::PF_STATE_SHIFT

crates/ptwm-core/src/tensor_record_compact.rs:108
pub const PF_STATE_SHIFT

Plane state source (`StateSource` value) in bits 2-3.

rsfn

ptwm_core::tensor_record_compact::plane_codec_ids

crates/ptwm-core/src/tensor_record_compact.rs:141
pub fn plane_codec_ids(table: &ExtensionTable) -> Vec<Option<CodecId>>

The codec id each Extension Table entry gives a plane that references it: the built-in codec for a built-in codec entry, `Identity` for a third-party plane codec (which has no wire id), and `None` for entries that are not plane codecs.

rsfn

ptwm_core::tensor_record_compact::record_binding

crates/ptwm-core/src/tensor_record_compact.rs:188
pub fn record_binding(meta: Option<&[u8]>, dtype_code: u16) -> Option<Vec<u8>>

Bound Source params for a record's shape metadata as the compact container stores it: `Some` only when the metadata is stored as `TF_SHAPE` and the shape and dtype bind.

rstype

ptwm_core::tensor_record_compact::RecordContext

crates/ptwm-core/src/tensor_record_compact.rs:127
pub struct RecordContext

What a compact record refers to outside itself.

rsconst

ptwm_core::tensor_record_compact::TF_DEPS

crates/ptwm-core/src/tensor_record_compact.rs:98
pub const TF_DEPS
rsconst

ptwm_core::tensor_record_compact::TF_DTYPE_NAME

crates/ptwm-core/src/tensor_record_compact.rs:100
pub const TF_DTYPE_NAME
rsconst

ptwm_core::tensor_record_compact::TF_INLINE_CHAIN

crates/ptwm-core/src/tensor_record_compact.rs:97
pub const TF_INLINE_CHAIN
rsconst

ptwm_core::tensor_record_compact::TF_PAYLOAD_HASH

crates/ptwm-core/src/tensor_record_compact.rs:96
pub const TF_PAYLOAD_HASH
rsconst

ptwm_core::tensor_record_compact::TF_RAW_META

crates/ptwm-core/src/tensor_record_compact.rs:101
pub const TF_RAW_META
rsconst

ptwm_core::tensor_record_compact::TF_SHAPE

crates/ptwm-core/src/tensor_record_compact.rs:99
pub const TF_SHAPE
rsfn

ptwm_core::tensor_record_compact::write_compact_record

crates/ptwm-core/src/tensor_record_compact.rs:208
pub fn write_compact_record(rec: &TensorRecord, ctx: &RecordContext<'_>, out: &mut Vec<u8>) -> Result<(), PtwmCoreError>

Append the compact encoding of `rec` to `out`. `rec.name` is not written. On error `out` is left unchanged.