Signet Forge 0.1.0
C++20 Parquet library with AI-native extensions
 DEMO
Loading...
Searching...
No Matches
forge.hpp
Go to the documentation of this file.
1// SPDX-License-Identifier: AGPL-3.0-or-later
2// Copyright 2026 Johnson Ogundeji
3#pragma once
4
28
31#include "signet/error.hpp"
32#include "signet/types.hpp"
33#include "signet/schema.hpp"
34#include "signet/statistics.hpp"
35#include "signet/memory.hpp"
36#include "signet/thrift/compact.hpp"
37#include "signet/thrift/types.hpp"
38#include "signet/column_writer.hpp"
39#include "signet/column_reader.hpp"
41
44#include "signet/encoding/rle.hpp"
45#include "signet/encoding/dictionary.hpp"
46#include "signet/encoding/delta.hpp"
47#include "signet/encoding/byte_stream_split.hpp"
49
52#include "signet/z_order.hpp"
54
57#include "signet/compression/codec.hpp"
58#include "signet/compression/snappy.hpp"
59#include "signet/compression/zstd.hpp"
60#include "signet/compression/lz4.hpp"
61#include "signet/compression/gzip.hpp"
63
66#include "signet/crypto/sha256.hpp"
67#include "signet/crypto/sha512.hpp"
68#include "signet/crypto/aes_core.hpp"
69#include "signet/crypto/aes_gcm.hpp"
70#include "signet/crypto/aes_ctr.hpp"
71#include "signet/crypto/key_metadata.hpp"
72#include "signet/crypto/cipher_interface.hpp"
73#if defined(SIGNET_ENABLE_COMMERCIAL) && SIGNET_ENABLE_COMMERCIAL
74#include "signet/crypto/pme.hpp"
75#include "signet/crypto/post_quantum.hpp"
76#endif
78
81#include "signet/bloom/xxhash.hpp"
82#include "signet/bloom/split_block.hpp"
84
87#include "signet/column_index.hpp"
88#if !defined(_WIN32) && !defined(__EMSCRIPTEN__)
89#include "signet/mmap_reader.hpp"
90#endif
92
95#include "signet/ai/vector_type.hpp"
96#include "signet/ai/quantized_vector.hpp"
97#include "signet/ai/tensor_bridge.hpp"
99
102#include "signet/ai/wal.hpp"
103#ifndef __EMSCRIPTEN__
104#include "signet/ai/streaming_sink.hpp"
105#endif
107
110#include "signet/ai/feature_writer.hpp"
111#include "signet/ai/feature_reader.hpp"
113
116#include "signet/ai/mpmc_ring.hpp"
117#include "signet/ai/column_batch.hpp"
118#include "signet/ai/event_bus.hpp"
120
123#if defined(SIGNET_ENABLE_COMMERCIAL) && SIGNET_ENABLE_COMMERCIAL
124# include "signet/ai/audit_chain.hpp"
125# include "signet/ai/row_lineage.hpp"
126# include "signet/ai/decision_log.hpp"
127# include "signet/ai/inference_log.hpp"
128# include "signet/ai/human_oversight.hpp"
129# include "signet/ai/log_retention.hpp"
130# include "signet/ai/compliance/compliance_types.hpp"
131# include "signet/ai/compliance/mifid2_reporter.hpp"
132# include "signet/ai/compliance/eu_ai_act_reporter.hpp"
133#endif
135
138#include "signet/interop/onnx_bridge.hpp"
139#include "signet/interop/arrow_bridge.hpp"
140#include "signet/interop/numpy_bridge.hpp"
142
145#include "signet/writer.hpp"
146#include "signet/reader.hpp"
aes_core.hpp
AES-256 block cipher implementation (FIPS-197).
aes_ctr.hpp
AES-256-CTR stream cipher implementation (NIST SP 800-38A).
aes_gcm.hpp
AES-256-GCM authenticated encryption (NIST SP 800-38D).
arrow_bridge.hpp
Arrow C Data Interface interop for SignetStack Signet Forge.
audit_chain.hpp
byte_stream_split.hpp
BYTE_STREAM_SPLIT encoding and decoding (Parquet encoding type 9).
cipher_interface.hpp
Abstract cipher interface, GCM/CTR adapters, CipherFactory, and platform CSPRNG.
codec.hpp
Compression codec interface and registry for Signet Forge.
column_batch.hpp
Column-major batch of feature rows for zero-copy tensor wrapping and WAL serialization in ML inferenc...
column_index.hpp
ColumnIndex, OffsetIndex, and ColumnIndexBuilder for predicate pushdown.
column_reader.hpp
PLAIN-encoded Parquet column decoder.
column_writer.hpp
PLAIN encoding writer for all Parquet physical types.
compact.hpp
Thrift Compact Protocol encoder and decoder for Parquet metadata serialization.
compliance_types.hpp
decision_log.hpp
delta.hpp
DELTA_BINARY_PACKED encoding and decoding (Parquet encoding type 5).
dictionary.hpp
Dictionary encoding and decoding for Parquet (PLAIN_DICTIONARY / RLE_DICTIONARY).
eu_ai_act_reporter.hpp
event_bus.hpp
Multi-tier event bus for routing SharedColumnBatch events through three tiers.
feature_reader.hpp
Versioned ML feature store reader with point-in-time queries.
feature_writer.hpp
Versioned ML feature store writer for appending typed feature vectors to Parquet.
gzip.hpp
GZIP compression codec for Signet Forge (wraps zlib).
human_oversight.hpp
inference_log.hpp
key_metadata.hpp
Key material, encryption configuration, and TLV serialization for Parquet Modular Encryption (PME).
log_retention.hpp
lz4.hpp
LZ4 raw-block compression codec for Signet Forge (wraps liblz4).
memory.hpp
Arena (bump-pointer) allocator for batch Parquet reads.
mifid2_reporter.hpp
mmap_reader.hpp
Memory-mapped I/O reader for Parquet files.
mpmc_ring.hpp
Lock-free bounded MPMC ring buffer based on Dmitry Vyukov's algorithm.
numpy_bridge.hpp
DLPack / NumPy interop for SignetStack Signet Forge tensor bridge.
onnx_bridge.hpp
ONNX Runtime interop for SignetStack Signet Forge tensor bridge.
pme.hpp
Parquet Modular Encryption (PME) orchestrator – encrypts and decrypts Parquet file components (footer...
post_quantum.hpp
Post-quantum cryptography: Kyber-768 KEM, Dilithium-3 signatures, X25519 ECDH, and HybridKem (Kyber+X...
quantized_vector.hpp
INT8/INT4 quantized vector storage for AI embeddings with SIMD acceleration.
rle.hpp
RLE/Bit-Packing Hybrid encoding and decoding (Parquet spec).
row_lineage.hpp
Per-row lineage tracking (Iceberg V3-style) with monotonic row IDs, mutation versioning,...
schema.hpp
Schema definition types: Column<T>, SchemaBuilder, and Schema.
sha256.hpp
SHA-256 hash function (NIST FIPS 180-4).
sha512.hpp
SHA-512 hash function (NIST FIPS 180-4).
snappy.hpp
Bundled, zero-dependency, header-only Snappy compression codec.
split_block.hpp
Split Block Bloom Filter as specified by the Apache Parquet format.
statistics.hpp
Per-column-chunk statistics tracker and little-endian byte helpers.
streaming_sink.hpp
Lock-free SPSC/MPSC ring buffers, StreamingSink for background Parquet compaction,...
tensor_bridge.hpp
Zero-copy tensor bridge: maps Parquet column data directly into ML-framework-compatible tensor views ...
types.hpp
Parquet Thrift struct types – C++ structs matching parquet.thrift, with Compact Protocol serialize/de...
types.hpp
Parquet format enumerations, type traits, and statistics structs.
vector_type.hpp
AI-native ML vector column type: VectorWriter/VectorReader for dense float embedding storage as FIXED...
wal.hpp
Write-Ahead Log (WAL) with sub-millisecond append, CRC-32 integrity, crash recovery,...
xxhash.hpp
Header-only xxHash64 implementation for Signet Forge.
z_order.hpp
Z-order curve (Morton code) encoding/decoding for multi-dimensional indexing.
zstd.hpp
ZSTD compression codec for Signet Forge (wraps libzstd).