Valerois AI Logo
ValeroisAI
Bifrost CSL
Bifrost CSL•Sub-quadratic Linear Architecture

Sub-quadratic
Intelligence.

Eliminating quadratic self-attention with continuous causal state convolutions. Zero KV-cache overhead, strictly constant O(1) memory, and 63× data efficiency.

Explore Valkir ModelsEmpirical Leaderboard
AMD RX 9070 XT(16 GB)
4.74B TokensTrained
O(1)Static Buffer
Valerois AI Depth Extrusion
Valerois AI Depth Extrusion
Valerois AI Depth Extrusion
Valerois AI Depth Extrusion
Valerois AI 3D Core Logo
Cumulative Tokens Trained
4.74 Billion
3 Phases • 36h cumulative
ROCm Inductor Speedup
37,243 tok/s
+26% vs Native PyTorch (29.5k)
Peak Training VRAM
4.87 GB
40% reduction via Inductor fusion
BF16 Checkpoint Size
223 MB
Ultra-compact 116.8M parameters
HumanEval AST Validity
54.0%
Crushes Pythia-160M (~20%)
Interactive Zero-Shot Inference Engine

Test Valkir 16L Live

Experience sub-quadratic generation speeds and true O(1) zero-KV memory scaling in real-time.

Valkir 16L|ROCm 7.2 Active
224.4tok/s
Input Prompt206 chars
SAMPLINGGreedy / ArgMax
STATE MEMORYO(1) Linear
Generated Stream
TTFT: 3.4 msTokens: 184
import numpy as np

class CausalRingBuffer:
    def __init__(self, kernel_size: int, hidden_dim: int):
        # Strictly constant O(1) allocation - no KV-cache explosion
        self.kernel_size = kernel_size
        self.hidden_dim = hidden_dim
        self.buffer = np.zeros((kernel_size, hidden_dim), dtype=np.float32)
        self.pointer = 0

    def step(self, x_token: np.ndarray) -> np.ndarray:
        """Insert new token state in O(1) time and retrieve context window."""
        self.buffer[self.pointer] = x_token
        self.pointer = (self.pointer + 1) % self.kernel_size
        # Unroll circular state without quadratic matrix allocation
        idx = (np.arange(self.kernel_size) + self.pointer) % self.kernel_size
        return self.buffer[idx]

# Telemetry: Valid AST verified in 3.4ms | Zero VRAM drift
THROUGHPUT224.4 tok/s
KV OVERHEAD0.00 MB
RECEPTIVE FIELD1,233 Tok
VERIFICATIONAST Pass ✓
Architectural Pillars

How Bifrost CSL Surpasses Transformers

A state-of-the-art linear operator combining hierarchical dilated convolutions with dynamic input-dependent gating.

Dual-Scale Temporal Convolutions

Short and dilated kernels simultaneously capture local code syntax and long-range narrative context. Delivers a 1,233-token receptive field with zero quadratic attention bottlenecks.

Hierarchical Linear Scaling • Zero KV-Cache

Dynamic Input-Dependent Gating

Replaces memory-heavy attention matrices with ultra-fast elementwise gating. Automatically routes salient tokens and closes for noise, achieving true O(1) constant inference memory.

Strict O(1) Constant Space • Sub-2ms First Token

SwiGLU FFN & LayerScale Stability

Expanded non-linear feedforward networks paired with adaptive LayerScale coefficients ensure rock-solid numerical stability and prevent gradient saturation across deep stacks.

2,048 Intermediate Dim • 100% Numerical Stability
Interactive 16-Layer Deep-Dive

Bifrost CSL Layer Architecture

Inspect how signals propagate through the 16 continuous causal layers without attention matrices or quadratic memory bottlenecks.

Active Inspection: Layers 5 - 12

MultiScale Dilated Long-Range Causality

The engine room of Bifrost CSL. Dilated convolutions leap across context boundaries with quadratic-free linear compute, expanding the temporal receptive field exponentially.

SHORT CONVK=16 (Dilation 1)
DILATED CONVK=32 (Dilation 2)
RECEPTIVE FIELD1,233 tokens
MEMORY BOTTLENECKZero ($O(1)$)
Forward Signal PropagationLinear Step
1RMSNorm + In_Proj Linear Map
Passed ✓
2Branch A: K=16 Local Stream
Passed ✓
3Branch B: K=32 Dilation=2 Wide Stream
Passed ✓
4Tensor Fusion: 0.5 * (x_short + x_dilated)
Passed ✓
5LayerScale Stabilization (γ = 0.1)
Passed ✓
Numerical Precision: Bfloat16Gradient Drift: 0.00%
Empirical Scaling Simulator

Sub-quadratic vs Quadratic Scaling

Drag the context slider to observe how standard attention architectures suffer exponential memory explosion while Bifrost CSL remains strictly flat.

Select Inference Context Length65,536 Tokens (64K Context)
1K Context4K Context16K Context64K Context128K Context
Standard Self-Attention (Transformer)Quadratic $O(N^2)$
3.31 GB
Required VRAM at 65,536 tokens
KV Cache: 3072.0 MB16GB Hardware Ceiling

As context expands, the $O(N^2)$ attention matrix floods memory bandwidth, causing latency throttling and eventual out-of-memory failure on edge GPUs.

Token Throughput:Declining (~48 tok/s)
Valerois Bifrost CSL (Valkir 16L)Strictly O(1) Constant
0.22 GB
Zero memory drift (Flat across any length)
State Buffer: 0.65 MB1.3% of 16GB Used

Continuous causal convolutions maintain a constant state register. Memory overhead never increases, allowing infinite streaming without KV cache.

Token Throughput:Flat 219+ tok/s (Zero Degradation)
Verified Checkpoint • Valkir 16L

Competing with 300B Tokens on 4.74B Data.

Valkir 16L was trained on a single AMD Radeon RX 9070 XT over ~36 cumulative hours across 3 phases (Base, Continual, and Marathon). At 116.8M parameters, it matches or outperforms Pythia-160M and OPT-125M—models trained with 63× more data.

HumanEval AST
54.0%
vs 20% Pythia
WinoGrande
51.62%
1,267 questions
ARC-Easy
33.08%
2,376 questions
BF16 Checkpoint
223 MB
Ultra-portable
View Full LeaderboardDownload Weights & Cards
valkir_16l_config.json
BF16
"architecture": "Bifrost-CSL",
"parameters": 116_800_000,
"hidden_dim": 640,
"n_layers": 16,
"kernel_short": 16 (dilation=1),
"kernel_long": 32 (dilation=2),
"receptive_field": 1233 tokens,
"throughput_inductor": "37,243 tok/s"
Hardware: AMD RX 9070 XTStatus: Verified
The Valerois Thesis

The Path Beyond Quadratic Transformers.

We founded Valerois AI on a single empirical insight: artificial intelligence cannot achieve universal adoption while shackled to quadratic memory scaling. By moving from attention matrices to continuous causal linear convolutions, we decouple reasoning power from exponential hardware costs.

Edge Sovereignty

Run frontier-tier reasoning locally on edge hardware with 223 MB weights and zero cloud dependency.

Linear Infinite Context

Stream 100k+ tokens with constant memory. No KV cache explosion, no memory exhaustion.

Data Hyper-Efficiency

Matching 300B token benchmarks with only 4.74B tokens through tight multi-scale inductive priors.