Absolute-Address Partial Residency

zenodo.org · 2026-09-30 · by Long Nguyen ·generic
Caches, memoization, and lookup tables reuse previously computed information when that informationremains valid and the corresponding object can be identified again. This principle itself is not new and is notclaimed as a contribution of this record.This record refers to the proposed structure as Absolute-Address Partial Residency (AAPR). AAPR askswhether the same principle can be applied to execution relations in a trained dense Transformer. Aftertraining, the model weights, architecture, and the learned function mapping an internal state to the next stateremain fixed during inference, while actual activations remain input-dependent. The invariance of the trainedmodel therefore does not by itself imply that execution paths can be reused. An additional condition must betested: can some input-dependent execution relations be observed once, recorded, and reused later?To test this, post-hoc identifiers were assigned to all target channels of a trained dense Transformer.Activation histories were recorded while passing multiple inputs through the unmodified model, and a statictransition mapping keyed by previous activation addresses was constructed. In an NF4 4-bit Gemma 3 4Benvironment, closed-task validation using 80 of 2,560 MLP down_proj output channels per layer reproducedthe same top-1 result as an oracle that referenced the actual residual at every hop, across two seeds andtwelve hops for several task categories. A fact_geo failure caused by insufficient calibration coveragerecovered after adding fact-type examples and rebuilding only the transition map.A path that physically computed only the selected rows matched a path that performed full computation andthen retained the same rows by masking. Validation answers were also preserved after portions of the originalL1-12 down_proj weights were physically released from VRAM.

Absolute-Address Partial Residency

Published August 31, 2026 | Version v1

Technical note Open

Absolute-Address Partial Residency

Authors/Creators

Description

Caches, memoization, and lookup tables reuse previously computed information when that information remains valid and the corresponding object can be identified again. This principle itself is not new and is not claimed as a contribution of this record. This record refers to the proposed structure as Absolute-Address Partial Residency (AAPR). AAPR asks whether the same principle can be applied to execution relations in a trained dense Transformer. After training, the model weights, architecture, and the learned function mapping an internal state to the next state remain fixed during inference, while actual activations remain input-dependent. The invariance of the trained model therefore does not by itself imply that execution paths can be reused. An additional condition must be tested: can some input-dependent execution relations be observed once, recorded, and reused later? To test this, post-hoc identifiers were assigned to all target channels of a trained dense Transformer. Activation histories were recorded while passing multiple inputs through the unmodified model, and a static transition mapping keyed by previous activation addresses was constructed. In an NF4 4-bit Gemma 3 4B environment, closed-task validation using 80 of 2,560 MLP down\_proj output channels per layer reproduced the same top-1 result as an oracle that referenced the actual residual at every hop, across two seeds and twelve hops for several task categories. A fact\_geo failure caused by insufficient calibration coverage recovered after adding fact-type examples and rebuilding only the transition map. A path that physically computed only the selected rows matched a path that performed full computation and then retained the same rows by masking. Validation answers were also preserved after portions of the original L1-12 down\_proj weights were physically released from VRAM.

Files

###

AAPR\_Dated\_Technical\_Record\_v1\_English.pdf

###

Files (190.7 kB)

| Name | Size | Download all | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | AAPR\_Dated\_Technical\_Record\_v1\_English.pdf md5:d273514a33b820111e14e8b5fdc5f2b5 | 190.7 kB | Preview Download |

Jump up

zenodo.org · 8 images · 37 links

Comments (0)

No comments yet