Command Palette
Search for a command to run...
DePass: Unified Feature Attributing by Simple Decomposed Forward Pass
DePass: Unified Feature Attributing by Simple Decomposed Forward Pass
Xiangyu Hong Che Jiang Kai Tian Biqing Qi Youbang Sun Ning Ding Bowen Zhou
Abstract
Attributing the behavior of Transformer models to internal computations is a central challenge in mechanistic interpretability. We introduce DePass, a unified framework for feature attribution based on a single decomposed forward pass. DePass decomposes hidden states into customized additive components, then propagates them with attention scores and MLP's activations fixed. It achieves faithful, fine-grained attribution without requiring auxiliary training. We validate DePass across token-level, model component-level, and subspace-level attribution tasks, demonstrating its effectiveness and fidelity. Our experiments highlight its potential to attribute information flow between arbitrary components of a Transformer model. We hope DePass serves as a foundational tool for broader applications in interpretability.