Feature Fusion Analysis

This project is primarily a personal research journey into understanding why resdual/skip connections and heterogeneous representation merging work — not merely that they work — from the standpoint of feature geometry and representation structure. The codebase provides a complete pipeline for training a simplified ResNet-style architecture on CIFAR-100, and another for analyzing how specific feature fusion methods applied at the skip connections affect intermediate representations.

Quick Start

1. Install requirements

pip install -r requirements.txt

2. Run all experiments and analysis

python run_all.py --device cuda

• --device can be cuda or cpu (default: cuda if available).

This will:

• Train models for all fusion methods and seeds.
• Run the analysis and generate plots in the analysis/ directory.

---

Research Motivation

Architectures such as ResNet are now canonical in deep learning, but residual connections themselves are not unique to convolutional networks. In Transformers, for example, residual connections are applied in an almost minimal form: representations are passed forward via simple addition around self-attention and feedforward blocks. These residual pathways are largely treated as structural necessities rather than objects of analysis.

At the other end of the spectrum lie models with explicit and complex feature fusion mechanisms, such as those found in multimodal machine learning. Vision–language models, audio–visual systems, and multi-sensor architectures routinely fuse heterogeneous representations using concatenation, learned projections, attention-based mixing, or gating mechanisms. In these settings, fusion is understood as a critical design decision that shapes representational alignment and downstream behavior.

Despite this wide range—from minimal additive skips in Transformers to rich fusion operators in multimodal models—the dominant justification for any particular fusion strategy remains empirical task performance. What is far less clear—and rarely visualized—is how different fusion operations reshape feature representations, and why certain operations lead to improved task-specific performance.

More concretely, this project aims to address and/or visualize the following:

• What geometric transformations do different fusion operations induce in feature space?
• How do these transformations evolve across network depth?
• In what sense do certain fusion strategies produce “better” representations?

By extracting and visualizing intermediate representations, the goal is to treat feature fusion as a representation-shaping operation whose effects can be directly inspected, compared, and reasoned about.

Experimental Setup

Architecture

• A lightweight ResNet‑style CNN
• Instrumented to support feature extraction at multiple depths
• Configurable fusion operations for each residual connection

Dataset

• CIFAR‑100

Fusion Operations

Four fusion operations are investigated. Each represents a mapping $f(x_{k-1},x_k) \rightarrow z_k$.

1. None - Simple pass through

$$ z_k = x_k $$

2. Add - Element-wise addition of features

$$ z_k = x_{k-1} \oplus x_{k} $$

3. Concat - $\text{Concatenation} \rightarrow \text{Conv}_{1\times1}$

$$ z_k = \text{ReLU} \left( \text{BatchNorm} \left( \text{Conv}_{1\times1}([x_{k-1};x_k]) \right) \right) $$

4. ProjectConcat - $\text{Conv}{1\times1} \rightarrow \text{Concatenation} \rightarrow \text{Conv}{1\times1}$

$$ x' = \text{Projection}(x) = \text{ReLU} \left( \text{BatchNorm} \left( \text{Conv}_{1\times1}(x) \right) \right) $$

$$ z_k = \text{ReLU} \left( \text{BatchNorm} \left( \text{Conv}_{1\times1}([x'_{k-1};x'_k]) \right) \right) $$

Training and Analysis Pipeline

The core training and analysis pipeline proceeds as follows:

1. Train a model for each fusion operation at 4 different random seeds, 16 models total.
2. Collect intermediate activation tensors, $x_k$ and $z_k$
3. Apply spatial aggregation (e.g., GAP) and store for downstream analysis
4. Analyze linear Centered Kernel Alignment (CKA) between fusion layer inputs, as well as CKA values between inputs and corresponding outputs.
5. Analyze effective ranks of concatenated fusion inputs and, separately, fusion outputs.

Metrics

The following metrics are used to analyze the effects different fusion operations have on representation shaping.

• Linear Centered Kernel Alignment
• Effective Rank

Linear Centered Kernel Alignment (CKA)

Linear Centered Kernel Alignment (Linear CKA) is a statistical technique used to quantify the similarity between two representations of the same dataset, such as feature embeddings produced by different layers of a neural network or different models. It is widely used in machine learning to understand the relationships between learned neural network feature representations.

Linear CKA is scale-invariant, and unlike local similarity metrics (e.g., cosine similarity, which operates on individual vectors), Linear CKA captures the overall structural similarity between two sets of representations in a global sense.

Mathematical Formulation:

Linear CKA measures how similar two matrices of representations $X$ and $Y$ are, where $X \in \mathbb{R}^{n \times p}$ and $Y \in \mathbb{R}^{n \times q}$. Here:

• Rows $n$ correspond to data points or examples.
• Columns $p$ in $X$, $q$ in $Y$ correspond to features or dimensions in the respective representations.

Before calculating the linear kernel, the matrices X and Y are centered to ensure the comparison focuses on the structure of the data and removes biases introduced by the means of the feature vectors. Linear CKA between two centered matrices $X$ and $Y$ is defined as,

$$\text{CKA}(X, Y) = \frac{| X^\top Y |_F^2}{| X^\top X |_F \cdot | Y^\top Y |_F}$$

Effective Rank

The Effective Rank $\text{erank}$ is a measure derived from the singular value decomposition (SVD) of a matrix, used to quantify the "dimensionality" or "complexity" of the data encoded in a representation. Effective Rank considers the distribution of the singular values and captures the "spread" of information across them. It is a valuable metric in data analysis and machine learning because it provides insights into the "informational richness" of representations, beyond just counting dimensions.

Mathematical Formulation

When you compute the singular value decomposition (SVD) of a matrix $X \in \mathbb{R}^{n \times d}$ you obtain,

$$X = U \Sigma V^\top$$

Where,

• $U$ and $V$ are orthogonal matrices corresponding to the left and right singular vectors, respectively.
• $\Sigma$ is a diagonal matrix of singular values $(\sigma_1, \sigma_2, \dots, \sigma_r)$, where $\sigma_1 \geq \sigma_2 \geq \dots \geq \sigma_r \geq 0$, and $r \leq \min(n, d) $ is the rank of $X$.

The Effective Rank of $X$ is a normalized measure of how "spread out" the singular values $\sigma_i$ are. It is defined as,

$$\text{erank}(X) = \exp(-\sum_{i=1}^r p_i \log(p_i))$$

where $p_i$ is the normalized proportion of the i-th singular value relative to the sum of all singular values.

Results

Validation Loss and Accuracy

• Similar training dynamics are observed between the four models with plateaus between 60 and 80 epochs. Concat and ProjectConcat models achieve both the lowest validation losses and highest validation accuracies of the four considered.

• Projection before concatenation appears to enable better alignment between feature spaces before combining, leading to better representation and downstream performance.

• Concatenation and convolution itself outperforms the traditional add operation likely because the $(1 \times 1)$ convolution learns how to optimally fuse the concatenated channels.

Linear CKA Fusion Inputs Pairs

• The None method shows the highest similarity among feature maps throughout the different layers of the model, indicationg highly redundant, less diverse representations learned by the network.

• Add appears to introduce moderate diversity between fusion inputs.

• Concat and ProjectConcat exhibit the lowest similarity implying that concatenation and/or convolution-based methods encourage richer, more diverse representations at deeper network layers.

Linear CKA of Fusion Input-Output Pairs

Alternating Similarity Pattern

• The higher similarity values between $x_{n-1}, z_n$ indicate that fusion outputs $z_n$ retain more information from the shallower feature maps $x_{n-1}$. This is because earlier-layer features tend to be less task-specific and capture broader, simpler patterns (e.g., edges or textures). These features require less transformation before forming the fused representation.
• The lower similarity values between $x_n, z_n$ demonstrate that fusion outputs $z_n$ undergo greater transformation to integrate more abstract representations from deeper layers $x_n$.

Fusion at Intermediate Layers

• At intermediate layers (e.g., $z_3 = \text{fusion2}(x_3, x_2)$), the network learns to decorrelate and transform the more abstract representations from the current layer $x_3$, while still retaining valuable foundational patterns from an earlier step $x_2$. The resulting fusion output $z_3$ is more diverse and enriched to provide meaningful representations for downstream layers.
• The low similarity between (x_n) and (z_n) at intermediate levels ((x_2)-(z_3) and (x_3)-(z_4)) reflects significant transformations to boost task-specific performance in subsequent layers.

Re-alignment of Features in Deeper Layers

• As the network nears its output layers (e.g., $z_4$), the representations begin converging on task-specific outputs, leading to higher similarity between input-ouput pairs.
• This aligns with well-established behaviors in deep neural networks, where later layers prioritize aligning features directly with the task space. Representations closer to the classifier head tend to encode abstract patterns that directly correspond to the task (e.g., class-level features in classification tasks) and so overlap more with input feature maps from nearby layers.

Effective Ranks of Fusion Inputs

• Shallow layers start with high diversity, as they process low-level features that capture general patterns (e.g., edges, textures).
• Deeper layers produce more task-specific features, reducing overall dimensionality and diversity as the model prioritizes relevant representations over raw feature diversity.
• Add generates the highest effective ranks overall, indicating the largest feature diversity in terms of raw fusion inputs.
• Concat and ProjectConcat maintain moderate effective ranks within their fusion inputs, but not to the extend of Add.

Effective Ranks of Fusion Outputs

U-shaped pattern of effective ranks

• The "U-shaped" erank pattern in the fusion outputs ((z_2 \to z_3 \to z_4)) indicates that diversity is initially high in the early fusion output due to reasons discussed earlier. Intermediate fusion outputs exhibit lower diversity due to feature compression and abstraction. Deeper layers recover diversity due to the network's focus on task-specific alignment in the final feature representations, particularly for effective fusion operations like Concat and ProjectConcat

Top Performers Concat and ProjectConcat

• High and persistent effective ranks in fusion outputs, especially at $z_4$ indicate a consistent ability to produce diverse and enriched feature representations. The learned projection mechanisms likely prevent feature redundancy in all fusion stages.

Poor Performers Add and None

• Add shows reasonable effective rank at the shallowest layers but experiences a steady, linear decline as it produces increasingly redundant fused representations at deeper layers.

• None fails to enrich representations with consistently low effective ranks showing its outputs are highly redundant, as expected without any fusion mechanism.

Conclusion

Through an analysis of fusion strategies applied to skip connections in a simplified ResNet-style architecture, this project highlights how different fusion operations influence feature geometry and representation structure at various network depths. The findings demonstrate that feature fusion is not merely a structural design but a powerful representation-shaping mechanism that directly impacts task-specific performance.

Among the methods studied:

• Concat and ProjectConcat provide the most effective representational diversity, particularly due to their ability to produce enriched, task-specific feature outputs while maintaining high effective ranks.
• Add, while preserving diversity in earlier layers, shows a decline in representation quality in deeper layers due to limited transformation ability.
• None, without any fusion mechanism, results in highly redundant representations with minimal diversity and is the weakest performer across all metrics.

Overall, the results suggest that maximizing the diversity of inputs and outputs while balancing meaningful alignment at deeper layers is a key factor in improving network performance. Fusion operations like Concat and ProjectConcat offer deeper insights into how heterogeneous representations can be optimally combined to improve task-specific learning.

Author

Jonathan Hestroffer