StyleGAN

> Most generators stir $z$ into every layer at the same time. StyleGAN split it apart: first map $z$ to an intermediate $w$, then inject $w$ at every resolution level through AdaIN. That single change untangled the latent space and made photorealistic faces a solved problem for seven years running.

Type: Build
Languages: Python
Prerequisites: Phase 8 · 03 (GANs), Phase 4 · 08 (Normalization), Phase 3 · 07 (CNNs)
Time: ~45 minutes

The Problem

A DCGAN maps $z$ to an image through a stack of transposed convolutions. The problem: $z$ controls everything — pose, lighting, identity, background — entangled together. Move along one axis of $z$, all four change. You cannot ask the model “same person, different pose” because the representation does not factor that way.

Karras et al. (2019, NVIDIA) proposed: stop feeding $z$ directly into conv layers. Feed a constant 4×4×512 tensor as the network input. Learn an 8-layer MLP that maps $z in Z to w in W$. Inject $w$ at every resolution via adaptive instance normalization (AdaIN): normalize each conv feature map, then scale and shift by affine projections of $w$. Add per-layer noise for stochastic detail (skin pores, hair strands).

The result: $W$ has roughly orthogonal axes for “high-level style” (pose, identity) vs “fine style” (lighting, color). You can swap styles between two images by using image A’s $w$ for the low-resolution levels and image B’s $w$ for the high. This unlocked editing, cross-domain stylization, and the entire “StyleGAN-inversion” line of research.

The Concept

!StyleGAN: mapping network + AdaIN + per-layer noise

Mapping network. $f: Z to W$, an 8-layer MLP. $Z = mathcal{N}(0, I)^{512}$. $W$ is not forced to be Gaussian — it learns a data-adapted shape.

Synthesis network. Starts from a learned constant 4×4×512. Each resolution block: upsample → conv → AdaIN(w_i) → noise → conv → AdaIN(w_i) → noise. Resolutions double: 4, 8, 16, 32, 64, 128, 256, 512, 1024.

AdaIN.

mathop{mathrm{AdaIN}}(x, y) = y_{text{scale}} cdot frac{x - mathop{mathrm{mean}}(x)}{mathop{mathrm{std}}(x)} + y_{text{bias}}

where $y_{text{scale}}$ and $y_{text{bias}}$ come from affine projections of $w$. Normalize per feature map, then restyle. “Style” here is the first- and second-order statistics of the feature map.

Per-layer noise. Single-channel Gaussian noise added to each feature map, scaled by a learned per-channel factor. Controls stochastic detail without affecting global structure.

Truncation trick. At inference, sample $z$, compute $w = text{mapping}(z)$, then $w’ = hat{w} + psi cdot (w – hat{w})$ where $hat{w}$ is the mean $w$ over many samples. $psi < 1$ trades diversity for quality. Almost every StyleGAN demo uses $psi approx 0.7$.

StyleGAN 1 → 2 → 3

In 2026 StyleGAN3 remains the default for (a) narrow-domain photorealism at high FPS, (b) few-shot domain adaptation (train on a new dataset with 100 images, freeze mapping), (c) inversion-based editing (find the $w$ that reconstructs a real photo, then edit that $w$). For open-domain text-to-image, it is not the tool — diffusion is.

gx-stylegan-mapping

Build It

code/main.py implements a toy “style-GAN lite” in 1-D: a mapping MLP, a synthesis function that takes a learned constant vector and modulates it with $w$-derived scale/bias, and per-layer noise. It shows that injecting $w$ via affine-modulation matches or beats concatenating $z$ into the generator’s input.

Step 1: mapping network

def mapping(z, M):
    h = z
    for i in range(num_layers):
        h = leaky_relu(add(matmul(M[f"W{i}"], h), M[f"b{i}"]))
    return h

Step 2: adaptive instance normalization

def adain(x, w_scale, w_bias):
    mu = mean(x)
    sd = std(x)
    x_norm = [(xi - mu) / (sd + 1e-8) for xi in x]
    return [w_scale * xi + w_bias for xi in x_norm]

Per-feature-map scale and bias come from $w$ via linear projection.

Step 3: per-layer noise

def add_noise(x, sigma, rng):
    return [xi + sigma * rng.gauss(0, 1) for xi in x]

Sigma per-channel is learnable.

Pitfalls

Droplet artifacts. StyleGAN 1 produced a blobby droplet in the feature maps because AdaIN zeroed out mean. StyleGAN 2’s weight demodulation fixes it by scaling the convolution weights instead.
Texture sticking. StyleGAN 1 and 2 textures followed pixel coordinates, not object coordinates (visible when interpolating). StyleGAN 3’s alias-free convolutions fix this with windowed sinc filters.
Mode coverage. Truncation ψ < 0.7 looks clean but samples from a narrow cone; use ψ = 1.0 if you need diversity.
Inversion is lossy. Inverting a real photo into $W$ is usually done through optimization or an encoder (e4e, ReStyle, HyperStyle). Results drift over many iterations.

Use It

| Use case | Approach |
|----------|----------|
| Photoreal human faces (anime, product, narrow) | StyleGAN3 FFHQ / custom fine-tune |
| Face editing from a photo | e4e inversion + StyleSpace / InterFaceGAN directions |
| Face swap / reenactment | StyleGAN + encoder + blending |
| Avatar pipelines | StyleGAN3 w/ ADA for low-data fine-tune |
| Domain adaptation from a few images | Freeze mapping network, fine-tune synthesis |
| Multi-modal or text-conditioned generation | Don't — use diffusion |

For product-grade demos where the answer is "photo of a person's face", StyleGAN beats diffusion on inference cost (single forward pass, <10ms on a 4090) and sharpness for the same quality bar.

Ship It

Save outputs/skill-stylegan-inversion.md. Skill takes a real photo and outputs: inversion method (e4e / ReStyle / HyperStyle), expected latent loss, editing budget (how far in $W$ you can move before artifacts), and a list of known-good editing directions (age, expression, pose).

Exercises

1. Easy. Run code/main.py with adain_on=True and adain_on=False. Compare the spread of outputs for a fixed latent vs perturbed latent.
2. Medium. Implement mixing regularization: for a training batch, compute $w_a$, $w_b$, and apply $w_a$ for the first half of synthesis and $w_b$ for the second half. Does the decoder learn disentangled styles?
3. Hard. Take a pretrained StyleGAN3 FFHQ model (ffhq-1024.pkl). Find the $w$ direction that controls "smile" by training an SVM on labelled samples; report how far you can push before identity drifts.

Key Terms

| Term | What people say | What it actually means |
|------|-----------------|-----------------------|
| Mapping network | "The MLP" | $f: Z to W$, 8 layers, decouples latent geometry from data statistics. |
| W space | "The style space" | Output of the mapping network; roughly disentangled. |
| AdaIN | "Adaptive instance norm" | Normalize feature map, then scale + shift by $w$-projection. |
| Truncation trick | "Psi" | $w = text{mean} + psi cdot (w - text{mean})$, $psi<1$ trades diversity for quality. | | Path-length regularization | "PL reg" | Penalizes large changes in image per unit change in $w$; makes $W$ smoother. | | Weight demodulation | "The StyleGAN2 fix" | Normalize conv weights instead of activations; kills droplet artifacts. | | Alias-free | "StyleGAN3's trick" | Windowed sinc filters; eliminates texture sticking to the pixel grid. | | Inversion | "Find w for a real image" | Optimize or encode $x to w$ so $G(w) approx x$. |

Production note: why StyleGAN still ships in 2026

StyleGAN3 on a 4090 generates a 1024² FFHQ face in under 10 ms — num_steps = 1, no VAE decode, no cross-attention pass. In production terms this is the floor latency for any image generator. A 50-step SDXL + VAE-decode pipeline at the same resolution is ~3 seconds. That is a 300× gap, and for narrow-domain products (avatar services, ID document pipelines, stock face generation) it wins on TCO.

Two operational consequences:

No scheduler, no batcher. Static batch at the target occupancy is optimal. Continuous batching (essential for LLMs and diffusion) provides zero benefit because every request takes the same FLOPs.
Truncation ψ is the safety knob. ψ < 0.7 samples from a narrow cone of the mapping network's range. This is the only lever the serving layer has over sample variance. Lower ψ at peak load, raise it for premium users.

AI Engineering from Scratch StyleGAN

AI Engineering from Scratch