ControlNet is an architectural extension for diffusion-based image generation models that introduces a trainable copy of a portion of the base model's encoder, conditioned on a spatial control signal such as an edge map, depth map, human pose skeleton, or semantic segmentation mask. The control signal is injected into the generation process at each relevant layer, allowing the output image to conform to a desired spatial structure while the base model still governs texture, style, and semantic content. Because only the auxiliary network's weights are trained and the original model weights remain frozen, ControlNet can be trained on relatively modest datasets and added or removed modularly without altering the base checkpoint.

Think of it like a stencil laid over a canvas: the stencil determines the shapes and outlines that must appear, while the painter still chooses the colors, medium, and overall mood. The stencil constrains composition without dictating artistry.

A character artist might extract a pose skeleton from a reference photograph using a pose estimator, feed that skeleton to ControlNet alongside a text prompt, and generate a new illustration where the figure adopts the exact same pose even though the appearance is entirely different. Without ControlNet, achieving a specific pose would require extensive prompt iteration with no guarantee of spatial accuracy.

ControlNet is closely related to Diffusion models, which form its base, Latent Space operations since the conditioning acts within the diffusion model's latent representation, Fine-tuning since the auxiliary network is trained separately, and Multimodal inputs since the conditioning signal is often a non-text modality such as depth or pose data.

You will encounter ControlNet in image generation pipelines built on Stable Diffusion, in creative tools like ComfyUI and Automatic1111, and in research on controllable synthesis for animation, fashion, and architectural visualization.