New training technique enhances representation quality in diffusion autoencoders
Researchers introduced SteeringDRL, a method that steers diffusion autoencoders toward stronger latent representations while maintaining image reconstruction quality.
A team of researchers has presented SteeringDRL, a training approach that combines gated residual U-Nets with a curriculum that gradually adjusts noise exposure. The method aims to guide diffusion autoencoders toward more robust latent representations without sacrificing image fidelity.
Analysis of training dynamics revealed two early-stage trajectories: a reconstruction-focused path that quickly optimises image similarity, and a disentanglement-focused path that balances reconstruction quality with the development of clearer internal representations. By modifying shortcut pathways in the diffusion U-Net and controlling early noise levels, the authors were able to influence which trajectory the model follows.
Experiments on standard disentanglement benchmarks showed that models trained with SteeringDRL achieved higher representation quality and exhibited reduced sensitivity to random initialisation. The approach also improved spatial disentanglement in object-centric learning, yielding better segmentation results on both synthetic and real-world image datasets.
