introduces two new components to enable unified multi-task controllable generation:
1. Mixture-of-Experts Adapters: Parallel convolutional modules, one per task, that adapt to each condition’s visual features.
2. Task-Aware HyperNetwork: Dynamically modulates the convolution kernels of a base model given embeddings of task instructions.
UniControl is trained on twelve distinct tasks spanning edges, regions, maps, and more. The overall model architecture remains consistent across tasks, while the conditioning components specialize.
Mixture-of-Experts Adapters
The adapters provide dedicated pathways for each task to process america phone number list its visual features appropriately. This overcomes the misalignment between diverse conditions needing specialized handling.
For example, a segmentation map pathway focuses more on spatial semantic relations than 3D geometry. In contrast, a depth adapter will emphasize global layout and surface orientations. With separate adapters per task, UniControl can extract nuanced representations tailored to each input type.