DREAM: Diffusion Rectification and Estimation-Adaptive Models

Jinxin Zhou^1,*    Tianyu Ding^2,*,†    Tianyi Chen²    Jiachen Jiang²    Ilya Zharkov²    Zhihui Zhu¹    Luming Liang^2,†
1Ohio State University     ²Microsoft
CVPR 2024
Project Page | Paper

Turning the top to the bottom by adding only three lines of code.

We present DREAM, a novel training framework representing Diffusion Rectification and Estimation-Adaptive Models, requiring minimal code changes (just three lines) yet significantly enhancing the alignment of training with sampling in diffusion models. DREAM features two components: diffusion rectification, which adjusts training to reflect the sampling process, and estimation adaptation, which balances perception against distortion. When applied to image super-resolution (SR), DREAM adeptly navigates the tradeoff between minimizing distortion and preserving high image quality. Experiments demonstrate DREAM's superiority over standard diffusion-based SR methods, showing a 2 to 3x faster training convergence and a 10 to 20x reduction in necessary sampling steps to achieve comparable or superior results. We hope DREAM will inspire a rethinking of diffusion model training paradigms.

Citation

If you find our work helpful, please kindly cite our work:

@article{zhou2023dream,
  title={DREAM: Diffusion Rectification and Estimation-Adaptive Models},
  author={Zhou, Jinxin and Ding, Tianyu and Chen, Tianyi and Jiang, Jiachen and Zharkov, Ilya and Zhu, Zhihui and Liang, Luming},
  journal={arXiv preprint arXiv:2312.00210},
  year={2023}
}

or

@InProceedings{Zhou_2024_CVPR,
    author    = {Zhou, Jinxin and Ding, Tianyu and Chen, Tianyi and Jiang, Jiachen and Zharkov, Ilya and Zhu, Zhihui and Liang, Luming},
    title     = {DREAM: Diffusion Rectification and Estimation-Adaptive Models},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {8342-8351}
}