Deep Learning-Driven Ultra-High-Definition Image Restoration: A Survey
Liyan Wang, Weixiang Zhou, Cong Wang, Kin-Man Lam, Zhixun Su, Jinshan Pan
Abstract
Ultra-high-definition (UHD) image restoration aims to specifically solve the problem of quality degradation in ultra-high-resolution images. Recent advancements in this field are predominantly driven by deep learning-based innovations, including enhancements in dataset construction, network architecture, sampling strategies, prior knowledge integration, and loss functions. In this paper, we systematically review recent progress in UHD image restoration, covering various aspects ranging from dataset construction to algorithm design. This serves as a valuable resource for understanding state-of-the-art developments in the field. We begin by summarizing degradation models for various image restoration subproblems, such as super-resolution, low-light enhancement, deblurring, dehazing, deraining, and desnowing, and emphasizing the unique challenges of their application to UHD image restoration. We then highlight existing UHD benchmark datasets and organize the literature according to degradation types and dataset construction methods. Following this, we showcase major milestones in deep learning-driven UHD image restoration, reviewing the progression of restoration tasks, technological developments, and evaluations of existing methods. We further propose a classification framework based on network architectures and sampling strategies, helping to clearly organize existing methods. Finally, we share insights into the current research landscape and propose directions for further advancements.
摘要
超高清(UHD)图像修复旨在针对性解决超高分辨率图像的质量退化问题。该领域的最新进展主要由基于深度学习的创新驱动,包括数据集构建、网络架构设计、采样策略优化、先验知识融合和损失函数改进等方面的增强。本文系统综述了UHD图像修复的最新进展,涵盖从数据集构建到算法设计的多个维度,为理解该领域前沿发展提供了重要参考。我们首先总结了多种图像修复子问题(如超分辨率、低光增强、去模糊、去雾、去雨、去雪等)的退化模型,并强调了其在UHD图像修复中面临的独特挑战;接着梳理了现有UHD基准数据集,并根据退化类型与数据集构建方法对文献进行分类;随后展现了深度学习驱动的UHD图像修复主要里程碑,回顾了修复任务、技术发展与现有方法的评估进展;进一步提出基于网络架构与采样策略的分类框架,助力系统性组织现有方法;最后,针对当前研究现状分享了见解并提出了未来发展方向的建议。
Introduction
Recently, with the rapid advancement of imaging and acquisition equipment, ultra-high-definition (UHD) images featuring high pixel density and resolutions (e.g., 3,840 × 2,160 pixels or higher), become widely used in fields such as video streaming [1], [2], virtual reality [3], medical imaging [4], and satellite remote sensing [5], [6]. This surge in applications has significantly heightened user demand for enhanced image clarity and detail performance. However, hardware limitations, restricted transmission bandwidth, and challenging acquisition environments often impede the production of high-quality UHD images. Common challenges include insufficient resolution and degradation factors such as blur, rain, snow, haze, low light, etc. Fig. 1 illustrates examples of image degradation under different conditions, highlighting how these issues compromise image resolution and overall quality. Consequently, UHD image restoration has emerged as a critical area of research within computer vision and image processing.
近年来,随着成像与采集设备的快速发展,具有高像素密度和超高分辨率(如3,840 × 2,160像素及以上)的超高清(UHD)图像在视频流媒体、虚拟现实、医学影像、卫星遥感等领域广泛应用。这一应用热潮显著提升了用户对图像清晰度与细节表现的需求。然而,硬件限制、传输带宽约束以及复杂采集环境往往导致高质量UHD图像难以生成,常见挑战包括分辨率不足以及模糊、雨雪、雾霾、低光等退化因素。如图1所示,不同条件下的图像退化实例展示了这些问题如何损害图像分辨率与整体质量。因此,UHD图像修复已成为计算机视觉与图像处理领域的关键研究方向。
UHD image restoration aims to recover high-quality UHD images from degraded inputs, addressing various sub-problems, such as UHD image super-resolution, deblurring, dehazing, low-light image enhancement, deraining, and desnowing. While deep learning-based image restoration methods [7]–[17] have achieved remarkable success on lower-resolution images (e.g., low-light image enhancement [18]–[20], dehazing [21], [22], desnowing [23], [24], deraining [25]–[28], and deblurring [29], [30]) thanks to advances in techniques like convolutional neural networks (CNNs) [31], [32] and Transformers [33], [34], their effectiveness remains limited when applied to UHD image restoration. This limitation stems from the fact that these models are often designed for low-resolution inputs and struggle to scale efficiently to UHD images. In addition, UHD images inherently possess more intricate details, a wider color gamut, and a significantly larger number of pixels, which pose unique computational challenges.
UHD图像修复旨在从退化输入中恢复高质量的UHD图像,涵盖超分辨率、去模糊、去雾、低光增强、去雨、去雪等多种子问题。尽管基于深度学习的图像修复方法在低分辨率图像上(如低光增强、去雾、去雪、去雨、去模糊)凭借卷积神经网络(CNN)与Transformer等技术取得了显著成功,但其在UHD图像修复中的效果仍受限。这主要因为此类模型通常针对低分辨率输入设计,难以高效扩展到UHD图像。此外,UHD图像本身包含更精细的细节、更广的色域及显著增多的像素,这带来了独特的计算挑战。
Since the introduction of the first large-scale dataset for UHD image super-resolution reconstruction tasks [35] in 2021, a range of UHD image restoration methods and corresponding datasets have been developed. These include multi-guided bilateral learning for UHD image dehazing [36] with the 4KID dataset; multi-scale separable-patch integration networks for video deblurring [37] with the 4DRK dataset; the Transformer-based LLFormer [38] with the UHD-LOL dataset and the Fourier embedding network UHDFour [39] with the UHD-LL dataset for UHD low-light image enhancement; and UHDformer [40], which explores feature transformation between high- and low-resolution, with the UHD-Haze/UHD-Blur datasets for UHD image restoration. Additionally, the dual interaction prior-driven network UHDDIP [41], paired with the UHDSnow/UHD-Rain datasets, addresses UHD image restoration.
These methods employ diverse strategies, including downsampling-enhancement-upsampling structure [37], [39], [41]–[48], encoder-decoder structures with stepwise up-downsampling [38], [49]–[51], and resampling-enhancement structures [52], [53]. The evolution of network models has transitioned from CNNs [37], [39], [42], [49] (emphasizing local feature extraction) to Transformer-based models [38], [41], [43], [48] (focusing on global modeling), and most recently to Multilayer Perceptrons (MLPs) [44]–[47] and Mamba [50] (aiming to reduce computational overhead). From 2021 to the present, approximately 20 studies have explored deep learning-based UHD image restoration methods. A concise summary of these developments is presented in Fig. 2.
自2021年首个面向超高清(UHD)图像超分辨率重建任务的大规模数据集提出以来,一系列UHD图像修复方法及对应数据集相继涌现。例如:
- 基于4KID数据集的多引导双边学习网络用于UHD图像去雾;
- 基于4DRK数据集的多尺度可分离补丁集成网络用于视频去模糊;
- 基于UHD-LOL数据集的Transformer架构LLFormer,以及基于UHD-LL数据集的傅里叶嵌入网络UHDFour用于UHD低光图像增强;
- 基于UHD-Haze/UHD-Blur数据集的UHDformer,探索高低分辨率间的特征变换以实现UHD图像修复;
- 结合UHDSnow/UHD-Rain数据集的双交互先验驱动网络UHDDIP处理UHD图像修复。
这些处理UHD图像的方法采用了多样化策略,包括:
- 下采样-增强-上采样结构;
- 分步上下采样的编码器-解码器结构;
- 重采样-增强结构。
网络模型的演进从侧重局部特征提取的CNN,转向聚焦全局建模的Transformer,再到近年旨在降低计算开销的MLP与Mamba。2021年至今,已有约20项研究探索基于深度学习的UHD图像修复方法,其发展脉络可简要总结如图2所示。
Although deep learning has dominated research on UHD image restoration, there is a lack of comprehensive and in-depth surveys on deep learning-based solutions. Therefore, our work systematically and comprehensively reviews the research in the field to provide a useful starting point for understanding major developments, limitations of existing approaches, and potential future research directions. The main contributions of this paper are threefold:
• We conduct a systematic review of research progress in deep learning-based UHD image restoration, covering problem definitions for various UHD restoration tasks, challenges, the development of benchmark datasets, and the improvements and limitations of existing methods.
• We propose an effective classification method for existing deep learning-based UHD image restoration methods and analyze representative benchmarks under different subtasks.
• Finally, we discuss the challenges faced by current deep learning methods and outline promising future directions.
尽管深度学习主导了UHD图像修复研究,但针对基于深度学习的解决方案仍缺乏全面且深入的综述。因此,本文通过系统性梳理与综合分析,为理解该领域的主要进展、现有方法的局限性及未来潜在研究方向提供参考。本文的主要贡献包括以下三点:
- 系统性综述:覆盖各类UHD修复任务的问题定义、挑战、基准数据集发展及现有方法的改进与局限性;
- 分类与基准分析:提出基于深度学习的UHD图像修复方法分类框架,并在不同子任务下分析代表性基准;
- 挑战与未来方向:探讨当前深度学习方法面临的挑战,并展望有潜力的研究方向。
Challenges
UHD images, affected by the aforementioned degradation processes, pose unique challenges for restoration due to their ultra-high resolution and dense pixel characteristics:
• High computational requirements: Compared to high-definition (HD) images, UHD images contain significantly more pixels. Processing such large-scale feature maps demands substantial computational resources and storage, necessitating advanced GPUs and high-performance hardware.
• Difficulty in recovering details: UHD images capture intricate details, and degradation effects are more pronounced at high resolutions. Although existing methods perform adequately with low-resolution images, they often struggle to preserve fine structures in UHD images, resulting in texture loss or artifacts in the restoration output.
• Lack of training datasets: Methods trained on low-resolution datasets cannot directly process UHD images, as they require large amounts of degraded-clear UHD image pairs for fine-tuning. However, most publicly available datasets are dominated by HD and lower-resolution images, with a notable lack of datasets specialized for UHD scenarios. This limitation significantly hinders the development and testing of algorithms.
• Poor algorithm adaptability: Many restoration algorithms are designed for low-resolution images and fail to scale effectively to UHD images. Developing new algorithms that accommodate the distinct characteristics of UHD images is an urgent need.
受前述退化过程影响的超高清(UHD)图像,因其超高分辨率与密集像素特性,在修复过程中面临独特挑战:
- 高计算需求:相较于高清(HD)图像,UHD图像包含显著更多的像素。处理此类大规模特征图需消耗大量计算资源与存储空间,必须依赖高端GPU及高性能硬件。
- 细节恢复困难:UHD图像包含精细细节,且退化效应在高分辨率下更显著。现有方法虽能处理低分辨率图像,却常难以有效保留UHD图像的细微结构,导致修复结果出现纹理丢失或伪影。
- 训练数据匮乏:基于低分辨率数据集训练的方法无法直接处理UHD图像,需大量退化-清晰UHD图像对进行微调。然而,公开数据集以HD及以下分辨率图像为主,专为UHD场景设计的数据集严重不足,极大阻碍算法开发与测试。
- 算法适应性差:多数修复算法针对低分辨率图像设计,难以有效扩展至UHD图像。亟需开发适应UHD图像特性的新算法
Datasets
This section highlights the key benchmark datasets developed for UHD image restoration, categorized by the specific challenges they address.
Approaches
This section presents a comprehensive and systematic review of existing deep learning-based UHD image restoration methods. This review covers various tasks, including super-resolution reconstruction, low-light image enhancement, dehazing, deblurring, deraining, and desnowing.
Technical
In this section, we examine the evolution of deep learningbased UHD image restoration methods, focusing on three critical aspects:
- network architecture
- sampling strategy
- loss function
1. Network Architectures
Existing UHD image restoration models employ a variety of advanced network architectures, including convolutional neural networks (CNNs) [31], UNet [62], pyramid networks [63], multilayer perceptrons (MLPs) [64], Transformers [33], and Mamba [65]. These architectures provide different processing methods and technical support for UHD image restoration tasks. To systematically analyze their characteristics and application scenarios, we categorize existing models into three forms based on UHD image processing methods:
- Downsampling-enhancement-upsampling structure,
- Encoder-decoder structure with stepwise up-downsampling,
- Resampling-enhancement structure.
现有UHD图像修复模型
现有模型采用多种先进网络架构,包括:
- 卷积神经网络(CNNs)
- UNet
- 金字塔网络
- 多层感知机(MLPs)
- Transformer
- Mamba
这些架构为UHD图像修复任务提供差异化处理方法与技术支持。为系统分析其特性与应用场景,依据UHD图像处理方法将现有模型分为三类:
- 下采样-增强-上采样结构
- 分步上下采样的编码器-解码器结构
- 重采样-增强结构
1.1 Downsampling-Enhancement-Upsampling
Figure 4: Summary of the downsampling-enhancement-upsampling structure for UHD image restoration. (a) The single-branch downsampling-enhancement-upsampling architecture focuses on the design of enhancement networks in the low-resolution space, utilizing some popular architectures such as CNN, UNet, MLP, and Transformer. (b) The dual-branch downsampling-enhancement-upsampling architecture explores the correlation between high- and low-resolution features or incorporates additional prior information to guide the reconstruction process.
1.1 下采样-增强-上采样结构
图4总结了用于超高清(UHD)图像恢复的下采样-增强-上采样结构。(a) 单分支下采样-增强-上采样架构侧重于在低分辨率空间中设计增强网络,采用流行架构如CNN、UNet、MLP和Transformer。(b) 双分支下采样-增强-上采样架构探索高分辨率与低分辨率特征之间的相关性,或融入额外先验信息以指导重建过程。
Figure 5: Overview of dual branch frameworks under the downsampling-enhancement-upsampling structure. DMixer [45] upsamples low-resolution features and merges them with high-resolution features for reconstruction. UDR-Mixer [47] feeds high-resolution features into the low-resolution branch to facilitate reconstruction. UHDformer [43] transforms features from high to low resolution and enhances high-resolution reconstruction through concatenation. UHDDIP [41] extracts gradient and normal priors in the low-resolution space to interact with low-resolution features, guiding high-resolution reconstruction.
图 5 总结了下采样-增强-上采样结构下的双分支框架。DMixer 上采样低分辨率特征并将其与高分辨率特征融合以进行重建。UDR-Mixer 将高分辨率特征输入低分辨率分支以促进重建。UHDformer 将特征从高分辨率转换为低分辨率,并通过拼接增强高分辨率重建。UHDDIP 在低分辨率空间中提取梯度和法线先验,以与低分辨率特征交互,指导高分辨率重建。
1.2 Encoder-Decoder with Stepwise Up-downsampling
Figure 6: Summary of the Encoder-Decoder structure with stepwise up-downsampling for UHD image restoration. UHDVD [37] and LapDehazeNet [49] progressively downsample the input image based on separable-patch and Laplace pyramid architectures to encode features respectively, while LLFormer [38] and Wave-Mamba [50] reduce the scale of potential features with stepwise and adopt the core components of the Axis-based Transformer block and the Low-Frequency State Space Block (LFSS Block) for restoration, respectively.
1.2 带逐步上下采样的编码器-解码器结构
图 6 总结了用于UHD图像恢复的带逐步上下采样的编码器-解码器结构。UHDVD 和 LapDehazeNet 分别基于可分离块(separable-patch)和拉普拉斯金字塔架构逐步下采样输入图像以编码特征;而 LLFormer 和 Wave-Mamba 通过逐步方式降低潜在特征尺度,并分别采用基于轴向的Transformer块和低频状态空间块(LFSS Block) 作为核心组件进行恢复。
1.3 Resampling-Enhancement
Figure 7: Summary of the resampling-enhancement structure for UHD image restoration. Unlike previous methods that rely on uniform, content-agnostic downsampling (represented by the gray arrows), (a) a non-uniform sampling enhancement network (NSEN) [52], which incorporates two core designs: 1) content-guided downsampling to generate detail-preserving low-resolution images, and 2) invertible pixel alignment that computes inverse functions to remove distortions induced during the downsampling process; and (b) a learning model-aware resampling (LMAR) [53], which focuses on obtaining compensated low-resolution features from UHD input images guided by model knowledge. These features are fed into the enhancer, along with original low-resolution features, and are subsequently upsampled to UHD results.
1.3 重采样-增强结构
图 7 总结了用于UHD图像恢复的重采样-增强结构。与依赖均匀、内容无关下采样(灰色箭头表示)的传统方法不同,(a) 非均匀采样增强网络(NSEN)包含两大核心设计:1) 内容引导下采样以生成保留细节的低分辨率图像;2) 可逆像素对齐,通过计算逆函数消除下采样过程中的失真。(b) 学习模型感知重采样(LMAR)专注于从UHD输入图像中获取模型知识引导的补偿低分辨率特征。这些特征与原始低分辨率特征一同输入增强器,最终上采样为UHD结果。
2 Sampling Strategy
在现有超高清(UHD)图像修复框架中,三种主要采样策略被广泛采用:
- 高倍率上下采样(如4×、8×、16×、32×),常用方法包括双三次插值、双线性插值和PixelUnshuffle。这些方法虽能减少计算负担,但高倍率下导致显著信息丢失,影响修复质量(例如,PixelUnshuffle在PSNR/SSIM指标最优,但双线性插值在LPIPS更优)。
- 逐步上下采样(如从2×降至16×),通常集成于编码器-解码器结构以提取多尺度特征。然而,下采样程度增加会不可逆损伤图像质量,后续上采样难以恢复。
- 内容相关重采样,通过自适应调整采样率解决上述问题。例如:
- 小波变换避免关键信息丢失,
- Min-p采样保留高置信度特征并丢弃次要特征,
- 非均匀下采样器根据图像细节丰富度动态采样,
- 模型感知重采样(LMAR) 利用模型知识定制采样,兼容现有插值方法且无需重新训练增强网络。
核心挑战在于平衡计算效率与信息保留,高倍率采样虽高效但牺牲细节,而内容相关策略通过智能优化(如动态采样和模型驱动)提升鲁棒性。
3 Loss Functions
1) Pixel-Level Loss
- 𝐿1 loss (Mean Squared Error, MAE)
- 𝐿2 loss (Mean Squared Error, MSE)
- 𝑆𝑚𝑜𝑜𝑡ℎ𝐿1 loss [66]
- 𝐶ℎ𝑎𝑟𝑏𝑜𝑛𝑛𝑖𝑒𝑟 loss
- 𝑇𝑉 (Total Variation) loss
2)𝑆𝑆𝐼𝑀 loss [67]
3)Frequency-Domain Loss [68]
4)Perceptual loss [69]
5)Adversarial loss [70]
Evaluation
1 Evaluation Metrics
1.1 Full-Reference Metrics
PSNR. Peak Signal-to-Noise Ratio (PSNR) [71]
SSIM. Structural Similarity Index Measure (SSIM) [67]
MAE/MSE. Mean Absolute Error (MAE) and Mean Squared Error (MSE)
LPIPS. LPIPS [72] assesses perceptual similarity by calculating the distance between feature representations in a neural network. This deep learning-based metric approximates human judgment of image quality. Lower LPIPS values correspond to higher perceptual similarity.
全参考指标
- PSNR(峰值信噪比)[71]:衡量修复图像与真实图像之间的信噪比,值越高表示质量越接近真实图像。
- SSIM(结构相似性指数)[67]:通过亮度、对比度和结构相似性综合评估图像质量,值越接近1表示修复效果越好。
- MAE/MSE(平均绝对误差/均方误差):直接计算像素级差异,MAE值越小表示误差越低,MSE值越小表示修复精度越高。
- LPIPS(基于学习的感知相似性指标)[72]:利用神经网络提取特征的距离评估感知相似性,更贴近人类对图像质量的判断。LPIPS值越低,表示感知质量越接近真实图像。
1.2 No-Reference Metrics
NIQE. The primary purpose of Natural Image Quality Evaluator (NIQE) [75] is to assess an image’s naturalness based on statistical models of natural scenes. It determines how visually natural and realistic an image appears without requiring reference images. Lower NIQE scores indicate more natural and realistic images.
MUSIQ. Multi-scale Image Quality (MUSIQ) [76] evaluates image quality by analyzing contrast preservation across multiple scales. It focuses on the preservation of fine details and textures in the image after processing. Higher MUSIQ values signify better image quality.
PI. Perceptual Index (PI) combines two metrics, MAE and NIQE, to evaluate image perceptual quality. It emphasizes both aesthetic appeal and naturalness. A lower PI score reflects better perceptual quality.
无参考指标
- NIQE(自然图像质量评估器)[75]:基于自然场景的统计模型评估图像的自然性,无需参考图像。NIQE值越低,表示图像越接近真实自然场景。
- MUSIQ(多尺度图像质量评估)[76]:通过多尺度对比度分析衡量细节和纹理的保留程度。MUSIQ值越高,表示图像质量越好。
- PI(感知指数):结合MAE与NIQE,综合评价图像的美学吸引力和自然性。PI值越低,表示感知质量越优。
2 Comparision Results
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
3 Computational Complexity
Future
1. Effective Processing Paradigm
As previously discussed, current UHD image restoration methods predominantly follow two main processing paradigms: the downsampling-enhancement-upsampling and the encoder-decoder with stepwise up-downsampling. These paradigms aim to reduce computational costs by employing high magnification or stepwise downsampling; however, a significant drawback is the information loss which degrades overall restoration quality. Although some researchers have attempted to avoid information loss through wavelet transforms [50], the connection with the augmentation network has not been considered, and its effectiveness requires further improvement. Additionally, advancements in learning model-aware resampling methods [53] have shown potential for preserving feature consistency between UHD image inputs and their corresponding low-resolution inputs without requiring retraining of the enhancement network. Thus, exploring the intrinsic connection between resampling and enhancement networks is a potential research direction for developing more effective processing paradigms.
高效处理范式
如前所述,当前UHD图像修复方法主要遵循两种处理范式:下采样-增强-上采样和分阶段上下采样的编码器-解码器结构。这些范式通过采用高倍率或分步下采样来降低计算成本,但其显著缺点是信息丢失导致修复质量下降。尽管已有研究者尝试通过小波变换[50]避免信息丢失,但未考虑与增强网络的关联,且其效果仍需改进。此外,模型感知重采样方法[53]的最新进展展现了潜力,能够在无需重新训练增强网络的前提下,保持UHD图像输入与其对应低分辨率输入之间的特征一致性。因此,探索重采样操作与增强网络的内在关联是开发更高效处理范式的潜在研究方向。
2 Lightweight Network Design
Existing deep learning models face significant computational challenges, which limit their application in UHD image restoration tasks. Current restoration methods, ranging from traditional CNNs to advanced Transformer architectures, demand substantial computational power, making them heavily reliant on high-end GPUs. Although some networks have been proposed to reduce these complexities through MLPs and novel Mamba frameworks, they fall short of enabling direct implementation on mobile devices. Consequently, future research should prioritize the development of lightweight network architectures that balance performance and efficiency, enabling broader applicability across diverse devices.
轻量化网络设计
现有深度学习模型面临显著的计算挑战,限制了其在UHD图像修复任务中的应用。当前修复方法(从传统CNN到先进的Transformer架构)需要大量计算资源,严重依赖高端GPU。尽管已有网络通过MLP和新型Mamba框架尝试降低复杂度,但仍无法实现在移动设备上直接部署。因此,未来研究需优先开发轻量化网络架构,平衡性能与效率,从而扩大其跨多样化设备的适用性。
3 Developing Real-World Benchmark Datasets
As shown in Table 1, except for the UHD-LL dataset [39], most existing benchmark datasets are artificially synthesized. Although the data shortage in UHD image restoration has been temporarily alleviated, a significant gap remains between synthetic images and real-world degraded images. Models trained on synthetic images often perform well on synthetic test samples but exhibit poor performance on real-world images. For this reason, it is imperative to develop large-scale paired training datasets comprising real-world UHD images.
开发真实世界基准数据集
如表1所示,除UHD-LL数据集外,现有基准数据集多为人工合成。尽管UHD图像修复领域的数据短缺问题暂时缓解,但合成图像与真实世界退化图像之间仍存在显著差距。基于合成数据训练的模型在合成测试样本上表现良好,但对真实世界图像的修复效果较差。因此,亟需构建包含真实世界UHD图像的大规模成对训练数据集。
4. Using Image Priors
Current UHD image restoration networks predominantly focus on learning intrinsic features; however, the incorporation of image priors plays a pivotal role in enhancing image quality. For instance, UHDDIP [41] incorporates gradient and normal priors into its model design, significantly improving structural integrity and detail preservation of restored images. This demonstrates the potential of leveraging image priors to guide the restoration process. Further exploration of alternative image priors opens promising avenues for advancing UHD image restoration methodologies.
利用图像先验
当前的UHD图像修复网络主要关注学习图像固有特征,但图像先验的融合对提升修复质量至关重要。例如,UHDDIP在模型设计中融入了梯度和法向先验,显著提高了修复图像的结构完整性和细节保留能力。这证明了利用先验信息引导修复过程的潜力。进一步探索其他类型的图像先验,将为UHD图像修复方法的创新提供机遇。
5. Specialized Evaluation Metrics
Most image quality evaluation metrics, such as PSNR, SSIM, and LPIPS, are designed for standard-resolution images and reflect image quality to a certain extent. However, the high-resolution characteristics of UHD images place greater demands on detail restoration and subjective visual perception. For example, the perceived quality of an image may be affected by local detail in different areas, which may not be evident in low-resolution images. Therefore, it is crucial to develop evaluation indicators specifically tailored for UHD images. These metrics should accurately reflect the unique requirements of UHD images in terms of detail representation and overall sensory quality.
专用评估指标
大多数图像质量评估指标(如PSNR、SSIM和LPIPS)针对标准分辨率图像设计,仅能部分反映图像质量。然而,UHD图像的高分辨率特性对细节修复和主观视觉体验提出了更高要求。例如,图像感知质量可能受不同区域局部细节的影响,而此类现象在低分辨率图像中并不明显。因此,需开发专门针对UHD图像的评估指标,以准确反映其在细节还原与整体感官质量上的独特需求。
6. UHD Images with Multiple Degradations
Current UHD image restoration algorithms are usually designed to address a single type of degradation. However, in practical applications, UHD images are often affected by a combination of degradation factors, leaving the need for mixed degradation processing largely unresolved. It is worth noting that SimpleIR [48] is the first to propose an all-in-one restoration method for UHD images. While all-in-one image restoration methods have shown significant progress on low-resolution images using prompt learning and dynamic network technologies, these techniques have not yet been applied to UHD images.
多退化类型UHD图像修复
现有UHD图像修复算法通常针对单一退化类型设计。然而,实际应用中UHD图像常受多种退化因素共同影响,导致混合退化处理需求尚未解决。值得注意的是,SimpleIR首次提出了面向UHD图像的一体化修复方法。尽管通过提示学习和动态网络技术,一体化修复方法已在低分辨率图像中取得显著进展,但这些技术尚未在UHD图像中应用。