Hiding Images in Diffusion Models by Editing Learned Score Functions通过编辑学习的分数函数在扩散模型中隐藏图像

Abstract  抽象

Hiding data using neural networks (i.e., neural steganography) has achieved remarkable success across both discriminative classifiers and generative adversarial networks. However, the potential of data hiding in diffusion models remains relatively unexplored. Current methods exhibit limitations in achieving high extraction accuracy, model fidelity, and hiding efficiency due primarily to the entanglement of the hiding and extraction processes with multiple denoising diffusion steps. To address these, we describe a simple yet effective approach that embeds images at specific timesteps in the reverse diffusion process by editing the learned score functions. Additionally, we introduce a parameter-efficient fine-tuning method that combines gradient-based parameter selection with low-rank adaptation to enhance model fidelity and hiding efficiency. Comprehensive experiments demonstrate that our method extracts high-quality images at human-indistinguishable levels, replicates the original model behaviors at both sample and population levels, and embeds images orders of magnitude faster than prior methods. Besides, our method naturally supports multi-recipient scenarios through independent extraction channels.
使用神经网络（即神经隐写术）隐藏数据在判别分类器和生成对抗网络中都取得了显着的成功。然而，数据隐藏在扩散模型中的潜力仍然相对未被探索。目前的方法在实现高提取精度、模型保真度和隐藏效率方面表现出局限性，这主要是由于隐藏和提取过程与多个去噪扩散步骤纠缠在一起。为了解决这些问题，我们描述了一种简单而有效的方法，通过编辑学习的分数函数，在反向扩散过程中的特定时间步长嵌入图像。此外，我们还引入了一种参数高效的微调方法，该方法将基于梯度的参数选择与低秩自适应相结合，以提高模型保真度和隐藏效率。综合实验表明，我们的方法提取了人类无法区分的高质量图像，在样本和总体水平上复制了原始模型行为，并且嵌入图像的速度比以前的方法快几个数量级。此外，我们的方法通过独立的提取通道自然支持多接收者场景。

The evolution of data hiding has paralleled advancements in digital media, progressing from traditional bitstream manipulations [8] to deep learning paradigms [3, 1, 60]. A prevailing scheme of hiding data with neural networks adopts an autoencoder architecture, where an encoding network embeds a secret message into some cover media, and a decoding network is responsible for retrieving the message. Despite demonstrated feasibility, its practical deployment faces three fundamental constraints. First, the requirement for secure transmission of the decoding network creates logistical vulnerabilities [11, 10]. Second, state-of-the-art steganalysis tools [4, 57, 33, 18, 52, 19, 12] can detect the embedded message from the cover media (commonly in the forms of digital images and videos) with high accuracy, leading to compromised secrecy. Last, multi-recipient scenarios necessitate complex key design and management [3, 65, 53, 25, 55].
数据隐藏的演变与数字媒体的进步并行，从传统的比特流作 [8] 发展到深度学习范式 [3,1,60]。  一种流行的用神经网络隐藏数据的方案采用自动编码器架构，其中编码网络将秘密消息嵌入到一些覆盖介质中，解码网络负责检索消息。尽管已证明可行性，但其实际部署面临三个基本限制。首先，对解码网络安全传输的要求造成了物流漏洞 [11,10]。  其次，最先进的隐写分析工具 [4、57、33、18、52、19、12] 可以高精度地检测来自封面介质（通常以数字图像和视频的形式）的嵌入信息，从而导致保密性受损。最后，多接收者场景需要复杂的密钥设计和管理 [3,65,53,25,55]。 

A paradigm shift emerges with hiding data in neural networks [44, 32, 51, 50, 5], where secret data is embedded directly into model parameters. This approach evades conventional steganalysis methods, which are typically designed to analyze stego multimedia rather than neural network weights. Initially, such hiding methods showed success in discriminative models [44, 32, 51, 50]. However, recent attention has shifted towards generative models due to their increased utility and wider adoption. Moreover, generative models can directly produce secret data without the need for a separate decoding network, thus inherently resolving transmission security issues. For instance, Chen et al. [5] proposed to embed images at specific locations of the learned distributions by deep generative models, which has proven effective with SinGANs [41], a variant of generative adversarial networks (GANs) [14].
在神经网络  中隐藏数据出现了范式转变[44,32,51,50,5]，其中秘密数据直接嵌入到模型参数中。这种方法避免了传统的隐写分析方法，这些方法通常设计用于分析隐存多媒体而不是神经网络权重。最初，这种隐藏方法在判别模型 中显示出成功[44,32,51,50]。  然而，由于生成模型的实用性增加和更广泛的采用，最近的注意力已转向生成模型。而且，生成模型可以直接生成秘密数据，无需单独的解码网络，从而从本质上解决了传输安全问题。例如，Chen 等  [5] 提出通过深度生成模型将图像嵌入到学习分布的特定位置，这已被证明对生成对抗网络（GAN） 的变体 SinGAN [41] 有效[14]。

The advent of diffusion models [47, 21] introduces new opportunities for this generative probabilistic hiding approach. Despite slightly different purposes (backdooring or watermarking), existing approaches [7, 36, 6] face critical bottlenecks (see Table 1). First, they are inadequate in hiding complex natural images of rich structures and textures, with reconstruction peak signal-to-noise ratio (PSNR) ≤25 dB. Second, they compromise model fidelity, with Fréchet inception distance (FID) [20] degradation exceeding 100%, raising detectability concerns. Last, they require full model retraining or fine-tuning (≥10 GPU hours) to learn parallel secret diffusion processes.
扩散模型 [47,21] 的出现为这种生成概率隐藏方法带来了新的机会。尽管目的略有不同（后门或水印），但现有方法 [7,36,6] 仍面临严重瓶颈（见表 1）。首先，它们不足以隐藏具有丰富结构和纹理的复杂自然图像，重建峰值信噪比（PSNR）dB。 ≤25 其次，它们损害了模型保真度，Fréchet 起始距离（FID） [20] 退化超过 100% ，引发了可检测性问题。最后，它们需要完整的模型重新训练或微调（ ≥10 GPU 小时）来学习并行的秘密扩散过程。

In this paper, we describe a simple yet effective approach to hiding images in diffusion models. We identify that editing the learned score functions (by inserting secret key-to-image mappings) at specific timesteps enables precise image embedding without disrupting the original chain of the reverse diffusion process. The stego diffusion model is publicly shared11Only the secret key of few bits is shared between the sender and recipient via a secure subliminal channel.
发送者和接收者之间仅通过安全的潜意识通道共享几位的密钥。
在本文中，我们描述了一种在扩散模型中隐藏图像的简单而有效的方法。我们发现，在特定时间步长编辑学习到的分数函数（通过插入密钥到图像映射）可以在不破坏反向扩散过程的原始链的情况下实现精确的图像嵌入。stego 扩散模型是公开共享的 1, replicating the original model behaviors in synthesizing in-distribution high-quality images. The secret image extraction is achieved in a single step via key-guided and timestep-conditioned “denoising.” To further improve model fidelity and hiding efficiency, we introduce a hybrid parameter-efficient fine-tuning (PEFT) method that combines gradient-based parameter selection [15] and low-rank adaptation (LoRA) [23], which reduces trainable parameters by 86.3% compared to full fine-tuning. Comprehensive experiments demonstrate the effectiveness of our method across four critical dimensions: 1) extraction accuracy, achieving 52.90 dB in PSNR for 32×32 images and 39.33 dB for 256×256 images; 2) model fidelity, maintaining nearly original FID scores (4.77 versus 4.79 on CIFAR10 [28]) with minimal sample-level distortion; 3) hiding efficiency, reducing embedding time to 0.04 and 0.18 GPU hours for 32×32 and 256×256 images, respectively; and 4) scalability, simultaneously embedding four images for different recipients with independent extraction keys.
，复制原始模型在合成分布式高质量图像时的行为。秘密图像提取是通过按键引导和时间步长条件的“去噪”一步实现的。为了进一步提高模型保真度和隐藏效率，我们引入了一种混合参数高效微调（PEFT）方法，该方法结合了基于梯度的参数选择 [15] 和低秩自适应（LoRA） [23]，与完全微调 86.3% 相比，该方法减少了可训练参数。综合实验证明了该方法在四个关键维度上的有效性：1）提取精度，图像的 PSNR 达到 52.90 dB， 39.33 图像达到 256×256 32×32 dB;2）模型保真度，保持接近原始的 FID 分数（与 CIFAR10 [28] 4.79 相比）， 4.77 样本水平失真最小;3）隐藏效率，分别减少图像的嵌入时间 0.04 和 0.18 GPU 小时数; 32×32 256×256 4）可扩展性，同时为不同的接收者嵌入四个图像，具有独立的提取键。

Neural Steganography. We focus primarily on hiding data in neural networks [44, 32, 51, 50, 5], which involves embedding secret data within neural network parameters or structures, ensuring covert communication without compromising model performance. Representative neural steganography strategies encompass replacing least significant bits of model parameters [44, 32], substituting redundant parameters [32, 51], mapping parameter values [44, 32, 50] or signs [44, 32] directly to secret messages, memorizing secret-labeled synthetic data [44], and hiding images in deep probabilistic models [5]. Despite their effectiveness, diffusion models have not been extensively considered in neural steganography.
神经隐写术。 我们主要关注在神经网络 中隐藏数据 [44,32,51,50,5]，这涉及在神经网络参数或结构中嵌入秘密数据，确保隐蔽通信而不影响模型性能。代表性的神经隐写术策略包括替换模型参数 的最低有效位 [44,32]，替换冗余参数 [32,51]，映射参数值 [44,32,50] 或符号 [44,32] 直接发送到秘密消息，记住秘密标记的合成数据 [44]，并将图像隐藏在深度概率模型 中[5]。 尽管扩散模型有效，但扩散模型尚未在神经隐写术中得到广泛考虑。

Diffusion Models represent a significant advancement in generative modeling, offering high-quality data synthesis through multi-step denoising diffusion processes. Inspired by non-equilibrium thermodynamics [43] and score matching in empirical Bayes [38, 24], diffusion models define a forward diffusion process, which progressively adds noise to data until it becomes random, and a reverse (generative) process, which learns to iteratively remove this noise, reconstructing the original data. In contrast to GANs, diffusion models optimize a likelihood-based objective [21, 46], typically resulting in more stable training and better mode coverage. Architecturally, diffusion models predominantly employ U-Nets [40] and Transformers [48], operating in either pixel [21, 47, 27] or latent [39, 35, 37] space.
扩散模型代表了生成建模的重大进步，通过多步骤去噪扩散过程提供高质量的数据合成。受非平衡热力学 [43] 和经验贝叶 斯[38,24] 中的分数匹配的启发，扩散模型定义了一种正向扩散过程，该过程逐渐向数据添加噪声，直到数据变得随机，以及一个反向（生成）过程，该过程学习迭代地消除这些噪声，重建原始数据。与 GAN 相比，扩散模型优化了基于似然的目标 [21,46]，通常可以带来更稳定的训练和更好的模式覆盖。在架构上，扩散模型主要采用 U-Nets [40] 和 Transformers [48]，在像素 [21,47,27] 或潜在 [39,35,37] 空间中运行。

Existing methods for hiding data in diffusion models typically involve concurrent training of secret reverse diffusion processes [7, 36, 6]. As a result, these approaches suffer from limited extraction accuracy, noticeable degradation in model fidelity, and high computational demands, especially when applied to complex natural images. Diffusion model editing techniques [13, 29, 54, 56], originally designed to alter or remove learned concepts and rules, may offer a potential remedy. Some have been adapted to embed invisible yet recoverable signals for watermarking purposes [11, 10]. Nevertheless, these editing methods do not fulfill the precise image reconstruction requirements we are looking for.
在扩散模型中隐藏数据的现有方法通常涉及秘密反向扩散过程的并发训练 [7,36,6]。  因此，这些方法存在提取精度有限、模型保真度明显下降和计算要求高的问题，尤其是在应用于复杂的自然图像时。扩散模型编辑技术 [13,29,54,56] 最初旨在改变或删除学习到的概念和规则，可能提供一种潜在的补救措施。有些已被改编为嵌入不可见但可恢复的信号以用于水印目的 [11,10]。  然而，这些编辑方法并不能满足我们正在寻找的精确图像重建要求。

Parameter-Efficient Fine-Tuning. PEFT methods can be broadly classified into two categories: selective and reparameterized fine-tuning. Selective methods strategically identify only a subset of parameters for adjustment based on simple heuristics [59] or parameter gradients [17, 63]. In contrast, reparameterized fine-tuning introduces additional trainable parameters to a pre-trained frozen backbone. Notable examples include prompt-tuning [30], adapters [22], and LoRA [23] and its variants [34, 61, 26, 16]. This paper presents a hybrid PEFT method for hiding images in pixel-space diffusion models, effectively combining the strengths of both selective and reparameterized methods.
参数高效的微调。 PEFT 方法大致可分为 两类：选择性和重新参数化微调。选择性方法仅根据简单的启发式 [59] 或参数梯度 [17,63] 战略性地识别出要调整的参数子集。相比之下，重新参数化微调为预训练的冻结主干引入了额外的可训练参数。值得注意的例子包括提示调整 [30]、适配器 [22] 和 LoRA [23] 及其变体 [34,61,26,16]。  本文提出了一种在像素空间扩散模型中隐藏图像的混合 PEFT 方法，有效地结合了选择性和重新参数化方法的优势。

In this section, we first provide the necessary preliminaries of diffusion models, which serve as the basis of the proposed method. Subsequently, we describe the typical image hiding scenario, and present in detail our diffusion-based steganography method (see Fig. 1).
在本节中，我们首先提供扩散模型的必要初步准备，作为所提出方法的基础。随后，我们描述了典型的图像隐藏场景，并详细介绍了我们基于扩散的隐写术方法（见图。1）。

Diffusion models, notably denoising diffusion probabilistic models (DDPMs) [21], have recently emerged as state-of-the-art deep generative models capable of synthesizing high-quality images through a defined diffusion process.
扩散模型，特别是去噪扩散概率模型（DDPM）[21]，最近出现了最先进的深度生成模型，能够通过定义的扩散过程合成高质量的图像。

Forward Diffusion Process. Starting from a clean image 𝒙0∼q⁢(𝒙0), Gaussian noise ϵ∼𝒩⁢(𝟎,𝐈) is iteratively added over T timesteps following a variance schedule {β1,…,βT}, described as
正向扩散过程。 从干净的图像 𝒙0∼q⁢(𝒙0) 开始，高斯噪声 ϵ∼𝒩⁢(𝟎,𝐈) 按照方差计划在时间步长上 T 迭代添加 {β1,…,βT} ，描述为

where each step is governed by
其中每个步骤由

Eventually, this process transforms the clean image 𝒙0 into nearly pure Gaussian noise 𝒙T.
最终，这个过程将干净的图像 𝒙0 转换为几乎纯的高斯噪声 𝒙T 。

Reverse Diffusion Process. A Markov chain with parameters 𝜽 is learned to iteratively recover the clean image from noise, modeled as
反向扩散过程。 学习具有参数 𝜽 的马尔可夫链，以迭代地从噪声中恢复干净的图像，建模为

Model parameters 𝜽 are optimized by minimizing a denoising objective [21]:
通过最小化去噪物镜 来优化模型参数 𝜽 [21]：

where ϵ𝜽⁢(⋅,⋅) is the learned noise estimation function (also known as the score function), implemented typically using a U-Net, and α¯t=∏i=1t(1−βi) controls the noise scale at timestep t. Eq. (4) can also be viewed as a variational bound of the log-likelihood of the observed data 𝒙0.
其中 ϵ𝜽⁢(⋅,⋅) 是学习到的噪声估计函数（也称为分数函数），通常使用 U-Net 实现，并在 α¯t=∏i=1t(1−βi) 时间步长 t 控制噪声尺度。式 （4）也可以看作是观测数据 𝒙0 的对数似然的变分界限。

Inference iteratively employs the trained and timestep-conditioned score function with optimal parameters 𝜽¯ to transform Gaussian noise into a clean image:
推理迭代地使用具有最佳参数 𝜽¯ 的训练和时间步长条件分数函数将高斯噪声转换为干净的图像：

for t∈{T,…,1}, where αt=1−βt, 𝒛∼𝒩⁢(𝟎,𝐈), and σt controls stochasticity at timestep t, with σt2=(1−α¯t−1)⁢βt/(1−α¯t). σ1 is usually set to zero to remove randomness at the final timestep. For simplicity, we slightly abuse the notation 𝒙t to denote both the noisy image and its denoised counterpart at timestep t.
对于 t∈{T,…,1} ，其中 αt=1−βt ， 𝒛∼𝒩⁢(𝟎,𝐈) ， 和 σt 控制时间步长 t 的随机性 ，使用 σt2=(1−α¯t−1)⁢βt/(1−α¯t) 。 σ1 通常设置为零以消除最终时间步长的随机性。为简单起见，我们稍微滥用了符号 𝒙t 来表示噪声图像及其在时间步长 t 处的去噪图像。

We consider a single-image hiding scenario following the formulation of the classical prisoner’s problem by Simmons [42], which involves three distinct parties:
我们考虑了 Simmons[42] 对经典囚徒问题的表述之后的单图像隐藏场景，该场景涉及三个不同的当事人：

• • 

  A sender, who embeds a secret image 𝒙s in a diffusion model with a secret key 𝒦s, and subsequently shares the stego diffusion model publicly;

  
  • 发送者，使用密钥将秘密图像 𝒙s 嵌入扩散模型中 𝒦s ，随后公开共享 stego 扩散模型;
• • 

  A recipient, who utilizes the privately shared secret key 𝒦s to extract the secret image 𝒙s from the publicly transmitted stego diffusion model;

  
  • 接收者，利用私有共享的密钥 𝒦s 从公开传输的 stego 扩散模型中提取秘密图像 𝒙s ;
• • 

  An inspector, who examines the publicly available diffusion models to verify that they retain normal generation functionality, and do not exhibit any suspicious behavior.

  
  • 检查员，检查公开可用的扩散模型，以验证它们是否保留正常的生成功能，并且没有表现出任何可疑行为。

The key objectives in this scenario are twofold: ensuring high accuracy in the extraction of the secret image by the authorized recipient, and preserving the sample-level and population-level fidelity of the diffusion model in image generation so that it remains undetected during scrutiny by the inspector.
这种情况下的主要目标是双重的：确保授权接收者提取秘密图像的高精度，并在图像生成中保持扩散模型的样本水平和总体水平保真度，以便在检查员的审查期间不被发现。

Basic Idea. Existing diffusion-based techniques [7, 6, 36] often embed secret images by modifying the entire Markov chain of the reverse diffusion process, resulting in significant challenges regarding extraction accuracy, model fidelity, and hiding efficiency. To resolve these, we propose a simple yet effective hiding method that operates at a single timestep. This process uses a secret key 𝒦s={ks,ts}, where ks is the seed to deterministically generate Gaussian noise 𝒛s, and ts is the selected timestep for image hiding. The secret key 𝒦s is privately communicated with the recipient, enabling precise, secure, and rapid one-step reconstruction of the secret image 𝒙s as
基本思想。 现有的基于扩散的技术 [7,6,36] 通常通过修改反向扩散过程的整个马尔可夫链来嵌入秘密图像，这在提取精度、模型保真度和隐藏效率方面带来了重大挑战。为了解决这些问题，我们提出了一种简单而有效的隐藏方法，该方法在单个时间步长内运行。这个过程使用一个密钥 𝒦s={ks,ts} ，其中 ks 是确定性地生成高斯噪声 𝒛s 的种子， ts 是图像隐藏的选定时间步长。密钥 𝒦s 与接收者私下通信，从而能够精确、安全、快速地一步重建秘密图像 𝒙s ，如

where 𝜽~ denotes the optimal parameters after editing.
其中 𝜽~ 表示编辑后的最佳参数。

Loss Function. To ensure minimal distortion in reconstructing the secret image 𝒙s, we define the extraction accuracy loss as
损失函数。 为了确保在重建秘密图像 𝒙s 时将失真降至最低，我们将提取精度损失定义为

By optimizing the learned score function ϵ𝜽⁢(𝒛s,ts) using the accuracy loss, the secret image is embedded within the diffusion model. Meanwhile, it is crucial that the stego diffusion model closely replicates the generative functionality of the original model. Therefore, we introduce an additional model fidelity loss to regulate the edited score function:
通过使用精度损失优化学习到的分数函数 ϵ𝜽⁢(𝒛s,ts) ，将秘密图像嵌入到扩散模型中。同时，stego 扩散模型紧密复制原始模型的生成功能至关重要。因此，我们引入了一个额外的模型保真度损失来调节编辑的分数函数：

where ϵ𝜽¯⁢(⋅,⋅) denotes the original score function and 𝒙t=α¯t⁢𝒙0+1−α¯t⁢ϵ. In contrast to the loss ℓa⁢(𝜽) in Eq. (7), which penalizes only the reconstruction error of the secret image 𝒙s at the specific timestep ts, the model fidelity loss ℓf⁢(𝜽) computes the expectation of the residual between noise estimated by the original and edited score functions. This expectation is taken across uniformly sampled timesteps t∼Uniform⁢({1,…,T}), clean images 𝒙0∼q⁢(𝒙0), and Gaussian noise ϵ∼𝒩⁢(𝟎,𝐈) to ensure complete adherence to the original diffusion process. Moreover, the loss ℓf⁢(𝜽) eliminates the need to retrain the original diffusion model or access the original training dataset, requirements typically imposed by existing methods [7, 6, 36]. Lastly, the overall loss function is a linear weighted summation of the two terms:
其中表示原始分数函数， 𝒙t=α¯t⁢𝒙0+1−α¯t⁢ϵ 而 ϵ𝜽¯⁢(⋅,⋅) 。与式 （7）中的损失相比，公式 （7）中的损失 ℓa⁢(𝜽) 仅惩罚秘密图像 𝒙s 在特定时间步长 ts 下的重建误差，模型保真度损失 ℓf⁢(𝜽) 计算了原始分数函数估计的噪声和编辑后的分数函数估计的噪声之间的残差的期望值。这种期望是在均匀采样的时间步长 t∼Uniform⁢({1,…,T}) 、干净的图像 𝒙0∼q⁢(𝒙0) 和高斯噪声 ϵ∼𝒩⁢(𝟎,𝐈) 中实现的，以确保完全遵守原始扩散过程。此外，这种损失 ℓf⁢(𝜽) 消除了重新训练原始扩散模型或访问原始训练数据集的需要，这是现有方法 通常提出的要求[7,6,36]。  最后，总体损失函数是两项的线性加权求和：

where λ is a trade-off parameter.
其中 λ 是权衡参数。

Multi-image Hiding. We extend the single-image hiding method to scenarios involving multiple recipients, each with a unique secret key. For a set of secret images 𝒳s={𝒙s(1),…,𝒙s(M)}, where 𝒙s(i) denotes the secret image intended for the i-th recipient, and M is the total number of secret images, equal to the number of recipients. Each secret image 𝒙s(i) is paired with a secret key 𝒦s(i)={ks(i),ts(i)}, forming the key collection 𝒦s={𝒦s(1),…,𝒦s(M)} distributed among M different recipients. The loss function in Eq. (9) is then modified as
多图像隐藏。 我们将单镜像隐藏方法扩展到涉及多个收件人的场景，每个收件人都有一个唯一的密钥。对于一组秘密图像 𝒳s={𝒙s(1),…,𝒙s(M)} ，其中 𝒙s(i) 表示用于第 i -个收件人的秘密图像，并且 M 是秘密图像的总数，等于收件人的数量。每个秘密映像 𝒙s(i) 都与一个密钥配对 𝒦s(i)={ks(i),ts(i)} ，形成分布在不同收件人之间的 M 密钥集合 𝒦s={𝒦s(1),…,𝒦s(M)} 。然后将式 （9）中的损失函数修改为：

where 𝒛s(i) is the Gaussian noise generated using the seed ks(i). After optimizing the pre-trained diffusion model using ℓ⁢(𝜽;𝒦s,𝒳s), the i-th recipient can extract her/his secret image 𝒙s(i) using the designated secret key 𝒦s(i). Without access to additional secret keys, each recipient is prevented from retrieving secret images intended for others.
其中 𝒛s(i) 是使用种子产生的高斯噪声 ks(i) 。使用 ℓ⁢(𝜽;𝒦s,𝒳s) 优化预训练扩散模型后， i 第 - 个接收者可以使用指定的密钥提取她/他的秘密图像 𝒙s(i) 𝒦s(i) 。如果无法访问其他密钥，则每个收件人都无法检索供其他人使用的秘密图像。

To improve both model fidelity and hiding efficiency, we introduce a hybrid PEFT technique that combines the advantages of selective and reparameterized PEFT strategies. Our method consists of three steps: 1) computing parameter-level sensitivity with respect to the editing loss function (in Eq. (9)), 2) identifying the most sensitive layers, and 3) applying reparameterized fine-tuning to these selected layers.
为了提高模型保真度和隐藏效率，我们引入了一种混合 PEFT 技术，该技术结合了选择性和重新参数化 PEFT 策略的优点。我们的方法包括三个步骤：1）计算相对于编辑损失函数的参数级灵敏度（如式（9）中），2）识别最敏感的层，以及 3）对这些选定的层应用重新参数化的微调。

Parameter Sensitivity Calculation. We quantify the importance of each parameter through gradient-based sensitivity analysis [17]. For parameter θi in the score function ϵ𝜽⁢(𝒛s,ts), the task-specific sensitivity gi is approximated by accumulating squared gradients over N iterations:
参数灵敏度计算。 我们通过基于梯度的敏感性分析 [17] 来量化每个参数的重要性。对于分数函数中的参数 θi ，特定于任务的敏感度 gi 是通过在迭代中 N 累积平方梯度来近似 ϵ𝜽⁢(𝒛s,ts) 的：

where i and j are the parameter and iteration indices, respectively. This metric identifies parameters most responsive to the hiding objective.
其中 i 和 分别 j 是参数和迭代索引。此指标标识对隐藏目标最灵敏的参数。

Sensitive Layer Selection. We extend parameter-level to layer-level sensitivity through spatial aggregation. Specifically, we begin by binarizing the sensitivity score of each parameter gi:
敏感层选择。 我们通过空间聚合将参数级的灵敏度扩展到层级。具体来说，我们首先对每个参数的灵敏度分数进行二值化 gi ：

where τ is a predefined threshold that controls the sensitive parameter sparsity. Next, we rank all network layers by their counts of sensitive parameters (i.e., those with bi=1), and select top-η layers for subsequent structured and computationally efficient fine-tuning. Critically, we avoid summing raw sensitivity scores within layers, as this could bias selection towards layers with disproportionately many parameters—even if those parameters are less impactful (known as the law of triviality). By prioritizing the number of sensitive parameters over their cumulative sensitivity, we ensure a balanced layer-wise assessment.
其中 τ 是控制敏感参数稀疏性的预定义阈值。接下来，我们根据敏感参数的计数对所有网络层（ 即 bi=1 ）进行排名，并选择顶层 η 进行后续结构化和计算效率的微调。至关重要的是，我们避免对层内的原始灵敏度分数求和，因为这可能会使选择偏向参数不成比例的层——即使这些参数的影响较小（称为琐碎定律）。通过优先考虑敏感参数的数量而不是其累积灵敏度，我们确保了平衡的层级评估。

LoRA-based Fine-tuning. For each selected sensitive layer, we implement a variant of LoRA for PEFT. Specifically, for linear layers, we apply the standard LoRA [23] by injecting trainable low-rank matrices Δ⁢𝐖=𝐀𝐁, where 𝐀∈ℝm×r and 𝐁∈ℝr×n with rank r≪min⁡{m,n}. For convolutional layers, we first reshape the 4D convolution filters into 2D matrices by flattening the input channel and filter dimensions, and then apply LoRA accordingly. To improve fine-tuning performance and accelerate convergence, we incorporate the rank stabilization trick proposed in [26], adjusting the scaling factor to be proportional to O⁢(1/r). Meanwhile, we apply the learning rate decoupling trick introduced by [16], setting different learning rates for the matrices 𝐀 and 𝐁, respectively.
基于 LoRA 的微调。 对于每个选定的敏感层，我们为 PEFT 实现了 LoRA 的变体。具体来说，对于线性层，我们通过注入可训练的低秩矩阵 Δ⁢𝐖=𝐀𝐁 来应用标准 LoRA [23]，其中 𝐀∈ℝm×r 和 𝐁∈ℝr×n 和 。 r≪min⁡{m,n} 对于卷积层，我们首先通过展平输入通道和滤波器维度将 4D 卷积滤波器重塑为 2D 矩阵，然后相应地应用 LoRA。为了提高微调性能并加速收敛，我们结合了[26] 中 提出的秩稳定技巧，将缩放因子调整为与 O⁢(1/r) 成正比。同时，我们应用了[16] 引入 的学习率解耦技巧，分别为矩阵 𝐀 和 𝐁 设置了不同的学习率。