Abstract
Text-to-audio (TTA) systems based on latent diffusion and flow matching achieve high fidelity in the full-band scenarios, but their iterative sampling nature hinders practical deployment in real-world applications. Although few-step generation methods such as MeanFlow substantially reduce the number of function evaluations (NFEs), two fundamental challenges still remain in extremely low-NFE models. The first is unreliability: in extremely low NFE environments, quality of generated samples becomes highly sensitive to the initial noise. The second is quality inconsistency: increasing the number of ODE steps does not guarantee monotonic improvements in general sample quality. We propose a hybrid noise skimming framework to overcome both issues through better initial noise selection in few-step generation. The proposed framework combines two complementary components: a learnable Noise Booster amortizes the initial noise selection at the distribution level by refining random noise toward more favorable regions and Single-Step Search which improves the sample quality as instance-wise adaptation using only a single additional search step. The learnable Noise Booster restores the reliability and raises the quality floor by refining initial noise and in contrast to increasing ODE steps, enlarging the search budget yields monotonic improvement in sample quality. Experiments at 44.1 kHz show that our framework achieves SOTA performance with only a one-step generation.
Hybrid Noise Skimming
Audio samples
02A metal clack followed by a horse neighing as gusts of wind blow and birds chirp in the background
03A man talking as a door slams shut followed by a door creaking
04Rain is falling continuously
05A few gunshots followed by a man speaking
06A series of compressed air spraying as a motor hums in the background
07Metal clinking with faint clip-clops of a horse
08A motor humming as wood is being scrapped and sanded
09A motorbike engine running as multiple car horns honk and a car alarm rings in the distance
10Dog panting followed by footsteps
11A drone whirring followed by a crashing sound
12Machinery banging and hissing
13Several faint clicks are followed by a person sneezing and coughing
14Humming of a large engine with wind blowing
15A sewing machine operating as a machine motor hisses loudly in the background
16A race car engine is idling and then the car accelerates away quickly
17Continuous quacking and rustling
18A train horn honks with a train racing by
19Synthesized rumbling followed by an explosion
20A sewing machine is in use as well as knocking and bleeping noises
Results
Table 1Objective comparison with representative TTA models on the AudioCaps test set.
| Model | NFE | SR (kHz) |
FD ↓ | IS ↑ | KL ↓ | CLAP ↑ | PQ ↑ | CE ↑ | CU ↑ | PC ↑ | AQA ↑ |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Diffusion / Flow TTA | |||||||||||
| GenAU-L | 200 | 16 | 10.03 | 15.63 | 1.51 | 0.317 | 5.69 | 3.39 | 5.00 | 3.17 | 0.862 |
| TangoFlux | 50 | 44.1 | 16.32 | 15.13 | 1.37 | 0.320 | 5.75 | 3.56 | 5.02 | 3.23 | 0.910 |
| Resonate | 50 | 44.1 | 11.80 | 12.83 | 1.38 | 0.406 | 5.75 | 3.48 | 4.99 | 3.32 | 0.911 |
| Accelerated TTA | |||||||||||
| AudioLCM | 2 | 16 | 21.79 | 8.85 | 1.61 | 0.242 | 5.53 | 3.26 | 4.87 | 3.00 | 0.819 |
| ConsistencyTTA | 1 | 16 | 19.56 | 9.88 | 1.48 | 0.262 | 5.85 | 3.50 | 4.94 | 3.50 | 0.842 |
| MeanAudio-L | 1 | 16 | 11.02 | 12.84 | 1.34 | 0.317 | 5.52 | 3.36 | 4.91 | 3.03 | 0.882 |
| SoundCTM-UNet | 2 | 16 | 15.71 | 9.29 | 1.50 | 0.252 | 5.66 | 3.31 | 4.89 | 3.30 | 0.836 |
| SoundCTM-DiT | 1 | 44.1 | 22.13 | 10.92 | 1.69 | 0.346 | 5.65 | 3.23 | 4.87 | 3.06 | 0.779 |
| SAO small-ARC | 8 | 44.1 | 47.92 | 9.23 | 2.42 | 0.181 | 6.28 | 3.55 | 5.40 | 2.72 | 0.656 |
| Proposed | 1 | 44.1 | 14.35 | 10.86 | 1.41 | 0.391 | 5.84 | 3.49 | 4.96 | 3.52 | 0.865 |
| 2 | 13.44 | 11.70 | 1.41 | 0.396 | 5.88 | 3.54 | 5.01 | 3.41 | 0.872 | ||
| 4 | 13.08 | 12.06 | 1.38 | 0.407 | 5.98 | 3.59 | 5.10 | 3.40 | 0.878 | ||
| 8 | 12.74 | 12.76 | 1.35 | 0.427 | 6.13 | 3.66 | 5.24 | 3.39 | 0.885 | ||
| 16 | 12.79 | 12.96 | 1.34 | 0.438 | 6.22 | 3.72 | 5.34 | 3.40 | 0.886 | ||
| 32 | 12.35 | 13.25 | 1.30 | 0.445 | 6.29 | 3.76 | 5.41 | 3.38 | 0.890 | ||
| 64 | 12.29 | 13.21 | 1.31 | 0.450 | 6.35 | 3.79 | 5.46 | 3.38 | 0.889 | ||
Table 2Human evaluation results. OVL = overall quality, REL = relevance to the caption.
| Model | NFE | OVL ↑ | REL ↑ |
|---|---|---|---|
| TangoFlux | 50 | 2.87 ± 0.12 | 3.73 ± 0.11 |
| SAO small-ARC | 8 | 2.99 ± 0.12 | 2.55 ± 0.13 |
| MeanAudio-L | 1 | 2.66 ± 0.11 | 3.04 ± 0.13 |
| SoundCTM-DiT | 1 | 2.50 ± 0.11 | 2.51 ± 0.12 |
| Proposed | 1 | 2.76 ± 0.11 | 3.21 ± 0.13 |
| 16 | 3.00 ± 0.11 | 3.42 ± 0.12 |