Skip to content
/ NeAF Public

[AAAI 2025] Representing Sounds as Neural Amplitude Fields: A Benchmark of Coordinate-MLPs and A Fourier Kolmogorov-Arnold Framework

License

Notifications You must be signed in to change notification settings

lif314/NeAF

Folders and files

NameName
Last commit message
Last commit date

Latest commit

efea75b · Dec 10, 2024

History

25 Commits
Aug 13, 2024
Aug 12, 2024
Nov 6, 2024
Jun 27, 2024
May 31, 2024
Dec 10, 2024
Jun 27, 2024
Aug 12, 2024
Jul 10, 2024
Aug 12, 2024
Jul 10, 2024
Aug 12, 2024

Repository files navigation

Representing Sounds as Neural Amplitude Fields: A Benchmark of Coordinate-MLPs and A Fourier Kolmogorov-Arnold Framework
[AAAI 2025]

Linfei Li · Lin Zhang* · Zhong Wang · Fengyi Zhang · Zelin Li · Ying Shen

NeAF teaser

What is Neural Amplitude Fields?

NeAF

Environment Configuration

Benchmark of Coordinate-MLPs in Audio Signal Representations

Overview

benchmark

Run

  • Testing on Bach, Counting, and Blues.
bash scripts/benchmark_MLPs_demo.sh
  • Testing on CSTR VCTK dataset.
bash scripts/benchmark_MLPs_vctk.sh
  • Testing on GTZAN dataset.
bash scripts/benchmark_MLPs_gtzan.sh

Fourier-ASR: A Fourier Kolmogorov-Arnold Framework

Overview

fourier

Run

  • Testing on Bach, Counting, and Blues.
bash scripts/benchmark_KANs_demo.sh
  • Testing on CSTR VCTK dataset.
bash scripts/benchmark_KANs_vctk.sh
  • Testing on GTZAN dataset.
bash scripts/benchmark_KANs_gtzan.sh

Ablation Experiments

Positional encoding is parameter-sensitive

  • RFF positional encoding is sensitive to the dimension parameter L .
bash scripts/benchmark_FFN_L.sh
  • RFF positional encoding is sensitive to the variance parameter σ .
bash scripts/benchmark_FFN_sigma.sh
  • NeFF positional encoding is sensitive to the dimension parameter L .
bash scripts/benchmark_NeRF_L.sh

Activation functions are parameter-sensitive

  • Gaussian-type activation functions are sensitive to the variance factor a .
bash scripts/benchmark_gaussian.sh
  • Sine-type activation functions are sensitive to the frequency factor ω .
# Sine
bash scripts/benchmark_siren.sh

# Incode-Sine
bash scripts/benchmark_incode-sine.sh

Periodic activation functions are sensitive to initialization schemes

bash scripts/benchmark_sensitive_init.sh

Fourier-ASR is parameter-insensitive

When model capacity is limited, larger Ω in the input layer improve the performance.

bash scripts/benchmark_Fourier_omega.sh

About

[AAAI 2025] Representing Sounds as Neural Amplitude Fields: A Benchmark of Coordinate-MLPs and A Fourier Kolmogorov-Arnold Framework

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published