Generating coherence-constrained multisensor signals using balanced mixing and spectrally smooth filters

Project Overview

This project provides a Python function for generating multi-channel audio signals with predefined spatial coherence.

Abstract

The spatial properties of a noise field can be described by a spatial coherence function. Synthetic multichannel noise signals exhibiting a specific spatial coherence can be generated by properly mixing a set of uncorrelated, possibly nonstationary, signals. The mixing matrix can be obtained by decomposing the spatial coherence matrix. As proposed in a widely used method [1], the decomposing can be performed using a Cholesky or an eigenvalue decomposition. The limitations of these two decompositions are discussed and addressed in [2]. First, specific properties of the mixing matrix are analyzed, namely the spectral smoothness and the mix balance. The first quantifies the mixing matrix-filters variation across frequency, and the second quantifies how much each input contributes to each output. Secondly, three methods based on the unitary Procrustes solution are proposed to enhance the spectral smoothness, the mix balance, and both properties jointly. A performance evaluation confirms the improvements of the mixing matrix in terms of objective measures. Further, the evaluation results show that the error between the target and the generated coherence is lowered by increasing the spectral smoothness of the mixing matrix.

Method

This method can generate multi-channel audio signals that exhibit a predefined spatial coherence. It also enhances specific properties of the mixing matrix obtained with the baseline approach (Cholesky or eigenvalue decompositions). These properties are:

• Spectral Variation: variation of the mixing matrix's filter response in the frequency domain. The mixing matrix is considered smooth if it slowly varies over frequency. For this reason, we denote a low Spectral Variation with 'Spectral Smoothness'. This variation can be quantified by the squared Frobenius norm of the difference between two frequency-bands adjacent mixing matrices.
• Coherence Error: accuracy of the generated spatial coherence at frequency bands that are not resolved by the chosen DFT length. A mixing matrix yields a low coherence error when the squared Frobenius norm of the difference between the target coherence matrix and the generated coherence matrix is low.
• Mix Balance: balance of the mixing, i.e., the number of input signals that contribute to each output signal. A balanced mixing matrix contains similar contributions from each input signal to each output signal. The balance can be quantified by the l1-norm of the mixing matrix.

The coherence error is inherently decreased by increasing the spectral smoothness. In addition, a smooth mixing matrix yields shorter impulse responses compared to non-smooth counterparts. The benefits of improving the smoothness and the balance are also perceptually evident. A smooth mix leads to less distortions in the output signals (especially during transients). A balanced mix is such that the input signals are filtered and summed similarly among all the channels, leading to an increased perceptual plausibility.

Settings

Different parameters can be chosen for the signal generation (e.g., sampling frequency, DFT length, spatial coherence model). The microphone positions are arranged in a matrix (M x 3), i.e., number of channels x coordinates xyz. The generator works for any arbitrary 3-D microphone constellation.

Supported spatial coherence models:

1. 3-D spherically isotropic diffuse sound field: spherical
2. 2-D cylindrically isotropic diffuse sound field: cylindrical
3. Corcos model (turbulent aerodynamic noise, e.g., atmospheric-wind noise): corcos

The first two models depend solely on the microphone positions. The third model has two additional parameters: speed and direction of the turbulent airflow. When selecting this model, it is suggested to use an inter-microphone distance of less than 2 cm to appreciate differences with respect to a noise field with uncorrelated signals.

Supported factorization methods to decompose the target coherence matrix and obtain the mixing matrix:

1. Cholesky decomposition: chd
2. Eigenvalue decomposition: evd

The chd yields a smooth but unbalanced mix. The evd yields a more balanced but non-smooth mix.

Three methods are available to enhance the properties of the mixing matrix:

1. Enhance smoothness: smooth
2. Enhance balance: balanced
3. Enhance balance and smoothness: balanced+smooth
4. No post-processing: standard

The smooth method enhances the smoothness (i.e., decreases the spectral variation) and lowers the coherence error while leaving almost unaltered the mix balance. The balanced method maximizes the balance but significantly increases the spectral variation and the coherence error. The balanced+smooth method enhances both properties with a reasonable trade-off. The standard method leaves the Cholesky or the eigenvalue decomposition unaltered.

Installation

pip install anf-generator

Example Code

import numpy as np
import anf_generator as anf

# Define signals
duration = 10  # Duration in seconds
num_channels = 4  # Number of microphones

# Define target spatial coherence
params = anf.CoherenceMatrix.Parameters(
  mic_positions=np.array([[0.04 * i, 0, 0] for i in range(num_channels)]),
  sc_type="spherical",
  sample_frequency=16000,
  nfft=1024,
)

# Generate "num_channels" mutually independent input signals of length "duration"
input_signals = np.random.randn(num_channels, duration * params.sample_frequency)

# Generate output signals with the desired spatial coherence
output_signals, coherence_target, mixing_matrix = anf.generate_signals(
  input_signals, params, decomposition='evd', processing='balance+smooth')

Two more test scripts are provided: tests/example.py and tests/example_babble.py.

Audio Examples

Click here to listen to examples generated using this method.

References

[1] E.A.P. Habets, I. Cohen and S. Gannot, 'Generating nonstationary multisensor signals under a spatial coherence constraint,' The Journal of the Acoustical Society of America, Vol. 124, Issue 5, pp. 2911-2917, Nov. 2008. Code

[2] D. Mirabilii, S. J. Schlecht, E.A.P. Habets, 'Generating coherence-constrained multisensor signals using balanced mixing and spectrally smooth filters', The Journal of the Acoustical Society of America, Vol. 149, 1425, 2021.