Towards the automatic assessment of spatial quality in the reproduced sound environment

Research Student: Dr Rob Conetta
Principal Supervisor: Dr Francis Rumsey
Co-supervisor: Dr Slawek Zielinski,
Thesis Supervisor: Dr Tim Brookes
Industrial partners: Prof Søren Bech, Bang & Olufsen, Denmark. David Meares, BBC Research and Development

Start date: 2006
End date: 2011

Part of QESTRAL (EPSRC Project Reference: EP/D041244/1)

Project Outline

This project formed part of QESTRAL (Quality Evaluation of Spatial Transmission and Reproduction using an Artificial Listener), creating and deveoping a method for the prediction of perceived spatial quality. The QESTRAL model is an objective evaluation model capable of accurately predicting changes to perceived spatial quality. It uses probe signals and a set of objective metrics to measure changes to low-level spatial attributes. A polynomial weighting function derived from regression analysis is used to predict data from listening tests, which employed spatial audio processes (SAPs) proven to stress those low-level attributes.

A listening test method was developed for collecting listener judgements of impairments to spatial quality. This involved the creation of a novel test interface to reduce the biases inherent in other similar audio quality assessment tests. Pilot studies were undertaken which established the suitability of the method.

Two large scale listening tests were conducted using 31 Tonmeister students from the Institute of Sound Recording (IoSR), University of Surrey. These tests evaluated 48 different SAPs, typically encountered in consumer sound reproduction equipment, when applied to 6 types of programme material. The tests were conducted at two listening positions to determine how perceived spatial quality was changed.

Analysis of the data collected from these listening tests showed that the SAPs created a diverse range of judgements that spanned the range of the spatial quality test scale and that listening position, programme material type and listener each had a statistically significant influence upon perceived spatial quality. These factors were incorporated into a database of 308 responses used to calibrate the model.

The model was calibrated using partial least-squares regression using target specifications similar to those of audio quality models created by other researchers. This resulted in five objective metrics being selected for use in the model. A method of post correction using an exponential equation was used to reduce non-linearity in the predicted results, thought to be caused by the inability of some metrics to scrutinise the highest quality SAPs. The resulting model had a correlation (r) of 0.89 and an error (RMSE) of 11.06% and performs similarly to models developed by other researchers. Statistical analysis also indicated that the model would generalise to a larger population of listeners.

Publications

Journal Papers

• Conetta R, Brookes T, Rumsey F, Zielinski S, Dewhirst M, Jackson P, Bech S, Meares D, George S. (2014) 'Spatial Audio Quality Perception (Part 1): Impact of Commonly Encountered Processes'. AUDIO ENGINEERING SOC JOURNAL OF THE AUDIO ENGINEERING SOCIETY, 62 (12), pp. 831-846.
  [full text]
• Conetta R, Brookes T, Rumsey F, Zielinski S, Dewhirst M, Jackson P, Bech S, Meares D, George S. (2014) 'Spatial Audio Quality Perception (Part 2): A Linear Regression Model'. AUDIO ENGINEERING SOC JOURNAL OF THE AUDIO ENGINEERING SOCIETY, 62 (12), pp. 847-860.
  [full text]

Conference & Convention Papers

• Rumsey, F., Zielinski, S., Jackson, P.J.B., Dewhirst, M., Conetta, R., George, S., Bech, S. & Meares, D. (2008), "QESTRAL (Part 1): Quality Evaluation of Spatial Transmission and Reproduction using an Artificial Listener", Proceedings of the Audio Engineering Society 125th Convention, Oct 2-5, San Francisco, USA, Preprint 7595.
  [abstract] [bib]
• Conetta, R., Rumsey, F., Zielinski, S., Jackson, P.J.B., Dewhirst, M., Bech, S., Meares, D. & George, S. (2008) "QESTRAL (Part 2): Calibrating the QESTRAL spatial quality model using listening test data", Proceedings of the Audio Engineering Society 125th Convention, Oct 2-5, San Francisco, USA, Preprint 7596.
  [abstract] [bib]
• Jackson, P.J.B., Dewhirst, M., Conetta, R., Rumsey, F., Zielinski, S., Bech, S., Meares, D. & George, S. (2008), "QESTRAL (Part 3): System and metrics for spatial quality prediction", Proceedings of the Audio Engineering Society 125th Convention, Oct 2-5, San Francisco, USA, Preprint 7597.
  [abstract] [bib]
• Dewhirst, M., Conetta, R., Rumsey, F., Jackson, P.J.B., Zielinski, S., Bech, S., Meares, D. & George, S. (2008), "QESTRAL (Part 4): Test signals, combining metrics and the prediction of overall spatial quality", Proceedings of the Audio Engineering Society 125th Convention, Oct 2-5, San Francisco, USA, Preprint 7598.
  [abstract] [bib]
• George, S., Zielinski, S., Rumsey, F., Conetta, R., Dewhirst, M., Jackson, P.J.B., Meares, D. & Bech, S. (2008), "An Unintrusive Objective Model for Predicting the Sensation of Envelopment Arising from Surround Sound Recordings", Proceedings of the Audio Engineering Society 125th Convention, Oct 2-5, San Francisco, USA, Preprint 7599.
  [abstract] [bib]
• Conetta, R., Dewhirst, M., Rumsey, F., Zielinski, S., Jackson, P.J.B., Bech, S., Meares, D. & George, S. (2008).
  "Calibration of the QESTRAL model for the prediction of spatial quality", Proceedings of the Institute of Acoustics 24th Reproduced Sound Conference, Nov 20-21, Brighton, UK.
  [abstract] [poster] [bib]

Conference abstracts

• Jackson, P.J.B., Rumsey, F., Zielinski, S., Dewhirst, M., Conetta, R., Bech, S., & Meares, D. (2008).
  "Prediction of spatial perceptual attributes of reproduced sound across the listening area".
  J. Acoust. Soc. Am., 123 (5, pt.2): 2979, Presented at Acoustics'08, Paris,July 2008.
  [abstract] [bib]
• Rumsey, F., Zielinski, S., Jackson, P.J.B., Dewhirst, M., Conetta, R., Bech, S., & Meares, D. (2008).
  "Measuring perceived spatial quality changes in surround sound reproduction".
  J. Acoust. Soc. Am., 123 (5, pt.2): 2980, Presented at Acoustics'08, Paris,July 2008.(invited).
  [abstract] [bib]

Poster

• Conetta, R., Jackson, P.J.B., Zielinski, S. & Rumsey, F. (2007)
  "Envelopment: What is it? A definition for multichannel audio".
  Presented at the 1st SpACE-Net Workshop, Jan 25, York, UK.
  [abstract] [poster] [bib]

Thesis

• Conetta, R. (2011)
  "Towards the automatic assessment of spatial quality in the reproduced sound environment".
  [thesis]