Developing Rationale for Experimental Designs Employed During Forthcoming NASA Quesst Mission Community Noise Campaigns

Since the publication of R.A. Fisher’s “The Arrangement of Field Experiments” in 1926, the enumerated principles that contributed to the improvement of agricultural field experiments in the early 20th century (randomization, replication, blocking, and appropriate variation of factors) remain hallmarks of all well-designed clinical trials and scientific endeavors in the 21st century. Beginning in 2026, NASA will conduct community noise campaigns with the first-of-kind X-59 experimental aircraft to 1) demonstrate the possibility of quiet supersonic flight over land and 2) to collect live data about annoyance (a categorical response) in relation to estimated noise levels (an experimental factor) produced by the new noise phenomenon, a low-noise “sonic thump”. The resulting predicted relationship between perceptual response and estimated noise level, e.g., mixed logistic regression or related models in the class of generalized linear (mixed) models, is believed to be a useful policy tool that domestic and international regulators can use when deciding whether existing prohibitions of supersonic flight over land can be replaced with a noise-based limit. The X-59 is engineered with the expectation that annoyance in the tested ranges should be a rare outcome, and, consequently, NASA convened an interdisciplinary team to examine and refine the rationale underlying the experimental design for future community campaigns. In this presentation, we review relevant literature on experiments with rare binary outcomes, relate the deliberations of the NASA team to tenets of good experimental design, and highlight several challenges and operational realities of this ambitious campaign.