Research Report: Building a File Observatory for Secure Parser Development

"Parsing untrusted data is notoriously challenging.Failure to handle maliciously crafted data correctly can (anddoes) lead to a wide range of vulnerabilities. The Languagetheoretic security (LangSec) philosophy seeks to obviate the needfor developers to apply ad hoc solutions by, instead, offeringformally correct and verifiable input handling throughout thesoftware development lifecycle. One of the key components indeveloping secure parsers is a broad coverage corpus that enablesdevelopers to understand the problem space for a given formatand to use, potentially, as seeds for fuzzing and other automatedtesting. In this paper, we offer an update on the developmentof a file observatory to gather and enable analysis on a diversecollection of files at scale. Specifically, we report on the additionof a bug tracker corpus and new analytic methods on our existingcorpus."