Research Report: Progress on Building a File Observatory for Secure Parser Development

Parsing untrusted data is notoriously challenging.Failure to handle maliciously crafted data correctly can (anddoes) lead to a wide range of vulnerabilities. The Languagetheoretic security (LangSec) philosophy seeks to obviate the needfor developers to apply ad hoc solutions by, instead, offeringformally correct and verifiable input handling throughout thesoftware development lifecycle. One of the key components indeveloping secure parsers is a broad coverage corpus that enablesdevelopers to understand the problem space for a given formatand to use, potentially, as seeds for fuzzing and other automatedtesting. In this paper, we offer an update on work reportedat the LangSec 2021 conference on the development of a fileobservatory to gather and enable analysis on a diverse collectionof files at scale. The initial focus of the observatory is on PortableDocument Format (PDF) files and file formats typically embeddedin PDFs. In this paper, we report on refactoring the ingestprocess, applying new analytic methods, and improving the User Interface.