NASA Logo

NTRS

NTRS - NASA Technical Reports Server

Back to Results
Reducing Truncation Error In Integer ProcessingImproved method of rounding off (truncation of least-significant bits) in integer processing of data devised. Provides for reduction, to extremely low value, of numerical bias otherwise generated by accumulation of truncation errors from many arithmetic operations. Devised for use in integer signal processing, in which rescaling and truncation usually performed to reduce number of bits, which typically builds up in sequence of operations. Essence of method to alternate direction of roundoff (plus, then minus) on alternate occurrences of truncated values contributing to bias.
Document ID
19950065240
Acquisition Source
Legacy CDMS
Document Type
Other - NASA Tech Brief
Authors
Thomas, J. Brooks
(Jet Propulsion Lab., California Inst. of Tech., Pasadena, CA.)
Berner, Jeffrey B.
(Jet Propulsion Lab., California Inst. of Tech., Pasadena, CA.)
Graham, J. Scott
(Jet Propulsion Lab., California Inst. of Tech., Pasadena, CA.)
Date Acquired
August 17, 2013
Publication Date
February 1, 1995
Publication Information
Publication: NASA Tech Briefs
Volume: 19
Issue: 2
ISSN: 0145-319X
Subject Category
Mathematics And Information Sciences
Report/Patent Number
NPO-18968
Accession Number
95B10081
Distribution Limits
Public
Copyright
Work of the US Gov. Public Use Permitted.

Available Downloads

There are no available downloads for this record.
No Preview Available