NASA Logo

NTRS

NTRS - NASA Technical Reports Server

Back to Results
Addressing the Big-Earth-Data Variety Challenge with the Hierarchical Triangular MeshWe have implemented an updated Hierarchical Triangular Mesh (HTM) as the basis for a unified data model and an indexing scheme for geoscience data to address the variety challenge of Big Earth Data. We observe that, in the absence of variety, the volume challenge of Big Data is relatively easily addressable with parallel processing. The more important challenge in achieving optimal value with a Big Data solution for Earth Science (ES) data analysis, however, is being able to achieve good scalability with variety. With HTM unifying at least the three popular data models, i.e. Grid, Swath, and Point, used by current ES data products, data preparation time for integrative analysis of diverse datasets can be drastically reduced and better variety scaling can be achieved. In addition, since HTM is also an indexing scheme, when it is used to index all ES datasets, data placement alignment (or co-location) on the shared nothing architecture, which most Big Data systems are based on, is guaranteed and better performance is ensured. Moreover, our updated HTM encoding turns most geospatial set operations into integer interval operations, gaining further performance advantages.
Document ID
20160014543
Document Type
Conference Paper
Authors
Rilee, Michael L.
(Science Systems and Applications, Inc. Greenbelt, MD, United States)
Kuo, Kwo-Sen
(Maryland Univ. College Park, MD, United States)
Clune, Thomas
(NASA Goddard Space Flight Center Greenbelt, MD, United States)
Oloso, Amidu
(Science Systems and Applications, Inc. Greenbelt, MD, United States)
Brown, Paul G.
(Paradigm4, Inc. Waltham, MA, United States)
Yu, Honfeng
(Nebraska Univ. Lincoln, NE, United States)
Date Acquired
December 7, 2016
Publication Date
December 5, 2016
Subject Category
Earth Resources And Remote Sensing
Computer Programming And Software
Documentation And Information Science
Report/Patent Number
GSFC-E-DAA-TN34964
Meeting Information
2016 IEEE International Confernece on Big Data(Big Data 2016)(Washington, DC)
Funding Number(s)
CONTRACT_GRANT: NNX12AD03A
CONTRACT_GRANT: NNG12HP06C
Distribution Limits
Public
Copyright
Public Use Permitted.
Keywords
SciDB
Hierarchical Triangular Mesh
Data Fusion

Available Downloads

NameType 20160014543.pdf STI
No Preview Available