Develop and deploy in AWS Architecture #2 for Hyrax in the Cloud

Description

Based on a change in scope and contract, OPeNDAP will develop an implementation of Architecture #2 for Hyrax in the Cloud (HiC). The details of Arch #2 are described in the cloud trade study report that can be found on the OPeNDAP web site (as well as other locations).https://www.opendap.org/pdf/EED2-TP-028_Rev01_WebAccessible%20API%20Cloud%20Study.pdf

The main difference between Arch #1 (implemented in PI 18.1 and Arch #2 is that data are subset directly from S3 in Arch #2 and the meta-information needed for that are made explicitly available using an enhanced DAP4 metadata response.

Objective
Summary
Implement Architecture #2

Description
OPeNDAP and THG examined three distinct architectures that can be used to implement DAP web services for data in HDF5 files that are stored in S3. Architecture #2 uses HTTP Range GET operations to subset HDF5 granules 'in place,' without first transferring the granule out of S3. This can reduce both access latency and total response time in many cases because typical accesses are limited to a fraction of the total data in a granule.

Acceptance Criteria
The Hyrax server can process a request for data and read only the data requested from S3 using zero or more HTTP Range-GET requests. This implementation should include HTTP 'connection reuse' and optionally parallel data transfers.

Assignee

James Gallagher

Reporter

James Gallagher

Priority

Medium

Labels

Fix versions

None

Epic Name

Architecture #2
Configure