Based on a change in scope and contract, OPeNDAP will develop an implementation of Architecture #2 for Hyrax in the Cloud (HiC). The details of Arch #2 are described in the cloud trade study report that can be found on the OPeNDAP web site (as well as other locations).https://www.opendap.org/pdf/EED2-TP-028_Rev01_WebAccessible%20API%20Cloud%20Study.pdf
The main difference between Arch #1 (implemented in PI 18.1 and Arch #2 is that data are subset directly from S3 in Arch #2 and the meta-information needed for that are made explicitly available using an enhanced DAP4 metadata response.
Implement Architecture #2
OPeNDAP and THG examined three distinct architectures that can be used to implement DAP web services for data in HDF5 files that are stored in S3. Architecture #2 uses HTTP Range GET operations to subset HDF5 granules 'in place,' without first transferring the granule out of S3. This can reduce both access latency and total response time in many cases because typical accesses are limited to a fraction of the total data in a granule.
The Hyrax server can process a request for data and read only the data requested from S3 using zero or more HTTP Range-GET requests. This implementation should include HTTP 'connection reuse' and optionally parallel data transfers.