Modify the BES command pipeline to recognize DMR++ applicability for a given path


As a data provider, I want Hyrax to recognize when it can use the DMR++ to provide access to data 'in place' on S3 so that users can access those data faster.

Part of the Architecture #2 story decomposition.

We will develop A#2 only for HDF5 files.

  1. DAP2 data requests may involve looking at a DMR (or DMR++) file.

  2. The DMR++ file uses xml:base to be the place where data are located. This needs to change. The xml:base attribute must be the name of the server that transmits the response to the client and the S3 location should be encoded elsewhere (e.g., another attribute, the chunks, ...). (moved to another ticket. jhrg 4/16/18)

  3. How are we going to recognize that a DMR++ is relevant for a given request? These are only relevant for data requests. Modify the data request part(s) of the command pipeline for the DMR++. Given that we will probably store the DMR++ in the MDS, we will also probably modify the metadata part(s) of the command pipeline too. The place to make this modification is in the BESDataResponseHandler::execute() and BESDap4ResponseHandler::execute()methods.

  4. Assume that the DMR++ response documents are in the MDS for any dataset where the DMR++ handler should be used and then, in the DMR++ document, the dmrpp:href attribute of the dapataset element will hold the location of the data.

  5. Metadata responses will be handled by populating the MDS with the correct information. Other tickets describe the population process.

Nathan suggests:

  • Modify the DHI to re-route this request to the DMR++ Handler and add a note that the DMR++ should be extracted from the MDS and not a file.

  • The DMR++ Handler will take it from there.


James Gallagher
June 1, 2018, 12:01 AM

I modified the BESDap4ResponseHandler and BESDataResponseHandler instead. These are in bes/dap, where the MDS is defined so the link pattern (where dispatch does not know about the dap module) is preserved. This also matches the pattern used for the DDS, DAS, DMR responses.

Your pinned fields
Click on the next to a field label to start pinning.


James Gallagher


James Gallagher