MDS bug - LMT of data not used

Description

Check LMT info for a dataset. The MDS does not look at the underlying data (file) to check for changes.

Modify the handlers that work with the local file system to return whether or not the item is stale. (Stale == LMT test)

Modify handlers that work with remote data to return whether or not the item is stale. (Stale == HTTP/1.1 caching tests)

However, some items in the MDS are put there by an external process and should not be tested for stale-ness because the MDS is a store for some things (and a cache in other cases).

In the current implementation, the MDS is either and cache or a store, not both. We should take this into consideration and probably make a story for this.

Files with 'add_method(...)' (I removed the handlers that don't read from data sources - jg) :

  • CSVRequestHandler.cc ------------- (comma separated values)

  • FitsRequestHandler.cc -------------- (Flexible Image Transport System)

  • FFRequestHandler.cc --------------- (FreeForm - ascii and binary data from various sources, described by a 'format' file)

  • GDALRequestHandler.cc ---------- (geospatial data, generally GeoTiff, but GDAL can be configured to read from lots of different sources)

  • DmrppRequestHandler.cc -------- (Read data from Amazon's S3 web object store - it actually is much more general and can read from any web object store)

  • NCRequestHandler.cc -------------- (netCDF files)

Files with 'add_handler(...)' :

  • HDF4RequestHandler.cc ---------- HDF4 data files

  • HDF5RequestHandler.cc ---------- HDF5 data files

Handlers I'm not sure about:

  • SQLHandler (not part of the regular build)

  • NCMLHandler (I have to think about just what using cached objects means for this code)

Environment

None

Assignee

Sam Lloyd

Reporter

James Gallagher

Labels

Fix versions

Story Points

2

Affects versions

Epic Link

Priority

Medium
Configure