Renaming of shared dimensions doesn't work (and should) - Could be duplicate of renaming a variable

Description

Hi James,

Our preliminary tests show that the NcML module still cannot recognize dimension name changes. I include an email below of our early correspondence with John Caron about the NcML schema, and the example for the dimension change in ncml that we desire.

If I attempt to "remove" existing dimensions in order to redefine my own, Hyrax would issue an error saying that this version of the module only supports removal of attributes and variables.

One consequence is that the fonc module wouldn't honor such changes in NcML either (though not recognizing, the NcML module never complained about the attempted dimension changes as long as I don't remove any existing dimension).

I wonder if this can be implemented in the module after all? Do you have an idea of how long it'd take to do it? Perhaps I didn't emphasize but this feature would be highly important for our data service.

I did verify that the fonc module no longer prepends "h4_", "h5_", or "ncml_" in front of the variable or attribute names when downloading in netcdf, which is good.

-Fan

On 06/18/11 10:24, John Caron wrote:
On 6/17/2011 11:44 AM, Gallagher James wrote:
John,

We have some users of ncml that are trying to rename dimensions using ncml that looks like:

where "aerosol_opt_001" is a variable in an existing file and it's dimension names in that file are some useless gibberish. When they slurp their ncml into panoply and IDV, the dimensions are renamed; our ncml code doesn't do this, however. I checked the on-line docs and it wasn't clear (but was sort of implied) that this syntax was supposed to rename dimensions. Is it?

Thanks,
James
--
James Gallagher
jgallagher at opendap.org
406.723.8663

John's reply:

I think that the above does work in NcML, because the new shared
dimension objects are created and added to the file, then they are used
by the variables as usual. ill make a note to improve the docs.

Environment

None

Activity

Show:
James Gallagher
April 9, 2012, 8:31 PM

Update/additions to this bug:

There are at least two more features we would like to have in this handler. Hopefully implementing these won't erase the gain in #'s 1 and 2?

i). To define and use new dimensions (I sent you this one before and you probably already had a ticket for it). This shall include supporting <remove> dimensions, and/or recognizing new <dimension> definitions in the "shape" attribute of a variable, defining new variables that can use the new dimension, etc.

ii). In"joinExisting" aggregation, the dimension values are simply "join"ed which shall instead be redefined.

Wonder if these are doable soon? To be able to do the aggregation correctly will be a significant milestone for our OPeNDAP service.

-Fan

p.s. Here is an example for ii). The "MAI1NXINT.5.2.0" data can be accessed from

http://goldsmr2.sci.gsfc.nasa.gov/opendap/MERRA/MAI1NXINT.5.2.0/contents.html

Suppose I only have 1980 data and if I have this ncml data to aggregate:

The aggregated "TIME_EOSGRID", with " units: minutes since 1980-01-01 00:00:00", shows repeated values like

TIME_EOSGRID, 0, 60, 120, 180, 240, 300, 360, 420, 480, 540, 600, 660, 720, 780, 840, 900, 960, 1020, 1080, 1140, 1200, 1260, 1320, 1380, 0, 60, 120, 180, 240, 300, 360, 420

while the values should really increment after 1380 to 1440 minutes, etc.

It may not be straightforward to figure out which variables are dimensional. It is the dimension to be aggregated. For our Merra data it can be read off of the "coordinates" attribute of the aggregated variable. I think at the minimum the handler shall allow defining new dimensional variables corresponding to the dimension along which the data is to be aggregated (I wonder if this is already there for "joinNew"). When I do this like:

I got this "superman" error in browser:

OUCH!

##BESError##Type# 1##Message# NCMLModule InternalError# #static std##string ncml_module##NCMLParser##convertNcmlTypeToCanonicalType#const std##string#### ASSERTION FAILED# condition## #ncmlType#empty## # Logic error# convertNcmlTypeToCanonicalType disallows empty## input###Administrator# admin#email#address#your#domain#name##

Kent Yang
August 1, 2017, 6:49 PM

Another error related to renaming dimensions:

This is from LP DAAC. They are trying to rename dimensions with NcML but get the failure message. There are two symptoms :

1) When removing dimensions with the following NcML:
<?xml version="1.0" encoding="UTF-8"?>
<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" location="LANDSAT_ARD_AGGREGATES/LANDSAT_ARD_SRB1/h005v002.ncml">
<remove type="dimension" name="northing"/>
<remove type="dimension" name="easting"/>
<dimension name="YDim" length="5000"/>
<dimension name="XDim" lenght="5000"/>
<variable name="SRB1" shape="XDim YDim"/>
</netcdf>

The error message is as follows:

NCMLModule ParseError: at *.ncml line=4: Illegal type in remove element: type=dimension This version of the parser can only remove type="attribute" or type="variable".

2) When I just simply use:

<dimension name="YDim" length="5000"/>
<dimension name="XDim" lenght="5000"/>
<variable name="SRB1" shape="XDim YDim"/>

I don't get errors but the variable name SRB1's dimensions are not changed.

Slav Korolev
November 29, 2018, 8:49 PM
Edited

In this ticket remove dimention should be implemented so that for ncml like this:

DDS output will be

Assignee

Slav Korolev

Reporter

James Gallagher

Labels

Fix versions

None

Time remaining

8h

Story Points

None

Time tracking

0m

Epic Link

Priority

Highest
Configure