Fileout netCDF cannot generate a valid netCDF file when string datatype has a _FillValue

Description

FilenetCDF out fails to generate either a netcdf3 or a netcdf4 for OMPS due to a string datatype variable has a _FillValue. I can confirm this is not a netCDF bug since netCDF’s nccopy can successfully generate a netCDF-3 file. It is a bug inside the filenetCDF out. I use NcML to rename the _FillValue to _FillValueStr, then I can successfully generate nc-4 or nc-3 files.

Using NCML I can change the string _FillValue to another name and generate netCDF-4 or netCDF-3 files.

http://alpaca:8080/opendap/hyrax/data/ncml/ges_2017_eval/rename_string_fillvalue.ncml.html

Environment

None

Activity

Show:
Kent Yang
August 17, 2017, 9:14 PM
Edited

James,

Great that you fix the filenetCDF-4.
For netCDF-3, I think you can map this DAP string array to a 2-D Char array. This is what nccopy does. Check the file
ftp://gamma.hdfgroup.org/pub/outgoing/ymuqun/myopendap/Hyrax-397/out.nc

This is the netcdf-3 file I generated with nccopy of the above HDF5 file via Hyrax. Particularly, the string variable is mapped to a 2-D char array and the fillvalue is not limited to one character. I think this case is allowed by netCDF-C since nccopy is distributed by Unidata's netCDF-C package.

char GeolocationData_UTC_CCSDA_A(DimAlongTrack, maxStrlen64) ;
GeolocationData_UTC_CCSDA_A:long_name = "UTC Image Midpoint Time" ;
GeolocationData_UTC_CCSDA_A:coverage_content_type = "referenceInformation" ;
GeolocationData_UTC_CCSDA_A:_FillValue = "0000-00-00T00:00:00.000000Z" ;
GeolocationData_UTC_CCSDA_A:valid_range = "2012-01-01T00:00:00.000000Z\n",
"2099-12-31T00:00:00.000000Z" ;
GeolocationData_UTC_CCSDA_AIMENSION_LABELS = "DimAlongTrack" ;
GeolocationData_UTC_CCSDA_A:origname = "UTC_CCSDA_A" ;
GeolocationData_UTC_CCSDA_A:fullnamepath = "/GeolocationData/UTC_CCSDA_A" ;

James Gallagher
August 17, 2017, 9:59 PM

Kent,

I haven't fixed anything yet - I don't have time to work on this now. I just was talking with Dennis @ Unidata and he told me about the restriction on NC_CHAR variables and _FillValue. I found out about the NC4 NC_STRING thing that looks like a fix by searching online and reading some of your earlier comments on this ticket.

Slav Korolev
May 22, 2019, 2:14 PM
Slav Korolev
June 21, 2019, 11:55 AM

NetCDF-3 there is no string type, only a char array type, so the fill value should be just one character, not an array of characters. For this reason has been decided for FillValue attribute to use just first character from original value and to add additional string attribute Orig_FillValue with original value of _FillValue.

James Gallagher
June 21, 2019, 4:10 PM

I’ll mark this as done. The HK-23 branch has been merged to master.

Assignee

Slav Korolev

Reporter

Kent Yang

Labels

Fix versions

None

Time remaining

16h

Story Points

None

Time tracking

0m

Epic Link

Priority

Highest
Configure