Change the Data Request Form code (all 3 versions) so that it URL encodes the query before using it.

Description

Currently Tomcat-8.5.31 ships configured to reject URL's that are not correctly URL encoded to the HTTP-1.1 standard. This is a part of a larger security push industry wide and we need to get out client code (the Data Request Forms) into compliance by updating our Data Request Form generation code so that the Form URL encodes the URL before an attempt is made to dereference it. This should not be applied to the text in the Data URL field in the form as it will make it impossible to read. Rather, we do the encoding as we send the request.

Here is the original message that brought this to my attention:

Hi,
Some of our users are reporting that some requests to our opendap are returning a 400 to them. For example this link returns a 400 error:
https://ladsweb.modaps.eosdis.nasa.gov/opendap/allData/6/MOD08_D3/2016/122/MOD08_D3.A2016122.006.2016123095613.hdf.ascii?XDim[0:1:359]

We consistently get a 400 error on that link but if you remove the XDim parameter the link works. Additionally, the response seems to be slightly different. For example with curl we simply receive a 400 HTTP code with no data while with Chrome browser we get a 400 code along with message:

Since this is a live system with a lot of requests it’s hard to associate error messages from tomcat with a particular request but I think when we make this request we receive this error message from tomcat:

I partially suspect that the log is unrelated to this error.

We’re running olfs version 1.17.0, libdap and bes version 3.19.1-1.
The olfs container is based off of the tomcat:8-jre8 image. The exact tomcat version is 8.5.31.0. The base OS is Debian 9.4.
The bes container is based off of centos:7 image. The bes image is identical to https://github.com/OPENDAP/hyrax-docker/blob/master/hyrax-1.14.0/besd/Dockerfile

Any help?

Navid

Environment

We’re running olfs version 1.17.0, libdap and bes version 3.19.1-1.

The olfs container is based off of the tomcat:8-jre8 image. The exact tomcat version is 8.5.31.0. The base OS is Debian 9.4.

The bes container is based off of centos:7 image. The bes image is identical to https://github.com/OPENDAP/hyrax-docker/blob/master/hyrax-1.14.0/besd/Dockerfile

Status

Assignee

Nathan Potter

Reporter

Nathan Potter

Labels

Epic Link

Components

Priority

Medium
Configure