We're updating the issue view to help you get more done. 

Data Transfer Errors in DMR++

Description

When I found the pipe() was leaking file descriptors, that fixed these errors, but there is another error.

Once in a while, the code gets a 500 http response. It should probably re-try those. However, when it gets that error, the excpetion catching mechanism will leak pthreads data because it does not call pthread_join on all the threads.

Here are the errors from a recent run:

1 2 3 4 5 6 7 8 9 10 [centos@ip-172-31-11-224 bin]$ grep Error $prefix/var/bes.log 2018-10-05T17:59:13GMT|&|25227|&|ERROR: BES Internal Error: HTTP status error: Expected an OK status, but got: 500 (CurlHandlePool.cc:251) (DmrppArray.cc:969) 2018-10-05T17:59:25GMT|&|1705|&|ERROR: BES Internal Error: HTTP status error: Expected an OK status, but got: 500 (CurlHandlePool.cc:251) (DmrppArray.cc:969) 2018-10-05T20:20:15GMT|&|22829|&|ERROR: BES Internal Error: HTTP status error: Expected an OK status, but got: 500 (CurlHandlePool.cc:251) (DmrppArray.cc:969) 2018-10-05T20:20:17GMT|&|22829|&|ERROR: BES Internal Error: No more libcurl handles. (Chunk.cc:489) (DmrppArray.cc:969) [centos@ip-172-31-11-224 bin]$ grep Error p18-2-test-harness.log 2018-10-05T17:59:13GMT|&|25227|&|ERROR: BES Internal Error: HTTP status error: Expected an OK status, but got: 500 (CurlHandlePool.cc:251) (DmrppArray.cc:969) 2018-10-05T20:20:15GMT|&|22829|&|ERROR: BES Internal Error: HTTP status error: Expected an OK status, but got: 500 (CurlHandlePool.cc:251) (DmrppArray.cc:969) [centos@ip-172-31-11-224 bin]$

Old info, now no longer true:
There are some Data Transfer Errors when reading data from S3. They fall into four groups:

  • Couldn't resolve host name

  • Couldn't connect to server

  • Problem with the SSL CA cert (path? access rights?)

  • SSL connect error

And most likely originate here:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 void dmrpp_easy_handle::read_data() { CURL *curl = d_handle; // Perform the request CURLcode curl_code = curl_easy_perform(curl); if (CURLE_OK != curl_code) { throw BESInternalError(string("Data transfer error: ").append(curl_easy_strerror(curl_code)), __FILE__, __LINE__); } // For HTTP, check the return code, for the file protocol, if curl_code is OK, that's good enough if (d_url.find("https://") == 0 || d_url.find("http://") == 0) { evaluate_curl_response(curl); } d_chunk->set_is_read(true); }

Environment

None

Status

Assignee

James Gallagher

Reporter

James Gallagher

Labels

None

Fix versions

Story Points

1

Affects versions

Hyrax 1.15.1

Epic Link

Priority

High