Uploaded image for project: 'Hyrax-Kanban'
  1. HK-10

'Too Many Open Files' error when the DMR++ handler is run on many variables (60+), or for many repetitions (60+)

    Details

    • Type: Bug
    • Status: Done (View workflow)
    • Priority: Highest
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: Hyrax 1.15.1
    • Labels:
      None

      Description

      Based on the error logs, it could be here, since this is the only call to curl_easy_strerror() that is not prefixed with some identifying string.

              CURLcode res = curl_easy_setopt(handle->d_handle, CURLOPT_URL, chunk->get_data_url().c_str());
              if (res != CURLE_OK) throw BESInternalError(string(curl_easy_strerror(res)), __FILE__, __LINE__);
      

      inside

      dmrpp_easy_handle *
      CurlHandlePool::get_easy_handle(Chunk *chunk)
      {
      }
      

      But that is not the case. It's actually

      2018-10-04T00:37:52GMT|&|1863|&|ERROR: BES Internal Error: Too many open files (BES error code: 1).
      2018-10-04T00:37:52GMT|&|1863|&| at: BESFileLockingCache.cc:493
      

      (I changed how BESError objects are logged when Verbose logging is turned on.)

      This is error comes from:

      static bool getSharedLock(const string &file_name, int &ref_fd)
      {
          BESDEBUG("cache2", "getSharedLock(): Acquiring cache read lock for " << file_name <<endl);
      
          int fd;
          if ((fd = open(file_name.c_str(), O_RDONLY)) < 0) {
              switch (errno) {
              case ENOENT:
                  return false;
      
              default:
                  throw BESInternalError(get_errno(), __FILE__, __LINE__);
              }
          }
      ...
      }
      

      I will change the BESDEBUG lines so that we can track lock/unlock operations and re-run the tests.

        Attachments

          Activity

            People

            • Assignee:
              jimg James Gallagher
              Reporter:
              jimg James Gallagher
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 day
                1d