ERROR: Remote end closed connection without response

[gchudnov]

When running clin process -d --env=staging -t $TOK ./nakadi/mops/paas.clin.yaml and there are lots of event-types / subsciptions to process, sometimes clin is failing with an exception, (see below).

EDIT: hm. somehow failing just a lot on TEST-env. Already made 8 attempts and it is failing sooner or later

Expected Behavior

If possible it would be great to avoid termination of the app and have some retry logic inside (or increase the amount of retries if they already present (?) ).

Actual Behavior

Just was running clin several times in a row and it was failing with the following errors:

('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
Traceback (most recent call last):
  File "/home/gchudnov/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/gchudnov/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 426, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/gchudnov/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    httplib_response = conn.getresponse()
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/http/client.py", line 1373, in getresponse
    response.begin()
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/http/client.py", line 319, in begin
    version, status, reason = self._read_status()
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/http/client.py", line 288, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/site-packages/requests/adapters.py", line 450, in send
    timeout=timeout
  File "/home/gchudnov/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/gchudnov/.local/lib/python3.7/site-packages/urllib3/util/retry.py", line 403, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/gchudnov/.local/lib/python3.7/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/home/gchudnov/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/gchudnov/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 426, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/gchudnov/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    httplib_response = conn.getresponse()
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/http/client.py", line 1373, in getresponse
    response.begin()
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/http/client.py", line 319, in begin
    version, status, reason = self._read_status()
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/http/client.py", line 288, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/site-packages/clin/run.py", line 184, in process
    processor.apply(task.target, task.envelope)
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/site-packages/clin/processor.py", line 55, in apply
    apply(env, envelope.spec)
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/site-packages/clin/processor.py", line 147, in apply_subscription
    sub.event_types, sub.owning_application, sub.consumer_group
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/site-packages/clin/clients/nakadi.py", line 84, in get_subscription
    resp = requests.get(url, headers=self._headers, params=params)
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/home/gchudnov/.pyenv/versions/3.7.13/lib/python3.7/site-packages/requests/adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Steps to Reproduce the Problem

1. on test-environemtn
2. yaml with lots of event-types / seubscriptions (?)
3. clin process -d --env=staging -t $TOK ./nakadi/mops/paas.clin.yaml

Specifications

• Version: 1.4.0
• Platform: Ubuntu Linux 18.04