Home » Python » Correct way to try/except using Python requests module?

Correct way to try/except using Python requests module?

Posted by: admin November 1, 2017 Leave a comment

    r = requests.get(url, params={'s': thing})
except requests.ConnectionError, e:
    print e #should I also sys.exit(1) after this?

Is this correct? Is there a better way to structure this? Will this cover all my bases?


Have a look at the Requests exception docs. In short:

In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception.

In the event of the rare invalid HTTP response, Requests will raise an HTTPError exception.

If a request times out, a Timeout exception is raised.

If a request exceeds the configured number of maximum redirections, a TooManyRedirects exception is raised.

All exceptions that Requests explicitly raises inherit from requests.exceptions.RequestException.

To answer your question, what you show will not cover all of your bases. You’ll only catch connection-related errors, not ones that time out.

What to do when you catch the exception is really up to the design of your script/program. Is it acceptable to exit? Can you go on and try again? If the error is catastrophic and you can’t go on, then yes, a call to sys.exit() is in order.

You can either catch the base-class exception, which will handle all cases:

    r = requests.get(url, params={'s': thing})
except requests.exceptions.RequestException as e:  # This is the correct syntax
    print e

Or you can catch them separately and do different things.

    r = requests.get(url, params={'s': thing})
except requests.exceptions.Timeout:
    # Maybe set up for a retry, or continue in a retry loop
except requests.exceptions.TooManyRedirects:
    # Tell the user their URL was bad and try a different one
except requests.exceptions.RequestException as e:
    # catastrophic error. bail.
    print e

As Christian pointed out:

If you want http errors (e.g. 401 Unauthorized) to raise exceptions, you can call Response.raise_for_status. That will raise an HTTPError, if the response was an http error.

An example:

    r = requests.get('http://www.google.com/nothere')
except requests.exceptions.HTTPError as err:
    print err

Will print:

404 Client Error: Not Found for url: http://www.google.com/nothere