Fix #6628 - JSONDecodeError are not deserializable

requests.exceptions.JSONDecodeError are not deserializable: calling
`pickle.dumps` followed by `pickle.loads` will trigger an error.

This is particularly a problem in a process pool, as an attempt to
decode json on an invalid json document will result in the entire
process pool crashing.

This is due to the MRO of the `requests.exceptions.JSONDecodeError`
class: the `__reduce__` method called when pickling an instance is not
the one from the JSON library parent: two out of three args expected
for instantiation will be dropped, and the instance can't be
deserialised.

By specifying in the class which parent `__reduce__` method should be
called, the bug is fixed as all args are carried over in the resulting
pickled bytes.
This commit is contained in:
Thomas Dehghani
2024-01-31 17:07:35 +01:00
parent 96b22fa18c
commit 3ff3ff21dd
2 changed files with 20 additions and 0 deletions
+10
View File
@@ -41,6 +41,16 @@ class JSONDecodeError(InvalidJSONError, CompatJSONDecodeError):
CompatJSONDecodeError.__init__(self, *args)
InvalidJSONError.__init__(self, *self.args, **kwargs)
def __reduce__(self):
"""
The __reduce__ method called when pickling the object must
be the one from the JSONDecodeError (be it json/simplejson)
as it expects all the arguments for instantiation, not just
one like the IOError, and the MRO would by default call the
__reduce__ method from the IOError due to the inheritance order.
"""
return CompatJSONDecodeError.__reduce__(self)
class HTTPError(RequestException):
"""An HTTP error occurred."""
+10
View File
@@ -2810,3 +2810,13 @@ class TestPreparingURLs:
assert r4 == 425
assert r5 == 425
assert r6 == 425
def test_json_decode_errors_are_serializable_deserializable():
json_decode_error = requests.exceptions.JSONDecodeError(
"Extra data",
'{"responseCode":["706"],"data":null}{"responseCode":["706"],"data":null}',
36,
)
deserialized_error = pickle.loads(pickle.dumps(json_decode_error))
assert repr(json_decode_error) == repr(deserialized_error)