[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Bulletproof json.dump?

On 2020-07-06, Chris Angelico wrote:

> On Mon, Jul 6, 2020 at 8:36 PM Adam Funk <a24061 at> wrote:
>> Hi,
>> I have a program that does a lot of work with URLs and requests,
>> collecting data over about an hour, & then writing the collated data
>> to a JSON file.  The first time I ran it, the json.dump failed because
>> there was a bytes value instead of a str, so I had to figure out where
>> that was coming from before I could get any data out.  I've previously
>> run into the problem of collecting values in sets (for deduplication)
>> & forgetting to walk through the big data object changing them to
>> lists before serializing.
>> Is there a "bulletproof" version of json.dump somewhere that will
>> convert bytes to str, any other iterables to list, etc., so you can
>> just get your data into a file & keep working?
> That's the PHP definition of "bulletproof" - whatever happens, no
> matter how bad, just keep right on going. If you really want some way

Well played!

> to write "just anything" to your file, I recommend not using JSON -
> instead, write out the repr of your data structure. That'll give a
> decent result for bytes, str, all forms of numbers, and pretty much
> any collection, and it won't break if given something that can't
> safely be represented.

Interesting point.  At least the TypeError message does say what the
unacceptable type is ("Object of type set is not JSON serializable").

"It is the role of librarians to keep government running in difficult
times," replied Dramoren.  "Librarians are the last line of defence
against chaos."                                       (McMullen 2001)