[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Bulletproof json.dump?

On Mon, Jul 6, 2020 at 8:36 PM Adam Funk <a24061 at> wrote:
> Hi,
> I have a program that does a lot of work with URLs and requests,
> collecting data over about an hour, & then writing the collated data
> to a JSON file.  The first time I ran it, the json.dump failed because
> there was a bytes value instead of a str, so I had to figure out where
> that was coming from before I could get any data out.  I've previously
> run into the problem of collecting values in sets (for deduplication)
> & forgetting to walk through the big data object changing them to
> lists before serializing.
> Is there a "bulletproof" version of json.dump somewhere that will
> convert bytes to str, any other iterables to list, etc., so you can
> just get your data into a file & keep working?

That's the PHP definition of "bulletproof" - whatever happens, no
matter how bad, just keep right on going. If you really want some way
to write "just anything" to your file, I recommend not using JSON -
instead, write out the repr of your data structure. That'll give a
decent result for bytes, str, all forms of numbers, and pretty much
any collection, and it won't break if given something that can't
safely be represented.