git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Bulletproof json.dump?


On 2020-07-06, Frank Millman wrote:

> On 2020-07-06 2:06 PM, Jon Ribbens via Python-list wrote:
>> On 2020-07-06, Chris Angelico <rosuav at gmail.com> wrote:
>>> On Mon, Jul 6, 2020 at 8:36 PM Adam Funk <a24061 at ducksburg.com> wrote:
>>>> Is there a "bulletproof" version of json.dump somewhere that will
>>>> convert bytes to str, any other iterables to list, etc., so you can
>>>> just get your data into a file & keep working?
>>>
>>> That's the PHP definition of "bulletproof" - whatever happens, no
>>> matter how bad, just keep right on going.
>> 
>> While I agree entirely with your point, there is however perhaps room
>> for a bit more helpfulness from the json module. There is no sensible
>> reason I can think of that it refuses to serialize sets, for example.
>> Going a bit further and, for example, automatically calling isoformat()
>> on date/time/datetime objects would perhaps be a bit more controversial,
>> but would frequently be useful, and there's no obvious downside that
>> occurs to me.
>> 
>
> I may be missing something, but that would cause a downside for me.
>
> I store Python lists and dicts in a database by calling dumps() when 
> saving them to the database and loads() when retrieving them.
>
> If a date was 'dumped' using isoformat(), then on retrieval I would not 
> know whether it was originally a string, which must remain as is, or was 
> originally a date object, which must be converted back to a date object.
>
> There is no perfect answer, but my solution works fairly well. When 
> dumping, I use 'default=repr'. This means that dates get dumped as 
> 'datetime.date(2020, 7, 6)'. I look for that pattern on retrieval to 
> detect that it is actually a date object.
>
> I use the same trick for Decimal objects.
>
> Maybe the OP could do something similar.

Aha, I think the default=repr option is probably just what I need;
maybe (at least in the testing stages) something like this:

try:
    with open(output_file, 'w') as f:
        json.dump(f)
except TypeError:
    print('unexpected item in the bagging area!')
    with open(output_file, 'w') as f:
        json.dump(f, default=repr)

and then I'd know when I need to go digging through the output for
bytes, sets, etc., but at least I'd have the output to examine.


-- 
Well, we had a lot of luck on Venus
We always had a ball on Mars