Python dict.get's Default Value is Always Evaluated

This is a gotcha I ran across in some production code that is obvious in retrospect. I was profiling the code to find places where we were calling "an_expensive_database_function" and came across code like this:

def doit(*args, **kwargs):
    value = kwargs.get('key', an_expensive_database_function())

The original author probably assumed that if 'key' was present in the kwargs dictionary an_expensive_database_function wouldn't be called; that it would be short circuited in the same manner as Boolean expressions. But since get is a function the arguments are always evaluated on the way into the function. So in this case even if the value of an_expensive_database_function was already present in the kwargs dictionary the database function would be called again.

Here is a "look before you leap" solution:

def doit(*args, **kwargs):
    value = kwargs.get('key')
    if value is None: 
        # assuming default value None isn't a valid value
        value = an_expensive_database_function()

Here is the "easier to ask forgiveness than permission" solution:

def doit(*args, **kwargs):
    try:
        value = kwargs['key']
    except KeyError:
        value = an_expensive_database_function()

Mar 22, 2012

Richard Klein said...

Thanks for the post. I do something similar to your "look before you leap" solution but shorten it up to one line.

value = kwargs.get('key') or expensive_database_function()

Steve Schwarz said...

Richard thanks for you comment.

That works well unless the dict's value(s) for which you don't want to call an_expensive_database_function can be "falsey": (), [], "". So that's why I've taken to check explicitly for the not present value getting returned from get() which makes the inline solution too verbose/costly:

value = kwargs.get("key") != None and kwargs.get("key") or an_expensive_database_function()

Of course if that isn't a possibility your solution is more succinct.

tech.agilitynerd.com