Avoid KeyError in pandas mapping: use dict.get with same-shape defaults to label DataFrame rows safely

Learn how to prevent KeyError in pandas when mapping from a lookup table. Use dict.get with shape-safe defaults to enrich DataFrame columns fast and robust.

When you enrich an input DataFrame with metadata from a static lookup table, a single missing key can derail the whole run with a KeyError. This guide shows how to make that mapping robust using pandas and a plain Python dict, while keeping the logic fast and readable.

Reproducing the issue

Start with a CSV-backed lookup that you convert to a dictionary keyed by the server name. Then label incoming rows by mapping the server column into attributes like type, cost, and location.

import pandas as pd
ref_df = pd.read_csv(r'C:\Location\format1.csv', sep=',')
ref_df['label'] = 'apache'
assoc_map = (
    ref_df
    .set_index('server')
    .T
    .to_dict('list')
)
print(assoc_map)
# {'ABC123': ['IBM', 1000, 'East Coast', 'apache'],
#  'ABC456': ['Dell', 800, 'West Coast', 'apache'],
#  'XYZ123': ['HP', 900, 'West Coast', 'apache']}
events_df = pd.read_csv(r'C:\Location\my_data.csv')
print(events_df)
#    server  busy       datetime
# 0  ABC123   24%  6/1/2024 0:02
# 1  ABC456   45%  6/1/2024 4:01
# 2  GHI100   95%  6/1/2024 9:10
# Direct indexing crashes when a key is missing
events_df['type'] = events_df['server'].map(lambda s: assoc_map[s][0])
events_df['cost'] = events_df['server'].map(lambda s: assoc_map[s][1])
events_df['location'] = events_df['server'].map(lambda s: assoc_map[s][2])
# KeyError: 'GHI100'

As soon as a server in the input is not present in the dictionary, the direct assoc_map[s] lookup raises KeyError.

Why it breaks

The root cause is the dictionary access itself. The slice that fails is assoc_map[s]. The subsequent [0], [1], [2] index into the list only happens after the dict lookup is already done. Trying to wrap the wrong part of the expression with .get doesn’t help. For example, assoc_map.get([s][0], None) looks like it protects the list index, but [s][0] simply evaluates back to s. That means the expression becomes assoc_map.get(s, None), which returns the entire list of attributes when the key exists, and None when it doesn’t. Mapping that as-is drops the full list into the column, or yields None, which is not what you want.

If you prefer to trace what’s happening step by step, replace the lambda with a small function so you can add print and examine the inputs and outputs in isolation.

The fix

Apply .get to the dictionary access and provide a default list with the same shape as the real value. Then index that list. This guarantees the indexing is always valid, even when a key is missing.

# Use a same-shape default value and index after .get
# Here we use empty strings as placeholders
events_df['type'] = events_df['server'].map(lambda s: assoc_map.get(s, [""]*3)[0])
events_df['cost'] = events_df['server'].map(lambda s: assoc_map.get(s, [""]*3)[1])
events_df['location'] = events_df['server'].map(lambda s: assoc_map.get(s, [""]*3)[2])

If you want to avoid repeating similar lines, iterate the targets and their positions.

for idx, col in enumerate(['type', 'cost', 'location']):
    events_df[col] = events_df['server'].map(
        lambda s: assoc_map.get(s, [""]*3)[idx]
    )

An equivalent variant uses an explicit membership check and returns a placeholder when the key is absent.

for idx, col in enumerate(['type', 'cost', 'location']):
    events_df[col] = events_df['server'].map(
        lambda s: assoc_map[s][idx] if s in assoc_map else ""
    )

You can also update multiple columns in one pass.

events_df.update(
    (
        col,
        events_df['server'].map(
            lambda s: assoc_map[s][pos] if s in assoc_map else ""
        )
    )
    for pos, col in enumerate(['type', 'cost', 'location'])
)

Why this matters

Real-world data rarely aligns perfectly with reference tables. A defensive lookup ensures that one unexpected key doesn’t break the entire labeling step. Using dict.get at the dictionary access point, combined with a shape-compatible default, keeps your mapping predictable and your DataFrame schema stable, even when upstream sources drift.

Wrap-up

When mapping attributes from a pandas-backed lookup, put .get on the dictionary access, not on the list indexing. Provide a default list with the right length, index into it, and use placeholders that fit your downstream expectations. If you need to inspect behavior in detail, replace lambdas with a small function to print inputs and intermediate values. This small adjustment eliminates KeyError while preserving clear, vectorized-style code.