Python AES-CBC Decryption Fails? Persist the Initialization Vector (IV) to Prevent UnicodeDecodeError

Learn why AES-CBC decryption in Python fails with UnicodeDecodeError when the IV differs, and how to fix it by saving the IV with the ciphertext from encryption.

When encrypting with AES in CBC mode, it’s easy to get tripped up by one subtle but critical detail: the initialization vector (IV). If the IV used at decryption doesn’t match the one from encryption, the recovered bytes will be incorrect, and any attempt to interpret them as UTF‑8 text can crash with a UnicodeDecodeError. Below is a minimal example that demonstrates the issue and a straightforward fix.

Problem setup

The following Python snippet performs AES-CBC encryption, writes the ciphertext to a file, then reads and decrypts it. It looks fine at first glance, but decryption later fails to produce valid text.

from Crypto.Cipher import AES
from Crypto.Util.Padding import pad, unpad
from Crypto.Hash import SHA256 as H256
def digest_hex(buf):
    h = H256.new(buf)
    return h.hexdigest()
secret = '1234'
print('Key length:', len(secret))
blk_sz = 16
msg = "The quick brown fox jumped over the lazy dog"
print('Plain text length:', len(msg))
kbuf = pad(secret.encode(), blk_sz)
print('akey length', len(kbuf))
msg_bytes = msg.encode()
enc_ctx = AES.new(kbuf, AES.MODE_CBC)
padded = pad(msg_bytes, blk_sz)
print('sha payload:', digest_hex(padded))
ct = enc_ctx.encrypt(padded)
print('Encrypted sha:', digest_hex(ct))
with open('data.bin', 'wb') as fh:
    fh.write(ct)
# ---- later, decrypt ----
dec_ctx = AES.new(kbuf, AES.MODE_CBC)
with open('data.bin', 'rb') as fh:
    raw = fh.read()
print('file contents sha:', digest_hex(raw))
pt = dec_ctx.decrypt(raw)
print('decrypted sha:', digest_hex(pt))
clear = unpad(pt, blk_sz)
print('Plain text:', clear.decode())

The decryption step produces non-text bytes, and decoding fails with an error similar to:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa8 in position 1: invalid start byte

What’s actually going wrong

In CBC mode, encryption and decryption must use the same IV. If the decryption phase uses a different IV, the first block will be transformed incorrectly, and that error cascades through the decrypted output. Even though the code successfully performs encryption and decryption calls, the bytes you get back are not the original plaintext, so decoding to UTF‑8 throws an error.

Fixing the issue

The solution is to persist the IV along with the ciphertext and supply that exact IV to the decryptor. One simple pattern is to prepend the IV to the ciphertext when writing to disk, then read it back and split the first 16 bytes as the IV for AES-CBC.

Encryption with IV persisted:

from Crypto.Cipher import AES
from Crypto.Util.Padding import pad
from Crypto.Hash import SHA256 as H256
def digest_hex(buf):
    h = H256.new(buf)
    return h.hexdigest()
secret = '1234'
blk_sz = 16
msg = "The quick brown fox jumped over the lazy dog"
kbuf = pad(secret.encode(), blk_sz)
enc_ctx = AES.new(kbuf, AES.MODE_CBC)
padded = pad(msg.encode(), blk_sz)
ct = enc_ctx.encrypt(padded)
with open('data.bin', 'wb') as fh:
    fh.write(enc_ctx.iv + ct)

Decryption using the stored IV:

from Crypto.Cipher import AES
from Crypto.Util.Padding import unpad
from Crypto.Hash import SHA256 as H256
def digest_hex(buf):
    h = H256.new(buf)
    return h.hexdigest()
secret = '1234'
blk_sz = 16
# kbuf must be constructed exactly as during encryption
from Crypto.Util.Padding import pad
kbuf = pad(secret.encode(), blk_sz)
with open('data.bin', 'rb') as fh:
    payload = fh.read()
iv, raw = payload[:16], payload[16:]
dec_ctx = AES.new(kbuf, AES.MODE_CBC, iv=iv)
pt = dec_ctx.decrypt(raw)
clear = unpad(pt, blk_sz)
print(clear.decode())

Why this matters

Without the correct IV, you will not get the original plaintext back, even though the API calls themselves complete without raising. That leads to confusing symptoms such as mismatched hashes and decoding failures. Persisting and reusing the same IV on decryption keeps the process symmetrical and prevents corrupted output.

Takeaways

When using AES in CBC mode, always save the IV generated for encryption and provide that exact IV on decryption. A practical pattern is to concatenate IV and ciphertext in one binary blob and reconstruct them in the same order when reading. Keeping this discipline avoids hard‑to‑trace errors like UnicodeDecodeError at the final decode step and ensures the decrypted bytes match the original message.