What Lowercasing Taught Me About Trusting Strings

Every so often a post reminds me that the most dangerous line of code in a system is the one that looks like it could not possibly be wrong. This week's version: calling .lower() on a string can be a security vulnerability. If your first reaction is skepticism, mine was too. Lowercasing is the plumbing of programming. We do it to normalize usernames, compare header names, canonicalize domains, and check things against blocklists. It feels like arithmetic. The problem is that case conversion is not a character-by-character mechanical operation. It is a linguistic one, defined by Unicode, and it has behavior that surprises almost everyone who has not been bitten before. Case is not symmetric, and not always local Two examples that break the mental model. Turkish has a dotless i, and correct locale-aware conversion maps between letters differently than English does, which means "the same" string can lowercase into two different results depending on locale settings. And there are characters outside ASCII whose lowercase form is an ASCII character, meaning a string that contains no k at all can become one that does after normalization. Sit with that second one for a moment, because it is the security-relevant shape. If you validate a string, then normalize it, you have validated something that no longer exists. Your check ran against one value and your system acts on another. That is the classic time-of-check versus time-of-use bug, except the mutation is not caused by an attacker racing you. It is caused by your own normalization call, quietly doing what the spec says it should do. I want to be careful not to overstate the specifics here, since the exact behavior depends on language runtime, Unicode version, and locale configuration. The generalizable lesson is what interests me. The pattern to look for in your own code Anywhere a string travels through this sequence, there is potential for trouble: Accept input. Check it against a rule: an allowlist, a blocklist, a com