Bare except and mutable defaults in 18 Python projects
← Back to Learn · By Ronak Arora · Published · Updated
Bare except:, mutable default arguments and wildcard imports are three of the most quoted Python pitfalls. We scanned 18 well-known Python projects — 1,892,200 lines of code — to see how often each really appears. In total there are 75 bare excepts, 60 mutable-default findings and 112 wildcard imports. That is about 0.13 findings per 1,000 lines, and the raw count overstates the problem once you read the code around them.
How we measured
The 18 projects are listed, with the commit scanned, in the open data file. We counted findings from REPO-SIGHT’s py-bare-except, py-mutable-default-arg and py-wildcard-import rules, then opened the code at every bare except and at the library-code mutable defaults to see what it did. A fourth rule, mutable class attributes, fired 1,988 times, more than these three combined, so we left it out. 4 projects have none of the three: click, jinja, requests and pytest.
Bare except: 75 findings, 6 projects
Only 6 of 18 projects contain a bare except at all, and one of them, SQLAlchemy, accounts for 49 of the 75. The rest are tornado (9), black (9), rich (6), scrapy (1) and flask (1).
A bare except catches everything, including KeyboardInterrupt and SystemExit. That is dangerous when the handler hides the error, and fine when it cleans up and passes the error on. This is how the 75 divide:
- Re-raises: 29. The handler does some cleanup and raises again, as in the first example below. 23 are in library code and 6 in tests. This is a legitimate pattern, and the main reason the rule alone overstates the problem.
- Swallows with
pass: 20. 17 are in tests or test-support code. 3 are in library code: two in SQLAlchemy’sorm/loading.pyand one indialects/oracle/provision.py. - Handles the error: 14. The handler returns a fallback value or records the failure, 7 in library code and 7 in tests.
- Test fixtures: 9. Every one of black’s 9 hits is in
tests/data/cases, which holds sample source files for the formatter to reformat. The bare excepts are the test input, not black’s own code. - Logs: 3. The handler logs the exception.
Re-raises — usually fine
try:
do_work()
except:
cleanup()
raise
Swallows — worth fixing
try:
do_work()
except:
pass
Out of 75 findings, only the 3 library swallows and perhaps a few of the 7 library handlers merit a second look. That is the real size of the problem in this data set.
Mutable defaults: 60 findings, 8 projects
The classic bug is a default such as items=[], which is created once and shared by every call. We found 60 findings at 49 distinct function definitions in 8 projects. 37 of the 49 are in tests or test-support code. Black has 17 findings, celery 13, django 9 and SQLAlchemy 8.
In the library cases we checked, the defaults were not mutated. Tornado’s ax_attrs default is rebound with set(ax_attrs) before use. Black’s Node.__init__ takes a copy with fixers_applied[:]. SQLAlchemy’s inspect_formatargspec replaces its {} defaults with kwonlydefaults or {}. FastAPI marks its own case with an explicit # noqa: B006, a sign the maintainers know about the pattern and chose it. The pattern is still worth avoiding, because the next person to edit the function may not know any of this.
Shared between calls
def add_item(item, items=[]):
items.append(item)
return items
One list per call
def add_item(item, items=None):
items = [] if items is None else items
items.append(item)
return items
Wildcard imports: 112 findings, 8 projects
74 of the 112 wildcard imports (66%) are in __init__.py files, where from .module import * is a common way to re-export a package’s public names. Django has 36 of them, httpx 16, SQLAlchemy 15 and pydantic 10. Another 32 are in tests, and 6 are elsewhere: five in attrs’ re-export modules such as attrs/validators.py and one in Pillow’s build backend.
Wildcard imports make it harder to see where a name comes from, so they are a poor habit in application code. In a package’s __init__.py they are usually deliberate.
Why these three rules exist: two short demonstrations
The counts above say how often the patterns appear. These two runs show what they do when they go wrong. Both were run on Python 3.
A mutable default is shared between calls. We called the function from the example above three times with three different items:
print(add_item("a"))
print(add_item("b"))
print(add_item("c"))
['a']
['a', 'b']
['a', 'b', 'c']
Each call looks independent, yet the third returns all three items, because the default list is created once, when the function is defined. That is the whole bug, and why the tornado, black and SQLAlchemy cases above are safe only because the code copies or rebinds the default before using it.
A bare except catches the Ctrl-C. We wrote two functions that send the process a SIGINT, the same signal Ctrl-C produces, inside a try block. One catches with a bare except: and the other with except Exception::
bare(): returns "bare except swallowed the Ctrl-C"
narrow(): KeyboardInterrupt propagated to the caller
The bare version swallowed the interrupt and carried on. The narrow version let it through, as KeyboardInterrupt is not a subclass of Exception. In a long-running program that difference decides whether a user can stop it.
Fixing each one, and checking the fix
We put the three patterns, plus a cleanup handler that re-raises, into a small package and ran REPO-SIGHT over it. The analyser reported five findings: py-mutable-default-arg on line 1, py-bare-except on lines 8 and 14, and py-wildcard-import on both lines of a package’s __init__.py. Then we rewrote each one and scanned again, and the count dropped to zero.
- Mutable default: use
Noneas the default and create the list inside the function, as in the “one list per call” version above. For a dataclass, usefield(default_factory=list). - Bare except that swallows: name the exception you expect, or at least use
except Exception as exc:and do something withexc, such as log it or return a fallback on purpose. - Bare except that cleans up and re-raises: if the cleanup should run whether or not the code fails, use
tryandfinally. We did, and the finding disappeared. If the cleanup must run only on failure, the re-raise is doing a real job, and the finding is one you can leave with a comment saying why. The rule flags the syntax in both cases, as the first section explained. - Wildcard import: import the names you want and list them in
__all__. We replaced twofrom .core import *lines with explicit names and an__all__list, and the findings went away. The rule does flag wildcards in__init__.pyfiles, which is why 74 of the 112 findings above are there. In a package that is a deliberate choice, and an explicit list costs a few lines and makes the public interface visible.
A cleanup order that matches the data
- Swallowed exceptions in library code. There are only 3 in this data set, and they are the ones that can hide a real failure.
- Library handlers that return a fallback. There are 7. Check that each names the errors it expects.
- Mutable defaults that the function modifies. We found none among the library cases we read, but the same check applies to yours.
- Wildcards outside
__init__.py. They hide where a name comes from, and are the easiest to remove. - Everything in tests and fixtures. Last, and only if you want a clean report.
What to take from this
- Read the handler, not just the rule. Half of the bare excepts here either re-raise or sit in tests and fixtures.
- Mature projects still have a few. Even in code with heavy review, a small number of swallowed exceptions exist. They are the ones to clean up first.
- Treat the count as a prompt. A rule that fires hundreds of times is telling you where to look, not how bad things are.
What this data does and does not show
- It counts syntax, not bugs. The three rules flag patterns. For mutable defaults, the library cases we read did not modify the default. For the bare excepts, we did not test whether any swallowed error hid a real failure, and we make no claim about the projects’ correctness.
- The classification is ours. We read the handler at each of the 75 bare excepts and sorted it into the five groups above. Another reader might place a few differently.
- It is 18 projects at one commit each. The projects are well regarded and heavily reviewed. Do not read the figures as typical of Python code in general. The data file lists every project and commit.
Frequently asked questions
Is a bare except ever correct?
It is acceptable when the handler cleans up and re-raises, which is the most common use in this data set. It is a bug when it silently discards the error. Catching Exception is usually the better default, because it leaves KeyboardInterrupt and SystemExit alone.
Why does REPO-SIGHT report the re-raising ones too?
The rule flags the syntax, and the report shows the code so you can judge. See how to improve Python code quality for before-and-after fixes.
How do I check my own project?
Paste a GitHub URL to get a per-file list of findings. The Python code analyzer page covers what is checked.