Magic

At work, we have this goofy Python that creates “magic” CLI args. Nowhere in the codebase are the arguments defined. If you run program.py --help you get the empty argparse message.

The “magic” happens in a dependency program we pull in (also ours) that interrogates one of the class methods program runs and builds a bunch of args at runtime.

Yes, this is bad, but that’s not why we’re here.

A Twist

The program codebase also comes with a bunch of config files in various formats that contain code to execute program. These are not actually part of program, but the live in its repo.

If a developer changes the innocuous method mentioned above, the arguments required to run program will change at runtime, and all the config files will be wrong.

Complaining

Yes, I did that, but it didn’t work.

I also checked my years of first hand experience against the Python Universe to see if this was just “pythonic.”

In [1]: import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.

Plan B

I was not at liberty to change the imported program with the magic, so I whipped up a bunch of tests in program that check the mystery method above, use its output and some hard-coded magic-knowledge to generate expectations on what is required from the CLI args.

Then I greps through the codebase to figure out which configs pertain to this, imports and parses them (including some homebrewed jsonette hacky parsing), and figures out if each one has the correct magic args.

This sucked, but it sucked so much less than all the magic happening silently and breakages not discovered until runtime.

Plan C(lean up that hardcoded garbage)

If the imported program with the magic changed, we were screwed. We were looking into the black box so hard. To fix that, another merge request (Yes, Gitlab. You’ve finally broken me down.), I added a test that ignored any knowledge obtained from inside the black box. All it did was try to run program with the args pulled from each config. There was a big fat try...except with a few targeted exception types around it.

This worked quite well… EXCEPT… argparse didn’t raise its custom exception all the way to userland. The magic program appeared to catch it and in turn raise a SystemExit exception. To show something useful to the user, that meant we had to use contextlib.redirect_stderr and then relay that to our test.

Punchline

Anywho, it’s not perfect, but it’s several steps closer to it than when I first ran into this. The most notable thing, though, and why I wrote this, was a final curveball at the end.

I wanted to demonstrate how awesome this was with a little demo video for the team. Tests running in one terminal, Neovim in the other with those hard to read configs. I made a change and the tests were still green!

What the hot fuck?

I’m a TDD guy, but I spend so much time retrofitting tests to lazy bastards’ code that stuff like this happens. It sucks, but it didn’t make any sense!

Then I changed something else and the tests were red (as expected)!

I kept notes on what worked and what didn’t and eventually a pattern emerged. Then I felt like a real turd.

TIL

argparse does a matching to allow abbreviations by default.