Use a logger, fool.

It’s basically just code that routes to different outputs based on what type of message you pass in. You can route them out to files and do all sorts of fancy stuff. At a minimum, though, they’ll clean up your console and set up future-you for success.

Hey, look. I’m logging!

>>> import logging
>>> logging.basicConfig(level=logging.WARNING)
>>> logging.debug('hi')
>>> logging.info('hi')
>>> logging.warning('hi')
WARNING:root:hi
>>> logging.error('hi')
ERROR:root:hi
>>> logging.critical('hi')
CRITICAL:root:hi

Only messages that pass a threshold go through.

Environmental Logging

It’s best to set log level based on an environment variable.

import logging
import os

LOG_LEVEL = os.environ.get("LOG_LEVEL", "WARNING").upper()
logging.basicConfig(level=LOG_LEVEL, format="%(asctime)s %(message)s")

Now, when you run the code, you can change the output without touching the dang code.

~/d/helpers (master↑4|●2) $ set -x LOG_LEVEL ERROR
~/d/helpers (master↑4|●2) $ ./run.sh analysis-only
PASS: resources_by_date / happy_path
PASS: resources_by_date / matches_expectation
PASS: resources_by_date / partial_match
PASS: resources_by_date / matches_expectation_full
PASS: resources_by_date / xxx-696
PASS: resources_by_date / xxx-700
PASS: resources_by_date / xxx-695


# Hrm... let's debug something.

~/d/helpers (master↑4|●2) $ set -x LOG_LEVEL dEbUG
~/d/helpers (master↑4|●2) $ ./run.sh analysis-only
2022-05-03 14:22:05,443
Analyzing results from x_resources_by_date / happy_path (AnalysisType.XXXS_IN_FILES)
2022-05-03 14:22:05,445 inputs:
2022-05-03 14:22:05,445 /Users/xxx/code/pi/test_data/xxx-656_ci-test/test_case_results/x_resources_by_date/happy_path/inputs/patient/member_infos.parquet
/Users/xxx/code/pi/test_data/xxx-656_ci-test/test_case_results/x_resources_by_date/happy_path/inputs/claims/claim.parquet
2022-05-03 14:22:05,445 outputs:
2022-05-03 14:22:05,445 /Users/xxx/code/pi/test_data/xxx-656_ci-test/test_case_results/x_resources_by_date/happy_path/outputs/12345/claim/Claim_x/1990_01_01.json.gz
2022-05-03 14:22:05,480
Analyzing results from x_resources_by_date / matches_expectation (AnalysisType.OUTPUTS_MATCH_FIXTURES)
2022-05-03 14:22:05,481 inputs:
2022-05-03 14:22:05,481 /Users/xxx/code/pi/test_data/xxx-656_ci-test/test_case_results/x_resources_by_date/matches_expectation/inputs/claims/claim.parquet
2022-05-03 14:22:05,481 outputs:
2022-05-03 14:22:05,481 /Users/xxx/code/pi/test_data/xxx-656_ci-test/test_case_results/x_resources_by_date/matches_expectation/outputs/12345/claim/Claim_x/1990_01_01.json.gz
2022-05-03 14:22:05,482 * input[0]
2022-05-03 14:22:05,483