Suppress noisy Alembic plugin setup logs#69916
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes CLI machine-readable output (e.g. --output json) being polluted by stdlib-logging output from dependencies (notably Alembic), by making [logging] namespace_levels apply consistently to both structlog and Python’s standard-library logging loggers. It also lowers the handler threshold when needed so per-namespace verbosity can be higher than the global level without being dropped.
Changes:
- Apply
namespace_levelsto stdlib loggers by setting per-logger levels afterdictConfig, and adjust the default handler level to the most verbose configured level so overrides can take effect. - Default
alembic.runtime.pluginstoWARNINGto suppress noisy plugin-registration logs during normal CLI commands. - Add regression tests for stdlib namespace filtering and the Alembic default; update config docs to reflect behavior and namespace hierarchy.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| shared/logging/src/airflow_shared/logging/structlog.py | Applies namespace levels to stdlib loggers, lowers handler threshold when required, and suppresses Alembic plugin logs by default. |
| shared/logging/tests/logging/test_structlog.py | Adds tests covering stdlib namespace filtering, allowing per-namespace verbosity below the global level, and the Alembic logger default. |
| airflow-core/src/airflow/config_templates/config.yml | Documents that namespace_levels applies to both structlog and stdlib logging and clarifies hierarchical namespace behavior. |
|
|
||
| logging.config.dictConfig(config) | ||
| for logger_name, level in parsed_namespace_log_levels.items(): | ||
| logging.getLogger(logger_name).setLevel(level) |
There was a problem hiding this comment.
logging.getLogger(logger_name).setLevel(level) sets a stdlib logger level directly on the global logger object, outside dictConfig. Once set, it stays set on that logger for the rest of the process & a later configure_logging() call that does not mention that same logger name will not reset it back. Seems like a quirk we should be aware about or handle in dictConfig?
There was a problem hiding this comment.
I was 90% sure the namespace logging working for stdlib logging already without needing this change? Did I make that up
There was a problem hiding this comment.
I have refocused this PR to just silence the chatty plugin.
The second issue I tried fixing can be seen when you run AIRFLOW__LOGGING__NAMESPACE_LEVELS=alembic.runtime.plugins=WARNING airflow dags list --columns dag_id,fileloc,owners,is_paused --output json in a branch not having this fix
|
Wrong fix IMO -- why is |
Alembic emits autogenerate plugin registration messages at INFO during CLI startup. Those unrelated lines contaminate machine-readable command output, while warnings and errors still need to remain visible.
8c4fb25 to
c994939
Compare
Alembic emits autogenerate plugin registration messages at INFO during CLI startup. Those unrelated lines contaminate machine-readable command output, while warnings and errors still need to remain visible.
closes: #69911
Was generative AI tooling used to co-author this PR?
GPT-5.6