Skip to content

Suppress noisy Alembic plugin setup logs#69916

Open
ephraimbuddy wants to merge 1 commit into
apache:mainfrom
astronomer:fix-stdlib-namespace-levels
Open

Suppress noisy Alembic plugin setup logs#69916
ephraimbuddy wants to merge 1 commit into
apache:mainfrom
astronomer:fix-stdlib-namespace-levels

Conversation

@ephraimbuddy

@ephraimbuddy ephraimbuddy commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Alembic emits autogenerate plugin registration messages at INFO during CLI startup. Those unrelated lines contaminate machine-readable command output, while warnings and errors still need to remain visible.

closes: #69911


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

GPT-5.6

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes CLI machine-readable output (e.g. --output json) being polluted by stdlib-logging output from dependencies (notably Alembic), by making [logging] namespace_levels apply consistently to both structlog and Python’s standard-library logging loggers. It also lowers the handler threshold when needed so per-namespace verbosity can be higher than the global level without being dropped.

Changes:

  • Apply namespace_levels to stdlib loggers by setting per-logger levels after dictConfig, and adjust the default handler level to the most verbose configured level so overrides can take effect.
  • Default alembic.runtime.plugins to WARNING to suppress noisy plugin-registration logs during normal CLI commands.
  • Add regression tests for stdlib namespace filtering and the Alembic default; update config docs to reflect behavior and namespace hierarchy.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
shared/logging/src/airflow_shared/logging/structlog.py Applies namespace levels to stdlib loggers, lowers handler threshold when required, and suppresses Alembic plugin logs by default.
shared/logging/tests/logging/test_structlog.py Adds tests covering stdlib namespace filtering, allowing per-namespace verbosity below the global level, and the Alembic logger default.
airflow-core/src/airflow/config_templates/config.yml Documents that namespace_levels applies to both structlog and stdlib logging and clarifies hierarchical namespace behavior.

Comment thread airflow-core/src/airflow/config_templates/config.yml

logging.config.dictConfig(config)
for logger_name, level in parsed_namespace_log_levels.items():
logging.getLogger(logger_name).setLevel(level)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logging.getLogger(logger_name).setLevel(level) sets a stdlib logger level directly on the global logger object, outside dictConfig. Once set, it stays set on that logger for the rest of the process & a later configure_logging() call that does not mention that same logger name will not reset it back. Seems like a quirk we should be aware about or handle in dictConfig?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was 90% sure the namespace logging working for stdlib logging already without needing this change? Did I make that up

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have refocused this PR to just silence the chatty plugin.

The second issue I tried fixing can be seen when you run AIRFLOW__LOGGING__NAMESPACE_LEVELS=alembic.runtime.plugins=WARNING airflow dags list --columns dag_id,fileloc,owners,is_paused --output json in a branch not having this fix

@ashb

ashb commented Jul 16, 2026

Copy link
Copy Markdown
Member

Wrong fix IMO -- why is alembic being imported so early?

Comment thread shared/logging/tests/logging/test_structlog.py Outdated
Alembic emits autogenerate plugin registration messages at INFO during CLI startup. Those unrelated lines contaminate machine-readable command output, while warnings and errors still need to remain visible.
@ephraimbuddy
ephraimbuddy force-pushed the fix-stdlib-namespace-levels branch from 8c4fb25 to c994939 Compare July 17, 2026 11:04
@ephraimbuddy ephraimbuddy changed the title Fix namespace_levels not applying to standard library loggers Suppress noisy Alembic plugin setup logs Jul 17, 2026
@ephraimbuddy
ephraimbuddy requested review from amoghrajesh and ashb July 17, 2026 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Alembic logs included in the stdout output when performing airflow cli commands

6 participants