Skip to content

Bump azure-mgmt-datafactory to 10.0 and fix compat#69901

Closed
amoghrajesh wants to merge 1 commit into
apache:mainfrom
amoghrajesh:bump-azure-10
Closed

Bump azure-mgmt-datafactory to 10.0 and fix compat#69901
amoghrajesh wants to merge 1 commit into
apache:mainfrom
amoghrajesh:bump-azure-10

Conversation

@amoghrajesh

Copy link
Copy Markdown
Contributor

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Noticed this failure in CI:

    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:280: error: No overload variant of "create_or_update" of "FactoriesOperations" matches argument types "str", "str",
    "Factory", "str | None", "dict[str, Any]"  [call-overload]
                return self.get_conn().factories.create_or_update(
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:280: note: Possible overload variants:
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:280: note:     def create_or_update(self, resource_group_name: str, factory_name: str, factory: Factory, *, content_type: str = ..., etag: str | None = ..., match_condition: MatchConditions | None = ..., **kwargs: Any) -> Factory
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:280: note:     def create_or_update(self, resource_group_name: str, factory_name: str, factory: IO[bytes], *, content_type: str = ..., etag: str | None = ..., match_condition: MatchConditions | None = ..., **kwargs: Any) -> Factory
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:341: error: Too many positional arguments for "get" of "LinkedServicesOperations"  [misc]
                return self.get_conn().linked_services.get(
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:551: error: Too many positional arguments for "get" of "DataFlowsOperations"  [misc]
                return self.get_conn().data_flows.get(
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:599: error: No overload variant of "create_or_update" of "DataFlowsOperations" matches argument types "str", "str",
    "str", "DataFlowResource | IO[Any]", "str | None", "dict[str, Any]"  [call-overload]
                return self.get_conn().data_flows.create_or_update(
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:599: note: Possible overload variants:
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:599: note:     def create_or_update(self, resource_group_name: str, factory_name: str, data_flow_name: str, data_flow: DataFlowResource, *, content_type: str = ..., etag: str | None = ..., match_condition: MatchConditions | None = ..., **kwargs: Any) -> DataFlowResource
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:599: note:     def create_or_update(self, resource_group_name: str, factory_name: str, data_flow_name: str, data_flow: IO[bytes], *, content_type: str = ..., etag: str | None = ..., match_condition: MatchConditions | None = ..., **kwargs: Any) -> DataFlowResource
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:629: error: No overload variant of "create_or_update" of "DataFlowsOperations" matches argument types "str", "str",
    "str", "DataFlowResource", "str | None", "dict[str, Any]"  [call-overload]
                return self.get_conn().data_flows.create_or_update(
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:629: note: Possible overload variants:
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:629: note:     def create_or_update(self, resource_group_name: str, factory_name: str, data_flow_name: str, data_flow: DataFlowResource, *, content_type: str = ..., etag: str | None = ..., match_condition: MatchConditions | None = ..., **kwargs: Any) -> DataFlowResource
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:629: note:     def create_or_update(self, resource_group_name: str, factory_name: str, data_flow_name: str, data_flow: IO[bytes], *, content_type: str = ..., etag: str | None = ..., match_condition: MatchConditions | None = ..., **kwargs: Any) -> DataFlowResource
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:925: error: No overload variant of "create_or_update" of "TriggersOperations" matches argument types "str", "str",
    "str", "TriggerResource", "str | None", "dict[str, Any]"  [call-overload]
                return self.get_conn().triggers.create_or_update(
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:925: note: Possible overload variants:
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:925: note:     def create_or_update(self, resource_group_name: str, factory_name: str, trigger_name: str, trigger: TriggerResource, *, content_type: str = ..., etag: str | None = ..., match_condition: MatchConditions | None = ..., **kwargs: Any) -> TriggerResource
    providers/microsoft/azure/src/airflow/providers/microsoft/azure/hooks/data_factory.py:925: note:     def create_or_update(self, resource_group_name: str, factory_name: str, trigger_name: str, trigger: IO[bytes], *, content_type: str = ..., etag: str | None = ..., match_condition: MatchConditions | None = ..., **kwargs: Any) -> TriggerResource
    Found 6 errors in 1 file (checked 4724 source files)

    Error 1 returned

Caused due to

116c116
< azure-mgmt-datafactory==9.3.0
---
> azure-mgmt-datafactory==10.0.0

The 10.0.0 release regenerated the SDK's operations classes and dropped the positional if_match/if_none_match parameters on create_or_update/get in favor of a keyword-only etag parameter (the standard azure-core conditional-request pattern, where etag maps to If-Match on writes and If-None-Match on reads).

Instead of band-aiding it, a better way would be to bump to 10.0+ lower bound cos current lower bound is ~5years old
image

User implications / backcompat

None for Airflow users - the hook's public API (if_match/if_none_match parameter names on update_factory, get_linked_service, get_dataflow, update_dataflow, create_dataflow, update_trigger) is unchanged. Only the internal SDK call shape and the minimum installed SDK version changed.


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@Lee-W

Lee-W commented Jul 15, 2026

Copy link
Copy Markdown
Member

There's already a #69800 fixing the same issue. That one also handles the match_condition part https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/datafactory/azure-mgmt-datafactory/CHANGELOG.md#breaking-changes

@eladkal

eladkal commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Suppressed by #69800

@eladkal eladkal closed this Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants