Added python gRPC API bindings by chinmaychahar · Pull Request #618 · dragonflyoss/api

chinmaychahar · 2026-02-01T05:37:09Z

This PR adds codegen Python gRPC bindings for Dragonfly APIs

Description

Added generated Python code from existing protobuf definitions
Added pyproject.toml for packaging
Added basic README

Related Issue

Motivation and Context

This API dependency package with generated Python gRPC code based on the existing .protoc files serves as step 1 for the Python SDK design

gaius-qi

Please remove v2 path => api/*.py

Copilot

Pull request overview

This PR adds Python gRPC API bindings for Dragonfly by generating Python code from existing protobuf definitions. This serves as the foundation (step 1) for a future Python SDK.

Changes:

Added generated Python gRPC stub files from protobuf definitions for scheduler, manager, dfdaemon, common, and errordetails APIs
Added pyproject.toml for Python package configuration with setuptools backend
Added basic README documenting the package

Reviewed changes

Copilot reviewed 12 out of 20 changed files in this pull request and generated 24 comments.

Show a summary per file

File	Description
python/pyproject.toml	Package configuration defining dependencies (grpcio, protobuf) and build system
python/api/v2/*.py	Generated Python gRPC stubs for v2 API (scheduler, manager, dfdaemon, common, errordetails)
python/api/v2/pycache/*.pyc	Compiled Python bytecode files (should NOT be committed)
python/README.md	Basic documentation for the package
.gitignore	Added .venv/ pattern for Python virtual environments

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-02T02:41:17Z

python/api/v2/scheduler_pb2_grpc.py

+        raise NotImplementedError('Method not implemented!')
+
+    def DeleteCachePeer(self, request, context):
+        """DeletCacheePeer releases cache peer in scheduler.


Typo in docstring: "DeletCacheePeer" should be "DeleteCachePeer" (remove the extra 'e'). This is auto-generated code from the protobuf definition, so the typo exists in the source .proto file and should be fixed there before regenerating.

Suggested change

"""DeletCacheePeer releases cache peer in scheduler.

"""DeleteCachePeer releases cache peer in scheduler.

Copilot · 2026-02-02T02:41:17Z

python/api/manager_pb2_grpc.py

+        raise NotImplementedError('Method not implemented!')
+
+    def ListSeedPeers(self, request, context):
+        """List acitve schedulers configuration.


Typo in docstring: "acitve" should be "active". This appears twice (lines 98 and 133). This is auto-generated code from the protobuf definition, so the typo exists in the source .proto file and should be fixed there before regenerating.

Copilot · 2026-02-02T02:41:17Z

python/api/manager_pb2_grpc.py

+        raise NotImplementedError('Method not implemented!')
+
+    def ListSchedulers(self, request, context):
+        """List acitve schedulers configuration.


Typo in docstring: "acitve" should be "active". This is the second occurrence of the same typo. This is auto-generated code from the protobuf definition, so the typo exists in the source .proto file and should be fixed there before regenerating.

Copilot · 2026-02-02T02:41:18Z

python/README.md

+## Python Dragonfly API
+
+Generated Python gRPC bindings for Dragonfly.
+
+Source protos in https://github.com/dragonflyoss/api
+


The Python package is missing essential metadata and configuration files. Consider adding: 1) A MANIFEST.in file to ensure non-Python files (like README.md) are included in distributions, 2) Package metadata fields in pyproject.toml (authors, license, repository URL, etc.), 3) A way to specify which packages to include (either explicitly listing them or using [tool.setuptools.packages.find]). Without these, the package may not install or distribute correctly.

Copilot · 2026-02-02T02:41:18Z

python/pyproject.toml

+]
+
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"


The pyproject.toml is missing several important metadata fields for a proper Python package distribution: 'authors', 'readme', 'license', 'homepage', and 'repository'. Additionally, the package should specify 'packages' or use a proper package discovery mechanism to ensure the API modules are included in the distribution. Consider adding: authors = [{name = "...", email = "..."}], readme = "README.md", license = {text = "..."}, and [tool.setuptools.packages.find] to automatically discover packages.

Suggested change

]

[build-system]

requires = ["setuptools"]

build-backend = "setuptools.build_meta"

]

authors = [

{ name = "Your Name", email = "you@example.com" },

]

readme = "README.md"

license = { text = "Proprietary" }

[project.urls]

homepage = "https://example.com/dragonfly-api"

repository = "https://example.com/dragonfly-api/repository"

[build-system]

requires = ["setuptools"]

build-backend = "setuptools.build_meta"

[tool.setuptools.packages.find]

include = ["dragonfly_api*"]

Copilot · 2026-02-02T02:41:22Z

python/api/v2/scheduler_pb2.py

+
+
+from . import common_pb2 as common__pb2
+from . import errordetails_pb2 as errordetails__pb2


Import of 'errordetails__pb2' is not used.

Suggested change

from . import errordetails_pb2 as errordetails__pb2

Copilot · 2026-02-02T02:41:22Z

python/api/v2/scheduler_pb2.py

+
+from . import common_pb2 as common__pb2
+from . import errordetails_pb2 as errordetails__pb2
+from google.protobuf import duration_pb2 as google_dot_protobuf_dot_duration__pb2


Import of 'google_dot_protobuf_dot_duration__pb2' is not used.

Copilot · 2026-02-02T02:41:22Z

python/api/v2/scheduler_pb2.py

+from . import common_pb2 as common__pb2
+from . import errordetails_pb2 as errordetails__pb2
+from google.protobuf import duration_pb2 as google_dot_protobuf_dot_duration__pb2
+from google.protobuf import empty_pb2 as google_dot_protobuf_dot_empty__pb2


Import of 'google_dot_protobuf_dot_empty__pb2' is not used.

Copilot · 2026-02-02T02:41:23Z

python/api/v2/scheduler_pb2.py

+from . import errordetails_pb2 as errordetails__pb2
+from google.protobuf import duration_pb2 as google_dot_protobuf_dot_duration__pb2
+from google.protobuf import empty_pb2 as google_dot_protobuf_dot_empty__pb2
+from google.protobuf import timestamp_pb2 as google_dot_protobuf_dot_timestamp__pb2


Import of 'google_dot_protobuf_dot_timestamp__pb2' is not used.

Suggested change

from google.protobuf import timestamp_pb2 as google_dot_protobuf_dot_timestamp__pb2

Copilot · 2026-02-02T02:41:23Z

python/api/v2/scheduler_pb2_grpc.py

+# Generated by the gRPC Python protocol compiler plugin. DO NOT EDIT!
+"""Client and server classes corresponding to protobuf-defined services."""
+import grpc
+import warnings


Import of 'warnings' is not used.

Suggested change

import warnings

gaius-qi

Please generate code by namely/protoc-all, refer to https://github.com/dragonflyoss/api/blob/main/hack/protoc.sh.

chinmaychahar · 2026-02-12T07:47:10Z

Sure, on it @gaius-qi

chinmaychahar · 2026-02-12T07:47:50Z

Should I address all Copilot comments as well or https://github.com/dragonflyoss/api/blob/main/hack/protoc.sh would be enough?

chinmaychahar · 2026-03-04T12:00:18Z

Hey @gaius-qi I regenerated Python bindings using namely/protoc-all and flattened api/v2/ to api/ per your review.

Currently only v2 protos are generated. Happy to add v1 bindings if needed.

Signed-off-by: chinmaychahar <chinmay.cc.06@gmail.com>

Copilot

Pull request overview

Copilot reviewed 12 out of 14 changed files in this pull request and generated 9 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copilot · 2026-03-04T12:11:00Z

python/api/manager_pb2_grpc.py

+import grpc
+
+from google.protobuf import empty_pb2 as google_dot_protobuf_dot_empty__pb2
+from pkg.apis.manager.v2 import manager_pb2 as pkg_dot_apis_dot_manager_dot_v2_dot_manager__pb2


The generated stub imports pkg.apis.manager.v2.manager_pb2, but this PR adds python/api/manager_pb2.py instead of a pkg/apis/manager/v2/manager_pb2.py module. As a result, importing manager_pb2_grpc will raise ModuleNotFoundError unless the generated files are placed in the expected pkg/apis/... package hierarchy (or the generation options/imports are adjusted).

Suggested change

from pkg.apis.manager.v2 import manager_pb2 as pkg_dot_apis_dot_manager_dot_v2_dot_manager__pb2

from . import manager_pb2 as pkg_dot_apis_dot_manager_dot_v2_dot_manager__pb2

Copilot · 2026-03-04T12:11:00Z

python/api/manager_pb2.py

+
+
+from google.protobuf import empty_pb2 as google_dot_protobuf_dot_empty__pb2
+from pkg.apis.common.v2 import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2


This generated module imports pkg.apis.common.v2.common_pb2, but the PR places common_pb2.py under python/api/ rather than pkg/apis/common/v2/. As-is, import ...manager_pb2 will fail; the generated Python files need to live in a directory hierarchy matching the proto paths (or be regenerated with matching import paths).

Suggested change

from pkg.apis.common.v2 import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2

from . import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2

Copilot · 2026-03-04T12:11:00Z

python/api/manager_pb2_grpc.py

+        """List acitve schedulers configuration.
+        """


Typo in docstring: "acitve" should be "active".

Copilot · 2026-03-04T12:11:00Z

python/pyproject.toml

+]
+
+[build-system]
+requires = ["setuptools"]


setuptools is unpinned in build-system.requires. Since this project uses PEP 621 [project] metadata, older setuptools versions can fail to build; consider requiring setuptools>=61 (and typically include wheel as well) to make builds reproducible.

Suggested change

requires = ["setuptools"]

requires = ["setuptools>=61", "wheel"]

Copilot · 2026-03-04T12:11:01Z

python/api/scheduler_pb2_grpc.py

+from pkg.apis.common.v2 import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2
+from pkg.apis.scheduler.v2 import scheduler_pb2 as pkg_dot_apis_dot_scheduler_dot_v2_dot_scheduler__pb2


These imports reference modules under pkg.apis.* (e.g. pkg.apis.common.v2.common_pb2), but this PR places the generated files under python/api/*.py. With the current layout, importing this module will fail because the pkg/apis/... Python package hierarchy doesn’t exist. Either generate the Python outputs into a directory structure that matches the proto paths (so pkg/apis/.../*_pb2.py exists under the package root) or regenerate with import paths that match python/api.

Suggested change

from pkg.apis.common.v2 import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2

from pkg.apis.scheduler.v2 import scheduler_pb2 as pkg_dot_apis_dot_scheduler_dot_v2_dot_scheduler__pb2

from . import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2

from . import scheduler_pb2 as pkg_dot_apis_dot_scheduler_dot_v2_dot_scheduler__pb2

Copilot · 2026-03-04T12:11:01Z

python/api/dfdaemon_pb2_grpc.py

+from pkg.apis.common.v2 import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2
+from pkg.apis.dfdaemon.v2 import dfdaemon_pb2 as pkg_dot_apis_dot_dfdaemon_dot_v2_dot_dfdaemon__pb2


Like the other generated modules, this file imports pkg.apis.* modules that are not present under python/ in this PR (the generated code is in python/api). Unless the package layout is changed to include pkg/apis/... under the Python package root (or the code is regenerated with matching import paths), importing these stubs will fail at runtime.

Suggested change

from pkg.apis.common.v2 import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2

from pkg.apis.dfdaemon.v2 import dfdaemon_pb2 as pkg_dot_apis_dot_dfdaemon_dot_v2_dot_dfdaemon__pb2

from . import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2

from . import dfdaemon_pb2 as pkg_dot_apis_dot_dfdaemon_dot_v2_dot_dfdaemon__pb2

Copilot · 2026-03-04T12:11:01Z

python/api/manager_pb2_grpc.py

+    def ListSeedPeers(self, request, context):
+        """List acitve schedulers configuration.
+        """
+        context.set_code(grpc.StatusCode.UNIMPLEMENTED)


The ListSeedPeers method docstring says "List acitve schedulers configuration", which is both misspelled and describes the wrong resource (seed peers vs schedulers). Since this is generated code, the fix likely needs to be applied in the source .proto comments and then the stubs regenerated.

Copilot · 2026-03-04T12:11:02Z

python/api/dfdaemon_pb2_grpc.py

+        raise NotImplementedError('Method not implemented!')
+
+    def UpdatePersistentTask(self, request, context):
+        """UpdatePersistentTask updates metadate of thr persistent task in p2p network.


Typos in docstring: "metadate of thr" should be "metadata of the".

Suggested change

"""UpdatePersistentTask updates metadate of thr persistent task in p2p network.

"""UpdatePersistentTask updates metadata of the persistent task in p2p network.

Copilot · 2026-03-04T12:11:02Z

python/api/dfdaemon_pb2_grpc.py

+    def UpdatePersistentCacheTask(self, request, context):
+        """UpdatePersistentCacheTask updates metadate of thr persistent cache task in p2p network.
+        """


Typos in docstring: "metadate of thr" should be "metadata of the".

chinmaychahar · 2026-03-04T12:14:11Z

The files were autogenerated @gaius-qi , should I address all of Copilot's comments?

gaius-qi requested a review from Copilot February 2, 2026 02:35

Copilot started reviewing on behalf of gaius-qi February 2, 2026 02:35 View session

gaius-qi requested changes Feb 2, 2026

View reviewed changes

Copilot AI reviewed Feb 2, 2026

View reviewed changes

gaius-qi requested changes Feb 2, 2026

View reviewed changes

maychin and others added 2 commits March 4, 2026 17:32

Added python gRPC API bindings

054c56e

Signed-off-by: chinmaychahar <chinmay.cc.06@gmail.com>

regenerate python bindings using namely/protoc-all

0bc3fc1

Signed-off-by: chinmaychahar <chinmay.cc.06@gmail.com>

chinmaychahar force-pushed the python-grpc-api branch from a3ef700 to 0bc3fc1 Compare March 4, 2026 12:02

chinmaychahar requested review from Copilot and gaius-qi March 4, 2026 12:03

Copilot started reviewing on behalf of chinmaychahar March 4, 2026 12:03 View session

Copilot AI reviewed Mar 4, 2026

View reviewed changes

	"""DeletCacheePeer releases cache peer in scheduler.
	"""DeleteCachePeer releases cache peer in scheduler.

-]
-[build-system]
-requires = ["setuptools"]
-build-backend = "setuptools.build_meta"
+]
+authors = [
+  { name = "Your Name", email = "you@example.com" },
+]
+readme = "README.md"
+license = { text = "Proprietary" }
+[project.urls]
+homepage = "https://example.com/dragonfly-api"
+repository = "https://example.com/dragonfly-api/repository"
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"
+[tool.setuptools.packages.find]
+include = ["dragonfly_api*"]



		from . import common_pb2 as common__pb2
		from . import errordetails_pb2 as errordetails__pb2

	from pkg.apis.manager.v2 import manager_pb2 as pkg_dot_apis_dot_manager_dot_v2_dot_manager__pb2
	from . import manager_pb2 as pkg_dot_apis_dot_manager_dot_v2_dot_manager__pb2



		from google.protobuf import empty_pb2 as google_dot_protobuf_dot_empty__pb2
		from pkg.apis.common.v2 import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2

	from pkg.apis.common.v2 import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2
	from . import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2

	requires = ["setuptools"]
	requires = ["setuptools>=61", "wheel"]

		from pkg.apis.common.v2 import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2
		from pkg.apis.scheduler.v2 import scheduler_pb2 as pkg_dot_apis_dot_scheduler_dot_v2_dot_scheduler__pb2

		from pkg.apis.common.v2 import common_pb2 as pkg_dot_apis_dot_common_dot_v2_dot_common__pb2
		from pkg.apis.dfdaemon.v2 import dfdaemon_pb2 as pkg_dot_apis_dot_dfdaemon_dot_v2_dot_dfdaemon__pb2

	"""UpdatePersistentTask updates metadate of thr persistent task in p2p network.
	"""UpdatePersistentTask updates metadata of the persistent task in p2p network.

Conversation

chinmaychahar commented Feb 1, 2026

Description

Related Issue

Motivation and Context

Uh oh!

gaius-qi left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

gaius-qi left a comment

Choose a reason for hiding this comment

Uh oh!

chinmaychahar commented Feb 12, 2026

Uh oh!

chinmaychahar commented Feb 12, 2026

Uh oh!

chinmaychahar commented Mar 4, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment