Skip to content

feat: implement validate API key check endpoint and GCS verification#616

Open
yiyche wants to merge 2 commits into
datacommonsorg:masterfrom
yiyche:feature/validate-api-key
Open

feat: implement validate API key check endpoint and GCS verification#616
yiyche wants to merge 2 commits into
datacommonsorg:masterfrom
yiyche:feature/validate-api-key

Conversation

@yiyche

@yiyche yiyche commented Jul 6, 2026

Copy link
Copy Markdown
Member

mainly added

  • api router file: pipeline/workflow/ingestion-helper/routes/ingestion.py (POST /ingestion/start)
  • api key validation: tests DC_API_KEY against https://api.datacommons.org/v2/node before job launch to fail fast on 401/403.
  • aditional GCS artifact verification: ensure bucket + file combo exists
  • check for already running ingestion jobs (comment on PR under DCP repo)

@yiyche yiyche requested a review from dwnoble July 6, 2026 18:22
@codacy-production

codacy-production Bot commented Jul 6, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 50 complexity

Metric Results
Complexity 50

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new ingestion router with /start and /config endpoints to validate, trigger, and inspect Cloud Run ingestion jobs. It also adds GCS bucket and file existence checks to the storage client, updates dependencies, and includes comprehensive unit tests. The review feedback focuses on improving robustness and error handling: guarding against potential AttributeErrors when auto-discovering job configurations, logging warnings for failed Cloud Run API requests, and updating error messages to account for permission issues during GCS bucket and file checks.

Comment thread pipeline/workflow/ingestion-helper/routes/ingestion.py Outdated
Comment thread pipeline/workflow/ingestion-helper/routes/ingestion.py
Comment thread pipeline/workflow/ingestion-helper/routes/ingestion.py
Comment thread pipeline/workflow/ingestion-helper/routes/ingestion.py Outdated
@yiyche yiyche force-pushed the feature/validate-api-key branch 4 times, most recently from 911ebb5 to b774d21 Compare July 9, 2026 04:51
Comment thread pipeline/workflow/ingestion-helper/routes/ingestion.py
Comment thread pipeline/workflow/ingestion-helper/routes/ingestion.py Outdated
Comment thread pipeline/workflow/ingestion-helper/routes/ingestion.py
Comment thread pipeline/workflow/ingestion-helper/routes/ingestion.py Outdated
Comment thread pipeline/workflow/ingestion-helper/routes/ingestion.py Outdated
@yiyche yiyche force-pushed the feature/validate-api-key branch from daf019b to 9add35d Compare July 10, 2026 18:18
@yiyche yiyche requested a review from dwnoble July 10, 2026 18:26
@yiyche yiyche enabled auto-merge (squash) July 10, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants