-
-
Notifications
You must be signed in to change notification settings - Fork 18
AI CI Reviewer Setup
This page documents the implementation of Custom AI CI Reviewer for the MiniPdf project (Issue #38).
MiniPdf uses a dual-layer AI code review system:
| Layer | Mechanism | Trigger | Result |
|---|---|---|---|
| Copilot Code Review | GitHub built-in reviewer via Rulesets | Every PR to main
|
Review comments on PR |
| AI Security Scan (CI) | Azure AI Foundry API in GitHub Actions | Every PR to main
|
✓ Pass / ✗ Fail check |
PR opened/updated to main
│
├──► CI: build job (dotnet build + test)
│ │
│ └──► CI: ai-security-scan job
│ ├── Get .cs/.csproj diff
│ ├── Send to Azure AI Foundry (GPT-4.1)
│ └── ✓ PASS or ✗ FAIL
│
└──► Copilot Code Review (built-in)
└── Review comments on PR conversation
- Go to Settings → Copilot → Code review
- Enable "Use custom instructions when reviewing pull requests"
- Click "Create ruleset for default branch"
Custom review instructions are defined in .github/copilot-code-review.md:
- Focus on .NET security and OWASP Top 10
- Check for hardcoded secrets in application source code
- Verify input validation and output encoding
- Ensure secure file I/O patterns
# Check ruleset exists
gh api repos/shps951023/MiniPdf/rulesets --jq '.[].name'
# Expected: "Copilot review for default branch"
# Check PR reviews
gh pr view <PR_NUMBER> --json reviews --jq '.reviews[] | {author: .author.login, state: .state}'
# Expected: copilot-pull-request-reviewer, COMMENTEDThe ai-security-scan job in .github/workflows/ci.yml:
-
Gets the PR diff — only
.csand.csprojfiles (source code only, not CI config or docs) -
Sends to Azure AI Foundry — with security-focused system prompt from
.github/copilot-code-review.md -
Parses the result — AI returns
{"passed": true/false, "issues": [...]} - Pass or Fail — exits with code 0 (pass) or 1 (fail), shown as ✓ / ✗ in GitHub checks
| Secret | Description | Example |
|---|---|---|
AZURE_OPENAI_API_KEY |
Azure AI Foundry API key | abc123... |
AZURE_OPENAI_ENDPOINT |
Azure endpoint (base URL only) | https://miniaitx.openai.azure.com |
AZURE_OPENAI_DEPLOYMENT |
Model deployment name | gpt-4.1 |
Important:
AZURE_OPENAI_ENDPOINTshould be the base URL only, not the full API path. The CI script automatically appends/openai/deployments/{deployment}/chat/completions?api-version=2025-01-01-preview.
Go to Settings → Secrets and variables → Actions → New repository secret
Cause: The job has if: github.event_name == 'pull_request'. Push events don't trigger it.
Solution: This is by design. Use PRs for all changes to main.
Cause: This action does not exist (or is private/deprecated).
Solution: Use GitHub's built-in Copilot Code Review via Rulesets instead (Settings → Copilot → Code review).
Cause: jq's // operator treats boolean false as falsy and falls through to the default.
Solution: Use explicit boolean check:
jq -r 'if .passed == true then "true" elif .passed == false then "false" else "unknown" end'Cause: The diff includes .yml workflow files where ${{ secrets.* }} is used — AI misinterprets this as secret exposure.
Solution: Only scan application source code files (.cs, .csproj), not CI configuration.
Cause: Endpoint set to full URL including path, causing double path.
Fix: Set endpoint to base URL only: https://your-resource.openai.azure.com
| Date | Change |
|---|---|
| 2026-02-22 | Initial copilot-code-review-action (failed — action doesn't exist) |
| 2026-02-22 | Switched to built-in Copilot Code Review via Rulesets |
| 2026-02-23 | Added ai-security-scan CI job with Azure AI Foundry |
| 2026-02-23 | Fixed jq boolean parsing and diff scope |
-
.github/workflows/ci.yml— CI workflow withai-security-scanjob -
.github/copilot-code-review.md— AI review custom instructions -
AGENTS.md— Project-level agent instructions