Skip to content

AI CI Reviewer Setup

Wei Lin edited this page Feb 22, 2026 · 1 revision

AI CI Reviewer Setup (Issue #38)

This page documents the implementation of Custom AI CI Reviewer for the MiniPdf project (Issue #38).

Overview

MiniPdf uses a dual-layer AI code review system:

Layer Mechanism Trigger Result
Copilot Code Review GitHub built-in reviewer via Rulesets Every PR to main Review comments on PR
AI Security Scan (CI) Azure AI Foundry API in GitHub Actions Every PR to main ✓ Pass / ✗ Fail check

Architecture

PR opened/updated to main
        │
        ├──► CI: build job (dotnet build + test)
        │         │
        │         └──► CI: ai-security-scan job
        │               ├── Get .cs/.csproj diff
        │               ├── Send to Azure AI Foundry (GPT-4.1)
        │               └── ✓ PASS or ✗ FAIL
        │
        └──► Copilot Code Review (built-in)
              └── Review comments on PR conversation

1. Copilot Code Review (Built-in)

Setup

  1. Go to Settings → Copilot → Code review
  2. Enable "Use custom instructions when reviewing pull requests"
  3. Click "Create ruleset for default branch"

Custom Instructions

Custom review instructions are defined in .github/copilot-code-review.md:

  • Focus on .NET security and OWASP Top 10
  • Check for hardcoded secrets in application source code
  • Verify input validation and output encoding
  • Ensure secure file I/O patterns

Verification

# Check ruleset exists
gh api repos/shps951023/MiniPdf/rulesets --jq '.[].name'
# Expected: "Copilot review for default branch"

# Check PR reviews
gh pr view <PR_NUMBER> --json reviews --jq '.reviews[] | {author: .author.login, state: .state}'
# Expected: copilot-pull-request-reviewer, COMMENTED

2. AI Security Scan (CI Job)

How It Works

The ai-security-scan job in .github/workflows/ci.yml:

  1. Gets the PR diff — only .cs and .csproj files (source code only, not CI config or docs)
  2. Sends to Azure AI Foundry — with security-focused system prompt from .github/copilot-code-review.md
  3. Parses the result — AI returns {"passed": true/false, "issues": [...]}
  4. Pass or Fail — exits with code 0 (pass) or 1 (fail), shown as ✓ / ✗ in GitHub checks

Required GitHub Secrets

Secret Description Example
AZURE_OPENAI_API_KEY Azure AI Foundry API key abc123...
AZURE_OPENAI_ENDPOINT Azure endpoint (base URL only) https://miniaitx.openai.azure.com
AZURE_OPENAI_DEPLOYMENT Model deployment name gpt-4.1

Important: AZURE_OPENAI_ENDPOINT should be the base URL only, not the full API path. The CI script automatically appends /openai/deployments/{deployment}/chat/completions?api-version=2025-01-01-preview.

Set Secrets

Go to Settings → Secrets and variables → Actions → New repository secret

Troubleshooting

Problem: ai-pr-review job skipped on push

Cause: The job has if: github.event_name == 'pull_request'. Push events don't trigger it.
Solution: This is by design. Use PRs for all changes to main.

Problem: github/copilot-code-review-action not found

Cause: This action does not exist (or is private/deprecated).
Solution: Use GitHub's built-in Copilot Code Review via Rulesets instead (Settings → Copilot → Code review).

Problem: AI returns passed: false but CI shows pass

Cause: jq's // operator treats boolean false as falsy and falls through to the default.
Solution: Use explicit boolean check:

jq -r 'if .passed == true then "true" elif .passed == false then "false" else "unknown" end'

Problem: AI flags CI workflow secrets as security issues

Cause: The diff includes .yml workflow files where ${{ secrets.* }} is used — AI misinterprets this as secret exposure.
Solution: Only scan application source code files (.cs, .csproj), not CI configuration.

Problem: AZURE_OPENAI_ENDPOINT error

Cause: Endpoint set to full URL including path, causing double path.
Fix: Set endpoint to base URL only: https://your-resource.openai.azure.com

Timeline

Date Change
2026-02-22 Initial copilot-code-review-action (failed — action doesn't exist)
2026-02-22 Switched to built-in Copilot Code Review via Rulesets
2026-02-23 Added ai-security-scan CI job with Azure AI Foundry
2026-02-23 Fixed jq boolean parsing and diff scope

Related Files

Clone this wiki locally