Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions .github/workflows/apply-gallery-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,18 @@ jobs:
contains(github.event.comment.body, 'Reject:')
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: write
pull-requests: write
Comment on lines +22 to +24
env:
PR_NUMBER: ${{ github.event.issue.number }}
REVIEW_COMMAND: ${{ github.event.comment.body }}
steps:
- name: Authorize commenter
env:
GH_TOKEN: ${{ secrets.GALLERY_UPDATE_TOKEN }}
GH_TOKEN: ${{ github.token }}
COMMENTER: ${{ github.event.comment.user.login }}
run: |
test -n "$GH_TOKEN" || { echo "GALLERY_UPDATE_TOKEN is required."; exit 1; }
permission=$(gh api "repos/$GITHUB_REPOSITORY/collaborators/$COMMENTER/permission" --jq .permission)
case "$permission" in
admin|maintain|write) ;;
Expand All @@ -42,9 +44,8 @@ jobs:

- name: Validate command target
env:
GH_TOKEN: ${{ secrets.GALLERY_UPDATE_TOKEN }}
GH_TOKEN: ${{ github.token }}
run: |
test -n "$GH_TOKEN" || { echo "GALLERY_UPDATE_TOKEN is required."; exit 1; }
head_ref=$(gh pr view "$PR_NUMBER" --repo "$GITHUB_REPOSITORY" --json headRefName --jq .headRefName)
base_ref=$(gh pr view "$PR_NUMBER" --repo "$GITHUB_REPOSITORY" --json baseRefName --jq .baseRefName)
head_repo=$(gh api "repos/$GITHUB_REPOSITORY/pulls/$PR_NUMBER" --jq .head.repo.full_name)
Expand Down Expand Up @@ -72,7 +73,7 @@ jobs:
- name: Apply rejected items
id: apply
env:
GH_TOKEN: ${{ secrets.GALLERY_UPDATE_TOKEN }}
GH_TOKEN: ${{ github.token }}
run: |
PR_BODY="$(gh pr view "$PR_NUMBER" --repo "$GITHUB_REPOSITORY" --json body --jq .body)" \
BASE_CATALOG_PATH="$RUNNER_TEMP/base-templates.json" \
Expand Down Expand Up @@ -106,7 +107,7 @@ jobs:

- name: Publish revised proposal
env:
GH_TOKEN: ${{ secrets.GALLERY_UPDATE_TOKEN }}
GH_TOKEN: ${{ github.token }}
run: |
if git diff --quiet -- static/templates.json static/retired-templates.json; then
gh pr comment "$PR_NUMBER" --repo "$GITHUB_REPOSITORY" --body "The requested items were already absent from the proposal."
Expand Down
21 changes: 11 additions & 10 deletions .github/workflows/audit-gallery-content.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,10 @@ jobs:
if: github.event_name != 'pull_request'
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: write
pull-requests: read
copilot-requests: write
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
with:
Expand Down Expand Up @@ -121,12 +125,10 @@ jobs:
node -e "const fs=require('fs');const value=fs.readFileSync(process.env.RUNNER_TEMP + '/copilot-skills.json','utf8');for(const name of ['cosmosdb-best-practices','humanizer'])if(!value.includes(name))throw new Error('Required Copilot skill unavailable: '+name)"

- name: Classify and promote high-confidence updates
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
if [ "${{ secrets.COPILOT_GITHUB_TOKEN != '' }}" != "true" ]; then
echo "Copilot classification skipped: COPILOT_GITHUB_TOKEN is not configured."
exit 0
fi
COPILOT_GITHUB_TOKEN="${{ secrets.COPILOT_GITHUB_TOKEN }}" node scripts/gallery-audit/index.mjs --classify-only --promote
node scripts/gallery-audit/index.mjs --classify-only --promote

- name: Validate proposed catalog
if: hashFiles('output/gallery-content-review/promotion-result.json') != ''
Expand All @@ -144,14 +146,12 @@ jobs:
echo "catalog=true" >> "$GITHUB_OUTPUT"
fi

- name: Publish draft pull request
- name: Publish proposal branch
if: success()
env:
GH_TOKEN: ${{ secrets.GALLERY_UPDATE_TOKEN }}
GH_TOKEN: ${{ github.token }}
UPDATE_BRANCH: automation/gallery-content-updates-${{ github.run_id }}-${{ github.run_attempt }}
NOTIFY_REVIEWER: jagord_microsoft
run: |
test -n "$GH_TOKEN" || { echo "GALLERY_UPDATE_TOKEN is required to publish the draft pull request."; exit 1; }
gh auth setup-git

if [ ! -f output/gallery-content-review/promotion-result.json ]; then
Expand All @@ -166,7 +166,8 @@ jobs:
git add static/templates.json static/retired-templates.json
git commit -m "chore: refresh gallery content"
git push origin "HEAD:$UPDATE_BRANCH"
gh pr create --draft --base main --head "$UPDATE_BRANCH" --reviewer "$NOTIFY_REVIEWER" --title "chore: refresh gallery content" --body-file output/gallery-content-review/promotion-summary.md
compare_url="https://github.com/$GITHUB_REPOSITORY/compare/main...$UPDATE_BRANCH?expand=1"
printf 'Proposal branch published. Create a draft pull request: %s\n' "$compare_url" >> "$GITHUB_STEP_SUMMARY"

- name: Add summary
if: always()
Expand Down
83 changes: 21 additions & 62 deletions docs/automated-gallery-maintenance.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,70 +32,28 @@ Each run creates a proposal from `main` on a new branch. Rejection commits there

## Required repository configuration

Create these Actions secrets before enabling scheduled promotion:
The maintenance workflows use the built-in Actions `GITHUB_TOKEN`; they require no authentication secrets.

| Secret | Purpose | Minimum access |
| Job | Scoped access |
| --- | --- | --- |
| `COPILOT_GITHUB_TOKEN` | Authenticates the pinned GitHub Copilot CLI used for classification | Active Copilot subscription, Copilot Requests account permission, public repository access only |
| `GALLERY_UPDATE_TOKEN` | Authorizes review commenters, pushes the automation branch, and creates or edits its draft PR | Fine-grained repository token with Administration read, Contents read/write, and Pull requests read/write |
| Scheduled/manual audit | Contents write, Pull requests read, Copilot requests write |
| Pull-request validation | Contents read, Pull requests read |
| Review-command application | Contents write, Pull requests write |

Use a maintained automation identity where organizational policy permits. Record the owner and rotation date outside the repository. Rotate either token immediately when its owner changes or access is suspected to be compromised.
Keep the repository default workflow permission read-only. Write permissions are granted only to jobs that publish or revise a proposal branch. The workflows never create, approve, or merge pull requests.

The workflow's normal `GITHUB_TOKEN` remains read-only. Write credentials are exposed only to the final draft-PR publication step.
### Enable the required repository settings

### Create the required tokens
An `AzureCosmosDB` owner or repository administrator must:

Create two separate fine-grained personal access tokens. They cannot be combined because **Copilot Requests** is available only when the token's resource owner is the user's personal account, while catalog publication needs repository permissions under the `AzureCosmosDB` organization.
1. Open **Settings > Actions > General > Workflow permissions**.
2. Keep **Read repository contents and packages permissions** as the default.
3. Keep **Allow GitHub Actions to create and approve pull requests** disabled, as required by enterprise policy.
4. Confirm the organization enables **Allow use of Copilot CLI billed to the organization**.

#### Create `COPILOT_GITHUB_TOKEN`
The job-scoped `permissions` blocks remain the source of least privilege. Do not add a PAT, GitHub App private key, or publisher secret as a fallback.

Use a maintained automation identity that has an active GitHub Copilot subscription and is allowed to use Copilot CLI by the organization or enterprise policy.

1. Sign in as the automation identity and open **Settings > Developer settings > Personal access tokens > Fine-grained tokens > Generate new token**, or open the [fine-grained token form](https://github.com/settings/personal-access-tokens/new).
2. Set a descriptive name such as `AzureCosmosDB gallery Copilot classification` and choose the shortest practical expiration allowed by policy.
3. For **Resource owner**, select the automation identity's **personal account**, not `AzureCosmosDB`.
4. For **Repository access**, select **Public repositories**. Classification reads only this public repository and does not need repository write access.
5. Under **Permissions > Account permissions**, add **Copilot Requests**. Do not add repository or organization write permissions.
6. Generate the token and copy it immediately; GitHub displays the value only once.

Copilot requests consume the token owner's Copilot premium-request allowance. The token fails if that identity loses its Copilot subscription or if Copilot CLI is disabled by organization or enterprise policy.

#### Create `GALLERY_UPDATE_TOKEN`

Use a maintained automation identity that is a member of `AzureCosmosDB` and has permission to create branches and pull requests in `AzureCosmosDB/gallery`.

1. From the same fine-grained token page, select **Generate new token**.
2. Set a descriptive name such as `AzureCosmosDB gallery maintenance publisher` and choose the shortest practical expiration allowed by policy.
3. For **Resource owner**, select **AzureCosmosDB**.
4. If prompted, enter a justification describing the weekly gallery-maintenance draft pull request workflow.
5. For **Repository access**, select **Only select repositories**, then select **gallery**.
6. Under **Repository permissions**, grant only:
- **Administration: Read-only**
- **Contents: Read and write**
- **Pull requests: Read and write**
7. Leave all other repository and organization permissions at their defaults. Administration read is used only to verify that a review-command commenter has write, maintain, or admin access. The token does not need Actions, Workflows, Pages, or permission to bypass branch protection.
8. Generate the token and copy it immediately.
9. If organization policy marks it `pending`, an `AzureCosmosDB` owner must approve it before the workflow can publish a branch or draft pull request. Until approval, it can read only public resources.

#### Add the repository secrets

An administrator with repository write access must add both values to `AzureCosmosDB/gallery`:

1. Open **Settings > Secrets and variables > Actions > Secrets**.
2. Select **New repository secret**.
3. Add the Copilot token with the exact name `COPILOT_GITHUB_TOKEN`.
4. Add the publisher token with the exact name `GALLERY_UPDATE_TOKEN`.
5. Never paste either value into an issue, pull request, workflow input, command argument, or log.

GitHub CLI can install the secrets without exposing them in shell history; each command prompts for the value:

```shell
gh secret set COPILOT_GITHUB_TOKEN --repo AzureCosmosDB/gallery
gh secret set GALLERY_UPDATE_TOKEN --repo AzureCosmosDB/gallery
gh secret list --repo AzureCosmosDB/gallery
```

`gh secret list` confirms only that the names exist. Validate the values by manually running **Audit gallery content**: classification must report `complete`, and an eligible proposal must be able to create or update the draft maintenance pull request.
Validate the configuration by manually running **Audit gallery content**: classification must report `complete`, and an eligible proposal must publish an automation branch and compare link. Open that link and create the draft pull request as a person.

### Configure the new-PR email notification

Expand All @@ -115,8 +73,8 @@ This uses GitHub's notification system and requires no SMTP credentials or mail-
An organization owner or repository administrator must complete these steps after this pull request is merged:

1. Approve GitHub Actions for the repository and allow the pinned actions used by the gallery workflows.
2. Create and install `COPILOT_GITHUB_TOKEN` and `GALLERY_UPDATE_TOKEN` by following the preceding token procedures. Confirm the organization-owned publisher token is approved before the first run.
3. Confirm the automation identity can use GitHub Copilot CLI and can create branches and pull requests in `AzureCosmosDB/gallery`.
2. Keep default workflow permissions read-only and keep Actions-created pull requests disabled.
3. Confirm **Allow use of Copilot CLI billed to the organization** is enabled.
4. Create and keep an active `Protect main` ruleset enabled with at least one approving review, stale approval dismissal, approval after the latest push, and required resolution of review threads. The audit job verifies these settings and fails before promotion when they are absent. Neither token needs permission to bypass it.
5. Configure `jagord_microsoft` to route `AzureCosmosDB` review-request notifications to `jagord@microsoft.com`.
6. Open **Actions > Audit gallery content**, choose **Run workflow**, and review the artifact and draft pull request from the first complete run.
Expand Down Expand Up @@ -176,14 +134,15 @@ Each maintenance run uploads `gallery-content-review-<run-id>` containing:
| Condition | Expected behavior | Maintainer action |
| --- | --- | --- |
| One source times out, rate limits, truncates, or returns malformed data | Source and run are marked partial; no promotion | Inspect the artifact, retry later, then fix or disable the source if persistent |
| Copilot token is missing | Classification is skipped; existing draft is preserved | Restore the secret and rerun manually |
| Copilot organization policy blocks Actions | Classification is incomplete; existing draft is preserved | Enable organization-billed Copilot CLI use and rerun manually |
| Copilot output fails schema validation twice | Classification is incomplete; existing draft is preserved | Inspect captured diagnostics and rerun after correcting the prompt or CLI issue |
| Promotion produces no catalog diff | No branch or draft pull request is published | Confirm the latest complete artifact contains no eligible changes |
| Build or tests fail | No branch or PR mutation | Fix on a normal reviewed PR, then rerun maintenance |
| Generated metadata is malformed | Do not merge the draft | Fix the ingestion or validation rule, regenerate from `main`, and review again |
| A `Reject:` comment fails | Catalogs and PR body remain unchanged | Use IDs from the latest PR body and confirm the commenter is an owner, member, or collaborator |
| Push or PR update returns 403 | Publication fails without changing `main` | Verify `GALLERY_UPDATE_TOKEN` has Contents and Pull requests read/write access and organization approval |
| Copilot CLI authentication fails | Classification is incomplete; existing draft is preserved | Verify the token owner still has Copilot access, rotate `COPILOT_GITHUB_TOKEN`, and rerun manually |
| Branch push returns 403 | Publication fails without changing `main` | Verify the audit job has job-scoped Contents write permission |
| Review-command PR update returns 403 | The proposal remains unchanged | Verify the review-command job has job-scoped Contents and Pull requests write permissions |
| Copilot CLI authentication fails | Classification is incomplete; existing draft is preserved | Verify organization-billed Copilot CLI use is enabled and rerun manually |
| A localized Microsoft Learn URL is committed manually | Tests fail with `localized Microsoft documentation URL` | Remove the locale path segment, for example change `/en-us/azure/cosmos-db/...` to `/azure/cosmos-db/...` |

## Rollback and disablement
Expand All @@ -207,7 +166,7 @@ npm run gallery:audit:fixtures
npm run build
```

Live deterministic discovery can be run with `npm run gallery:audit`. It uses public endpoints and may be affected by network availability and rate limits. Copilot classification requires `COPILOT_GITHUB_TOKEN` and should not be run with broad tool permissions.
Live deterministic discovery can be run with `npm run gallery:audit`. It uses public endpoints and may be affected by network availability and rate limits. Copilot classification accepts `COPILOT_GITHUB_TOKEN`, `GH_TOKEN`, or `GITHUB_TOKEN` in precedence order and should not be run with broad tool permissions.

## Configuration changes

Expand Down
12 changes: 6 additions & 6 deletions docs/gallery-content-discovery-and-maintenance-proposal.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ The workflow can edit catalog files only on `automation/gallery-content-updates-
- No Azure resources, hosted APIs, databases, queues, storage accounts, model endpoints, or other backend services are used.
- Automation mutates only the repository in which the workflow runs. External repositories remain read-only discovery sources.
- Discovery uses public HTTP endpoints and the GitHub API with the workflow's scoped `GITHUB_TOKEN`.
- Relevance analysis uses GitHub Copilot CLI with a user-owned fine-grained token stored as the `COPILOT_GITHUB_TOKEN` Actions secret and limited to the **Copilot Requests** account permission and public repository access.
- Branch and pull request publication uses a separate `GALLERY_UPDATE_TOKEN` Actions secret scoped to this repository with contents and pull-request write access.
- The write token is exposed only to the final publication step, after tests and a full static build pass.
- Relevance analysis uses GitHub Copilot CLI with the built-in Actions `GITHUB_TOKEN` and the `copilot-requests: write` workflow permission. The organization must enable **Allow use of Copilot CLI billed to the organization**.
- Proposal branch publication uses the built-in Actions `GITHUB_TOKEN` with job-scoped contents write access. Enterprise policy requires a person to create the draft pull request from the emitted compare link.
- Repository default workflow permissions remain read-only; write access exists only in the publishing jobs after tests and a full static build pass.
- Pull requests are always drafts and are never approved or merged by automation.
- No candidate or audit finding changes the published catalog until a person merges the draft pull request.

Expand Down Expand Up @@ -180,7 +180,7 @@ These deterministic checks prioritize recall and keep the model input small. Cop
Run GitHub Copilot CLI programmatically after deterministic collection:

```shell
COPILOT_GITHUB_TOKEN="$COPILOT_GITHUB_TOKEN" \
GITHUB_TOKEN="$GITHUB_TOKEN" \
node scripts/gallery-audit/index.mjs --classify-only --promote
```

Expand Down Expand Up @@ -324,11 +324,11 @@ The workflow should:
8. validate Copilot output and combine it with deterministic evidence;
9. apply eligible changes to the per-run automation branch;
10. run focused tests and the full Docusaurus build;
11. open a new draft pull request with a separate write token;
11. publish the proposal branch and a compare link for a person to open as a draft pull request;
12. write a concise Actions job summary; and
13. upload the artifact bundle.

The normal Actions `GITHUB_TOKEN` remains scoped to read-only repository access. Copilot CLI authentication is separate: `COPILOT_GITHUB_TOKEN` requires an active Copilot subscription and the **Copilot Requests** account permission, with no repository write permission. `GALLERY_UPDATE_TOKEN` is a distinct repository-scoped credential used only by the publish step.
The built-in Actions `GITHUB_TOKEN` receives only the permissions declared for each job. Validation remains read-only; audit publication receives contents write, pull-request read, and Copilot-request write access; review-command application receives contents and pull-request write access. No stored authentication secret is required, and Actions never create, approve, or merge pull requests.

Use timeouts, concurrency with `cancel-in-progress: false`, bounded response sizes, redirect limits, and per-source request limits. Pin third-party actions to reviewed commit SHAs before enabling the schedule.

Expand Down
Loading
Loading