-
Notifications
You must be signed in to change notification settings - Fork 382
Migrate Python checks to CI Gate #11522
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
e1b262e
6cdf761
c126768
b0d6b2f
62cde21
6e9597f
295e2f0
60c7525
9fa8055
524568e
2b191d9
021600f
21f0f56
cff9838
a3a70d1
fbec5e5
9c8dab7
afdc2d8
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| --- | ||
| changeKind: internal | ||
| packages: | ||
| - "@typespec/http-client-python" | ||
| --- | ||
|
|
||
| Move Python emitter PR validation into GitHub Actions CI Gate. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,242 @@ | ||
| name: Python CI | ||
|
|
||
| on: | ||
| workflow_call: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| build: | ||
| name: "Build & Regenerate (Linux)" | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
|
|
||
| - uses: ./.github/actions/setup | ||
|
|
||
| - uses: actions/setup-python@v7 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Install repo dependencies | ||
| run: pnpm install | ||
|
|
||
| - name: Setup http-client-python | ||
| # Avoid npm lifecycle side effects and lockfile rewrites in CI; setup still runs build + generator install. | ||
| run: | | ||
| npm install --ignore-scripts --package-lock=false | ||
| npm run setup | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: Prepare Python environment | ||
| # Run prepare.py directly so missing tox/dev dependencies fail the job instead of being swallowed by the npm wrapper. | ||
| run: venv/bin/python eng/scripts/setup/prepare.py | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: Regenerate | ||
| run: npm run regenerate | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: Pre-build wheels | ||
| run: | | ||
| venv/bin/python tests/install_packages.py build azure tests | ||
| venv/bin/python tests/install_packages.py build unbranded tests | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: Check for unstaged changes | ||
| run: npm run check-unstaged | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: Upload generated artifacts | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: python-generated | ||
| include-hidden-files: true | ||
| path: | | ||
| packages/http-client-python/tests/generated | ||
| packages/http-client-python/tests/.wheels | ||
| packages/http-client-python/dist | ||
| packages/http-client-python/generator/dist | ||
| retention-days: 1 | ||
|
|
||
| test: | ||
| name: "Mock API Tests" | ||
| needs: build | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
|
|
||
| - uses: ./.github/actions/setup | ||
|
|
||
| - uses: actions/setup-python@v7 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Download generated artifacts | ||
| uses: actions/download-artifact@v8 | ||
| with: | ||
| name: python-generated | ||
| path: packages/http-client-python | ||
|
|
||
| - name: Install repo dependencies | ||
| run: pnpm install | ||
|
|
||
| - name: Setup http-client-python | ||
| # Generated artifacts come from the build job; downstream jobs only need the Python package installed and test tools prepared. | ||
| run: | | ||
| npm install --ignore-scripts --package-lock=false | ||
| npm run install | ||
| venv/bin/python eng/scripts/setup/prepare.py | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: Test | ||
| run: npm run test:generator -- --env=test | ||
| working-directory: packages/http-client-python | ||
|
|
||
| typecheck: | ||
| name: "Type Checking" | ||
| needs: build | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
|
|
||
| - uses: ./.github/actions/setup | ||
|
|
||
| - uses: actions/setup-python@v7 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Download generated artifacts | ||
| uses: actions/download-artifact@v8 | ||
| with: | ||
| name: python-generated | ||
| path: packages/http-client-python | ||
|
|
||
| - name: Install repo dependencies | ||
| run: pnpm install | ||
|
|
||
| - name: Setup http-client-python | ||
| # Generated artifacts come from the build job; downstream jobs only need the Python package installed and test tools prepared. | ||
| run: | | ||
| npm install --ignore-scripts --package-lock=false | ||
| npm run install | ||
| venv/bin/python eng/scripts/setup/prepare.py | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: Mypy & Pyright | ||
| run: npm run test:generator -- --env=mypy,pyright | ||
| working-directory: packages/http-client-python | ||
|
|
||
| lint: | ||
| name: "Lint & Format" | ||
| needs: build | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
|
|
||
| - uses: ./.github/actions/setup | ||
|
|
||
| - uses: actions/setup-python@v7 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Download generated artifacts | ||
| uses: actions/download-artifact@v8 | ||
| with: | ||
| name: python-generated | ||
| path: packages/http-client-python | ||
|
|
||
| - name: Install repo dependencies | ||
| run: pnpm install | ||
|
|
||
| - name: Setup http-client-python | ||
| # Generated artifacts come from the build job; downstream jobs only need the Python package installed and test tools prepared. | ||
| run: | | ||
| npm install --ignore-scripts --package-lock=false | ||
| npm run install | ||
| venv/bin/python eng/scripts/setup/prepare.py | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: Pylint | ||
| run: npm run test:generator -- --env=lint | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: Lint source | ||
| run: npm run lint | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: Build TypeSpec Prettier plugin | ||
| run: pnpm --filter "@typespec/prettier-plugin-typespec..." run build | ||
|
|
||
| - name: Format check source | ||
| run: npm run format:check | ||
| working-directory: packages/http-client-python | ||
|
|
||
| docs: | ||
| name: "Docs Validation" | ||
| needs: build | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
|
|
||
| - uses: ./.github/actions/setup | ||
|
|
||
| - uses: actions/setup-python@v7 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Download generated artifacts | ||
| uses: actions/download-artifact@v8 | ||
| with: | ||
| name: python-generated | ||
| path: packages/http-client-python | ||
|
|
||
| - name: Install repo dependencies | ||
| run: pnpm install | ||
|
|
||
| - name: Setup http-client-python | ||
| # Generated artifacts come from the build job; downstream jobs only need the Python package installed and test tools prepared. | ||
| run: | | ||
| npm install --ignore-scripts --package-lock=false | ||
| npm run install | ||
| venv/bin/python eng/scripts/setup/prepare.py | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: API View & Sphinx | ||
| run: npm run test:generator -- --env=apiview,sphinx | ||
| working-directory: packages/http-client-python | ||
|
|
||
| windows-test: | ||
| name: "Build & Regenerate && Test (Windows)" | ||
| runs-on: windows-latest | ||
| steps: | ||
| - name: Enable Git long paths | ||
| run: git config --system core.longpaths true | ||
|
|
||
| - uses: actions/checkout@v7 | ||
|
|
||
| - uses: ./.github/actions/setup | ||
|
|
||
| - uses: actions/setup-python@v7 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Install repo dependencies | ||
| run: pnpm install | ||
|
|
||
| - name: Setup http-client-python | ||
| # Windows validates build/regenerate/test in one job, so run the full setup instead of restoring Linux artifacts. | ||
| run: | | ||
| npm install --ignore-scripts --package-lock=false | ||
| npm run setup | ||
| .\venv\Scripts\python.exe eng\scripts\setup\prepare.py | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: Regenerate | ||
| run: npm run regenerate | ||
| working-directory: packages/http-client-python | ||
|
|
||
| - name: Test | ||
| run: npm run test:generator -- --env=test | ||
| working-directory: packages/http-client-python |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -55,9 +55,3 @@ extends: | |
| parameters: | ||
| DependsOn: InitStage | ||
| Condition: eq('true', stageDependencies.InitStage.outputs['InitJob.InitStep.RunJava']) | ||
|
|
||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. do you need to delete some fileS?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, I could clear some unnecessary script files inner http-client-python folder but I want to do the deletion in the next PR to keep this PR clear. |
||
| # Run python stages if RunPython == true | ||
| - template: /packages/http-client-python/eng/pipeline/templates/ci-stages.yml | ||
| parameters: | ||
| DependsOn: InitStage | ||
| Condition: eq('true', stageDependencies.InitStage.outputs['InitJob.InitStep.RunPython']) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that tool was letting multiple filters to be defined in the same task
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It can define multiple filters in one task, but here the two filters need different quantifier behavior. The core filter uses
predicate-quantifier: "every"so excluded-only changes do not trigger core CI, while the Python filter needs the defaultsomebehavior so any Python-related change triggers Python CI. Since the quantifier applies to the whole paths-filter step, these need to stay split unless we change the filtering approach.