Skip to content

Commit 2562fdf

Browse files
committed
Release daos-fsspec 1.4.1
0 parents  commit 2562fdf

108 files changed

Lines changed: 26195 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.dockerignore‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
.git
2+
.venv
3+
__pycache__
4+
*.pyc
5+
*.egg-info
6+
build
7+
dist
8+
.pytest_cache
9+
.ruff_cache
10+
.mypy_cache
11+
.hypothesis
12+
.coverage

‎.gitattributes‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
*.pyi linguist-detectable=true
2+
tests/ linguist-detectable=false
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
---
2+
name: Bug Report
3+
about: Report a bug to help us improve daos-fsspec
4+
title: ''
5+
labels: bug
6+
assignees: ''
7+
---
8+
9+
## Environment
10+
11+
- **daos-fsspec version**: <!-- e.g. 1.4.1 -->
12+
- **Python version**: <!-- e.g. 3.11.4 -->
13+
- **DAOS version**: <!-- e.g. 2.6.4 -->
14+
- **OS**: <!-- e.g. Ubuntu 22.04 -->
15+
- **Installation method**: <!-- pip install, from source, etc. -->
16+
17+
## Description
18+
19+
<!-- A clear and concise description of what the bug is. -->
20+
21+
## Steps to Reproduce
22+
23+
```python
24+
# Minimal code that reproduces the issue
25+
import fsspec
26+
27+
fs = fsspec.filesystem("dfs", pool="mypool", container="mycont")
28+
# ...
29+
```
30+
31+
## Expected Behavior
32+
33+
<!-- What you expected to happen. -->
34+
35+
## Actual Behavior
36+
37+
<!-- What actually happened. Include full tracebacks if applicable. -->
38+
39+
```
40+
Paste traceback here
41+
```
42+
43+
## Additional Context
44+
45+
<!-- Any other context: DAOS pool/container config, network setup, whether the issue is intermittent, etc. -->
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
name: Feature Request
3+
about: Suggest a new feature or improvement
4+
title: ''
5+
labels: enhancement
6+
assignees: ''
7+
---
8+
9+
## Problem
10+
11+
<!-- A clear description of the problem or limitation you're experiencing. -->
12+
13+
## Proposed Solution
14+
15+
<!-- How you'd like this to work. Include code examples if applicable. -->
16+
17+
```python
18+
# Example of how the feature would be used
19+
import fsspec
20+
21+
fs = fsspec.filesystem("dfs", pool="mypool", container="mycont")
22+
# ...
23+
```
24+
25+
## Alternatives Considered
26+
27+
<!-- Any alternative solutions or workarounds you've considered. -->
28+
29+
## Additional Context
30+
31+
<!-- Any other context: use case, scale requirements, related DAOS features, etc. -->

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
## What
2+
3+
<!-- Brief description of the change. -->
4+
5+
## Why
6+
7+
<!-- Motivation: bug fix, feature, refactor, etc. Link to issue if applicable. -->
8+
9+
## How
10+
11+
<!-- Key implementation details reviewers should focus on. -->
12+
13+
## Testing
14+
15+
<!-- How was this tested? New tests added? Existing tests still pass? -->
16+
17+
- [ ] `ruff check` passes
18+
- [ ] `ruff format --check` passes
19+
- [ ] `pytest tests/ -m "not live"` passes (all unit tests)
20+
- [ ] Tested against live DAOS cluster (if applicable)

‎.github/dependabot.yml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
version: 2
2+
updates:
3+
- package-ecosystem: "pip"
4+
directory: "/"
5+
schedule:
6+
interval: "weekly"
7+
open-pull-requests-limit: 5
8+
labels:
9+
- "dependencies"
10+
- package-ecosystem: "github-actions"
11+
directory: "/"
12+
schedule:
13+
interval: "weekly"
14+
open-pull-requests-limit: 5
15+
labels:
16+
- "dependencies"

‎.github/workflows/release.yml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
name: Release
2+
on:
3+
workflow_dispatch:
4+
inputs:
5+
testpypi_run_id:
6+
description: Successful TestPyPI run whose artifacts will be published
7+
required: true
8+
confirmation:
9+
description: Type publish-1.4.1 to publish to production PyPI
10+
required: true
11+
permissions:
12+
contents: read
13+
actions: read
14+
jobs:
15+
verify:
16+
if: github.ref == 'refs/heads/main' && inputs.confirmation == 'publish-1.4.1'
17+
runs-on: ubuntu-24.04
18+
timeout-minutes: 15
19+
steps:
20+
- name: Verify tested source
21+
env:
22+
GH_TOKEN: ${{ github.token }}
23+
RUN_ID: ${{ inputs.testpypi_run_id }}
24+
REPOSITORY: ${{ github.repository }}
25+
EXPECTED_SHA: ${{ github.sha }}
26+
run: |
27+
[[ "$RUN_ID" =~ ^[0-9]+$ ]]
28+
gh api "repos/$REPOSITORY/actions/runs/$RUN_ID" > run.json
29+
python - <<'PYTHON'
30+
import json, os
31+
run = json.load(open('run.json'))
32+
assert run['conclusion'] == 'success'
33+
assert run['head_sha'] == os.environ['EXPECTED_SHA']
34+
assert run['path'] == '.github/workflows/testpypi.yml'
35+
assert run['event'] == 'workflow_dispatch'
36+
PYTHON
37+
publish:
38+
needs: verify
39+
runs-on: ubuntu-24.04
40+
timeout-minutes: 15
41+
environment: pypi
42+
permissions:
43+
actions: read
44+
id-token: write
45+
steps:
46+
- uses: actions/download-artifact@v8
47+
with:
48+
name: release-dist
49+
path: dist
50+
run-id: ${{ inputs.testpypi_run_id }}
51+
github-token: ${{ github.token }}
52+
- uses: pypa/gh-action-pypi-publish@release/v1
53+
with:
54+
attestations: false

‎.github/workflows/test.yml‎

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
name: Tests
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
workflow_dispatch:
9+
10+
permissions:
11+
contents: read
12+
13+
jobs:
14+
unit-tests:
15+
runs-on: ubuntu-24.04
16+
timeout-minutes: 20
17+
strategy:
18+
fail-fast: false
19+
matrix:
20+
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
21+
22+
steps:
23+
- uses: actions/checkout@v7
24+
25+
- name: Set up Python ${{ matrix.python-version }}
26+
uses: actions/setup-python@v7
27+
with:
28+
python-version: ${{ matrix.python-version }}
29+
30+
- name: Install dependencies
31+
run: |
32+
pip install -e ".[test]"
33+
34+
- name: Run unit tests
35+
run: pytest tests/ -v -m "not live" --timeout=120 --timeout-method=thread --cov=daos_fsspec --cov-report=term-missing --cov-report=xml
36+
37+
- name: Upload coverage
38+
if: matrix.python-version == '3.12'
39+
uses: actions/upload-artifact@v7
40+
with:
41+
name: coverage-report
42+
path: coverage.xml
43+
44+
lint:
45+
runs-on: ubuntu-24.04
46+
timeout-minutes: 10
47+
steps:
48+
- uses: actions/checkout@v7
49+
50+
- name: Set up Python
51+
uses: actions/setup-python@v7
52+
with:
53+
python-version: "3.12"
54+
55+
- name: Install ruff
56+
run: pip install ruff
57+
58+
- name: Lint
59+
run: ruff check daos_fsspec/ tests/
60+
61+
- name: Format check
62+
run: ruff format --check daos_fsspec/ tests/
63+
64+
typecheck:
65+
runs-on: ubuntu-24.04
66+
timeout-minutes: 10
67+
steps:
68+
- uses: actions/checkout@v7
69+
70+
- name: Set up Python
71+
uses: actions/setup-python@v7
72+
with:
73+
python-version: "3.12"
74+
75+
- name: Install dependencies
76+
run: |
77+
pip install "mypy>=1.0,<1.19" fsspec
78+
pip install --no-deps -e .
79+
80+
- name: Type check
81+
run: mypy daos_fsspec/ --ignore-missing-imports
82+
83+
build:
84+
runs-on: ubuntu-24.04
85+
timeout-minutes: 10
86+
steps:
87+
- uses: actions/checkout@v7
88+
89+
- name: Set up Python
90+
uses: actions/setup-python@v7
91+
with:
92+
python-version: "3.12"
93+
94+
- name: Install build tools
95+
run: pip install build
96+
97+
- name: Build sdist and wheel
98+
run: python -m build
99+
100+
- name: Verify wheel contents
101+
run: |
102+
pip install dist/*.whl
103+
python -c "import daos_fsspec; print(daos_fsspec.__version__)"
104+
python -c "from daos_fsspec import DFSFileSystem; print('OK')"
105+
106+
- name: Upload artifacts
107+
uses: actions/upload-artifact@v7
108+
with:
109+
name: dist
110+
path: dist/
111+
112+
live-tests:
113+
name: live-tests (DAOS ${{ matrix.daos_version }})
114+
if: github.event_name == 'workflow_dispatch' || github.ref == 'refs/heads/main'
115+
runs-on: ubuntu-24.04
116+
timeout-minutes: 45
117+
strategy:
118+
fail-fast: false
119+
matrix:
120+
daos_version: ["2.6.5", "2.8.0"]
121+
122+
steps:
123+
- uses: actions/checkout@v7
124+
125+
- name: Disable transparent hugepages
126+
run: |
127+
echo "THP before: $(cat /sys/kernel/mm/transparent_hugepage/enabled)"
128+
echo never | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
129+
echo "THP after: $(cat /sys/kernel/mm/transparent_hugepage/enabled)"
130+
131+
- name: Prepare memory for RAM-backed SCM
132+
run: |
133+
sudo sysctl vm.nr_hugepages=0
134+
free -h
135+
136+
- name: Run live tests against DAOS ${{ matrix.daos_version }}
137+
env:
138+
DAOS_VERSION: ${{ matrix.daos_version }}
139+
shell: bash -euo pipefail {0}
140+
run: |
141+
chmod +x docker/test.sh
142+
docker/test.sh --live --skip-unit 2>&1 | tee live-test.log
143+
144+
- name: Dump cluster state on failure
145+
if: failure()
146+
run: |
147+
docker ps -a || true
148+
docker logs daos-test-server 2>&1 | tail -100 || true
149+
docker logs daos-test-client 2>&1 | tail -50 || true
150+
docker exec daos-test-server cat /tmp/daos_engine.log 2>&1 | tail -50 || true
151+
152+
- name: Publish result to job summary
153+
if: always()
154+
run: |
155+
{
156+
echo "### DAOS ${{ matrix.daos_version }} live integration tests"
157+
echo '```'
158+
grep -E 'passed|failed|error' live-test.log | tail -5 || true
159+
echo '```'
160+
} >> "$GITHUB_STEP_SUMMARY"
161+
162+
- name: Upload live test log
163+
if: always()
164+
uses: actions/upload-artifact@v7
165+
with:
166+
name: live-test-log-${{ matrix.daos_version }}
167+
path: live-test.log

0 commit comments

Comments
 (0)