Skip to content

Commit 7b94e65

Browse files
committed
added asyncValueStore implementation
1 parent 4a77c21 commit 7b94e65

17 files changed

+2159
-383
lines changed

docs/api.rst

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
21
API Reference
32
=============
43

@@ -43,6 +42,13 @@ uberjob.ValueStore
4342
:members:
4443

4544

45+
uberjob.AsyncValueStore
46+
---------------------
47+
48+
.. autoclass:: uberjob.AsyncValueStore
49+
:members:
50+
51+
4652
uberjob.stores
4753
--------------
4854

docs/async.rst

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
AsyncValueStore Support
2+
======================
3+
4+
Starting with version 1.1.0, uberjob supports asynchronous value stores, providing better performance for I/O-bound operations.
5+
6+
AsyncValueStore Interface
7+
------------------------
8+
9+
The :class:`~uberjob.AsyncValueStore` abstract base class defines the interface for all asynchronous value stores:
10+
11+
.. code-block:: python
12+
13+
class AsyncValueStore(ABC):
14+
@abstractmethod
15+
async def read(self):
16+
"""Read the value from the store asynchronously."""
17+
18+
@abstractmethod
19+
async def write(self, value) -> None:
20+
"""Write a value to the store asynchronously."""
21+
22+
@abstractmethod
23+
async def get_modified_time(self) -> dt.datetime | None:
24+
"""Get the modified time of the stored value asynchronously."""
25+
26+
Available Implementations
27+
------------------------
28+
29+
uberjob provides several implementations of :class:`~uberjob.AsyncValueStore`:
30+
31+
- :class:`~uberjob.stores.AsyncJsonFileStore`: For storing JSON-serializable values in files asynchronously
32+
- :class:`~uberjob.stores.AsyncTextFileStore`: For storing string values in files asynchronously
33+
- :class:`~uberjob.stores.AsyncBinaryFileStore`: For storing binary data in files asynchronously
34+
- :class:`~uberjob.stores.AsyncPickleFileStore`: For storing Python objects in files asynchronously
35+
- :class:`~uberjob.stores.AsyncTouchFileStore`: For creating empty marker files asynchronously
36+
37+
Example Usage
38+
-----------
39+
40+
Here's an example of using :class:`~uberjob.stores.AsyncJsonFileStore`:
41+
42+
.. code-block:: python
43+
44+
import asyncio
45+
from uberjob.stores import AsyncJsonFileStore
46+
47+
async def main():
48+
# Create an async JSON file store
49+
store = AsyncJsonFileStore("data.json")
50+
51+
# Write data asynchronously
52+
await store.write({"key": "value"})
53+
54+
# Read data asynchronously
55+
data = await store.read()
56+
print(data) # {"key": "value"}
57+
58+
# Get the modified time
59+
modified_time = await store.get_modified_time()
60+
print(f"Last modified: {modified_time}")
61+
62+
asyncio.run(main())
63+
64+
Utility Functions
65+
---------------
66+
67+
uberjob also provides async versions of utility functions for file operations:
68+
69+
- :func:`~uberjob.stores.async_staged_write`: Asynchronously write to a staged file that is atomically moved to the target path
70+
- :func:`~uberjob.stores.async_staged_write_path`: Lower-level function that yields a staging path
71+
- :func:`~uberjob.stores.async_get_modified_time`: Get the modified time of a file asynchronously
72+
73+
Performance Considerations
74+
------------------------
75+
76+
Asynchronous value stores are most beneficial in scenarios where:
77+
78+
- Many I/O operations are performed concurrently
79+
- I/O operations take significant time (network operations, large files)
80+
- The application already uses asyncio for other purposes
81+
82+
For small, local files or CPU-bound workloads, the synchronous value stores may still be more appropriate.

docs/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,3 +17,4 @@ The lessons below introduce important concepts in uberjob.
1717
lesson6
1818
lesson7
1919
lesson8
20+
async

0 commit comments

Comments
 (0)