|
10 | 10 |
|
11 | 11 | ## What is CLAIMED? |
12 | 12 |
|
13 | | -CLAIMED has three interlocking layers: |
| 13 | +CLAIMED began life as a **library of reusable AI and data components** — ready-made, |
| 14 | +shareable building blocks for the recurring chores of machine learning: reading and |
| 15 | +writing data from COS/S3, NLP preprocessing, model training, and benchmarking. That |
| 16 | +component library is still here (`claimed.components.*`) and still the foundation |
| 17 | +everything else builds on. |
| 18 | + |
| 19 | +What turned CLAIMED from a handy library into a genuinely powerful framework is the |
| 20 | +**C3 Component Compiler**. C3 takes *any* notebook, Python script, or R script and |
| 21 | +compiles it into a fully containerised, runnable, composable component — complete with |
| 22 | +a CLI, a Docker image, and KubeFlow Pipelines / CWL / Kubernetes descriptors — with |
| 23 | +zero boilerplate. Suddenly every piece of code you write becomes a first-class, |
| 24 | +reusable CLAIMED component, not just the ones someone hand-packaged in advance. C3 is |
| 25 | +the heart of the framework today. |
| 26 | + |
| 27 | +From there, CLAIMED grew outward by **integrating two complementary projects** rather |
| 28 | +than reinventing them: |
| 29 | + |
| 30 | +- **MLX** (`claimed.mlx`) brings **experiment and asset tracking** — full provenance |
| 31 | + for every dataset, model, and job — and powers the grid-compute backend that |
| 32 | + distributes component runs across heterogeneous clusters. |
| 33 | +- **terratorch-iterate** (the `iterate2` engine, shipped inside `claimed`) adds |
| 34 | + **hyperparameter optimisation (HPO) and neural architecture search (NAS)** on top of |
| 35 | + your components, with MLflow logging, Optuna-driven search, and Ray parallelisation. |
| 36 | + |
| 37 | +Put together, CLAIMED lets you go from a single script all the way to a tracked, |
| 38 | +optimised, cluster-scale AI pipeline — without ever leaving the framework. |
14 | 39 |
|
15 | 40 | | Layer | Package / module | Purpose | |
16 | 41 | |---|---|---| |
17 | | -| **C3** – Component Compiler | `claimed.c3` | Turns notebooks, Python scripts, and R scripts into fully containerised, executable AI components | |
| 42 | +| **C3** – Component Compiler | `claimed.c3` | Turns notebooks, Python scripts, and R scripts into fully containerised, executable, composable AI components | |
| 43 | +| **Component Library** | `claimed.components.*` | The original ready-to-use components for COS/S3 I/O, benchmarking, NLP, training, and more | |
18 | 44 | | **MLX** – ML eXchange backend | `claimed.mlx` | Tracks datasets, models, jobs and other assets; powers the grid-compute backend | |
19 | | -| **Component Library** | `claimed.components.*` | Ready-to-use components for COS/S3 I/O, benchmarking, NLP, training, and more | |
| 45 | +| **iterate** – HPO / NAS | `claimed` (`iterate2`) | Hyperparameter optimisation and neural architecture search via MLflow, Optuna, and Ray | |
20 | 46 |
|
21 | 47 | --- |
22 | 48 |
|
|
0 commit comments