Hybrid AI system for palaeontological visualisation. PaleoVis bridges symbolic knowledge representation and generative artificial intelligence, combining deductive reasoning (OWL ontologies and GraphDB) with inductive reasoning (hyperbolic graph neural networks and diffusion models) to reconstruct extinct animals from fossil photographs.
Reconstructing extinct organisms from fossilized remains requires balancing scientific constraints with generative inference. PaleoVis addresses this challenge by integrating a structured knowledge base of taxonomy and morphology with deep learning pipelines. By querying living anatomical analogues and mapping hierarchical phylogenetic relationships, the system guides image diffusion models to generate visually compelling and scientifically grounded palaeontological reconstructions.
PaleoVis is composed of five interconnected modules:
| Module | Technology | Purpose |
|---|---|---|
| 1 — Knowledge base | OWL / GraphDB | Taxonomic ontology, SPARQL analogue search |
| 2 — Multimodal GNN | ViT + RGCN (Poincaré) | Encode bones, find living analogues |
| 3 — LLM prompt layer | Llama-3-8B (Ollama) | Scientific prompt generation, scale control |
| 4 — Generative pipeline | SDXL + ControlNet | Palaeontological reconstruction |
| 5 — RLAIF loop | DPO (trl) | Preference optimisation from oracle feedback |
-
Knowledge Base (Module 1)
- Built on Web Ontology Language (OWL) and hosted in GraphDB.
- Encodes taxonomic structures, palaeo-habitats, and morphological metadata.
- Executes SPARQL queries to extract anatomical and phylogenetic constraints.
-
Multimodal GNN (Module 2)
- Utilizes Vision Transformers (ViT) to extract visual features from fossil input photographs.
- Embeds relational taxonomic hierarchies using Relational Graph Convolutional Networks (RGCN) within Poincaré hyperbolic space.
- Matches fossil feature vectors against living biological analogues.
-
LLM Prompt Layer (Module 3)
- Powered by a local Llama-3-8B instance via Ollama.
- Translates retrieved analogue metadata, scale specifications, and anatomical rules into structured prompts.
-
Generative Pipeline (Module 4)
- Combines Stable Diffusion XL (SDXL) with ControlNet.
- Uses structural fossil constraints to guide the generation of soft tissue, integument, and environmental context.
-
RLAIF Loop (Module 5)
- Employs Direct Preference Optimization (DPO) via the
trllibrary. - Refines model outputs using preference feedback from palaeontological oracle models to prioritize scientific accuracy over pure aesthetic liberty.
- Employs Direct Preference Optimization (DPO) via the
See docs/filesystem.md for the full annotated directory tree and repository structure.
See docs/acceptance_criteria.md for module-by-module pass/fail specifications and test procedures.
- Python 3.10+
- CUDA-compatible GPU (recommended for GNN and Diffusion inference)
- GraphDB Free running locally on port 7200
- Ollama installed locally
-
Clone the repository and install dependencies:
git clone [https://github.com/SupPanda9/PaleoVis.git](https://github.com/SupPanda9/PaleoVis.git) cd PaleoVis pip install -r requirements.txt -
Pull the local LLM model:
ollama pull llama3
-
Ensure GraphDB Free is running locally:
Host URL: http://localhost:7200
-
Run the full ETL pipeline:
python etl/run_all.py Launch the user interface:
-
Launch the user interface:
python ui/app.py