Skip to content
SupPanda9Public

About

Hybrid AI system combining Knowledge Graphs and Generative Diffusion Models to reconstruct extinct species from fossil photographs.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

PaleoVis

Hybrid AI system for palaeontological visualisation. PaleoVis bridges symbolic knowledge representation and generative artificial intelligence, combining deductive reasoning (OWL ontologies and GraphDB) with inductive reasoning (hyperbolic graph neural networks and diffusion models) to reconstruct extinct animals from fossil photographs.


Overview

Reconstructing extinct organisms from fossilized remains requires balancing scientific constraints with generative inference. PaleoVis addresses this challenge by integrating a structured knowledge base of taxonomy and morphology with deep learning pipelines. By querying living anatomical analogues and mapping hierarchical phylogenetic relationships, the system guides image diffusion models to generate visually compelling and scientifically grounded palaeontological reconstructions.


Architecture

PaleoVis is composed of five interconnected modules:

Module Technology Purpose
1 — Knowledge base OWL / GraphDB Taxonomic ontology, SPARQL analogue search
2 — Multimodal GNN ViT + RGCN (Poincaré) Encode bones, find living analogues
3 — LLM prompt layer Llama-3-8B (Ollama) Scientific prompt generation, scale control
4 — Generative pipeline SDXL + ControlNet Palaeontological reconstruction
5 — RLAIF loop DPO (trl) Preference optimisation from oracle feedback

Detailed Module Breakdown

  1. Knowledge Base (Module 1)

    • Built on Web Ontology Language (OWL) and hosted in GraphDB.
    • Encodes taxonomic structures, palaeo-habitats, and morphological metadata.
    • Executes SPARQL queries to extract anatomical and phylogenetic constraints.
  2. Multimodal GNN (Module 2)

    • Utilizes Vision Transformers (ViT) to extract visual features from fossil input photographs.
    • Embeds relational taxonomic hierarchies using Relational Graph Convolutional Networks (RGCN) within Poincaré hyperbolic space.
    • Matches fossil feature vectors against living biological analogues.
  3. LLM Prompt Layer (Module 3)

    • Powered by a local Llama-3-8B instance via Ollama.
    • Translates retrieved analogue metadata, scale specifications, and anatomical rules into structured prompts.
  4. Generative Pipeline (Module 4)

    • Combines Stable Diffusion XL (SDXL) with ControlNet.
    • Uses structural fossil constraints to guide the generation of soft tissue, integument, and environmental context.
  5. RLAIF Loop (Module 5)

    • Employs Direct Preference Optimization (DPO) via the trl library.
    • Refines model outputs using preference feedback from palaeontological oracle models to prioritize scientific accuracy over pure aesthetic liberty.

File Layout

See docs/filesystem.md for the full annotated directory tree and repository structure.


Acceptance Criteria

See docs/acceptance_criteria.md for module-by-module pass/fail specifications and test procedures.


Quickstart

Prerequisites

  • Python 3.10+
  • CUDA-compatible GPU (recommended for GNN and Diffusion inference)
  • GraphDB Free running locally on port 7200
  • Ollama installed locally

Setup & Run

  1. Clone the repository and install dependencies:

    git clone [https://github.com/SupPanda9/PaleoVis.git](https://github.com/SupPanda9/PaleoVis.git)
    cd PaleoVis
    pip install -r requirements.txt
    
  2. Pull the local LLM model:

    ollama pull llama3
    
  3. Ensure GraphDB Free is running locally:

Host URL: http://localhost:7200

  1. Run the full ETL pipeline:

    python etl/run_all.py
    Launch the user interface:
    
  2. Launch the user interface:

    python ui/app.py

About

Hybrid AI system combining Knowledge Graphs and Generative Diffusion Models to reconstruct extinct species from fossil photographs.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages