Add API models for BRIGHT(v1.1)

[KennethEnevoldsen]

still ongoing.

targeting: embeddings-benchmark/mteb#4340

run using:

from pathlib import Path

import mteb
import mteb.cache

cache_path = Path(__file__).parent.parent / "mteb_results" / "remote"

cache = mteb.ResultCache(cache_path)
cache.get_models()

names = [
    # "ByteDance-Seed/Seed1.5-Embedding",
    # "google/text-embedding-004",
    "voyageai/voyage-large-2-instruct",
    "openai/text-embedding-3-large",
    "Cohere/Cohere-embed-english-v3.0",
]

for name in names:
    print(name)
    model = mteb.get_model(name)
    bench = mteb.get_benchmark("BRIGHT(v1.1)")
    res = mteb.evaluate(model, bench, cache=cache)

Checklist

• My model has a model sheet, report, or similar
• My model has a reference implementation in mteb/models/model_implementations/, this can be as an API. Instruction on how to add a model can be found here
  • No, but there is an existing PR ___
• The results submitted are obtained using the reference implementation
• My model is available, either as a publicly accessible API or publicly on e.g., Huggingface
• I solemnly swear that for all results submitted I have not trained on the evaluation dataset including training splits. If I have, I have disclosed it clearly.

[github-actions]

Model Results Comparison

Reference models: intfloat/multilingual-e5-large, google/gemini-embedding-001
New models evaluated: voyageai/voyage-large-2-instruct
Tasks: BrightBiologyRetrieval, BrightEarthScienceRetrieval, BrightEconomicsRetrieval, BrightPonyRetrieval, BrightPsychologyRetrieval, BrightRoboticsRetrieval, BrightStackoverflowRetrieval, BrightSustainableLivingRetrieval

Results for voyageai/voyage-large-2-instruct

task_name	intfloat/multilingual-e5-large	voyageai/voyage-large-2-instruct	Max result	Model with max result	In Training Data
BrightBiologyRetrieval	0.0174	0.2268	0.3387	lightonai/Reason-ModernColBERT	False
BrightEarthScienceRetrieval	0.1506	0.2573	0.4170	lightonai/Reason-ModernColBERT	False
BrightEconomicsRetrieval	0.0706	0.1981	0.2455	lightonai/Reason-ModernColBERT	False
BrightPonyRetrieval	0.1302	0.0164	0.1517	BAAI/bge-m3	False
BrightPsychologyRetrieval	0.0879	0.2527	0.3104	lightonai/Reason-ModernColBERT	False
BrightRoboticsRetrieval	0.1112	0.1169	0.2181	lightonai/Reason-ModernColBERT	False
BrightStackoverflowRetrieval	0.0694	0.171	0.2425	lightonai/Reason-ModernColBERT	False
BrightSustainableLivingRetrieval	0.0961	0.1529	0.2021	lightonai/Reason-ModernColBERT	False
Average	0.0917	0.174	0.2658	nan	-

---

---

Note: Content truncated due to GitHub API limits. See the full report in the workflow artifacts.

[github-actions]

This pull request has been automatically marked as stale due to inactivity.

[KennethEnevoldsen]

will merge these and add more as a I have them