Skip to content

Commit ccaae8d

Browse files
ThireusThireus
authored
Merge branch 'ggml-org:master' into master
2 parents a138a5a + 4adac43 commit ccaae8d

130 files changed

Lines changed: 5048 additions & 3331 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.devops/nix/package.nix‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
vulkan-loader,
1919
openssl,
2020
shaderc,
21+
spirv-headers,
2122
useBlas ?
2223
builtins.all (x: !x) [
2324
useCuda
@@ -145,6 +146,7 @@ effectiveStdenv.mkDerivation (finalAttrs: {
145146
ninja
146147
pkg-config
147148
git
149+
spirv-headers
148150
]
149151
++ optionals useCuda [
150152
cudaPackages.cuda_nvcc

‎.github/workflows/build-riscv.yml‎

Lines changed: 6 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -47,22 +47,10 @@ jobs:
4747
steps:
4848
- name: Install dependencies
4949
run: |
50-
sudo apt-get update
51-
52-
# Install necessary packages
53-
sudo apt-get install -y libatomic1 libtsan2 gcc-14 g++-14 cmake build-essential wget git-lfs
54-
5550
# Set gcc-14 and g++-14 as the default compilers
5651
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-14 100
5752
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-14 100
5853
59-
if ! which rustc; then
60-
# Install Rust stable version
61-
sudo apt-get install -y rustup
62-
rustup install stable
63-
rustup default stable
64-
fi
65-
6654
git lfs install
6755
6856
- name: GCC version check
@@ -74,12 +62,12 @@ jobs:
7462
id: checkout
7563
uses: actions/checkout@v6
7664

77-
# FIXME: Enable when ggml-org/ccache-action works on riscv64
78-
# - name: ccache
79-
# uses: ggml-org/ccache-action@v1.2.21
80-
# with:
81-
# key: ubuntu-riscv64-native-sanitizer-${{ matrix.sanytizer }}-${{ matrix.build_type }}
82-
# save: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }}
65+
- name: ccache
66+
uses: ggml-org/ccache-action@afde29e5b5422e5da23cb1f639e8baecadeadfc3 # https://github.com/ggml-org/ccache-action/pull/1
67+
with:
68+
key: ubuntu-riscv64-native-sanitizer-${{ matrix.sanitizer }}-${{ matrix.build_type }}
69+
evict-old-files: 1d
70+
save: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }}
8371

8472
- name: Build
8573
id: cmake_build

‎.github/workflows/build.yml‎

Lines changed: 8 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1001,22 +1001,14 @@ jobs:
10011001
steps:
10021002
- name: Install dependencies
10031003
run: |
1004-
sudo apt-get update
1005-
10061004
# Install necessary packages
1007-
sudo apt-get install -y libatomic1 libtsan2 gcc-14 g++-14 cmake build-essential libssl-dev wget git-lfs
1005+
sudo apt-get update
1006+
sudo apt-get install -y libssl-dev
10081007
10091008
# Set gcc-14 and g++-14 as the default compilers
10101009
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-14 100
10111010
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-14 100
10121011
1013-
if ! which rustc; then
1014-
# Install Rust stable version
1015-
sudo apt-get install -y rustup
1016-
rustup install stable
1017-
rustup default stable
1018-
fi
1019-
10201012
git lfs install
10211013
10221014
- name: Check environment
@@ -1032,13 +1024,12 @@ jobs:
10321024
id: checkout
10331025
uses: actions/checkout@v6
10341026

1035-
# FIXME: Enable when ggml-org/ccache-action works on riscv64
1036-
# - name: ccache
1037-
# uses: ggml-org/ccache-action@v1.2.21
1038-
# with:
1039-
# key: ubuntu-cpu-riscv64-native
1040-
# evict-old-files: 1d
1041-
# save: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }}
1027+
- name: ccache
1028+
uses: ggml-org/ccache-action@afde29e5b5422e5da23cb1f639e8baecadeadfc3 # https://github.com/ggml-org/ccache-action/pull/1
1029+
with:
1030+
key: ubuntu-cpu-riscv64-native
1031+
evict-old-files: 1d
1032+
save: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }}
10421033

10431034
- name: Build
10441035
id: cmake_build

‎CODEOWNERS‎

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,21 @@
11
# collaborators can optionally add themselves here to indicate their availability for reviewing related PRs
2-
# multiplie collaborators per item can be specified
2+
# multiple collaborators per item can be specified
3+
#
4+
# ggml-org/ci : CISC, danbev, ggerganov, netrunnereve, ngxson, taronaeo
5+
# ggml-org/ggml-cann : hipudding
6+
# ggml-org/ggml-cuda : JohannesGaessler, am17an, IMbackK, ORippler
7+
# ggml-org/ggml-hexagon : lhez, max-krasnyansky
8+
# ggml-org/ggml-metal : ggerganov
9+
# ggml-org/ggml-opencl : lhez, max-krasnyansky
10+
# ggml-org/ggml-rpc : rgerganov
11+
# ggml-org/ggml-sycl : arthw
12+
# ggml-org/ggml-vulkan : 0cc4m, jeffbolznv
13+
# ggml-org/ggml-webgpu : reeselevine
14+
# ggml-org/ggml-zdnn : taronaeo
15+
# ggml-org/llama-common : ggerganov, aldehir, angt, danbev, ngxson, pwilkin
16+
# ggml-org/llama-mtmd : ngxson
17+
# ggml-org/llama-server : ggerganov, ngxson, allozaur, angt, ServeurpersoCom
18+
# ggml-org/llama-webui : allozaur
319

420
/.devops/*.Dockerfile @ngxson
521
/.github/actions/ @ggml-org/ci

‎convert_hf_to_gguf.py‎

Lines changed: 58 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10893,7 +10893,64 @@ def set_gguf_parameters(self):
1089310893
self.gguf_writer.add_moe_latent_size(latent_size)
1089410894

1089510895
def set_vocab(self):
10896-
super().set_vocab()
10896+
# The NemotronH config uses pattern characters (e.g. '-') that may not
10897+
# be supported by the installed transformers version. AutoTokenizer
10898+
# internally calls AutoConfig which triggers this parsing failure.
10899+
# Using trust_remote_code=True to load the model's own config class.
10900+
tokens: list[str] = []
10901+
toktypes: list[int] = []
10902+
10903+
from transformers import AutoTokenizer
10904+
tokenizer = AutoTokenizer.from_pretrained(self.dir_model, trust_remote_code=True)
10905+
10906+
# Pad vocab size (from Mamba2Model/GraniteHybridModel)
10907+
self.hparams["pad_vocab_size_multiple"] = 8 # Setting this here since GraniteHybridModel.set_vocab() isn't being invoked now.
10908+
# From Mamba2Model.set_vocab():
10909+
vocab_size = self.hparams["vocab_size"]
10910+
pad_vocab = self.hparams.get("pad_vocab_size_multiple", 16)
10911+
# ref: https://stackoverflow.com/a/17511341/22827863
10912+
vocab_size = -(vocab_size // -pad_vocab) * pad_vocab
10913+
self.hparams["vocab_size"] = vocab_size
10914+
10915+
assert max(tokenizer.vocab.values()) < vocab_size
10916+
10917+
tokpre = self.get_vocab_base_pre(tokenizer)
10918+
10919+
reverse_vocab = {id_: encoded_tok for encoded_tok, id_ in tokenizer.vocab.items()}
10920+
added_vocab = tokenizer.get_added_vocab()
10921+
10922+
added_tokens_decoder = tokenizer.added_tokens_decoder
10923+
10924+
for i in range(vocab_size):
10925+
if i not in reverse_vocab:
10926+
tokens.append(f"[PAD{i}]")
10927+
toktypes.append(gguf.TokenType.UNUSED)
10928+
else:
10929+
token: str = reverse_vocab[i]
10930+
if token in added_vocab:
10931+
if not added_tokens_decoder[i].normalized:
10932+
previous_token = token
10933+
token = tokenizer.decode(tokenizer.encode(token, add_special_tokens=False))
10934+
if previous_token != token:
10935+
logger.info(f"{repr(previous_token)} is encoded and decoded back to {repr(token)} using AutoTokenizer")
10936+
10937+
if added_tokens_decoder[i].special or self.does_token_look_special(token):
10938+
toktypes.append(gguf.TokenType.CONTROL)
10939+
else:
10940+
token = token.replace(b"\xe2\x96\x81".decode("utf-8"), " ") # pre-normalize user-defined spaces
10941+
toktypes.append(gguf.TokenType.USER_DEFINED)
10942+
else:
10943+
toktypes.append(gguf.TokenType.NORMAL)
10944+
tokens.append(token)
10945+
10946+
# From TextModel.set_vocab_gpt2():
10947+
self.gguf_writer.add_tokenizer_model("gpt2")
10948+
self.gguf_writer.add_tokenizer_pre(tokpre)
10949+
self.gguf_writer.add_token_list(tokens)
10950+
self.gguf_writer.add_token_types(toktypes)
10951+
10952+
special_vocab = gguf.SpecialVocab(self.dir_model, load_merges=True)
10953+
special_vocab.add_to_gguf(self.gguf_writer)
1089710954

1089810955
# The tokenizer _does_ add a BOS token (via post_processor type
1089910956
# TemplateProcessing) but does not set add_bos_token to true in the

‎docs/ops.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -22,13 +22,13 @@ Legend:
2222
| ARANGE | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
2323
| ARGMAX | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
2424
| ARGSORT | ❌ | ✅ | ✅ | ✅ | ✅ | 🟡 | 🟡 | ✅ | ✅ | ❌ | ❌ |
25-
| CEIL | ❌ | ❌ | ✅ | 🟡 | ❌ | ❌ | ✅ | 🟡 | ✅ | ❌ | ❌ |
25+
| CEIL | ❌ | ❌ | ✅ | 🟡 | ✅ | ❌ | ✅ | 🟡 | ✅ | ❌ | ❌ |
2626
| CLAMP | ❌ | ✅ | ✅ | ✅ | ✅ | 🟡 | 🟡 | 🟡 | ✅ | ❌ | ❌ |
2727
| CONCAT | ❌ | ✅ | ✅ | 🟡 | ✅ | 🟡 | ✅ | ✅ | ✅ | ❌ | ❌ |
28-
| CONT | ❌ | 🟡 | ✅ | ✅ | 🟡 | 🟡 | 🟡 | ✅ | 🟡 | ❌ | ❌ |
28+
| CONT | ❌ | 🟡 | ✅ | ✅ | ✅ | 🟡 | 🟡 | ✅ | 🟡 | ❌ | ❌ |
2929
| CONV_2D | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ |
3030
| CONV_2D_DW | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
31-
| CONV_3D | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
31+
| CONV_3D | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
3232
| CONV_TRANSPOSE_1D | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
3333
| CONV_TRANSPOSE_2D | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
3434
| COS | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
@@ -46,7 +46,7 @@ Legend:
4646
| EXPM1 | ❌ | ❌ | ✅ | 🟡 | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ |
4747
| FILL | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ |
4848
| FLASH_ATTN_EXT | ❌ | 🟡 | ✅ | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 | ❌ | ❌ |
49-
| FLOOR | ❌ | ❌ | ✅ | 🟡 | ❌ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
49+
| FLOOR | ❌ | ❌ | ✅ | 🟡 | ✅ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
5050
| GATED_DELTA_NET | ❌ | ❌ | ✅ | ❌ | 🟡 | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ |
5151
| GATED_LINEAR_ATTN | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
5252
| GEGLU | ❌ | ✅ | ✅ | ✅ | 🟡 | ✅ | ✅ | 🟡 | ✅ | ❌ | ❌ |
@@ -84,10 +84,10 @@ Legend:
8484
| REPEAT_BACK | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
8585
| RMS_NORM | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
8686
| RMS_NORM_BACK | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
87-
| ROLL | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
87+
| ROLL | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
8888
| ROPE | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
8989
| ROPE_BACK | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
90-
| ROUND | ❌ | ❌ | ✅ | 🟡 | ❌ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
90+
| ROUND | ❌ | ❌ | ✅ | 🟡 | ✅ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
9191
| RWKV_WKV6 | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
9292
| RWKV_WKV7 | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
9393
| SCALE | ❌ | 🟡 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
@@ -116,6 +116,6 @@ Legend:
116116
| TIMESTEP_EMBEDDING | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
117117
| TOP_K | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
118118
| TRI | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
119-
| TRUNC | ❌ | ❌ | ✅ | 🟡 | ❌ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
119+
| TRUNC | ❌ | ❌ | ✅ | 🟡 | ✅ | ❌ | 🟡 | 🟡 | ✅ | ❌ | ❌ |
120120
| UPSCALE | ❌ | 🟡 | ✅ | ✅ | ✅ | 🟡 | ✅ | ✅ | ❌ | ❌ | ❌ |
121-
| XIELU | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ |
121+
| XIELU | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ |

0 commit comments

Comments
 (0)