Cosine distance returns ±Infinity for small nonzero float32 vectors, corrupting MATCH ranking; vec_normalize returns non-finite values (v0.1.9)

[JoeyLYZ666]

Current Behavior

With distance_metric=cosine, sufficiently small but still nonzero and representable float32 vectors can produce non-finite cosine distances. The same numerical issue also affects vec0 MATCH ordering and vec_normalize.

All vector elements below are finite float32 values. The apparent failure occurs during the squared-magnitude calculation: for example, 1e-25 itself is representable and nonzero, but (1e-25)^2 = 1e-50 underflows in float32.

Scalar vec_distance_cosine against [1, 0]:

second operand	expected cosine distance	returned
[1, 0]	0	0.0
[1e-20, 0]	0	about -2.7e-06
[1e-25, 0]	0	-Infinity
[-1e-25, 0]	2	+Infinity
[1e-25, 1e-25]	1 - 1/sqrt(2) ≈ 0.292893	-Infinity

This also violates positive scale invariance. For any nonzero vector v and positive scalar c,

cosine_distance(q, v) == cosine_distance(q, c * v)

should hold, since positive scaling does not change vector direction.

For example:

q = [1, 0]
v = [1, 0]
c * v = [1e-25, 0]

vec_distance_cosine(q, v)     = 0.0
vec_distance_cosine(q, c * v) = -Infinity

MATCH ranking

The same behavior affects ranking in a cosine vec0 table.

Example rows:

id1 = [0.9, 0.1]
id2 = [1e-25, 0]
id3 = [1e-25, 1e-25]
id4 = [-1e-25, 0]
id5 = [1e-30, 0]

with query:

[1, 0]

Their true cosine distances are approximately:

id2 = 0
id5 = 0
id1 = 0.006116
id3 = 0.292893
id4 = 2

However, on v0.1.9, the small-norm rows can receive non-finite distances such as -Infinity or +Infinity.

A representative result is:

id5  -Infinity
id3  -Infinity
id2  -Infinity
id1   0.0062
id4  +Infinity

The clearest ordering violation is between id1 and id3.

id1 has a true cosine distance of about 0.006116, while id3 has a true distance of about 0.292893, so id1 should rank ahead of id3.

Instead, id3 receives -Infinity and is ranked ahead of id1.

vec_normalize

The same issue is visible in vec_normalize:

vec_normalize([1e-15, 0])  -> approximately [1, 0]
vec_normalize([1e-20, 0])  -> approximately [1.0000026, 0]
vec_normalize([1e-22, 0])  -> approximately [1.0097, 0]
vec_normalize([1e-25, 0])  -> [Infinity, NaN]

For a nonzero vector [1e-25, 0], the mathematically normalized result is [1, 0].

Steps to Reproduce

1. Install sqlite-vec v0.1.9:

pip install sqlite-vec==0.1.9

2. Run the following minimal reproducer:

import math
import sqlite3
import struct

import sqlite_vec


def blob(v):
    return struct.pack(f"{len(v)}f", *v)


con = sqlite3.connect(":memory:")
con.enable_load_extension(True)
sqlite_vec.load(con)

query = [1.0, 0.0]
original = [1.0, 0.0]
scaled = [1e-25, 0.0]

d_original = con.execute(
    "SELECT vec_distance_cosine(?, ?)",
    (blob(query), blob(original)),
).fetchone()[0]

d_scaled = con.execute(
    "SELECT vec_distance_cosine(?, ?)",
    (blob(query), blob(scaled)),
).fetchone()[0]

print("sqlite-vec:", con.execute("SELECT vec_version()").fetchone()[0])
print("distance(query, original):", d_original)
print("distance(query, scaled):  ", d_scaled)

assert math.isfinite(d_original)
assert math.isfinite(d_scaled)
assert math.isclose(
    float(d_original),
    float(d_scaled),
    rel_tol=1e-6,
    abs_tol=1e-6,
)

Observed on v0.1.9:

distance(query, original): 0.0
distance(query, scaled):   -inf

The assertion fails because positive scaling changes a finite cosine distance into -Infinity.

The issue can also be reproduced directly with SQL:

SELECT vec_distance_cosine(
  vec_f32('[1,0]'),
  vec_f32('[1e-25,0]')
);

which returns:

-Infinity

Additional affected behavior can be reproduced with:

CREATE VIRTUAL TABLE t USING vec0(
  id integer primary key,
  v float[2] distance_metric=cosine
);

and with:

SELECT vec_normalize(vec_f32('[1e-25,0]'));

which returns non-finite values.

Expected Behavior

For nonzero vectors, cosine distance should remain finite and preserve positive scale invariance.

In particular:

cosine_distance([1,0], [1,0])

and

cosine_distance([1,0], [1e-25,0])

should both be approximately 0.

More generally, cosine distances for nonzero finite vectors should remain within the expected [0, 2] range, subject to normal floating-point error.

vec0 MATCH ordering should be consistent with those cosine distances. A vector with true distance approximately 0.0061 should not rank behind one with true distance approximately 0.2929 because the latter was converted to -Infinity.

vec_normalize should return a finite unit vector for a nonzero representable input such as [1e-25, 0], or explicitly reject inputs it cannot safely normalize, rather than returning [Infinity, NaN].

Likely Mechanism

The current cosine-distance and normalization calculations appear to accumulate squared magnitudes in float32.

For a vector component:

x = 1e-25

the input remains nonzero and representable, but:

x * x = 1e-50

underflows during the float32 squared-magnitude calculation.

This can make the computed magnitude zero even though the original vector is nonzero.

For cosine distance, this effectively leads to a division by zero:

dot / sqrt(magnitude_squared)

which can produce +Infinity, -Infinity, or other non-finite results depending on the inputs.

The gradual loss of accuracy observed in vec_normalize around 1e-20 to 1e-22, followed by [Infinity, NaN] around 1e-25, appears consistent with the same numerical issue.

Environment

• sqlite-vec v0.1.9 (vec_version())
• Installed from the sqlite-vec Python package
• Python 3.12
• SQLite 3.x
• Windows 11
• In-memory SQLite database
• No other extensions loaded

The issue was originally found using a metamorphic relation that checks whether positive scaling preserves cosine distance and nearest-neighbor ordering.