SGD batching improvements, LBFGS and a Gaussian Process - #435

Open

Craigacp wants to merge 29 commits into

oracle:mainfrom

Craigacp:sgd-batching

Commits on Mar 19, 2026

Removing deprecated loss method.
Craigacp
committed
dce9e04
Expanding SGD losses so they natively work on batches. Following through into the trainers and parameters. This commit calls a lot of methods which need to be created in the la package.
Craigacp
committed
800d3e2
Implementing more la functions.
Craigacp
committed
3d47d92
Working on matrix batching.
Craigacp
committed
de8a985
Replacing ArrayMatrix with Matrix.aggregate.
Craigacp
committed
71d0088
Adding createTargetArray and fixing bugs in matrix multiplication order.
Craigacp
committed
0010c24
Small fixes for AbstractSGDTrainer and DenseMatrix, adding a densify option to DataOptions.
Craigacp
committed
e124786
Adding loss & batchLoss methods to SGDObjective, making the loss and gradient methods on SGDObjective return a record not a pair, and adding a few methods to DenseMatrix and DenseVector in support …
Craigacp
committed
52cad1d
Fixing a bug in DenseMatrix.subtract.
Craigacp
committed
998c033
Flipping the classification losses around so we're always minimising the loss.
Craigacp
committed
96c7d33
Adding L-BFGS, a linear regression using it, and a linear classifier using it.
Craigacp
committed
114c47d
Refactoring LinearTrainer to share code.
Craigacp
committed
49fb65c
More logging, less print statements.
Craigacp
committed
3ab71d8
Changing logging level in line search.
Craigacp
committed
0df75fb
More small logging changes to line search.
Craigacp
committed
e78742c
Adding in convergence limit check.
Craigacp
committed
61c379f
Adding smoke tests and a few small tidy ups.
Craigacp
committed
32818d1
Adding multi-label linear trainer.
Craigacp
committed
efe05b6
LinearTrainer uses example weights.
Craigacp
committed
026de8f
Initial implementation of gaussian process regression.
Craigacp
committed
4002df2
Batching predictions.
Craigacp
committed
380d834
Adding l2 regularisation to LinearTrainer.
Craigacp
committed
4c98d10
Updating gaussian process for protobuf serialization.
Craigacp
committed
f90a304
Finishing GP serialization.
Craigacp
committed
d098568
Fix linear trainer so it descends the gradient properly.
Craigacp
committed
c1ac1f1
Adding a Wolfe line search and fixing a bunch of direction bugs in LBFGS.
Craigacp
committed
0925d36
Bunch of small cleanups and moving the GP over to the new deserialisation cache world.
Craigacp
committed
aba863d
Rearranging the convergence checks in LBFGS so the line search is skipped if the gradient is zero.
Craigacp
committed
69b6475
Turning off the wine-quality test.
Craigacp
committed
06ad26a

Navigation Menu

Uh oh!

SGD batching improvements, LBFGS and a Gaussian Process - #435

SGD batching improvements, LBFGS and a Gaussian Process#435
Craigacp wants to merge 29 commits into
oracle:mainfrom
Craigacp:sgd-batching

Commits on Mar 19, 2026

Removing deprecated loss method.

Expanding SGD losses so they natively work on batches. Following through into the trainers and parameters. This commit calls a lot of methods which need to be created in the la package.

Implementing more la functions.

Working on matrix batching.

Replacing ArrayMatrix with Matrix.aggregate.

Adding createTargetArray and fixing bugs in matrix multiplication order.

Small fixes for AbstractSGDTrainer and DenseMatrix, adding a densify option to DataOptions.

Adding loss & batchLoss methods to SGDObjective, making the loss and gradient methods on SGDObjective return a record not a pair, and adding a few methods to DenseMatrix and DenseVector in support …

Fixing a bug in DenseMatrix.subtract.

Flipping the classification losses around so we're always minimising the loss.

Adding L-BFGS, a linear regression using it, and a linear classifier using it.

Refactoring LinearTrainer to share code.

More logging, less print statements.

Changing logging level in line search.

More small logging changes to line search.

Adding in convergence limit check.

Adding smoke tests and a few small tidy ups.

Adding multi-label linear trainer.

LinearTrainer uses example weights.

Initial implementation of gaussian process regression.

Batching predictions.

Adding l2 regularisation to LinearTrainer.

Updating gaussian process for protobuf serialization.

Finishing GP serialization.

Fix linear trainer so it descends the gradient properly.

Adding a Wolfe line search and fixing a bunch of direction bugs in LBFGS.

Bunch of small cleanups and moving the GP over to the new deserialisation cache world.

Rearranging the convergence checks in LBFGS so the line search is skipped if the gradient is zero.

Turning off the wine-quality test.