Fix XGBRegressor binary:logistic ONNX conversion producing wrong predictions by JOSH1024 · Pull Request #771 · onnx/onnxmltools

JOSH1024 · 2026-06-05T07:05:23Z

Summary

Fixes #726.

XGBRegressor models using objective="binary:logistic" can produce incorrect ONNX predictions when tree outputs are non-zero (for example when subsample < 1.0).

The converter currently uses base_score directly as base_values in the TreeEnsembleRegressor, which mixes probability-space values with tree outputs that are represented in logit space.

This change:

Converts base_score from probability space to logit space for binary:logistic.
Emits an intermediate TreeEnsembleRegressor output.
Adds a Sigmoid node after the tree ensemble output.
Adds a regression test reproducing the issue reported in Converted XGBoost model outputs wrong results #726.

Reproduction

Using the reproducer from #726:

Before this change:

Expected: 0.64148873
ONNX:     0.69600594
Allclose: False

After this change:

Expected: 0.64148873
ONNX:     0.64148873
Allclose: True

Testing

Added regression test:

tests/xgboost/test_xgboost_issues.py::test_issue_726_binary_logistic_subsample

Executed:

pytest tests/xgboost/test_xgboost_issues.py -v

Result:

tests/xgboost/test_xgboost_issues.py::TestXGBoostIssues::test_issue_676 PASSED
tests/xgboost/test_xgboost_issues.py::TestXGBoostIssues::test_issue_726_binary_logistic_subsample PASSED

========================= 2 passed =========================

Also ran:

pytest tests/xgboost -v

Observed no additional failures relative to main.

Validation

Reproduced the issue locally.
Verified the generated ONNX graph changes from:

TreeEnsembleRegressor

to:

TreeEnsembleRegressor -> Sigmoid

Added a regression test covering the reported scenario.
Verified ONNX predictions now match XGBoost predictions for the reproducer.

Related: #758

JOSH1024 · 2026-06-05T08:54:02Z

hi contributers
@vinitra, @wenbingl , @prasanthpul and @xadupre it would be wonderful if any of you could u pls review my pr ??

JOSH1024 · 2026-06-09T05:14:40Z

@take-cheeze could u please review this pr ?

take-cheeze · 2026-06-09T06:30:44Z

@JOSH1024 Sorry. Seems I don't have write access to this repository
@onnx/sig-converters May approve this PR to merge

JOSH1024 · 2026-06-09T09:25:46Z

@take-cheeze thanks for letting me know

JOSH1024 · 2026-06-09T09:27:37Z

@andife it would be wonderful if u could review this pr?

Copilot

Pull request overview

Fixes incorrect ONNX predictions for xgboost.XGBRegressor models using objective="binary:logistic" by aligning the ONNX graph with XGBoost’s logit-space accumulation + sigmoid behavior.

Changes:

Convert base_score from probability space to logit space for binary:logistic when populating TreeEnsembleRegressor.base_values.
Emit an intermediate TreeEnsembleRegressor output and apply a Sigmoid node to produce probability outputs matching XGBoost.
Add a regression test reproducing issue #726 with subsample < 1.0.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
tests/xgboost/test_xgboost_issues.py	Adds regression test coverage for `binary:logistic` subsample scenario (but currently contains an invalid import that breaks tests).
onnxmltools/convert/xgboost/operator_converters/XGBoost.py	Adjusts XGBRegressor conversion for `binary:logistic` to use logit base values and an explicit Sigmoid post-processing node.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

JOSH1024 · 2026-06-09T09:56:31Z

@andife i have done the things which copilot said

andife · 2026-06-09T11:24:20Z

could you fix dco

JOSH1024 · 2026-06-09T11:26:42Z

yes done @andife

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

JOSH1024 · 2026-06-10T05:07:35Z

@justinchuby i have updated is it ok now ?

JOSH1024 · 2026-06-10T05:27:10Z

@andife Could you please review and merge this PR if you think the changes look correct? Thanks!

andife · 2026-06-10T13:28:37Z

@xadupre

JOSH1024 · 2026-06-10T16:27:44Z

@andife i am not understand why is ubuntu and windows checks are fail could u please help me ?

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

…ansform regression Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

JOSH1024 · 2026-06-10T18:40:09Z

@andife could u check now ?

…te_existing kwarg Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

JOSH1024 · 2026-06-10T19:10:58Z

@andife could u run the checks?

…core Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

JOSH1024 · 2026-06-11T05:11:58Z

@andife could u re run the checks?

JOSH1024 · 2026-06-11T05:33:03Z

@andife is this ok becasue those ubuntu and windows check fails are there in all the merged pr also ?? so if everything else looks good could u please approve it

JOSH1024 · 2026-06-11T06:11:55Z

@xadupre is this ok because those ubuntu and windows check fails are there in all the merged pr also ?? so if everything else looks good could u please approve it

The old guard skipped remapping any true/false child ID equal to 0, intending to skip dummy placeholders on LEAF nodes. But it also silently skipped valid branch pointers whose child happened to be node 0 (e.g. root's left child after set_new_numbers). This caused broken tree traversal in ONNX, producing wrong predictions. Fix: skip only when the current node is itself a LEAF. Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

JOSH1024 · 2026-06-11T08:31:15Z

The 2 failing XGBoost tests (test_xgb0_empty_tree_classifier, test_xgb_classifier_13) on windows-latest py==3.10 xgboost<2 are pre-existing failures unrelated to this change. They are skipped on xgboost>=2 and fail due to an existing compatibility issue between xgboost<2 and onnxruntime==1.16.3 on Windows.

JOSH1024 · 2026-06-11T08:31:36Z

@andife can u check now

JOSH1024 · 2026-06-12T05:05:18Z

@andife ??

andife · 2026-06-12T05:06:51Z

@xadupre Can you review this PR?

JOSH1024 · 2026-06-12T10:04:21Z

@xadupre could u please look into this pr ??

JOSH1024 · 2026-06-13T07:19:47Z

@xadupre could u pls look into this ??

JOSH1024 · 2026-06-15T06:22:17Z

@xadupre can u look into this

JOSH1024 · 2026-06-15T09:02:20Z

@andife could u pls help me because there is no response from @xadupre

andife · 2026-06-15T14:31:46Z

@onnx/sig-converters-approvers

JOSH1024 · 2026-06-16T04:33:32Z

@andife i have been requesting @xadupre and @onnx/sig-converters-approvers from last week and did'nt get any reply from their side so could u pls help me what should i do?

justinchuby · 2026-06-16T05:34:39Z

I'll ping Xavier tomorrow

JOSH1024 · 2026-06-17T05:23:39Z

@justinchuby could u help me here also

JOSH1024 · 2026-06-18T04:12:02Z

@justinchuby ??

…g, base_score logit transform, and TreeEnsembleClassifier label workarounds Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

JOSH1024 force-pushed the test-pr758-fix branch from 220d9e8 to 5203674 Compare June 5, 2026 07:09

andife requested a review from Copilot June 9, 2026 09:36

Copilot started reviewing on behalf of andife June 9, 2026 09:36 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread tests/xgboost/test_xgboost_issues.py Outdated

github-advanced-security AI found potential problems Jun 9, 2026

View reviewed changes

Comment thread tests/xgboost/test_xgboost_issues.py Fixed

JOSH1024 force-pushed the test-pr758-fix branch from b63eac7 to 2b081fa Compare June 9, 2026 09:53

JOSH1024 force-pushed the test-pr758-fix branch from 3a9952d to 0d1b329 Compare June 9, 2026 11:24

justinchuby requested a review from Copilot June 9, 2026 17:26

Copilot started reviewing on behalf of justinchuby June 9, 2026 17:26 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread onnxmltools/convert/lightgbm/operator_converters/LightGbm.py

andife requested a review from Copilot June 10, 2026 16:33

Copilot started reviewing on behalf of andife June 10, 2026 16:33 View session

Copilot AI reviewed Jun 10, 2026

View reviewed changes

Comment thread onnxmltools/convert/xgboost/operator_converters/XGBoost.py Outdated

Comment thread tests/xgboost/test_xgboost_converters.py Outdated

Comment thread tests/xgboost/test_xgboost_issues.py

JOSH1024 force-pushed the test-pr758-fix branch from 19b374f to f3a7ce4 Compare June 10, 2026 17:36

joshua added 4 commits June 10, 2026 23:07

Fix XGBRegressor binary logistic conversion

edf1522

Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

Fix XGBoost categorical test inputs

1dd5b99

Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

Stabilize binary classifier post transforms

61870b7

Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

Fix XGBRegressor binary:logistic ONNX conversion and LightGBM post_tr…

f0c4d1c

…ansform regression Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

joshua added 2 commits June 11, 2026 00:28

Fix base_score logit transform: gate on XGBoost version, drop overwri…

0e6ba8a

…te_existing kwarg Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

Fix binary classifier weights for onnxruntime >=1.22

3c667ae

Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

joshua added 2 commits June 11, 2026 10:29

Fix all-stump binary classifier: synthesize class weights from base_s…

63758ce

…core Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

Skip degenerate all-stump tests on XGBoost >=2; fix stump class weights

1fe1040

Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

JOSH1024 force-pushed the test-pr758-fix branch from 32729d9 to 9ff4905 Compare June 11, 2026 08:20

andife requested a review from xadupre June 12, 2026 05:06

JOSH1024 force-pushed the test-pr758-fix branch from cd7b880 to 11abb85 Compare June 15, 2026 06:08

JOSH1024 mentioned this pull request Jun 16, 2026

dumps OnnxDiscrepancyCheck result on disc so they can be picked up later and gathered for a dashboard microsoft/Olive#2503

Merged

5 tasks

JOSH1024 force-pushed the test-pr758-fix branch 2 times, most recently from bb1cba6 to 1b0078c Compare June 18, 2026 09:44

Fix SparkML/XGBoost binary classifier regressions: tree node remappin…

1b0078c

…g, base_score logit transform, and TreeEnsembleClassifier label workarounds Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>

Conversation

JOSH1024 commented Jun 5, 2026

Summary

Reproduction

Testing

Validation

Uh oh!

JOSH1024 commented Jun 5, 2026

Uh oh!

JOSH1024 commented Jun 9, 2026

Uh oh!

take-cheeze commented Jun 9, 2026

Uh oh!

JOSH1024 commented Jun 9, 2026

Uh oh!

JOSH1024 commented Jun 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

JOSH1024 commented Jun 9, 2026

Uh oh!

andife commented Jun 9, 2026

Uh oh!

JOSH1024 commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

JOSH1024 commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JOSH1024 commented Jun 10, 2026

Uh oh!

andife commented Jun 10, 2026

Uh oh!

JOSH1024 commented Jun 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JOSH1024 commented Jun 10, 2026

Uh oh!

JOSH1024 commented Jun 10, 2026

Uh oh!

JOSH1024 commented Jun 11, 2026

Uh oh!

JOSH1024 commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JOSH1024 commented Jun 11, 2026

Uh oh!

JOSH1024 commented Jun 11, 2026

Uh oh!

JOSH1024 commented Jun 11, 2026

Uh oh!

JOSH1024 commented Jun 12, 2026

Uh oh!

andife commented Jun 12, 2026

Uh oh!

JOSH1024 commented Jun 12, 2026

Uh oh!

JOSH1024 commented Jun 13, 2026

Uh oh!

JOSH1024 commented Jun 15, 2026

Uh oh!

JOSH1024 commented Jun 15, 2026

Uh oh!

andife commented Jun 15, 2026

JOSH1024 commented Jun 9, 2026 •

edited

Loading

JOSH1024 commented Jun 10, 2026 •

edited

Loading

JOSH1024 commented Jun 11, 2026 •

edited

Loading