Skip to content

Add results from the openmm840 qa#117

Open
jthorton wants to merge 22 commits into
mainfrom
openmm840_qa
Open

Add results from the openmm840 qa#117
jthorton wants to merge 22 commits into
mainfrom
openmm840_qa

Conversation

@jthorton

Copy link
Copy Markdown
Contributor

A draft of the results object from a local qa run of openmm-8.4.0.

@IAlibay IAlibay left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of small thoughts.

Comment thread openfe_benchmarks/results/2026-03-18-openmm-840-qa-testing/submission.yaml Outdated
Comment thread openfe_benchmarks/results/2026-03-18-openmm-840-qa-testing/submission.yaml Outdated
Comment thread openfe_benchmarks/results/2026-03-18-openmm-840-qa-testing/submission.yaml Outdated
@jthorton

Copy link
Copy Markdown
Contributor Author

TODO:

  • upload the raw results to zenodo
  • make the submission yaml again using the script and add the link the the zenodo entry
  • ask for review

@jaclark5 jaclark5 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here are some suggested changes to be consistent between the submission.yaml and the prepare_metadata_submission.py

Comment thread openfe_benchmarks/scripts/_example_generate_results_local.py Outdated
Comment thread openfe_benchmarks/results/2026-03-18-openmm-840-qa-testing/submission.yaml Outdated
Comment thread openfe_benchmarks/results/2026-03-18-openmm-840-qa-testing/submission.yaml Outdated
@jthorton

jthorton commented Jun 3, 2026

Copy link
Copy Markdown
Contributor Author

Update _example_plan_rbfe.py to make sure that the "system_name" and "system_group" are saved in the mapping annotations.

@jaclark5

jaclark5 commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator

I think I updated everything according to the meeting today.
@jthorton you'll have to:

  • rerun the results file generation
  • create zenodo record, here's an initial draft of the description output by the script
    zenodo_description.md

@jaclark5

jaclark5 commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

Update _example_plan_rbfe.py to make sure that the "system_name" and "system_group" are saved in the mapping annotations.

This should be resolved

Comment thread openfe_benchmarks/scripts/prepare_metadata_submission.py Outdated
Comment thread openfe_benchmarks/results/2026-03-18-openmm-840-qa-testing/submission.yaml Outdated
Comment thread openfe_benchmarks/scripts/prepare_metadata_submission.py
openmm_version: TODO
openff_toolkit_version: TODO
mapper: "KartografAtomMapper 1.2.0 (LSA)"
forcefield: ["ff14SB", "phosaa10", "tip3p_HFE_multivalent", "tip3p_standard"]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is listed here and in the protocol settings is this the searchable tag? If so we probably want the small molecule forcefield here as well.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or is the tags list intended to be the searchable keys?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tags list are intended to be searchable keys generally, but we could also remove the ff from there and expect it to be searchable from this field only, what do you think?

Gufe seems to separated the small molecule force field settings from forcefield which is the protein and solvent force field in OpenMMSystemGeneratorFFSettings. I did neglect to add the small molecule force field here. I'll resolve that

This submission describes the charge_annihilation_set, jacs_set RBFE benchmark
(charge_annihilation_set: egfr, irak4_s2, irak4_s3; jacs_set: p38, tyk2) prepared with
tip3p_HFE_multivalent/ff14SB/phosaa10/tip3p_standard and am1bcc_at. The submission contains 160
edges, 45 unique ligands, and 1 unique proteins. Note this means the charge annihilation sets are

@jthorton jthorton Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 unique proteins

This is wrong

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, it's because we don't set the component name or molprops, only the atoms and bonds are distinguishing features. @jthorton do you have a standard way of distinguishing proteins? If not, I'll figure something out.

summary: |
This submission describes the charge_annihilation_set, jacs_set RBFE benchmark
(charge_annihilation_set: egfr, irak4_s2, irak4_s3; jacs_set: p38, tyk2) prepared with
tip3p_HFE_multivalent/ff14SB/phosaa10/tip3p_standard and am1bcc_at. The submission contains 160

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Force fields are listed again but not the small molecule force field or charges.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed!

Comment on lines +267 to +272
if protocol_settings.partial_charges:
_add_value_with_keys(
self.partial_charges,
protocol_settings.partial_charges,
keys,
)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pulls charges from the protocol; if there are charges on the ligand, they take priority. In this case, it reports at_am1bcc (the default), but the submission used nagl. We should make this blank so the user has to fill it in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants