Skip to content

soar-eval: per-module eval harness for comparing architectural changes #581

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
kimjune01 wants to merge 6 commits into
base: development
Choose a base branch
from
Draft

soar-eval: per-module eval harness for comparing architectural changes #581

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
2948386
soar-eval: transfer + per-module eval harness
kimjune01 Apr 11, 2026
3ed17d3
rename eval.py to transfer_eval.py for clarity
kimjune01 Apr 12, 2026
5673550
remove experimental RL-substate files from PR scope
kimjune01 Apr 12, 2026
8517326
fix agent column in diff output, use relative paths in base agent
kimjune01 Apr 12, 2026
14834fd
soar-eval: add policy.json with timing noise thresholds
kimjune01 Apr 12, 2026
4147546
soar-eval: fix path independence, add smem/epmem/perf suites
kimjune01 Apr 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions soar-eval/agents/bw-op-subgoal-base.soar
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# BW-Op-Subgoal agent WITHOUT the hardcoded initialization.
# The eval harness sources a generated task initialization separately.
#
# Path is relative to the repo root. Run from the repo root or set
# the working directory before sourcing this file.

pushd "UnitTests/SoarTestAgents/Chunking/tests/BW-Op-Subgoal/BW-Op-Subgoal"
source _firstload.soar
pushd all
source all_source.soar
popd
pushd elaborations
source elaborations_source.soar
popd
source _readme.soar
# NOTE: initialize-blocks-world.soar is NOT sourced — task generator provides it
source move-block.soar
pushd move-block
source move-block_source.soar
popd
popd
Loading