Test IT Python Integrations

The repository contains new versions of adapters for python test frameworks.

Pytest:

Behave:

Nose:

Robotframework:

Commons:

Compatibility

Test IT	Behave	Nose	Pytest	RobotFramework
3.5	2.0	2.0	2.0	2.0
4.0	2.1	2.1	2.1	2.1
4.5	2.5	2.5	2.5	2.5
4.6	2.8	2.8	2.8	2.8
5.0	3.2	3.2	3.2	3.2
5.2	3.3	3.3	3.3	3.3
5.3	3.6.1.post530	3.6.1.post530	3.6.1.post530	3.6.1.post530
5.4	3.6.6.post540	3.6.6.post540	3.6.6.post540	3.6.6.post540
5.5	3.10.1.post550	3.10.1.post550	3.10.1.post550	3.10.1.post550
5.6	3.11.6.post560	3.11.6.post560	3.11.6.post560	3.11.6.post560
5.7	4.2.1.post570	4.2.1.post570	4.2.1.post570	4.2.1.post570
Cloud	4.2.0 +	4.2.0 +	4.2.0 +	4.2.0 +

1. For current versions, see the releases tab.
2. Starting with 5.2, we have added a TMS postscript, which means that the utility is compatible with a specific enterprise version.
3. If you are in doubt about which version to use, check with the support staff. support@yoonion.ru

What's new in 4.0.0?

• New logic with a fix for test results loading

• Added sync-storage subprocess usage for worker synchronization on port 49152 by defailt.

• importRealtime=false is a default mode (previously true)

• limitations:

• Versions before 4.2.2 are not compatible with adapterMode=2 when using parallelization (e.g., pytest --testit -n 4). Please use adapterMode=1or update to 4.2.2+ .

How to run 4.0+ locally?

You can change nothing, it's full compatible with previous versions of adapters for local run on all OS.

How to run in an isolated environment?

The adapter tries to download the correct sync-storage version from GitHub. If that is not possible, prepare the sync-storage binary yourself:

1. Find the sync-storage version for your adapter release: check out the release tag and look for the SYNC_STORAGE_VERSION value. You can use our script.

2. Download the correct binary for your OS and version from our release page.

3. Place the downloaded binary at ./build/.caches/syncstorage-v0.3.3-windows_amd64.exe in your test project (the version in the filename is an example).

4. Run your tests as usual; the adapter will pick up and run the version specified in SYNC_STORAGE_VERSION.

How to run 4.0+ with CI/CD?

For CI/CD pipelines, we recommend starting the sync-storage instance before the adapter and waiting for its completion within the same job.

It can be OK for adapterMode=2 and automatic creation of new test-run + call for curl -v http://127.0.0.1:49152/wait-completion || true in the end.

There is a guide how to do everything with adapterMode 1 or 0:

You can see how we implement this here.

• to get the latest version of sync-storage, please use our script

• To download a specific version of sync-storage, use our script and pass the desired version number as the first parameter. Sync-storage will be downloaded as .caches/syncstorage-linux-amd64

1. Create an empty test run using testit-cli or use an existing one, and save the testRunId. 1.1 (alternative) You can use curl + jq to create empty test run, there is an example for github actions:

mkdir -p "$(dirname "${{ env.TEMP_FILE }}")"
BASE_URL="${{ env.TMS_URL }}"
BASE_URL="${BASE_URL%/}"
BODY=$(jq -nc \
    --arg projectId "${{ env.TMS_PROJECT_ID }}" \
    --arg name "${{ env.TMS_TEST_RUN_NAME }}" \
    '{projectId: $projectId, name: $name}')

curl -sS -f -X POST "${BASE_URL}/api/v2/testRuns" \
    -H "accept: application/json" \
    -H "Content-Type: application/json" \
    -H "Authorization: PrivateToken ${{ env.TMS_PRIVATE_TOKEN }}" \
    -d "$BODY" \
    | jq -er '.id' > "${{ env.TEMP_FILE }}"

echo "TMS_TEST_RUN_ID=$(<${{ env.TEMP_FILE }})" >> $GITHUB_ENV
echo "TMS_TEST_RUN_ID=$(<${{ env.TEMP_FILE }})" >> .env
export TMS_TEST_RUN_ID=$(<${{ env.TEMP_FILE }})

2. Start sync-storage with the correct parameters as a background process (alternatives to nohup can be used). Stream the log output to the service.log file:

nohup .caches/syncstorage-linux-amd64 --testRunId ${{ env.TMS_TEST_RUN_ID }} --port 49152 \
    --baseURL ${{ env.TMS_URL }} --privateToken ${{ env.TMS_PRIVATE_TOKEN }}  > service.log 2>&1 & 

3. Start the adapter using adapterMode=1 or adapterMode=0 for the selected testRunId.
4. Wait for sync-storage to complete background jobs by calling:

curl -v http://127.0.0.1:49152/wait-completion?testRunId=${{ env.TMS_TEST_RUN_ID }} || true

5. You can read the sync-storage logs from the service.log file.

Legacy behavior

(Hint) To enable legacy behavior without sync-storage you can use flag legacyWorkflow=true (TMS_LEGACY_WORKFLOW)

General

Supported test frameworks:

1. Pytest
2. Behave
3. RobotFramework
4. Nose

🚀 Warning

Since 3.0.0 version:

• If the externalId annotation is not specified, then its contents will be a hash of a fully qualified method name.