Skip to content

{{VAR}} resolves in action steps but NOT in the final-verification checkpoint (literal template compared) #87

@komal-lt

Description

@komal-lt

Summary

{{VAR}} references resolve correctly in action steps but are NOT resolved in a step's final-verification checkpoint — the checkpoint compares against the literal template string (e.g. {{global.KANE_PROJECT}}) instead of the resolved value, so the assertion fails even though the page is correct.

Environment

@testmuai/kane-cli 0.4.1, --variables-file variables.json with KANE_PROJECT = "CLI Project".

Repro

## Search for the dogfood project
Open {{TM_BASE_URL}}/projects. Type "{{KANE_PROJECT}}" into the search box.

## Verify the filtered result
Verify the filtered projects list contains a project named "{{KANE_PROJECT}}".
  • Step "Search" resolves the variable: it types CLI Project into the search box (correct).
  • Step "Verify" does not: the checkpoint looks for a project literally named {{global.KANE_PROJECT}}.

Actual

Run summary (verification step):

The run attempted to confirm that the Projects list contains a specific expected project (shown as “{{global.KANE_PROJECT…}}” in the step description). … the search box contains “CLI Project”, and the list shows a single project card titled “CLI Project”. The verification step failed because the expected project name did not appear in the filtered results.

So the page is correct (CLI Project is shown), but the checkpoint compared it to the unresolved {{global.KANE_PROJECT}} template → false failure.

Expected

{{VAR}} should resolve consistently — in verification/assertion checkpoints, not only in action steps. The namespaced {{global.X}} / {{secrets.user.X}} form that appears in checkpoints suggests the resolver runs for actions but the verification path keeps the raw template.

Workaround

Avoid embedding {{VAR}} as a quoted exact-match in an assertion; phrase it relative to a prior step ("the project you searched for"). But assertions referencing variables should just work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions