Skip to content

trim ui-test-artist levels and move them into the ui test partition - #74543

Draft
davidsbailey wants to merge 3 commits into
migrate-ui-test-other-v2from
migrate-ui-test-artist-levels-and-trim
Draft

trim ui-test-artist levels and move them into the ui test partition#74543
davidsbailey wants to merge 3 commits into
migrate-ui-test-other-v2from
migrate-ui-test-artist-levels-and-trim

Conversation

@davidsbailey

Copy link
Copy Markdown
Member

Testing story

Deployment notes

@davidsbailey
davidsbailey force-pushed the migrate-ui-test-artist-levels-and-trim branch from 0db591e to 76d4837 Compare August 11, 2026 21:55
davidsbailey and others added 3 commits August 11, 2026 23:00
Only three of the ten Artist levels are load bearing, and the unit is
about to have its levels cloned into "UI Test "-prefixed copies under
dashboard/test/ui/config. Cloning the other seven would put seven level
definition files in the test tree that no test reads.

Kept, and why each:

  1 Standalone_Artist_1  the unit's landing level, reached by
                         /api/hour/begin/ui-test-artist -> /s/ui-test-artist/reset
  2 Standalone_Artist_2  loaded by student_learning/maze_signed_{in,out}.feature
                         to show that progress in ui-test-maze does not appear
                         in a different unit. Left at position 2 so neither
                         feature changes.
  3 Standalone_Artist_9  the only remaining level with free_play true, so
                         pressing runButton alone completes it, and now the
                         last level in the lesson, which is what turns the
                         congrats dialog's button into Finish.

Dropping Standalone_Artist_9 from position 10 to position 3 is the sole
feature edit: hoc_certificates.feature drives that level through
runButton -> Finish -> /congrats -> certificate.

The script_json also picks up a fresh serialized_at and an empty
jit_pl_concepts_lessons key. Both are serializer drift, already present in
the eight most recently regenerated ui-test script_jsons.

No [reset db] needed. ScriptSeed.destroy_outdated_objects covers ScriptLevel
and LevelsScriptLevel, so the incremental seed deletes the seven removed
script_levels once the file's md5 changes. Measured by reseeding the old
file back to ten and the new one down to three.

Verified against a local dashboard, since test-studio still serves the
ten-level unit: hoc_certificates.feature:9, maze_signed_out.feature:6,
maze_signed_in.feature:4 and ai_differentiation_chat.feature:73 pass under
./runner.rb -l, and ai_differentiation_threads.feature passes under
./runner.rb -l --eyes.

Needs a baseline accept in Applitools: ai_differentiation_threads takes four
full-page snapshots of /courses/ui-test-artist/units/1, which now shows three
progress bubbles instead of ten.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run bin/curriculum/clone_ui_test_levels.rb on ui-test-artist, so the unit
stops referencing production levels:

  Standalone_Artist_1 -> UI Test Standalone_Artist_1   (new)
  Standalone_Artist_2 -> UI Test Standalone_Artist_2   (new)
  Standalone_Artist_9 -> UI Test Standalone_Artist_9   (already existed)

Only two level files are new. UI Test Standalone_Artist_9 was cloned by
#73718 when ui-test-csf was migrated, and the script's get-or-create found
it by name, so the two units share the one clone rather than minting a
second copy.

The clones differ from their sources only in created_at, audit_log, and the
ordering of null-valued properties. These are plain Artist levels: no
contained levels, no project template, no sublevels, no encrypted
properties, no rubrics.

levelbuilder_mode is off in locals.yml here, so the script was loaded under
`rails runner` with Rails.application.config.levelbuilder_mode set for that
process. Its `__FILE__ == $0` guard exists for this; the code path is
otherwise identical.

Verified locally: `rake seed:custom_levels_ui_tests` picks up both new files,
reseeding ui-test-artist from the regenerated script_json lands all three
script_levels on the UI Test copies, and the five features that exercise the
unit still pass against a local dashboard — hoc_certificates.feature:9,
maze_signed_out.feature:6, maze_signed_in.feature:4,
ai_differentiation_chat.feature:73, and ai_differentiation_threads.feature
under --eyes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…artition

The "Add a level using the lesson edit page" scenario of
teacher_tools/levelbuilder/lesson_edit_page.feature searches the Add Level
dialog for "Standalone_Artist_1" and adds the first result. With the clones the
commit below adds, that search returns "UI Test Standalone_Artist_1" first --
level search orders by updated_at descending and the ui test seed tasks run
after the production ones, so a clone always sorts ahead of the level it was
copied from -- and saving the lesson failed:

  Error Saving: RuntimeError at /lessons/137
  UI Test levels may only be used in ui-test-* scripts, but
  "temp-script-1786473466-625459" references: UI Test Standalone_Artist_1

Teaching level search to hide UI Test levels would fix the symptom only until
production levels stop being seeded in test environments, which is where the
partitioned curriculum data work is headed: then this test has no production
Artist level to search for at all. So point it at test content on both sides
of the reference instead.

- TestController#create_migrated_script names its unit
  "ui-test-temp-script-...", #create_course names its course
  "ui-test-temp-course-...", and the two levelbuilder steps that type a name
  into the new unit and new course forms do the same. A ui-test- unit is the
  only kind that may reference a UI Test level.
- create_migrated_script seeds its script_level with "UI Test Applab test",
  the copy made earlier in this stack when the csp ui test units were
  migrated, and now uses find_by_name! so a missing level fails loudly
  instead of pushing nil.
- The feature searches for and asserts on "UI Test Standalone_Artist_1".

The unit rename and the seed level are one change, not two: a temp unit named
"temp-script-..." cannot hold "UI Test Applab test" either.

Verified by replaying the failing path in `rails runner`, building the temp
unit the way create_migrated_script does and attaching both levels to a
script_level:

  ui-test-temp-script-...: UI Test Applab test            -> attached
  ui-test-temp-script-...: UI Test Standalone_Artist_1    -> attached
  temp-script-...:         UI Test Applab test            -> REFUSED
  temp-script-...:         UI Test Standalone_Artist_1    -> REFUSED

the refusals carrying the same message drone reported. The dialog's own query,
name LIKE %UI Test Standalone_Artist_1% narrowed to Artist, returns exactly
that level.

Nothing else keys off the prefix at runtime: it selects a definition file's
directory, which only matters when levelbuilder_mode is on, and it satisfies
ScriptLevel's partition validation. The four features using these steps --
lesson_edit_page, new_unit_page, script_edit_page, modular_courses -- assert
nothing about the temp names.

Two things to know. The features were not run locally: the lesson edit page
needs levelbuilder_mode on the dev server, and levelbuilder features write
temp unit and course files into the working tree. Drone covers them. When they
are run that way, those stray files now land under dashboard/test/ui/config,
which rake seed:ui_test globs, so delete them before committing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@davidsbailey
davidsbailey force-pushed the migrate-ui-test-artist-levels-and-trim branch from 76d4837 to 8a252e5 Compare August 11, 2026 23:15
@davidsbailey
davidsbailey changed the base branch from migrate-ui-test-csp-levels to migrate-ui-test-other-v2 August 11, 2026 23:26
@davidsbailey davidsbailey changed the title Migrate UI test artist levels and trim trim ui-test-artist levels and move them into the ui test partition Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant