Skip to main content

Agentic Testing Loop

The Windows agentic loop builds the current hidden-plugin jar, stages it into the installed RuneLite sideload directory, launches RuneLite through ca.arnah.runelite.LauncherHijack, and treats only observed client postconditions as success.

Windows launcher​

runAgenticTest depends on FatJarWithHidden. WindowsRuneLiteLauncher then:

  1. refuses to stage while a RuneLite process is running;
  2. atomically replaces %USERPROFILE%\.runelite\sideloaded-plugins\n3Plugins.jar;
  3. parses %LOCALAPPDATA%\RuneLite\config.json;
  4. validates the bundled JVM, configured classpath, main class, and dependencies;
  5. clears the configured output directory;
  6. launches the configured JVM arguments, classpath, and ca.arnah.runelite.LauncherHijack;
  7. waits for RuneLite and propagates its exit code.

Locked jars, malformed config, missing dependencies, and child launch failures are terminal launcher errors.

Test options​

TestOptions parses these immutable JVM properties:

PropertyMeaning
n3.test.scriptExact target plugin name.
n3.test.profileExact, case-sensitive stored label beginning with TEST_; exactly one match is required.
n3.test.unlockPasswordOptional one-run unlock value. When absent, n3profiles.unlockPassword is used.
n3.test.outputDirResult and screenshot directory; default build/n3-test-results.
n3.test.timeoutSecondsGeneral test deadline; default 300. Tutorial Island uses 1800.

The runner is hidden, enabled by default, and inactive unless n3.test.mode=true. It calls ProfileStore.load(char[]), clears the temporary password array, and does not persist, log, screenshot, or serialize the supplied unlock value.

Startup reconciles the current GameState, so an already-visible login screen does not need another event. The runner distinguishes missing password, bad decryption, missing/duplicate profile, legacy login failure, Jagex login failure, 60-second login timeout, and target-plugin start failure. The wrapper starts only after an observed LOGGED_IN.

TutorialIslandTestPlugin sets startPaused=false, restores the prior configuration on shutdown, and fails when the base plugin cannot start.

Results and acceptance​

TestResultWriter writes schema-version 2 result.json and client screenshots below the output directory. When observable workflows are registered, each screenshot is paired with their immutable state, transition reason, tick, and bounded recent history under workflowTimeline. Dispatch acceptance is evidence that an operation was queued, not that it succeeded. Tests must record the before state, observed event/dispatch evidence, and the required after-state.

For Tutorial Island, live acceptance requires both hidden plugins in the runtime catalogue, automatic login to the exact test profile, native listener evidence where required, observed settings/door/dialogue postconditions, and fresh result/screenshots.

Run with RuneLite closed:

.\gradlew.bat runAgenticTest `
  -Pn3.test.script="Tutorial Island Test" `
  -Pn3.test.profile="TEST_1" `
  -Pn3.test.unlockPassword="." `
  -Pn3.test.timeoutSeconds=1800 `
  --console plain --no-daemon

Adding a wrapper​

  1. Register a hidden [Feature]TestPlugin in runelite-plugin.properties.
  2. Keep it inert outside test mode.
  3. Start the base plugin and fail immediately if startup is rejected.
  4. Observe completion/failure state rather than treating dispatch as success.
  5. Call TestRunnerPlugin.getInstance(pluginManager).completeTest(result).