Agentic QA

Requested by: Zoltan F. Request: Add priority field to checks to enable prioritized execution and better control over check execution order. Current Problem: No option to specify check execution order All checks run in parallel with no control over sequencing Cannot prioritize critical checks for faster feedback Risk of "endless checks" problem with large check suites Proposed Solutions: Option 1: Sequential Ordering Add order field to checks Execute checks one by one in specified order Option 2: Priority-Based Execution (Preferred) Add priority levels (P1, P2, P3) Execute checks by priority (top priority first) Supports parallel execution within same priority level All checks could be P1 if needed More lightweight and flexible than strict ordering Use Cases: Run critical checks first for faster feedback on key functionality Better resource management for large check suites Control execution flow for dependent checks Business Value: Faster feedback loops for critical checks Reduced waiting time for essential test results Better resource allocation Prevents issues with unlimited parallel execution Current Workaround: Include complete flow in a single check to control execution sequence within that check. Quotes: "Is it possible to add prio field to the check list? With that top prio would run first to get feedback faster (as I see the tests run parallel)" "I think check suite is great idea but it must be limited somehow to be able to manage the scenarios and manage and separate the requirements and their coverage. From execution point it is also recommended to have some kind of prioritized execution or limit what could we execute parallel. Endless checks are dangerous item as I see."

Current Limitation: Users can only edit the test session name after a session is created. URLs and check suite selection cannot be modified. User Feedback: "Edit Session allows to edit only Session name" Requested Functionality: Enable users to edit existing test sessions to: Change the selected check suite Modify the list of URLs/pages to test Update session scope without creating a new session Use Cases: Fix URL typos without recreating the entire session Add or remove pages from an existing test session Switch to a different check suite for the same URLs Adjust test scope based on initial results Reuse session configuration with different parameters Current Workaround: Users must create a new test session to change URLs or check suite. Business Value: Improved user experience and workflow efficiency Reduced friction in test session management Faster iteration during test setup Less redundant session creation Technical Considerations: Need to consider impact on test history and audit trail May need to track session modifications for compliance Should preserve original configuration for reference Consider whether to allow edits on completed vs. running sessions Priority: Medium - Quality of life improvement Status: Currently by design (intentional limitation) Related: Commit f3a9a46 - "refactor redirect_to edit" (October 2025) Documented in MVP release notes as intentional limitation Source: Consolidated Feedback Analysis - November 2025

Requested by: Zoltan F. Request: Better management and visualization of multiple test session executions with ability to filter and group by check suite or scope. Current Behavior: Re-run functionality exists at test session level ✓ Execution counter shows total check executions Counter stays same if re-run with same scope (confusing) No grouping or filtering by check suite No grouping or filtering by scope (URLs/pages) Difficult to track execution history User Quote: "Is it possible to manage the runs and re-run under the original session somehow like the test cases what could be executed many times but the test cases is still the same." Desired Functionality: 1. Execution History Management View all executions of a test session as a list Each execution shows: timestamp, duration, results, scope changes Link related executions together Similar to test case execution history in traditional test management 2. Filtering and Grouping Options Filter test sessions by check suite Filter by scope (specific URLs or page types) Group re-runs of the same test session View execution trends over time 3. Better Execution Counter Clearer indication of what the counter represents Show number of times session was run vs total check executions Historical execution count Use Cases: Track how a specific check suite performs over time Compare results across multiple executions Filter to see only relevant test sessions Understand test session re-run history Audit trail for test executions Current Confusion: User re-executed the same session but execution number stayed the same, leading to confusion about what the counter represents. Status: 🔍 Discussion Needed Team wants to hear more about this use case in upcoming sessions. New design may address some aspects. Related: Execution counter will be clarified/redesigned in new UI New design may make current view "redundant" Priority: Medium - Workflow improvement for power users

Reported by: Oleh B. Issue: When a check fails during test session execution, the failure results and error reasons are not displayed in the UI until the page is manually reloaded. Specifics: Browser: Chrome Timing: Occurs during active test session execution Workaround: Manual page reload reveals the results Impact: Disrupts debugging workflow Users don't see failures in real-time Have to constantly refresh to check status Poor user experience during test monitoring Expected Behavior: Check failures should appear immediately when they occur Results and error reasons should be visible without page reload Real-time updates as checks complete (pass or fail) Technical Analysis: Likely issues: WebSocket/polling not updating UI with failure events Frontend state not refreshing on failure responses Race condition between check completion and UI update Browser-specific rendering issue (Chrome) Testing Needed: Does this occur in other browsers (Firefox, Safari, Edge)? Is it specific to one installation? Does it happen consistently or intermittently? Priority: Medium - Affects test monitoring and debugging

Reported by: Oleh B. Issue: URLs must be fully qualified with protocol and path. Simple domain entries fail, causing confusion and failed test runs. Current Behavior: ❌ galaxus.de - FAILS ✅ https://www.galaxus.de/ - WORKS Expected Behavior: System should automatically normalize URLs: Auto-add https:// if no protocol specified Handle www subdomain intelligently Add trailing slash if needed Validate and correct common URL formats Use Cases: Users copy domains from browser (often without protocol) Quick test setup without worrying about URL format Reduced friction in test session creation Business Impact: Improved user experience Reduced setup errors Fewer failed test sessions due to URL format issues More intuitive for non-technical users Technical Implementation: URL validation and normalization on frontend Helpful error messages if URL truly invalid Preview/confirmation of normalized URL Examples of URLs to Handle: example.com → https://example.com/ www.example.com → https://www.example.com/ example.com/page → https://example.com/page http://example.com → Keep as-is (respect user's protocol choice)

Reported by: Oleh B. Issue: During instance creation in the Project-Code field, after the first incorrect input, all subsequent correct inputs are incorrectly reflected as failed until the page is reloaded. Steps to Reproduce: Start instance creation process Enter incorrect value in Project-Code field Receive validation error (expected) Enter correct values in subsequent attempts All correct inputs still show as failed (bug) Reload page Validation works correctly again Impact: Confusing onboarding experience for new users Users may think correct values are wrong Forces page reload to continue setup Browser: Chrome Technical Analysis: Likely a frontend validation state issue where error state is not properly cleared after the first validation failure. Expected Behavior: First wrong input: Show validation error ✓ Subsequent correct inputs: Clear error and show success No page reload required Workaround: Reload the page after first validation error.

Feature Request: Implement a comprehensive check versioning system to be able to experiment with multiple check description versions. Parent Issue: Previously, when a check suite was modified, the changes were immediately reflected in all test sessions (including completed ones). -> This bug was fixed. Proposed Enhancement: Implement a check versioning system that: Version Tracking - Automatically create a new version when a check is modified - Store version number and author of each change - Maintain full history of all check modifications Allow version selection on single check reruns - Allow viewing and selection of different versions for single-check re-run - Ensure the execution details reflect with check version applied every time Audit Trail - Track who made changes and when - Provide changelog for each check Use Cases: Understanding test result trends over time Debugging why results changed between runs Rollback to previous check versions if needed Technical Implementation Considerations: Version storage strategy (separate table vs embedded documents) UI for version history display Version comparison tools

Issue: "I have an element on the page which can not be detected (slider) in DOM it is <input type='range' ....>" Root Cause: The computer vision model used for element detection does not recognize range slider inputs. This is not a Playwright MCP limitation but a limitation of the CV model. Impact: Cannot test pages with range slider inputs Blocker for applications that use sliders for user input Reproducible across different pages Technical Details: Element Type: <input type="range"> Detection Method: Computer Vision Status: Acknowledged as CV model limitation Potential Solutions: Train/update CV model to recognize slider elements Add fallback to DOM-based detection for known element types Provide manual element selector option for unsupported elements Business Impact: Blocks testing for e-commerce sites with price range filters, audio/video controls, and other slider-based interfaces.

User Request: "If you have some scenario with transferring via few pages than you have to specify all of them during Test Session creation. Otherwise Agentic QA will not transfer to other pages (next to the Home page) and will not run checks." Current Behavior: Entry points have to be listed explicitly during Test Session creation Each page is tested separately against the whole check suite No exploratory mode available by default Sequential multi-step checks are supported, but may lead to inconsistent results and can only verify against single expected behavior Desired Behavior: Automatic page navigation as part of check execution Entry of the landing page is sufficient -> agents explore the envinroment till they find a relevant entry point Ability to define flows that span multiple pages Context preservation across page transitions Support of multiple verifications across the same flow Current Limitation: Complex multi-page scenarios require deeper customization and specific check description "prompting" Workaround: List all pages explicitly during test session creation Include multiple steps into check descriptions