QualWeb CLI is a command-line accessibility evaluation tool that can check web pages with modules for Accessibility Conformance Testing (ACT) rules, WCAG techniques and best practices. This QualWeb CLI tutorial explains the basic command, how to organize multilingual URL checks, why report outcomes need interpretation and how EARL-style evidence can support a repeatable localization quality-assurance process.
Translating a site changes more than text. A localized route may render a different component, insert an additional heading, display a new validation message or load a language-specific template. A repeatable command-line audit helps teams compare target-language pages systematically, provided they also confirm that the target rendered successfully and that the findings belong to the correct language and product version.
This guide gives a practical workflow for installing QualWeb CLI, testing individual and multiple URLs, understanding ACT versus WCAG-technique findings, handling uncertain outcomes, preserving reports and connecting automated evidence with keyboard, screen-reader and linguistic testing. It also explains the maintenance move to the QualWeb monorepo and avoids treating a tool exit or a low issue count as accessibility certification.
- What to do first: define a meaningful user task, choose the correct language and reproduce the relevant page state.
- What this guide covers: repeatable URL-based checks with report provenance, EARL-style outputs and locale-specific evidence.
- What the tool can prove: results from its actual tested conditions, not complete translation quality or conformance.
- What to preserve: route, locale, build, viewport, interaction state, affected element and a reproducible test.
- What a useful finish looks like: confirmed repair, targeted retest, human review and a clear release decision.
What QualWeb CLI actually does
The QualWeb CLI project documents installation through npm and a `qw` command. Its basic form is `qw -u https://example.org/`, while documentation also shows a report format option such as `-r earl`. Check current package help and the maintained QualWeb monorepo before adopting options in CI, because legacy component repositories were archived in late 2024 and a command or version copied from an old example may not describe the version your team installed.
Automated accessibility reports contain rule-based observations, not a universal measure of user success. An ACT rule describes a consistent way to test a specific condition; a WCAG technique is a documented way of satisfying or investigating part of a success criterion; best practices can add helpful signals without themselves defining a formal conformance claim. The same element may be covered differently by different modules, so avoid summing all raw counts as if they represented unique failures.
For a multilingual site, build a manifest of actual locale URLs and define the user state expected at each one. A Japanese product page, a German checkout error and an Arabic account settings page are different test fixtures. Before evaluating output, verify status, title, main content, page language and redirects so the CLI is not quietly scanning a generic error screen or the source-language homepage.
This article focuses on QualWeb’s command-line evaluation model, its ACT and technique modules and evidence exchange, not an axe wrapper, Lighthouse score tutorial or general browser-inspector walkthrough.
The practical distinction between detected evidence and accessible user experience matters in every language. A rule result is tied to an element, configuration, view or sampled product state. A user experiences a complete action: understanding an instruction, choosing a control, recovering from an error and knowing what happened. For that reason, keep three questions separate: what the tool observed, what a person can successfully do, and whether the target-language content communicates the intended meaning. This guide repeatedly brings those questions together without pretending one can stand in for the others.
A reliable ten-stage workflow
1. Verify the package and documentation
Read the current monorepo and CLI help, check the runtime supported by the installed release and record the exact package version. Before drawing a conclusion, name the language, region and concrete page state under investigation. This makes an observation reproducible for someone who does not share the original tester’s browser session or assumptions. A successful process should distinguish a tool setup failure from a genuine product issue and should identify which evidence still needs human confirmation.
Confirm the supported runtime and the command provided by your installed QualWeb package rather than copying commands from archived repositories. Save package versions in the project lockfile. An evaluation that fails before loading a page must be recorded as a tooling or environment error.
2. Install in an isolated project
Use the documented npm installation or a project-scoped dependency, and keep lockfiles so results can be compared across builds. Before drawing a conclusion, name the language, region and concrete page state under investigation. This makes an observation reproducible for someone who does not share the original tester’s browser session or assumptions. A successful process should distinguish a tool setup failure from a genuine product issue and should identify which evidence still needs human confirmation.
Define the target as a real locale route and user purpose, such as a localized help article, registration screen or payment-failure page. Each target needs a known expected title and language. This prevents teams from comparing one translated product with a different generic fallback page.
3. Run one known public URL
Start with the documented `qw -u` form on a stable test page and inspect both command output and the rendered page in a browser. Before drawing a conclusion, name the language, region and concrete page state under investigation. This makes an observation reproducible for someone who does not share the original tester’s browser session or assumptions. A successful process should distinguish a tool setup failure from a genuine product issue and should identify which evidence still needs human confirmation.
Use the application rather than a proxy for translation state: load genuine localized resources, direction and regional configuration. Verify BCP 47 language tags and final navigated URLs. A successful HTTP response is insufficient if it serves a soft-404 or source-language template.
4. Create a locale URL manifest
List each language route and what page state it should represent, including translated titles and routes rather than guessing URL patterns. Before drawing a conclusion, name the language, region and concrete page state under investigation. This makes an observation reproducible for someone who does not share the original tester’s browser session or assumptions. A successful process should distinguish a tool setup failure from a genuine product issue and should identify which evidence still needs human confirmation.
Command-line runs require attention to when content becomes testable. If an application hydrates late or redirects behind authentication, the produced report may describe the wrong state. Inspect a representative evaluated page manually and document current tool support for such execution contexts.
5. Confirm navigation outcomes
Test redirects, authentication needs and failed requests before believing the accessibility evaluator reached the intended document. Before drawing a conclusion, name the language, region and concrete page state under investigation. This makes an observation reproducible for someone who does not share the original tester’s browser session or assumptions. A successful process should distinguish a tool setup failure from a genuine product issue and should identify which evidence still needs human confirmation.
Save command output and errors separately and know how the chosen version signals unsuccessful evaluation. A process finishing is not proof that checks completed. Test a deliberately invalid URL once in a controlled environment to learn how infrastructure errors are reported.
6. Choose useful report output
Capture the default machine or console result and evaluate EARL output when an evidence exchange format is needed. Before drawing a conclusion, name the language, region and concrete page state under investigation. This makes an observation reproducible for someone who does not share the original tester’s browser session or assumptions. A successful process should distinguish a tool setup failure from a genuine product issue and should identify which evidence still needs human confirmation.
Read the ACT, WCAG-technique and best-practice modules as separate evidence categories. ACT test rules formalize repeatable conditions; techniques give approaches related to WCAG; neither turns an entire product into a single pass/fail fact. Trace each important assertion to its actual target.
7. Interpret module boundaries
Keep ACT rules, WCAG techniques and best-practice outcomes distinguishable and map them to the underlying tested conditions. Before drawing a conclusion, name the language, region and concrete page state under investigation. This makes an observation reproducible for someone who does not share the original tester’s browser session or assumptions. A successful process should distinguish a tool setup failure from a genuine product issue and should identify which evidence still needs human confirmation.
When multiple modules mention the same element, compare the selectors and underlying behavior before opening several tickets. A shared missing name can appear in many reports and languages. Keep per-locale provenance while consolidating the change at its genuine shared component.
8. Cluster duplicate root causes
Aggregate repeated missing-label findings by shared component while retaining the evidence for every affected locale. Before drawing a conclusion, name the language, region and concrete page state under investigation. This makes an observation reproducible for someone who does not share the original tester’s browser session or assumptions. A successful process should distinguish a tool setup failure from a genuine product issue and should identify which evidence still needs human confirmation.
Preserve source URL, final URL, locale, product revision, tool version, rules and the original report artifact. If an EARL export is used, validate that it contains the assertions and subjects expected by the team. Data format alone does not confer completeness or authority.
9. Retest controlled fixes
Keep tool version, page version, state, viewport and locale stable as much as practicable between baseline and retest. Before drawing a conclusion, name the language, region and concrete page state under investigation. This makes an observation reproducible for someone who does not share the original tester’s browser session or assumptions. A successful process should distinguish a tool setup failure from a genuine product issue and should identify which evidence still needs human confirmation.
Upgrade-induced changes should be compared with a stable baseline, not interpreted immediately as product regressions. Rerun the same pages, identify new or changed rules, and separate environmental failures from new code defects. A repaired control also deserves one hands-on verification.
10. Add meaningful human testing
Follow up with keyboard navigation, assistive-technology checks, translated copy review and task completion tests on representative locales. Before drawing a conclusion, name the language, region and concrete page state under investigation. This makes an observation reproducible for someone who does not share the original tester’s browser session or assumptions. A successful process should distinguish a tool setup failure from a genuine product issue and should identify which evidence still needs human confirmation.
Use automated URL evaluation as a triage layer, then perform keyboard and assistive-technology tests on important flows. A linguist should review monetary terms, irreversible actions and error instructions. Record the human checks alongside structured tool output without merging their methods or claims.
24 field cases: multilingual problems, investigation and repair
The following exercises are separate test situations rather than a declaration that the tool will automatically catch every condition described. Each one joins an observable localization problem to a check that can be reproduced and a repair that can be independently confirmed. Some require visual, keyboard, semantic, linguistic or assistive-technology work in addition to QualWeb CLI. That boundary is essential: an auditor should not label an issue automatically checked if a human found it by another method.
Case 01: Run succeeds on the wrong page
Situation and user risk. A realistic test begins with the user, not the dashboard. A German route redirects to the English home page and the CLI reports no serious accessibility findings. The central question is whether someone working in the target language can recognize the necessary information and complete the intended action without guessing. A component can look polished in a design review while its spoken name, relationship or behavior fails under a different script, viewport or state.
Investigation. Check the final URL, document language, visible heading and response behavior before accepting the report. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Fix locale routing and rerun the intended German page rather than saving a misleading green report. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 02: A missing translation yields fallback
Situation and user risk. This situation is especially easy to miss when product teams copy a source-language screenshot into a translation ticket. A Japanese service error is replaced by a generic English 404 page during automated URL evaluation. The important evidence is the actual localized interaction, including the words a person encounters, the order in which controls appear and the consequence of making a selection. A faithful translation cannot rescue a component whose behavior is inaccessible.
Investigation. Verify that the requested resource exists in the fixture and record any fallback route and status. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Treat unavailable localized content as a product defect separate from any accessibility-rule outcome. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 03: CLI package differs from old README
Situation and user risk. Here is a useful quality-assurance exercise for a world-facing interface: A copied installation command refers to an archived component repository rather than the maintained QualWeb monorepo. Rather than assigning the entire problem to a translator or engineer immediately, separate what is visibly wrong from what can be established through inspection. That prevents a wording change from hiding a structural defect or a markup fix from leaving an incorrect instruction in place.
Investigation. Compare installed package version and command help with current source documentation. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Pin a verified version and update the team procedure before relying on legacy flags. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 04: ACT outcomes are misread
Situation and user risk. A realistic test begins with the user, not the dashboard. A report contains different outcomes across rules, and an analyst interprets all non-pass labels as failures. The central question is whether someone working in the target language can recognize the necessary information and complete the intended action without guessing. A component can look polished in a design review while its spoken name, relationship or behavior fails under a different script, viewport or state.
Investigation. Identify each outcome’s meaning and scope in the evaluator’s report and associated rule definition. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Report confirmed failures separately from inapplicable or unresolved checks and document follow-up steps. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 05: WCAG technique finding is overstated
Situation and user risk. This situation is especially easy to miss when product teams copy a source-language screenshot into a translation ticket. A technique-based check surfaces a pattern, but the team describes its result as a full WCAG 2.2 certification. The important evidence is the actual localized interaction, including the words a person encounters, the order in which controls appear and the consequence of making a selection. A faithful translation cannot rescue a component whose behavior is inaccessible.
Investigation. Trace the technique to the relevant success criterion and assess its applicability to the page. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. State the exact observed check and complete other necessary tests before making broader claims. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 06: Best-practice issue inflates compliance total
Situation and user risk. Here is a useful quality-assurance exercise for a world-facing interface: The report lists useful design suggestions and the dashboard treats them as formal conformance failures. Rather than assigning the entire problem to a translator or engineer immediately, separate what is visibly wrong from what can be established through inspection. That prevents a wording change from hiding a structural defect or a markup fix from leaving an incorrect instruction in place.
Investigation. Separate module categories and record the specific rationale attached to each finding. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Prioritize sound design improvements without misrepresenting the normative basis. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 07: Two modules flag the same element
Situation and user risk. A realistic test begins with the user, not the dashboard. A form field appears in both an ACT-related finding and a WCAG technique result. The central question is whether someone working in the target language can recognize the necessary information and complete the intended action without guessing. A component can look polished in a design review while its spoken name, relationship or behavior fails under a different script, viewport or state.
Investigation. Compare target selectors, rule description and the actual user-facing fault. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Create one root-cause ticket with linked results and retest both relevant checks. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 08: Arabic page has wrong language code
Situation and user risk. This situation is especially easy to miss when product teams copy a source-language screenshot into a translation ticket. A page renders Arabic text but declares the document language as English. The important evidence is the actual localized interaction, including the words a person encounters, the order in which controls appear and the consequence of making a selection. A faithful translation cannot rescue a component whose behavior is inaccessible.
Investigation. Audit the localized URL and confirm its actual HTML language attribute in browser source. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Set an accurate BCP 47 language tag and retest pronunciation manually with a screen reader. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 09: Polish labels remain source language
Situation and user risk. Here is a useful quality-assurance exercise for a world-facing interface: The interface route changes but error labels in a reusable template remain in English. Rather than assigning the entire problem to a translator or engineer immediately, separate what is visibly wrong from what can be established through inspection. That prevents a wording change from hiding a structural defect or a markup fix from leaving an incorrect instruction in place.
Investigation. Compare the rendered failure state with the expected localized fixture, not only the CLI summary. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Correct the localization resource mapping and run the evaluation against that state again. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 10: Locale URLs vary by region
Situation and user risk. A realistic test begins with the user, not the dashboard. French for France and French for Canada share most wording but different payment disclosures and navigation paths. The central question is whether someone working in the target language can recognize the necessary information and complete the intended action without guessing. A component can look polished in a design review while its spoken name, relationship or behavior fails under a different script, viewport or state.
Investigation. Specify both route variants in the URL manifest and confirm which product configuration each renders. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Test materially distinct regional variants instead of collapsing all French routes into one sample. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 11: Login wall blocks the scan
Situation and user risk. This situation is especially easy to miss when product teams copy a source-language screenshot into a translation ticket. A protected settings page redirects CLI requests to the sign-in screen even though authenticated humans can reach it. The important evidence is the actual localized interaction, including the words a person encounters, the order in which controls appear and the consequence of making a selection. A faithful translation cannot rescue a component whose behavior is inaccessible.
Investigation. Inspect the actual document returned under the CLI execution environment and its authentication state. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Use a supported authenticated evaluation workflow or separately inspect the rendered state in browser tools. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 12: Single-page application loads late
Situation and user risk. Here is a useful quality-assurance exercise for a world-facing interface: The evaluation occurs before asynchronous translation bundles have finished replacing placeholders. Rather than assigning the entire problem to a translator or engineer immediately, separate what is visibly wrong from what can be established through inspection. That prevents a wording change from hiding a structural defect or a markup fix from leaving an incorrect instruction in place.
Investigation. Compare the evaluated DOM with a manual page captured after all meaningful content settles. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Use a tool-supported readiness strategy where available, or validate the state through a controlled browser check. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 13: CJK line breaks change control area
Situation and user risk. A realistic test begins with the user, not the dashboard. A long Chinese text block pushes a submit control below a persistent footer in a narrow layout. The central question is whether someone working in the target language can recognize the necessary information and complete the intended action without guessing. A component can look polished in a design review while its spoken name, relationship or behavior fails under a different script, viewport or state.
Investigation. Pair automated evidence with visual reflow and keyboard testing at the target viewport. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Fix flexible layout and re-evaluate the exact translated form, not merely the CSS in isolation. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 14: Report file lacks provenance
Situation and user risk. This situation is especially easy to miss when product teams copy a source-language screenshot into a translation ticket. The team archives a JSON output without indicating which release, language or URL produced it. The important evidence is the actual localized interaction, including the words a person encounters, the order in which controls appear and the consequence of making a selection. A faithful translation cannot rescue a component whose behavior is inaccessible.
Investigation. Inspect artifacts for source URL, run time, package version and rule configuration. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Store a small manifest beside every report so later auditors can reconstruct the test. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 15: EARL output is treated as proof
Situation and user risk. Here is a useful quality-assurance exercise for a world-facing interface: An exported EARL-style report looks formal and its format is mistaken for a declaration of complete conformance. Rather than assigning the entire problem to a translator or engineer immediately, separate what is visibly wrong from what can be established through inspection. That prevents a wording change from hiding a structural defect or a markup fix from leaving an incorrect instruction in place.
Investigation. Read the assertions and targets actually represented in the output and identify omitted manual checks. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Use structured evidence to support review, not to replace judgment about coverage or accessibility. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 16: Test fixture contains private data
Situation and user risk. A realistic test begins with the user, not the dashboard. A staging locale URL includes real customer names in query parameters that would be copied into logs. The central question is whether someone working in the target language can recognize the necessary information and complete the intended action without guessing. A component can look polished in a design review while its spoken name, relationship or behavior fails under a different script, viewport or state.
Investigation. Inspect the input manifest and report destinations before automation runs. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Use synthetic records and redact secret tokens from artifacts while preserving reproducible state. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 17: One shared component appears in every locale
Situation and user risk. This situation is especially easy to miss when product teams copy a source-language screenshot into a translation ticket. The same untranslated icon button produces repeated missing-name alerts across twenty language routes. The important evidence is the actual localized interaction, including the words a person encounters, the order in which controls appear and the consequence of making a selection. A faithful translation cannot rescue a component whose behavior is inaccessible.
Investigation. Group findings by stable component identifier while retaining affected language and route evidence. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Fix centrally and sample retest across script families and representative product flows. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 18: Path names use non-Latin scripts
Situation and user risk. Here is a useful quality-assurance exercise for a world-facing interface: A localized route uses characters that a shell script mishandles while constructing CLI arguments. Rather than assigning the entire problem to a translator or engineer immediately, separate what is visibly wrong from what can be established through inspection. That prevents a wording change from hiding a structural defect or a markup fix from leaving an incorrect instruction in place.
Investigation. Verify actual UTF-8 encoding and quote URL arguments safely before running evaluations. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Preserve exact canonical URL strings and avoid brittle ad hoc byte replacements. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 19: Broken output is swallowed by pipeline
Situation and user risk. A realistic test begins with the user, not the dashboard. A continuous-integration step captures only standard output and discards evaluation errors and setup failures. The central question is whether someone working in the target language can recognize the necessary information and complete the intended action without guessing. A component can look polished in a design review while its spoken name, relationship or behavior fails under a different script, viewport or state.
Investigation. Record process status, diagnostic output and artifact existence separately from rule counts. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Make environment failures visible so a missing report cannot be recorded as a passed accessibility audit. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 20: File of multiple URLs mixes environments
Situation and user risk. This situation is especially easy to miss when product teams copy a source-language screenshot into a translation ticket. A batch accidentally checks production English content and staging Spanish content with different releases. The important evidence is the actual localized interaction, including the words a person encounters, the order in which controls appear and the consequence of making a selection. A faithful translation cannot rescue a component whose behavior is inaccessible.
Investigation. Review the manifest and annotate each URL with environment, commit and expected locale. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Separate incomparable builds or declare them explicitly before drawing regression conclusions. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 21: Localized CAPTCHA blocks a task
Situation and user risk. Here is a useful quality-assurance exercise for a world-facing interface: The audited page is accessible until a challenge appears during checkout that a CLI page scan cannot complete. Rather than assigning the entire problem to a translator or engineer immediately, separate what is visibly wrong from what can be established through inspection. That prevents a wording change from hiding a structural defect or a markup fix from leaving an incorrect instruction in place.
Investigation. Follow the real transaction journey with an appropriate test account and investigate challenge accessibility. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Provide an accessible alternative and document the limitation of static URL evaluation. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 22: False positive hides a real bug
Situation and user risk. A realistic test begins with the user, not the dashboard. The team dismisses a recurring alert as harmless across every language and later overlooks a genuine related regression. The central question is whether someone working in the target language can recognize the necessary information and complete the intended action without guessing. A component can look polished in a design review while its spoken name, relationship or behavior fails under a different script, viewport or state.
Investigation. Review representative instances and preserve the documented exception’s exact conditions. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Retest exceptions after shared markup or locale changes instead of broad permanent suppression. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 23: Rule results differ after upgrade
Situation and user risk. This situation is especially easy to miss when product teams copy a source-language screenshot into a translation ticket. A new QualWeb version changes evaluation coverage, making issue counts diverge without a code change. The important evidence is the actual localized interaction, including the words a person encounters, the order in which controls appear and the consequence of making a selection. A faithful translation cannot rescue a component whose behavior is inaccessible.
Investigation. Compare rule configurations, package version and test fixtures before declaring a regression. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Keep an intentional tool-upgrade baseline and investigate new findings on their merits. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
Case 24: Keyboard journey succeeds but language fails
Situation and user risk. Here is a useful quality-assurance exercise for a world-facing interface: The CLI finds no detectable markup errors, but a confirmation message uses the wrong date or amount terminology. Rather than assigning the entire problem to a translator or engineer immediately, separate what is visibly wrong from what can be established through inspection. That prevents a wording change from hiding a structural defect or a markup fix from leaving an incorrect instruction in place.
Investigation. Run a language and transaction-content review alongside automated accessibility evidence. Apply QualWeb’s ACT-rule, WCAG-technique and best-practice evaluation modules to the relevant state when the mechanism supports the check, and use an independent manual test when it does not. Preserve the route, language tag, browser or platform, build, viewport and exact triggering action. If a reported element is hidden or the route redirected, establish that fact before attributing the result to the target interface.
Repair and acceptance. Correct meaning and retest the action’s consequences; do not claim that automated accessibility found the translation error. For sign-off, repeat the original action with the changed component and compare at least one neighboring language or layout where a shared implementation could regress. Record whether the fix addresses programmatic semantics, wording, layout or a combination; no one-line status such as passed can replace that distinction.
How to interpret results without overstating them
A raw finding total is a count of observations under a given evaluation configuration. It is not necessarily a count of unique user barriers: one broken component may produce several alerts, and the same shared template may recur across many routes. Conversely, a small count does not mean low risk if a single unlabelled payment control prevents a user from completing a necessary transaction. Inspect the underlying elements, explain the task consequence, and group duplicates by genuine root cause without discarding evidence from affected languages.
Use at least three levels of certainty in the issue register. A confirmed defect has a reproducible observation and an applicable accessibility or language requirement. A needs-review item is a signal whose relevance or impact requires human judgment. An environment or coverage gap means the intended product state was not evaluated; for example, a login redirect or a missing translation bundle. Treating the third category as a pass makes release statistics unreliable.
For each high-impact finding, write a short user-centered statement: what action the person intended, what the interface communicated or failed to communicate, what was observed, and what must change. Add a minimal reproduction path, a stable reference to the affected component, the source of the observation and the retest. A native speaker should confirm meaning; a keyboard or assistive-technology tester should confirm operation where relevant; and an engineer should explain the technical correction in terms a non-engineer can verify.
A multilingual coverage matrix worth maintaining
Do not multiply language count by page count and assume the result represents complete coverage. Choose locales according to script, direction, market configuration, route variation, fonts, plural rules and business risk. A representative sampling plan can include an English baseline, a long-word language, a right-to-left language, a CJK layout, a language requiring combining marks, and a region where data formats or legal wording differ. These are examples of risk dimensions, not a claim that one language automatically represents every other language in its family.
- Routes: home, sign-in, recovery, checkout or application, account settings and help, with localized errors and confirmation states.
- Scripts: Latin, non-Latin and mixed-direction content where the product actually supports them, including realistic personal names and addresses.
- Layouts: phone, tablet, desktop, zoomed text and reduced available height where controls can be clipped.
- Actions: keyboard navigation, focus, field validation, dialogs, disclosures, menus, tables and links.
- Meaning: money, deadlines, destructive actions, consent, status, next steps and user-generated strings.
- Evidence: tested build, route, locale, state, tool version, issue, owner, remediation and retest.
A good release report can be compact even when the underlying testing is substantial. It should say which product and build were examined, which locales and critical tasks were represented, which checks were automated, which were performed manually, and which issues remain open. Include clear exclusions and owners. If an important locale could not be tested, describe that limitation directly instead of turning missing evidence into an optimistic conclusion.
Frequently asked questions
What is the basic QualWeb CLI command?
The project documents `qw -u` followed by the URL. Check the installed command help and current maintained repository for version-specific options. The most useful next step is to connect that answer to a specific user journey and a recorded test condition. Check the actual target-language content, note the criterion or product expectation at issue, and make the result repeatable rather than relying on an isolated score. A human review remains important wherever the decision depends on language, meaning, usability or lived assistive-technology experience.
What is EARL?
EARL is an accessibility evaluation results vocabulary used for structured assertions. A machine-readable report improves evidence exchange but does not make a test comprehensive. The most useful next step is to connect that answer to a specific user journey and a recorded test condition. Check the actual target-language content, note the criterion or product expectation at issue, and make the result repeatable rather than relying on an isolated score. A human review remains important wherever the decision depends on language, meaning, usability or lived assistive-technology experience.
Are ACT rules identical to WCAG success criteria?
No. ACT rules define repeatable tests for specific conditions; WCAG success criteria state accessibility requirements. The most useful next step is to connect that answer to a specific user journey and a recorded test condition. Check the actual target-language content, note the criterion or product expectation at issue, and make the result repeatable rather than relying on an isolated score. A human review remains important wherever the decision depends on language, meaning, usability or lived assistive-technology experience.
What are WCAG techniques?
Techniques explain ways of meeting or testing aspects of WCAG criteria, but a single technique result is not a whole-product conformance statement. The most useful next step is to connect that answer to a specific user journey and a recorded test condition. Check the actual target-language content, note the criterion or product expectation at issue, and make the result repeatable rather than relying on an isolated score. A human review remains important wherever the decision depends on language, meaning, usability or lived assistive-technology experience.
Do I need one report per locale?
Keep the original language and route identity in every result. You may aggregate reports later, provided their provenance remains traceable. The most useful next step is to connect that answer to a specific user journey and a recorded test condition. Check the actual target-language content, note the criterion or product expectation at issue, and make the result repeatable rather than relying on an isolated score. A human review remains important wherever the decision depends on language, meaning, usability or lived assistive-technology experience.
Can this audit a protected app?
Only if the chosen execution approach can reach its intended authenticated state. Verify actual rendered content and follow current tool documentation. The most useful next step is to connect that answer to a specific user journey and a recorded test condition. Check the actual target-language content, note the criterion or product expectation at issue, and make the result repeatable rather than relying on an isolated score. A human review remains important wherever the decision depends on language, meaning, usability or lived assistive-technology experience.
What if the evaluator reaches a redirect?
Record the final destination and confirm it is the desired language page before interpreting findings. The most useful next step is to connect that answer to a specific user journey and a recorded test condition. Check the actual target-language content, note the criterion or product expectation at issue, and make the result repeatable rather than relying on an isolated score. A human review remains important wherever the decision depends on language, meaning, usability or lived assistive-technology experience.
Does no finding equal accessible?
No. Automated evaluations cannot judge all aspects of understanding, usability, keyboard operation or assistive-technology behavior. The most useful next step is to connect that answer to a specific user journey and a recorded test condition. Check the actual target-language content, note the criterion or product expectation at issue, and make the result repeatable rather than relying on an isolated score. A human review remains important wherever the decision depends on language, meaning, usability or lived assistive-technology experience.
How should I compare two builds?
Keep URL, locale, meaningful state and rule-engine version comparable, then examine changes at the individual finding level. The most useful next step is to connect that answer to a specific user journey and a recorded test condition. Check the actual target-language content, note the criterion or product expectation at issue, and make the result repeatable rather than relying on an isolated score. A human review remains important wherever the decision depends on language, meaning, usability or lived assistive-technology experience.
What should accompany an issue report?
A reproducible route, language, tested state, relevant rule, expected behavior, observed impact, owner and evidence of retest. The most useful next step is to connect that answer to a specific user journey and a recorded test condition. Check the actual target-language content, note the criterion or product expectation at issue, and make the result repeatable rather than relying on an isolated score. A human review remains important wherever the decision depends on language, meaning, usability or lived assistive-technology experience.
Sources, standards and further reading
For technical behavior and current installation details, prefer the maintained project documentation and published standards. Tool releases change, and the scope of a rule is not always obvious from its short result label. The following are source references for the workflow; they are not endorsements or a substitute for reproducing a finding on your own product.
For complementary workflows, compare the following established eduKate guides rather than repeating their scope in this one:
The standard for a practical, reader-serving guide is simple: state what the reader can do, explain what the evidence means, show an example that could fail in the real world, and leave a clear route to a reliable next action. For multilingual accessibility, a translated product is finished only when its information and functions remain understandable and usable for the people it serves. Keep the automated report, the human judgment and the actual user outcome visible as separate but connected parts of that work.