Skip to main content

Reporting

SHAFT Engine ships with two complementary report formats, each suited to a different audience and workflow. You can enable or disable them independently using properties.

Report Types at a Glance​

ReportBest ForDefault State
Allure ReportDeep debugging — step-by-step logs, screenshots, videos✅ Opens automatically after execution
Execution SummaryQuick supplemental feedback — fast, portable, no setup⚙️ Opt-in via property

SHAFT-generated HTML report attachments use a shared SHAFT report theme across execution summaries, performance reports, accessibility reports, locator health, failure briefs, failure traces, and flake profiles. The same theme gives each surface consistent headers, status chips, metric cards, table wrapping, and dark-mode behavior whether opened directly or from Allure.

Allure Report​

The Allure Report is the primary report in SHAFT. It opens automatically in your default browser at the end of every test run and provides a rich, single-file HTML report with:

  • Step-by-step action logs with failure screenshots attached by default
  • Video recordings (when enabled)
  • Animated GIFs of browser sessions
  • Test history and trend graphs across multiple runs
  • Environment information and categories

Failure Trace Viewer​

When a test fails, SHAFT attaches a failure trace viewer to Allure by default. The viewer collects the ordered Selenium action timeline, exception and stacktrace, source-code frame fallback, available page or native source snapshot, locator health summary, and related artifacts without generating heavy trace artifacts for passing tests.

Allure receives one SHAFT trace-report attachment on failed tests:

  • shaft-trace.zip, which contains shaft-trace.json, SHAFT Trace Report.html, shaft-network.har, and the native Playwright trace ZIP when Playwright tracing is enabled and available

The archived shaft-trace.json includes an actions array with stable fields for automation agents:

{
  "id": "action-3",
  "category": "element",
  "name": "CLICK",
  "status": "failed",
  "startTime": "2026-06-26T10:13:13.000Z",
  "durationMs": 184,
  "locator": "By.id: pay",
  "url": "https://shop.example/checkout",
  "message": "Click \"Pay\"",
  "exception": {
    "type": "org.openqa.selenium.ElementClickInterceptedException",
    "message": "element click intercepted"
  },
  "attachments": ["screenshot - Click (48213 bytes)"],
  "metadata": {
    "elementName": "Pay"
  },
  "actionability": {
    "matchCount": 1,
    "displayed": true,
    "enabled": true,
    "rect": {"x": 42, "y": 318, "width": 120, "height": 36},
    "centerPoint": {"x": 102, "y": 336},
    "css": {"display": "block", "visibility": "visible", "pointerEvents": "auto"},
    "obscuringElement": {"selector": "div.modal-backdrop"},
    "recommendation": "Wait for or close the blocking element before retrying the action."
  }
}

For failed Selenium element actions, the optional actionability object explains why the target was not actionable. It includes locator match count, element state, rectangle and center point, viewport/CSS details, any center-point blocking element found with document.elementFromPoint, warnings from best-effort diagnostics, and a deterministic recommendation. SHAFT collects this only for failed traced actions and applies the same redaction rules used by the trace JSON.

For failed Appium touch actions, the same actions array includes mobile session metadata when the driver exposes it:

{
  "category": "touch",
  "name": "swipeElementIntoView",
  "status": "failed",
  "locator": "By.id: checkout_list",
  "metadata": {
    "gestureParameters": "targetText=Pay now, movement=VERTICAL",
    "platformName": "Android",
    "automationName": "UiAutomator2",
    "appPackage": "com.example.checkout",
    "appActivity": ".CheckoutActivity",
    "context": "NATIVE_APP",
    "orientation": "PORTRAIT",
    "windowSize": "1080x1920",
    "nativePageSourceExcerpt": "<hierarchy password=\"********\">..."
  }
}

Context changes from driver.browser().setContext(...) and MCP mobile_switch_context are also recorded as mobile-context trace events with contextBefore, contextAfter, and requestedContext. Unsupported Appium metadata fields use deterministic fallback text instead of surfacing raw provider errors in the trace.

When shaft.trace.includeNetwork=true, Selenium browser sessions also add redacted network entries to shaft-trace.json and a HAR-like shaft-network.har file to the archive. The trace viewer exposes Network, Console, and Observability tabs so you can inspect HTTP method, URL, status, duration, bounded body preview, browser console logs, and unsupported-driver warnings. Common headers, cookies, passwords, tokens, and sensitive URL values are masked before attachment.

SHAFT also persists the archive and a local index under target/shaft-traces/<safe-test-id>/:

  • shaft-trace.zip
  • index.json

Use shaft.trace.mode=failure for normal runs. Switch to retry or always only while actively debugging.

src/main/resources/properties/custom.properties
shaft.trace.enabled=true
shaft.trace.mode=failure
shaft.trace.includeCodeContext=true
shaft.trace.includeFullPageSnapshots=true
shaft.trace.includeNativePageSource=true
shaft.trace.includeNetwork=true
shaft.trace.includeConsole=true
shaft.trace.maxArtifactMb=50

Evidence Level Profiles​

evidenceLevel is the profile-level evidence switch. Its default is FAILURE_ONLY, so passing tests keep reports small while failed and broken tests still get failure screenshots, page source, diagnostics, and trace artifacts.

Profiles are applied after file, CLI, and default granular evidence properties are loaded. The code-based setter also applies the selected profile immediately. For FAILURE_ONLY, BALANCED, FAST, and FULL, the profile overrides granular controls such as screenshotParams_whenToTakeAScreenshot, whenToTakePageSourceSnapshot, createAnimatedGif, videoParams_recordVideo, captureWebDriverLogs, attachFullLog, shaft.diagnostics.enabled, and shaft.trace.mode. Set evidenceLevel=CUSTOM when you want those granular controls to win. In code, call evidenceLevel(...) after granular setters when you want a profile to win; call evidenceLevel("CUSTOM") before granular setters when you want custom values to win.

LevelEffect
FAILURE_ONLYDefault. Screenshots and page source only on failures; GIF, video, WebDriver logs, and full log off; diagnostics and trace enabled for failures.
BALANCEDValidation screenshots plus failure page source, diagnostics, and trace; GIF, video, WebDriver logs, and full log off.
FASTMinimal evidence: screenshots, page source, GIF, video, WebDriver logs, full log, diagnostics, and trace off.
FULLRich evidence: screenshots and page source always, GIF and video on, WebDriver logs and full log on, diagnostics on, and trace mode always.
CUSTOMNo profile override. Use this before setting granular evidence controls directly.
src/main/resources/properties/custom.properties
evidenceLevel=FAILURE_ONLY

Open Allure Report Manually​

SHAFT writes convenience scripts to your project root during the first run. Use them to regenerate and serve the Allure report at any time:

Windows
generate_allure_report.bat
macOS / Linux
./generate_allure_report.sh

Disable Auto-Opening the Allure Report​

The Allure report opens automatically by default (allure.automaticallyOpen=true). To disable this:

src/main/resources/properties/custom.properties
allure.automaticallyOpen=false

Skip Allure Report Generation​

If you only need raw allure-results and do not want SHAFT to create the HTML report, helper script, archive, or live watch process, disable report generation:

src/main/resources/properties/custom.properties
allure.generateReport=false

Generate an Allure Report Archive (for CI/CD)​

To generate a portable ZIP archive of the Allure report that you can publish as a CI/CD artifact:

src/main/resources/properties/custom.properties
allure.generateArchive=true

Customize the Allure Report​

You can customize the report title and logo displayed in the generated HTML:

src/main/resources/properties/custom.properties
allure.customTitle=My Regression Suite
allure.customLogo=https://example.com/my-logo.png

Tune Allure 3 Awesome Report Output​

When SHAFT generates an Allure 3 report, it writes the corresponding allurerc.yaml file for the Allure Awesome plugin. In addition to the title, output directory, logo, and history settings, you can control the Awesome plugin options that affect packaging, theme mode, language, browser opening, and the hierarchy shown in the report tree. SHAFT-generated Allure 3 reports use the same light and dark color tokens as this user guide. Allure 2 reports keep their existing UI.

note

These options apply to Allure 3 report generation. allure.automaticallyOpen controls whether SHAFT opens the final report after test execution, while allure.open is passed to the Allure CLI configuration itself.

src/main/resources/properties/custom.properties
# Generate a self-contained HTML report file (default: true)
allure.singleFile=true

# Allure 3 color mode (default: auto)
allure.theme=auto

# Allure UI language code (default: en)
allure.reportLanguage=en

# Let the Allure CLI open the report after generation (default: false)
allure.open=false

# Comma-separated Allure label hierarchy (default: package,testClass)
allure.groupBy=package,testClass

Monitor Allure 3 Results While Tests Run​

When allure.realtimeMonitoring=true and SHAFT resolves an Allure 3 CLI, SHAFT starts native allure watch against allure-results while tests run. The watch process is stopped before final report generation and archiving. This is disabled by default, and is ignored when Allure 2 compatibility mode is active or the Allure CLI cannot be resolved.

src/main/resources/properties/custom.properties
allure.realtimeMonitoring=true

Common allure.groupBy presets include:

PresetValueBest For
Code hierarchypackage,testClassDefault Java package and class navigation
Suite hierarchyparentSuite,suite,subSuiteTestNG or suite-based organization
BDD hierarchyepic,feature,storyBusiness-readable Allure annotation reports
Module hierarchymodule,parentSuite,suite,subSuiteMulti-module projects

Full-page image attachments open at the available modal width in SHAFT-generated Allure 3 reports. Use the modal scrollbar to inspect the rest of tall screenshots.

HTML attachments created through SHAFT's attachment reporter are also fitted to the available Allure preview width. SHAFT injects lightweight containment rules so wide tables, preformatted blocks, and long values wrap instead of forcing a horizontal page scroll.

Troubleshoot Allure CLI Generation​

SHAFT runs Allure report generation synchronously so the generated command, exit code, standard output, and standard error are visible in execution logs. If report generation fails, check the log entries around Executing Allure report generation command, Allure generate stdout, Allure generate stderr, and Allure report generation command exited with code ... before changing report settings.

Accumulate History and Reports Across Runs​

By default, SHAFT keeps Allure history to enable trend graphs across runs. You can control this behavior:

src/main/resources/properties/custom.properties
# Accumulate history to show trends across runs (default: true)
allure.accumulateHistory=true

# Keep previous HTML report files in the report directory (default: true)
allure.accumulateReports=true

# Clean the results directory before each run (default: true)
allure.cleanResultsDirectory=true

Execution Summary Report​

The Execution Summary report is a lightweight supplement to the Allure report. It is disabled by default and is most useful when you want a quick, portable HTML summary without opening the full Allure report.

  • No extra setup required — just enable the property.
  • Portable — a fast self-contained HTML file you can share or archive.

Enable the Execution Summary Report​

src/main/resources/properties/custom.properties
openExecutionSummaryReportAfterExecution=true

Locator Health Report​

The Locator Health Report is disabled by default. Enable it when you want to find slow, flaky, ambiguous, stale, or healed web locators across a full run. SHAFT records lightweight metrics during element identification and writes HTML and JSON reports under execution-summary/locator-health/. The JSON export is attached to Allure, and the HTML dashboard is attached when shaft.locatorHealth.attachDashboard=true.

Recorded metrics include lookup count, unique-match, no-match, multi-match, and stale rates, average and p95 lookup time, polling attempts, timeout count, slow lookups, SHAFT Heal attempts, accepted recoveries, selected replacement locator, and confidence when the provider exposes it. Each locator also gets a health score, selector-smell labels, and plain-language recommendations for risky patterns such as absolute XPath, index-heavy XPath, generated IDs, text-only selectors, and deep CSS chains. When the failure trace viewer is enabled, failed-test trace JSON also includes the current locator health snapshot.

src/main/resources/properties/custom.properties
shaft.locatorHealth.enabled=true
shaft.locatorHealth.warnBelowScore=70
shaft.locatorHealth.attachDashboard=true
shaft.locatorHealth.failBelowScore=-1
slowLocatorThresholdMillis=750

Keep shaft.locatorHealth.failBelowScore=-1 while introducing the report. Set it to a score threshold only after the suite has a stable locator-health baseline. The legacy locatorHealthReportEnabled=true and failOnLocatorHealthWarnings=false keys remain supported.

Flake and Auto-Wait Profiler​

The Flake and Auto-Wait Profiler is disabled by default. Enable it when you want Allure attachments that explain slow element actions, assertions, retries, wait polling, locator churn, and evidence overhead.

SHAFT attaches flake-profile.json and Flake Profile HTML for tests that produce profiler signals. The suite-level profile is attached during engine tear down. Element action duration excludes screenshot capture and report attachment time; those costs are recorded separately as screenshot, page snapshot, and report attachment evidence. Assertion and verification duration is measured around the validation step itself.

src/main/resources/properties/custom.properties
shaft.flakeProfiler.enabled=true
shaft.flakeProfiler.attachPerTest=true
shaft.flakeProfiler.failOnSevereFlakeRisk=false
shaft.flakeProfiler.slowActionThresholdMs=2000

Keep shaft.flakeProfiler.failOnSevereFlakeRisk=false while collecting a baseline. Turn it on only after the suite has a realistic slow-action threshold.

Video Recording​

SHAFT can record a video of every test session and automatically attach it to the Allure report. Because evidenceLevel=FAILURE_ONLY is the default, set evidenceLevel=CUSTOM before enabling video directly, or set evidenceLevel=FULL to enable the full rich-evidence profile.

src/main/resources/properties/custom.properties
evidenceLevel=CUSTOM
videoParams_recordVideo=true

Where to Put Your Properties File​

Create a custom.properties file under src/main/resources/properties/ in your project. SHAFT automatically creates a properties/ directory with some core files on first run — just add your file there:

your-project/
└── src/
    └── main/
        └── resources/
            └── properties/
                ├── default/               ← auto-generated by SHAFT
                └── custom.properties      ← your file

Control Video Recording Scope​

You can control which part of the session is recorded:

src/main/resources/properties/custom.properties
evidenceLevel=CUSTOM
videoParams_recordVideo=true
# Scope options: DriverSession | TestMethod
videoParams_scope=DriverSession
ScopeDescription
DriverSessionRecords the entire browser/driver session (default)
TestMethodRecords each test method as a separate video

Animated GIFs​

Animated GIFs are disabled by default. SHAFT creates them when you explicitly set evidenceLevel=CUSTOM and createAnimatedGif=true, and also enables them automatically for retry attempts when forceCaptureSupportingEvidenceOnRetry=true. GIF frames are captured in the background and do not force every action screenshot to be attached to the report. Generated GIF files are written under video.folder before they are attached to Allure:

src/main/resources/properties/custom.properties
evidenceLevel=CUSTOM
createAnimatedGif=true
# Delay between frames in milliseconds (default: 500)
animatedGif_frameDelay=500

CI/CD Pipeline (archive report as artifact)​

src/main/resources/properties/custom.properties
# Do not open the browser during CI
allure.automaticallyOpen=false
# Generate a ZIP artifact to publish from the pipeline
allure.generateArchive=true

Active Debugging (rich visuals)​

src/main/resources/properties/custom.properties
# Allure opens automatically by default; add rich evidence
evidenceLevel=FULL

Quick Pass/Fail Summary​

src/main/resources/properties/custom.properties
# Enable fast portable summary alongside the Allure report
openExecutionSummaryReportAfterExecution=true

All Reporting Properties​

For the full list of reporting-related properties and their default values, see the Properties List.

Key properties quick reference:

PropertyDefaultDescription
allure.automaticallyOpentrueOpen the Allure report in the browser automatically after the run
allure.generateReporttrueGenerate the Allure HTML report, helper script, archive, and live watch process
allure.generateArchivefalseGenerate a portable ZIP archive of the Allure report
allure.accumulateHistorytrueAccumulate history across runs for trend graphs
allure.accumulateReportstrueKeep previous Allure HTML report files
allure.cleanResultsDirectorytrueClean Allure results directory before each run
allure.customTitleTest run reportCustom title in the Allure report header
allure.customLogo(SHAFT default logo)Custom logo URL for the Allure report
allure.realtimeMonitoringfalseStart native Allure 3 allure watch while tests run
allure.singleFiletrueGenerate an Allure 3 Awesome report as a self-contained HTML file
allure.themeautoDefault Allure 3 Awesome color mode: auto, light, or dark
allure.reportLanguageenLanguage code used by the Allure 3 Awesome report UI
allure.openfalsePass the browser-open flag to Allure 3 report generation
allure.groupBypackage,testClassComma-separated label hierarchy for the Allure 3 report tree
openExecutionSummaryReportAfterExecutionfalseOpen the Execution Summary report after the run
evidenceLevelFAILURE_ONLYEvidence profile: FAILURE_ONLY, BALANCED, FAST, FULL, or CUSTOM
locatorHealthReportEnabledfalseGenerate end-of-run HTML/JSON locator health reports and Allure attachments
shaft.locatorHealth.enabledfalseGenerate end-of-run locator health reports
shaft.locatorHealth.warnBelowScore70Mark locators below this health score as risky
shaft.locatorHealth.attachDashboardtrueAttach the HTML locator health dashboard to Allure
shaft.locatorHealth.failBelowScore-1Fail the run when any locator score is below this threshold; -1 disables score-based failure
slowLocatorThresholdMillis750Lookup duration that marks a locator as slow
failOnLocatorHealthWarningsfalseFail the run when locator health warnings are present
shaft.flakeProfiler.enabledfalseAttach opt-in flake and auto-wait timing profiles to Allure
shaft.flakeProfiler.attachPerTesttrueAttach each test's JSON/HTML profile when profiler signals exist
shaft.flakeProfiler.failOnSevereFlakeRiskfalseFail the run when severe flake-risk actions are found
shaft.flakeProfiler.slowActionThresholdMs2000Duration threshold used to flag slow and severe-risk actions
shaft.trace.enabledtrueAttach the SHAFT failure trace viewer artifacts
shaft.trace.modefailureTrace mode: failure, retry, or always
shaft.trace.includeCodeContexttrueInclude the best matching source-code frame and snippet
shaft.trace.includeFullPageSnapshotstrueInclude web snapshots when available
shaft.trace.includeNativePageSourcetrueInclude Appium/native page source when available
shaft.trace.includeNetworktrueCapture redacted Selenium browser network evidence in the trace metadata and HAR-like export
shaft.trace.includeConsoletrueCapture browser console evidence in the trace metadata when the driver exposes browser logs
shaft.trace.maxArtifactMb50Maximum size for a single trace bundle entry
videoParams_recordVideofalseEnable video recording of test sessions
videoParams_scopeDriverSessionScope of video recording
createAnimatedGiffalseCreate an animated GIF; retries can enable it automatically
animatedGif_frameDelay500Delay between GIF frames (milliseconds)
screenshotParams_whenToTakeAScreenshotValidationPointsOnlyGranular screenshot policy; profile levels except CUSTOM override it
whenToTakePageSourceSnapshotfailuresOnlyGranular page-source policy; profile levels except CUSTOM override it