Playwright

About Playwright

Playwright is an open-source automation framework for web testing that supports Chromium, Firefox, and WebKit, enabling fast, reliable end-to-end testing across all modern browsers.

Features

Set Up the Playwright Reporter

To submit your test results using the Playwright reporter, follow these steps:

1. Install the TestMonitor reporter package for Playwright:

   $ npm install @testmonitor/playwright-reporter

2. Configure the reporter in your playwright.config.ts file using the token:

   {
       "reporter": [
           "@testmonitor/playwright-reporter",
           {
               "domain": "dev.testmonitor.test",
               "token": "<token>"
           }
       ]
   }

3. Optionally, you can specify these configuration parameters:

   {
       "reporter": [
           "@testmonitor/playwright-reporter",
           {
               // Optional parameters 
               milestoneId: 12,          // Use this to reference an existing milestone
               milestone: 'Version 2.1', // Or use this to create or match a milestone by name
               testEnvironmentId: 3      // Target test environment ID in TestMonitor 
           }
       ]
   }

To improve visibility into your test runs, the Playwright integration allows uploading screenshots, videos and trace files for failed tests, which can be enabled in you Playwright settings:

• Screenshots:
  Set screenshot: 'on', 'only-on-failure' or 'on-first-failure' in your use settings to capture screenshots automatically.
• Trace files:
  Set trace: 'on', 'retain-on-failure' or 'on-first-retry' to save trace files for each test.
• Videos:
  Set video: 'on', 'retain-on-failure' or 'on-first-retry' to record videos.

To help you configure Playwright for integration with TestMonitor, consider the following sample setup for your playwright.config.ts file:

import { defineConfig } from '@playwright/test';

export default defineConfig({
    // Base configuration
    // ...
 
    // Artifact settings 
    use: {
        screenshot: 'only-on-failure',
        trace: 'on',
        video: 'retain-on-failure'
    },
});

Captured media will be automatically attached to the relevant test result in TestMonitor when a Playwright test fails.

Run Tests and Send Results to TestMonitor

Execute your Playwright tests as usual:

$ npx playwright test

Alternatively, you may use your preferred development environment, such as Playwright UI or Visual Studio Code, to execute your tests.

The reporter will automatically send test cases, results, and artifacts to TestMonitor after execution.

View and Analyze Playwright Results in TestMonitor

Once the results have been submitted, the reporter will generate a URL that enables you to access and review the Playwright test results.

You can access a comprehensive overview of all Playwright test runs in the Track test runs section:

1. Go to Track
2. Open the filter dialog by clicking the icon.
3. Identify and select the Reporter filter, then proceed to close the filter dialog.
4. Activate the Reporter filer and select Playwright from the list.

Deactivate the Playwright Integration

If the integration is no longer required or your project comes to an end you can easily deactivate the integration:

1. Go to the Project Settings.
2. Choose the Integrations option from the menu on the left.
3. Select the Playwright panel.
4. In the top right corner, click on Deactivate... .
5. Click Deactivate to confirm.

Troubleshooting

If you encounter errors involving the Playwright integration with TestMonitor, here are some steps you can take to troubleshoot the issue:

Confirm your domain and token are correct

Verify that your Playwright reporter configuration includes the correct domain and token, as specified in your integration settings on TestMonitor.

Check your daily submission quota

Ensure you have not exceeded your daily submission quota. Submission limits are based on your subscription plan: Starter permits up to 500 results per day, Professional up to 2,500, and Enterprise up to 5,000. Additionally, a maximum of 10 submissions per minute is allowed.

Inspect the integration log

The integration log assists in identifying the reasons for integration failures by listing all encountered issues while interacting with Playwright. To access the log, follow these instructions:

1. Open the integration page
2. Click on the link in the right panel that reports the number of integration issues.

Investigate the log entry by reviewing the error type and description. You can dismiss the error by clicking on the check button on the right.

Refresh your integration token

If you continue to experience authentication errors, it is advisable to refresh your integration token to confirm that you are using the correct credentials.

1. Click on the three-dotted menu in the top-right corner.
2. Select Refresh token...
3. Confirm your action.
4. Ensure that you revise your Playwright configuration to use the updated token.

Contact support

If these solutions do not resolve your issue, feel free to reach out to our support team for additional assistance.