Selenium

About Selenium

Selenium is a robust open-source framework designed to automate the testing of websites and web applications. It integrates seamlessly with modern development workflows and offers support for multiple programming languages, including Java, JavaScript, C#, PHP, Python, and Ruby.

Features

By combining TestNG with Selenium, you enable advanced integration features such as:

Creating Selenium test cases, along with their associated results and execution run in TestMonitor.
Identification of previously uploaded Selenium test cases, enabling you to submit multiple execution runs over time.
Uploading of test run artifacts, such as screenshots.
Configuration of imported test case placement and milestone behavior.

Prerequisites

Before you begin, ensure you have:

Admin or project manager role in TestMonitor
A Maven-based TestNG suite that’s ready for configuration
Java 11+ installed

When all these requirements are met, you can proceed with setting up the TestNG listener.

If your Selenium setup does not yet utilize TestNG, please refer to the TestNG documentation for detailed setup instructions.

Activate the TestNG Integration

Activate the TestNG integration for your project as follows:

Go to the Project Settings.
Choose the Integrations option from the menu on the left.
In the top right corner, click on Browse....
Find and select the TestNG panel.
Select the Activate button to enable the TestNG integration.
Optionally, select the Default milestone for your TestNG test runs. Alternatively, specify the desired milestone directly in your listener configuration.

Set Up the TestNG Listener

To submit your Selenium test results using the TestNG listener, follow these steps:

Add the TestMonitor TestNG listener dependency to your pom.xml file:

<dependency> 
  <groupId>com.testmonitor</groupId> 
  <artifactId>testng-reporter</artifactId> 
  <version>1.0</version> 
</dependency>

Run mvn install to install the dependencies.

Add the TestMonitor TestNG listener to your TestNG suite XML file(s):

<suite name="All">
  <test name="Tests">
    <classes>
      ...
    </classes>
  </test>

  <listeners>
    <listener class-name="com.testmonitor.testng.TestNGReporter" />
  </listeners>
</suite>

Set the environment variables of your project, containing your domain and integration token:
```
TESTMONITOR_DOMAIN=mydomain.testmonitor.com
TESTMONITOR_TOKEN=mytoken
```
Optionally, you can set additional variables:
- TESTMONITOR_MILESTONE_ID: Set the milestone for new automation test runs.
- TESTMONITOR_MILESTONE_NAME: Finds or creates a milestone matching the specified name for new automation test runs.
- TESTMONITOR_TEST_ENVIRONMENT_ID: Sets the test environment ID for new automation test runs.
- TESTMONITOR_ATTACHMENT_ON_FAILURE: Determines if attachments should be send when a test case fails (default: true).
- TESTMONITOR_VERBOSE: Shows extra details about what the reporter is doing behind the scenes (default: false).

To enhance test visibility, the TestNG integration supports automatic screenshot uploads when enabled in your tests.

To capture screenshots on failure, implement the HasWebdriver interface in your test cases. This gives the reporter access to the web driver, allowing screenshots to be taken if a test fails:

public class MyTestCase implements HasWebdriver {
    protected WebDriver driver;

    @BeforeMethod(alwaysRun = true)
    private void setUp() {
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--headless=new");
        this.driver = new ChromeDriver(options);
    }


    @Test
    public void myTest() {
        this.driver.get("https://www.testmonitor.com/");
    }

    public WebDriver getDriver() {
        return this.driver;
    }
}

Captured media will be automatically attached to the relevant test result in TestMonitor when a Selenium test fails.

Try it out yourself: view a Selenium test example to see how everything fits together.

Run Tests and Send Results to TestMonitor

Execute your Selenium tests as usual:

$ mvn test -p all

The reporter will automatically send test cases, results, and artifacts to TestMonitor after execution.

View and Analyze Selenium Results in TestMonitor

Once the results have been submitted, the reporter will generate a URL that enables you to access and review the Selenium test results.

You can access a comprehensive overview of all Selenium test runs in the Track test runs section:

Go to Track
Open the filter dialog by clicking the icon.
Identify and select the Reporter filter, then proceed to close the filter dialog.
Activate the Reporter filer and select TestNG from the list.

Deactivate the TestNG Integration

If the integration is no longer required or your project comes to an end you can easily deactivate the integration:

Go to the Project Settings.
Choose the Integrations option from the menu on the left.
Select the TestNG panel.
In the top right corner, click on Deactivate... .
Click Deactivate to confirm.

Deactivating the integration will halt the submission of any future automated tests. All submitted test cases, test runs, and results will continue to be accessible within your project for your review and analysis.

Troubleshooting

If you encounter errors involving the TestNG integration with TestMonitor, here are some steps you can take to troubleshoot the issue:

Confirm your domain and token are correct

Verify that your environment variables include the correct domain and token, as specified in your integration settings on TestMonitor.

Ensure your Import folder and milestone settings are valid

Before submitting test runs, it is essential to establish and configure a milestone within your TestMonitor integration. You have the option to set a milestone directly through the integration configuration or to specify the milestone ID in your environment variables.

Inspect the integration log

The integration log assists in identifying the reasons for integration failures by listing all encountered issues while interacting with TestNG. To access the log, follow these instructions:

Open the integration page
Click on the link in the right panel that reports the number of integration issues.

Investigate the log entry by reviewing the error type and description. You can dismiss the error by clicking on the check button on the right.

Refresh your integration token

If you continue to experience authentication errors, it is advisable to refresh your integration token to confirm that you are using the correct credentials.

Click on the three-dotted menu in the top-right corner.
Select Refresh token...
Confirm your action.
Ensure that you revise your TestNG configuration to use the updated token.

Contact support

If these solutions do not resolve your issue, feel free to reach out to our support team for additional assistance.