Waldo updates
Waldo updates

Validation Tests βœ…




We've come to realize that about 20% of the tests that were created on Waldo would not pass on their very first replay, which causes issues with tests created as dependencies off of them and causes a lot of frustration.

Now, instead of just trusting blindly that a new test should pass and allowing users to create other tests dependent off of potentially failing tests, we validate every new test created before we allow you to use it. That is, we stress test them several times on all the device configurations on your account, and we verify that they output consistent and reliable results.

Until these tests pass validation, the newly created tests will be categorized as "Pending validation" and will NOT be included in automatic runs. You will also not be able to use them as dependencies either until they have passed.

Validation Tests.png

Flaky tests: Catch 'em all! 🎣




We’re excited to launch Flaky detection : a fundamental addition to our platform and how people can now evaluate their test results.

Flakiness is probably the worst thing that can happen to automated end-to-end testing. A β€œflaky” test is simply one that yields different outcomes when played multiple times.

With Flaky detection, you can identify quickly when a test is flaky:

Flaky Matrix.png

Thanks to the replay switch you can see all the different outcomes of your test during the last run and identify why your test is flaky.


If you want to learn more about flaky detection you can read our last blog article and our product documentation.


Introducing Waldo Live πŸŽ‰





We are thrilled to announce the release of a new and exciting addition to Waldo!

Introducing Waldo Live.

  • Launch your app directly in your web browser with a single click.
  • Inspect HTTP requests, app logs, and analytics events (iOS only)!
  • Switch between device models, OS versions, and languages with a single click. 😎

Even better? Connect your GitHub account to Waldo and access your live app directly from your Pull Requests in GitHub!


Filter by folder





You can now filter your tests by Folder in the Tests page header.

This adds more options for displaying/comparing/sorting your tests in both By Folder and Sorted List modes.

Screen Shot 2021-09-08 at 5.23.44 PM.png

Group Tests πŸ—‚





To better help you manage and organize your tests, we are excited to introduce a new Group Tests feature for Waldo. Now you can group your tests within folders that you create on the fly.

In addition, this feature introduces new view types, as well as new sorting and filtering capabilities, for your tests. Together, these enable you to navigate and search your tests more effectively.

You can also execute common actions (such as Run, Move, and Mute) from a brand-new action bar. As you will discover, the action bar appears whenever you select one or more folders/tests with the checkboxes.

Finally, the folder view supports Drag And Drop, so feel free to play around with it!

Group Test.png

GitHub Actions πŸ™





We now officially support GitHub Actions as one of our CI providers.

Screenshot 2021-06-21 at 17.58.46.png

The following documentation will guide you through the steps needed to integrate Waldo into your current GitHub Actions workflow.


Improved Slack notification πŸ“£





Capture d’écran 2021-06-17 aΜ€ 16.06.22.png

We added the Commit Message and the Branch Name to your Slack notifications! Make sure to set up the GitHub integration to benefit from this enhancement.

Assertion Time Limit ⏱





You can now configure Waldo to automatically move to the next step of your test, even if some of the assertions on that step are failing.

Screen Shot 2021-06-15 at 12.52.05 PM.png

This feature allows you to more easily handle the case where a step is known to take a significant amount of time to complete. For example, if your app exports some data on a particular step and you know that it normally takes at least 45 seconds to finish. With this new feature, you can now adjust the assertion time limit to 1 minute for that step. This will ensure that Waldo allows enough time for the export to complete.

Screen Shot 2021-06-15 at 12.52.20 PM.png

Options to fix similarity assertion πŸ‘“





Two options are available now to fix a similarity assertion:

  1. Use actual screen as a reference going forward: this screen will be the new baseline for this device configuration,

  2. Update the screen similarity level: this will change the value of the similarity assertion, this option impacts all device configurations. This is advised when your screen changes a lot from run to run.

Screen Shot 2021-06-02 at 12.25.42 PM.png

Multiple Slack channels notifications πŸ“£





Do you have multiple applications on Waldo? This feature is for you! You will be able to choose a different Slack channel for each application.

Screen Shot 2021-05-26 at 2.15.13 PM.png