The Chromium Chronicle #4: Test your Web Platform Features with WPT
Episode 4: by Robert in Waterloo, ON (July, 2019)
Previous episodes
If you work on Blink, you might know of web_tests (formerly LayoutTests). web-platform-tests (WPT) lives inside web_test/external/wpt
. WPT is the preferred way to test web-exposed features as it is shared with other browsers via GitHub. It has two main types of tests: reftests and testharness.js tests.
reftests take and compare screenshots of two pages. By default, screenshots are taken after the load
event is fired; if you add a reftest-wait
class to the <html>
element, the screenshot will be taken when the class is removed. Disabled tests mean diminishing test coverage. Be aware of font-related flakiness; use the Ahem
font when possible.
testharness.js is a JavaScript framework for testing anything except rendering. When writing testharness.js tests, pay attention to timing, and remember to clean up global state.
Flaky timeout & potential leaked states:
Don't
<script>
promise_test(async t => {
assert_equals(await slowLocalStorageTest(), "expected", "message");
localStorage.clear();
});
</script>
A better test with long timeout & cleanup:
Do
<meta name="timeout" content="long">
<script>
promise_test(async t => {
t.add_cleanup(() => localStorage.clear());
assert_equals(await slowLocalStorageTest(), "expected", "message");
});
</script>
Use testdriver.js if you need automation otherwise unavailable on the web. You can get a user gesture from test_driver.bless
, generate complex, trusted inputs with test_driver.action_sequence
, etc.
WPT also provides some useful server-side features through file names. Multi-global tests (.any.js
and its friends) run the same tests in different scopes (window
, worker
, etc.); .https.sub.html
asks the test to be loaded over HTTPS with server-side substitution support like below:
var anotherOrigin = "https://{{hosts[][www1]}}:{{ports[https][0]}}/path/to/page.html";
Some features can also be enabled in query strings. baz.html?pipe=sub|header(X-Key,val)|trickle(d1)
enables substitution, adds X-Key: val
to the headers of the response, and delays 1 second before responding. Search for "pipes" on web-platform-tests.org for more.
WPT can also test behaviors that are not included in specs yet; just name the test as .tentative
. If you need Blink internal APIs (e.g. testRunner
, internals
), put your tests in web_tests/wpt_internal
.
Changes made to WPT are automatically exported to GitHub. You will see comments from a bot in your CL. GitHub changes from other vendors are also continuously imported. To receive automatically filed bugs when new failures are imported, create an OWNERS
file in a subdirectory in WPT:
# TEAM: your-team@chromium.org
# COMPONENT: Blink>YourComponent
# WPT-NOTIFY: true
emails-here-will-be-cc@chromium.org
Additional Resources
- Want to find out how your tests run on other browsers, and how interoperable your feature is? Use wpt.fyi.
- Looking for more documentation on APIs, guidelines, examples, tips and more? Visit web-platform-tests.org.