August 14, 2019

Announcing the WebKit Tracking Prevention Policy

Surfin’ Safari

Today we are publishing the WebKit Tracking Prevention Policy, covering:

  • What types of tracking WebKit will prevent.
  • When other tracking countermeasures come into play such as limiting capabilities and informed user consent.
  • How WebKit handles unintended impact of our tracking prevention.

We’d like to thank Mozilla for their anti-tracking policy which served as inspiration for ours.

Please send us your comments or questions about this policy to @webkit on Twitter.

August 14, 2019 10:44 PM

August 07, 2019

Release Notes for Safari Technology Preview 89

Surfin’ Safari

Safari Technology Preview Release 89 is now available for download for macOS Mojave and the macOS Catalina beta. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 247433-248024.

JavaScript

  • Implemented nullish coalescing with the ?? operator for ESNext (r247819)

Web API

  • Added referrerpolicy attribute support for <script> elements (r247509)
  • Changed window.openDatabase to be overridable (r247434)
  • Fixed an IndexedDB error where the starting version change transaction may be neglected (r247649)
  • Fixed the ability to unbold selected text when the system font is used (r247439)
  • Made storing cross-origin top-level prefetches in HTTP cache optional (r247860)
  • Moving right by word boundary right before an object element followed by a <br> element hangs (r247881)
  • Removed support for beforeload on link=prefetch (r247481)

Compatibility

  • Fixed Daring Fireball long press highlights that were unnecessarily inflated due to false illegibility (r247792)
  • Fixed contextual menu actions for YouTube videos (r247901)

Accessibility

  • Exposed the aria-label attribute for <video> elements (r247891)

Media

  • Enabled a WebRTC debug mode without encryption (r247438)
  • Fixed support for FLAC-in-MP4 (r247934)

Web Inspector

  • Added support for console.screenshot with detached (not in main DOM tree) <img> and <picture> elements (r247814)
  • Added support for console.screenshot with ImageData and ImageBitmap (r247812)
  • Added a “Show Grid” navigation item for the Images collection in the Resources tab (r248004)
  • Added an indicator/separator around items in the Images collection in the Resources tab (r248019)
  • Add special case for about:blank resources to show “Resource has no content” message (r247747)
  • Fixed display of application/xml content for XHR requests (r247533)
  • Fixed issues toggling multiple breakpoints when they’re collapsed into one line (r247639)
  • Fixed Command-X (⌘X) to cut selected properties in the Styles sidebar (r247760)
  • Made the Changes panel in the Elements Tab render with LTR text direction (r247492)

WebGPU

  • Implemented errors for GPURenderPipeline creation (r247764)
  • Added descriptive error messages in WHLSL (r247834)
  • Changed checker to setError() when a property access node can’t commit its base type in WHLSL (r247676)
  • Changed to return the zero-value enum in the enum-from-integer constructor when the integer is not a valid enum value in WHLSL (r247675)
  • Made enums work in WHLSL (r247666)
  • Updated matrix memory layout to match HLSL by laying out columns linearly in WHLSL (r247468)

August 07, 2019 05:00 PM

August 05, 2019

Philippe Normand: Review of the Igalia Multimedia team Activities (2019/H1)

Igalia WebKit

This blog post takes a look back at the various Multimedia-related tasks the Igalia Multimedia team was involved in during the first half of 2019.

GStreamer Editing Services

Thibault added support for the OpenTimelineIO open format for editorial timeline information. Having many editorial timeline information formats supported by OpenTimelineIO reduces …

By Philippe Normand at August 05, 2019 01:30 PM

August 02, 2019

Changing Page Settings on iOS Using Web Inspector

Surfin’ Safari

If you’ve ever used Web Inspector before, chances are you’ve used (or are at least familiar with) the Develop menu. It holds action items and toggles for various settings of the browser, like whether local files (e.g. URLs beginning with file://) can be loaded or whether CSS is applied to each page.

All of these items apply to the entire browser, meaning that if you Disable Styles on one page, every other page will be affected.

Additionally, these items have no effect when using Web Inspector to inspect a remote target, like an iOS device or simulator. Checking Disable Styles in the Develop menu will not have any affect on the remote target.

In order to support this development workflow, Web Inspector has added a device settings menu that allows these settings to be toggled per-page when being remotely inspected.

Clicking on the device settings menu icon will show a popover with many of the same settings as the Develop menu.

Since these settings apply per-page and only on the remote target, the corresponding actions in the Develop menu are disabled, as they have no effect on a remote target:

  • Disable Images
  • Disable Styles
  • Disable JavaScript
  • Disable Site-specific Hacks
  • Disable Cross-Origin Restrictions
  • WebRTC
    • Allow Media Capture on Insecure Sites
    • Disable ICE Candidate Restrictions
    • Use Mock Capture Devices

Along these lines, the device settings menu is only shown when using Web Inspector to inspect a remote target.

Device settings are not preserved between Web Inspector sessions. Closing Web Inspector (or disconnecting the inspected device) will cause all previously set device settings for the inspected page to reset.

Device settings are preserved across navigations, however, so long as Web Inspector stays open/connected.

User Agent

The first item in the device settings menu is the User Agent editor. It contains a list of common user agents, as well as an option to input a custom user agent (Other…).

Each time the User Agent is modified, the inspected page will automatically reload so that the new User Agent is applied.

Disable Toggles

Each of these toggles, when checked, disables a specific piece of functionality in the inspected page.

  • Images will prevent any not-yet loaded images from loading, but will have no effect on any already loaded images.
  • Styles will immediately disable all CSS on the page, including inline <style>s and any style DOM attributes.
  • JavaScript will cause the page to ignore any future JavaScript from being run, including new <script> elements (the underlying resource isn’t even requested) and callbacks for previously added event listeners.
  • Site-specific Hacks controls whether workarounds are made by WebKit to support compatibility on certain sites.
    • A list of these sites can be found in Source/WebCore/page/Quirks.cpp.
    • If you develop a site that is found in that list, we strongly encourage developing and testing with Site-specific Hacks disabled.
  • Cross-Origin Restrictions controls whether CORS rules/restrictions are active for any future network requests.

WebRTC Toggles

These toggles focus specifically on functionality related to WebRTC.

  • Allow Media Capture on Insecure Sites will allow WebRTC media capture to be used/tested on insecure (e.g. non-https) pages for any future calls of getUserMedia.
  • Disable ICE Candidate Restrictions will prevent host ICE candidates from being filtered for new connection attempts.
  • Use Mock Capture Devices will replace all capture devices with a mock “Bip-Bop” device for any future calls of getUserMedia.
  • Disable Encryption will cause future connections to be established and future streams (from those connections) to be transmitted without any form of encryption.

Feedback

You can try out changing device settings with iOS 12.2 or later. Let us know how it works for you. Send feedback on Twitter (@webkit, @dcrousso) or by filing a bug.

August 02, 2019 05:56 AM

July 24, 2019

Release Notes for Safari Technology Preview 88

Surfin’ Safari

Safari Technology Preview Release 88 is now available for download for macOS Mojave and the macOS Catalina beta. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 247047-247433.

JavaScript

  • Enabled Intl.PluralRules and Intl.NumberFormatToParts by default (r247247)

WebRTC

  • Registered a MediaStreamTrack as media producer only if it is a capture track (r247208, r247382)

Web API

  • Fixed XHR CORS requests getting logged twice on the server (r247276)

Pointer Events

  • Updated to respect pointer capture when dispatching mouse boundary events and updating :hover (r247148)

Rendering

  • Changed to avoid extra backing store on elements with overflow: scroll and visibility: hidden (r247420)
  • Changed to trigger a compositing update when a video element is changing (r247187)

Accessibility

  • Added accessibility announcement notifications to show, dismiss and selection change for the datalist suggestions view (r247418)
  • Exposed HTML datalist accessibility (r247295)
  • Enhanced support of aria-haspopup per ARIA 1.1 specification. (r247071)
  • Implemented support for ARIA roles: insertion, deletion, subscript, superscript, and time (r247349)
  • Fixed ignored role="presentation" on HTML <table> elements for VoiceOver (r247330)

WebGL

  • Hooked up WebGL’s back buffer in ANGLE backend on macOS (r247315)

WebGPU

  • Added most of the remainder of the standard library (r247174)
  • Implemented GPUError and error scopes (r247366)
  • Made the destructor of VariableDeclaration non-virtual (r247124)
  • Optimized the lexer (r247171)
  • Removed the phase resolveCallsInFunctions (r247170)

Web Inspector

  • Added support for pasting copied DOM nodes in the Elements tab (r247054)
  • Fixed dismissing a blank property to no longer cause the rule to appear in the Changes panel (r247406)
  • Fixed an issue where unbalanced quotes or parenthesis weren’t displayed as properly closed after editing values (r247196)
  • Updated descendant DOM breakpoints be enabled, disabled, or deleted from a collapsed parent (r247053)

Bug Fixes

  • Fixed paste from Notes into Excel 365 spreadsheet (r247222)
  • Fixed video playback on xfinity.com/stream on macOS Catalina (r247213)

July 24, 2019 05:00 PM

July 10, 2019

Release Notes for Safari Technology Preview 87

Surfin’ Safari

Safari Technology Preview Release 87 is now available for download for macOS Mojave and the macOS Catalina beta. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 246691-247047.

Web API

  • Changed openDatabase to return an empty object when WebSQL is disabled (r246707)
  • Added an experimental behavior to prevent a 5 second delay for initial paint on pages that are using Google’s anti-flicker optimization when content blockers are enabled (r246764)

Web Sockets

  • Added support for sending blob messages when using web sockets platform APIs (r246964)

Payment Request

  • Changed to set state to Closed when show() is called during an active session (r246863)

Experimental Web Shading Language (WHLSL)

Rendering

  • Fixed incorrect clipping with overflow: scroll inside overflow: hidden with border-radius (r246845)
  • Fixed the preview of a <picture> element to match the element bounds (r246695)

WebGPU

  • Made WebGPU enabled only on Mojave and later (r246888)

Web Inspector

  • Implemented console.countReset (r246850)
  • Implemented console.timeLog (r246798)
  • Added additional demo audits for other WebInspectorAudit functions (r247042)
  • Enable the Show Shadow DOM navigation item by default in the Elements tab (r246821)

Known Issues

  • Open tabs are blank on initial launch after upgrading to Safari Technology Preview 87 and loading all tabs from the last session. To correct this behavior, reload open tabs or relaunch Safari Technology Preview.

July 10, 2019 05:00 PM

July 08, 2019

WebDriver is Coming to Safari in iOS 13

Surfin’ Safari

As anyone who has developed a mobile-friendly site can tell you, mobile browsers and desktop browsers are different. As device capabilities continue to evolve and users move more of their browsing to mobile browsers, web developers are facing an increasing need to write automated tests for the mobile web in the same way that desktop-oriented site content is tested.

Starting in iOS 13, Safari now includes native support for the W3C WebDriver standard. This feature builds on existing support for WebDriver in desktop Safari, first introduced in Safari 10 and macOS Sierra. This blog post explains how to get started with WebDriver for iOS, how to port your existing desktop-oriented tests, and a few details regarding safeguards and changes in behavior.

Getting Started

Control via WebDriver is exposed to developers via the /usr/bin/safaridriver executable, which hosts a driver that handles REST API requests sent by WebDriver test clients. In order to run WebDriver tests on an iOS device, it must be plugged into a macOS host that has a new enough version of safaridriver. Support for hosting iOS-based WebDriver sessions is available in safaridriver included with Safari 13 and later. Older versions of safaridriver do not support iOS WebDriver sessions.

If you’ve never used safaridriver on macOS before, you’ll first need to run safaridriver --enable and authenticate as an administrator. Then, you must enable Remote Automation on every device that you intend to use for WebDriver. To do this, toggle the setting in Settings → Safari → Advanced → Remote Automation.

Remote Automation Setting on iOS

Lastly, to run tests on the device, you’ll need to plug it into the macOS machine, trust the host, and ensure that it is unlocked when you try to start the WebDriver session.

Since safaridriver is capable of starting WebDriver sessions using Safari on the macOS host or Safari on an attached iOS device, it needs a little extra hint to know that you want to use a device or a desktop machine. To facilitate this, safaridriver supports many new capabilities that may be provided with a new session request. The most important new capability to use is ‘platformName’: ‘ios’; this tells the driver you want an iOS host instead of a macOS host. You can also specify the host(s) you wish to use by specifying device type, device UDID, device name, OS version, OS build, and browser version via standardized and extension capabilities. Lastly, safaridriver supports running WebDriver tests on iOS Simulator devices whose runtime is iOS 13 or later. To use a simulator for testing, add the capability ‘safari:useSimulator’: true. For a full list of supported capabilities and their semantics, please consult the safaridriver(1) man page by executing man safaridriver.

Safeguards

While it’s awesome to be able to write automated tests that run in Safari, a REST API for browser automation introduces a potential vector for malicious software. To support a valuable automation API without sacrificing a user’s privacy or security, Safari’s driver implementation introduces extra safeguards that further restrict browsing in WebDriver sessions. These safeguards also double as extra insurance against “flaky tests” by providing enhanced test isolation. Some of these safeguards are described below.

Just like on macOS, Safari on iOS isolates WebDriver tests by using a separate set of windows, tabs, preferences, and persistent storage. When a WebDriver session is active, existing tabs are hidden and a distinctively-colored WebDriver window is shown instead. Automation windows are easy to recognize by their orange Smart Search field. Similar to browsing in a Private Browsing window, WebDriver tests that run in an Automation window always start from a clean slate and cannot access Safari’s normal browsing history, AutoFill data, or other sensitive information. Aside from the obvious privacy benefits, this restriction also helps to ensure that tests are not affected by a previous test session’s persistent state such as local storage or cookies. When the WebDriver session terminates, the orange window goes away, preexisting tabs are restored, and any state accumulated during the WebDriver session is destroyed.

As a WebDriver test is executing in an Automation window, any attempts to interact with the window or web content could derail the test by unexpectedly changing the state of the page. To prevent this from happening, Safari installs a “glass pane” over the Automation window while the test is running. This blocks any stray interactions (mouse, keyboard, resizing, and so on) from affecting the Automation window. If a running test gets stuck or fails, it’s possible to interrupt the running test by tapping on the screen and choosing to end the session. When an automation session is interrupted, the test’s connection to the browser is permanently severed and cannot be resumed. Notifications and system gestures (i.e., slide in from top/bottom) are also suppressed while WebDriver tests are running in order to avoid flakiness.

Glass Pane

A non-goal of this release is to support general-purpose device automation via the WebDriver API. The W3C WebDriver API focuses on automating interactions with web content, and that is also the focus of the WebDriver support added in iOS 13. Other means exist to change system-level configuration with the XCTest Framework, and are complimentary to WebDriver. There are other ways in which web content interacts with the rest of iOS, and most of these are suppressed in order to provide a stable testing environment. Safari for iOS does not allow WebDriver-initiated navigations to be handled outside of Safari. In other words, clicking a tel:// link will not offer to place a phone call, and clicking an app store link will not redirect the user to the App Store. Similarly, non-Safari content displayed by the system–such as update dialogs, app notifications, incoming calls, etc.–are suppressed while a WebDriver session is active. Lastly, the software keyboard is suppressed when focusing text fields in WebDriver sessions. To simulate text input with WebDriver, use the existing Perform Actions endpoint or Element Send Keys endpoint.

Porting Tests

A major goal of iOS 13 is to make it easier to develop and consume desktop-like web experiences on mobile devices. Towards that end, a major goal for WebDriver on iOS is to make it easy to port existing desktop-oriented WebDriver tests to work with Desktop-class browsing in Safari on iPadOS. As much as we’d like to treat macOS and iOS devices the same in our tests, there are important differences that affect how you may want to use WebDriver.

The first notable difference is handling of user input, namely touch and clicks. In this initial version of WebDriver on iOS, simulated input sources include keyboard, mouse, and single-finger touches. The Perform Actions command and Element Click command synthesize touches using mouse inputs. If your existing test uses mouse clicks, double, clicks, and drags, safaridriver converts these interactions into equivalent single finger touches; no code needs to be changed. Multiple touch input sources (i.e., fingers) are not supported in this release. We experimented with offering this via the existing Perform Actions endpoint, but found that it was difficult to author WebDriver commands that performed well-known gestures such as flick, pinch, pan, and so on. This is partly due to the low-level nature of the Actions API, and partly due to the timing-sensitive nature of gesture recognizers used by Safari and WebKit. Apple is working within the W3C WebDriver working group to define a more expressive, intuitive, and stable API for performing gestures in WebDriver tests.

A less surprising difference between macOS and iOS WebDriver is that endpoints to manipulate the OS window are generally not supported by Safari on iOS. Specifically, the Set Window Rect, Minimize Window, and Maximize Window endpoints will return “unsupported operation”. For more detailed information concerning which endpoints are supported, see the W3C WebDriver Commands reference.

Summary

With the introduction of native WebDriver support in Safari on iOS 13, it’s now possible to run the same automated tests of desktop-oriented web content on desktop and mobile devices equally. Safari’s support comes with new, exclusive safeguards to simultaneously protect user security and privacy and also help you write more stable and consistent tests. You can try out Safari’s WebDriver support today by installing a beta of macOS Catalina and iOS 13.

If you encounter unexpected behavior in Safari’s WebDriver support, or have other suggestions for improvement, please file a bug at https://bugreport.apple.com/. For quick questions or feedback, email web-evangelist@apple.com or tweet @webkit on Twitter. Happy testing!

July 08, 2019 05:00 PM

June 28, 2019

Michael Catanzaro: On Version Numbers

Igalia WebKit

I’m excited to announce that Epiphany Tech Preview has reached version 3.33.3-33, as computed by git describe. That is 33 commits after 3.33.3:

Epiphany about dialog displaying the version number

I’m afraid 3.33.4 will arrive long before we  make it to 3.33.3-333, so this is probably the last cool version number Epiphany will ever have.

I might be guilty of using an empty commit to claim the -33 commit.

I might also apologize for wasting your time with a useless blog post, except this was rather fun. I await the controversy of your choice in the comments.

By Michael Catanzaro at June 28, 2019 04:07 PM

June 26, 2019

Release Notes for Safari Technology Preview 86

Surfin’ Safari

Safari Technology Preview Release 86 is now available for download for macOS Catalina betas and macOS Mojave. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 246093-246691.

Pointer Events

  • Added support for chorded button interactions (r246103, r246149)
  • Updated to fire pointerout and pointerleave events after firing pointercancel (r246122)

JavaScript

  • Added support for String.prototype.matchAll (r246567))
  • Changed to throw a TypeError exception if Proxy’s set trap returns falsy value (r246346)
  • Fixed JSON.parse to throw a syntax error when called without arguments (r246162)

Web Assembly

  • Added support for Anyref tables, Table.get and Table.set for Anyref only (r246139)
  • Added support for multiple tables (r246571)
  • Added support for Table.size, grow and fill instructions (r246577)
  • Fixed float64 “select” on ARM64 devices (r246134)

WebRTC

  • Updated mediaDevices.enumerateDevices() to list the system default audio devices with deviceId as “default” (r246215)

Web API

  • Fixed IntersectionObserver rootMargin detection failure when root is an element (r246432)
  • Updated to avoid generating a new XSLT-based document when already changing the document. (r246182)

CSS

  • Implemented tab-size with units (r246193)
  • Included touch-action in the computed styles list (r246314)

Security

  • Added wildcard to Access-Control-Allow-Methods and Access-Control-Allow-Headers (r246238)
  • Changed blob URLs to inherit CSP policy from their parent (r246277)
  • Changed data URLs to inherit their CSP policy from their parent (r246129)

Experimental Web Shading Language (WHLSL)

  • Enabled compute shaders (r246427)
  • Implemented array references (r246394)
  • Implemented loop expressions (r246121)
  • Implemented out-of-bounds and nullptr behavior (r246438)
  • Supported matrices (r246579)

WebGPU

  • Removed GPUBuffer.setSubData and implemented GPUDevice.createBufferMapped (r246217)

Web Inspector

  • Included JavaScript call trees when exporting or importing Timeline recordings (r246292)
  • Improved the performance of the CSS pretty printer by using a Worker (r246178)
  • Split the General panel in the Settings tab into sub panels (r246510)
  • Ensured that indent type and size settings are respected everywhere that there’s a CodeMirror editor (r246419)
  • Added support for respecting case sensitive or regex global settings when searching in a resource (r246502)
  • Changed to ensure that adding a DOM, Event, or URL breakpoint is enabled globally (r246523)
  • Added support for multiline keys when using “Copy Path to Property” (r246271)
  • Fixed an issue where longhand CSS properties that were overridden by shorthands didn’t have a strikethrough (r246223)
  • Fixed an issue where query parameters in the Network tab would truncate if the value contained a = (r246559)
  • Fixed an issue where the Inspector Style Sheet was missing when grouping resources by path (r246509)

June 26, 2019 05:00 PM

June 21, 2019

A New Bytecode Format for JavaScriptCore

Surfin’ Safari

In revision r237547 we introduced a new bytecode format for JavaScriptCore (JSC). The goals of the new format were to improve memory usage and allow the bytecode to be cached on disk, while the previous format was optimized for interpreter throughput at the cost of memory usage.

In this post, we will start with a quick overview of JSC’s bytecode, key aspects of the old bytecode format and the optimizations it enabled. Next, we will look into the new format and how it affects interpreter execution. Finally, we will look at the impact of the new format on memory usage and performance and how this rewrite improved type safety in JavaScriptCore.

Background

Before JSC executes any JavaScript code, it must lex, parse and generate bytecode for it. JSC has 4 tiers of execution:

  • Low Level Interpreter (LLInt): the start-up interpreter
  • Baseline JIT: a template JIT
  • DFG JIT: a low-latency optimizing compiler
  • FTL JIT: a high-throughput optimizing compiler

Execution starts by interpreting the bytecode, at the lowest tier, and as the code gets executed more it gets promoted to a higher tier. This is described in a lot more details in this blog post about the FTL.

The bytecode is the source of truth throughout the whole engine. The LLInt executes the bytecode. The baseline is a template JIT, which emits snippets of machine code for each bytecode instruction. Finally, the DFG and FTL parse the bytecode and emit DFG IR, which is then run through an optimizing compiler.

Because the bytecode is the source of truth, it tends to stay alive in memory throughout the whole program execution. In JavaScript-heavy websites, such as Facebook or Reddit, the bytecode is responsible for 20% of the overall memory usage.

The Bytecode

To make things more concrete, let’s look at a simple JavaScript program, learn how to inspect the bytecode generated by JSC and how to interpret the bytecode dump.

// double.js
function double(a) {
    return a + a;
}
double(2);

If you run the above program with jsc -d double.js, JSC will dump all the bytecode it generates to stderr. The bytecode dump will contain the bytecode generated for double:

[   0] enter
[   1] get_scope          loc4
[   3] mov                loc5, loc4
[   6] check_traps
[   7] add                loc7, arg1, arg1, OperandTypes(126, 126)
[  13] ret                loc7

Each line starts with the offset of the instruction in brackets, followed by the opcode name and its operands. Here we can see the operands loc for local variables, arg for function arguments and OperandTypes, which is metadata about the predicted types of the arguments.

Old Bytecode Format

The old bytecode format had a few issues that we wanted to fix:

  • It used too much memory.
  • The instruction stream was writable, which prevented memory-mapping the bytecode stream.
  • It had optimizations that we no longer benefited from, such as direct threading.

In order to better understand how we addressed these issues in the new format, we need a basic understanding of the old bytecode format. In the old format, instructions could be in one of two forms: unlinked, which is compact and optimized for storage and linked, which is inflated and optimized for execution, containing memory addresses of runtime objects directly in the instruction stream.

Unlinked Instructions

The instructions were encoded using variable-width encoding. The opcode and each operand took as little space as possible, ranging from 1 to 5 bytes. Take the add instruction from the program above as an example, it would take 6 bytes: one for the opcode (add), one for each of the registers (loc7, arg1 and arg1 again) and two bytes for the operand types.

Unlinked instructions

Linking / Linked Instructions

Before executing the bytecode it needed to be linked. Linking inflated all the instructions, making the opcode and each of the operands pointer-sized. The opcode was replaced by the actual pointer to the implementation of the instruction and the profiling-related metadata replaced with the memory address of the newly allocated profiling data structures. The add from above took 40 bytes to represent:

Linked instructions

Execution

The bytecode is executed by the LLInt, which is written in a portable assembly called offlineasm. Here are a few notes about offlineasm that may help understand the code snippets that will follow:

  • Temporary registers are t0-t5, argument registers are a0-a3 and return registers are r0 and r1. For floating point, the equivalent registers are ft0-ft5, fa0-fa3 and fr. cfp and sp are special registers that hold the call frame and stack pointer respectively.
  • Instructions have one of the following suffixes: b for byte, h for 16-bit word, i for 32-bit word, q for 64-bit word and p for pointer (either 32- or 64-bit depending on the architecture).
  • Macros are lambda expressions that take zero or more arguments and return code. Macros may be named or anonymous and can be passed as arguments to other macros.

If you want to learn more about offlineasm, LowLevelInterpreter.asm is the main offlineasm file in JSC and it contains a more in depth explanation of the language at the top.

The old bytecode format had two big advantages related to interpreter throughput: direct threading and inline caching.

Direct Threading

The linked bytecode had the actual pointer to the offlineasm implementation of the instruction in place of the opcode, which made executing the next instruction as simple as advancing the program counter (PC) by the size of the current instruction and making an indirect jump. That is illustrated in the following offlineasm code:

macro dispatch(instructionSize)
    addp instructionSize * PtrSize, PC
    jmp [PC]
end

Notice that dispatch is a macro, this way the code is duplicated at the end of every instruction. This is very efficient, since it’s just an addition plus an indirect branch. The duplication reduces branch prediction pollution since we don’t have all instructions jumping to a common label and sharing the same indirect jump to the next instruction.

Inline Caching

Since the instruction stream is writable and all the arguments are pointer-sized, we can store metadata in the instruction stream itself. The best example of this is for the get_by_id instruction, which is emitted when we load from an object in JavaScript.

object.field

This emits a get_by_id for loading the property field from object. Since this is one of the most common operations in JavaScript, it’s crucial that it be fast. JSC uses inline caching as a way of speeding this up. The way this was done in the interpreter was by reserving space in the instruction stream to cache metadata about the load. More specifically, we recorded the StructureID of the object we are loading from and the memory offset from which we have to load the value. The LLInt implementation of get_by_id looked like the following:

_llint_op_get_by_id:
    // Read operand 2 from the instruction stream as a signed integer,
    // i.e. the virtual register of `object`
    loadisFromInstruction(2, t0)

    // Load `object` from the stack and its structureID
    loadConstantOrVariableCell(t0, t3, .opGetByIdSlow)
    loadi JSCell::m_structureID[t3], t1

    // Read the cached StructureID and verify that it matches the object's
    loadisFromInstruction(4, t2)
    bineq t2, t1, .opGetByIdSlow

    // Read the PropertyOffset of `field` and load the actual value from it
    loadisFromInstruction(5, t1)
    loadPropertyAtVariableOffset(t1, t3, t0)

    // Read the virtual register where the result should be stored and store
    // the value to it
    loadisFromInstruction(1, t2)
    storeq t0, [cfr, t2, 8]
    dispatch(constexpr op_get_by_id_length)

.opGetByIdSlow
    // Jump to C++ slow path
    callSlowPath(_llint_slow_path_get_by_id)
    dispatch(constexpr op_get_by_id_length)

New Bytecode Format

When designing the new bytecode, we had two major goals: it should be more compact and easily cacheable on disk. With these two goals, we expected significant improvements on memory usage and set ourselves to improve runtime performance through caching. This shaped how we encoded instructions as well as runtime metadata.

The first and biggest change is that we no longer have a separate linked encoding for execution. This immediately means that the bytecode can no longer be direct threaded, since the address of the instruction could not be stored to disk, as it changes with every program invocation. Removing this optimization was a deliberate choice of the design.

In order to make the single format suitable for both storage and execution, each instruction can be encoded as either narrow or wide.

Narrow Instructions

In a narrow instruction, the opcode and its operands each take 1-byte. Here’s the add instruction again, this time as a narrow instruction in the new format:

Narrow

Ideally, all instructions would be encoded as narrow, but not all operands fit into a single byte. If any of the operands (or the opcode for that matter) requires more than 1 byte, the whole instruction is promoted to wide.

Wide Instructions

A wide instruction consists of a special 1-byte opcode, op_wide, followed by a series of 4-byte slots for the original opcode and each of its arguments.

Wide Instructions

An important thing to notice here is that it is not possible to have a single wide operand – if one operand overflows, the whole instruction becomes wide. This is an important compromise, because even though it might seem wasteful at first, it makes the implementation much simpler: the offset of any given operand is the same regardless of the instruction being narrow or wide. The only difference is whether the instruction stream is treated as an array of 4-byte values or 1-byte values.

Linking / Metadata Table

The other fundamental part of the new bytecode is the metadata table. During linking, instead of constructing a new instruction stream, we initialize a side table with all the writable data associated with any given instruction. Conceptually, the table is 2-dimensional, its first index is the opcode for the instruction, e.g. add, which points to a monomorphic array of metadata entries for that specific instruction. For example, we have a struct called OpAdd::Metadata to hold the metadata for the add instruction, so accessing metadataTable[op_add] will result in a pointer of type OpAdd::Metadata*. Additionally, each instruction has a special operand, metadataID, which is used as the second index in the metadata table.

Metadata

For compactness, the metadata table is laid out in memory as a single block of memory. Here’s a representation of what the table above would actually look like in memory.

Metadata Memory Table

At the start of the table is the header, an array containing an integer for each opcode representing the offset from the start of the table where the metadata for that opcode starts. The next region is the payload, which contains the actual metadata for each opcode.

Execution

The biggest changes to execution are that the interpreter is now indirect threaded, and for any writable or runtime metadata, we need to read from the metadata table. Another interesting aspect is how wide instructions are executed.

Indirect Threading

The process of mapping from opcode to instruction address, which used to be done as part of linking in the old format, is now part of the interpreter dispatch. This means that extra loads are required, and the dispatch macro now looks like the following:

macro dispatch(instructionSize)
    addp instructionSize, PC
    loadb [PC], t0
    leap _g_opcodeMap, t1
    jmp [t1, t0, PtrSize]
end

Metadata Table

The same kind of “inline caching”† still applies to get_by_id, but given that we need to write the metadata at runtime and the instruction stream is now read-only, this data needs to live in the metadata table.

Loads from the metadata table are a bit costly, since the CodeBlock needs to be loaded from the call frame, the metadata table from the CodeBlock, the array for the current opcode from the table and then finally the metadata entry from the array based on the current instruction’s metadataID.

† I write “inline caching” in quotes here since it’s not technically inline anymore.

Wide Instruction Execution

The execution of wide instructions is surprisingly simple: there are two functions for each instruction, one for the narrow version and another for the wide. We take advantage of offlineasm to generate the two functions from a single implementation. (e.g. the implementation of op_add will automatically generate its wide version, op_add_wide.)

By default, the narrow version of the instruction is executed, until the special instruction op_wide is executed. All it does is read the next opcode, and dispatch to its wide version, e.g. op_add_wide. Once the wide opcode is done, it goes back to dispatching a narrow instruction, since every wide instruction needs the 1-byte op_wide prefix.

Memory Usage

The new bytecode format uses approximately 50% less memory, which means a reduction of 10% in overall memory usage for JavaScript-heavy websites, such as Facebook or Reddit.

Here’s a chart comparing the bytecode size for reddit.com, apple.com, facebook.com and gmail.com.

Bytecode size charts for reddit.com, apple.com, facebook.com and gmail.com

Notice that in the After case the Metadata has been merged with Linked to represent the memory used by the metadata table and Unlinked represents the actual bytecode instructions. The reason is that a significant part of what lives in the new metadata used to live in the old linked bytecode. Otherwise, the comparison would look skewed, since the whole Linked bar would be gone and the Metadata would seem to have doubled in size, when in reality it was part of the data from Linked that moved into Metadata.

Performance

As discussed earlier, indirect threading increases the overhead of the interpreter dispatch. However, given the average complexity of the bytecode instructions in JSC we did not expect the dispatch overhead to play a meaningful role in the overall performance of the interpreter. This is further amortized by the multiple JIT tiers. We profiled purely switching from direct to indirect threading in CLoop, the C++ backend for LLInt, and the results were neutral even with the JITs disabled.

Loads from the metadata table are costly, involving a large chain of loads. In order to reduce this chain, we pin a callee save register in the interpreter to hold the pointer to the metadata table at all times. Even with this optimization, it takes three loads to access the metadata entry. This was a 10% interpreter slowdown, but was neutral when running with all JIT tiers enabled.

Bonus: Type Safety

Changing the bytecode format required changes across the whole engine, so we decided to take this as an opportunity to improve the bytecode-related infrastructure, increasing the type safety, readability and maintainability of the code.

To support all this changes, first we needed to change how instructions were specified. Originally, instructions were declared in a JSON file, with their respective name and length (the number of operands plus one, for the opcode). Here is how the add instruction was declared:

{ "name": "op_add", "length": 5 }

When accessing the instruction and its operands from C++ the code would look roughly like this:

SLOW_PATH_DECL(slow_path_add)
{
    JSValue lhs = OP_C(2).jsValue();
    JSValue rhs = OP_C(3).jsValue();
    ...
}

Notice that operands were accessed by their offset: OP_C was a macro that would access the Instruction* pc argument (hidden here by the SLOW_PATH_DECL macro) at the given offset. However, the type of the data at the specified offset is unknown, so the result would have to be cast to the desired type. This is error-prone and makes the code harder to understand. The only way to learn which operands an instruction had (and their types) was to look for usages of it and look at variable names and type casts.

With the new infrastructure, when declaring an instruction, a name and type must be given for each of the operands, as well as declaring all the data that will be stored in the metadata table.

op :add,
    args: {
        dst: VirtualRegister,
        lhs: VirtualRegister,
        rhs: VirtualRegister,
        operandTypes: OperandTypes,
    },
    metadata: {
        arithProfile: ArithProfile,
    }

With this extra information, it is now possible to generate a fully typed struct for each instruction, which results in safer C++ code:

SLOW_PATH_DECL(slow_path_add)
{
   OpAdd bytecode = pc->as<OpAdd>();
   JSValue lhs = GET_C(bytecode.m_lhs);
   JSValue rhs = GET_C(bytecode.m_rhs);
   ...
}

In the example above, we first need to convert the generic Instruction* pc to the instruction we want to access, which will perform a runtime check. If the opcode matches, it returns an instance of the generated struct, OpAdd, which includes the fields m_lhs and m_rhs, both of type VirtualRegister as specified in our instruction declaration.

The extra information also allowed us to replace a lot of mechanical code with safer, generated code. For example, we auto generate all the type safe code for writing instructions to the instruction stream, which automatically does the narrow/wide fitting. We also generate all the code to dump the instructions for debugging.

Conclusion

JavaScriptCore has a new bytecode format that uses 50% less memory on average and is available on Safari 12.1 and Safari Technology Preview. The work on the caching API for the new bytecode is already underway in the WebKit repository, and anyone interested is welcome to follow along and contribute on bugs.webkit.org! You can also get in touch with me on Twitter with any questions or comments.

June 21, 2019 05:00 PM

June 14, 2019

Michael Catanzaro: An OpenJPEG Surprise

Igalia WebKit

My previous blog post seems to have resolved most concerns about my requests for Ubuntu stable release updates, but I again received rather a lot of criticism for the choice to make WebKit depend on OpenJPEG, even though my previous post explained clearly why there are are not any good alternatives.

I was surprised to receive a pointer to ffmpeg, which has its own JPEG 2000 decoder that I did not know about. However, we can immediately dismiss this option due to legal problems with depending on ffmpeg. I also received a pointer to a resurrected libjasper, which is interesting, but since libjasper was removed from Ubuntu, its status is not currently better than OpenJPEG.

But there is some good news! I have looked through Ubuntu’s security review of the OpenJPEG code and found some surprising results. Half the reported issues affect the library’s companion tools, not the library itself. And the other half of the issues affect the libmj2 library, a component of OpenJPEG that is not built by Ubuntu and not used by WebKit. So while these are real security issues that raise concerns about the quality of the OpenJPEG codebase, none of them actually affect OpenJPEG as used by WebKit. Yay!

The remaining concern is that huge input sizes might cause problems within the library that we don’t yet know about. We don’t know because OpenJPEG’s fuzzer discards huge images instead of testing them. Ubuntu’s security team thinks there’s a good chance that fixing the fuzzer could uncover currently-unknown multiplication overflow issues, for instance, a class of vulnerability that OpenJPEG has clearly had trouble with in the past. It would be good to see improvement on this front. I don’t think this qualifies as a security vulnerability, but it is certainly a security problem that would facilitate discovering currently-unknown vulnerabilities if fixed.

Still, on the whole, the situation is not anywhere near as bad as I’d thought. Let’s hope OpenJPEG can be included in Ubuntu main sooner rather than later!

By Michael Catanzaro at June 14, 2019 02:43 PM

June 12, 2019

Release Notes for Safari Technology Preview 85

Surfin’ Safari

Safari Technology Preview Release 85 is now available for download for macOS Catalina betas and macOS Mojave. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 245618-246093.

Web Authentication

  • Added support for Attestation Conveyance Preference (r245638)

Pointer Events

  • Updated to ensure that capturing data managed by the PointerCaptureController gets cleared upon navigation (r245809, r246031)
  • Changed compatibility mouse events to be able to be prevented while the pointer is pressed (r245699)
  • Exposed navigator.maxTouchPoints (r246070)
  • Changed to only allow pointer capture if the pointer is in the active buttons state (r246074)
  • Changed to not prevent mouseover, mouseout, mouseenter, and mouseleave events while the pointer is down (r245695)
  • Changed toElement and fromElement on PointerEvent to be null (r245867)
  • Changed mouseenter and pointerenter events to be fired from the bottom up (r246061)

Editing

  • Fixed inserting a newline in contenteditable add one character, not two (r245912, r245980)

Media

  • Added an option to mute audio capture automatically when page is not visible (r246002)
  • Allowed for resizing camera video feeds to very small resolutions (r246049)
  • Fixed createAnswer() SDP Rejected by setLocalDescription() (r245829)
  • Fixed playing one video at a time for multiple videos with audio, and the autoplay and playinline attributes (r245712)
  • Changed to hide MediaCapabilities.encodingInfo() when the platform does not support it. (r245636)

CSS

  • Changed precedence for min-width to always win over max-width (r245966)
  • Fixed font-optical-sizing to apply the correct variation value (r245672)
  • Updated to include searching sub-selectors when determining the property whitelist for selector (r245664)
  • Updated to preserve CSS Grid repeat() notation when serializing declared values (r245798)

Web API

  • Fixed bounding client rect for an inline element in Intersection Observer (r245642)
  • Implemented feature policy self, none, and * parsing (r245625)
  • Implemented imagesrcset and imagesizes attributes on link rel=preload (r246045)
  • Implemented Promise.allSettled (r245869)
  • Fixed programmatic scroll in right-to-left overflow with async scrolling enabled (r245771)
  • Added support for preferred presentation size when pasting an image (r245637)
  • Made the computed width of non-replaced inline return computed style (r245768)

JavaScript

  • Implemented support for Numeric Separators (r245655)
  • Implemented opwide16 and opwide32 and introduced 16-bit version bytecode (r245906)
  • Fixed InferredValue to not be a JSCell (r246073)
  • Reduced metadata footprint (r245658)
  • Changed createListFromArrayLike to throw a type error if the value is not an object (r245675)

WebAssembly

  • Added support for anyref in globals (r245765)

Web GPU

  • Updated vertex buffers and Input State API (r245905)

WHLSL

  • Enforced variable lifetimes (r245945)
  • Implemented property resolver (r245680, r245722)
  • Improved the speed of parsing and lexing the standard library (r246052)

Web Inspector

  • Exposed a way to get the current Audit version from within an Audit (r245909)
  • Allowed arbitrary JSON data to be returned as part of the result of an Audit (r245914)
  • Changed CSS modifications to be shared for rules that match multiple elements (r245991)
  • Updated the debugger navigation sidebar to always reveal the active call frame when hitting a breakpoint (r246026)
  • Moved the overlay rulers to the opposite vertical or horizontal side if they intersect the highlighted nodes so that no content is obstructed (r245728)
  • Added a setting to show overlay rulers and guides whenever element selection is enabled (r245730)

June 12, 2019 08:00 PM

June 10, 2019

Javier Fernández: A new terminal-style line breaking with CSS Text

Igalia WebKit

The CSS Text 3 specification defines a module for text manipulation and covers, among a few other features, the line breaking behavior of the browser, including white space handling. I’ve been working lately on some new features and bug fixing for this specification and I’d like to introduce in this posts the last one we made available for the Web Platform users. This is yet another contribution that came out the collaboration between Igalia and Bloomberg, which has been held for several years now and has produced many important new features for the Web, like CSS Grid Layout.

The feature

I guess everybody knows the white-space CSS property, which allows web authors to control two main aspects of the rendering of a text line: collapsing and wrapping. A new value break-spaces has been added to the ones available for this property, which allows web authors to emulate a terminal-like line breaking behavior. This new value operates basically like pre-wrap, but with two key differences:

  • any sequence of preserved white space characters takes up space, even at the end of the line.
  • a preserved white space sequence can be wrapped at any character, moving the rest of the sequence, intact, to the line bellow.

What does this new behavior actually mean ? I’ll try to explain it with a few examples. Lets start with a simple but quite illustrative demo which tries to emulate a meteorology monitoring system which shows relevant changes over time, where the gaps between subsequent changes must be preserved:



 #terminal {
  font: 20px/1 monospace;
  width: 340px;
  height: 5ch;
  background: black;
  color: green;
  overflow: hidden;
  white-space: break-spaces;
  word-break: break-all;
 }


    

Another interesting use case for this feature could be a logging system which should preserve the text formatting of the logged information, considering different window sizes. The following demo tries to describe this such scenario:



body { width: 1300px; }
#logging {
  font: 20px/1 monospace;
  background: black;
  color: green;

  animation: resize 7s infinite alternate;

  white-space: break-spaces;
  word-break: break-all;
}
@keyframes resize {
  0% { width: 25%; }
  100% { width: 100%; }
}

Hash: 5a2a3d23f88174970ed8 Version: webpack 3.12.0 Time: 22209ms Asset Size Chunks Chunk Names pages/widgets/index.51838abe9967a9e0b5ff.js 1.17 kB 10 [emitted] pages/widgets/index img/icomoon.7f1da5a.svg 5.38 kB [emitted] fonts/icomoon.2d429d6.ttf 2.41 kB [emitted] img/fontawesome-webfont.912ec66.svg 444 kB [emitted] [big] fonts/fontawesome-webfont.b06871f.ttf 166 kB [emitted] img/mobile.8891a7c.png 39.6 kB [emitted] img/play_button.6b15900.png 14.8 kB [emitted] img/keyword-back.f95e10a.jpg 43.4 kB [emitted] . . .

Use cases

In the demo shown before there are several cases that I think it’s worth to analyze in detail.

A breaking opportunity exists after any white space character

The main purpose of this feature is to preserve the white space sequences length even when it has to be wrapped into multiple lines. The following example tries to describe this basic use case:



.container {
  font: 20px/1 monospace;
  width: 5ch;
  white-space: break-spaces;
  border: 1px solid;
}

XX XX

The example above shows how the white space sequence with a length of 15 characters is preserved and wrapped along 3 different lines.

Single leading white space

Before the addition of the break-spaces value this scenario was only possible at the beginning of the line. In any other case, the trailing white spaces were either collapsed or hang, hence the next line couldn’t start with a sequence of white spaces. Lets consider the following example:



.container {
  font: 20px/1 monospace;
  width: 3ch;
  white-space: break-spaces;
  border: 1px solid;
}

XX XX

Like when using pre-wrap, the single leading space is preserved. Since break-spaces allows breaking opportunities after any white space character, we break after the first leading white space (” |XX XX”). The second line can be broken after the first preserved white space, creating another leading white space in the next line (” |XX | XX”).

However, lets consider now a case without such first single leading white space.



.container {
  font: 20px/1 monospace;
  width: 3ch;
  white-space: break-spaces;
  border: 1px solid;
}

XXX XX

Again, it s not allowed to break before the first space, but in this case there isn’t any previous breaking opportunity, so the first space after the word XX should overflow (“XXX | XX”); the next white space character will be moved down to the next line as preserved leading space.

Breaking before the first white space

I mentioned before that the spec states clearly that the break-space feature allows breaking opportunities only after white space characters. However, it’d be possible to break the line just before the first white space character after a word if the feature is used in combination with other line breaking CSS properties, like word-break or overflow-wrap (and other properties too).



.container {
  font: 20px/1 monospace;
  width: 4ch;
  white-space: break-spaces;
  overflow-wrap: break-word;
  border: 1px solid;
}

XXXX X

The two white spaces between the words are preserved due to the break-spaces feature, but the first space after the XXXX word would overflow. Hence, the overflow-wrap: break-word feature is applied to prevent the line to overflow and introduce an additional breaking opportunity just before the first space after the word. This behavior causes that the trailing spaces are moved down as a leading white space sequence in the next line.

We would get the same rendering if word-break: break-all is used instead overflow-wrap (or even in combination), but this is actualy an incorrect behavior, which has the corresponding bug reports in WebKit (197277) and Blink (952254) according to the discussion in the CSS WG (see issue #3701).

Consider previous breaking opportunities

In the previous example I described a combination of line breaking features that would allow breaking before the first space after a word. However, this should be avoided if there are previous breaking opportunities. The following example is one of the possible scenarios where this may happen:



.container {
  font: 20px/1 monospace;
  width: 4ch;
  white-space: break-spaces;
  overflow-wrap: break-word;
  border: 1px solid;
}

XX X X

In this case, we could break after the second word (“XX X| X”), since overflow-wrap: break-word would allow us to do that in order to avoid the line to overflow due to the following white space. However, white-space: break-spaces only allows breaking opportunities after a space character, hence, we shouldn’t break before if there are valid previous opportunities, like in this case in the space after the first word (“XX |X X”).

This preference for previous breaking opportunities before breaking the word, honoring the overflow-wrap property, is also part of the behavior defined for the white-space: pre-wrap feature; although in that case, there is no need to deal with the issue of breaking before the first space after a word since trailing space will just hang. The following example uses just the pre-wrap to show how previous opportunities are selected to avoid overflow or breaking a word (unless explicitly requested by word-break property).



.container {
  font: 20px/1 monospace;
  width: 2ch;
  white-space: pre-wrap;
  border: 1px solid;
}

XX
overflow-wrap:
break-word
word-break:
break-all

In this case, break-all enables breaking opportunities that are not available otherwise (we can break a word at any letter), which can be used to prevent the line to overflow; hence, the overflow-wrap property doesn’t take any effect. The existence of previous opportunities is not considered now, since break-all mandates to produce the longer line as possible.

This new white-space: break-spaces feature implies a different behavior when used in combination with break-all. Even though the preference of previous opportunities should be ignored if we use the word-break: break-all, this may not be the case for the breaking before the first space after a word scenario. Lets consider the same example but using now the word-break: break-all feature:



.container {
  font: 20px/1 monospace;
  width: 4ch;
  white-space: break-spaces;
  overflow-wrap: break-word;
  word-break: break-all;
  border: 1px solid;
}

XX X X

The example above shows that using word-break: break-all doesn’t produce any effect. It’s debatable whether the use of break-all should force the selection of the breaking opportunity that produces the longest line, like it happened in the pre-wrap case described before. However, the spec states clearly that break-spaces should only allow breaking opportunities after white space characters. Hence, I considered that breaking before the first space should only happen if there is no other choice.

As a matter of fact, specifying break-all we shouldn’t considering only previous white spaces, to avoid breaking before the first white space after a word; the break-all feature creates additional breaking opportunities, indeed, since it allows to break the word at any character. Since break-all is intended to produce the longest line as possible, this new breaking opportunity should be chosen over any previous white space. See the following test case to get a clearer idea of this scenario:



.container {
  font: 20px/1 monospace;
  width: 4ch;
  white-space: break-spaces;
  overflow-wrap: break-word;
  word-break: break-all;
  border: 1px solid;
}

X XX X

Bear in mind that the expected rendering in the above example may not be obtained if your browser’s version is still affected by the bugs 197277(Safari/WebKit) and 952254(Chrome/Blink). In this case, the word is broken despite the opportunity in the previous white space, and also avoiding breaking after the ‘XX’ word, just before the white space.

There is an exception to the rule of avoiding breaking before the first white space after a word if there are previous opportunities, and it’s precisely the behavior the line-break: anywhere feature would provide. As I said, all these assumptions were not, in my opinion, clearly defined in the current spec, so that’s why I filed an issue for the CSS WG so that we can clarify when it’s allowed to break before the first space.

Current status and support

The intent-to-ship request for Chrome has been approved recently, so I’m confident the feature will be enabled by default in Chrome 76. However, it’s possible to try the feature in older versions by enabling the Experimental Web Platform Features flag. More details in the corresponding Chrome Status entry. I want to highlight that I also implemented the feature for LayoutNG, the new layout engine that Chrome will eventually ship; this achievement is very important to ensure the stability of the feature in future versions of Chrome.

In the case of Safari, the patch with the implementation of the feature landed in the WebKit’s trunk in r244036, but since Apple doesn’t announce publicly when a new release of Safari will happen or which features it’ll ship, it’s hard to guess when the break-spaces feature will be available for the web authors using such browser. Meanwhile, It’s possible to try the feature in the Safari Technology Preview 80.

Finally, while I haven’t see any signal of active development in Firefox, some of the Mozilla developers working on this area of the Gecko engine have shown public support for the feature.

The following table summarizes the support of the break-spaces feature in the 3 main browsers:

Chrome Safari Firefox
Experimental M73 STP 80 Public support
Ship M76 Unknown Unknown

Web Platform Tests

At Igalia we believe that the Web Platform Tests project is a key piece to ensure the compatibility and interoperability of any development on the Web Platform. That’s why a substantial part of my work to implement this relatively small feature was the definition of enough tests to cover the new functionality and basic use cases of the feature.

white-space overflow-wrap word-break
pre-wrap-008
pre-wrap-015
pre-wrap-016
break-spaces-003
break-spaces-004
break-spaces-005
break-spaces-006
break-spaces-007
break-spaces-008
break-spaces-009
break-word-004
break-word-005
break-word-006
break-word-007
break-word-008
break-all-010
break-all-011
break-all-012
break-all-013
break-all-014
break-all-015

Implementation in several web engines

During the implementation of a browser feature, even a small one like this, it’s quite usual to find out bugs and interoperability issues. Even though this may slow down the implementation of the feature, it’s also a source of additional Web Platform tests and it may contribute to the robustness of the feature itself and the related CSS properties and values. That’s why I decided to implement the feature in parallel for WebKit (Safari) and Blink (Chrome) engines, which I think it helped to ensure interoperability and code maturity. This approach also helped to get a deeper understanding of the line breaking logic and its design and implementation in different web engines.

I think it’s worth mentioning some of these code architectural differences, to get a better understanding of the work and challenges this feature required until it reached web author’s browser.

Chrome/Blink engine

Lets start with Chrome/Blink, which was especially challenging due to the fact that Blink is implementing a new layout engine (LayoutNG). The implementation for the legacy layout engine was the first step, since it ensures the feature will arrive earlier, even behind an experimental runtime flag.

The legacy layout relies on the BreakingContext class to implement the line breaking logic for the inline layout operations. It has the main characteristic of handling the white space breaking opportunities by its own, instead of using the TextBreakIterator (based on ICU libraries), as it does for determining breaking opportunities between letters and/or symbols. This design implies too much complexity to do even small changes like this, especially because is very sensible in terms of performance impact. In the following diagram I try to show a simplified view of the classes involved and the interactions implemented by this line breaking logic.

The LayoutNG line breaking logic is based on a new concept of fragments, mainly handled by the NGLineBreaker class. This new design simplifies the line breaking logic considerably and it’s highly optimized and adapted to get the most of the TextBreakIterator classes and the ICU features. I tried to show a simplified view of this new design with the following diagram:

In order to describe the work done to implement the feature for this web engine, I’ll list the main bugs and patches landed during this time: CR#956465, CR#952254, CR#944063,CR#900727, CR#767634, CR#922437

Safari/WebKit engine

Although as time passes this is less probable, WebKit and Blink still share some of the layout logic from the ages prior to the fork. Although Blink engineers have applied important changes to the inline layout logic, both code refactoring and optimizations, there are common design patterns that made relatively easy porting to WebKit the patches that implemented the feature for the Blink’s legacy layout. In WebKit, the line breaking logic is also implemented by the BreakingContext class and it has a similar architecture, as it’s described, in a quite simplified way, in the class diagram above (it uses different class names for the render/layout objects, though) .

However, Safari supports for the mac and iOS platforms a different code path for the line breaking logic, implemented in the SimpleLineLayout class. This class provides a different design for the line breaking logic, and, similar to what Blink implements in LayoutNG, is based on a concept of text fragments. It also relies as much as possible into the TextBreakIterator, instead of implementing complex rules to handle white spaces and breaking opportunities. The following diagrams show this alternate design to implement the line breaking process.

This SimpleLineLayout code path in not supported by other WebKit ports (like WebKitGtk+ or WPE) and it’s not available either when using some CSS Text features or specific fonts. There are other limitations to use this SimpleLineLayout codepath, which may lead to render the text using the BreakingContext class.

Again, this is the list of bugs that were solved to implement the feature for the WebKit engine: WK#197277, WK#196169, WK#196353, WK#195361, WK#177327, WK#197278

Conclusion

I hope that at this point these 2 facts are clear now:

  • The white-space: break-spaces feature is a very simple but powerful feature that provides a new line breaking behavior, based on unix-terminal systems.
  • Although it’s a simple feature, on the paper (spec), it implies a considerable amount of work so that it reaches the browser and it’s available for web authors.

In this post I tried to explain in a simple way the main purpose of this new feature and also some interesting corner cases and combinations with other Line Breaking features. The demos I used shown 2 different use cases of this feature, but there are may more. I’m sure the creativity of web authors will push the feature to the limits; by then, I’ll be happy to answer doubts, about the spec or the implementation for the web engines, and of course fix the bugs that may appear once the feature is more used.

Igalia logo
Bloomberg logo

Igalia and Bloomberg working together to build a better web

Finally, I want to thank Bloomberg for supporting the work to implement this feature. It’s another example of how non-browser vendors can influence the Web Platform and contribute with actual features that will be eventually available for web authors. This is the kind of vision that we need if we want to keep a healthy, open and independent Web Platform.

By jfernandez at June 10, 2019 08:11 PM

June 06, 2019

What’s New in the Payment Request API for Apple Pay

Surfin’ Safari

Since announcing last April that WebKit supports the W3C Payment Request API for Apple Pay, we’ve been hard at work adding even more features to the API. From support for phonetic names in the Apple Pay payment method to new standard features like the paymentmethodchange event and PaymentResponse.retry(), WebKit’s Payment Request implementation is now more capable than ever. Anything you could’ve done in the Apple Pay JS API can now be done directly with the Payment Request API using the Apple Pay payment method.

Let’s take a closer look at some of the improvements we’ve made to Payment Request in the past year.

The paymentmethodchange event

When the user selects a card in the Apple Pay payment sheet, WebKit now dispatches the paymentmethodchange event to your PaymentRequest object. WebKit populates the event’s methodDetails attribute with an ApplePayPaymentMethod dictionary, which you can use to determine information about the card such as its type.

Listening for paymentmethodchange is optional, but if you do, be sure to call the event’s updateWith() method with a promise for updated payment details.

const request = new PaymentRequest(...);
request.onpaymentmethodchange = (event) => {
    // Compute new details based on event.methodDetails
    const detailsUpdatePromise = computeDetails(event.methodDetails);
    event.updateWith(detailsUpdatePromise);
};

Whether or not you listen for the paymentmethodchange event, WebKit continues to support payment details modifiers, a declarative way to vary payment details based on the user’s selected card. See last year’s blog post for more information about modifiers.

Requesting Phonetic Names

The Apple Pay payment method now supports requesting a phonetic spelling of your customer’s name as part of shipping and billing contact information. Here’s an Apple Pay payment method that requests a shipping contact containing a localized name, a phonetic name, and a postal address:

const applePayMethod = {
    supportedMethods: "https://apple.com/apple-pay",
    data: {
        ...,
        requiredShippingContactFields: ["name", "phoneticName", "postalAddress"],
    },
};

When your customer authorizes payment, WebKit provides the phonetic name as part of the ApplePayPayment dictionary that’s stored in the PaymentResponse‘s details attribute.

const request = new PaymentRequest([applePayMethod], ...);
request.show().then((response) => {
    const shippingContact = response.details.shippingContact;
    const phoneticFamilyName = shippingContact.phoneticFamilyName;
    const phoneticGivenName = shippingContact.phoneticGivenName;
});

Reporting Errors

When handling contact information, you might encounter an error that needs to be resolved by the customer before they can proceed with the transaction. Common examples include invalid addresses, unserviceable addresses, and missing contact information.

So that you can report these errors to your customers, WebKit now supports Payment Request’s fine-grained error reporting capabilities in the PaymentDetailsUpdate dictionary. You can specify shippingAddressErrors for errors with shipping address attributes, payerErrors for errors with names, email addresses, and phone numbers, and paymentMethodErrors for errors specific to the selected payment method. The Apple Pay payment method treats paymentMethodErrors as a sequence of ApplePayError objects.

Here’s how you might report an error for an invalid postal code:

const request = new PaymentRequest([applePayMethod], details, { requestShipping: true });
request.onshippingaddresschange = (event) => {
    // Compute new details based on shippingAddress
    const detailsUpdate = computeDetails(request.shippingAddress);

    // Check for an invalid postal code
    if (!isValid(request.shippingAddress.postalCode))
        detailsUpdate.shippingAddressErrors = { postalCode: "Invalid postal code" };

    event.updateWith(detailsUpdate);
};

When you report errors, WebKit displays them in the Apple Pay payment sheet and prompts your customer to make corrections — perhaps by editing their shipping address or selecting from a list of stored address. Here’s what the shipping address error we specified above looks like on an iPhone.

When your customer taps on the erroneous shipping address, the Apple Pay sheet displays your custom error message. If they decide to edit their address, the fields you flagged with errors (postalCode, in this case) are highlighted to help guide them through making corrections.

Once the user finishes making corrections, WebKit fires the shippingaddresschange event again. If you are satisfied with the corrections, you can proceed with the transaction by calling the updateWith() method again without specifying errors.

When handling errors in shippingaddresschange, keep in mind that WebKit redacts some parts of the shipping contact before payment authorization. The rules vary by country, but in general, WebKit reveals only the contact information necessary to calculate tax and shipping costs. Once the customer authorizes payment, WebKit reveals all the requested contact information.

Retrying Authorizations

While many errors can be detected when handling events like shippingaddresschange, the redaction rules mentioned above mean that some errors can only be detected once the user has authorized payment. For instance, you might require an email address to send a purchase confirmation, only to find after authorization that the customer did not provide a valid one.

To handle these cases, WebKit now implements PaymentResponse‘s retry() method, allowing you to ask the user to retry the payment after correcting for errors.

When you detect an error in a PaymentResponse, you can call retry() with a PaymentValidationErrors dictionary. Like in PaymentDetailsUpdate, you can specify shippingAddress errors, payer errors, and paymentMethod errors. When you call retry(), it returns a promise that resolves once the PaymentResponse is re-authorized.

Here’s how you might handle an invalid email address:

const handleResponse = (response) => {
    if (!isValid(response.payerEmail)) {
        response.retry({
            payer: { email: "Invalid email address" },
        }).then(() => { handleResponse(response) });
        return;
    }
    response.complete("success");
};

const request = new PaymentRequest([applePayMethod], ...);
request.show().then(response => handleResponse(response));

Keep in mind that retry() is for errors that can be corrected by the user. If you detect a fatal error — for example, the card was declined or an item is no longer in stock — call PaymentResponse‘s complete() method with a PaymentComplete value of 'fail'. The Apple Pay payment sheet will display an appropriate error to the user and then dismiss itself.

Availability

These new Payment Request features are available starting in Safari 12.1 on macOS, Safari on iOS 12.2, and Safari Technology Preview.

Feedback

We’ve received some great developer feedback so far, and we look forward to more of your bug reports and feature requests. If you find a Payment Request bug in WebKit, please report it at bugs.webkit.org. On Twitter, you can reach the WebKit team at @webkit, or our web technologies evangelist Jonathan Davis at @jonathandavis.

June 06, 2019 05:25 PM

CPU Timeline in Web Inspector

Surfin’ Safari

Web Inspector now includes a new CPU usage timeline that lets developers measure a page’s CPU usage, estimate its energy impact, and more easily investigate sources of script execution that may be contributing to poor energy utilization.

Energy Efficiency

Web applications for productivity and entertainment are commonplace. For many users, popular websites are often left open in the foreground of their browser or inactive in a background tab for extended periods of time. The activity on all of these pages may contribute to power drain. Many of the devices used to browse the web are battery powered. To all of those users, battery life matters!

Ensuring web applications make the best use of limited resources is a difficult problem. Having a powerful set of tools makes it easier for developers to investigate and identify performance issues. Many different variables affect energy use on web pages, but the most prominent is CPU activity. Monitoring CPU usage can provide an overall estimate of the energy efficiency of a web page and can highlight cause for concern if activity is too high.

CPU Timeline

The new CPU usage timeline is enabled by default. Open the Timelines tab of Web Inspector and you will see a new CPU timeline in the overview at the top. Select the timeline to show its detail view, which including a breakdown of the main thread activity, an energy impact rating, and more.

New CPU Usage Timeline New CPU Usage Timeline New CPU Usage Timeline New CPU Usage Timeline

The CPU timeline overview shows CPU usage samples taken every 500ms. You can use these samples to visualize activity spikes, and how they correlate with networking, layout and rendering, and JavaScript. To see a detailed overview of CPU usage in a specific section of a recording, click and drag to make a time range selection.

Main Thread Breakdown

The first section of the detail view is a breakdown of the work being performed on the main thread. While the total CPU usage across all threads is important, the work performed on the main thread is particularly important for web content.

CPU Timeline Main Thread Breakdown

Principal tasks such as evaluating script, painting, layout, and style resolution are performed on the main thread. The chart allows you to quickly see a comparison of where time is spent in each of these categories.

The main thread is primarily active in order to service user interaction events such as scrolling and clicking or tapping on a page. This means that the main thread may have periods of inactivity. The number inside the chart shows the approximate number of milliseconds the main thread was active. This number may be much smaller than the total selected time range in the overview.

Different interactions produce different main thread workloads. For example, scrolling the page will increase the time spent painting, resizing the window will cause additional layouts, and typing into a form may execute scripts.

Energy Impact

The Energy Impact section provides an estimate of the battery drain caused by the page. This is based off of the average CPU usage for the selected time range.

Extended periods of high CPU utilization will show “High” energy impact, since sustained high CPU usage will drain power and have a noticeable effect on battery life. When making use of this section there are two important scenarios to consider and measure:

  • Idle recordings – An idle page can be one sitting in the foreground not being interacted with by the user, or a background tab that is still alive. It is important that idle pages use as little energy and CPU as possible. Recordings of idle pages should strive for a “Low” rating (less than 3% average CPU usage).
     
  • Interactive recordings – Recordings that include interactivity such as navigating, scrolling, clicking, typing, etc. should expect higher CPU usage because the page is responding to the user. Periods of interactivity should strive to maintain a “Medium” rating (less than 30% average CPU usage) and avoid “High” if possible.

  • While WebKit does work to throttle timers and limit the impact of background tabs or obscured content, idle pages may still perform costly work. Likewise foreground pages that are constantly performing script, such as timers or excessive event handling, may drain battery without any obvious indication.

    It is important to select a sufficiently large time range so that a single spike in CPU activity doesn’t dominate the average. Page load for example will briefly use very high CPU activity as the initial cost to download and render the page is high. A short time range selection during page load, or during a period of user interaction with the page, may show a high Energy Impact but may not drain the battery if the usage is not sustained. Normal browsing scenarios have idle periods between loads while users read or more slowly interact with page content. A selection range of 15 seconds or more will produce a more realistic estimate of battery drain than a short selection.

    Per-Thread Details

    The center of the detail view includes a larger CPU usage graph broken down by thread. The legend shows the different colors for the main thread, worker threads, and other threads. Hover the graph and you get the exact values for the nearest sample. Below the graph is the Main Thread indicator strip. This shows each of the individual samples of main thread activity. Clicking on this strip will take you directly to the event happening at that time in the event’s associated timeline.

    For an even more detailed breakdown by individual threads and thread groups you can expand the lower Threads section. While the main thread and worker threads will be individually graphed, the remaining threads are grouped. WebKit threads include any threads performing work known to WebKit such as Garbage Collection, JIT compilation, bmalloc scavanger, and more. CPU usage on any remaining threads are put into an unclassified group.

    Statistics and Sources

    These sections provide more detailed insight into the activity that occurred on the page within the selected time range.

    The Statistics section enumerates different kinds of activities that occured on the page and their individual frequencies. Activity includes network requests and entry into script via timers, events, or observer callbacks. The values are sorted by frequency so you can immediately see the most commonly occurring activities which may warrant investigation.

    The Sources section pinpoints and aggregates the sources of entries into script. This includes timer installations, event handlers, and observer handlers. Values are again sorted by frequency so you can immediately see the hottest entry points. For timers, we include the function name where the timer was installed. This makes it easy to detect and investigate unexpected timer registrations.

    The two sections work together to make a powerful debugging tool. Categories of script entries can be selected from the Statistics section and the Sources section will filter to show the corresponding script entry points. For example, by selecting requestAnimationFrame the Sources list will immediately filter down to show only the animation timer installations. With one more click you can then jump to the associated code and place breakpoints for further debugging.

    Timeline Import & Export

    Web Inspector now supports importing and exporting timeline recordings. Now when you capture a recording of a performance issue you can export the recording to share with others, attach to a bug report, or just save for later analysis.

    Import and export works across all of the timelines. This makes it useful for all kinds of performance analysis, not just the new CPU timeline. If you captured a recording with an interesting JavaScript and Events profile or one with JavaScript Allocations heap snapshots showing a memory leak, export and import will work great in each of these cases.

    Feedback

    You can try out the new CPU Usage Timeline in the latest Safari Technology Preview. Let us know how it works for you. Stay tuned for upcoming blog posts on best practices for energy efficient web content. Send feedback on Twitter (@webkit, @JosephPecoraro) or by filing a bug.

    June 06, 2019 02:00 PM

    June 05, 2019

    Safari Technology Preview 84, with Safari 13 Features, is Now Available

    Surfin’ Safari

    Safari Technology Preview Release 84 is now available for download for macOS Mojave. With this release, Safari Technology Preview is now available for betas of macOS Catalina. If you already have Safari Technology Preview installed, you can update from the Software Update pane of System Preferences.

    This release covers the same revisions of WebKit from Safari Technology Preview 83, but includes new Safari and WebKit features that will be present in Safari 13. The following Safari 13 features are new to Safari Technology Preview 84:

    Refreshed Favorites Design. The Favorites page has been visually refreshed, and now includes Show More and Show Less actions.

    Switch to Tab from Smart Search Field. The Smart Search Field now offers switching to an already-open tab when a search query matches the title or URL of an open tab.

    Warnings for Weak Passwords. When signing into a website with a weak password, Safari will prompt you to visit the website in a new tab to upgrade the password to an Automatic Strong Password. Safari uses the well-known URL for changing passwords (/.well-known/change-password), allowing websites to take users directly to their change password pages. The password list in Safari Preferences has also been updated to flag weak passwords.

    Many more WebKit features in Safari 13 are present in this release of Safari Technology Preview and have been in past releases. You can read more about these changes in What’s New in Safari 13 Beta.

    June 05, 2019 05:00 PM

    Creating Web Inspector Audits

    Surfin’ Safari

    This post is a followup to the Audits in Web Inspector post, and explains the capabilities of and how to write an audit.

    Test Case Format

    The actual test that is run in the page is just a stringified JavaScript function. async functions are allowed, as well as non-async functions that return a Promise. The only requirement is that the return value of the function (or Promise) conforms to the following rules:

    • Returning true/false will translate to a Pass/Fail result with no additional data.
    • Returning a string value will translate to the corresponding result (e.g. "pass" is a Pass result) with no additional data.
    • Returning an object gives the most flexibility, as it allows for additional data other than the audit’s result to be displayed in the Audit tab. Result levels are first retrieved from the "level" value (if it exists), and are translated in the same way as if a string was returned (e.g. "pass" is a Pass result). Alternatively, using any result string as a key with a value of true will have the same effect (e.g. "pass": true).

    There are five result levels:

    • "pass" corresponds to a Pass result, which is where everything was as it should be.
    • "warning" corresponds to a Warning result, which is a “soft pass” in that there was nothing wrong, but there were things that should be changed.
    • "fail" corresponds to a Fail result, which is an indication that something is not as it should be.
    • "error" corresponds to an Error result, which occurs when the JavaScript being run threw an error.
    • "unsupported" corresponds to an Unsupported result, which is a special case that can be used to indicate when the data being tested isn’t supported by the current page (e.g. missing some API).

    There are also three additional pieces of data that can be returned within the result object and have a dedicated specialized interface:

    • domNodes, which is an array of DOM nodes that will be displayed in the Audit tab much the same as if they were logged to the console.
    • domAttributes, which is an array of strings, each of which will be highlighted if present on any DOM nodes within domNodes.
    • errors, which is an array of Error objects and can be used as a way of exposing errors encountered while running the audit.
      • If this array has any values, the result of the audit is automatically changed to Error.
      • If an error is thrown while running an audit, it will automatically be added to this list.

    For custom data, you can add it to the result object and it will display in the Audit tab, so long as it’s JSON serializable and doesn’t overlap with any of the items above.

    Container Structure

    Web Inspector Audits follow a simple and highly flexible structure in the form of JSON objects. These objects fall into two main categories: tests and groups.

    The format for a test is as follows:

    {
        "type": "test-case",
        "name": "...",
        "test": "<stringified JavaScript function>"
    }
    

    The format for a group is as follows:

    {
        "type": "test-group",
        "name": "...",
        "tests": [...]
    }
    

    In this case, the values inside tests can be both individual test cases, or additional groups.

    Both tests and groups also support a number of optional properties:

    • description is a basic string value that is displayed in the Audit tab as a way of providing more information about that specific audit, such as what it does or what it’s trying to test.
    • supports can be used as an alternative to feature-checking, in that it prevents the audit from even being run unless the number value matches Web Inspector’s Audit version number, which can be found at the bottom of the Web Inspector window when in edit mode in the Audit tab. At the time of writing this post, the current version is 3.
    • setup is similar to a test case’s test value, except that it only has an effect when supplied for a top-level audit. The idea behind this value is to be able to share code between all audits in a group, as it is executed before the first audit in a group.

    Specially Exposed Data

    Since audits are run from Web Inspector, it’s possible for additional information to be exposed to each test function being executed. Much of this data is already exposed to Web Inspector, but was never accessible via JavaScript in any way.

    The information is exposed via a WebInspectorAudit object that is passed to each test function. Note that this object is shared between all test under a given top-level audit, which is defined as an audit with no parent. As such, attaching data to this object to be shared between test is accepted and encouraged.

    Version

    Accessing Web Inspector’s Audit version number from within a test is as simple as getting the value of WebInspectorAudit.Version.

    Resources

    The following all relate to dealing with resources loaded by the page, and are held by WebInspectorAudit.Resources.

    • getResources() will return a list of objects, each corresponding to a specific resource loaded by the inspected page, identified by a string url, string mimeType, and audit-specific id.
    • getResourceContent(id) will return an object with the string data and boolean base64Encoded contents of the resource with the given audit-specific id.

    DOM

    The following all relate to dealing with the DOM tree, and are held by WebInspectorAudit.Resources.

    • hasEventListeners(node[, type]) returns true/false depending on whether the given DOM node has any event listeners, or has an event listener for the given type (if specified).

    Accessibility

    The following all relate to dealing with the accessibility tree, and are held by WebInspectorAudit.Accessibility. Further information can be found in the WAI-ARIA specification.

    • getElementsByComputedRole(role[, container]) returns an array of DOM nodes that match the given role that are children of the container DOM node (if specified) or the main document.
    • getActiveDescendant(node) returns the active descendant of the given DOM node.
    • getMouseEventNode(node) returns the DOM node that would handle mouse events which is or is a child of the given DOM node.
    • getParentNode(node) returns the parent DOM node of the given DOM node in the accessibility tree.
    • getChildNodes(node) returns an array of DOM nodes that are children of the given DOM node in the accessibility tree.
    • getSelectedChildNodes(node) returns an array of currently selected DOM nodes that are children of the given DOM node in the accessibility tree.
    • getControlledNodes(node) returns an array of DOM nodes that are controlled by the given DOM node.
    • getFlowedNodes(node) returns an array of DOM nodes that are flowed to from the given DOM node.
    • getOwnedNodes(node) returns an array of DOM nodes that are owned by the given DOM node.
    • getComputedProperties(node) returns an object that contains various accessibility properties for the given DOM node. Since HTML allows for “incorrect” values in markup (e.g. an invalid value for an attribute), the following properties use the computed value determined by WebKit:
      • busy is a boolean related to the aria-busy attribute.
      • checked is a string related to the aria-checked attribute.
      • currentState is a string related to the aria-current attribute.
      • disabled is a boolean related to the aria-disabled attribute.
      • expanded is a boolean related to the aria-expanded attribute.
      • focused is a boolean that indicates whether the given node is focused.
      • headingLevel is a number related to the aria-level attribute and the various HTML heading elements.
      • hidden is a boolean related to the aria-hidden attribute.
      • hierarchicalLevel is a number related to the aria-level attribute.
      • ignored is a boolean that indicates whether the given node is currently being ignored in the accessibility tree.
      • ignoredByDefault is a boolean that indicates whether the given node is always ignored by the accessibility tree.
      • invaludStatus is a string related to the aria-invalid attribute.
      • isPopUpButton is a boolean related to the aria-haspopup attribute.
      • label is a string related to the aria-label attribute
      • liveRegionAtomic is a boolean related to the aria-atomic attribute.
      • liveRegionRelevant is an array of strings related to the aria-relevant attribute.
      • liveRegionStatus is a string related to the aria-live attribute.
      • pressed is a boolean related to the aria-pressed attribute.
      • readonly is a boolean related to the aria-readonly attribute.
      • required is a boolean related to the aria-required attribute.
      • role is a string related to the role attribute.
      • selected is a boolean related to the aria-selected attribute.

    Getting Started

    As mentioned in the Audits in Web Inspector post, the Demo Audit and Accessibility audits that are included as part of Web Inspector can be used as good starter templates for the formatting and structure of a Web Inspector audit.

    Alternatively, the eslint.json audit, which runs ESLint against every *.js resource that was loaded from the main origin of the inspected page, can serve as a more complex example of an async audit that makes use of some of the above specially exposed data.

    Feedback

    The Audit tab was added in Safari Technology Preview 75. Is there a workflow that isn’t supported, or a piece of data that isn’t exposed, that you’d like to use when writing/running audits? Do you have an idea for a default audit that should be included in Web Inspector? Let us know! Please feel free to send us feedback on Twitter (@dcrousso or @jonathandavis) or by filing a bug.

    June 05, 2019 02:41 PM

    Audits in Web Inspector

    Surfin’ Safari

    Often when creating web pages, there are particular rules or guidelines that are set by the site author, or by an included library/framework. In those cases, it can be tedious to find any cases where those rules/guidelines may be broken, assuming one even has the inclination to do so. As a simple example, remembering to set a title or alt on <img> is often forgotten, but it’s important to include them as it makes it easier for accessibility tools to understand and/or navigate the page.

    For those repeated cases, where one wants to check if the page conforms to an expectation, we’ve created a new tool, which we call Web Inspector Audits, that can help simplify that process.

    Audit Tab

    Web Inspector Audits can be found in the new Audit tab, represented as a tree of audits, each of which can be run against the inspected page. An audit can be either a group of other audits (e.g. a test group), or an individual test case.

    Each audit can have one of five different result levels, each denoted by a different icon.

    • Passed represents a result where everything was as it should be.
    • Warning is a “soft pass” in that there was nothing wrong, but there were things that should be changed.
    • Failed is an indication that something is not as it should be.
    • An Error result occurs when the JavaScript being run threw an error.
    • Unsupported is a special case that can be used to indicate when the data being tested isn’t supported by the current page (e.g. missing some API).

    Audits are also able to be disabled, to allow for more control as to what is actually run. This is done by clicking the Edit button at the bottom of the navigation sidebar.

    Default Audits

    Web Inspector itself also provides a few audits that are included by default and cannot be deleted (only disabled).

    Demo Audit is a basic exposé of the functionality/interface of audits, and can be used as a starter template for writing your own audits.

    Accessibility includes a handful of test cases that check for DOM accessibility best practices on the inspected page, such as the example mentioned at the beginning of this post, in accordance with the WAI-ARIA specification. The Accessibility audit serves as a great starting point for making sure that your page is accessible.

    Import and Export

    When we first began thinking about the concept of audits, one thing that was forefront in our minds was that both the audit and its result should be portable. This was the primary reason that we chose to use JSON as the format for audits, as well as their results.

    All audits are able to be imported and exported from any Web Inspector, regardless of whether that particular audit is able to be run on the current page (see the Unsupported explanation above). Any imported audits are persistent across Web Inspector sessions, meaning you can import an audit while inspecting one page, open Web Inspector on another page, and be able to run that same audit on the new page without having to re-import. This makes it easy for you to share audits you’ve used (or even written) with other developers out in the world.

    Audit results are similarly importable/exportable, but they are not preserved across sessions. This way, if you run an audit and encounter any non-Passed result, you can export it and share the data with those around you, possibly helping you find the right way to act on it.

    Demo

    The simplest way to see how Web Inspector Audits work is to simply run them on any page.

    As a demonstration of a more advanced usage, eslint.json is an audit that runs ESLint against every *.js resource that was loaded from the main origin of the inspected page. If you’re interested in editing this audit, or even writing your own, please see the upcoming post about creating Web Inspector Audits.

    Feedback

    The Audit tab was added in Safari Technology Preview 75. Is there a workflow that isn’t supported, or a piece of data that isn’t exposed, that you’d like to use when writing/running audits? Do you have an idea for a default audit that should be included in Web Inspector? Let us know! Please feel free to send us feedback on Twitter (@dcrousso or @jonathandavis) or by filing a bug.

    June 05, 2019 02:40 PM

    May 29, 2019

    Release Notes for Safari Technology Preview 83

    Surfin’ Safari

    Safari Technology Preview Release 83 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 245007-245618.

    Web Authentication

    • Enabled WebAuthN by default on macOS (r245589)
    • Changed to cancel the pending request when a new request is made (r245043)
    • Changed to return InvalidStateError to sites whenever authenticators return such error (r245262)

    Pointer Events

    • Fixed isPrimary property of pointercancel events to match previous events for that pointer (r245020)
    • Fixed calling preventDefault() on pointerdown to prevent “compatibility” mouse events (r245585)

    Storage Access API

    • Changed to only consume the user gesture when the user explicitly denies access and made document.hasStorageAccess() return true when the feature is off (r245025)

    Rendering

    • Implemented backing-sharing in compositing layers, allowing overlap layers to paint into the backing store of another layer (r245170)
    • Changed layers painting into shared backing to contribute to overlap (r245502)
    • Changed to repaint when the set of backing-sharing layers changes (r245220)
    • Fixed rendering of backing-sharing layers with transforms (r245205)
    • Fixed layer bounds for sharing layers that paint with transforms (r245208)
    • Fixed layer-related flashing with composited overflow: scroll (r245602)
    • Fixed overflow: scroll that becomes non-scrollable to stop being composited (r245212)
    • Fixed content disappearing when the scroller hosting a shared layer becomes non-scrollable (r245206)
    • Fixed rendering issues when layer contents become non-opaque (r245207)

    Service Workers

    • Changed to terminate a service worker instance when its SWServer is destroyed (r245200)
    • Changed a service worker process to app nap when all of its clients app nap (r245299)

    CSS

    • Implemented line-break: anywhere (r245275)
    • Implemented a modern “clearfix” with display: flow-root (r245494)
    • Implemented page-break-* and -webkit-column-break-* as legacy-shorthands. (r245276)
    • Fixed font-optical-sizing applying the wrong variation value (r245598)
    • Updated CSS grid when changing auto repeat type (r245295)
    • Updated to use max size to compute auto repeat CSS grid tracks (r245279)

    WebRTC

    • Defined a media buffering policy (r245039)
    • Allowed sequential playback of media files when initial playback started with a user gesture (r245467)
    • Fixed video stream freezing when the front camera orientation changes (r245033)

    WebDriver

    • Fixed the in-view center point for elements larger than the viewport (r245320)

    Web API

    • Changed to preserve DOM selection after clicking a button that hides itself on mousedown (r245238)
    • Limited the number of prefetches of a given page (r245171)

    Web Inspector

    • Changed the user gesture toggle to also force a user interaction flag (r245366)
    • Fixed the colors in the network table waterfall container for Dark Mode (r245484)
    • Fixed context menu items in the DOM tree when not clicking directly on the node representation in the Elements tab (r245495)
    • Fixed Storage tab crashes when adding new local storage or session storage entries (r245535)
    • Fixed the CPU timeline and Memory timeline bars sometimes incorrectly drawing and jumping around while scrolling (r245498)
    • Made it easier to switch to a DOM node in the Elements tab from a returned DOM node in the Audit tab (r245497)

    May 29, 2019 05:00 PM

    May 22, 2019

    Privacy Preserving Ad Click Attribution For the Web

    Surfin’ Safari

    A typical website is made of numerous components coming from a wide variety of sources. Many of the sources that make up a website are opaque to the user, and some third-party resources are designed to identify and track users as they browse the web, often in order to retarget ads and measure ad campaign effectiveness.

    The combination of third-party web tracking and ad campaign measurement has led many to conflate web privacy with a web free of advertisements. We think that’s a misunderstanding. Online ads and measurement of their effectiveness do not require Site A, where you clicked an ad, to learn that you purchased something on Site B. The only data needed for measurement is that someone who clicked an ad on Site A made a purchase on Site B.

    Today we are presenting a new technology to allow attribution of ad clicks on the web while preserving user privacy. We used the following principles as we designed this technology:

    • Users should not be uniquely identified across websites for the purposes of ad click attribution. This means the combined data of an ad click and a conversion should not be attributable to a single user at web scale. To achieve this, our design has the following properties:
      • Up to 64 ad campaigns can be measured in parallel per website where ads are placed by an advertiser. This low number means ad campaign IDs cannot be turned into user identifiers.
      • Up to 64 conversion events can be distinguished on the advertiser’s own website. This means conversion IDs are also restricted from being turned into user identifiers.
    • Only websites that users visit should be involved in measuring ad clicks and conversions. This means that opaque third-parties should not receive ad click attribution reports and we enforce it by requiring that the ad link is part of a first-party webpage and by only reporting on which first-party website a conversion happened.
    • The browser should act on behalf of the user and do its best to preserve privacy while reporting on ad click attribution. We achieve this by:
      • Sending attribution reports in a dedicated Private Browsing Mode even though the user is in regular browsing mode.
      • Disallowing data like cookies for reporting purposes.
      • Delaying reports randomly between 24 and 48 hours.
      • Not supporting Privacy Preserving Ad Click Attribution at all when the user is in Private Browsing Mode.
    • The browser vendor should not learn about the user’s ad clicks or conversions. For this reason, we designed the feature to do all of its work on-device. The browser vendor does not see any of the ad click attribution data.

    Critically, our solution avoids placing trust in any of the parties involved — the ad network, the merchant, or any other intermediaries — and dramatically limits the entropy of data passed between them to prevent communication of a tracking identifier.

    Ad Click Attribution in a Nutshell

    Here’s a simple example of ad click attribution:
    An online store runs an ad on a search engine website. If a user clicks the ad and eventually buys something, both the online store and the search engine website where the ad was placed want to know; they want the purchase to be attributed to the ad click so that the store knows where to focus their advertising budget. Such attribution is used for measurement of which ads are effective.

    Traditional, Privacy-Invasive Ad Click Attribution

    Traditionally, ad click attribution has been done through the use of cookies and so-called “tracking pixels.” Here’s an illustration of how this works:

    Privacy invading ad click attribution

    The illustration above shows the user John:

    1. Searching for “grill” on search.example,
    2. Clicking an ad which takes him to shop.example, and
    3. Finally adding a $90 grill to a shopping cart.

    Following each action on shop.example, shop.example fires a tracking pixel (a request for an invisible image) to search.example to report progress toward a purchase.

    In browsers without appropriate privacy protections, search.example will identify John through his cookies every time shop.example fires such a tracking pixel to search.example. This pervasive technology allows search.example to learn everything John does on shop.example and all other websites that fire similar tracking pixels. Even worse, all these pixels fire regardless of whether John has clicked an ad or not.

    Needless to say, tracking pixels that carry cookies enable sites such as search.example to build up a huge profile of people’s interests, purchasing power, habits, age, et cetera. We refer to this as cross-site tracking and Safari prevents it from happening through the WebKit feature Intelligent Tracking Prevention (ITP).

    As more and more browsers acknowledge the problems of cross-site tracking, we should expect privacy-invasive ad click attribution to become a thing of the past.

    Privacy Preserving Ad Click Attribution

    We propose a modern way of doing ad click attribution that doesn’t allow for cross-site tracking of users but does provide a means of measuring the effectiveness of online ads. It is built into the browser itself and runs on-device which means that the browser vendor does not get to see what ads are clicked or which purchases are made.

    Privacy Preserving Ad Click Attribution has three steps:

    1. Store ad clicks. This is done by the page hosting the ad at the time of an ad click.
    2. Match conversions against stored ad clicks. This is done on the website the ad navigated to as a result of the click. Conversions do not have to happen right after a click and do not have to happen on the specific landing page, just the same website.
    3. Send out ad click attribution data. This is done by the browser after a conversion matches an ad click.

    Let’s go through these steps in detail and which steps we’ve taken to preserve the user’s privacy.

    Step 1: Store Ad Clicks

    Anchor elements, often referred to as links, now support two new, optional attributes called adDestination and adCampaignID.

    As shown in the illustration below, adDestination is the domain the ad click is navigating the user to, and adCampaignID is the identifier of the ad campaign.

    Store ad clicks

    If the user clicks the ad link on search.example, the browser will follow the navigation, through potential redirects, to make sure that the user actually lands on shop.example. If so, the browser stores the ad click, comprising the following data (presented here in plain English): The user clicked shop.example’s ad campaign 55 on search.example.

    Here are the important privacy aspects of this step:

    • The link needs to be an element on the first-party website (the main frame), not a link in an iframe. This is to meet user expectations and to be able to provide control to the user. Users can only be expected to understand which first-party website they clicked an ad on and which first-party website they made a purchase on. We also think it’s important that the first-party website that serves the ad is the one attributed for the performance of the ad campaign.
    • Neither search.example nor shop.example can read the stored ad click data or detect that it exists.
    • The browser only stores ad clicks for a limited time. In WebKit’s implementation that is seven days.
    • The entropy of the ad campaign ID needs to be properly restricted to not become a cross-site tracking vector. WebKit’s implementation allows a value between 0 and 63, i.e. a maximum of 64 shop.example ad campaigns running in parallel on search.example.

    Step 2: Match Conversions Against Stored Ad Clicks

    To achieve ad click attribution, the browser needs to be able to match conversions with stored ad clicks. What are conversions?

    • Adding an item to the shopping cart is a conversion.
    • Signing up for a new service is a conversion.
    • Entering shipping or payment information is a conversion.
    • Pulling the trigger and actually buying something is a conversion.

    Matching conversions to ad clicks allows shop.example to understand that a specific ad campaign may be effective in getting customers to add items to their shopping carts but something in the checkout flow throws them off.

    How does Privacy Preserving Ad Click Attribution detect a conversion and match it with a stored ad click? It makes use of the legacy tracking pixels!

    Match conversions ad clicks

    In the illustration above, an existing request to the existing tracking pixel is redirected by search.example on its own server infrastructure to a well-known location in order to signal to the browser that this is in fact a conversion happening. Note that privacy protections such as ITP will typically make sure that no cookies are sent in this request.

    The path parameter “20” at the end of the well-known location is the conversion data. This gives shop.example an opportunity to say something about the conversion such as where in the sales funnel the customer is, what the value of the conversion is, what time of day it is, or whatever they decide is relevant for them.

    The redirect to the well-known location may also include an optional priority parameter which indicates the importance of this particular conversion in the case of multiple conversions matching the same stored ad click.

    Here are the important privacy aspects of this step:

    • Neither search.example nor shop.example know whether there is any stored ad click data to be matched against.
    • Neither search.example nor shop.example are told by the browser whether there was a match or not.
    • The entropy of the ad conversion data needs to be properly restricted to not become a cross-site tracking vector. WebKit’s implementation allows a value between 0 and 63, i.e. six bits to distinguish conversion events. As mentioned earlier, shop.example decides what goes into these bits. For instance, they may spend two bits on monetary value in four buckets: {less than $10, between $10 and $50, between $51 and $200, above $200}.

    We expect to also implement a JavaScript API to send this information to the .well-known location to remove the requirement for tracking pixels but we’d like to openly discuss what should go into that API since it is much more forward looking than retrofitting existing tracking pixels.

    Step 3: Send Out Ad Click Attribution Data

    Now we come to the third and final step — the browser reports that a conversion happened for a user that had previously clicked an ad.

    Send Ad Click Attribution Data

    Once the browser has matched a conversion against a stored ad click, it sets a timer, randomized between 24 and 48 hours. When that timer fires, the browser makes an ephemeral, stateless POST request to the same well-known location. In our example, the request would go to https://search.example/.well-known/ad-click-attribution/20/55, with the referrer request header set to https://shop.example.

    In plain English this report would say: 24 to 48 hours ago, some user who previously clicked shop.example’s ad campaign 55 on search.example, converted with data 20 on shop.example.

    Once the ephemeral, stateless POST request goes out, the stored ad click is consumed and cannot be converted further. This is in part why we have the minimum delay of 24 hours. During that delay, shop.example has the opportunity to signal further conversions, for instance down a sales funnel, and only the most important conversion will be sent in the POST request. The importance is controlled through the optional priority parameter in the conversion redirect, as mentioned above.

    Here are the important privacy aspects of this step:

    • Neither search.example nor shop.example know that an attribution request has been scheduled.
    • The 24–48 hour delay makes sure a conversion that happens shortly after an ad click will not allow for speculative profiling of the user by search.example. The randomness in the delay makes sure that the request does not in itself reveal when during the day the conversion happened. If shop.example wants time of day data, they will have to spend some of their six bits of conversion data on it.
    • The ephemeral, stateless request makes sure the request is not associated with state built up through other browsing. Ephemeral in this sense is referred to as Private Browsing in Safari.
    • The well-known location allows for a simple rule if Content Blockers wants to block such conversion reporting.

    Privacy Considerations

    For ad click attribution to happen, some bits of data about what happened across two websites need to be sent. Today’s practice of ad click attribution has no practical limit on the bits of data, which allows for full cross-site tracking of users using cookies. This is privacy invasive and thus we are obliged to prevent such ad click attribution from happening in Safari and WebKit.

    But by keeping the entropy of attribution data low enough, we believe the reporting can be done in a privacy preserving way.

    Here is a summary of our privacy considerations for Privacy Preserving Ad Click Attribution:

    • Only links served on first-party pages should be able to store ad click attribution data. This ensures that users have a chance of understanding how Privacy Preserving Ad Click Attribution works.
    • Neither the website where the ad click happens nor the website where the conversion happens should be able to see whether ad click data has been stored, has been matched, or is scheduled for reporting.
    • Ad clicks should only be stored for a limited time, such as a week. Users cannot be expected to understand that a purchase they make today is attributed to an ad click they made months ago.
    • The entropy of both ad campaign ID and conversion data needs to be restricted to a point where this data cannot be repurposed for cross-site tracking of users. We propose six bits each for these two pieces of data, or values between 0 and 63.
    • Ad click attribution requests should be delayed randomly between 24 to 48 hours. This makes sure that a conversion that happens shortly after an ad click will not allow for speculative cross-site profiling of the user. The randomness in the delay makes sure the request does not in itself reveal when during the day the conversion happened.
    • The browser should not guarantee any specific order in which multiple ad click attribution requests are sent, since the order itself could be abused to increase the entropy and allow for cross-site tracking of users.
    • The browser should use an ephemeral session (a.k.a. private or incognito mode) to make ad click attribution requests.
    • The browser should not use or accept any credentials such as cookies, client certificates, or Basic Authentication in ad click attribution requests or responses.
    • The browser should offer a way to turn ad click attribution on and off. We intend to have the default setting to be on to encourage websites to move to this technology and abandon general cross-site tracking.
    • The browser should not enable ad click attribution in private/incognito mode.

    Try It Out In Safari Technology Preview!

    We’re happy to offer Privacy Preserving Ad Click Attribution as an experimental feature in Safari Technology Preview 82+.

    First, enable the Develop menu, then go to the Experimental Features submenu.

    Experimental Features menu

    There you’ll find “Ad Click Attribution” which enables the feature itself, and “Ad Click Attribution Debug Mode” which enables debug logging for developers and shortens the 24–48 hour delay to a static one minute delay, also for use by developers.

    Debugging the Link Attributes

    A cross-site anchor element that wants to push ad click attribution data into the browser looks like this:
    <a href="https://some.site.example" addestination="https://shop.example" adcampaignid="55">

    To debug such elements, you use the Web Inspector’s console with the “Preserve Log” setting enabled. Here are a few examples of console warnings you may see if there’s something wrong with your attribution attributes:

    Both adcampaignid and addestination need to be set for Ad Click Attribution to work.

    Ad Click Attribution is only supported in the main frame.

    This tells you the anchor element is not part of the main frame.

    addestination can not be the same site as the current website. This technology is meant for cross-site attribution of ad clicks. There is no need for it within the same website.

    Debugging Storage of Ad Clicks

    For debugging anything beyond the anchor element, you need to use the system log (syslog). Here’s how you achieve that:

    1. Enable Ad Click Attribution Debug Mode in the Develop–>Experimental Features submenu.
    2. In your macOS Terminal, run: log stream -info | grep AdClickAttribution.

    Now if you click a cross-site element with adDestination and adCampaignID attributes, you should expect to see the following in your syslog:
    Storing an ad click.

    Debugging Conversions

    A conversion is signaled through a same-site HTTP redirect to /.well-known/ad-click-attribution/[a decimal value between 0 and 63 representing the conversion data]. Same-site here means search.example needs to be the server redirecting to https://search.example/.well-known/ad-click-attribution/. The reason for this is that search.example should be in control of when stored ad clicks on its site are consumed. Note that the conversion redirect is done as a subresource on shop.example so we don’t mean same-site as the main frame.

    Once you do such a redirect, the syslog might feature one of the following error messages:

    Conversion was not accepted because the HTTP redirect was not same-site. This is the requirement mentioned above, i.e. it has to be search.example redirecting to search.example/.well-known/ad-click-attribution/.

    Conversion was not accepted because it was requested in an HTTP redirect that is same-site as the first-party. Again, this technology is meant for cross-site attribution of ad clicks. There is no need for it within the same website.

    Conversion was not accepted because the URL's protocol is not HTTPS or the URL contains one or more of username, password, query string, and fragment. The request to the well-known location has to be HTTPS and cannot contain a username, password, query string, or fragment.

    Conversion was not accepted because the URL path did not start with /.well-known/ad-click-attribution/.

    Conversion was not accepted because the conversion data could not be parsed or was higher than the allowed maximum of 63.

    Conversion was not accepted because the URL path contained unrecognized parts. This is a catch-all error message for when the URL has unrecognized path elements or is not of the correct length.

    Detecting Successful Conversions

    If you got everything right in the redirect to the well-known location, you should see the following message in the syslog:
    Got a conversion with conversion data: 20 and priority: 0.

    Here you see the priority parameter. It is a way for the server to signal how important a particular conversion is so that the browser can report the most important one. Take the sales funnel example. There, multiple conversions will happen in succession: add to shopping cart, enter shipping info, enter payment info, and finalize purchase. Most likely, the finalized purchase is the conversion that should be reported together with the ad campaign ID. Priority can be 0 to 63, higher means higher priority, and the priority value is only used for internal bookkeeping, i.e. not sent in any request.

    Henceforth, lets assume the redirect is done with conversion data 20 and priority 12, like so:
    https://search.example/.well-known/ad-click-attribution/20/12

    Now, if there’s a stored ad click that matches this conversion, you’ll see detailed conversion information in the syslog:

    Converted a stored ad click with conversion data: 20 and priority: 12. This is when a previously unconverted ad click is converted.

    Re-converted an ad click with a new one with conversion data: 20 and priority: 12 because it had higher priority. This is for when there’s a conversion of higher priority that matches an already scheduled conversion request. The ad click is kept, re-converted with the high priority conversion, and scheduled for reporting.

    Replaced a previously converted ad click with a new one with conversion data: 20 and priority: 12 because it had higher priority. This is for when there is a different ad click (the user may have clicked more than one ad) with a scheduled conversion request but with lower priority. The newly converted ad click with higher priority replaces the old one.

    Finally, you’ll see the scheduling of the report request in the syslog:
    Setting timer for firing conversion requests to the debug mode timeout of 60 seconds where the regular timeout would have been 111003 seconds. This is special-cased for Ad Click Attribution Debug Mode. Instead of the 24 to 48 hour delay, there’s only a 60 second delay. The log message tells what the real delay would have been, in this case ≈31 hours.

    Receiving Conversion Reports

    When the scheduled timer fires, an HTTP POST request is made to ./well-known/ad-click-attribution/[conversion data]/[ad campaign ID], effectively reporting that a conversion happened for a user that previously clicked an associated advertisement. In our example, this request would go to:
    https://search.example/.well-known/ad-click-attribution/20/55
    … with the referrer request header set to:
    https://shop.example/

    When this request is about to go out, you’ll see the following syslog entry:
    About to fire an attribution request for a conversion.

    If something went wrong with the request, you’ll see it in the syslog:
    Received error: [error message] for ad click attribution request.

    Where To Send Feedback and Bug Reports

    Privacy Preserving Ad Click Attribution is in the early stage of being proposed as a standard through the W3C Web Platform Incubator Community Group (WICG). Please join the discussion and file issues to discuss how this technology fits with your use cases.

    If you find that the experimental feature in Safari Technology Preview doesn’t work as explained, please file a WebKit bug at https://bugs.webkit.org and CC John Wilander.

    For technical inquiries on Privacy Preserving Ad Click Attribution, you’ll find me on Twitter: @johnwilander

    May 22, 2019 01:00 PM

    May 15, 2019

    Debugging Media in Web Inspector

    Surfin’ Safari

    Media elements, such as <video> and <audio>, can sometimes be tricky to debug due to the sheer flexibility of the available functionality. Content can be loaded in a variety of different ways, such as a direct stream from a single file, byte-range requests from a single file, or multipart streams from different sources. In the past it was difficult to reconcile how content made the journey to being displayed on a page, as well as how it interacted with the other content around it. In macOS Mojave 10.14.4 we’ve added special functionality that makes it easier to understand how media content relates to network and JavaScript activity, specifically how that activity can stall or even prevent playback.

    Correlating Media Events and Network Activity

    Sometimes media playback may become stalled due to network activity, such as if a byte range request takes too long to load. Previously there was no easy way to identify these issues solely from Web Inspector. Now, however, Web Inspector provides an option called “Group Media Requests” that organizes network activity based on the related top-level media element. This makes it easier to keep track of related resources, such as byte range requests or items that are part of a media playlist.

    Since media elements aren’t resources it doesn’t make sense to have the usual “waterfall”. Instead, we’ve created a special timeline that keeps a log of playback related events fired by the media element, such as loadstart or stalled, as well as some basic state about the media element, such as whether it’s paused. A dashed line represents “paused”, a solid line represents “playing”, and no line represents “stopped”. If a <video> element enters fullscreen a grey area is shown indicating the time in which that <video> remained in fullscreen mode. On macOS <video> elements can also be rendered in a special way, called Power Efficient Video Playback, which is indicated by a green area.

    Identifying Media Issues Caused by JavaScript

    Sometimes, however, issues may arise due to incorrect JavaScript usage or layout issues. For these cases, we’ve also added the same event information to a new timeline in the Timelines tab called Media, as well as in the general timeline overview.

    Similar to the Network tab, correlating media events with nearby script activity can be a quick way to diagnose issues in situations where JavaScript causes the wrong activity to occur. In addition to this dedicated Media timeline all media events are also shown in the general timeline overview.

    Diagnosing issues with Power Efficient Video Playback

    Whenever a <video> element is about to render a frame on macOS, it makes a decision about whether to use the computer’s CPU/GPU or the display itself to composite the frame. In cases where the display can be used for frame compositing, it is often more power efficient. If your website contains full-screen video content, you should strive to follow these guidelines to ensure that your <video> is able to use display compositing as often as possible.

    • The <video> must be playing.
    • The <video> must be full-screen.
    • The <video> must not be occluded in any way (e.g. not showing controls or any content “above” it).

    Keep in mind that these are ways in which the web content itself can prevent power efficient video playback. There are other reasons that aren’t controllable from web content as to why a video may not enter a power efficient playback state, such as being connected to an external display or the device itself not supporting it.

    WebDriver

    Starting in macOS Mojave 10.14.4, WebDriver is able to access the total count of display composited frames which can be used for continuous integration testing.

    let video = document.querySelector("video");
    let videoPlaybackQuality = video.getVideoPlaybackQuality();
    videoPlaybackQuality.displayCompositedVideoFrames
    

    Each time a frame is display composited, that count will increment, so if the count does not increment, the <video> is not using display compositing.

    If you develop websites where the main content is a <video>, consider adopting these guidelines to provide a better experience for your users.

    Feedback

    These enhancements are available starting in macOS Mojave 10.14.4 and iOS 12.2. Is there additional information you’d like us to visualize with respect to media elements? Are there other elements that fit into this “category” for which you’d like to see similar instrumentation? Let us know! Please feel free to send us feedback on Twitter (@dcrousso or @jonathandavis) or by filing a bug.

    May 15, 2019 06:00 PM

    Release Notes for Safari Technology Preview 82

    Surfin’ Safari

    Safari Technology Preview Release 82 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 244684-245007.

    Web API

    • Enabled Fetch Keep Alive by default (r244931)
    • Changed to resolve the percentage values of inset properties against proper box. (r244906)
    • Changed to use the normal loading path for ping loads (r244700)
    • Fixed propertyRegistry() to be overridden for SVGFEFloodElement and SVGFEMergeElement (r244690)
    • Fixed programmatic scrolling with async overflow scroll (r244947)
    • Marked U+01C0 as a homograph of U+006C (r244903)
    • Updated WebKit to rely more on ICU’s UTF-8 support (r244828)

    JavaScript

    • Fixed crash when opening the inspector when using already in-flight generators and async functions (r244915)
    • Changed setting a frame src attribute to not run a JavaScript URL synchronously (r244892)

    Media

    • Changed WebAudio Node JS wrappers to not be collected if events can be fired (r244977)
    • Updated getDisplayMedia to require a user gesture (r244749)
    • Updated XMLHttpRequest to propagate user gestures for media playback (r244823)

    Storage

    • Made a change to stop IndexedDB transactions to release locked database files when network process is ready to suspend (r244687)
    • Updated to terminate Service Worker processes that use too much CPU (r244979)
    • Updated to terminate unresponsive Service Worker processes (r244936)

    CSS

    • Fixed font-weight: 1000 to be parsed successfully (r244817)
    • Fixed transform sometimes being left in a bad state after an animation (r244800)

    Accessibility

    • Fixed the hit-point of a link that spans multiple lines (r244983)

    Web Inspector

    • Updated support for dragging a node to the console to log the node (r244946)
    • Added support for recording actions performed on WebGL 2 contexts (r244908)

    Web Driver

    • Used a more informative key to indicate automation availability (r244791)

    Web Authentication

    • Added a check to require a call to WebAuthN from a focused document (r244938)

    WebGPU

    • Moved gpu API entry point from window to navigator (r244777)

    Browser Changes

    • Changed unchecking “Allow websites to ask for permission to send notifications” to also disable prompting for HTML5 notifications

    May 15, 2019 05:00 PM

    May 10, 2019

    Dark Mode in Web Inspector

    Surfin’ Safari

    Web Inspector on macOS Mojave now supports Dark Mode.

    Web Inspector: Light/Dark color schemes

    Dark Mode in Web Inspector was introduced in Safari Technology Preview last year. This article highlights implementation details which could be helpful for anyone adapting Dark Mode for their websites or web views.


    How do I enable Dark Mode for Web Inspector?

    Web Inspector matches macOS appearance. To enable Dark Mode in macOS Mojave, go to “System Preferences → General” and select “Appearance: Dark” (see How to use Dark Mode on your Mac for more details). Note that it’s not possible to enable Dark Mode only for Web Inspector.

    Dark Mode in Web Inspector is supported in macOS Mojave and later.

    Why Dark Mode isn’t supported on macOS 10.13 High Sierra and below?
    Web Inspector uses native form controls. macOS Mojave has a dark version of each form control (text fields, buttons, checkboxes, drop-downs, radio buttons, and etc.).

    Dark form controls on dark background and light form controls on light background in macOS Mojave

    macOS 10.13 High Sierra and below don’t support Dark Mode and only have light version of form controls. Without the proper form control styles, Dark Mode experience would be inconsistent so we decided not to do it.


    Resources tab: light/dark color schemes

    We tried to preserve syntax highlighting colors of the light theme. To keep text readable, we increased luminosity — by increasing brightness and sometimes changing the hue.

    Network tab: light/dark color schemes

    For some cases, like the bars in Network tab, we didn’t have to change anything. The same colors are readable in both light and dark themes.

    Console tab

    SVG and currentColor

    Instead of maintaining a separate set of icons for Dark Mode, we changed the colors of the existing icons with CSS wherever possible.

    SVG Icons

    Most of Web Inspector icons are SVG files included inline in HTML:

    <svg style="color: blue">
        <use xlink:href="Images/Paint.svg#root"/>
    </svg>
    

    Paint.svg:

    <svg>
        <path fill="currentColor" d="..."/>
    </svg>
    

    currentColor is the color of the parent element. In this example, it’s blue.

    currentColor only works for inline SVGs. It doesn’t work for SVGs included via <img> element or CSS (e.g. via background-image or content). For these cases, we used CSS filters to invert the colors.

    CSS filters

    The very first iteration of Dark Mode in Web Inspector was a single CSS rule:

    body {
      filter: invert()
    }
    

    Inverting colors rotated the hue by 180 degrees. Notice how the selected color changed from blue to orange. We fixed that by rotating the hue another 180 degrees:

    body {
       filter: invert() hue-rotate(180deg)
    }
    

    This had several issues. The colors didn’t exactly match the Dark Mode Human Interface Guidelines. The dark shadows turned light.

    We ended up not using this filter for body, but we used it for icons included via CSS (e.g. via background-image or content), where currentColor didn’t work.

    CSS variables

    Implementing Dark Mode took over 1,000 lines of CSS. We used CSS variables extensively to reduce code duplication.

    Previously, we had variable names such as --text-color-dark-gray and --text-color-light-gray. In Dark Mode, they would have to be inverted to provide desired contrast with the background: --text-color-dark-gray would become light gray, and --text-color-light-gray would become dark gray.

    We started using semantic names such as --text-color-secondary instead of --text-color-dark-gray, and --text-color-tertiary instead of --text-color-light-gray.

    Contributing

    Please report bugs regarding Dark Mode in Web Inspector on webkit.org/new-inspector-bug. Make sure to include “Dark Mode” in the title.

    If you’re interested in contributing or have any questions, please stop by the #webkit-inspector IRC channel.

    May 10, 2019 05:00 PM

    May 07, 2019

    Dark Mode Support in WebKit

    Surfin’ Safari

    With the introduction of Dark Mode in macOS Mojave last year, web developers have been asking for support in Safari to style web content that matches the system appearance. With the Safari 12.1 update in macOS 10.14.4, dark mode support in WebKit has arrived.

    WebKit.org Getting Started page shown in light modeWebKit.org Getting Started page shown in dark mode

    Dark Mode Behaviors

    By default there are no behavior changes with how pages look when a user is in dark mode. This preserves the standard assumptions web designers have had for almost thirty years — that a page defaults to a white background and black text. To change these defaults in dark mode would be a web compatibility nightmare.

    However, this leaves a large area of the screen with potentially bright content in dark mode. For simple content, an app could transform colors in the document for dark mode. This is what Mail does in macOS Mojave — it displays simple email messages with a dark mode interpretation.

    Not all web content is simple. For this reason Safari and WebKit do not auto-darken web content — documents will need to opt-in to dark mode. The main way to signal that your content supports dark mode it to adopt the new color-scheme style property, specified in this proposal.

    You would use this inherited property in your stylesheet like this:

    :root {
        color-scheme: light dark;
    }
    

    Specifying the values light and dark on the root element lets the engine know both modes are supported by the document. This changes the default text and background colors of the page to match the current system appearance. Also standard form controls, scrollbars, and other named system colors change their look automatically. Without this declaration, it would not be safe for the engine to use dark form controls or a dark color scheme, since many documents are designed with an assumed light color scheme.

    For example, this simple page, with color-scheme: light dark specified, will be entirely ready to use in both appearances.

    Example page in light modeExample page in dark mode

    These default background and text colors can be different values when using the high contrast accessibility mode or with different browser and system versions. You should not assume they will always be the same color values.

    The color-scheme style property is also supported on style rules that select specific elements in the document, not just the root element. This is useful if you have content or components that need a specific color scheme, specifically content with form controls, since dark mode form controls would not look correct on light backgrounds.

    There is also <meta name="color-scheme">, specified in this proposal. This meta tag declaration allows apps like Mail to know early on in parsing which color scheme to use for the document, as it can affect the auto-darkening transformations that get applied to the document’s colors. Declaring <meta name="color-scheme" value="light dark"> in a rich email message lets Mail know it supports its own styling for dark mode. You can also specify "light only", informing Mail that it should not transform your light color scheme email message.

    Note: The color-scheme property and meta tag name were renamed from supported-color-schemes in Safari Technology Preview 81. The old name is supported by WebKit, Safari, and Mail in macOS 10.14.4. This is a developing standard that could change more in the future.

    Styling For Dark Mode

    Defining color-scheme will get you going for simple content. For most web content, you will need to adopt the prefers-color-scheme media query, specified in this proposal, to style elements with custom colors or images. You can use this media query anywhere media queries are supported, such as in <picture> elements or window.matchMedia() for script triggers.

    The best way to deploy a dark and light color scheme in your documents is to utilize CSS variables. Then you can easily specify the colors in one place with the media query, and use those variables throughout your stylesheets. When the media query matches, the variables will change wherever they are used — auto switching with any appearance change.

    Here is an example using the media query to specify color values with CSS variables:

    :root {
        color-scheme: light dark;
        --special-text-color: hsla(60, 100%, 50%, 0.5);
        --border-color: black;
    }
    
    @media (prefers-color-scheme: dark) {
        :root {
            --special-text-color: hsla(60, 50%, 70%, 0.75);
            --border-color: white;
        }
    }
    
    .special {
        color: var(--special-text-color);
        border: 1px solid var(--border-color);
    }
    

    Images and Dark Mode

    For most hero images, they will still look great in dark mode — they might even pop off the page more than ever! As mentioned before, you can use the prefers-color-schemes media query in <picture> elements or style rules to select a different image if you have a dark mode version.

    For our simple example page, we can select a different hero image of the Mojave desert:

    <picture>
        <source srcset="mojave-night.jpg" media="(prefers-color-scheme: dark)">
        <img src="mojave-day.jpg">
    </picture>
    
    Example page in light modeExample page in dark mode

    You might also have images for controls or other interactive elements that you will need to update for dark mode. Many sites today have adopted vector images for these glyphs, such as SVG or web fonts. Fonts will naturally adapt to your dark mode text colors, but SVG images will need some additional styling to look correct. One trick you can use with SVG is the currentColor keyword to have embedded SVGs use the current text color — thus automatically adapting to your dark mode text colors. You can also use CSS variables to style colors inside of SVG, just like your other elements.

    For specific images that you can’t style any other way, you can use a CSS invert() filter. If the image also has color, you might consider using invert() hue-rotate(180deg) to moderately preserve the image intent. In any case you should not use an overall filter on broad sections of content, since that can have high memory and performance costs. Using direct styling of colors is the best approach for color accuracy and performance.

    Debugging Dark Mode

    Web Inspector now includes a navigation bar button in the Elements tab to toggle dark or light mode, depending on your current overall system preference. This allows you to quickly test pages in both appearances without having to toggle the whole system appearance in System Preferences. The Styles sidebar updates to reflect the current matching rules, as well as any CSS color variables.

    Dark Mode Web Inspector Toggle

    Examples

    As you might have noticed, if you are using Safari on macOS 10.14.4, the WebKit.org site already supports dark mode. Since we added support a while ago in Safari Technology Preview, a number of other sites have also adopted dark mode. Here are some examples you can check out today:

    • Twitter — The social networking service.
    • Emojipedia — Home of emoji meanings.
    • NSHipster — A journal of the overlooked bits in Objective-C, Swift, and Cocoa.
    • CSS DB — A database of staged CSS features.
    • Stuff & Nonsense — A design studio, specializing in creative design for digital products and web.

    May 07, 2019 05:00 PM

    May 01, 2019

    Release Notes for Safari Technology Preview 81

    Surfin’ Safari

    Safari Technology Preview Release 81 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 244110-244684.

    Dark Mode

    • Renamed supported-color-schemes to color-scheme (r244408)
    • Standardized the <meta name="color-scheme"> separator (r244413)

    JavaScript

    • Added support for incremental bytecode cache updates (r244143)

    WebRTC

    • Allowed the MediaSource API to be enabled via website policy (r244197)
    • Prevented restarting WebRTC stats timer if backend is stopped (r244632)
    • Supported RTCDataChannel blob binaryType (r244196)
    • Added support for parsing FairPlayStreaming PSSH boxes. (r244439)

    Media

    • Updated AudioSession route sharing policy (r244223)
    • Fixed layout issues occuring when entering full screen mode (r244545)

    Layout

    • Added @page margin support (r244202)
    • Fixed synthesized baseline for CSS Grid and CSS Flex (r244213)

    Accessibility

    • Removed deprecated Accessibility Object Model events (r244582)

    Web API

    • Changed requestAnimationFrame to execute before the next frame (r244182)
    • Changed visited link hash to be computed only once (r244642)
    • Fixed link clicks in PDFs to not send referrer headers (r244405)

    Storage

    • Changed to clear cache storage structures before removing all related files (r244162)
    • Delayed initialization of quota users until the first quota request (r244112)
    • Fixed blob type not getting stored correctly in IndexedDB when IDBObjectStore has autoIncrement and keyPath options (r244436)

    Security

    • Changed to Ignore X-Frame-Options header when frame-ancestors CSP directive is present (r244589)
    • Implemented remembering device orientation permission for the duration of the browsing session (r244382)

    Web Inspector

    • Added line and column numbers with source location when logging CSP messages to the console (r244563)
    • Added support for drag-and-drop to import files in the Canvas and Audit tabs (r244278)
    • Added support for drag-and-drop to import HAR archives in the Network tab (r244576)
    • Added support to show Worker imported scripts in the Open Quickly dialog even if Web Inspector is opened after the Worker was created (r244363)
    • Changed to prevent starting property selection when pressing the right mouse button in the Styles sidebar (r244616)
    • Changed to show a resource in the Debugger even if an extension script throws a parse error (r244398)
    • Changed to include the number of Network Requests in the Statistics section of the CPU Usage timeline (r244411)
    • Ensured that object previews with Promises don’t add unnecessary catch handlers that prevent rejectionhandled events from being fired in the Console (r244312)
    • Ensured that tracking stops for the CPU Usage Timeline when disconnecting Web Inspector (r244478)
    • Fixed an issue where navigating to a resource sometimes shows a different resource (r244412)
    • Made starting and stopping a recording more consistent and reliable in the Timelines tab (r244195)
    • Provided a more obvious way to return to the overview when viewing imported recordings in the Canvas tab (r244560)
    • Updated to preserve multiple selection in the Elements tab when switching tabs (r244154)
    • Updated to disable special breakpoints when they are deleted in the Sources tab (r244274)

    WebDriver

    • Added hooks to make it possible to easily run WPT WebDriver web server by itself (r244492)
    • Fixed JavaScript evaluation failures if a dialog is shown (r244230)
    • Fixed enter fullscreen error handling code (r244236)
    • Fixed the Set Cookie endpoint to correctly set subdomain cookies (r244281)

    Web GPU

    • Implemented support for indexed drawing (r244147)
    • Prevented narrowing conversions during Metal function calls on 32-bit platforms (r244235)

    Privacy Preserving Ad Click Attribution

    • Added the privacy preserving Ad Click Attribution API as an experimental feature (r244581)

    May 01, 2019 08:30 PM

    April 24, 2019

    Intelligent Tracking Prevention 2.2

    Surfin’ Safari

    Note: Read about past updates to this technology in other blog posts about Intelligent Tracking Prevention, and the Storage Access API.

    The beta releases of iOS 12.3 and Safari on macOS Mojave 10.14.5 include an updated version of Intelligent Tracking Prevention (ITP). For purposes of developer communication, we’ll refer to it as ITP 2.2.

    Tracking Via Link Decoration Caps Client-Side Cookies to 1 Day of Storage

    As of ITP 2.2, persistent cookies set through document.cookie are capped to one day of storage when both of the following conditions are met:

    1. A domain classified with cross-site tracking capabilities was responsible for navigating the user to the current webpage.
    2. The final URL of the navigation mentioned above has a query string and/or a fragment identifier.

    The rest of this blog post explores this in detail.

    What Is Cross-Site Tracking Via Link Decoration?

    Let us first explain the reason for this change. Intelligent Tracking Prevention was designed to protect user privacy by preventing cross-site tracking. Since ITP 1.0 was launched, we have seen increased usage of URL query strings for cross-site tracking purposes.

    social.example link decoration to news.example, blog.example, and shop.example

    In the illustration above, the social network website social.example is adding a “click ID” as a query parameter to every outgoing link. Each of these click IDs are connected to the real user ID “Jane Doe” in social.example’s database.

    Once Jane lands on the destination page, the social network uses JavaScript to read the click ID in the URL query string and store it in a first-party cookie. How does the social network gain scripting powers on the destination website? The destination website can import scripts from social networks or ad networks to integrate a feature, and these scripts can be augmented to read, store, and report back incoming click IDs.

    From this point, every time Jane happens to visit news.example, blog.example, or shop.example, social.example’s script reads the click ID first-party cookie, sends it to its server, and identifies her. Now social.example can enrich its profile of Jane by tracking her activities on these websites and they can serve her individually targeted ads on these websites, all because she once navigated there through a link on social.example.

    This shows how click IDs can effectively work like user IDs for cross-site tracking.

    Further Consequences of Cross-Site Tracking Through Link Decoration

    There are two additional consequences of link decoration that are concerning for user privacy and for web compatibility.

    User IDs are Transferred When Users Share Links

    When tracking information is added to links, it can be used to connect people and devices when the links are shared. Jane Doe in the example above may want to share the news.example link with a coworker. Through link decoration, her social.example user ID is now copied into her coworker’s web browser and can be used to connect her with her coworker on social.example’s platform.

    Some Websites Fail to Load When They Receive Unknown Query Parameters

    Many websites simply fail to load when they receive query parameters that their servers don’t recognize. In one case we came across, a link to a comedy website was shared on a popular social network and the website rendered blank because of the social network’s link decoration. The following comments ensued:

    • Commenter 1: “The link is broken.”
    • Poster: “Sorry about that. Here’s the link again …”
    • Commenter 1: “That link is broken too.”
    • Commenter 2: “Yeah, the social network messes with the links. You have to use a link shortener.”

    Exactly When Should I Expect Cookies to Expire?

    Here’s an example of when ITP 2.2 will cap cookie persistence to one day:

    1. The website social.example has been classified by ITP as having cross-site tracking capabilities. How this classification works is explained in detail in the original ITP blog post and the ITP 2.0 blog post.
    2. The user clicks a link on social.example.
    3. The click results in a cross-site navigation that lands on shop.example and the landing URL has a query string and/or a fragment identifier.
    4. The shop.example webpage sets a persistent cookie through document.cookie. This cookie will now have a maximum expiry of one day.

    What About Ad Click Attribution?

    You might recall from our original ITP blog post that we mentioned link decoration as a way to achieve ad attribution in navigations. Such attribution should serve the destination site information about what ad was clicked and on which site, not information about who the user is.

    The kind of link decoration described above is being used for cross-site tracking of users.

    What If My Website Uses Query Strings or Fragments?

    Just like with cookies and other web storage, there are legitimate and privacy-infringing uses of query strings and fragments, and cross-site trackers try to make their activities indistinguishable from good use. We are obliged to prevent cross-site tracking for Safari users.

    It has been suggested that ITP should simply remove any link decoration when the click source has cross-site tracking capabilities. We opted not to do so because of legitimate use and the risk of breaking website compatibility (see the comedy example above).

    Note though, the one-day cap on document.cookie storage only applies if your site was navigated to by another site with cross-site tracking capabilities, and only for the current webpage.

    Developers, Here’s What You Can Do About It

    We have discussed with developers how cross-site tracking through link decoration came about. After all, user IDs are stored in websites’ first-party cookies — a storage space they manage.

    Our impression is that many developers never understood it was happening; in many cases, changes to third-party JavaScript embedded on websites introduced link decoration without web developers’ knowledge. However, once developers know that their websites are leveraged for cross-site tracking purposes, they can choose to filter out trackers’ link decoration.

    We hope that web developers will join us in better protecting user privacy while concurrently creating the best user experiences on the web.

    April 24, 2019 05:00 PM

    April 17, 2019

    Release Notes for Safari Technology Preview 80

    Surfin’ Safari

    Safari Technology Preview Release 80 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 243538-244110.

    WebGPU

    • Updated setBlendColor, setViewport, setScissorRect in GPURenderPassEncoder (r244093)
    • Replaced unsigned longs in WebGPU with uint64_t (r243658)
    • Standardized WebGPU object reference counting and creation logic (r243563)
    • Removed WebMetal experimental feature in favor of WebGPU (r243666)

    Web API

    • Implemented ResizeObserver (r243643)
    • Added support for “noreferrer” window feature to window.open() (r243705)
    • Added support for <object>.contentWindow (r243638)
    • Changed window.closed to true immediately when close() is invoked (r243661)
    • Changed to close the service worker database on network process suspension (r244097)
    • Changed Fetch to allow used body replacement in Request constructor (r243757)
    • Fixed HTML fragment serialization to not strip whitespace from URL attribute values (r243821)
    • Made someWindow.frames, someWindow.self, someWindow.window always return someWindow, even without a browsing context (r243669)
    • Removed conditional parsing of <noembed> content in the HTML parser (r243782)
    • Fixed the loadstart event for XMLHttpRequestUpload to be correctly initialized (r243765)
    • Fixed getBoundingClientRect returning an empty rect on a collapsed range (r243635)
    • Fixed the select element not showing a popup if the element lost focus while the popup was previously shown (r243601)
    • Pasting a table from Confluence strips table cell content (r243653)
    • Updated to ensure resetting the storage quota takes into account third party origins (r243806)

    SVG Animation

    • Fixed SVG Animation (SMIL) on <text> or <tspan> to work correctly on the second run (r243780)

    Media

    • Added support for muting screen capture and camera/microphone independently (r243899)

    CSS

    • Implemented white-space: break-spaces value (r244036)
    • Removed functionality for -apple-trailing-word (r243819)
    • Allowed FontFace names which start with a number (r243637)

    Accessibility

    • Prevented <svg> elements with labels and no accessible contents from getting exposed as empty AXGroups (r244029)
    • Changed to automatically compute accessibility labels for Apple Pay buttons (r244061)

    Web Inspector

    • Added support for showing WebGPU contexts in the Canvas Tab (r243763)
    • Added support for showing the resource initiator in the summary of the headers in the Network Tab (r243614)
    • Adjusted the energy impact thresholds in the CPU Usage Timeline (r243704)
    • Ensured that the Script Profiler debugging thread is not counted as part of the page usage in the CPU Usage Timeline (r243679)
    • Disabling a breakpoint for a specific event listener no longer removes it from the list of breakpoints (r243722)
    • Prevented breakpoints for attribute modifications from firing when breakpoints are disabled (r243719)
    • Prevented single clicks from following links in text editors that are not read-only (r243826)

    Safari Extensions

    • Legacy Safari Extensions (.safariextz files) are no longer supported. Safari App Extensions and Content Blockers, which can take advantage of powerful native APIs and frameworks as well as web technologies, can be distributed with apps in the App Store or from developers’ websites. You can learn more at developer.apple.com/safari/extensions/.

    April 17, 2019 05:00 PM

    April 11, 2019

    Link Click Analytics and Privacy

    Surfin’ Safari

    We’ve recently received questions on what we refer to as link click analytics, and specifically an internal setting for disabling the Ping attribute for anchor elements.

    Privacy By Default

    WebKit always strives for privacy by default. To name three unique such efforts – we partition third-party data storage and ServiceWorkers by default, we partition HTTP caches by default, and our Intelligent Tracking Prevention (ITP) feature is on by default in Safari. To the best of our knowledge, no other browser on the market offers similar protections.

    However, there are cases when users want even stronger privacy guarantees and are willing to trade some functionality or web compatibility for it. Two such examples are Private Browsing and Content Blockers.

    Let’s have a look at how Safari and WebKit’s privacy features play into link click analytics.

    What Is Link Click Analytics?

    The goal of link click analytics is to report to a web server that a navigational link click happened and that the user is leaving the webpage. Such auditing can be used for first-party web analytics as well as third-party cross-site tracking. The latter is where ITP comes in.

    How Can Websites Do Link Click Analytics?

    There are several ways for websites to do link click analytics. The ones we see in use today are:

    • Synchronous XHR (XMLHttpRequest).
    • Asynchronous XHR or Fetch, with a delay.
    • First-party bounce tracking.
    • The Beacon API.
    • The Ping attribute.

    Let’s go through the details of these techniques.

    Synchronous XHR

    Synchronous XHR is the synchronous version of XMLHttpRequest. Such synchronous calls often cause hangs on the web since they block the webpage while it waits for the server’s response. Therefore, web browsers are actively trying to remove the API.

    In the context of link click analytics, a site that is willing to inconvenience users might do something like the following (please don’t use this technology):

    window.addEventListener("unload", function(event) {
      let xhr = new XMLHttpRequest(),
          data = captureTrackingData(event);
    
      xhr.open("post", "/log", false); // 'false' implies synchronous.
      xhr.send(data);
    });
    

    The code above triggers when the user clicks a link and the current webpage unloads. The synchronous XHR blocks the navigation until it’s done which delays the navigation significantly. For users, this is perceived as poor performance.

    Asynchronous XHR or Fetch, With a Delay

    Another popular way to do link click analytics is through asynchronous XHR or Fetch, not in the least because they allow for cross-site requests to third-party trackers.

    While not blocking all execution on the webpage, this introduces an artificial delay to the navigation which users experience as poor performance. We’ve seen between 100 and 350 ms delays in such link click analytics scripts, and even some that do a busy loop while waiting. This is what a delayed navigation with asynchronous XHR looks like:

    const clickTime = 350; // Milliseconds.
    
    theLink.addEventListener("click", function(event) {
      event.preventDefault(); // Cancel the user's link click.
    
      let xhr = new XMLHttpRequest(),
          data = captureTrackingData(event);
    
      xhr.open("post", "/log", true); // 'true' implies asynchronous.
      xhr.send(data);
      setTimeout(function() {
        window.location.href = theLink.getAttribute("href");
      }, clickTime); // Navigate properly after 350 ms.
    });
    

    First Party Bounce Tracking

    You might recall from our earlier blog posts that ITP 2.0 detects first-party bounce trackers and classifies them as any other kind of cross-site tracker.

    Say the user clicks on a news.example link on the social.example website. Instead of navigating them straight to their destination news.example, they are rapidly navigated through trackerOne.example and trackerTwo.example before reaching news.example.

    social.example → trackerOne.example → trackerTwo.example → news.example

    This is a way to let trackerOne and trackerTwo do link click analytics and the increased load time is again bad for performance.

    The Beacon API

    While not specifically built for link click analytics, the Beacon API is a way of sending arbitrary analytics and/or tracking data without affecting the user experience. This is how Beacon can be used for link click analytics:

    window.addEventListener("unload", function(event) {
      let data = captureTrackingData(event);
      navigator.sendBeacon("https://tracker.example/", data);
    });
    

    Beacon requests are guaranteed to be initiated before the page unloads but do not block the webpage or delay the navigation.

    The Ping Attribute

    Ping is an attribute on anchor elements, popularly referred to as just links. The purpose of Ping is link click analytics, plain and simple. Here’s how it’s used:

    <a href="https://news.example" ping="https://tracker.example/going-to-news-example">Read the news</a>
    

    The Ping request to tracker.example above does not block or delay the navigation to news.example.

    What Can WebKit Do About this?

    As can be seen above, websites have several ways to go about logging or tracking a user’s clicks. The first three — synchronous XHR, Fetch with delay, and first-party bounce tracking — all hurt performance and make the web experience worse. The latter two — Beacon and Ping — still log clicks but do so without hurting performance.

    Just turning off the Ping attribute or the Beacon API doesn’t solve the privacy implications of link click analytics. Instead, it creates an incentive for websites to adopt tracking techniques that hurt the user experience. In effect, the choice between supporting Ping and not is not one of privacy, rather it is a choice between a good user experience and a bad one.

    So our approach is to have ITP block cookies and downgrade the referrer header (see the section on Origin-Only Referrer in our ITP 2.0 blog post) for all the link click analytics techniques listed when the request goes to a third-party domain classified with cross-site tracking capabilities. ITP also cleans up website data for first-party bounce trackers.

    The distinction we’re making here is between analytics of link clicks in general versus third-party analytics of link clicks tied to individual users. The latter is what ITP prevents and what we think is the right balance for on-by-default privacy protections.

    What Can Users Do About This?

    For users who want to fully block third-party link click analytics, WebKit and Safari supports Content Blockers. The effect of such load blocking may include blocking of ads or third-party widgets. If you want to install a Content Blocker, check out the App Store where you’ll find plenty of offerings. If you are a developer who wants to build a Content Blocker, see Apple Developer Documentation.

    A Final Note On Hidden Feature Flags

    As a detail of its implementation, WebKit makes use of the User Defaults mechanism on Apple platforms. These flags or pieces of configuration data are not exposed in Safari’s menus or in Safari Settings. Instead, they are used to control the inner workings of features to, for instance, enable quality assurance testing.

    Say WebKit decides to obsolete synchronous XHR altogether on Apple platforms. Such a change might be put behind a User Defaults flag so that it’s easy for engineers to assess whether a reported issue stems from the obsoleted synchronous XHR, or something else.

    Until recently, Safari supported an internal User Defaults flag to disable support for the Ping attribute. It was never our intention to surface this flag as a customer setting. We think it’s misguided to offer users the ability to disable web-facing features if doing so doesn’t disable or prevent the ends of that technology. Instead, Intelligent Tracking Prevention and Content Blockers offer users different levels of support for categorically affecting link click analytics.

    April 11, 2019 10:00 PM

    April 08, 2019

    Philippe Normand: Introducing WPEQt, a WPE API for Qt5

    Igalia WebKit

    WPEQt provides a QML plugin implementing an API very similar to the QWebView API. This blog post explains the rationale behind this new project aimed for QtWebKit users.

    Qt5 already provides multiple WebView APIs, one based on QtWebKit (deprecated) and one based on QWebEngine (aka Chromium). WPEQt aims to provide …

    By Philippe Normand at April 08, 2019 10:20 AM

    April 05, 2019

    New WebKit Features in Safari 12.1

    Surfin’ Safari

    There are many improvements and new web platform features in WebKit that are now available with the release of Safari 12.1, included with macOS Mojave 10.14.4 and iOS 12.2. This release delivers web platform features that improve website integration with the operating system, new real-time communication capabilities, more compatible encrypted media support, and features that help developers reduce the need for polyfills. Safari provides all of these features while at the same time improving user privacy, JavaScript performance, responsiveness, stability, and security.

    This post takes a quick look at the improvements and new features Safari 12.1 brings to the web. To learn even more about what’s in Safari 12.1, including changes to Safari itself, read the Safari 12.1 Release Notes.

    Dark Mode for the Web

    With users’ ability to choose between light and dark color schemes, websites can start to look out of place, or worse, become a blinding and painful experience. To help websites better integrate with the dark appearance setting in macOS Mojave, WebKit provides the supported-color-schemes property and prefers-color-scheme media query to allow a webpage to support light and dark color schemes.

    Web content authors can use the @media(prefers-color-scheme: dark) media query to provide dark mode styles that override a default light theme. Alternatively, @media(prefers-color-scheme: light) can be used to provide light styles that override a default dark theme.

    If you’re reading this post in Safari 12.1 for macOS Mojave, you can toggle in and out of Dark Mode in System Preferences → General to see that webkit.org itself provides light and dark color schemes.

    Web Inspector has also been improved with a new toggle in the Elements tab to easily switch between the light and dark mode. This helps developers test both color schemes without switching the whole system.

    Dark Mode Web Inspector Toggle

    Intelligent Tracking Prevention

    Updates to Intelligent Tracking Prevention add new restrictions to cookies, further reducing the ability of hidden third parties to track users across websites they visit. To do that, support for partitioned cookies was removed for domains Intelligent Tracking Prevention identifies as having cross-site tracking capabilities. Going a step further, Intelligent Tracking Prevention now also limits long-term tracking for JavaScript first-party cookies.

    Read more about these changes in the “Intelligent Tracking Prevention 2.1” blog post.

    Payment Request API

    The Payment Request API has been updated with granular errors, support for default addresses and contacts configured in Wallet and Apple Pay settings, and special field support for Japan. These changes now bring the Payment Request API to parity with the Apple Pay JS payment system, but with all of the benefits of web standards compatibility. Payment Request is now the recommended way to pay implement Apple Pay on the web.

    For more details, see the Payment Request API specifications.

    WebRTC Improvements

    Major improvements to WebRTC include support for video simulcast and the VP8 video codec. Video simulcast supports a technique for encoding video content with different parameters such as frame size or bit rate for serving the same content to multiple clients through a central server. WebKit support for the VP8 video codec is only available in WebRTC. The codec improves website compatibility, permitting video exchange with a wider range of WebRTC endpoints. Because H.264 has hardware support on Apple hardware and has been tuned for power efficiency, WebKit continues to use H.264 as its default codec in WebRTC for a better user experience.

    More details about these WebRTC improvements in WebKit are available in the blog post “On the Road to WebRTC 1.0, Including VP8”.

    Modern Encrypted Media Extensions API

    WebKit has an updated implementation of the Encrypted Media Extensions (EME) API for providing encrypted video or audio content. This updated version also no longer uses a webkit- prefix. The modern EME implementation includes a number of web developer features and conveniences not found in the original, prefixed API. Developers are encouraged to use feature detection to determine if the prefixed or the non-prefixed version of the API is available to ensure compatibility with older versions.

    See the Encrypted Media Extensions standards specifications for more information.

    Change Codecs and Containers in MSE

    Media Source Extensions have been supported by WebKit on macOS since Safari 8. It provides web-based streaming of audio and video media content. With MSE, media streams can be created and controlled by JavaScript. New in this release, WebKit now supports transitioning codecs or bytestream containers using a new changeType() method on SourceBuffer. This means developers can dynamically switch between codecs and containers within a single SourceBuffer rather than switching between multiple MediaSource instances which can cause noticeable delays in the transition.

    Learn more about the changeType() method in the Media Source Extensions Codec Switching feature incubation specifications.

    Intersection Observer

    The Intersection Observer API provides a way for developers to know when a particular element on a page becomes visible. This makes it possible for developers to avoid costly polling techniques that waste battery to periodically compute the position of an element compared to another element or the viewport. This allows for more power efficient implementations of website behaviors like lazily loaded images that also work for cross-origin content.

    You can learn more by reading the “IntersectionObserver in WebKit” blog post.

    Web Share API

    The Web Share API adds navigator.share(), a promise-based API developers can use to invoke a native sharing dialog provided the host operating system. This allows users to share text, links, and other content to an arbitrary destination of their choice, such as apps or contacts.

    navigator.share({
        title: document.title,
        text: "The WebKit Blog",
        url: "https://webkit.org/blog"
    });
    

    Learn more from the Web Share API specifications.

    Color Input

    WebKit added support for <input type="color">, a form control that uses native platform UI to allow users to choose a color as an input value. The value of the element is an RGB color specified in a seven-character hexadecimal format.


    The availability of this input means developers no longer need to develop or maintain custom code to provide color selection capability on websites in Safari.

    Read more about color input in the HTML Living Standard.

    Data Lists

    Another newly supported element in WebKit is the <datalist> element. The <datalist> element contains a list of <option> elements that provides suggested values for <input> elements. A typical pattern that <datalist> is used for is providing auto-complete functionality for text inputs. It can also be used with inputs of other types such as search, tel, url, email, date, month, week, time, datetime-local, and number. To connect an <input> to a <datalist>, add a list attribute to the <input> element that specifies the id attribute of the <datalist>.




    <label for="star-system-chooser">Select star system: </label><input id="star-system-chooser" list="star-system-options" type="text" name="star-system-choice">
    <datalist id="star-system-options">
        <option>Polaris</option>
        <option>Alpha Centauri</option>
        <option>Nysa</option>
        <option>Arcturus</option>
        <option>40 Eridani A</option>
        <option>Raxxla</option>
    </datalist>
    

    Support for <datalist> also helps developers reduce the amount of custom code they develop and maintain in order to provide this functionality.

    Learn more about data lists from the HTML Living Standard.

    Better Text Decorations

    Support for two new text decoration properties give web authors more control over the look of their text content.

    The text-underline-offset allows for placing an underline at a specific location below the baseline. The text-decoration-thickness property specifies the thickness using a length value — a number followed by a unit (e.g. 1px or 0.5em). Rather than using absolute lengths, using em units in both of these properties is preferable here so that the line thickness and offset scales with the font size. You can also specify from-font to both of these properties which will pull the relevant metric from the used font file itself.

    Web Inspector Improvements

    To help developers achieve more efficient editing and debugging workflows, Web Inspector has been improved to support multiple-selection for several key areas including DOM tree nodes, entries in the Cookies table, and style rules in the Styles sidebar.

    Web Inspector Storage Tab Cookies Table with multiple selections in light modeWeb Inspector Storage Tab Cookies Table with multiple selections in dark mode

    The Network tab now provides security information for all loaded resources. When selecting a resource in the Network tab, a resource preview panel is displayed. Clicking the “Security” tab provides security details including a button to show the certificate sent in response.

    Web Inspector Network Tab resource security certificate in light modeWeb Inspector Network Tab resource security certificate in dark mode

    Web Inspector also provides new media debugging tools to help developers working on web applications that deliver media content. The Network tab now includes media requests that can be grouped to more easily see related requests. The network loading timeline also shows new markers for grouped requests that show a pop-up with event timing details when clicked.

    In the Timelines tab, developers can add a new Media timeline to see media events that can be correlated to code. These media events also include special events when entering or exiting a special low-power video playback mode. Developers can help users save battery life by setting up fullscreen video to achieve low power playback. It requires providing video in fullscreen playback on nothing but a black background with no other elements overlaying the video. In this setup, when users play video fullscreen on recent MacBook Pro hardware running on battery power without an external display video is presented in an unconverted format that saves power. This is especially helpful for users to save a significant amount of battery life when they are playing long duration video content.

    Feedback

    All of these improvements are available to users running iOS 12.2 and macOS Mojave 10.14.4, and most of them are also available on macOS High Sierra and macOS Sierra. These features were also available to web developers with Safari Technology Preview releases. Changes in this release of Safari were included in the following Safari Technology Preview releases: 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76. Download the latest Safari Technology Preview release to stay on the forefront of future web features. You can also use the WebKit Feature Status page to watch for changes to your favorite web platform features.

    We love hearing from you. Send a tweet to @webkit or @jonathandavis to share your thoughts on this release, and any features you were hoping for that didn’t make it. If you run into any issues, we welcome your bug reports for Safari, or WebKit bugs for web content issues.

    April 05, 2019 06:20 PM

    April 03, 2019

    Release Notes for Safari Technology Preview 79

    Surfin’ Safari

    Safari Technology Preview Release 79 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 242836-243538.

    Web Authentication

    • Implemented FIDO AppID extension (r243193)

    Accessibility

    • Implemented support for new meter ARIA role (r243198)

    Service Workers

    • Changed to use a ServiceWorker process per registrable domain (r242905)

    Browser Changes

    • Blocked all plugins smaller than 5px × 5px (r242949)

    CSS

    • Fixed CSS grid container baseline alignment for orthogonal items (r243432)
    • Fixed a single leading space not being considered as a word break even when word-break: break-all is set (r243437)

    Web API

    • Changed to use smaller default storage quotas for third-party frames (r243247)
    • Fixed Intl.DateTimeFormat to obey 2-digit hour (r243512)
    • Fixed toggling display: contents to display: none failing to hide the element (r243444)

    Web Inspector

    • Added support for import and export of timeline recordings in the Timelines tab (r243024)
    • Added support for HAR import in the Network tab (r242948)
    • Added a toggle to switch between live activity and Imported HAR resource collections in the Network tab (r243004)
    • Added a HAR extension for resource priority in the Network tab (r243031)
    • Added a HAR extension for serverIPAddress port number in the Network tab (r243087)
    • Added icons for WebGL2 contexts in the Canvas tab (r243424)
    • Added support for style attribute changes to get tracked in the Changes sidebar (r243038)
    • Added support for a way to filter out all Console messages, evaluations, or results (r243260)
    • Added support for ::-webkit-scrollbar* rules being shown in the Styles sidebar (r242939)
    • Added a flattened Timeline overview that shows all records, one per line in the Timelines tab (r243301)
    • Changed to show when preventDefault() was called on an event or not in the Timelines tab (r243269)
    • Changed to support DOM searches to be case sensitive in the Search tab (r243207)
    • Fixed the WebGL action icon to not invert when selected in the Canvas tab (r243426)
    • Fixed changes to CSS resources only taking effect once editing stops (r243208)
    • Fixed getEventListeners to work for any EventTarget in the Console (r242940)
    • Fixed pausing in an inline script on a page with a URL query creating an Extra Script in the Debugger tab (r242929)
    • Fixed selecting a node in a different frame to automatically switch the Console prompt’s execution context (r243214)
    • Fixed HAR Export duplicating blocked time and send time if there was no DNS or Connect block in the Network tab (r242896)
    • Fixed Page Weight indicator clearing on pages with zero length resources (r243452)
    • Fixed canvas being rendered twice per frame in the Canvas tab (r243400)
    • Fixed collapsing a section not shrinking the space used by that section in the Source tab (r243271)
    • Fixed disabled breakpoints banner to be sticky in the Sources tab (r243225)
    • Fixed group by path names to not use the resource file name in the Sources tab (r243300)
    • Fixed editing the selector to not hide the rule in the Styles sidebar (r243264)
    • Fixed the “jump to effective property” button not hiding after overridden property becomes effective in the Styles sidebar (r242973)
    • Fixed export on about:blank causing the suggested filename to contain a colon and silently fail in the Timelines tab (r243355)
    • Gave long thread names a tooltip if they could be truncated in the CPU Usage timeline (r243026)
    • Included window as part of any event listener chain in the DOM (r243244)
    • Provided $event in the Console when paused on an event listener (r243161)
    • Provided a way to capture a screenshot of a node from within the page (r242992)32
    • Provided a way to get the contents of resources for audits in the Audit tab (r242941)
    • Provided option to group by path in the Sources tab (r243180)
    • Reduced the intensity of the orange overlay color for dark mode in the CPU Usage timeline (r243025)

    April 03, 2019 05:00 PM

    March 27, 2019

    Introducing the JetStream 2 Benchmark Suite

    Surfin’ Safari

    Today we are announcing a new version of the JetStream JavaScript benchmark suite, JetStream 2. JetStream 2 combines a variety of JavaScript and WebAssembly benchmarks, covering an array of advanced workloads and programming techniques, and reports a single score that balances them using a geometric mean. JetStream 2 rewards browsers that start up quickly, execute code quickly, and run smoothly. Optimizing the overall performance of our JavaScript engine has always been a high priority for the WebKit team. We use benchmarks to motivate wide-reaching and maintainable optimizations to the WebKit engine, often leading us to make large architectural improvements, such as creating an entirely new optimizing compiler, B3, in 2015. JetStream 2 mixes together workloads we’ve been optimizing for years along with a new set of workloads we’ll be improving for years to come.

    When we released JetStream 1 in 2014, it was a cross-cutting benchmark, measuring the performance of the latest features in the JavaScript ecosystem, at that time. One of our primary goals with JetStream 1 was to have a single benchmark we could use to measure overall JavaScript performance. However, since 2014, a lot about the JavaScript ecosystem has changed. Two of the major changes were the release of an extensive update to the JavaScript language in ES6, and the introduction of an entirely new language in WebAssembly. Even though we created JetStream 1 as a benchmark to track overall engine performance, we found ourselves creating and using new benchmarks in the years since its release. We created ARES-6 in 2017 to measure our performance on ES6 workloads. For the last two years, we’ve also tracked two WebAssembly benchmarks internal to the WebKit team. In late 2017, the V8 team released Web Tooling Benchmark, a benchmark we’ve used to improve regular expression performance in JavaScriptCore. Even though JetStream 1 was released after Kraken, we found ourselves continuing to track Kraken after its release because we liked certain subtests in it. In total, prior to creating JetStream 2, the WebKit team was tracking its performance across six different JavaScript and WebAssembly benchmarks.

    Gating JavaScriptCore’s performance work on a relative ordering of six different benchmark suites is impractical. It proved non-obvious how to evaluate the value of a change based on the collective results of six different benchmark suites. With JetStream 2, we are moving back to having a single benchmark — which balances the scores of its subtests to produce a single overall score — to measure overall engine performance. JetStream 2 takes the best parts of these previous six benchmarks, adds a group of new benchmarks, and combines them into a single new benchmark suite. JetStream 2 is composed of:

    • Almost all JetStream 1 benchmarks.
    • New benchmarks inspired by subtests of Kraken.
    • All of ARES-6.
    • About half of the Web Tooling Benchmark.
    • New WebAssembly tests derived from the asm.js benchmarks of JetStream 1.
    • New benchmarks covering areas not tested by the other six benchmark suites.

    New Benchmarks

    JetStream 2 adds new benchmarks measuring areas not covered by these previous benchmark suites. An essential component of JetStream 1 was the asm.js subset of benchmarks. With the release of WebAssembly, the importance of asm.js has lessened since many users of asm.js are now using WebAssembly. We converted many of the asm.js benchmarks from JetStream 1 into WebAssembly in JetStream 2. We also created a few new WebAssembly benchmarks. One such new test is richards-wasm, which models a hybrid JS and WebAssembly application. Richards-wasm is an implementation of Martin Richard’s system language benchmark that is implemented partially in WebAssembly and partially in JavaScript. The benchmark models a JavaScript application that frequently calls into helper methods defined in WebAssembly.

    JetStream 2 adds two new Web Worker tests. It’s essential for JetStream 2 to measure Web Worker performance given how popular they are across the web. The first worker benchmark we created is bomb-workers. Bomb-workers runs all of SunSpider in parallel — so it stresses how well the browser handles running a lot of JavaScript in parallel. The second benchmark we created is segmentation. Segmentation runs four Web Workers in parallel to compute the time series segmentation over a sample data set. This benchmark is derived from the same algorithm used in WebKit’s performance dashboard.

    JetStream 2 adds three new benchmarks emphasizing regular expression performance: OfflineAssembler, UniPoker, and FlightPlanner. UniPoker and FlightPlanner stress the performance of Unicode regular expressions — a feature that was new in ES6. OfflineAssembler is the JavaScriptCore offline assembler parser and AST generator translated from Ruby into JavaScript. UniPoker is a 5 card stud poker simulation using the Unicode code points for playing cards to represent card values. FlightPlanner parses aircraft flight plans that consist of named segments, e.g. takeoff, climb, cruise, etc, that include waypoints, airways, or directions and times, and then processes those segments with an aircraft profile to compute course, distance and forecast times for each resulting leg in the flight plan, as well as the total flight plan. It exercises Unicode regular expression code paths because the segment names are in Russian.

    JetStream 2 also adds two new general purpose JavaScript benchmarks: async-fs and WSL. Async-fs models performing various file system operations, such as adding and removing files, and swapping the byte-order of existing files. Async-fs stresses the performance of DataView, Promise, and async-iteration. WSL is an implementation of an early version of WHLSL — a new shading language proposal for the web. WSL measures overall engine performance, especially stressing various ES6 constructs and throw.

    You can read about each benchmark in JetStream 2 in the summary page.

    Benchmarking Methodology

    Each benchmark in JetStream 2 measures a distinct workload, and no single optimization technique is sufficient to speed up all benchmarks. Some benchmarks demonstrate tradeoffs, and aggressive or specialized optimizations for one benchmark might make another benchmark slower. Each benchmark in JetStream 2 computes an individual score. JetStream 2 weighs each benchmark equally.

    When measuring JavaScript performance on the web, it’s not enough to just consider the total running time of a workload. Browsers may perform differently for the same JavaScript workload depending on how many times it has run. For example, garbage collection runs periodically, making some iterations take longer than others. Code that runs repeatedly gets optimized by the browser, so the first iteration of any workload is usually more expensive than the rest.

    For this reason, JetStream 1 categorized each benchmark into one of two buckets: latency or throughput. Latency tests either measured startup performance or worst case performance. Throughput tests measured sustained peak performance. Like JetStream 1, JetStream 2 measures startup, worst case, and peak performance. However, unlike JetStream 1, JetStream 2 measures these metrics for every benchmark.

    JetStream 2 scores JavaScript and WebAssembly differently. For all but one of the JavaScript benchmarks in JetStream 2, individual benchmark scores equally weigh startup performance, worst case performance, and average case performance. These three metrics are crucial to running performant JavaScript in the browser. Fast startup times lead browsers to loading pages more quickly and allow users to interact with the page sooner. Good worst case performance ensures web applications run without hiccups or visual jank. Fast average case performance makes it so that the most advanced web applications can run at all. To measure these three metrics, each of these benchmarks run for N iterations, where N is usually 120, but may vary based on the total running time of each iteration. JetStream 2 reports the startup score as the time it takes to run the first iteration. The worst case score is the average of the worst M iterations, excluding the first iteration. M is always less than N, and is usually 4. For some benchmarks, M can be less than 4 when N is smaller than 120. The average case score is the average of all but the first of the N iterations. These three metrics are weighed equally using the geometric mean.

    WSL is JetStream 2’s one JavaScript benchmark that is an exception to the above scoring technique. WSL uses a different scoring mechanism because it takes orders of magnitude longer than the other benchmarks to run through a single iteration. It instead computes its score as the geometric mean over two metrics: the time it takes to compile the WSL standard library, and the time it takes to run through the WSL specification test suite.

    JetStream 2’s WebAssembly benchmarks are scored in two parts that equally weigh startup time and total execution time. The first part is the startup time, which is the time it takes until the WebAssembly module is instantiated. This is the time it takes the browser to put the WebAssembly code in a runnable state. The second part is the execution time. This is the time it takes to run the benchmark’s workload after it is instantiated. Both metrics are crucial for JetStream 2 to measure. Good startup performance makes it so that WebAssembly applications load quickly. Good execution time makes it so WebAssembly benchmarks run quickly and smoothly.

    In total, JetStream 2 includes 64 benchmarks. Each benchmark’s score is computed as outlined above, and the final score is computed as the geometric mean of those 64 scores.

    Optimizing Regular Expressions

    The Web Tooling Benchmark made us aware of some performance deficiencies in JavaScriptCore’s Regular Expression processing and provided a great way to measure any performance improvements for changes we made. For some background, the JavaScriptCore Regular Expression engine, also know as YARR, has both a JIT and interpreter matching engine. There were several types of patterns that were commonly used in the WebTooling Benchmark tests that JavaScriptCore didn’t have support for in the YARR JIT. This included back references, and nested greedy and non-greedy groups.

    A back reference takes the form of /^(x*) 123 \1$/, where we match what is in the parenthesized group and then match the same thing again later in the string when a prior group is referenced. For the example given here, the string "x 123 x" would match as well as the string “xxxxx 123 xxxxx”. It matches any line that begins with 0 or more 'x', has " 123 " in the middle, and then ends with the same number of 'x' characters as the string started with. JIT support for back references was added for both Unicode and non-Unicode patterns. We also added JIT support for patterns with the ignore case flag that process ASCII and Latin1 strings. Back reference matching of Unicode ignore case patterns is more problematic due to the large amount of case folding data required. So, we revert to the YARR interpreter for Unicode regular expressions that contain back references that also ignore character case.

    Before discussing the next improvement we made to the regular expression engine, some background is in order. The YARR Regular Expression engine uses a backtracking algorithm. When we want to match a pattern like /a[bc]*c/, we process forward in the pattern and when we fail to match a term, we backtrack to the prior term to see if we can try the match a little differently. The [bc]* term will match as many b’s and/or c’s in a row as possible. The string “abc” matches the pattern, but it needs to backtrack when matching the 'c' the first time. This happens because the middle [bc]* term will match both the 'b' and the 'c', and when we try to match the final 'c' term in the pattern, the string has been exhausted. We backtrack to the [bc]* term and reduce the length of that term’s match from “bc” to just “b” and then match the final 'c' term. This algorithm requires state information to be saved for various term types so that we can backtrack. Before we started this work, backtracking state consisted of counts and pointers into the term’s progress in a subject string.

    We had to extend how backtracking state was saved in order to add support for counted captured parenthesized groups that are nested within longer patterns. Consider a pattern like /a(b|c)*bc/. In addition to the backtracking information for the contents of the captured parenthesized group, we also need to save that group’s match count, and start and end locations as part of the captured group’s backtracking state. This state is saved in nodes on a singly linked list stack structure. The size of these parenthesized group backtracking nodes is variable, depending on the amount of state needed for all nested terms including the nested capture group’s extents. Whenever we begin matching a parenthesized group, either for the first or a subsequent time, we save the state for all the terms nested within that group as a new node on this stack. When we backtrack to the beginning of a parenthesized group, to try a shorter match or to backtrack to the prior terms, we pop the captured group’s backtracking node and restore the saved state.

    We also made two other performance improvements that helped our JetStream 2 performance. One is matching longer constant strings at once and the other is canonicalizing constructed character classes. We have had the optimization to match multiple adjacent fixed characters in a pattern as a group for some time, where we could match up to 32bits of character data at once, e.g four 8 bit characters. Consider the expression /starting/, which simply looks for the string “starting”. These optimizations allowed us to match “star” with one 32 bit load-compare-branch sequence and then the trailing “ting” with a second 32 bit load-compare-branch sequence. The recent change was made for 64 bit platforms and allows us to match eight 8 bit characters at a time. With this change, this regular expression is now matched with a single 64 bit load-compare-branch sequence.

    The latest Regular Expression performance improvement we did that provided some benefit to JetStream 2 was to coalesce character classes. For background, character classes can be constructed from individual characters, ranges of characters, built in escape characters, or built in character classes. Prior to this change, the character class [\dABCDEFabcdef], which matches any hex digit, would check for digit characters by comparing a character value against ‘0’ and ‘9’, and then individually compare it against each of the alphabetic characters, 'A', 'B', etc. Although this character class could have been written as [\dA-Fa-f], it makes sense for YARR to do this type of optimization for the JavaScript developer. Our change to character class coalescing now does this. We merge individual adjacent characters into ranges, and adjacent ranges into larger ranges. This often reduces the number of compare and branch instructions. Before this optimization, /[\dABCDEFabcdef]/.exec(“X”) would require 14 compares and branches to determine there wasn’t a match. Now it takes just 4 compares and branches.

    The performance impact of these optimizations was dramatic for some of the Web Tooling Benchmark tests. Adding the ability to JIT greedy nested parens improved the performance of coffeescript by 6.5x. JIT’ing non-greedy nested parens was a 5.8x improvement to espree and a 3.1x improvement to acorn. JIT’ing back references improved coffeescript by another 5x for a total improvement of 33x on that test. These changes also had smaller, but still measurable improvements on other JetStream 2 tests.

    Performance Results

    The best way for us to understand the state of performance in JavaScriptCore is both to track our performance over time and to compare our performance to other JavaScript engines. This section compares the performance of the prior released version of Safari, Safari 12.0.3, the newly released Safari 12.1 from macOS 10.14.4, and the latest released versions of Chrome, Chrome 73.0.3683.86, and Firefox, Firefox 66.0.1.

    All numbers are gathered on a 2018 MacBook Pro (13-inch, Four Thunderbolt 3 Ports, MacBookPro15,2), with a 2.3GHz Intel Core i5 and 8GB of RAM. The numbers for Safari 12.0.3 were gathered on macOS 10.14.3. The numbers for Safari 12.1, Chrome 73.0.3683.86, and Firefox 66.0.1 were gathered on macOS 10.14.4. The numbers are the average of five runs of the JetStream 2 benchmark in each browser. Each browser was quit and relaunched between each run.

    JetStream 2 Scores. Bigger is Better.

    The above figure shows that Safari 12.1 is the fastest browser at running JetStream 2. It is 9% faster than Safari 12.0.3, 8% faster than Chrome 73.0.3683.86, and 68% faster than Firefox 66.0.1.

    Conclusion

    JetStream 2 is a major update to the JetStream benchmark suite and we’re excited to be sharing it with you today. JetStream 2 includes a diverse set of 64 JavaScript and WebAssembly benchmarks, making it the most extensive and representative JavaScript benchmark we’ve ever released. We believe that engines optimizing for this benchmark will lead to better JavaScript performance on the web. The JavaScriptCore team is focused on improving JetStream 2 and has already made a 9% improvement on it in Safari 12.1.

    We’d love to hear any feedback you have on JetStream 2 or the optimizations we’ve made for it. Get in touch with Saam or Michael on Twitter with any feedback you have.

    March 27, 2019 05:00 PM

    Michael Catanzaro: Epiphany 3.32 and WebKitGTK 2.24

    Igalia WebKit

    I’m very pleased to (belatedly) announce the release of Epiphany 3.32 and WebKitGTK 2.24. This Epiphany release contains far more changes than usual, while WebKitGTK continues to improve steadily as well. There are a lot of new features to discuss, so let’s dive in.

    Dazzling New Address Bar

    The most noticeable change is the new address bar, based on libdazzle’s DzlSuggestionEntry. Christian put a lot of effort into designing this search bar to work for both Builder and Epiphany, and Jan-Michael helped integrate it into Epiphany. The result is much nicer than we had before:

    The address bar is a central component of the user interface, and this clean design is important to provide a quality user experience. It should also leave a much better first impression than we had before.

    Redesigned Tabs Menu

    Epiphany 3.24 first added a tab menu at the end of the tab bar. This isn’t very useful if you have only a few tabs open, but if you have a huge number of tabs then it’s useful to help navigate through them. Previously, this menu only showed the page titles of the tabs. For 3.32, Adrien has converted this menu to a nice popover, including favicons, volume indicators, and close buttons. These enhancements were primarily aimed at making the browser easier to use on mobile devices, where there is no tab bar, but they’re nice improvement for desktop users, too.

    (On mobile, the tab rows are much larger, to make touch selection easier.)

    Touchpad Gestures

    Epiphany now supports touchpad gestures. Jan-Michael first added a three-finger swipe to Epiphany, for navigating back and forward. Then Alexander (Exalm) decided to go and rewrite it, pushing the implementation down into WebKit to share as much code as possible with Safari. The end result is a two-finger swipe. This was much more involved than I expected as it required converting a bunch of Apple-specific Objective C++ code into cross-platform C++, but the end result was worth the effort:

    Applications that depend on WebKitGTK 2.24 may opt-in to these gestures using webkit_settings_set_enable_back_forward_navigation_gestures().

    Alexander also added pinch zoom.

    Variable Fonts

    Carlos Garcia decided to devote some attention to WebKit’s FreeType font backend, and the result speaks for itself:

    Emoji 🦇

    WebKit’s FreeType backend has supported emoji for some time, but there were a couple problems:

    • Most emoji combinations were not supported, so while characters like🧟(zombie) would work just fine, characters like 🧟‍♂️(man zombie) and 🧟‍♀️(woman zombie) were broken. Carlos fixed this. (Technically, only emoji combinations using a certain character code were broken, but that was most of them.)
    • There was no code to prefer emoji fonts for rendering emoji, meaning emoji would almost always be displayed in non-ideal fonts, usually DejaVu, resulting in a black and white glyph rather than color. Carlos fixed this, too. This seems to work properly in Firefox on some websites but not others, and it’s currently WONTFIXed in Chrome. It’s good to see WebKit ahead of the game, for once. Note that you’ll see color on this page regardless of your browser, because WordPress replaces the emoji characters with images, but I believe only WebKit can handle the characters themselves. You can test your browser here.

    Improved Adaptive Mode

    First introduced in 3.30, Adrien has continued to improve adaptive mode to ensure Epiphany works well on mobile devices. 3.32 is the first release to depend on libhandy. Adrien has converted various portions of the UI to use libhandy widgets.

    Reader Mode

    Jan-Michael’s reader mode has been available since 3.30, but new to 3.32 are many style improvements and new preferences to choose between dark and light theme, and between sans and serif font, thanks to Adrian (who is, confusingly, not Adrien). The default, sans on light background, still looks the best to me, but if you like serif fonts or dark backgrounds, now you can have them.

    JPEG 2000

    Wait, JPEG 2000? The obscure image standard not supported by Chrome or Firefox? Why would we add support for this? Simple: websites are using it. A certain piece of popular server-side software is serving JPEG 2000 images in place of normal JPEGs and even in place of PNG images to browsers with Safari-style user agents. (The software in question doesn’t even bother to change the file extensions. We’ve found far too many images in the wild ending in .png that are actually JPEG 2000.) Since this software is used on a fairly large number of websites, and our user agent is too fragile to change, we decided to support JPEG 2000 in order to make these websites work properly. So Carlos has implemented JPEG 2000 support, using the OpenJPEG library.

    This isn’t a happy event for the web, because WebKit is only as secure as its least-secure dependency, and adding new obscure image formats is not a step in the right direction. But in this case,  it is necessary.

    Mouse Gestures

    Experimental mouse gesture support is now available, thanks to Jan-Michael, if you’re willing to use the command line to enable it:

    $ gsettings set org.gnome.Epiphany.web:/org/gnome/epiphany/web/ enable-mouse-gestures true

    With this, I find myself closing tabs by dragging the mouse down and then to the right. Down and back up will reload the tab. Straight to the left is Back, straight to the right is Forward. Straight down will open a new tab. I had originally hoped to use the right mouse button for this, as in Opera, but turns out there is a difference in context menu behavior: whereas Windows apps normally pop up the context menu on button release, GTK apps open the menu on button press. That means the context menu would appear at the start of every mouse gesture. And that is certainly no good, so we’ve opted to use the middle mouse button instead. We aren’t sure whether this is a good state of things, and need your feedback to decide where to go with this feature.

    Improved Fullscreen Mode

    A cool side benefit of using libdazzle is that the header bar is now available in fullscreen mode by pressing the mouse towards the top of the screen. There’s even a nice animation to show the header bar sliding up to the top of the screen, so you know it’s there (animation disabled for fullscreen video).

    The New Tab Button

    Some users were disconcerted that the new tab button would jump from the end of the tab bar (when multiple tabs are open) back up to the end of the header bar (when there is only one tab open). Now this button will remain in one place: the header bar. Since it will no longer appear in the tab bar, Jan-Michael has moved it back to the start of the header bar, where it was from 3.12 through 3.22, rather than the end. This is mostly arbitrary, but makes for a somewhat more balanced layout.

    The history of the new tab button is rather fun: when the new tab button was first added in 3.8, it was added at the end of the header bar, but moved to the start in 3.12 to be more consistent with gedit, then moved back to the end in 3.24 to reduce the distance it would need to move to reach the tab bar. So we’ve come full circle here, twice. Only time will tell if this nomadic button will finally be able to stay put.

    New Icon

    Yes, most GNOME applications have a new icon in 3.32, so Epiphany is not special here. But I just can’t resist the urge to show it off. Thanks, Jakub!

    And More…

    It’s impossible to mention all the improvements in 3.32 in a single blog post, but I want to squeeze a few more in.

    Alexander (Exalm) landed several improvements to Epiphany’s theme, especially the incognito mode theme, which needed work to look good with the new Adwaita in 3.32.

    Jan-Michael added an animation for completed downloads, so we don’t need to annoyingly pop open the download popover anymore to let you know that your download has completed.

    Carlos Garcia added support for automation mode. This means Epiphany can now be used for running automated tests with WebDriver (e.g. with Selenium). Using the new automation mode, I’ve upstreamed support for running tests with Epiphany to the Web Platform Tests (WPT) project, the test suite used by web engine developers to test standards conformance.

    Carlos also reworked the implementation of script dialogs so that they are now modal only to their associated web view, not modal to the entire application. This means you can just close the browser tab if a particular website is abusing script dialogs in a problematic way, e.g. by continuously opening new dialogs.

    Patrick has improved the directory layout Epiphany uses to store data on disk to avoid storing non-configuration data under ~/.config, and reworked the internals of the password manager to mitigate Spectre-related concerns. He also implemented Happy Eyeballs support in GLib, so Epiphany will now fall back to an IPv4 connection if IPv6 is available but broken.

    Now Contains 100% Less Punctuation!

    Did you notice any + signs missing in this blog? Following GTK+’s rename to GTK, WebKitGTK+ has been renamed to WebKitGTK. You’re welcome.

    Whither Pop!_OS?

    Extra Credit

    Although Epiphany 3.32 has been the work of many developers, as you’ve seen, I want to give special credit Epiphany’s newest maintainer, Jan-Michael. He has closed a considerable number of bugs, landed too many improvements to mention here, and has been a tremendous help. Thank you!

    Now, onward to 3.34!

    By Michael Catanzaro at March 27, 2019 12:41 PM

    March 20, 2019

    Release Notes for Safari Technology Preview 78

    Surfin’ Safari

    Safari Technology Preview Release 78 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 242087-242836.

    Pointer Events

    • Enabled support for Pointer Events by default (r242232)
    • Added support for Pointer Events on macOS (r242137)

    Web Inspector

    • Enabled the CPU Usage timeline by default (r242741, r242739, r242567, r242243)
    • Added an Energy Impact section to the CPU Usage timeline (r242300)
    • Added Statistics and Sources sections to the CPU Usage timeline (r242562)
    • Added CPU Usage categorization per-thread and per-worker (r242466, r242104, r242242, r242740)
    • Added support for system accent color throughout the user interface (r242118, r242578, r242296)
    • Added node contextual menu items for event listeners sorted by node in the Elements tab (r242820)
    • Added support for shift-clicking a variable swatch to replace the text with the variable’s value in the Styles sidebar (r242602)
    • Added support for Control-Space to toggle the auto-completion menu in the Styles sidebar (r242218)
    • Added a go-to button for overridden CSS properties to jump to the effective property in the Styles sidebar (r242622)
    • Added the version number in the UI of the Audit tab (r242395)
    • Added export Canvas recording as HTML in the Canvas tab (r242809)
    • Changed to remove event listeners once a recording is ready in the Canvas tab (r242511)
    • Changed to stop showing the auto-stop UI when not inspecting a page in the Timelines tab (r242244)
    • Changed enabling “Stop when page loads” toggle to stop any active recording if the load event has already occurred in the Timelines tab (r242370)
    • Changed to allow image collections to be filtered by type in the Sources tab (r242829)
    • Changed breakpoints to be disabled when an audit is running (r242315)
    • Changed to not enable breakpoints when the source location changes in the Debugger tab (r242314)
    • Changed to highlight the node when hovering event listeners sorted by node in the Elements tab (r242590)
    • Fixed color swatches for valid inputs with leading or trailing whitespace in the Canvas tab (r242553)
    • Fixed DOM, URL, and Event breakpoints to grey out when all breakpoints are disabled in the Debugger tab (r242318)
    • Fixed the ability to select records in the bottom 16px of the timeline overview graph (r242510)
    • Fixed unreadable text in bezier curve editor numeric input fields in dark mode (r242151)
    • Fixed popover colors when transitioning to and from dark mode (r242144)
    • Fixed DOM event breakpoints firing when breakpoints are disabled (r242588)
    • Fixed frequent flashing of DOM node attributes in the Elements tab (r242322)
    • Fixed the keyboard shortcut for the Settings tab on non-US keyboards (r242821)
    • Fixed no results when opening to Search tab (r242105)
    • Improved handling past recordings in the Timelines tab (r242785)
    • Updated to only show changes for the given node in the CSS Changes sidebar (r242556)

    WebDriver

    • Fixed string not getting terminated with a null character (r242352)

    Web API

    • Adjusted XMLHttpRequest Content-Type handling (r242284)
    • Added support for the referrerpolicy attribute (r242534)
    • Implemented further CORS restrictions (r242786)

    Accessibility

    • Added remote search support for keyboard focusable element search type (r242528)

    Media

    • Removed HEVC as a codec requiring hardware support (r242357)

    March 20, 2019 05:00 PM

    March 19, 2019

    Michael Catanzaro: Epiphany Technology Preview Upgrade Requires Manual Intervention

    Igalia WebKit

    Jan-Michael has recently changed Epiphany Technology Preview to use a separate app ID. Instead of org.gnome.Epiphany, it will now be org.gnome.Epiphany.Devel, to avoid clashing with your system version of Epiphany. You can now have separate desktop icons for both system Epiphany and Epiphany Technology Preview at the same time.

    Because flatpak doesn’t provide any way to rename an app ID, this means it’s the end of the road for previous installations of Epiphany Technology Preview. Manual intervention is required to upgrade. Fortunately, this is a one-time hurdle, and it is not hard:

    $ flatpak uninstall org.gnome.Epiphany

    Uninstall the old Epiphany…

    $ flatpak install gnome-apps-nightly org.gnome.Epiphany.Devel org.gnome.Epiphany.Devel.Debug

    …install the new one, assuming that your remote is named gnome-apps-nightly (the name used locally may differ), and that you also want to install debuginfo to make it possible to debug it…

    $ mv ~/.var/app/org.gnome.Epiphany ~/.var/app/org.gnome.Epiphany.Devel

    …and move your personal data from the old app to the new one.

    Then don’t forget to make it your default web browser under System Settings -> Details -> Default Applications. Thanks for testing Epiphany Technology Preview!

    By Michael Catanzaro at March 19, 2019 06:39 PM

    March 12, 2019

    On the Road to WebRTC 1.0, Including VP8

    Surfin’ Safari

    Safari 11 was the first Safari version to support WebRTC. Since then, we have worked to continue improving WebKit’s implementation and compliance with the spec. I am excited to announce major improvements to WebRTC in Safari 12.1 on iOS 12.2 and macOS 10.14.4 betas, including VP8 video codec support, video simulcast support and Unified Plan SDP (Session Description Protocol) experimental support.

    VP8 Video Codec

    The VP8 video codec is widely used in existing WebRTC solutions. It is now supported as a WebRTC-only video codec in Safari 12.1 on both iOS and macOS betas. By supporting both VP8 and H.264, Safari 12.1 can exchange video with any other WebRTC endpoint. H.264 is the default codec for Safari because it is backed by hardware acceleration and tuned for real-time communication. This provides a great user experience and power efficiency. We found that, on an iPhone 7 Plus in laboratory conditions, the use of H.264 on a 720p video call increases the battery life by up to an hour compared to VP8. With H.264, VP8 and Unified Plan, Safari can mix H.264 and VP8 on a single connection. It is as simple as doing the following:

    const localStream = await navigator.mediaDevices.getUserMedia({audio: true, video: true});
    var pc = new RTCPeerConnection();
    // Create a transceiver to send video.
    pc.addTransceiver('video', { direction: 'sendonly'}).sender.replaceTrack(localStream.getVideoTracks()[0]);
    // Create a transceiver to receive video.
    pc.addTransceiver('video', { direction: 'recvonly'});
    // Send the offer to the other party.
    sendOffer(await pc.createOffer());
    // Other party selects H.264 for the 1st transceiver and VP8 for the 2nd transceiver.
    

    Video Simulcast

    To further improve WebRTC support for multi-party video conferencing, simulcast is now supported for both H.264 and VP8. Kudos to the libwebrtc community, including Cosmo Software, for making great progress in that important area. Simulcast is a technique that encodes the same video content with different encoding parameters, typically different frame sizes and bit rates. This is particularly useful when the same content is sent to several clients through a central server, called an SFU. As the clients might have different constraints (screen size, network conditions), the SFU is able to send the most suitable stream to each client. Each individual encoding can be controlled using RTCRtpSender.setParameters on the sender side. Simulcast is currently activated using SDP munging. Future work should allow simulcast activation using RTCPeerConnection.addTransceiver, as per specification.

    Unified Plan

    WebRTC uses SDP as the format for negotiating the configuration of a connection. While previous versions of Safari used Plan B SDP only, Safari is now transitioning to the standardized version known as Unified Plan SDP. Unified Plan SDP can express WebRTC configurations in a much more flexible way, as each audio or video stream transmission can be configured independently.

    If your website uses at most one audio and one video track per connection, this transition should not require any major changes. If your website uses connections with more audio or video tracks, adoption may be required. With Unified Plan SDP enabled, the support of the WebRTC 1.0 API, in particular the transceiver API, is more complete and spec-compliant than ever. To detect whether Safari uses Unified Plan, you can use feature detection:

    function usesUnifiedPlan()
    {
    return RTCRtpTransceiver.prototype.hasOwnProperty("currentDirection”);
    }
    

    Unified Plan is an experimental feature that is currently turned on by default in Safari Technology Preview and turned off by default in Safari in iOS 12.2 and macOS 10.14.4 betas. This can be turned on using the Develop menu on macOS and Safari settings on iOS.

    Additional API updates

    Safari also comes with additional improvements, including better support of capture device selection, experimental support of the screen capture API, and deprecation of the WebRTC legacy API.

    Web applications sometimes want to select the same capture devices used on a previous call. Device IDs will remain stable across browsing sessions as soon as navigator.mediaDevices.getUserMedia is successfully called once. Your web page can implement persistent device selection as follows:

    let stream;
    let deviceId = window.localStorage.getItem('preferredMicrophoneId');
    const constraint = deviceId ? {audio : {deviceId: {exact: deviceId}}} : {audio: true};
    try {
    stream = await navigator.mediaDevices.getUserMedia(constraint);
    } catch (e) {
    // ...
    }
    deviceId = stream.getAudioTracks()[0].getSettings().deviceId;
    window.localStorage.setItem('preferredMicrophoneId', deviceId);
    

    Existing fingerprinting mitigations remain in place, including the filtering of information provided by navigator.mediaDevices.enumerateDevices as detailed in the blog post, “A Closer Look Into WebRTC”.

    Initial support of the screen capture API is now available as an experimental feature in Safari Technology Preview. By calling navigator.mediaDevices.getDisplayMedia, a web application can capture the main screen on macOS.

    Following the strategy detailed in “A Closer Look Into WebRTC”, the WebRTC legacy API was disabled by default in Safari 12.0. Support for the WebRTC legacy API is removed from iOS 12.2 and macOS 10.14.4 betas. Should your application need support of the WebRTC legacy API, we recommend the use of the open source adapter.js library as a polyfill.

    Feedback

    We always appreciate your feedback. Send us bug reports, or feel free to tweet @webkit on Twitter, or email web-evangelist@apple.com.

    March 12, 2019 05:00 PM

    March 06, 2019

    Release Notes for Safari Technology Preview 77

    Surfin’ Safari

    Safari Technology Preview Release 77 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 241432-242087.

    WebRTC

    • Fixed getUserMedia with an ideal deviceId constraint to always select the correct device (r241489)
    • Made navigator.mediaDevices SecureContext (r241602)

    Web API

    • Fixed WebSocket to not fire events after being stopped (r241599)
    • Fixed Same-Site Lax cookies to be sent with cross-site redirect from a client-initiated load (r241918)
    • Updated the MIME type parser (r241863)

    CSS Grid

    • Changed to handle indefinite percentages in fit-content() (r241746)

    Accessibility

    • Fixed HTML5 <footer> tag to be read as an ARIA Landmark to VoiceOver (r242051)
    • Fixed incorrectly mapping the <meter> element to progressbar (r241989)

    Media

    • Fixed setting fullscreen pseudo-classes outside of root when entering fullscreen from inside of a shadow root (r241484)
    • Changed inline WebVTT styles to start with ::cue (r241608)

    Web Inspector

    • Added emulation of user gestures from Web Inspector console (r241633)
    • Added built-in accessibility tests to the Audit tab (r241639)
    • Added a macOS user-agent option in the device settings menu (r241870)
    • Added an experimental setting to merge Resources and Debugger tabs into a single Sources tab (r242049)
    • Changed how selected items are tracked in trees and tables to improve performance (r241652)
    • Changed Command-/ (⌘/) to toggle the edited property in the Styles sidebar (r242063)
    • Changed typing ; to not focus on the next property when there’s open parenthesis or open comment in the Styles sidebar (r241740)
    • Changed clicking a CSS property or selector to always select its text in the Styles sidebar (r241980)
    • Fixed all actions after an offscreen path modification getting marked as offscreen path errors in the Canvas tab (r241616)
    • Fixed an issue where taking a second Canvas recording wouldn’t select the Initial State of the new recording (r241601)
    • Fixed clearing the selection when dragging a selection outside of the selected element in the Console (r242078)
    • Fixed the dim appearance of commas in CSS selectors for dark mode (r241638)
    • Fixed the DOM Content Loaded blue marker for dark mode (r241975)
    • Fixed the Network Overview Graph segments box shadow for dark mode (r242072)
    • Fixed node-specific event breakpoints to get cleared or recalculated on navigation (r241874)
    • Fixed highlighting a node when hovering inside an object preview (r242076)
    • Fixed the navigation sidebar showing “No Search Results” when a slow search is in progress (r242080)
    • Fixed Scripts Timeline loading indicator margins in right-to-left mode (r241857)
    • Fixed valid values in style attributes getting reported as unsupported property values in the Styles sidebar (r241623)
    • Improved the Scripts timeline on-going-recording in dark mode (r241757)
    • Improved categorization of JavaScript Class instances in the Heap Snapshot instances view (r241787)
    • Provided a way to avoid a Timeline recording auto-stop on reload (r241750)
    • Provided a way to make searches case-sensitive or use a regular expression (r242018)

    Bug Fixes

    • Fixed a bug that caused loading on some sites to hang in Safari Technology Preview (r241658)

    Safari App Extensions

    • Made the window.safari object available in frames opened to safari-extension:// resources

    March 06, 2019 09:00 PM

    February 26, 2019

    Frédéric Wang: Review of Igalia's Web Platform activities (H2 2018)

    Igalia WebKit

    This blog post reviews Igalia’s activity around the Web Platform, focusing on the second semester of 2018.

    Projects

    MathML

    During 2018 we have continued discussions to implement MathML in Chromium with Google and people interested in math layout. The project was finally launched early this year and we have encouraging progress. Stay tuned for more details!

    Javascript

    As mentioned in the previous report, Igalia has proposed and developed the specification for BigInt, enabling math on arbitrary-sized integers in JavaScript. We’ve continued to land patches for BigInt support in SpiderMonkey and JSC. For the latter, you can watch this video demonstrating the current support. Currently, these two support are under a preference flag but we hope to make it enable by default after we are done polishing the implementations. We also added support for BigInt to several Node.js APIs (e.g. fs.Stat or process.hrtime.bigint).

    Regarding “object-oriented” features, we submitted patches private and public instance fields support to JSC and they are pending review. At the same time, we are working on private methods for V8

    We contributed other nice features to V8 such as a spec change for template strings and iterator protocol, support for Object.fromEntries, Symbol.prototype.description, miscellaneous optimizations.

    At TC39, we maintained or developed many proposals (BigInt, class fields, private methods, decorators, …) and led the ECMAScript Internationalization effort. Additionally, at the WebAssembly Working Group we edited the WebAssembly JS and Web API and early version of WebAssembly/ES Module integration specifications.

    Last but not least, we contributed various conformance tests to test262 and Web Platform Tests to ensure interoperability between the various features mentioned above (BigInt, Class fields, Private methods…). In Node.js, we worked on the new Web Platform Tests driver with update automation and continued porting and fixing more Web Platform Tests in Node.js core.

    We also worked on the new Web Platform Tests driver with update automation, and continued porting and fixing more Web Platform Tests in Node.js core. Outside of core, we implemented the initial JavaScript API for llnode, a Node.js/V8 plugin for the LLDB debugger.

    Accessibility

    Igalia has continued its involvement at the W3C. We have achieved the following:

    We are also collaborating with Google to implement ATK support in Chromium. This work will make it possible for users of the Orca screen reader to use Chrome/Chromium as their browser. During H2 we began implementing the foundational accessibility support. During H1 2019 we will continue this work. It is our hope that sufficient progress will be made during H2 2019 for users to begin using Chrome with Orca.

    Web Platform Predictability

    On Web Platform Predictability, we’ve continued our collaboration with AMP to do bug fixes and implement new features in WebKit. You can read a review of the work done in 2018 on the AMP blog post.

    We have worked on a lot of interoperability issues related to editing and selection thanks to financial support from Bloomberg. For example when deleting the last cell of a table some browsers keep an empty table while others delete the whole table. The latter can be problematic, for example if users press backspace continuously to delete a long line, they can accidentally end up deleting the whole table. This was fixed in Chromium and WebKit.

    Another issue is that style is lost when transforming some text into list items. When running execCommand() with insertOrderedList/insertUnorderedList on some styled paragraph, the new list item loses the original text’s style. This behavior is not interoperable and we have proposed a fix so that Firefox, Edge, Safari and Chrome behave the same for this operation. We landed a patch for Chromium. After discussion with Apple, it was decided not to implement this change in Safari as it would break some iOS rich text editor apps, mismatching the required platform behavior.

    We have also been working on CSS Grid interoperability. We imported Web Platform Tests into WebKit (cf bugs 191515 and 191369 and at the same time completing the missing features and bug fixes so that browsers using WebKit are interoperable, passing 100% of the Grid test suite. For details, see 191358, 189582, 189698, 191881, 191938, 170175, 191473 and 191963. Last but not least, we are exporting more than 100 internal browser tests to the Web Platform test suite.

    CSS

    Bloomberg is supporting our work to develop new CSS features. One of the new exciting features we’ve been working on is CSS Containment. The goal is to improve the rendering performance of web pages by isolating a subtree from the rest of the document. You can read details on Manuel Rego’s blog post.

    Regarding CSS Grid Layout we’ve continued our maintenance duties, bug triage of the Chromium and WebKit bug trackers, and fixed the most severe bugs. One change with impact on end users was related to how percentages row tracks and gaps work in grid containers with indefinite size, the last spec resolution was implemented in both Chromium and WebKit. We are finishing the level 1 of the specification with some missing/incomplete features. First we’ve been working on the new Baseline Alignment algorithm (cf. CSS WG issues 1039, 1365 and 1409). We fixed related issues in Chromium and WebKit. Similarly, we’ve worked on Content Alignment logic (see CSS WG issue 2557) and resolved a bug in Chromium. The new algorithm for baseline alignment caused an important performance regression for certain resizing use cases so we’ve fixed them with some performance optimization and that landed in Chromium.

    We have also worked on various topics related to CSS Text 3. We’ve fixed several bugs to increase the pass rate for the Web Platform test suite in Chromium such as bugs 854624, 900727 and 768363. We are also working on a new CSS value ‘break-spaces’ for the ‘white-space’ property. For details, see the CSS WG discussions: issue 2465 and pull request. We implemented this new property in Chromium under a CSSText3BreakSpaces flag. Additionally, we are currently porting this implementation to Chromium’s new layout engine ‘LayoutNG’. We have plans to implement this feature in WebKit during the second semester.

    Multimedia

    • WebRTC: The libwebrtc branch is now upstreamed in WebKit and has been tested with popular servers.
    • Media Source Extensions: WebM MSE support is upstreamed in WebKit.
    • We implemented basic support for <video> and <audio> elements in Servo.

    Other activities

    Web Engines Hackfest 2018

    Last October, we organized the Web Engines Hackfest at our A Coruña office. It was a great event with about 70 attendees from all the web engines, thank you to all the participants! As usual, you can find more information on the event wiki including link to slides and videos of speakers.

    TPAC 2018

    Again in October, but this time in Lyon (France), 12 people from Igalia attended TPAC and participated in several discussions on the different meetings. Igalia had a booth there showcasing several demos of our last developments running on top of WPE (a WebKit port for embedded devices). Last, Manuel Rego gave a talk on the W3C Developers Meetup about how to contribute to CSS.

    This.Javascript: State of Browsers

    In December, we also participated with other browser developers to the online This.Javascript: State of Browsers event organized by ThisDot. We talked more specifically about the current work in WebKit.

    New Igalians

    We are excited to announce that new Igalians are joining us to continue our Web platform effort:

    • Cathie Chen, a Chinese engineer with about 10 years of experience working on browsers. Among other contributions to Chromium, she worked on the new LayoutNG code and added support for list markers.

    • Caio Lima a Brazilian developer who recently graduated from the Federal University of Bahia. He participated to our coding experience program and notably worked on BigInt support in JSC.

    • Oriol Brufau a recent graduate in math from Barcelona who is also involved in the CSSWG and the development of various browser engines. He participated to our coding experience program and implemented the CSS Logical Properties and Values in WebKit and Chromium.

    Coding Experience Programs

    Last fall, Sven Sauleau joined our coding experience program and started to work on various BigInt/WebAssembly improvements in V8.

    Conclusion

    We are thrilled with the web platform achievements we made last semester and we look forward to more work on the web platform in 2019!

    February 26, 2019 11:00 PM

    Miguel A. Gómez: Hole punching in WPE

    Igalia WebKit

    As you may (or may not) know, WPE (and WebKitGtk+ if the proper flags are enabled) uses OpengGL textures to render the video frames during playback.

    In order to do this, WPE creates a playbin and uses a custom bin as videosink. This bin is composed by some GStreamer-GL components together with an appsink. The GL components ensure that the video frames are uploaded to OpenGL textures, while the appsink allows the player to get a signal when a new frame arrives. When this signal is emitted, the player gets the frame as a texture from the appsink and sends it to the accelerated compositor to be composed with the rest of the layers of the page.

    This process is quite fast due to the hardware accelerated drawings, and as the video frames are just another layer that is composited, it allows them to be transformed and animated: the video can be scaled, rotated, moved around the page, etc.

    But there are some platforms where this approach is not viable, maybe because there’s no OpenGL support, or it’s too slow, or maybe the platform has some kind of fast path support to take the decoded frames to the display. For these cases, the typical solution is to draw a transparent rectangle on the brower, in the position where the video should be, and then use some platform dependent way to put the video frames in a display plane below the browser, so they are visible through the transparent rectangle. This approach is called hole punching, as it refers to punching a hole in the browser to be able to see the video.

    At Igalia we think that supporting this feature is interesting, and following our philosophy of collaborating upstream as much as possible, we have added two hole punching approaches to the WPE upstream trunk: GStreamer hole punch and external hole punch.

    GStreamer hole punch

    The idea behind this implementation is to use the existent GStreamer based MediaPlayer to perform the media playback, but replace the appsink (and maybe other GStreamer elements) with a platform dependant video sink that is in charge of putting the video frames on the display. This can be enabled with the -DUSE_GSTREAMER_HOLEPUNCH flag.

    Of course, the current implementation is not complete cause the platform dependent bits need to be added to have the full functionality. What it currently does is to use a fakevideosink so the video frames are not shown, and draw the transparent rectangle on the position where the video should be. If you enable the feature and play a video, you’ll see the transparent rectangle and you’ll be able to hear the video sound (and even use the video controls as they work), but nothing else will happen.

    In order to have the full functionality there are a couple of places in the code that need to be modified to create the appropriate platform dependend elements. These two places are inside MediaPlayerPrivateGStreamerBase.cpp, and they are the createHolePunchVideoSink() and setRectangleToVideoSink() functions.

    GstElement* MediaPlayerPrivateGStreamerBase::createHolePunchVideoSink()
    {
        // Here goes the platform-dependant code to create the videoSink. As a default
        // we use a fakeVideoSink so nothing is drawn to the page.
        GstElement* videoSink =  gst_element_factory_make("fakevideosink", nullptr);
    
        return videoSink;
    }
    
    static void setRectangleToVideoSink(GstElement* videoSink, const IntRect& rect)
    {
        // Here goes the platform-dependant code to set to the videoSink the size
        // and position of the video rendering window. Mark them unused as default.
        UNUSED_PARAM(videoSink);
        UNUSED_PARAM(rect);
    }
    

    The first one, createHolePunchVideoSink() needs to be modified to create the appropriate video sink to use for the platform. This video sink needs to have some method that allows setting the position where the video frames are to be displayed, and the size they should have. And this is where setRectangleToVideoSink() comes into play. Whenever the transparent rectangle is painted by the browser, it will tell the video sink to render the frames to the appropriate position, and it does so using that function. So you need to modify that function to use the appropriate way to set the size and position to the video sink.

    And that’s all. Once those changes are made the feature is complete, and the video should be placed exactly where the transparent rectangle is.

    Something to take into account is that the size and position of the video rectangle are defined by the CSS values of the video element. The rectangle won’t be adjusted to fit the aspect ratio of the video, as that must be done by the platform video sink.

    Also, the video element allows some animations to be performed: it can be translated and scaled, and it will properly notify the video sink about the animated changes. But, of course, it doesn’t support rotation or 3D transformations (as the normal video playback does). Take into account that there might be a small desynchronization between the transparent rectangle and the video frames size and position, due to the asynchronicity of some function calls.

    Playing a video with GStreamer hole punch enabled.

    External hole punch

    Unlike the previous feature, this one doesn’t rely at all on GStreamer to perform the media playback. Instead, it just paints the transparent rectangle and lets the playback to be handled entirely by an external player.

    Of course, there’s still the matter about how to synchronize the transparent rectangle position and the external player. There would be two ways to do this:

    • Implement a new WebKit MediaPlayerPrivate class that would communicate with the external player (through sockets, the injected bundle or any other way). WPE would use that to tell the platform media player what to play and where to render the result. This is completely dependant of the platform, and the most complex solution, but it would allow to use the browser to play content from any page without any change. But precisely as it’s completely platform dependant, this is not valid approach for upstream.
    • Use javascript to communicate with the native player, telling it what to play and where, and WPE would just paint the transparent rectangle. The problem with this is that we need to have control of the page to add the javascript code that controls the native player, but on the other hand, we can implement a generic approach on WPE to paint the transparent rectangle. This is the option that was implemented upstream.

    So, how can this feature be used? It’s enabled by using the -DUSE_EXTERNAL_HOLEPUNCH flag, and what it does is add a new dummy MediaPlayer to WPE that’s selected to play content of type video/holepunch. This dummy MediaPlayer will draw the transparent rectangle on the page, according to the CSS values defined, and won’t do anything else. It’s up to the page owners to add the javascript code required to initiate the playback with the native player and position the output in the appropriate place under the transparent rectangle. To be a bit more specific, the dummy player will draw the transparent rectangle once the type has been set to video/holepunch and load() is called on the player. If you have any doubt about how to make this work, you can give a look to the video-player-holepunch-external.html test inside the ManualTests/wpe directory.

    This implementation doesn’t support animating the size and position of the video… well, it really does, as the transparent rectangle will be properly animated, but you would need to animate the native player’s output as well, and syncing the rectangle area and the video output is going to be a challenging task.

    As a last detail, controls can be enabled using this hole punch implementation, but they are useless. As WPE doesn’t know anything about the media playback that’s happening, the video element controls can’t be used to handle it, so it’s just better to keep them disabled.

    Using both implementations together

    You may be wondering, is it possible to use both implementations at the same time? Indeed it is!! You may be using the GStreamer holepunch to perform media playback with some custom GStreamer elements. At some point you may find a video that is not supported by GStreamer and you can just set the type of the video element to video/holepunch and start the playback with the native player. And once that video is finished, start using the GStreamer MediaPlayer again.

    Availability

    Both hole punch features will be available on the upcoming stable 2.24 release (and, of course, on 2.25 development releases). I hope they are useful for you!

    By magomez at February 26, 2019 10:25 AM

    February 21, 2019

    Intelligent Tracking Prevention 2.1

    Surfin’ Safari

    Note: Read about past updates to this technology in other blog posts about Intelligent Tracking Prevention, and the Storage Access API.

    The beta releases of iOS 12.2 and Safari 12.1 on macOS High Sierra and Mojave include an updated version of Intelligent Tracking Prevention (ITP). For purposes of developer communication, we’ll refer to it as ITP 2.1.

    With this update we further reduce trackers’ ability to establish user identities across sites.

    A Single Set of Cookies Per Site

    Previous versions of ITP allowed domains that were classified with tracking capabilities to store partitioned cookies, i.e. additional sets of cookies keyed off of the top site. As of ITP 2.1, partitioned cookies are no longer supported and third-parties classified with cross-site tracking capabilities now have to use the Storage Access API to get any cookie access.

    The revised timeline for classified domains’ cookies and website data looks like this:

    There are multiple reasons for this change:

    • Simplify for developers. We’ve received reports of existing site behavior getting tripped up by partitioned cookies. For instance, one site had a script which if it concluded that the user was logged out, overwrote its cookies with blank values to make sure no remnant data persisted (except for the cookie names). When this site was rendered in a third-party iframe, this cookie overwriting would instead create a new set of blank cookies.
    • Lower the memory footprint. As of ITP 1.1, partitioned cookies are session cookies which means they are not persisted. By getting rid of partitioned cookies, the storage session doesn’t have to keep multiple sets of cookies in memory.
    • Simplify ITP adoption for other WebKit ports. Partitioned cookies require support in the HTTP layer below WebKit. By removing support for them, we increase ITP’s flexibility and can make it work on more platforms.

    Client-Side Cookies Capped to 7 Days of Storage

    Cookies can either be set in HTTP responses or through the document.cookie API, the latter sometimes referred to as client-side cookies. With ITP 2.1, all persistent client-side cookies, i.e. persistent cookies created through document.cookie, are capped to a seven day expiry.

    The reasons for this change cover privacy, security, and performance:

    • Cross-site trackers have started using first-party sites’ own cookie jars for the purpose of persistent tracking. The first-party storage space is especially troublesome for privacy since all tracker scripts in the first-party context can read and write each other’s data. Say social.example writes a user tracking ID as a news.example first-party cookie. Now analytics.example, adnetwork.example, and video.example can leverage or cross pollinate that user tracking ID through their scripts on news.example.
    • Cookies available in document.cookie can be stolen by speculative execution attacks on memory. Therefore, they should not carry sensitive information such as credentials.
    • Cookies available in document.cookie can be stolen by cross-site scripting attacks. Again, therefore, they should not carry sensitive information such as credentials.
    • The proliferation of cookies slows down page and resource loads since cookies are added to every applicable HTTP request. Additionally, many cookies have high entropy values which means they cannot be compressed efficiently. We come across sites with kilobytes of cookies sent in every resource request.
    • There is a size limit on outgoing cookie headers for performance reasons, and websites risk hitting this limit when cross-site trackers add first-party cookies. We’ve investigated reports of news site subscribers getting spuriously logged out, and found that trackers were adding so many cookies that the news site’s legitimate login cookie got pushed out.

    Mike West (Google) has provided related statistics on cookies.

    Will This Change Log Users Out?

    Authentication cookies should be Secure and HttpOnly to protect them against man-in-the-middle attacks, cross-site scripting attacks, and speculative execution attacks. Cookies created through document.cookie cannot be HttpOnly which means authentication cookies should not be affected by the lifetime cap. If they are, you need to set your authentication cookies in an HTTP response and mark them Secure and HttpOnly.

    Implementation Details

    • Only cookies created through document.cookie are affected by this change.
    • The change covers cookies created in first-party contexts as well as in third-party iframes.
    • Session cookies are not affected, they remain session cookies.
    • Persistent cookies that have an expiry shorter than seven days keep their shorter expiry.

    Verified Partitioned Cache

    WebKit implemented partitioned caches more than five years ago. A partitioned cache means cache entries for third-party resources are double-keyed to their origin and the first-party eTLD+1. This prohibits cross-site trackers from using the cache to track users. Even so, our research has shown that trackers, in order to keep their practices alive under ITP, have resorted to partitioned cache abuse. Therefore, we have developed the verified partitioned cache.

    When a partitioned cache entry is created for a domain that’s classified by ITP as having cross-site tracking capabilities, the entry gets flagged for verification. After seven days, if there’s a cache hit for such a flagged entry, WebKit will act as if it has never seen this resource and load it again. The new response is then compared to the cached response and if they match in the ways we care about for privacy reasons, the verification flag is cleared and the cache entry is from that point considered legitimate. However, if the new response does not match the cache entry, the old entry is discarded, and a new one is created with the verification flag set, and the verification process starts over.

    ITP currently does this verification for permanent redirects since that’s where we see abuse today.

    Removed Compatibility Fix for Popups

    As mentioned in the ITP 2.0 blog post, we implemented a temporary compatibility fix for third-party popups, automatically forwarding storage access under the opener page. In ITP 2.1, we’ve removed the compatibility fix for popups that dismiss before they receive user interaction, i.e. dismiss before they receive a tap, click, or text entry.

    Removed Support For the DNT (“Do Not Track”) Signal

    Apple believes that privacy is a fundamental human right and that users should not be tracked across the web without their knowledge or their consent. Intelligent Tracking Prevention was enabled by default for all users when it was released in 2017, and prevented users from being tracked by third parties they didn’t engage with. ITP has been enhanced since then in response to trackers abusing web technologies to track users and additional on-by-default privacy functionality has been added to Safari, such as browser fingerprinting defenses.

    The introduction of ITP 2.1 coincides with Safari removing support for the Do Not Track (DNT) signal. The DNT signal was an attempt by web stakeholders to offer users an off-by-default way to ask servers not to track them. Importantly, DNT did not offer technical enforcement to prevent tracking of users by websites. Apple supported the DNT project starting in 2011, but since then, the vast majority of websites unfortunately have not changed their behavior in response to the DNT signal for the users who elected to turn it on. Instead, online tracking and tracking techniques have become more pervasive and sophisticated in spite of the DNT project.

    The DNT project recently ended without the publication of a standard, in part “because there has not been sufficient deployment of these extensions (as defined) to justify further advancement.” Given the lack of deployment of DNT and Safari’s on by default privacy protections such as ITP, Safari removed support for DNT so that users are not presented with a misleading and ineffective privacy control that, if anything, only offered additional browser fingerprinting entropy.

    February 21, 2019 06:30 PM

    February 20, 2019

    Release Notes for Safari Technology Preview 76

    Surfin’ Safari

    Safari Technology Preview Release 76 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 240780-241432.

    Dark Mode

    • Fixed overflow element scrollbar for dark mode content

    WebRTC

    • Fixed RTCRtpSender.getCapabilities(“video”) before initial offer breaking VP8

    Web API

    • Simplified and streamlined code that creates an appropriate document based on MIME type
    • Aligned with Fetch on data: URLs

    CSS

    • Fixed scroll-snap-align to be honored on a child with non-visible overflow

    Rendering

    • Added basic scrollability for async overflow scrolling
    • Fixed first paint getting blocked by async or deferred scripts
    • Fixed jumpy async overflow scrolling if the main thread is busy
    • Fixed rendering for async overflow scroll with border-radius

    Payment Request

    • Updated to allow requiring a phonetic name for shipping contacts

    Media

    • Updated inline WebVTT styles to start with ::cue
    • Updated to always stop ducking audio after audio capture ends

    Web Inspector

    • Added the ability to Import and Export Heap Snapshots
    • Added context menu items to copy a resource’s HTTP request or response data
    • Added keyboard shortcut to the Export button tooltip in the Audit tab
    • Added icons for media event types
    • Fixed selection getting broken after deleting the selected node
    • Fixed the Memory timeline starting from zero when both CPU and Memory timeline are enabled
    • Fixed missing resource data for the document on reload in the Resources tab
    • Fixed timeline Detail views to reset properly when a new time range selection contains nothing in the Timelines tab
    • Fixed timeline graphs that have drawing issues with multiple discontinuities in the Timelines tab
    • Fixed timeline time range selection to show duration alongside start and end in the Timelines tab
    • Fixed clicking on an empty space in the overview to deselect any selected record bar in the Timelines tab
    • Reduced the time spent updating related resources in the Details sidebar of the Resources tab when loading a page with lots of resources
    • Updated the Styles sidebar to automatically close unbalanced quotes and parenthesis when editing values
    • Updated the Styles sidebar with an easier way to select a single line

    Browser Changes

    • Added a “Reset All to Defaults” option in the Experimental Features menu of the Develop menu

    February 20, 2019 06:00 PM

    February 11, 2019

    Víctor Jáquez: Review of Igalia’s Multimedia Activities (2018/H2)

    Igalia WebKit

    This is the first semiyearly report about Igalia’s activities around multimedia, covering the second half of 2018.

    Great length of this report was exposed in Phil’s talk surveying mutimedia development in WebKitGTK and WPE:

    WebKit Media Source Extensions (MSE)

    MSE is a specification that allows JS to generate media streams for playback for Web browsers that support HTML 5 video and audio.

    Last semester we upstreamed the support to WebM format in WebKitGTK with the related patches in GStreamer, particularly in qtdemux, matroskademux elements.

    WebKit Encrypted Media Extensions (EME)

    EME is a specification for enabling playback of encrypted content in Web bowsers that support HTML 5 video.

    In a downstream project for WPE WebKit we managed to have almost full test coverage in the YoutubeTV 2018 test suite.

    We merged our contributions in upstream, WebKit and GStreamer, most of what is legal to publish, for example, making demuxers aware of encrypted content and make them to send protection events with the initialization data and the encrypted caps, in order to select later the decryption key.

    We started to coordinate the upstreaming process of a new implementation of CDM (Content Decryption Module) abstraction and there will be even changes in that abstraction.

    Lighting talk about EME implementation in WPE/WebKitGTK in GStreamer Conference 2018.

    WebKit WebRTC

    WebRTC consists of several interrelated APIs and real time protocols to enable Web applications and sites to captures audio, or A/V streams, and exchange them between browsers without requiring an intermediary.

    We added GStreamer interfaces to LibWebRTC, to use it for the network part, while using GStreamer for the media capture and processing. All that was upstreamed in 2018 H2.

    Thibault described thoroughly the tasks done for this achievement.

    Talk about WebRTC implementation in WPE/WebKitGTK in WebEngines hackfest 2018.

    Servo/media

    Servo is a browser engine written in Rust designed for high parallelization and high GPU usage.

    We added basic support for <video> and <audio> media elements in Servo. Later on, we added the GstreamerGL bindings for Rust in gstreamer-rs to render GL textures from the GStreamer pipeline in Servo.

    Lighting talk in the GStreamer Conference 2018.

    GstWPE

    Taking an idea from the GStreamer Conference, we developed a GStreamer source element that wraps WPE. With this source element, it is possible to blend a web page and video in a single video stream; that is, the output of a Web browser (to say, a rendered web page) is used as a video source of a GStreamer pipeline: GstWPE. The element is already merged in the gst-plugins-bad repository.

    Talk about GstWPE in FOSDEM 2019

    Demo #1

    Demo #2

    GStreamer VA-API and gst-MSDK

    At last, but not the least, we continued helping with the maintenance of GStreamer-VAAPI and gst-msdk, with code reviewing and on-going migration of the internal library to GObject.

    Other activities

    The second half of 2018 was also intense in terms of conferences and hackfest for the team:


    Thanks to bear with us along all this blog post and to keeping under your radar our work.

    By vjaquez at February 11, 2019 12:52 PM

    February 06, 2019

    Release Notes for Safari Technology Preview 75

    Surfin’ Safari

    Safari Technology Preview Release 75 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 239991-240780.

    WebRTC

    • Enabled Screen Capture (r240635)
    • Updated track sources to be unmuted whenever re-enabled after setDirection changes (r240152)
    • Updated Screen Capture to correctly handle rotation for local video playback (r240000)

    Intelligent Tracking Prevention

    • Added the ability to enable and disable Intelligent Tracking Prevention Debug Mode in the Develop menu (r240129)

    Web Authentication

    • Changed the nonce in the CTAP kInit command to weak random values (r240081, r240103)

    Media

    • Allowed <video> elements not in the DOM to AirPlay (r240537)

    Accessibility

    • Added support for returning relative frames for accessibility (r240219)
    • Changed the AXSelected attribute on RadioButton to not be settable (r240205)

    Layout

    • Fixed the position of layers nested inside of composited overflow-scroll (r240271)
    • Updated animations to only trigger layer re-compositing when necessary (r240012)

    Web API

    • Implemented PerformanceObserver.supportedEntryTypes (r240454)
    • Fixed a bug where sendBeacon to a previously-unvisited HTTPS domain would always fail (r240094)
    • Improved Fetch data: URL handling to align with standards (r240706)

    WebDriver

    • Added support for running subtests (r240649)
    • Added support for simulating single touches to Automation.performInteractionSequence (r240554)
    • Updated Automation.computeElementLayout to return Visual Viewport-aware coordinates (r240195)
    • Updated POST commands to require a valid parameters object (r240648)

    Web Inspector

    • Added Changes sidebar panel to Elements tab (r240559, 240699)
    • Added support for CSS Color 4 color syntaxes (r240550)
    • Added supports key to test/group for compatibility in the Audit tab (r240469)
    • Added a mechanism for Web Inspector to edit page settings on a remote target (r240540)
    • Added a mechanism for Web Inspector to edit page WebRTC settings on a remote target (r240644, r240664)
    • Added a mechanism for Web Inspector to edit the user agent of a remote target (r240549)
    • Added grouped CSS rules by resource to the Changes sidebar panel (r240741)
    • Added a CPU usage timeline in the Timelines tab (r240457)
    • Added new mechanisms for getting related Accessibility nodes and properties for a given node for audits in the Audit tab (r240277, r240309)
    • Changed the style of the device settings override popover content (r240737)
    • Changed the Elements tab to toggle visibility for all selected nodes (r240639)
    • Enabled computed style cascades by default in the Styles sidebar (r240691)
    • Excluded Debugger Threads from CPU usage values in Web Inspector (r240522)
    • FIxed a bug that prevented audits from being imported in the Audit tab (r240538)
    • Improved Dark Mode appearance within Memory timeline (r240518)
    • Improved invalid Audit and Recording JSON error messages (r240471)
    • Updated the Memory Timeline View to be responsive when resizing (r240763)
    • Updated the Audit tab to ensure that each test run creates its own injected objects (r240188)
    • Updated the Network waterfall column to redraw when adding or removing new columns (r240347)
    • Updated Web Inspector to show the uncaught exception view for unhandled promise rejections (r240499)

    CSS

    • Updated CSS Grid to properly handle static positions of absolute-positioned inside grid items (r240333)
    • Implemented flow-relative inset properties (r240334)
    • Implemented flow-relative margin, padding and border shorthands (r240251)
    • Changed to reject unitless length quirk in the inset shorthand (r240588)

    Browser Changes

    • Removed support for the expired Do Not Track standard to prevent potential use as a fingerprinting variable.

    Bug Fixes

    • Fixed snapshots removed too late after swiping back on Twitter (r240542)
    • Fixed history navigations to twitter.com that were leading to a 403 HTTP error (r240750)
    • Fixed a loading hang that could occur on history navigation (r240725)
    • Updated the MIME-type parser (r240591)

    February 06, 2019 06:00 PM

    February 04, 2019

    IntersectionObserver in WebKit

    Surfin’ Safari

    Web authors often need to know when a particular element on a page becomes visible. The element could be an ad or a video whose viewability we want to compute. Or we might want to defer loading an image until it is visible. A common way to solve this problem is using polling, periodically computing the position of an element with respect to the viewport. However, polling is inefficient, wastes battery life, and doesn’t work for cross-origin content. The IntersectionObserver API offers a better solution to this problem, and is now available in Safari Technology Preview, macOS 10.14.4 beta, and iOS 12.2 beta.

    API Overview

    Each IntersectionObserver has a set of target elements and observes the intersection of these targets with a particular root element or with the viewport. When the intersection of a target and root crosses a specified threshold, the observer invokes a callback. Here’s a simple example to get started:

    var intersectionCount = 0;
    function callback(entries) {
        entries.forEach((entry) => {
            if (entry.isIntersecting)
                intersectionCount++;
        });
    }
    var observer = new IntersectionObserver(callback);
    observer.observe(target);
    

    In the example above, we haven’t specified a root element so we’re observing intersections with the viewport. We pass the IntersectionObserver constructor the callback function we want it to use. We’ll get a callback as soon as any portion of the target intersects the viewport, and we’ll get another callback as soon as the target has no intersection with the viewport.

    What if we want to know when all or most of an element is visible, not just any portion of it? We can specify an intersection threshold when constructing the observer:

    observer = new IntersectionObserver(callback, { threshold: 0.95 } );
    

    Now, we’ll get a callback once 95% of a target intersects the viewport and another callback once the intersection ratio drops below 95%.

    Extending this idea, we can provide a list of thresholds that we’re interested in:

    observer = new IntersectionObserver(callback, {
        threshold: [ 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1 ] });
    

    Here’s a live demo that uses a list of thresholds. Scroll the target into view, and then slowly scroll it out of view and back into view, and notice observations getting logged as the intersection ratio crosses a threshold.

    The same idea works for a target that’s inside an iframe (demo). In this case, there are two ways to scroll the target out of view — scrolling the window so that the iframe is out of view, and scrolling the iframe itself — and both will trigger IntersectionObserver callbacks.

    Suppose we want advance notice that an element is about to become visible. The IntersectionObserver constructor lets us specify a margin by which to expand the root when computing intersections.

    observer = new IntersectionObserver(callback, { rootMargin: '100px 0px' } );
    

    When computing intersections, the viewport will be expanded by 100 pixels at the top and bottom, and will not be expanded on the left or right.

    Using negative root margin values, we can shrink the root when computing intersections. For example, in this demo, we shrink the viewport by 50px on all four sides:

    observer = new IntersectionObserver(callback, {
        threshold: [ 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1 ],
        rootMargin: '-50px'
    });
    

    Finally, we can specify a particular element to intersect with rather than the viewport:

    observer = new IntersectionObserver(callback, { root: someElement } );
    

    Let’s now take a look at the entries that get passed to the callback each time it is invoked. Suppose we want to to compute how long an element is at least 80% visible:

    observer = new IntersectionObserver((entries) => {
        entries.forEach((entry) => {
            if (entry.isIntersecting)
                entry.target.startTime = entry.time;
            else if (entry.target.startTime)
                entry.target.duration = entry.time - entry.target.startTime;
        });
    }, { threshold: 0.8 });
    

    Each intersection observer entry is a dictionary containing details about the intersection between a particular target and the root. This includes a timestamp, the intersection rectangle, and an intersection ratio that represents the fraction of the target that’s contained within the intersection rectangle.

    More examples

    Lazy image load

    Suppose we have a long page with many images. Rather than loading all these images at once, including those that are off-screen, we can start with low-resolution placeholder images, and defer loading high-resolution versions until an image starts to intersect (or, using rootMargin, is about to intersect) with the viewport. Here’s a demo illustrating this approach. To make it easier to see the high-resolution images getting loaded, the demo doesn’t use rootMargin, and waits 500ms after an image starts to intersect the viewport before switching to a high-res version.

    Triggering an animation when an element becomes visible

    To ensure that an animation only starts once the animated content is visible, we can trigger the animation inside an IntersectionObserver callback:

    function intersectionCallback(entries) {
        for (let entry of entries) {
            if (entry.isIntersecting) {
                entry.target.style.transform = 'translateX(20px)';
                entry.target.style.transition = 'transform 2s';
            } else {
                entry.target.style.transform = 'none';
                entry.target.style.transition-property = 'none';
            }
        }
    }
    

    Every time a target element starts intersecting the viewport, we’ll start an animation to slide it to the left, and we’ll put the element back to its original position once it no longer intersects. This demo has a few examples of animations that get triggered by intersection observations.

    Feedback

    We’re excited to have IntersectionObserver available in Safari Technology Preview, macOS 10.14.4 beta, and iOS 12.2 beta! Please try it out and file bugs for any issues you run into.

    February 04, 2019 07:45 PM

    January 25, 2019

    Removing Legacy SPDY Protocol Support

    Surfin’ Safari

    This is a guest post from Apple’s CFNetwork team about changes to the URLSession networking framework used by Safari and WebKit on Apple platforms. These changes may affect compatibility with your websites.

    SPDY is a network protocol designed to improve the performance of web browsing and other HTTP-based communication by addressing many limitations of the HTTP/1.1 protocol. Safari gained support for the SPDY protocol in 2014 with Apple’s OS X Yosemite and iOS 8 releases. This was achieved by adding SPDY support to the Foundation networking URLSession API.

    Many components of the SPDY protocol proved to be successful improvements over HTTP/1.1, so the Internet Engineering Task Force (IETF) drafted a new protocol specification, HTTP/2, based on many of the same ideas, along with other improvements. Safari and URLSession added support for the HTTP/2 protocol in 2015 with Apple’s OS X El Capitan and iOS 9 releases.

    Unlike HTTP/2 and HTTP/1.1, the SPDY protocol is not specified by an Internet standard from the IETF. Rather, it is an experimental protocol that served as a stepping stone to HTTP/2. In fact, many browsers and servers that supported SPDY quickly migrated to HTTP/2, and furthermore, have since dropped support for SPDY entirely (e.g., Chrome, Firefox, and Cloudflare).

    Support for the legacy SPDY protocol is deprecated in the recently released Apple Developer Betas of macOS Mojave 10.14.4 and iOS 12.2. In an upcoming release, legacy SPDY protocol support will be completely removed from Safari along with Apple’s URLSession (and deprecated NSURLConnection) API. At the same time, support for the legacy NPN TLS Extension will be removed. NPN, which was added for SPDY, also was never standardized and has been replaced by ALPN.

    Removing legacy SPDY protocol support will have little impact to many users of Safari and other applications using URLSession. Servers that did add support for SPDY continued to support HTTP/1.1, so connections to these servers will continue to work by gracefully switching to using HTTP/1.1 (if upgrades to add HTTP/2 support have not occurred), but may suffer from a loss of performance in doing so. To help identify any unexpected cases where SPDY is still being used, a new deprecation log message is generated by URLSession once per process launch if SPDY is negotiated with a server:

    WARNING: URLSession (or NSURLConnection) is using SPDY protocol, which is deprecated and will be removed in a future release

    This log message is available starting in the Apple Developer Betas of macOS Mojave 10.14.4 and iOS 12.2 and will be found in the system log.

    For many, removal of legacy SPDY support requires no action, as the URLSession API will automatically switch to the best protocol available. However, if you own or operate a web server that still supports the SPDY and/or NPN experimental technologies, we encourage upgrading to HTTP/2 now so that your clients can take advantage of its many benefits. If you have any systems which prevent transitioning from SPDY to HTTP/2, please let us know by contacting our Web Technologies Evangelist or by filing a bug report with details.

    January 25, 2019 05:30 PM

    January 23, 2019

    Release Notes for Safari Technology Preview 74

    Surfin’ Safari

    Safari Technology Preview Release 74 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 239566-239991.

    Fetch API

    • Implemented abortable fetch (r239644)

    Web Animations

    • Changed to compute animation effect timing properties in batch (r239723)

    Web Authentication

    • Changed the nonce in the CTAP kInit command to weak random values (r239852)
    • Imported U2F command and response converters from Chromium (r239665)
    • Added support for U2F HID Authenticators on macOS (r239752)

    WebRTC

    • Enabled MDNS ICE candidate support by default (r239936)

    Media

    • Changed <video> elements to not enter “paused” state when playing to the end over AirPlay (r239857)
    • Defined page media state flags for display capture (r239840)
    • Changed to deactivate audio session whenever possible (r239694)

    CSS

    Web API

    • Added an override of the session configuration for cookieAcceptPolicy (r239832)
    • Enabled visual viewport API by default (r239897)
    • Changed to no longer trigger a main frame load when doing a history navigation if the source and destination history items are clones
    • Fixed Bulgarian TLDs to not punycode-encode URLs with Bulgarian Cyrillic characters (r239967)
    • Fixed an issue where view is unresponsive for 5 seconds after swiping back on sites like quoteunquoteapps.com (r239720)
    • Fixed the parsed protocol of JavaScript URLs with embedded newlines and carriage returns to match the parsed protocol in Chrome and Firefox (r239642)
    • Fixed unwanted page navigation after showing and dismissing contextual menu with control-click (r239686)

    Service Workers

    • Fixed Service Worker fetch handler that resulted in a bad referrer (r239749)
    • Updated Service Worker fetch to obey its referrer policy (r239659)

    JavaScript

    • Optimized Object.prototype.toString (r239612)

    Accessibility

    • Updated the accessibility string for the <hr> HTML tag to use “separator” instead of “rule” (r239613)

    Web Inspector

    • Added a Setting toggle for Source Maps (r239635)
    • Added the ability to allow audits to be enabled or disabled in the Audits tab (r239858)
    • Added the ability for audits to determine whether a give node has event listeners when running an audit (r239987)
    • Added the ability for audits to query for all nodes with a given computed Accessibility role when running an audit (r239986)
    • Changed to automatically disable breakpoints when running an audit in the Audit tab (r239646)
    • Changed the Network tab to show secure connection details per-request (r239698)
    • Changed to use the save sheet instead of a dialog when clicking the “Export” button in the Network tab (r239647)
    • Extended XHR breakpoints to work with Fetch (r239703)
    • Fixed the color picker to accurately show color when the saturation value is not 100% (r239597)
    • Fixed support for “white” as a recognized color keyword (r239690)
    • Fixed text field and completion popover fonts to match in the Events breakpoint popover (r239953)
    • Fixed showing completions for events that start with “DOM” when typing uppercase “DOM” in Event breakpoints (r239955)
    • Fixed vertical alignment of group titles in the Settings tab (r239932)
    • Fixed read-only properties from being able to be deleted in the Styles sidebar (r239585)
    • Fixed pressing down key on an empty value field discarding the completion popover in the Styles sidebar (r239935)
    • Fixed selection becoming lost in the Styles sidebar when Web Inspector is blurred (r239588)
    • Fixed undo to prevent it reverting all changes at once in the Styles sidebar (r239762)
    • Implemented queryObjects Command Line API (r239583)
    • Included globalThis in default JavaScript completions (r239630)
    • Prevented starting a selection when clicking on a property that is being discarded in the Styles sidebar (r239766)

    WebDriver

    • Changed element click and other commands to appropriately handle user prompts (r239610)
    • Updated the Create Window implementation to support the newest specification (r239988)

    Storage

    • Fixed IndexedDB storage of Crypto keys in private browsing mode (r239746)
    • Fixed deleting IndexedDB databases to ensure open databases are not missed (r239960)
    • Made “Disable Web SQL” on by default (r239885)

    Security

    • Fixed CSP violation reports to bypass CSP checks (r239634)

    January 23, 2019 09:45 PM

    January 09, 2019

    Release Notes for Safari Technology Preview 73

    Surfin’ Safari

    Safari Technology Preview Release 73 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 239091-239566.

    Web API

    • Fixed navigator.userAgent in service workers to correctly reflect the customUserAgent set by the client (r239534)
    • Fixed offsetLeft and offsetParent to adjust across shadow boundaries (r239313)
    • Fixed an incorrect value for key property in keydown and keyup events generated holding the Control (⌃) key (r239361)
    • Removed support for <meta http-equiv=set-cookie> (r239342)

    JavaScript

    • Added support for globalThis (r239464)
    • Added support for BigInt in JSON.stringify (r239544)
    • Implemented support for BigInt ValueDiv into DFG (r239158)
    • Implemented DFG and FTL support for typeof in BigInt (r239141)
    • Implemented the “well-formed JSON.stringify” proposal (r239537)
    • Optimized Object.keys by caching key results (r239324)

    CSS

    • Fixed percentages in relative offsets for CSS grid items (r239502)
    • Fixed thick overlines and line-throughs growing in the wrong direction (r239357)
    • Fixed CSS variables to work for colors in the border property (r239516)
    • Prevented attempting to animate invalid CSS properties (r239174)
    • Updated CSS Properties and Values API to use new cycle fallback behavior (r239365)
    • Updated repeated background images with zero size to display the background color (r239504)

    Media

    • Changed EME MediaKeySystemConfiguration distinctiveIdentifier and persistentState to default to optional (r239373)
    • Changed inline WebVTT styles should override styles from Captions settings in System Preferences (r239181)
    • Updated PiP video to switch tabs before returning video to inline (r239447)
    • Changed playing a media element synchronously after setting srcObject to succeed without user gesture (r239189)
    • Fixed a flicker when exiting element-fullscreen (r239475)

    WebRTC

    • Ensured the first video frame of a MediaStream is rendered (r239319)
    • Fixed the MediaStream calculation of width or height when constraints contain only one of them (r239163)
    • Ensured that an ended event fires on MediaStreamTrack when a USB camera is unplugged (r239531)
    • Made RTCRtpSender.setParameters activate specific encodings (r239252)
    • Made sure getSenders() and getReceivers() do not return closed transceiver senders or receivers (r239238)
    • Implemented non-timeslice mode encoding for MediaRecorder (r239145)
    • Made sure RTCRtpTransceiver.stopped is true when a remote description with the corresponding m section rejected is applied (r239192)

    Web Animations

    • Changed Web Animations to ensure an animation’s finished state only occurs once when updating animations (r239269)

    Web Authentication

    • Changed userPresence to always be true (r239523)

    WebGPU

    • Implemented WebGPUBindGroupLayoutDescriptor and its supporting dictionaries (r239288)
    • Added basic implementation of vertex buffers with Metal Shading Language in WebGPU (r239139)

    Web Inspector

    • Enabled the display of m3u8 content as text in the Network tab (r239343)
    • Ensured that localized strings in default audits respond to locale changes in the Audit tab (r239449)
    • Fixed clipping of “Clear Filters” button in short windows in the Audit tab (r239394)
    • Fixed descriptions of default tests in the Audit tab (r239433)
    • Provided localization support for the display of “percentage passed” results in the Audit tab (r239404)
    • Fixed autoformat for minified JavaScript (r239467)
    • Fixed alignment of path view in the Canvas tab (r239312)
    • Fixed initial focus state of the recording auto-capture input in the Canvas tab (r239420)
    • Fixed selected text color when Inspector window is inactive in dark mode (r239386)
    • Fixed colors of ThreadTreeElement status icon when hovered in dark mode (r239430)
    • Fixed background color of Type profiler popovers in dark mode (r239484)
    • Improved readability for tables containing object previews in dark mode (r239443)
    • Fixed which element gets selected after pressing an arrow key after undoing deletion of a DOM node in the Elements tab (r239405)
    • Fixed Charles Proxy errors when opening some HAR files exported from Safari (r239478)
    • Fixed disappearing reload button after changing more than one setting (r239179)
    • Fixed the spacing of “E” icon on the debugger dashboard (r239432)
    • Fixed the rendering of column sort controls for dark mode in the Layers tab (r239385)
    • Increased the default column width for some localizations in the Layers tab (r239345)
    • Fixed the alignment of experimental setting editors for some localizations in the Settings tab (r239521)
    • Fixed a problem in the Styles sidebar where focus was lost when the inspector is blurred (r239527)
    • Fixed shift-clicking a color-swatch to change formats in the Styles sidebar (r239413)
    • Fixed data corruption when toggling selected properties in the Styles sidebar (r239251)
    • Improved the usability of the Computed styles sidebar when the panel is narrow (r239399)
    • Made the “esc” key select the entire property when editing the name or value in the Styles sidebar (r239481)
    • Prevented table selection from becoming corrupted when deleting selected cookies (r239175)
    • Fixed labelling of Intersection Observer callbacks in the Timelines tab (r239397)

    January 09, 2019 06:00 PM

    January 08, 2019

    Carlos García Campos: Epiphany automation mode

    Igalia WebKit

    Last week I finally found some time to add the automation mode to Epiphany, that allows to run automated tests using WebDriver. It’s important to note that the automation mode is not expected to be used by users or applications to control the browser remotely, but only by WebDriver automated tests. For that reason, the automation mode is incompatible with a primary user profile. There are a few other things affected by the auotmation mode:

    • There’s no persistency. A private profile is created in tmp and only ephemeral web contexts are used.
    • URL entry is not editable, since users are not expected to interact with the browser.
    • An info bar is shown to notify the user that the browser is being controlled by automation.
    • The window decoration is orange to make it even clearer that the browser is running in automation mode.

    So, how can I write tests to be run in Epiphany? First, you need to install a recently enough selenium. For now, only the python API is supported. Selenium doesn’t have an Epiphany driver, but the WebKitGTK driver can be used with any WebKitGTK+ based browser, by providing the browser information as part of session capabilities.

    from selenium import webdriver
    
    options = webdriver.WebKitGTKOptions()
    options.binary_location = 'epiphany'
    options.add_argument('--automation-mode')
    options.set_capability('browserName', 'Epiphany')
    options.set_capability('version', '3.31.4')
    
    ephy = webdriver.WebKitGTK(options=options, desired_capabilities={})
    ephy.get('http://www.webkitgtk.org')
    ephy.quit()
    

    This is a very simple example that just opens Epiphany in automation mode, loads http://www.webkitgtk.org and closes Epiphany. A few comments about the example:

    • Version 3.31.4 will be the first one including the automation mode.
    • The parameter desired_capabilities shouldn’t be needed, but there’s a bug in selenium that has been fixed very recently.
    • WebKitGTKOptions.set_capability was added in selenium 3.14, if you have an older version you can use the following snippet instead
    from selenium import webdriver
    
    options = webdriver.WebKitGTKOptions()
    options.binary_location = 'epiphany'
    options.add_argument('--automation-mode')
    capabilities = options.to_capabilities()
    capabilities['browserName'] = 'Epiphany'
    capabilities['version'] = '3.31.4'
    
    ephy = webdriver.WebKitGTK(desired_capabilities=capabilities)
    ephy.get('http://www.webkitgtk.org')
    ephy.quit()
    

    To simplify the driver instantation you can create your own Epiphany driver derived from the WebKitGTK one:

    from selenium import webdriver
    
    class Epiphany(webdriver.WebKitGTK):
        def __init__(self):
            options = webdriver.WebKitGTKOptions()
            options.binary_location = 'epiphany'
            options.add_argument('--automation-mode')
            options.set_capability('browserName', 'Epiphany')
            options.set_capability('version', '3.31.4')
    
            webdriver.WebKitGTK.__init__(self, options=options, desired_capabilities={})
    
    ephy = Epiphany()
    ephy.get('http://www.webkitgtk.org')
    ephy.quit()
    

    The same for selenium < 3.14

    from selenium import webdriver
    
    class Epiphany(webdriver.WebKitGTK):
        def __init__(self):
            options = webdriver.WebKitGTKOptions()
            options.binary_location = 'epiphany'
            options.add_argument('--automation-mode')
            capabilities = options.to_capabilities()
            capabilities['browserName'] = 'Epiphany'
            capabilities['version'] = '3.31.4'
    
            webdriver.WebKitGTK.__init__(self, desired_capabilities=capabilities)
    
    ephy = Epiphany()
    ephy.get('http://www.webkitgtk.org')
    ephy.quit()
    

    By carlos garcia campos at January 08, 2019 05:22 PM

    December 19, 2018

    Release Notes for Safari Technology Preview 72

    Surfin’ Safari

    Safari Technology Preview Release 72 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 238607-239091.

    Web Authentication

    • Added support for CTAP HID authenticators on macOS (r238729)

    CSS Painting API

    • Added CSS Painting API support for multiple worklets. (r239067)
    • Changed the CSS Painting API to pass this to the paint callback, and repaint when properties change. (r238686)
    • Updated the CSS Painting API to allow image values in inputProperties (r238872)
    • Updated the CSS Painting API to scale the display list when drawing (r238839)

    Web API

    • Changed CSS subresource loads to not be observable from resource timing if the stylesheet is opaque (r238698)
    • Changed overflow scrolling layers to be self-painting (r238725)
    • Changed to prevent computing animated values when there is no relevant animation (r239090)
    • Fixed XMLHttpRequest to prevent removing spaces from content-types before processing (r239040)
    • Fixed HEAD requests becoming changed to GET requests after 301, 302, and 303 redirects (r238891)
    • Fixed document.lastModified to work for non-rendered documents (r238628)
    • Fixed IndexedDB with binary data (Uint8Array) and autoIncrement key in store (r238677)
    • Modernized version checks for same site cookie support (r238614)
    • Restricted the DeviceMotion and DeviceOrientation APIs to secure contexts (r239087)

    WebRTC

    • Added support for firing the devicechange event when more capture device information is revealed when getUserMedia is granted by user (r238796)
    • Fixed the devicechange event to not fire in frames that can’t access capture devices (r239006)
    • Changed scaled video frames to be resized in letterbox mode (r239016)
    • Fixed DataChannels created asynchronously to open properly (r239064)
    • Fixed calling getUserMedia in a link that was opened with target="_blank" to work properly the second time (r238873)
    • Fixed the case of a sender created through addTransceiver whose track is populated using addTrack (r238845)

    Web Inspector

    • Enabled selection of multiple properties by default in the Styles sidebar (r238649)
    • Fixed $0 being shown for all selected elements in the Elements tab (r238859)
    • Fixed jumping from Computed to Styles to correctly select the property in the Elements tab (r238762)
    • Fixed the overlay bounds rulers to match the element when the page is scrolled (r239079)
    • Fixed disclosure triangles for RTL to be flipped and aligned right (r238711)
    • Fixed the selection of properties of read-only rules in the Styles sidebar (r238813)
    • Updated the selected DOM node scope highlight to improve contrast in the Elements tab (r238671)
    • Updated ⌘A to select all visible nodes in the Elements tab (r238858)
    • Fixed “Add Default Audits” shown when there are no filter results in the Audit tab (r238831)
    • Fixed empty results folder shown after clearing filters in the Audit tab (r238648)
    • Fixed navigating while tests are running to properly stop and clear all results in the Audit tab (r238841)
    • Fixed the sidebar briefly showing “No Filter Results” before populating default audits in the Audit tab (r238842)
    • Updated to save the expanded state of test groups in the Audit tab (r238843)
    • Updated test tree elements to start out collapsed in the Audit tab (r238822)
    • Updated tests to support async operations in the Audit tab (r238850)

    Web Share API

    • Fixed the share overlay to properly stick to the Safari window (r238941)

    Media

    • Fixed the lifetime of the HTMLMediaElement to be properly handled in asynchronous actions (r238788)

    JavaScript

    • Enabled .mjs content when loaded from file:// (r238673)
    • Added BigInt support for logic operations (r238861)
    • Implemented BigInt support for << and >> (r238790)

    Service Workers

    • Fixed updating a service worker during a navigation load sometimes causing the load to fail (r238683)

    December 19, 2018 06:00 PM

    December 08, 2018

    Philippe Normand: Web overlay in GStreamer with WPEWebKit

    Igalia WebKit

    After a year or two of hiatus I attended the GStreamer conference which happened in beautiful Edinburgh. It was great to meet the friends from the community again and learn about what’s going on in the multimedia world. The quality of the talks was great, the videos are published …

    By Philippe Normand at December 08, 2018 02:09 PM

    December 05, 2018

    Release Notes for Safari Technology Preview 71

    Surfin’ Safari

    Safari Technology Preview Release 71 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 237919-238607.

    Dark Mode

    • Enabled Dark Mode CSS Support by default in the Experimental Features menu (r238213)
    • Added experimental support for a supported-color-schemes CSS property (r238001)
    • Updated prefers-color-scheme media query matching to better match the draft spec (r238200)
    • Changed the default document background and text colors in dark mode and when dark is listed as a supported-color-schemes on the document or body element (r238212)
    • Added support for forcing color scheme (dark or light mode) in Web Inspector’s Elements tab (r238570)

    Web Animations

    • Enabled Web Animations by default in the Experimental Features menu (r238095)

    Intersection Observer

    • Enabled Intersection Observer by default in the Experimental Features menu (r238129)

    Web Authentication

    • Added Web Authentication as an experimental feature with support for USB-based CTAP2 devices (r238466)

    Web API

    • Fixed click event and touch event listeners on the body element (r237978)
    • Fixed input element gaining focus when a selectstart event listener on the document prevents the default action (r238409)
    • Fixed phantom focus events and blur events firing when clicking between text input fields with addEventListener (r238440)
    • Fixed Content-Type parameter values to allow empty quoted strings (r238124)
    • Changed CSS Painting API to pass size, arguments, and input properties to the paint callback (r237981)

    SVG

    • Fixed incorrectly sized monospace text when an SVG is included as an <img> (r238447)
    • Fixed SVG <use> element inside a shadow tree to be able to reference an element in the same tree (r238524)
    • Fixed updating href on textPath to properly update its rendering (r238464)

    CSS

    • Unprefixed text-decoration CSS3 properties (r238002)
    • Fixed normal-flow-only flex items to correctly respect z-index (r238058)
    • Fixed overlays with -webkit-overflow-scrolling:touch to become scrollable after added text makes it taller (r238266)

    Rendering

    • Made compositing updates incremental (r238090)
    • Changed to update the transform of a composited layer when the layer also needs repainted (r238170)
    • Fixed the caret disappearing at the end of a password field when the caps lock indicator is shown (r238522)
    • Fixed invisible dotted underlines that skip descenders (r237948)
    • Added support for dynamic pseudo-classes on elements with display: contents (r238097)
    • Fixed positioned text underlines that can look like a strike-through (r237955)

    Intelligent Tracking Prevention

    • Enabled a cap on partitioned cache max age (r237949)

    WebRTC

    • Required <iframe allow="display"> for an iframe to use getDisplayMedia (r238091)
    • Added support for RTCRtpCodecParameters.sdpFmtpLine (r238172)
    • Added support for transportStats and peerConnectionStats (r238171)
    • Enabled setting RTCRtpTransceiver.direction (r237974)
    • Fixed RTCPeerConnection.getTransceivers to always expose all transceivers (r238150)

    Web Inspector

    • Added support for the tab bar to automatically add a new Audit tab for running page audits when enabled in Web Inspector Experimental settings (r238500)
    • Added default tests to the Audit tab (r238334)
    • Added support for drag-and-drop for importing Audits in the Audits tab and Recordings in the Canvas tab (r238198)
    • Changed to not start the Canvas tab with auto-capture enabled (r238169)
    • Added support for the copy keyboard shortcut and context menu to the Cookies table (r238569)
    • Fixed the Debugger Popover to work when paused in a Worker (r238003)
    • Fixed Elements tab to allow selecting and deleting multiple DOM nodes (r238602)
    • Fixed missing minimum sizes in Network graphs that could cause no graph to be shown (r238168)
    • Added secure certificate details per-request in the Network tab (r238122)
    • Added a button to show the system certificate dialog in the Network tab (r238350)
    • Fixed a potential issue for watch expressions that never show results (r238005)
    • Added support for ⌘A (Command-A) to select all properties in the Styles sidebar (r238135)
    • Added support for ⇧Click (Shift-click) on a property to extend the selection in the Styles sidebar (r238201)
    • Fixed inline swatches when multiple properties selection is enabled in the Styles sidebar (r238120)
    • Changed Pressing ⇥ (Tab) or Enter to start editing the focused property in the Styles sidebar (r238096)
    • Added support to allow expanding properties to show list of overridden values in the Computed sidebar (r238589)
    • Fixed the experimental Computed sidebar in Dark Mode (r238594)
    • Updated tables to recalculate scrollable height when resized (r238203)
    • Updated tables to support Select All ⌘A (Command-A) (r238140)
    • Updated tables to support extending the row selection with the ⇧ (Shift) key (r238121)
    • Updated tables with no selection to select the first or last row when pressing the down or up arrow key (r238139)
    • Added a Media timeline in the Timelines tab (r238484)
    • Updated TreeOutline to re-use multiple-selection logic from Table (r238599)
    • Updated TreeOutline to prevent selecting the first child of an expanded element when pressing the right arrow key (r238305)
    • Added a button to clear filters with “No Filter Results” in the Navigation sidebar (r238502)
    • Made the “Reload Web Inspector” button more reliable when toggling experimental features (r238378)

    CSS Grid

    • Fixed absolute positioned child getting incorrectly sized when using auto-fit, generating spurious collapsed tracks (r238491)
    • Fixed align-self:center and position:sticky to work together (r238551)
    • Fixed grid container sizing under min-content height (r238488)
    • Implemented Baseline Alignment for grid items (r238457)

    Media

    • Fixed frame re-ordering preventing iframes from getting enqueued (r238086)
    • Enabled external playback for video in element fullscreen (r238327)
    • Fixed SourceBuffer to prevent throwing an error when appending a second init segment after changeType() (r238054)

    Payment Request

    • Fixed canMakePayment() to not consider serialized payment method data (r238042)

    JavaScript

    • Added BigInt support into ValueAdd (r237972)

    December 05, 2018 06:00 PM

    November 14, 2018

    Release Notes for Safari Technology Preview 70

    Surfin’ Safari

    Safari Technology Preview Release 70 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 237630-237919.

    Web Animations

    • Implemented getTiming() and updateTiming() (r237853)
    • Implemented Animation.updatePlaybackRate() (r237854)
    • Changed to not reset pending tasks when setting a null effect (r237856)
    • Made document.getAnimations() return declarative animations in the correct order (r237726)
    • Updated the API to allow the auto composite value (r237855)
    • Updated the Web Animations API to remove all of the ReadOnly interfaces (r237852)

    Intersection Observer

    • Added support to allow cross-document intersection observing (r237737)
    • Fixed isIntersecting computation when 0 is not a threshold (r237798)
    • Updated to account for CSS zoom when computing client rects (r237862)

    CSS

    • Added CSS Custom Properties API Support for syntax="*" and <length> with proper cycle handling (r237697)
    • Fixed <picture> container to update when the prefers-color-scheme media query changes (r237878)
    • Fixed CSS grid elements with justify-content:space-around that have extra whitespace (r237884)
    • Implemented text-underline-offset and text-decoration-thickness (r237903, r237835)

    WebRTC

    • Added support for sender and receiver getCapabilities (r237911)
    • Added onremovetrack support for RTCPeerConnection removed tracks (r237908)
    • Changed MediaRecorder to fire dataavailable event when all tracks are ended and stop() is called (r237642, r237650, r237675)
    • Fixed calling sender.replaceTrack() twice producing a new transceiver and its corresponding m= section (r237910)
    • Fixed sender.replaceTrack() fails with InvalidStateError if the transceiver.direction is “inactive” (r237916)
    • Fixed RTCTrackEvent.streams should be SameObject (r237688)
    • Updated to prevent revealing device IDs until the user has granted permission to capture (r237643)
    • Updated enumerateDevices to not expose devices that are not available to getUserMedia (r237770)
    • Updated to prevent prompting the user again after denying a getDisplayMedia request (r237879)

    Rendering

    • Fixed requestAnimationFrame causing bad location of position:fixed inside overflow:auto and iframe (r237754)
    • Fixed overlays with -webkit-overflow-scrolling:touch to become scrollable after added text makes it taller (r237849)

    Web Inspector

    • Added low-power video playback enter and exit events in Timelines and Network waterfalls (r237669)
    • Added support to show save and restore stack for recorded 2D Canvases (r237808)
    • Added missing context menu items for links in the Styles sidebar (r237660)
    • Added support to toggle selected properties by pressing Space or ⌘/ (Command-/) in the Styles sidebar (r237692)
    • Added support for multiple selection and multiple deletion of cookie records (r237746)
    • Created a setting for auto-recording newly created contexts in the Canvas tab (r237670)
    • Implemented copying and deletion of multiple properties in the Styles sidebar (r237659)
    • Updated to capture changes to <canvas> (i.e., canvas size) that would affect the recorded context in the Canvas tab (r237777)

    Storage

    • Added a storage limit for IndexedDB (r237700)
    • Changed IndexedDB to allow storing RTCCertificate (r237779)

    Media

    • Fixed MediaSource.isTypeSupported('video/mp4; codecs="hvc1.1.6.L60.B0') getting improperly rejected (r237655)

    Payment Request

    • Changed the errorFields of PaymentResponse.retry() to be optional (r237776)

    Security UI

    • Added a warning in the Smart Search field when loading non-secure pages

    November 14, 2018 06:00 PM

    November 12, 2018

    Web High Level Shading Language

    Surfin’ Safari

    This article is introducing a new graphics shading language for the Web named Web High Level Shading Language (WHLSL, pronounced “whistle”). The language is insprired by HLSL, the dominant shading language for graphics app developers. It extends HLSL for the Web platform to be safe and secure. It’s easy to read and write, and is well-specified using formal techniques.

    Background

    Over the past few decades, 3D graphics have changed significantly, and the APIs programmers use to write 3D applications have also changed accordingly. Five years ago, state-of-the-art graphics applications would use OpenGL to perform their rendering. However, the past few years have seen a shift in the 3D graphics industry toward newer, lower-level graphics frameworks that better match the behavior of real hardware. In 2014, Apple created the Metal framework, which lets iOS and macOS apps use the full power of the GPU. In 2015, Microsoft created Direct3D 12, a major update to Direct3D which allows for console-level efficiency for rendering and compute. In 2016, the Khronos Group published the Vulkan API, which is primarily used on Android, that offers similar advantages.

    Just like how WebGL brought OpenGL to the Web, the Web community is pursuing bringing this type of new, low-level 3D graphics API to the platform. Last year, Apple established the WebGPU Community Group inside the W3C to standardize a new 3D graphics API which provides the benefits of these native APIs, but is also suitable for the Web environment. This new Web API is implementable on top of Metal, Direct3D, and Vulkan. All of the major browser vendors are participating and contributing to the standardization effort.

    Each of these modern 3D graphics APIs uses shaders, and WebGPU is no different. Shaders are programs that take advantage of the specialized architecture of GPUs. In particular, GPUs are better than CPUs at heavy parallel numerical processing. To take advantage of both architectures, modern 3D apps use a hybrid design, using both the CPU and the GPU for different tasks. By leveraging the best traits of each, modern graphics APIs provide a powerful framework for developers to create complex, rich, and fast 3D apps. Apps designed for Metal use the Metal Shading Language, apps designed for Direct3D 12 use HLSL, and apps designed for Vulkan use SPIR-V or GLSL.

    Language Requirements

    Just like its native counterparts, WebGPU needs a shader language. This language needs to meet several requirements that make it well-tailored for the Web platform.

    The language needs to be safe. No matter what the application does, the shader must only be able to read or write data from the Web page’s domain. Without this guarantee, a malicious website could run a shader that reads pixels out of other parts of your screen, even from native apps.

    The language needs to be well-specified. The language specification has to be explicit about whether every single possible string of characters is a valid program or not. As with all other Web formats, a shading language for the Web must be precisely specified to guarantee interoperability between browsers.

    The language also needs to be well-specified so that it can be used as a compilation target. Many rendering teams write shaders in their own custom in-house language, and then cross-compile to whichever language is necessary. For this reason, the language should have a reasonably small set of unambiguous grammar and type checking rules that compiler writers can reference when emitting this language.

    This language needs to be translatable to Metal Shading Language, HLSL (or DXIL), and SPIR-V. This is because WebGPU is designed to work on top of Metal, Direct3D 12, and Vulkan, so the shaders need to be able to be represented in a form that each of those APIs can accept.

    The language needs to be performant. The entire reason developers want to use the GPU in the first place is for performance. The compiler itself needs to run quickly, and programs produced by the compiler need to run efficiently on real GPUs.

    This language needs to evolve with the WebGPU API. WebGPU features such as the binding model and tessellation model interact deeply with the shading language. Though it is feasible to have the language developed independently of the API, having the WebGPU API and shading language in the same forum ensures the goals are shared, and makes development more streamlined.

    The language needs to be easy for a developer to read and write. There are two pieces to this: firstly, the language should be familiar to both GPU programmers and CPU programmers. GPU programmers are important clients because they have experience writing shaders. CPU programmers are important because GPUs are increasingly being used for purposes beyond rendering, including machine learning, computer vision, and neural networks. For them, the language should be compatible with familiar programming language concepts and syntax.

    The second piece to this is that the language should be human-readable. The culture of the Web is one where anyone can start writing a webpage with just a text editor and a browser. This democratization of content is one of the Web’s greatest strengths. This culture has created a rich ecosystem of tools and inspectors, where tinkerers can investigate how any webpage works simply by View-Source. Having a single canonical human-readable language will greatly aid in community adoption of the WebGPU API.

    All the major languages used on the web today are human-readable, with one exception. The WebAssembly Community Group expected that parsing a bytecode would be more performant than parsing a text language. However, that turned out not to be true; Asm.js, which is JavaScript source, is still faster than WebAssembly for many use cases.

    Similarly, using a bytecode format such as WebAssembly doesn’t obviate the browser from needing to run optimization passes over the source code. Every major browser runs optimization passes on the bytecode prior to execution. Unfortunately, the desires of simpler compilers never ended up panning out.

    There is active debate in the Community Group about whether or not this human-readable language should be the one that’s natively accepted by the API, but the group agrees that the language that shaders are written in should be easily readable and writable.

    A New Language? Really?

    While there are a number of existing languages, none have been designed with both the Web and modern graphics applications in mind, and none that address the requirements listed above. Before we describe WHLSL, let’s look at some existing languages.

    Metal Shading Language is very similar to C++, which means it has all the power of bit-casts and raw pointers. It’s extremely powerful; the same source code can even be compiled for CPUs and GPUs. It’s extremely easy to port existing CPU-side code to Metal Shading Language. Unfortunately, all this power has some downsides. In Metal Shading Language, you could, for example, write a shader that casts a pointer to an integer, adds 17, casts it back to a pointer, and dereferences it. This is a security problem because it means the shader could access any resource that happens to be in the address space of the application, which is contrary to the Web’s security model. Theoretically, it could be possible to specify a dialect of Metal Shading Language that doesn’t have raw pointers, but pointers are so fundamental to the C and C++ languages that the result would be completely unfamiliar. C++ also heavily relies on undefined behavior, so any effort to fully specify each of C++’s numerous features would be unlikely to be successful.

    HLSL is the supported language that Direct3D shaders written in. It’s currently the most popular realtime shading language in the world, and is therefore the most familiar language to graphics programmers. There are multiple implementations, but there is no formal specification, making it difficult to create consistent, interoperable implementations. Nonetheless, given HLSL’s ubiquity, it is valuable to adopt its syntax as much as possible in the design of WHLSL.

    GLSL is the language used by WebGL, and was adopted by WebGL for the web platform. However, reaching cross-browser interoperability was extremely difficult due to incompatibilities in GLSL compilers. There remains a long tail of security and portability bugs with GLSL still being investigated. Also, GLSL is showing its age. It’s limited in that it doesn’t have pointer-like objects, or the ability to have variable length arrays. Its input and outputs are global variables with hardcoded names.

    SPIR-V was designed to be a low-level universal intermediate format for the actual shading languages that developers would use. People do not author SPIR-V; they use a human-readable language, and then convert it into SPIR-V bytecode using a tool.

    There are a few challenges with adopting SPIR-V for the web. First, SPIR-V was not written with security as a first principle, and it’s unclear whether it can be modified to satisfy the security requirements of the web. Forking the SPIR-V language means developers would have to recompile their shaders, possibly being forced to rewrite their source code anyway. Additionally, browsers would still be unable to trust incoming bytecode, and would be required to validate programs to make sure they are not doing anything insecure. And since Windows and macOS/iOS don’t support Vulkan, the incoming SPIR-V would still need to be translated/compiled into another language. Weirdly, this would mean on those two platforms, the starting point and the ending point are both human readable, but the bit in between is obfuscated with no benefit.

    Second, a significant amount of the SPIR-V specification exists inside separate documents known as “execution environments.” A SPIR-V execution environment currently doesn’t exist for the Web, and without one of these execution environments, many critical pieces of SPIR-V are undefined, such as which of the over 50 optional capabilities are supported.

    Third, many graphics applications such as Babylon.js require dynamically modifying shaders at runtime. Using a bytecode format means that these applications would have to include a compiler written in JavaScript that runs in the browser to produce the bytecode from the dynamically created shader. This would significantly increase the bloat of these sites and would lead to worse performance.

    Though JavaScript is the canonical language for the Web, its properties make it a poor candidate for a shading language. One of its strengths is its flexibility, but this dynamism leads to many conditionals and divergent control flow, which GPUs are not designed to execute efficiently. It is also garbage-collected, which is a procedure that definitely isn’t well-suited for GPU hardware.

    WebAssembly is another familiar possibility, but it also doesn’t map well to the architecture of GPUs. For example, WebAssembly assumes a single dynamically-sized heap, but GPU programs operate with access to multiple dynamically-sized buffers. There isn’t a high-performance way to map between the two models without recompiling.

    Therefore, after a fairly exhaustive search for a suitable language, we couldn’t find one which adequately meets the requirements of the project. So, the Community Group is making a new language. Creating a new language is a large task, but we feel there is an opportunity to produce something new that uses modern programming language design principles and fulfills our requirements.

    WHLSL

    WHLSL is a new shading language that fits the Web platform. It’s being developed by the WebGPU Community Group at the W3C, and the group is working on a specification, a compiler, and a CPU-side interpreter to show correctness.

    The language is based on HLSL, but simplifies and extends it. We’d really like existing HLSL shaders to just work as WHLSL shaders. Since WHLSL is a well-specified powerful and expressive shading language, some HLSL shaders will need tweaks to work, but as a result, WHLSL can guarantee safety and other benefits outlined above.

    For example, here is an example vertex shader from Microsoft’s DirectX-Graphics-Samples repository. It works as a WHLSL shader without any changes:

    VSParticleDrawOut output;
    output.pos = g_bufPosVelo[input.id].pos.xyz;
    float mag = g_bufPosVelo[input.id].velo.w / 9;
    output.color = lerp(float4(1.0f, 0.1f, 0.1f, 1.0f), input.color, mag);
    return output;
    

    And here’s the associated pixel shader, which works as a WHLSL shader completely unmodified:

    float intensity = 0.5f - length(float2(0.5f, 0.5f) - input.tex);
    intensity = clamp(intensity, 0.0f, 0.5f) * 2.0f;
    return float4(input.color.xyz, intensity);
    

    Basics

    Let’s talk about the language itself.

    Just like in HLSL, the primitive data types are bool, int, uint, float, and half. Doubles are not supported because they don’t exist in Metal, and software emulation would be too slow. Bools don’t have a particular bit representation and thus cannot be present in shader inputs/outputs or resources. This same restriction is present in SPIR-V, and we’d like to be able to use OpTypeBool in the generated SPIR-V code. WHLSL also includes smaller integral types of char, uchar, short, and ushort, which are available directly in Metal Shading Language, can be specified in SPIR-V by specifying 16 in OpTypeFloat, and can be emulated in HLSL. Emulation of these types is faster than emulation of doubles because the types are smaller and their bit representation is less complicated.

    WHLSL doesn’t provide C-style implicit conversions. We’ve found implicit conversions to be a common source of errors in shaders, and forcing the programmer to be explicit about where the conversions occur eliminates this often frustrating and mysterious class of bugs. This is a similar approach that languages such as Swift have taken. Additionally, a lack of implicit conversions keeps the specification and the compiler simple.

    Just like in HLSL, there are vector types and matrix types such as float4 and int3x4. Rather than add a bunch of “x1” single-element vectors and matrices, we opted to keep the standard library simple, since a single-element vector is already representable as a scalar and a single-element-matrix is already representable as a vector. This is consistent with the desire to eliminate implicit conversions, and requiring an explicit conversion between float1 and float is cumbersome and needlessly verbose.

    So, the following is a valid snippet of a shader:

    int a = 7;
    a += 3;
    float3 b = float3(float(a) * 5, 6, 7);
    float3 c = b.xxy;
    float3 d = b * c;
    

    I mentioned earlier that no implicit conversions are allowed, but you may have noticed in the above snippet, 5 is not written as 5.0. This is because literals are represented as a special type that can be unified with other numeric types. When the compiler sees the above code, it knows the multiplication operator requires the arguments to be the same type, and the first argument is clearly a float. So, when the compiler sees float(a) * 5 it says “well, I know the first argument is a float, so that means I must be using the (float, float) overload, so let’s unify the 5 with the second argument, and thus the 5 becomes a float.” This works even when both arguments are literals, because literals have a preferred type. So, 5 * 5 will get the (int, int) overload, 5u * 5u will get the (uint, uint) overload, and 5.0 * 5.0 will get the (float, float) overload.

    One difference between WHLSL and C is that WHLSL zero-initializes all uninitialized variables at their declaration site. This prevents non-portable behavior across OSes and drivers, or even worse, reading whatever value happened to be there before your shader started executing. It also means that all constructible types in WHLSL have a zero-value.

    Enums

    Because they don’t incur any runtime cost and are extremely useful, WHLSL has native support for enums.

    enum Weekday {
        Monday,
        Tuesday,
        Wednesday,
        Thursday,
        PizzaDay
    }
    

    The underlying type for an enum defaults to int, but you can override the type, e.g. enum Weekday : uint. Similarly, enum values can have an underlying value like Tuesday = 72. Because enums have defined types and values, they can therefore be used in buffers, and they can be casted between their underlying type and the enum type. When you want to refer to a value in code, you qualify it like Weekday.PizzaDay similar to how enum classes work in C++. This means that enum values don’t pollute the global namespace, and values of independent enums won’t collide.

    Structs

    Structs in WHLSL work similarly to HLSL and C.

    struct Foo {
        int x;
        float y;
    }
    

    Simply designed, they avoid inheritance, virtual methods, and access control. It’s impossible to have a “private” member of a struct. Because structs don’t have access control, there is no need for structs to have member functions. Free functions can see every member of every struct.

    Arrays

    Like other shading languages, arrays are value types that are passed and returned from functions by value (aka “copy-in copy-out,” like regular scalars). You make one using the following syntax:

    int[3] x;
    

    Just like any variable declaration, this will zero-fill the contents of the array, and is therefore an O(n) operation. We wanted to put the brackets after the type instead of after the variable name for two reasons:

    1. Putting all type information in a single place makes the parser simpler (avoiding the clockwise/spiral rule)
    2. Avoiding ambiguity when multiple variables are declared in a single statement (e.g. int[10] x, y;)

    One critical way we ensure safety of the language is performing bounds checking on every array access. There are a number of ways we make this potentially expensive operation efficient. Array indexes are uint, which reduce the check to a single comparison. Arrays are not sparsely implemented, and contain a length member which is available at compile-time, making access have near-zero cost.

    Whereas arrays are value types, WHLSL achieves reference semantics using two other types: safe pointers and array references.

    Safe Pointers

    The first is the safe pointer. Some form of reference semantics, which is the behavior pointers allow for, are used in almost every CPU-side programming language. Including pointers in WHLSL will make it easier for developers to migrate existing CPU-side code to the GPU, thereby allowing for easy porting of things like machine learning, computer vision, and signal processing applications.

    To satisfy the safety requirement, WHLSL uses safe pointers, which are guaranteed to either point to something valid or be null. Like C, you can create a pointer to an lvalue by using the & operator and can dereference one by using the * operator. Unlike C, you can’t index through a pointer as-if it were an array. You can’t cast it to and from a scalar value, and it doesn’t have a specific bit pattern representation. Therefore, it can’t exist in a buffer or as a shader input/output.

    Just like in OpenCL and Metal Shading Language, the GPU has different heaps, or address spaces that values can exist within. WHLSL has 4 different heaps: device, constant, threadgroup, and thread. All reference types must be tagged with the address space they point into.

    The device address space corresponds to the majority of memory on the device. This memory is readable and writable, and corresponds to Unordered Access Views in Direct3D and device memory in Metal Shading Language. The constant address space corresponds to a read-only region of memory, typically optimized for data being broadcast to every thread. As such, writing to an lvalue that exists in the constant address space is a compile error. Lastly, the threadgroup address space corresponds to a readable and writable region of memory that is shared between each thread in a threadgroup. It can only be used in compute shaders.

    By default, values exist within the thread address space:

    int i = 4;
    thread int* j = &i;
    *j = 7;
    // i is now 7
    

    Because all variables are zero-initialized, pointers are null-initialized. Therefore, the following is valid:

    thread int* i;
    

    Trying to dereference this pointer will cause either trapping or clamping, as described later.

    Array References

    Array references are similar to pointers, but they can be used with the subscript operator to access multiple elements in the array reference. Whereas arrays’ lengths are known at compile time and must be stated inside the type declaration, an array reference’s length is only known at runtime. Just like pointers, they must be associated with an address space, and they may be nullptr. Just like arrays, they are indexed using uints for single-comparison bounds checks, and they can’t be sparse.

    They correspond to the OpTypeRuntimeArray type in SPIR-V and one of Buffer, RWBuffer, StructuredBuffer, or RWStructuredBuffer in HLSL. In Metal, it is represented as a tuple of a pointer and a length. Just like array accesses, all operations are checked against the array reference’s length. Buffers are passed into the entry points from the API via array references or pointers.

    You can make an array reference from an lvalue by using the @ operator:

    int i = 4;
    thread int[] j = @i;
    j[0] = 7;
    // i is 7
    // j.length is 1
    

    Just as you might expect, using @ on pointer j creates an array reference that points to the same thing as j:

    int i = 4;
    thread int* j = &i;
    thread int[] k = @j;
    k[0] = 7;
    // i is 7
    // k.length is 1
    

    Using @ on an array makes the array reference point to that array:

    int[3] i = int[3](4, 5, 6);
    thread int[] j = @i;
    j[1] = 7;
    // i[1] is 7
    // j.length is 3
    

    Functions

    Functions look very similar to their C counterparts. For example, here is a function in the standard library:

    float4 lit(float n_dot_l, float n_dot_h, float m) {
        float ambient = 1;
        float diffuse = max(0, n_dot_l);
        float specular = n_dot_l < 0 || n_dot_h < 0 ? 0 : n_dot_h * m;
        float4 result;
        result.x = ambient;
        result.y = diffuse;
        result.z = specular;
        result.w = 1;
        return result;
    }
    

    This example shows how similar WHLSL functions are to C: function declarations and calls (e.g. to max()) have similar syntax, arguments and parameters are matched up pairwise in order, and ternary expressions are supported.

    Operators and Operator Overloading

    However, something else is going on here, too. When the compiler sees n_dot_h * m, it doesn’t intrinsically know how to perform that multiplication. Instead, the compiler will turn that into a call to operator*(). Then, the specific operator*() is chosen via the standard function overload resolution algorithm. This is important because it means you can write your own operator*() function, and teach WHLSL how to multiply your own types.

    This even works for operations like ++. Though pre- and post-increment have different behaviors, they both get overloaded to the same function: operator++(). Here’s an example from the standard library:

    int operator++(int value) {
        return value + 1;
    }
    

    This operator will be called for both pre-increment and post-increment, and the compiler is smart enough to do the right thing with the result. This solves the problem that C++ runs into where those operators are distinct, and are differentiated using an extra dummy int argument. For post-increment, the compiler will emit code to save the value to an anonymous variable, call operator++(), assign the result, and use the saved value for further processing.

    Operator overloading is used all over the language. It’s how vector and matrix multiplication is implemented. It’s how arrays are indexed. It’s how swizzle operators work. Operator overloading provides power and simplicity; the core language doesn’t have to know about each of these operations directly because they are implemented by overloaded operators.

    Generated Properties

    WHLSL doesn’t just stop at operator overloading, though. An earlier example included b.xxy where b is a float3. This is an expression that means “make a 3-element vector where the first two elements have the same value as b.x and the third element has the same value as b.y.” So it’s sort of like a member of the vector, except it isn’t actually associated with any storage; instead, it’s computed during the time it’s accessed. These “swizzle operators” are present in every realtime shading language, and WHLSL is no exception. The way they’re supported is by marking them as a generated property, like in Swift.

    Getters

    The standard library includes many functions of the following form:

    float3 operator.xxy(float3 v) {
        float3 result;
        result.x = v.x;
        result.y = v.x;
        result.z = v.y;
        return result;
    }
    

    When the compiler sees a property access to a member that doesn’t exist, it can call the operator passing the object as the first argument. Colloquially, we call this a getter.

    Setters

    The same approach even works for setters:

    float4 operator.xyz=(float4 v, float3 c) {
        float4 result = v;
        result.x = c.x;
        result.y = c.y;
        result.z = c.z;
        return result;
    }
    

    Using setters is very natural:

    float4 a = float4(1, 2, 3, 4);
    a.xyz = float3(7, 8, 9);
    

    The implementation of the setter creates a copy of the object with the new data. When the compiler sees an assignment to a generated property, it calls the setter and assigns the result to the original variable.

    Anders

    A generalization of getters and setters is the ander, which works with pointers. It exists as a performance optimization, so setters don’t have to create a copy of the object. Here’s an example:

    thread float* operator.r(thread Foo* value) {
        return &value->x;
    }
    

    Anders are more powerful than either getters or setters, because the compiler can use anders to implement either reads or assignments. When reading from a generated property via an ander, the compiler invokes the ander and then dereferences the result. When writing to it, the compiler invokes the ander, dereferences the result, and assigns to the result of that. Any user-defined type can have any combination of getters, setters, anders, and indexers; if the same type has an ander and either a getter or a setter, the compiler will prefer using the ander.

    Indexers

    But what about matrices? In most realtime shading languages, matrices aren’t accessed with members corresponding to their columns or rows. Instead, they are accessed using array syntax, e.g. myMatrix[3][1]. Vector types also usually have this kind of syntax. So how does this work? More operator overloading!

    float operator[](float2 v, uint index) {
        switch (index) {
            case 0:
                return v.x;
            case 1:
                return v.y;
            default:
                /* trap or clamp, more on this below */
        }
    }
    

    float2 operator[]=(float2 v, uint index, float a) {
        switch (index) {
            case 0:
                v.x = a;
                break;
            case 1:
                v.y = a;
                break;
            default:
                /* trap or clamp, more on this below */
        }
        return v;
    }
    

    As you can see, indexing uses operators too, and can therefore be overloaded. Vectors get these “indexers” too, so myVector.x and myVector[0] are synonyms for each other.

    Standard Library

    We designed the standard library based on the Microsoft Docs describing the HLSL standard library. The WHLSL standard library mostly includes math operations, which work both on scalar values and element-wise on vectors and matrices. All the standard operators you would expect are defined, including logical and bitwise operations, like operator*() and operator<<(). All the swizzle operators, getters, and setters are defined for vectors and matrices, where applicable.

    One of the design principles of WHLSL was to keep the language itself small so as much as possible could be defined in the standard library. Of course, not all the functions in the standard library can be expressed in WHLSL (like bool operator*(float, float)) but almost everything else is implemented in WHLSL. For example, this function is part of the standard library:

    float smoothstep(float edge0, float edge1, float x) {
        float t = clamp((x - edge0) / (edge1 - edge0), 0, 1);
        return t * t * (3 - 2 * t);
    }
    

    Because the standard library is designed to match HLSL as much as possible, most of the functions in it are already present in HLSL directly. So a compilation of WHLSL’s standard library to HLSL would choose to omit these functions and instead use the built-in versions. This will happen, for instance, for all the vector/matrix indexers — the GPU should never actually see the code above; the code generation step in the compiler should use the intrinsic instead. However, different shading languages have different intrinsics, so every function is defined to allow for correctness testing. Similarly, WHLSL includes a CPU-side interpreter, which uses the WHLSL implementations of these functions when executing WHLSL programs. This is true for every WHLSL function including the texture sampling functions.

    Not every function in HLSL’s standard library is present in WHLSL. For example, HLSL supports printf(). However, implementing such a function in Metal Shading Language or SPIR-V would be quite difficult. We included as many functions from the HLSL standard library as is reasonable in the Web environment.

    Variable Lifetime

    But if there are pointers in the language, how should we deal with use-after-free problems? For example, consider the following snippet:

    thread int* foo() {
        int a;
        return &a;
    }
    …
    int b = *foo();
    

    In languages like C, this code has undefined behavior. So, one solution is for WHLSL to just forbid this kind of structure and throw a compilation error when it sees something like this. However, this would require tracking the values that every pointer could possibly point to, which is a difficult analysis in the presence of loops and function calls. Instead, WHLSL makes every variable behave as if it has global lifetime.

    This means that this WHLSL snippet is completely valid and well-defined, for two reasons:

    1. Declaring a without an initializer will zero-fill it. Therefore, the value of a is well-defined. This zero-filling will occur each time foo() is called.
  • All variables have global lifetime (similar to C’s static keyword). Therefore, a never goes out of scope.

  • This global lifetime is only possible because recursion is disallowed (which is common for shading languages), which means there will not be any reentrancy problems. Similarly, shaders cannot allocate or free memory, so the compiler knows at compile-time every piece of memory that a shader could possibly access.

    So, for example:

    thread int* foo() {
        int a;
        return &a;
    }
    …
    thread int* x = foo();
    *x = 7;
    thread int* y = foo();
    // *x equals 0, because the variable got zero-filled again
    *y = 8;
    // *x equals 8, because x and y point to the same variable
    

    Most variables won’t need to truly be made global, so there isn’t a big impact on performance. If the compiler can prove that it is unobservable whether or not a particular variable actually has global lifetime, the compiler is free to keep the variable local. Because the pattern of returning pointers to locals is discouraged in other languages (indeed, many other shading languages don’t even have pointers), examples like this one will be relatively rare.

    Stages of Compilation

    WHLSL doesn’t make use of a preprocessor as other languages do. In other languages, the preprocessor’s primary purpose is to include multiple source files together. On the web, however, there is no direct file access, and usually the entire shader is presented in one downloaded resource. In many shading languages, the preprocessor is used to conditionally enable rendering features inside a large ubershader, but WHLSL allows for this use case by using specialization constants instead. Moreover, the many variants of preprocessors are incompatible in subtle ways, so the benefit of a preprocessor for WHLSL doesn’t outweigh the complexity of creating a specification for one.

    WHLSL is designed for a two-stage compilation. In our research, we’ve found that many 3D engines want to compile a large corpus of shaders, and each compilation includes large libraries of functions that are duplicated between the different compilations. Instead of compiling these support functions multiple times, a better solution is to compile the entire library once, and then allow a second stage to select which entry points from the library should be used together.

    This two-stage compilation means that as much of the compilation should be done in the first pass, so it isn’t run multiple times for families of shaders. This is the reason entry points in WHLSL are marked as either vertex, fragment, or compute. Letting the first stage of the compilation know which functions are entry points of which type lets more of the compilation occur in the first stage rather than the second stage.

    This second compilation stage also provides a convenient place to specify specialization constants. Recall that WHLSL doesn’t have a preprocessor, which is the traditional way for features to be enabled and disabled in HLSL. Engines often tailor a single shader to a particular situation by enabling a rendering effect or switching out a BRDF with the flip of a switch. The technique of including every rendering option in a single shader, and specializing the single shader based on which effects to enable, is so common it has a name: ubershaders. Instead of preprocessor macros, WHLSL programmers can use specialization constants, which work the same way as SPIR-V’s specialization constants. From the language’s point of view, they are just scalar constants. However, the values for these constants are supplied during this second compilation stage, making it super easy to configure the program at runtime.

    Because a single WHLSL program can include multiple shaders, the inputs and outputs to the shader aren’t represented by global variables in the way that other shading languages do it. Instead, the inputs and the outputs for a particular shader are associated with that shader itself. Inputs are represented as arguments to the shader’s entry point and outputs are represented as the return value of the entry point.

    The following shows how to describe a compute shader entry point:

    compute void ComputeKernel(device uint[] b : register(u0)) {
       …
    }
    

    Safety

    WHLSL is a safe language. This means that it is impossible to access information outside of a website’s origin. One of the ways WHLSL achieves this is by eliminating undefined behavior, as described above regarding uniformity.

    Another way WHLSL achieves safety is by performing bounds checks of array/pointer accesses. There are three ways these bounds checks may occur:

    1. Trapping. When a trap occurs in a program, the shader stage immediately exits, filling in 0s for all of the shader stage’s outputs. The draw call continues, and the next stage of the graphics pipeline gets run.
      Because trapping introduces new control flow, it has an effect on the uniformity of the program. Traps are issued inside bounds checks, which means they are necessarily present in non-uniform control flow. It may be okay for some programs that don’t use uniformity for anything, but in general this makes traps difficult to use.
    2. Clamping. Array index operations can clamp the index to the size of the array. This doesn’t involve new control flow, so it doesn’t have any affect on uniformity. It is even possible to “clamp” a pointer access or a zero-length array access by disregarding writes and returning 0s for reads. This is possible because the set of things you can do with a pointer in WHLSL is limited, so we can simply make each of those operations do some well-defined thing with a “clamped” pointer.
    3. Hardware and Driver Support. Some hardware and drivers already includes a mode where out-of-bounds accesses can’t happen. With this method, the mechanism by which the hardware forbids out-of-bounds accesses is implementation-defined. One example is the ARB_robustness OpenGL Extension. Unfortunately, WHLSL should be runnable on almost all modern hardware, and simply not enough APIs/devices support these kinds of modes.

    Whichever method the compiler uses, it should not affect the uniformity of the shader; in other words, it can’t turn an otherwise valid program into an invalid one.

    In order to determine the best behavior for bounds checks, we ran some performance experiments. We took some of the kernels used in the Metal Performance Shaders framework and made two new versions: one that uses clamping, and one that uses trapping. The kernels we picked were ones that do lots of array accesses: for example, multiplying large matrices. We ran this benchmark on a variety of devices at varying data sizes. We made sure that none of the traps were actually hit and none of the clamps actually had any effect, so we can be sure we were measuring the common case of a correctly-written program.

    We expected trapping to be generally faster, because redundant traps can be eliminated by the downstream compiler. However, we discovered that there wasn’t one clear winner. On some devices, trapping was significantly faster than clamping, and on other devices, clamping was significantly faster than trapping. These results show that the compiler should be able to choose which method is best for the particular device it’s being run on, rather than being forced to always choose one method.

    Chart of iPhone 6 vs iPhone X runtime scores

    Shader Signatures

    WHLSL supports a language feature of HLSL called “semantics.” They are used to identify variables between shader stages and from the WebGPU API. There are four types of semantics:

    • Built-in variables, e.g. uint vertexID : SV_VertexID
    • Specialization constants, e.g. uint numlights : specialized
    • Stage in/out semantics, e.g. float2 coordinate : attribute(0)
    • Resource semantics, e.g. device float[] coordinate : register(u0)

    As described above, WHLSL programs accept their inputs and outputs in the form of function parameters, not global variables.

    However, shaders often have multiple outputs. The most common example of this is the vertex shader passing multiple output values to the interpolator to be fed as inputs into the fragment shader.

    In order to accommodate this, the return value of a shader can be a struct, and the individual fields are treated independently. In fact, this works recursively — the struct can contain another struct, and its members are also treated independently. Nested structs are flattened, and all the fields which aren’t structs are gathered and treated as shader outputs.

    Shader parameters work the same way. An individual parameter can be a shader input, or it can be a struct with a collection of shader inputs. Structs can also contain other structs. Variables inside these structs are treated independently, as if they were additional parameters to the shader.

    After all these structs have been flattened into a set of inputs and a set of outputs, each item in the sets must have a semantic. Each built-in variable must have a particular type and must only be used in a particular shader stage. Specialization constants must only have simple scalar types.

    Stage in/out variables have the attribute semantic rather than the traditional HLSL semantics because many shaders pass around data that don’t match the canned semantics HLSL provides. In HLSL, it’s common to see generic data packed into the COLOR semantic, because COLOR is a float4 and the data fits inside a float4. Instead, the approach both SPIR-V and Metal Shading Language (via [[user(n)]]) take is to assign an identifier to each stage in/out variable, and use the assignments to match the variables between shader stages.

    Resource semantics should be familiar to HLSL programmers. WHLSL includes both resource semantics and address spaces, but both of these have different purposes. The address space of the variable is used to determine which cache and memory hierarchy it should be accessed within. The address space is necessary because it persists even through pointer operations; a device pointer can’t be set to point to a thread variable. In WHLSL, the resource semantic is only used to identify a variable from the WebGPU API. However, for consistency with HLSL, the resource semantic must “match” the address space of the variable it’s being put on. For example, you can’t put register(s0) on a texture. You can’t put register(u0) on a constant resource. Arrays in WHLSL don’t have address space (because they are value types, not reference types) so if an array appears as a shader argument, it is treated as if it was a device resource for the purposes of matching semantics.

    Just like Direct3D, WebGPU has a two-level binding model. Resource descriptors are aggregated into sets, and sets can be switched out in the WebGPU API. WHLSL matches HLSL by modeling this by an optional space parameter inside resource semantics: register(u0, space1).

    “Logical Mode” restrictions

    WHLSL is designed with the requirement that it is compilable to Metal Shading Language, SPIR-V, and HLSL (or DXIL). SPIR-V has many different operating modes, targeted by different embedding APIs. Specifically, we’re interested in the flavor of SPIR-V that Vulkan targets.

    This flavor of SPIR-V is a flavor of SPIR-V called Logical Addressing Mode. In SPIR-V Logical Mode, variables cannot have pointer type. Similarly, a pointer cannot be used in a Phi operation. The result of this is that each pointer must point to exactly one thing for all time; a pointer is simply a name for a value.

    Because WHLSL needs to be compilable to SPIR-V, WHLSL must not be more expressive than SPIR-V. Therefore, WHLSL has some restrictions to make it expressible in SPIR-V Logical Mode. These restrictions aren’t surfaced as an optional mode for WHLSL; instead, they’re part of the language itself. Eventually, we hope these restrictions can be lifted in a future version of the language, but until then, the language is restricted.

    These restrictions are:

    • Pointers and array references must not occur inside device, constant, or threadgroup memory
    • Pointers and array references must not occur inside arrays or array references
    • Pointers and array references must not be assigned outside of their initializer (in their declaration)
    • Functions that return pointers or array references must only have a single return point
    • Ternary expressions must not result in pointers

    With these restrictions, the compiler knows exactly what every pointer points to.

    But not so fast! Recall from above that thread variables have global lifetime, which means they behave as-if they were declared at the beginning of the entry point. What if the runtime gathered all these local variables together, sorted by type, and aggregated all the variables with the same type into arrays? Then, a pointer could simply be an offset into the appropriate array. A pointer can’t be recast to point to a different type in WHLSL, which means the appropriate array is determined statically by the compiler. Therefore, thread pointers don’t need to abide by the restrictions above. But, this technique doesn’t work for the pointers in the other address spaces; it only works for thread pointers.

    Resources

    WHLSL supports textures, samplers, and array references for buffers. Just like in HLSL, texture types in WHLSL look like Texture2D<float4>. The presence of these angle brackets don’t imply templates or generics; the language doesn’t have facilities for those (for simplicity). The only types that are allowed to have them are a finite set of built-in types. This design is a middle ground between allowing these types, which are present in HLSL, but also allowing further development of the language in a way that the Community Group can use the angle bracket characters.

    Depth textures are distinct from non-depth-textures because they are different types in Metal Shading Language, so the compiler needs to know which one to emit when it’s emitting Metal Shading Language. Because WHLSL doesn’t support member functions, texture sampling isn’t done like texture.Sample(…); instead, it’s done with free functions like Sample(texture, …).

    Samplers are not specialized; there is one sampler type for all use cases. You can use this sampler for both depth textures and non-depth textures. Depth textures support things like comparison operations in the sampler. If the sampler is configured to include a depth comparison and it’s used with a non-depth texture, the depth operation is ignored.

    The WebGPU API will automatically emit some resource barriers at particular places, which means the API needs to know which resources are used in a shader. Therefore, the “bindless” model of resources can’t be used. This means that all resources are listed as explicit inputs to a shader. Similarly, the API wants to know which resources are used for reading and which are used for writing; the compiler knows this statically by inspecting the program. There is no language-level support for “const” or a distinction between StructuredBuffer and RWStructuredBuffer because that information is already present in the program.

    Current Work

    The WebGPU community group is working on a formal language specification written with OTT that describes WHLSL with the same level of rigor that other Web languages employ. We’re also working on a compiler that can produce Metal Shading Language, SPIR-V, and HLSL. In addition, the compiler includes a CPU-side interpreter to show correctness of an implementation. Please try it out!

    Future Directions

    WHLSL is still nascent, and there is still a long way to go before the design of the language is complete. We would love to hear from you about your desires, concerns, and use cases! Please feel free to file issues in our GitHub repository about your ideas and thoughts!

    For the first proposal, we wanted to satisfy the constraints outlined at the beginning of this post, yet provide ample opportunity for expanding the language. One natural evolution of the language could add facilities for abstractions of types, like protocols or interfaces. WHLSL includes simple structs with no access control or inheritance. Other shading languages like Slang model type abstractions as a set of methods that must be present inside the struct. However, Slang runs into a problem where it is impossible to make an existing type adhere to a new interface. Once the struct is defined, you can’t add new methods to it; the curly brace has closed the struct forever. This problem is solved with extensions, similar to Objective-C or Swift, which can retroactively add methods into a struct after the struct has been defined. Java solved this problem by encouraging authors to add new classes, called adapters, which only exist to implement an interface, and plumb each call through to the implementing type.

    The WHLSL approach is much simpler; by using free functions instead of struct methods, we can use a system like Haskell’s type classes. Here, a type class defines a set of arbitrary functions that must exist, and a type adheres to the type class by implementing them. A solution like this could potentially be added to the language in the future.

    Wrapping Up

    This describes a new shading language named WHLSL that the W3C’s WebGPU Community Group owns. The goals of the language are satisfied by its familiar, HLSL-based syntax, safety guarantees, and simple, extensible design. As such, it represents the best-supported way to write shaders to be used in the WebGPU API. However, the WebGPU Community Group is unsure whether or not WHLSL programs should be supplied to the WebGPU API directly, or whether they should be compiled to an intermediate form before delivery to the API. Either way, WebGPU programmers should be writing in WHLSL because it fits best with the API.

    Please get involved! We’re doing this work on the WebGPU GitHub project. We’ve been working on a formal specification for the language, a reference compiler to emit Metal Shading Language and SPIR-V, and a CPU-side interpreter to validate correctness. We welcome everyone to try it out, and let us know how it goes!

    For more information, you can contact me at mmaxfield@apple.com or @Litherum, or you can contact our evangelist, Jonathan Davis.

    November 12, 2018 11:21 PM

    Michael Catanzaro: The GNOME (and WebKitGTK+) Networking Stack

    Igalia WebKit

    WebKit currently has four network backends:

    • CoreFoundation (used by macOS and iOS, and thus Safari)
    • CFNet (used by iTunes on Windows… I think only iTunes?)
    • cURL (used by most Windows applications, also PlayStation)
    • libsoup (used by WebKitGTK+ and WPE WebKit)

    One guess which of those we’re going to be talking about in this post. Yeah, of course, libsoup! If you’re not familiar with libsoup, it’s the GNOME HTTP library. Why is it called libsoup? Because before it was an HTTP library, it was a SOAP library. And apparently somebody thought that when Mexican people say “soap,” it often sounds like “soup,” and also thought that this was somehow both funny and a good basis for naming a software library. You can’t make this stuff up.

    Anyway, libsoup is built on top of GIO’s sockets APIs. Did you know that GIO has Object wrappers for BSD sockets? Well it does. If you fancy lower-level APIs, create a GSocket and have a field day with it. Want something a bit more convenient? Use GSocketClient to create a GSocketConnection connected to a GNetworkAddress. Pretty straightforward. Everything parallels normal BSD sockets, but the API is nice and modern and GObject, and that’s really all there is to know about it. So when you point WebKitGTK+ at an HTTP address, libsoup is using those APIs behind the scenes to handle connection establishment. (We’re glossing over details like “actually implementing HTTP” here. Trust me, libsoup does that too.)

    Things get more fun when you want to load an HTTPS address, since we have to add TLS to the picture, and we can’t have TLS code in GIO or GLib due to this little thing called “copyright law.” See, there are basically three major libraries used to implement TLS on Linux, and they all have problems:

    • OpenSSL is by far the most popular, but it’s, hm, shall we say technically non-spectacular. There are forks, but the forks have problems too (ask me about BoringSSL!), so forget about them. The copyright problem here is that the OpenSSL license is incompatible with the GPL. (Boring details: Red Hat waves away this problem by declaring OpenSSL a system library qualifying for the GPL’s system library exception. Debian has declared the opposite, so Red Hat’s choice doesn’t gain you anything if you care about Debian users. The OpenSSL developers are trying to relicense to the Apache license to fix this, but this process is taking forever, and the Apache license is still incompatible with GPLv2, so this would make it impossible to use GPLv2+ software except under the terms of GPLv3+. Yada yada details.) So if you are writing a library that needs to be used by GPL applications, like say GLib or libsoup or WebKit, then it would behoove you to not use OpenSSL.
    • GnuTLS is my favorite from a technical standpoint. Its license is LGPLv2+, which is unproblematic everywhere, but some of its dependencies are licensed LGPLv3+, and that’s uncomfortable for many embedded systems vendors, since LGPLv3+ contains some provisions that make it difficult to deny you your freedom to modify the LGPLv3+ software. So if you rely on embedded systems vendors to fund the development of your library, like say libsoup or WebKit, then you’re really going to want to avoid GnuTLS.
    • NSS is used by Firefox. I don’t know as much about it, because it’s not as popular. I get the impression that it’s more designed for the needs of Firefox than as a Linux system library, but it’s available, and it works, and it has no license problems.

    So naturally GLib uses NSS to avoid the license issues of OpenSSL and GnuTLS, right?

    Haha no, it uses a dynamically-loadable extension point system to allow you to pick your choice of OpenSSL or GnuTLS! (Support for NSS was started but never finished.) This is OK because embedded systems vendors don’t use GPL applications and have no problems with OpenSSL, while desktop Linux users don’t produce tivoized embedded systems and have no problems with LGPLv3. So if you’re using desktop Linux and point WebKitGTK+ at an HTTPS address, then GLib is going to load a GIO extension point called glib-networking, which implements all of GIO’s TLS APIs — notably GTlsConnection and GTlsCertificate — using GnuTLS. But if you’re building an embedded system, you simply don’t build or install glib-networking, and instead build a different GIO extension point called glib-openssl, and libsoup will create GTlsConnection and GTlsCertificate objects based on OpenSSL instead. Nice! And if you’re Centricular and you’re building GStreamer for Windows, you can use yet another GIO extension point, glib-schannel, for your native Windows TLS goodness, all hidden behind GTlsConnection so that GStreamer (or whatever application you’re writing) doesn’t have to know about SChannel or OpenSSL or GnuTLS or any of that sad complexity.

    Now you know why the TLS extension point system exists in GIO. Software licenses! And you should not be surprised to learn that direct use of any of these crypto libraries is banned in libsoup and WebKit: we have to cater to both embedded system developers and to GPL-licensed applications. All TLS library use is hidden behind the GTlsConnection API, which is really quite nice to use because it inherits from GIOStream. You ask for a TLS connection, have it handed to you, and then read and write to it without having to deal with any of the crypto details.

    As a recap, the layering here is: WebKit -> libsoup -> GIO (GLib) -> glib-networking (or glib-openssl or glib-schannel).

    So when Epiphany fails to load a webpage, and you’re looking at a TLS-related error, glib-networking is probably to blame. If it’s an HTTP-related error, the fault most likely lies in libsoup. Same for any other GNOME applications that are having connectivity troubles: they all use the same network stack. And there you have it!

    P.S. The glib-openssl maintainers are helping merge glib-openssl into glib-networking, such that glib-networking will offer a choice of GnuTLS or OpenSSL and obsoleting glib-openssl. This is still a work in progress. glib-schannel will be next!

    P.S.S. libcurl also gives you multiple choices of TLS backend, but makes you choose which at build time, whereas with GIO extension points it’s actually possible to choose at runtime from the selection of installed extension points. The libcurl approach is fine in theory, but creates some weird problems, e.g. different backends with different bugs are used on different distributions. On Fedora, it used to use NSS, but now uses OpenSSL, which is fine for Fedora, but would be a license problem elsewhere. Debian actually builds several different backends and gives you a choice, unlike everywhere else. I digress.

    By Michael Catanzaro at November 12, 2018 04:51 AM

    November 07, 2018

    Release Notes for Safari Technology Preview 69

    Surfin’ Safari

    Safari Technology Preview Release 69 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 237231-237630.

    CSS

    • Enabled Conic Gradients by default (r237402)

    Web API

    • Enabled the <datalist> element by default (r237245)
    • Added new image type for the CSS Paint API (r237276)
    • Changed the CSS Painting API to give a 2D rendering context (r237344)
    • Changed the CSS Painting API to parse paint() and store paint callbacks (r237243)
    • Changed Web Animations to not create a DocumentTimeline to suspend or resume animations (r237231)

    Intersection Observer

    • Handled zero-area intersections (r237284)

    Dark Mode

    • Changed to use a dark appearance scrollbar when the page background is dark or the document supports dark mode (r237466)

    JavaScript

    • Implemented BigInt support for ^ (r237296)

    Media

    • Added support for MediaKeyEncryptionScheme (r237268)
    • Updated to be able to adapt between H.264 streams with and without EditList (r237271)
    • Updated TextTrack cues to be updated more often than every 250ms (r237376)
    • Fixed timestampOffset to prevent introducing floating-point rounding errors to incoming samples (r237274)
    • Fixed the WebVTT region parameter and value to be separated by : (r237270)

    Web Inspector

    • Added corner rounding to the network timing blocks in the Network tab waterfall (r237432)
    • Changed to more aggressively snap timing blocks together in the Network tab (r237430)
    • Fixed CSP JSON request payload without a MIME-type to be pretty-printed (r237396)
    • Added fullscreen enter and exit events in the Timelines and Network waterfalls (r237431)
    • Fixed the Quick Open dialog to show named scripts that appear in the Debugger sidebar (r237327)
    • Fixed the Quick Open dialog for line and column to have a caret indicate the position (r237232)
    • Fixed malformed popovers for function source code (r237401)
    • Prevented the Canvas tab from listening for “space” key press when the tab is not visible (r237560)
    • Improved Canvas Recording loading speed when the “Frame” tree element is expanded (r237436)
    • Updated the Canvas tab to show a warning when the path moves offscreen (r237574)

    WebRTC

    • Changed MediaRecorder to fire a stop event when all tracks are ended (r237311)
    • Changed to handle MDNS resolution of candidates through libwebrtc directly (r237568)

    Payment Request

    • Implemented MerchantValidationEvent.methodName (r237521)
    • Implemented PaymentResponse.retry() (r237597)
    • Updated PaymentRequest.canMakePayment() to resolve to true whenever Apple Pay is available (r237594)

    Storage

    • Added a deprecation warning to the console for Web SQL (r237591)
    • Fixed iteration of cursors skipping records if updated or deleted in IndexedDB (r237590)

    November 07, 2018 06:00 PM

    Michael Catanzaro: Mesa Update Breaks WebKitGTK+ in Fedora 29

    Igalia WebKit

    If you’re using Fedora and discovered that WebKitGTK+ is displaying blank pages, the cause is a bad mesa update, mesa-18.2.3-1.fc29. This in turn was caused by a GCC bug that resulted in miscompilation of mesa.

    To avoid this bug, downgrade to mesa-18.2.2-1.fc29:

    $ sudo dnf downgrade mesa*

    You can also update to mesa-18.2.4-2.fc29, but this build has not yet reached updates-testing, let alone stable, so downgrading is easier for now. Another workaround is to run your application with accelerated compositing mode disabled, to avoid OpenGL usage:

    $ WEBKIT_DISABLE_COMPOSITING_MODE=1 epiphany

    On the bright side of things, from all the bug reports I’ve received over the past two days I’ve discovered that lots of people use Epiphany and notice when it’s broken. That’s nice!

    Huge thanks to Dave Airlie for quickly preparing the fixed mesa update, and to Jakub Jelenik for handling the same for GCC.

    By Michael Catanzaro at November 07, 2018 02:28 AM

    November 03, 2018

    Michael Catanzaro: WebKitGTK+ 2.22.2 and 2.22.3, Media Source Extensions, and YouTube

    Igalia WebKit

    Last month, I attended the Web Engines Hackfest (hosted by Igalia in A Coruña, Spain) and also the WebKit Contributors Meeting (hosted by Apple in San Jose, California). These are easily the two biggest WebKit development events of the year, and it’s always amazing to meet everyone in person yet again. A Coruña is an amazing city, and every browser developer ought to visit at least once. And the Contributors Meeting is a no-brainer event for WebKit developers.

    One of the main discussion points this year was Media Source Extensions (MSE). MSE is basically a way for browsers to control how videos are downloaded. Until recently, if you were to play a YouTube video in Epiphany, you’d notice that the video loads way faster than it does in other browsers. This is because WebKitGTK+ — until recently — had no support for MSE. In other browsers, YouTube uses MSE to limit the speed at which video is downloaded, in order to reduce wasted bandwidth in case you stop watching the video before it ends. But with WebKitGTK+, MSE was not available, so videos would load as quickly as possible. MSE also makes it harder for browsers to offer the ability to download the videos; you’ll notice that neither Firefox nor Chrome offer to download the videos in their context menus, a feature that’s been available in Epiphany for as long as I remember.

    So that sounds like it’s good to not have MSE. Well, the downside is that YouTube requires it in order to receive HD videos, to avoid that wasted bandwidth and to make it harder for users to download HD videos. And so WebKitGTK+ users have been limited to 720p video with H.264 and 480p video with WebM, where other browsers had access to 1080p and 1440p video. I’d been stuck with 480p video on Fedora for so long, I’d forgotten that internet video could look good.

    Unfortunately, WebKitGTK+ was quite late to implement MSE. All other major browsers turned it on several years ago, but WebKitGTK+ dawdled. There was some code to support MSE, but it didn’t really work, and was disabled. And so it came to pass that, in September of this year, YouTube began to require MSE to access any WebM video, and we had a crisis. We don’t normally enable major new features in stable releases, but this was an exceptional situation and users would not be well-served by delaying until the next release cycle. So within a couple weeks, we were able to release WebKitGTK+ 2.22.2 and Epiphany 3.30.1 (both on September 21), and GStreamer 1.14.4 (on October 2, thanks to Tim-Philipp Müller for expediting that release). Collectively, these releases enabled basic video playback with MSE for users of GNOME 3.30. And if you still use of GNOME 3.28, worry not: you are still supported and can get MSE if you update to Epiphany 3.28.5 and also have the aforementioned versions of WebKitGTK+ and GStreamer.

    MSE in WebKitGTK+ 2.22.2 had many rough edges because it was a mad rush to get the feature into a minimally-viable state, but those issues have been polished off in 2.22.3, which we released earlier this week on October 29. Be sure you have WebKitGTK+ 2.22.3, plus GStreamer 1.14.4, for a good experience on YouTube. Unfortunately we can’t provide support for older software versions anymore: if you don’t have GStreamer 1.14.4, then you’ll need to configure WebKitGTK+ with -DENABLE_MEDIA_SOURCE=OFF at build time and suffer from lack of MSE.

    Epiphany 3.28.1 uses WebKitSettings to turn on the “enable-mediasource” setting. Turn that on if your application wants MSE now (if it’s a web browser, it certainly does). This setting will be enabled by default in WebKitGTK+ 2.24. Huge thanks to the talented developers who made this feature possible! Enjoy your 1080p and 1440p video.

    By Michael Catanzaro at November 03, 2018 04:19 AM

    Michael Catanzaro: On WebKit Build Options (Also: How to Accidentally Disable Important Security Features!)

    Igalia WebKit

    When building WebKitGTK+, it’s a good idea to stick to the default values for the build options. If you’re building some sort of embedded system and really know what you’re doing, then OK, it might make sense to change some settings and disable some stuff. But Linux distros are generally well-advised to stick to the defaults to avoid creating problems for users.

    One exception is if you need to disable certain features to avoid newer dependencies when building WebKit for older systems. For example, Ubuntu 18.04 disables web fonts (ENABLE_WOFF2=OFF) because it doesn’t have the libbrotli and libwoff2 dependencies that are required for that feature to work, hence some webpages will display using subpar fonts. And distributions shipping older versions of GStreamer will need to disable the ENABLE_MEDIA_SOURCE option (which is missing from the below feature list by mistake), since that requires the very latest GStreamer to work.

    Other exceptions are the ENABLE_GTKDOC and ENABLE_MINIBROWSER settings, which distros do want. ENABLE_GTKDOC is disabled by default because it’s slow to build, and ENABLE_MINIBROWSER because, well, actually I don’t know why, you always want that one and it’s just annoying to find it’s not built.

    OK, but really now, other than those exceptions, you should probably leave the defaults alone.

    The feature list that prints when building WebKitGTK+ looks like this:

    --  ENABLE_ACCELERATED_2D_CANVAS .......... OFF
    --  ENABLE_DRAG_SUPPORT                     ON
    --  ENABLE_GEOLOCATION .................... ON
    --  ENABLE_GLES2                            OFF
    --  ENABLE_GTKDOC ......................... OFF
    --  ENABLE_ICONDATABASE                     ON
    --  ENABLE_INTROSPECTION .................. ON
    --  ENABLE_JIT                              ON
    --  ENABLE_MINIBROWSER .................... OFF
    --  ENABLE_OPENGL                           ON
    --  ENABLE_PLUGIN_PROCESS_GTK2 ............ ON
    --  ENABLE_QUARTZ_TARGET                    OFF
    --  ENABLE_SAMPLING_PROFILER .............. ON
    --  ENABLE_SPELLCHECK                       ON
    --  ENABLE_TOUCH_EVENTS ................... ON
    --  ENABLE_VIDEO                            ON
    --  ENABLE_WAYLAND_TARGET ................. ON
    --  ENABLE_WEBDRIVER                        ON
    --  ENABLE_WEB_AUDIO ...................... ON
    --  ENABLE_WEB_CRYPTO                       ON
    --  ENABLE_X11_TARGET ..................... ON
    --  USE_LIBHYPHEN                           ON
    --  USE_LIBNOTIFY ......................... ON
    --  USE_LIBSECRET                           ON
    --  USE_SYSTEM_MALLOC ..................... OFF
    --  USE_WOFF2                               ON

    And, asides from the exceptions noted above, those are probably the options you want to ship with.

    Why are some things disabled by default? ENABLE_ACCELERATED_2D_CANVAS is OFF by default because it is experimental (i.e. not great :) and requires CairoGL, which has been available in most distributions for about half a decade now, but still hasn’t reached Debian yet, because the Debian developers know that the Cairo developers consider CarioGL experimental (i.e. not great!). Many of our developers use Debian, and we’re not keen on having two separate sets of canvas bugs depending on whether you’re using Debian or not, so best keep this off for now. ENABLE_GLES2 switches you from desktop GL to GLES, which is maybe needed for embedded systems with crap proprietary graphics drivers, but certainly not what you want when building for a general-purpose distribution with mesa. Then ENABLE_QUARTZ_TARGET is for building on macOS, not for Linux. And then we come to USE_SYSTEM_MALLOC.

    USE_SYSTEM_MALLOC disables WebKit’s bmalloc memory allocator (“fast malloc”) in favor of glibc malloc. bmalloc is performance-optimized for macOS, and I’m uncertain how its performance compares to glibc malloc on Linux. Doesn’t matter really, because bmalloc contains important heap security features that will be disabled if you switch to glibc malloc, and that’s all you need to know to decide which one to use. If you disable bmalloc, you lose the Gigacage, isolated heaps, heap subspaces, etc. I don’t pretend to understand how any of those things work, so I’ll just refer you to this explanation by Sam Brown, who sounds like he knows what he’s talking about. The point is that, if an attacker has found a memory vulnerability in WebKit, these heap security features make it much harder to exploit and take control of users’ computers, and you don’t want them turned off.

    USE_SYSTEM_MALLOC is currently enabled (bad!) in openSUSE and SUSE Linux Enterprise 15, presumably because when the Gigacage was originally introduced, it crashed immediately for users who set address space (virtual memory allocation) limits. Gigacage works by allocating a huge address space to reduce the chances that an attacker can find pointers within that space, similar to ASLR, so limiting the size of the address space prevents Gigacage from working. At first we thought it made more sense to crash than to allow a security feature to silently fail, but we got a bunch of complaints from users who use ulimit to limit the address space used by processes, and also from users who disable overcommit (which is required for Gigacage to allocate ludicrous amounts of address space), and so nowadays we just silently disable Gigacage instead if enough address space for it cannot be allocated. So hopefully there’s no longer any reason to disable this important security feature at build time! Distributions should be building with the default USE_SYSTEM_MALLOC=OFF.

    The openSUSE CMake line currently looks like this:

    %cmake \
      -DCMAKE_BUILD_TYPE=Release \
      -DLIBEXEC_INSTALL_DIR=%{_libexecdir}/libwebkit2gtk%{_wk2sover} \
      -DPORT=GTK \
    %if 0%{?suse_version} == 1315
      -DCMAKE_C_COMPILER=gcc-7 \
      -DCMAKE_CXX_COMPILER=g++-7 \
      -DENABLE_WEB_CRYPTO=OFF \
      -DUSE_GSTREAMER_GL=false \
    %endif
    %if 0%{?suse_version} <= 1500
      -DUSE_WOFF2=false \
    %endif
      -DENABLE_MINIBROWSER=ON \
    %if %{with python3}
      -DPYTHON_EXECUTABLE=%{_bindir}/python3 \
    %endif
    %if !0%{?is_opensuse}
      -DENABLE_PLUGIN_PROCESS_GTK2=OFF \
    %endif
    %ifarch armv6hl ppc ppc64 ppc64le riscv64 s390 s390x
      -DENABLE_JIT=OFF \
    %endif
      -DUSE_SYSTEM_MALLOC=ON \
      -DCMAKE_EXE_LINKER_FLAGS="-Wl,--as-needed -Wl,-z,now -pthread" \
      -DCMAKE_MODULE_LINKER_FLAGS="-Wl,--as-needed -Wl,-z,now -pthread" \
      -DCMAKE_SHARED_LINKER_FLAGS="-Wl,--as-needed -Wl,-z,now -pthread"

    which all looks pretty reasonable to me: certain features that require “newer” dependencies are disabled on the old distros, and NPAPI plugins are not supported in the enterprise distro, and JIT doesn’t work on odd architectures. I would remove the ENABLE_JIT=OFF lines only because WebKit’s build system should be smart enough nowadays to disable it automatically to save you the trouble of thinking about which architectures the JIT works on. And I would also remove the -DUSE_SYSTEM_MALLOC=ON line to ensure users are properly protected.

    By Michael Catanzaro at November 03, 2018 03:29 AM

    October 24, 2018

    Release Notes for Safari Technology Preview 68

    Surfin’ Safari

    Safari Technology Preview Release 68 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 236793-237231.

    Web API

    • Enabled <input type="color"> support (r236942)
    • Fixed the Document and Window objects to lose their browsing context as soon as its iframe is removed from the document (r236862)
    • Fixed incorrect input.checked when parsing its children (r236795)
    • Fixed window.navigator and window.performance to not become null after the window loses its browsing context (r237185, r237209)
    • Changed to restrict browsing context lookup by name to frames that are related to one another (r237112)
    • Changed anchor target to be ignored on activation when the download attribute is set (r236987)
    • Changed target="_blank" on anchors to imply rel="noopener" (r237144)
    • Fixed incorrect garbage collection of JavaScript node wrappers held by MutationObserver and MutationRecord (r236850, r236801)

    WebRTC

    • Added VP8 support to WebRTC (r236821)
    • Added support for IceCandidate stats (r236963)
    • Added support for reporting “display composited video frames” through the VideoPlaybackQuality object (r236875)
    • Added support for RTCPeerConnection.generateCertificate (r237140)
    • Added support for RTCConfiguration.certificates (r237202)
    • Implemented error handler of MediaRecorder (r237106)

    Media

    • Added support for inline WebVTT styles (r237187)
    • Changed to use nanoseconds as MaximumTimeScale (r237208)
    • Changed to only report the supported WebGL version (r237018)

    CSS

    • Added prefers-color-scheme media query support for styling dark mode content (r237156)
    • Changed to resolve inset properties to computed style when there is over-constrainment (r236979)

    Rendering

    • Fixed slow tiling for CSS gradients (r237230)

    JavaScript

    • Added BigInt support for the bitwise | operator (r236901)

    Web Inspector

    • Added the ability to go directly from an event in the overview view to the specialized timeline for that event (r237195)
    • Added support for showing redirect requests in the Network and Timelines tabs (r236995)
    • Added table support for multiple selection and ⌘ Click (Command-Click) behavior (r236853)
    • Changed to use the iframe name attribute for FrameTreeElement (r236885)
    • Changed to allow multiple canvases to be recorded at the same time (r236952)
    • Created a special Network waterfall for media events (r237028)
    • Exposed Server Timing response headers in the Network tab (r237151)
    • Fixed the Canvas recording sidebar scroll position after switching tabs (r237196)
    • Fixed the color contrast of the disabled Record button in the Canvas tab (r236986)
    • Fixed capturing previously saved states and adding them to the recording payload in the Canvas tab (r237198)
    • Fixed previews to be removed when the parent view is hidden in the Canvas tab (r237090)
    • Fixed clicking the initiator link in the Network tab table to automatically switch to the Preview section (r236923)
    • Fixed dark mode contrast issues (r236953, r237085, r237150)
    • Fixed style editor warnings to not look like errors in dark mode (r237125)
    • Fixed unreadable text when hovering over CSS properties while holding the ⌘ (Command) key in dark mode (r237143)
    • Fixed the “goto arrow” color for the selected DOM breakpoint tree element in dark mode (r237078)
    • Fixed the detail view to be correctly shown after sorting the Network table (r237043)
    • Fixed the detail view reverting to “Response” when new requests are added in the Network tab (r237061)
    • Fixed the Open Resource dialog to show the path to the resource to disambiguate resources with the same name (r236918)
    • Fixed the toolbar getting hidden when Web Inspector is docked to side (r237131)
    • Fixed ⌃G (Control-G) to not wipe the line when jumping to the line in a CSS file (r237212)
    • Grouped media network entries by the node that triggered the request (r236927)
    • Indented all network entries when “Group by Node” is enabled (r237006)

    WebDriver

    • Fixed a bug where some key combinations such as ⌘A (Command-A) can cause the WebDriver session to hang (r236939)
    • Fixed a crash when a WebDriver session is terminated while waiting for simulated inputs to be handled (r236852)
    • Fixed a hang when creating a WebDriver session for Safari Technology Preview (macOS Mojave only).

    Payment Request

    • Changed to abort requests after details settle when the user cancels (r236922)

    Apple Pay

    • Fixed new shipping methods getting ignored when updating after the shippingaddresschange event (r237142)
    • Changed payment authorization results with ApplePayErrors to never be considered final (r237134)

    Web Animations

    • Fixed setting animation-name:none after a fill:forwards animation has completed to correctly revert to the unanimated style (r236809)

    WebCrypto

    • Changed ECDSA to be able to handle invalid signature inputs (r236820)

    October 24, 2018 05:00 PM

    October 21, 2018

    Manuel Rego: Igalia at TPAC 2018

    Igalia WebKit

    Just a quick update before boarding to Lyon for TPAC 2018. This year 12 igalians will be at TPAC, 10 employees (Álex García Castro, Daniel Ehrenberg, Javier Fernández, Joanmarie Diggs, Martin Robinson, Rob Buis, Sergio Villar, Thibault Saunier and myself) and 2 coding experience students (Oriol Brufau and Sven Sauleau). We will represent Igalia in the different working groups and breakout sessions.

    On top of that Igalia will have a booth in the solutions showcase where we’ll have a few demos of our last developments like: WebRTC, MSE, CSS Grid Layout, CSS Box Alignment, MathML, etc. Showing them in some low-end boards like the Raspebrry Pi using WPE an optimized WebKit port for embedded platforms.

    Thread by W3C Developers announcing my talk.

    In my personal case I’ll be attending the CSS Working Group (CSSWG) and Houdini Task Force meetings to follow the work Igalia has been doing on the implementation of different standards. In addition, I’ll be giving a talk about how to contribute to the evolution of CSS on the W3C Developer Meetup that happens on Monday. I’ll try to explain how easy is nowadays to provide feedback to the CSSWG and have some influence on the different specifications.

    Tweet by Daniel Ehrenberg about the Web Platform position.

    Last but not least, Igalia Web Platform Team is hiring, we’re looking for people willing to work on web standards from the implementation on the different browser engines, to the discussions with the standard bodies or the definition of test suites. If you’re attending TPAC and you want to work on a flat company focused on free software development, probably you are a good candidate to join us. Read the position announcement and don’t hesitate to talk to any of us there about that.

    See you at TPAC tomorrow!

    October 21, 2018 07:00 AM