February 19, 2020

Release Notes for Safari Technology Preview 101

Surfin’ Safari

Safari Technology Preview Release 101 is now available for download for macOS Catalina and macOS Mojave. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 255473-256576.

Web Inspector

  • Added a special breakpoint for controlling whether debugger statements pause in the Sources tab (r255887)
  • Changed to encode binary web socket frames using base64 (r256497)
  • Fixed elements closing tag showing reversed in RTL mode (r256374)
  • Fixed the bezier editor popover to be strictly LTR (r255886)
  • Fixed dragging handles in the easing popover selecting sidebar text (r255888)
  • Updated some cookie table column headers to not be localizable (r255896)

Media

  • Corrected TextTrack sorting with invalid BCP47 language (r255997)
  • Fixed AirPlay sometimes stopping after 60 minutes of playback (r255581)

Apple Pay

  • Redacted billing contact during payment method selection (r256071)

JavaScript

  • Added support for BigInt literal as PropertyName (r256541)

Web Animations

  • Fixed accelerated animations freezing on a render tree rebuild (r255663)
  • Fixed an event loop cycle between an animation finishing and it being removed from GraphicsLayerCA (r256181)
  • Fixed an issue where out-of-view transitions could trigger high memory use (r256095)
  • Prevented playing an accelerated animation that was canceled before it was committed (r255810)

WebAuthn

  • Changed authenticatorGetAssertion to be sent without pinAuth if user verification is discouraged (r256001)

WebRTC

  • Aligned getDisplayMedia() with standards specifications (r256034)
  • Fixed not processing newly gathered ICE candidates if the document is suspended (r256009)

CSS

  • Fixed CSS rules with the same selector from several large stylesheets getting applied in the wrong order (r255671)

Rendering

  • Fixed pages that trigger a redirect sometimes getting left blank (r256452)

Web API

  • Disallowed setting base URL to a data or JavaScript URL (r256191)
  • Fixed highlight text decorations to work with all decoration types and colors (r256451)
  • Implemented OffscreenCanvas.copiedImage (r256505)
  • Added standard gamepad mapping for GameControllerGamepads (r256215)
  • Tightened up stylesheet loading (r255693)
  • Fixed quantifiers after lookahead assertions to be syntax errors in Unicode patterns only (r255689)
  • Fixed \0 identity escapes to be syntax errors in Unicode patterns only (r255584)

IndexedDB

  • Fixed iteration of cursors skipping records if deleted (r256414)

Back-forward Cache

  • Updated to remember if legacy TLS was used in the back-forward cache (r256073)

February 19, 2020 09:10 PM

February 05, 2020

Release Notes for Safari Technology Preview 💯

Surfin’ Safari

Safari Technology Preview Release 100 is now available for download for macOS Catalina and macOS Mojave. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 254696-255473.

Web Inspector

  • Added links to Web Inspector Reference documentation (r254730)
  • Renamed the Canvas Tab to be the Graphics Tab, and included basic information and graphical representations of all Web Animation objects that exist in the inspected page (r255396)
  • Allowed developers to evaluate arbitrary JavaScript in isolated worlds created by Safari App Extensions via the execution context picker in the Console (r255191)

Web Animations

  • Added support for the options parameter to getAnimations() (r255149)
  • Changed animations to run accelerated even if other animations targeting the same element are not accelerated (r255383)
  • Fixed changing the delay of an accelerated animation to correctly seek (r255422)
  • Fixed a leak of CSS Animations when removing its animation-name property (r255371)
  • Separated setting a timeline’s current time from updating its animations (r255260)
  • Updated all DocumentTimeline objects when updating animations (r255141)

WebAuthn

  • Fixed User Verification (UV) option present on a CTAP2 authenticatorMakeCredential while the authenticator has not advertised support for it (r254710)

Media

  • Added support for allow="fullscreen" feature policy (r255162)
  • Changed EME to only emit an array of persistent-usage-records when more than one record is discovered (r254896)
  • Corrected VTT Cue Style handling to match the specification (r255151, r255227)
  • Fixed decoder glitches when watching videos on CNN.com (r254761)
  • Fixed AirPlay placard not visible when AirPlay is entered in fullscreen mode (r255103)
  • Fixed video sound sometimes continuing to play in page cache (r254814)
  • Fixed HTMLMediaElement to not remove the media session at DOM suspension time (r255116)

Web API

  • Added finite timeout when synchronously terminating a service worker (r254706)
  • Fixed :matches() to correctly combine with pseudo elements (r255059)
  • Fixed automatic link replacement via “Smart links” to emit insertLink input events (r254945)
  • Disabled Service Workers before terminating an unresponsive service worker process (r255438)
  • Implemented “create a potential-CORS request” (r254821)
  • Implemented transferable property of OffscreenCanvas (r255315)
  • Improved performance speed of index records deletion in IndexedDB (r255318)
  • Made pasteboard markup sanitization more robust (r254800)
  • Used Visible Position to calculate Positions for highlights (r254785)

CSS

  • Fixed EXIF orientation ignored for some CSS images (r254841)
  • Fixed elements no longer stay fixed with elastic overscroll (r255037)

WebRTC

  • Added support for MediaRecorder.requestData (r255085)

JavaScript

  • Fixed DateMath to accept more ISO-8601 timezone designators even if they are not included in ECMA262 to produce expected results in the wild code (r254939)

WebGL2

  • Implemented sub-source texImage2D and texSubImage2D (r255316)

February 05, 2020 09:10 PM

January 24, 2020

ResizeObserver in WebKit

Surfin’ Safari

For years now, web developers have desired the ability to design components that are responsive to their container instead of the viewport. Developers are used to using media queries against viewport width for responsive designs, but having media queries based on element sizes is not possible in CSS because it could result in circular dependencies. Thus, a JavaScript solution was required.

ResizeObserver was introduced to solve this problem, allowing authors to observe changes to the layout size of elements. It was first made available in Chrome 64 in January 2018, and it’s now in Safari Technology Preview releases (and Epiphany Technology Preview). ResizeObserver was enabled by default as of Safari Technology Preview 97.

API Overview

A script creates a ResizeObserver with a callback which will be called with ‘observations’, and registers/unregisters callbacks using .observe(element), and .unobserve(element). Each call to observe(element) adds that element to the set of elements observed by this ResizeObserver instance.

The callback provided to the constructor is called with a collection of observerEntries which contain data about the state of CSS boxes being observed, if those boxes actually changed size. The observer itself also has a .disconnect() method which stops the active delivery of observed changes to the callback. Here’s a simple example:

const callback = (entries) => {
  console.log(`${entries.length} resize observations happened`)
  Array.from(entries).forEach((entry) => {
    let rect = entry.contentRect;
    console.log(
      entry.target,
      `size is now ${rect.width}w x ${rect.height}h`
    )
  })
}

const myObserver = new ResizeObserver(callback)

myObserver.observe(targetElementA)
myObserver.observe(targetElementB)

What we are observing with ResizeObserver is changes to the size of CSS Boxes that we have observed. Since we previously had no information on these boxes before observing, and now we do, this creates an observable effect. Assuming that targetElementA and targetElementB are in the DOM, we will see a log saying that 2 resize observations happened, and providing some information about the elements and sizes of each. It will look something like:

"2 resize observations happened"
"<div class='a'>a</div>" "size is now 1385w x 27h"
"<div class='b'>b</div>" "size is now 1385w x 27h"

Similarly, this means that while it is not an error to observe an element that isn’t in the DOM tree, no observations will occur until a box is actually laid out (when it is inserted, and creates a box). Removing an observed element from the DOM tree (which wasn’t hidden) also causes an observation.

How Observations are Delivered

ResizeObserver strictly specifies when and how things happen and attempts to ensure that calculation and observation always happen “downward” in the tree, and to help authors avoid circularity. Here’s how that happens:

  1. Boxes are created.
  2. Layout happens.
  3. The browser starts a rendering update, and runs the steps up to and including the Intersection Observer steps.
  4. The system gathers and compares the box sizes of observed element with their previously recorded size.
  5. ResizeObserver callback is called passing ResizeObserverEntry objects containing information about the new sizes.
  6. If any changes are incurred during the callback, then layout happens again, but here, the system finds the shallowest at which depth a change occurred (measured in simple node depth from the root). Any changes that are related to something deeper down in the tree are delivered at once, while any that are not are queued up and delivered in the next frame, and an error message will be sent to the Web Inspector console: (ResizeObserver loop completed with undelivered notifications).
  7. Subsequent steps in the rendering updates are executed (i.e. painting happens).

Note

In Safari Technology Preview, entries contain a .contentRect property reflecting the size of the Content Box. After early feedback, the spec is being iterated on in backward compatible ways which will also provide a way to get the measure of the Border Box. Future versions of this API will also allow an optional second argument to .observe which allows you to specify which boxes (Content or Border) you want to receive information about.

Useful Example

Suppose that we have a component containing an author’s profile. It might be used on devices with many sized screens, and in many layout contexts. It might even be provided for reuse as a custom element somehow. Further, these sizes can change at runtime for any number of reasons:

  • On a desktop, the user resizes their window
  • On a mobile device, the user changes their orientation
  • A new element comes into being, or is removed from the DOM tree causing a re-layout
  • Some other element in the DOM changes size for any reason (some elements are even user resizable)

Depending on the amount of space available to us at any given point in time, we’d like to apply some different CSS—laying things out differently, changing some font sizes, perhaps even using different colors.

For this, let’s assume that we follow a ‘responsive first’ philosophy and make our initial design for the smallest screen size. As available space gets bigger, we have another design that should take effect when there are 768px available, and still another when there are at least 1024px. We’ll make these designs with our page using classes “.container-medium” and “.container-large”. Now all we have to do is add or remove those classes automatically.

/* Tell the observer how to manage the attributes */
const callback = (entries) => {
  entries.forEach((entry) => {
    let w = entry.contentRect.width
    let container = entry.target

    // clear out any old ones
    container.classList.remove('container-medium', 'container-large')

    // add one if a 'breakpoint' is true
    if (w > 1024) {
      container.classList.add('container-large')
    } else if (w > 768) {
      container.classList.add('container-medium')
    }
  }) 
}

/* Create the instance **/
const myObserver = new ResizeObserver(callback)

/* Find the elements to observe */
const profileEls = [...document.querySelectorAll('.profile')]

/* .observe each **/
profileEls.forEach(el => myObserver.observe(el))

Now, each .profile element will gain the class of .container-medium or .container-large if their available size meets our specified criteria, and our designs will always be appropriately applied based on their available size. You can, of course, combine this with a MutationObserver or as a Custom Element in order to account for elements which might come into existence later.

Feedback

We’re excited to have ResizeObserver available in Safari Technology Preview! Please try it out and file bugs for any issues you run into.

January 24, 2020 06:00 PM

January 22, 2020

Release Notes for Safari Technology Preview 99

Surfin’ Safari

Safari Technology Preview Release 99 is now available for download for macOS Catalina and macOS Mojave. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 253789-254696.

Legacy Plug-Ins

  • Removed support for Adobe Flash

Web Inspector

  • Elements
    • Enabled the P3 color picker (r253802)
    • Added RGBA input fields for the P3 color picker (r254243)
    • Added support for manipulating the value with the arrow keys in the color picker (r254094)
    • Added color() suggestion when editing a CSS property that accepts color values (r254316)
  • Sources
    • Allowed editing of style sheets injected by Safari App Extensions (r254186)
  • Console
    • Ensured that the clear button is always visible, even at smaller widths (r253800)

Web API

  • Added support for using valid non-zero width and height attributes to become the default aspect ratio of <img> (r254669)
  • Added a check to ensure Service Workers terminate after a period of time when thread blocking (r253898)
  • Aligned Range.intersectsNode() with the DOM specification (r254018)
  • Changed <iframe> attributes to be processed on srcdoc attribute removal (r254498)
  • Changed <img>.naturalWidth to return the density-corrected intrinsic width (r254229)
  • Changed <link> with non-CSS type to not be retrieved (r253992)
  • Changed Object.keys to throw if called on a module namespace object with uninitialized binding (r254390)
  • Changed Object.preventExtensions to throw if not successful (r254626)
  • Changed Document.createAttribute() to take in a localName, not a qualifiedName (r254021)
  • Changed the supported MIME types for image encoding to be supported image MIME types (r254541)
  • Denied Notification API access for non-secure contexts (r253899)
  • Fixed dispatchEvent() to not clear the event’s isTrusted flag when it returns early (r254016)
  • Fixed String.prototype.replace() incorrectly handling named references on regular expressions without named groups (r254088)
  • Fixed URL parser in Fetch not always using UTF-8 (r254672)
  • Fixed encoding entities correctly in <style> element during XML serialization of text (r253988)
  • Removed the low priority resource load for sendBeacon to reduce failure rates (r253847)
  • Updated Fetch to Handle empty Location value (r253814)

Cookies

  • Fixed document.cookie to not do a sync IPC to the network process for iframes that do not have storage access (r254556)

CSS

  • Added support for image-set() standard syntax (r254406)
  • Added support for rendering highlights specified in CSS Highlight API (r253857)
  • Implemented a network error when fetching a linked stylesheet resource fails (r254043)
  • Improved performance by invalidating only affected elements after media query evaluation changes (r253875)
  • Fixed rejected changes between similar unprefixed and prefixed gradient syntax (r254164)
  • Excluded implicit CSS grid tracks from the resolved value (r254561)

Media

  • Enabled HDR Media Capabilities by default (r253853)
  • Fixed specification violation in Font Loading API (r254220)
  • Ignored URL host for schemes that are not using host information (r253946)
  • Implemented “create a potential-CORS request” (r254000)
  • Implemented transceiver setCodecPreferences (r253966)
  • Made text track loading set same-origin fallback flag (r254031)
  • Fixed MediaKeySession.load() failing (r253852)

WebRTC

  • Removed the certificate info checks related to getUserMedia (r253827)

Payment Request

  • Converted the payment method data IDL in the PaymentRequest constructor (r253986)

Web Animations

  • Stopped creating CSS Animations for <noscript> elements (r254201)

JavaScript

  • Fixed invalid date parsing for ISO 8601 strings when no timezone given (r254038)
  • Fixed RegExp.prototype[Symbol.replace] to support named capture groups (r254195)

Web Share API

  • Added support for a user gesture to allow using the Web Share API even when preceded by an XHR call (r254178)

WebDriver

  • Reimplemented the “Execute Async Script” command with Promises to match the specification (r254329)
  • Fixed handling of session timeouts for values higher than MAX_INT (r253883)
  • Fixed scripts being executed in the wrong page context after a history navigation (r254328)

IndexedDB

  • Improved performance by removing the timer for pending operations in IDBTransaction (r253807)

January 22, 2020 06:00 PM

January 16, 2020

Paulo Matos: Cross-Arch Reproducibility using Containers

Igalia WebKit

I present the use of containers for cross architecture reproducibility using docker and podman, which I then go on to apply to JSC. If you are trying to understand how to create cross-arch reproducible environments for your software, this might help you!

More…

By Paulo Matos at January 16, 2020 04:00 PM

January 08, 2020

Release Notes for Safari Technology Preview 98

Surfin’ Safari

Safari Technology Preview Release 98 is now available for download for macOS Catalina and macOS Mojave. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 252823-253789.

Web Inspector

  • Elements
    • Removed the “Show/Hide Shadow DOM” navigation item (r253706)
    • Restricted showing paint flashing and compositing borders to the Web Inspector session (r253739)
    • Ensure that a bezier swatch is shown for CSS timing function keywords (r253758)
    • Fixed hovering over an invalid value while holding ⌘ to change the color of the text (r253405)
    • Fixed the Classes input to not display on top of other content (r253167)
  • Network
    • Fixed pressing ⌘F when no network item is selected to focus the filter bar (r253160)
  • Sources
    • Fixed non-regex local overrides to not apply to resources that only contain the URL instead of completely matching the URL (r253246)
  • Storage
    • Added support for filtering IndexedDB stores and indexes (r253161)
  • Audit
    • Fixed selected item before entering edit mode not being reselected after exiting edit mode (r253759)
    • Fixed importing a result with DOM nodes that don’t match the inspected page appearing as empty lines (r253757)
  • Console
    • Ensure copying an evaluation result does not include the saved variable index (r253169)
  • Search
    • Added basic “No Search Results” text with a clickable help navigation item that reveals and focuses the navigation sidebar search input when there is no active search (r253165)

Web Animations

  • Enabled Web Animations CSS Integration, a new implementation of CSS Animations and CSS Transitions, by default (r252945)
  • Fixed layout of element children with forwards-filling opacity animation that can be incorrect after removal (r252879)
  • Implemented Animation.commitStyles() (r252966)

Media

  • Enabled the Generic Text Track Cue API (r253695)

Rendering

  • Ensured transparency layers are properly ended when only painting root background (r253692)
  • Fixed an issue where elements could jump to the wrong position after some compositing-related style changes (r252935)

Web API

  • Implemented OffscreenCanvas.convertToBlob (r253474)
  • Changed setting toString or valueOf on a cross-origin Location object to throw a SecurityError (r253418)
  • Fixed an incorrect association of the URL object with the port value (r252998)
  • Prevented synchronous XHR in beforeunload and unload event handlers (r253213)

CSS

  • Changed to not perform range checking for calc() at parse time (r252983)
  • Changed media queries in img sizes attribute to evaluate dynamically (r252828)
  • Implemented the clamp() function (r253105)
  • Improved computed values of calc() functions to match the specification (r253079)

JavaScript

  • Changed Object.prototype.isPrototypeOf() to check if the passed in value is a non-object first (r253264)

WebRTC

  • Added protection for WebRTC network monitoring to wait forever in edge cases (r253203)
  • Fixed audio elements that resumed playback after getUserMedia (r253742)

Clipboard API

  • Added sanitization for HTML and image data written using clipboard.write (r253486)

Browser Changes

  • Changed to issue the load sooner on swipe back/forward navigation (r253360)
  • Re-disabled TLS 1.0 and TLS 1.1 by default (r253292)

WebAssembly

  • Changed to validate and generate bytecode in a single pass (r253140)

January 08, 2020 09:15 PM

Angelos Oikonomopoulos: A Dive Into JavaScriptCore

Igalia WebKit

Recently, the compiler team at Igalia was discussing the available resources for the WebKit project, both for the purpose of onboarding new Igalians and for lowering the bar for third-party contributors. As compiler people, we are mainly concerned with JavaScriptCore (JSC), WebKit’s javascript engine implementation. There are many high quality blog posts on the webkit blog that describe various phases in the evolution of JSC, but finding one’s bearings in the actual source can be a daunting task.

The aim of this post is twofold: first, document some aspects of JavaScriptCore at the source level; second, show how one can figure out what a piece of code actually does in a large and complex source base (which JSC’s certainly is).

In medias res

As an exercise, we’re going to arbitrarily use a commit I had open in a web browser tab. Specifically, we will be looking at this snippet:

Operands<Optional<JSValue>> mustHandleValues(codeBlock->numParameters(), numVarsWithValues);
int localsUsedForCalleeSaves = static_cast<int>(CodeBlock::llintBaselineCalleeSaveSpaceAsVirtualRegisters());
for (size_t i = 0; i < mustHandleValues.size(); ++i) {
    int operand = mustHandleValues.operandForIndex(i);
    if (operandIsLocal(operand) && VirtualRegister(operand).toLocal() < localsUsedForCalleeSaves)
	continue;
    mustHandleValues[i] = callFrame->uncheckedR(operand).jsValue();
}

This seems like a good starting point for taking a dive into the low-level details of JSC internals. Virtual registers look like a concept that’s good to know about. And what are those “locals used for callee saves” anyway? How do locals differ from vars? What are “vars with values”? Let’s find out!

Backstory

Recall that JSC is a multi-tiered execution engine. Most Javascript code is only executed once; compiling takes longer than simply interpreting the code, so Javascript code is always interpreted the first time through. If it turns out that a piece of code is executed frequently though1, compiling it becomes a more attractive proposition.

Initially, the tier up happens to the baseline JIT, a simple and fast non-optimizing compiler that produces native code for a Javascript function. If the code continues to see much use, it will be recompiled with DFG, an optimizing compiler that is geared towards low compilation times and decent performance of the produced native code. Eventually, the code might end up being compiled with the FTL backend too, but the upper tiers won’t be making an appearence in our story here.

What do tier up and tier down mean? In short, tier up is when code execution switches to a more optimized version, whereas tier down is the reverse operation. So the code might tier up from the interpreter to the baseline JIT, but later tier down (under conditions we’ll briefly touch on later) back to the baseline JIT. You can read a more extensive overview here.

Diving in

With this context now in place, we can revisit the snippet above. The code is part of operationOptimize. Just looking at the two sites it’s referenced in, we can see that it’s only ever used if the DFG_JIT option is enabled. This is where the baseline JIT ➞ DFG tier up happens!

The sites that make use of operationOptimize both run during the generation of native code by the baseline JIT. The first one runs in response to the op_enter bytecode opcode, i.e. the opcode that marks entry to the function. The second one runs when encountering an op_loop_hint opcode (an opcode that only appears at the beginning of a basic block marking the entry to a loop). Those are the two kinds of program points at which execution might tier up to the DFG.

Notice that calls to operationOptimize only occur during execution of the native code produced by the baseline JIT. In fact, if you look at the emitted code surrounding the call to operationOptimize for the function entry case, you’ll see that the call is conditional and only happens if the function has been executed enough times that it’s worth making a C++ call to consider it for optimization.

The function accepts two arguments: a vmPointer which is, umm, a pointer to a VM structure (i.e. the “state of the world” as far as this function is concerned) and the bytecodeIndex. Remember that the bytecode is the intermediate representation (IR) that all higher tiers start compiling from. In operationOptimize, the bytecodeIndex is used for

Again, the bytecodeIndex is a parameter that has already been set in stone during generation of the native code by the baseline JIT.

The other parameter, the VM, is used in a number of things. The part that’s relevant to the snippet we started out to understand is that the VM is (sometimes) used to give us access to the current CallFrame. CallFrame inherits from Register, which is a thin wrapper around a (maximally) 64-bit value.

The CodeBlock

In this case, the various accessors defined by CallFrame effectively treat the (pointer) value that CallFrame consists of as a pointer to an array of Register values. Specifically, a set of constant expressions

struct CallFrameSlot {
    static constexpr int codeBlock = CallerFrameAndPC::sizeInRegisters;
    static constexpr int callee = codeBlock + 1;
    static constexpr int argumentCount = callee + 1;
    static constexpr int thisArgument = argumentCount + 1;
    static constexpr int firstArgument = thisArgument + 1;
};

give the offset (relative to the callframe) of the pointer to the codeblock, the callee, the argument count and the this pointer. Note that the first CallFrameSlot is the CallerFrameAndPC, i.e. a pointer to the CallFrame of the caller and the returnPC.

The CodeBlock is definitely something we’ll need to understand better, as it appears in our motivational code snippet. However, it’s a large class that is intertwined with a number of other interesting code paths. For the purposes of this discussion, we need to know that it

  • is associated with a code block (i.e. a function, eval, program or module code block)
  • holds data relevant to tier up/down decisions and operations for the associated code block

We’ll focus on three of its data members:

int m_numCalleeLocals;
int m_numVars;
int m_numParameters;

So, it seems that a CodeBlock can have at least some parameters (makes sense, right?) but also has both variables and callee locals.

First things first: what’s the difference between callee locals and vars? Well, it turns out that m_numCalleeLocals is only incremented in BytecodeGeneratorBase<Traits>::newRegister whereas m_numVars is only incremented in BytecodeGeneratorBase<Traits>::addVar(). Except, addVar calls into newRegister, so vars are a subset of callee locals (and therefore m_numVarsm_numCalleelocals).

Somewhat surprisingly, newRegister is only called in 3 places:

So there you have it. Callee locals

  1. are allocated by a function called newRegister
  2. are either a var or a temporary.

Let’s start with the second point. What is a var? Well, let’s look at where vars are created (via addVar):

There is definitely a var for every lexical variable (VarKind::Stack), i.e. a non-local variable accessible from the current scope. Vars are also generated (via BytecodeGenerator::createVariable) for

So, intuitively, vars are allocated more or less for “every JS construct that could be called a variable”. Conversely, temporaries are storage locations that have been allocated as part of bytecode generation (i.e. there is no corresponding storage location in the JS source). They can store intermediate calculation results and what not.

Coming back to the first point regarding callee locals, how come they’re allocated by a function called newRegister? Why, because JSC’s bytecode operates on a register VM! The RegisterID returned by newRegister wraps the VirtualRegister that our register VM is all about.

Virtual registers, locals and arguments, oh my!

A virtual register (of type VirtualRegister) consists simply of an int (which is also called its offset). Each virtual register corresponds to one of

There is no differentiation between locals and arguments at the type level (everything is a (positive) int); However, virtual registers that map to locals are negative and those that map to arguments are nonnegative. In the context of bytecode generation, the int

It feels like JSC is underusing C++ here.

In all cases, what we get after indexing with a local, argument or constant is a RegisterID. As explained, the RegisterID wraps a VirtualRegister. Why do we need this indirection?

Well, there are two extra bits of info in the RegisterID. The m_refcount and an m_isTemporary flag. The reference count is always greater than zero for a variable, but the rules under which a RegisterID is ref’d and unref’d are too complicated to go into here.

When you have an argument, you get the VirtualRegister for it by directly adding it to CallFrame::thisArgumentoffset.

When you have a local, you map it to (-1 - local) to get the corresponding Virtualregister. So

local vreg
0 -1
1 -2
2 -3

(remember, virtual registers that correspond to locals are negative).

For an argument, you map it to (arg + CallFrame::thisArgumentOffset()):

argument vreg
0 this
1 this + 1
2 this + 2

Which makes all the sense in the world when you remember what the CallFrameSlot looks like. So argument 0 is always the `this` pointer.

If the vreg is greater than some large offset (s_firstConstantRegisterIndex), then it is an index into the CodeBlock's constant pool (after subtracting the offset).

Bytecode operands

If you’ve followed any of the links to the functions doing the actual mapping of locals and arguments to a virtual register, you may have noticed that the functions are called localToOperand and argumentToOperand. Yet they’re only ever used in virtualRegisterForLocal and virtualRegisterForArgument respectively. This raises the obvious question: what are those virtual registers operands of?

Well, of the bytecode instructions in our register VM of course. Instead of recreating the pictures, I’ll simply encourage you to take a look at a recent blog post describing it at a high level.

How do we know that’s what “operand” refers to? Well, let’s look at a use of virtualRegisterForLocal in the bytecode generator. BytecodeGenerator::createVariable will allocate2 the next available local index (using the size of m_calleeLocals to keep track of it). This calls into virtualRegisterForLocal, which maps the local to a virtual register by calling localToOperand.

The newly allocated local is inserted into the function symbol table, along with its offset (i.e. the ID of the virtual register).

The SymbolTableEntry is looked up when we generate bytecode for a variable reference. A variable reference is represented by a ResolveNode3.

So looking into ResolveNode::emitBytecode, we dive into BytecodeGenerator::variable and there’s our symbolTable->get() call. And then the symbolTableEntry is passed to BytecodeGenerator::variableForLocalEntry which uses entry.varOffset() to initialize the returned Variable with offset. It also uses registerFor to retrieve the RegisterID from m_calleeLocals.

ResolveNode::emitBytecode will then pass the local RegisterID to move which calls into emitMove, which just calls OpMov::emit (a function generated by the JavaScriptCore/generator code). Note that the compiler implicitly converts the RegisterID arguments to VirtualRegister type at this step. Eventually, we end up in the (generated) function

template<OpcodeSize __size, bool recordOpcode, typename BytecodeGenerator>
static bool emitImpl(BytecodeGenerator* gen, VirtualRegister dst, VirtualRegister src)
{
    if (__size == OpcodeSize::Wide16)
	gen->alignWideOpcode16();
    else if (__size == OpcodeSize::Wide32)
	gen->alignWideOpcode32();
    if (checkImpl<__size>(gen, dst, src)) {
	if (recordOpcode)
	    gen->recordOpcode(opcodeID);
	if (__size == OpcodeSize::Wide16)
	    gen->write(Fits<OpcodeID, OpcodeSize::Narrow>::convert(op_wide16));
	else if (__size == OpcodeSize::Wide32)
	    gen->write(Fits<OpcodeID, OpcodeSize::Narrow>::convert(op_wide32));
	gen->write(Fits<OpcodeID, __size>::convert(opcodeID));
	gen->write(Fits<VirtualRegister, __size>::convert(dst));
	gen->write(Fits<VirtualRegister, __size>::convert(src));
	return true;
    }
    return false;
}

where Fits::convert(VirtualRegister) will trivially encode the VirtualRegister into the target type. Specifically the mapping is nicely summed up in the following comment

// Narrow:
// -128..-1  local variables
//    0..15  arguments
//   16..127 constants
//
// Wide16:
// -2**15..-1  local variables
//      0..64  arguments
//     64..2**15-1 constants

You may have noticed that the Variable returned by BytecodeGenerator::variableForLocalEntry already has been initialized with the virtual register offset we set when inserting the SymbolTableEntry for the local variable. And yet we use registerFor to look up the RegisterID for the local and then use the offset of the VirtualRegister contained therein. Surely those are the same? Oh well, something for a runtime assert to check.

Variables with values

Whew! Quite the detour there. Time to get back to our original snippet:

Operands<Optional<JSValue>> mustHandleValues(codeBlock->numParameters(), numVarsWithValues);
int localsUsedForCalleeSaves = static_cast<int>(CodeBlock::llintBaselineCalleeSaveSpaceAsVirtualRegisters());
for (size_t i = 0; i < mustHandleValues.size(); ++i) {
    int operand = mustHandleValues.operandForIndex(i);
    if (operandIsLocal(operand) && VirtualRegister(operand).toLocal() < localsUsedForCalleeSaves)
	continue;
    mustHandleValues[i] = callFrame->uncheckedR(operand).jsValue();
}

What are those numVarsWithValues then? Well, the definition is right before our snippet:

unsigned numVarsWithValues;
if (bytecodeIndex)
    numVarsWithValues = codeBlock->numCalleeLocals();
else
    numVarsWithValues = 0;

OK, so this looks straighforward for a change. If the bytecodeIndex is not zero, we’re doing the tier up from JIT to DFG in the body of a function (i.e. at a loop entry). In that case, we consider all our callee locals to have values. Conversely, when we’re running for the function entry (i.e. bytecodeIndex == 0), none of the callee locals are live yet. Do note that the variable is incorrectly named. Vars are not the same as callee locals; we’re dealing with the latter here.

A second gotcha is that, whereas vars are always live, temporaries might not be. The DFG compiler will do liveness analysis at compile time to make sure it’s only looking at live values. That must have been a fun bug to track down!

Values that must be handled

Back to our snippet, numVarsWithValues is used as an argument to the constructor of mustHandleValues which is of type Operands<Optional<JSValue>>. Right, so what are the Operands? They simply hold a number of T objects (here T is Optional<JSValue>) of which the first m_numArguments correspond to, well, arguments whereas the remaining correspond to locals.

What we’re doing here is recording all the live (non-heap, obviously) values when we try to do the tier up. The idea is to be able to mix those values in with the previously observed values that DFG’s Control Flow Analysis will use to emit code which will bail us out of the optimized version (i.e. do a tier down). According to the comments and commit logs, this is in order to increase the chances of a successful OSR entry (tier up), even if the resulting optimized code may be slightly less conservative.

Remember that the optimized code that we tier up to makes assumptions with regard to the types of the incoming values (based on what we’ve observed when executing at lower tiers) and wil bail out if those assumptions are not met. Taking the values of the current execution at the time of the tier up attempt ensures we won’t be doing all this work only to immediately have to tier down again.

Operands provides an operandForIndex method which will directly give you a virtual reg for every kind of element. For example, if you had called Operands<T> opnds(2, 1), then the first iteration of the loop would give you

operandForIndex(0)
-> VirtualRegisterForargument(0).offset()
  -> VirtualRegister(argumentToOperand(0)).offset()
    -> VirtualRegister(CallFrame::thisArgumentOffset).offset()
      -> CallFrame::thisArgumentOffset

The second iteration would similarly give you CallFrame::thisArgumentOffset + 1.

In the third iteration, we’re now dealing with a local, so we’d get

operandForIndex(2)
-> virtualRegisterForLocal(2 - 2).offset()
  -> VirtualRegister(localToOperand(0)).offset()
    -> VirtualRegister(-1).offset()
      -> -1

Callee save space as virtual registers

So, finally, what is our snippet doing here? It’s iterating over the values that are likely to be live at this program point and storing them in mustHandleValues. It will first iterate over the arguments (if any) and then over the locals. However, it will use the “operand” (remember, everything is an int…) to get the index of the respective local and then skip the first locals up to localsUsedForCalleeSaves. So, in fact, even though we allocated space for (arguments + callee locals), we skip some slots and only store (arguments + callee locals - localsUsedForCalleeSaves). This is OK, as the Optional<JSValue> values in the Operands will have been initialized by the default constructor of Optional<> which gives us an object without a value (i.e. an object that will later be ignored).

Here, callee-saved register (csr) refers to a register that is available for use to the LLInt and/or the baseline JIT. This is described a bit in LowLevelInterpreter.asm, but is more apparent when one looks at what csr sets are used on each platform (or, in C++).

platform metadataTable PC-base (PB) numberTag notCellMask
X86_64 csr1 csr2 csr3 csr4
x86_64_win csr3 csr4 csr5 csr6
ARM64~/~ARM64E csr6 csr7 csr8 csr9
C_LOOP 64b csr0 csr1 csr2 csr3
C_LOOP 32b csr3 - - -
ARMv7 csr0 - - -
MIPS csr0 - - -
X86 - - - -

On 64-bit platforms, offlineasm (JSC’s portable assembler) makes a range of callee-saved registers available to .asm files. Those are properly saved and restored. For example, for X86_64 on non-Windows platforms, the returned RegisterSet contains registers r12-r15 (inclusive), i.e. the callee-saved registers as defined in the System V AMD64 ABI. The mapping from symbolic names to architecture registers can be found in GPRInfo.

On 32-bit platforms, the assembler doesn’t make any csr regs available, so there’s nothing to save except if the platform makes special use of some register (like C_LOOP does for the metadataTable 4).

What are the numberTag and notCellMask registers? Out of scope, that’s what they are!

Conclusion

Well, that wraps it up. Hopefully now you have a better understanding of what the original snippet does. In the process, we learned about a few concepts by reading through the source and, importantly, we added lots of links to JSC’s source code. This way, not only can you check that the textual explanations are still valid when you read this blog post, you can use the links as spring boards for further source code exploration to your heart’s delight!

Footnotes

1 Both the interpreter – better known as LLInt – and the baseline JIT keep track of execution statistics, so that JSC can make informed decisions on when to tier up.

2 Remarkably, no RegisterID has been allocated at this point – we used the size of m_calleeLocals but never modified it. Instead, later in the function (after adding the new local to the symbol table!) the code will call addVar which will allocate a new “anonymous” local. But then the code asserts that the index of the newly allocated local (i.e. the offset of the virtual register it contains) is the same as the offset we previously used to create the virtual register, so it’s all good.

3 How did we know to look for the ResolveNode? Well, the emitBytecode method needs to be implemented by subclasses of ExpressionNode. If we look at how a simple binary expression is parsed (and given that ASTBuilder defines BinaryOperand as std::pair<ExpressionNode*, BinaryOpInfo>), it’s clear that any variable reference has already been lifted to an ExpressionNode.

So instead, we take the bottom up approach. We find the lexer/parser token definitions, one of which is the IDENT token. Then it’s simply a matter of going over its uses in Parser.cpp, until we find our smoking gun. This gets us into createResolve aaaaand

return new (m_parserArena) ResolveNode(location, ident, start);

That’s the node we’re looking for!

4 C_LOOP is a special backend for JSC’s portable assembler. What is special about it is that it generates C++ code, so that it can be used on otherwise unsupported architectures. Remember that the portable assembler (offlineasm) runs at compilation time.

January 08, 2020 12:00 PM

Angelos Oikonomopoulos: A Dive Into JavaScriptCore

Igalia WebKit

This post is an attempt to both document some aspects of JSC at the source level and to show how one can figure out what a piece of code actually does in a source base as large and complex as JSC's.

January 08, 2020 12:00 PM

December 20, 2019

New WebKit Features in Safari 13

Surfin’ Safari

This year’s releases of Safari 13 for macOS Catalina, iPadOS, iOS 13, and watchOS 6 include a tremendous number of WebKit improvements for the web across Apple’s platforms. Of particular note is the number of features that enhance website compatibility to bring a true desktop-class web browsing experience to Safari on iPadOS. This release is also packed with updates for improved privacy, performance, and a host of new tools for web developers.

Here’s a quick look at the new WebKit enhancements available with these releases.

Desktop-class Browsing on iPad

WebKit provides the heart of this new experience with deep, fundamental changes that deliver a great desktop website experience on a touch device. With the exception of iPad mini, Safari on iPad will now send a user-agent string that is identical to Safari on macOS. Beyond just a user-agent change, WebKit added new support for web standards to provide the needed compatibility and quality. That included adding new support for Pointer Events, the Visual Viewport API, and programmatic paste. You can read more details about support for those standards in the sections below. In addition, WebKit added Media Source Extensions support in Safari on iPadOS to improve compatibility with the desktop-variants of streaming video websites.

Beyond foundational new web standards support in WebKit, there are many other refinements for the desktop browsing experience on iPad. Page scaling behaviors have been fine-tuned to prevent horizontal scrolling on wide webpages with responsive design viewport tags. When a webpage is scaled down to fit entirely within the viewport, WebKit will increase the font size of content to ensure text is comfortably legible. WebKit added support for automatic Fast Tap on links and buttons to help make navigating web content feel more responsive. Improved hardware keyboard support adds the ability to scroll with the arrow keys and perform focus navigation. Find on page now works like Safari on desktop, highlighting all of the matching terms on the page with a special highlight for the current selection. The behavior of editing callout menus for text selections was polished to avoid overlapping in page controls provided by many document editing web applications. Last but not least, Safari includes support for background downloads, as well as background file uploads.

This new experience on iPad means significant changes for web developers to consider for their web technology projects. Safari on iPad is a chameleon; it can respond to servers as either a desktop device or a mobile device under different circumstances. Most of the time, Safari on iPad presents a macOS user-agent when loading a webpage. If Safari is moved into a one-third size when multitasking the desktop site will be scaled to fit the one-third size without reloading and losing your place. But loading or reloading a webpage while Safari is in one-third size will provide an iOS user-agent since the mobile layout is better suited to the smaller viewport.

Now more than ever before, web developers need to take great care in providing a single responsive web design that uses feature detection instead of relying on separate desktop and mobile sites dependent on the user-agent. Developers should be sure to test their desktop website experience on an iPad to ensure it works well for users.

Pointer Events

WebKit added support for Pointer Events to provide DOM events for generic, hardware-agnostic pointer input such as those generated by a mouse, touch, or stylus. It adds a layer of abstraction that makes it easier for web developers to handle a variety of input devices. Similar to mouse events, pointer events include coordinates, a target element, button states, but they also supports additional properties related to other forms of input, such as pressure, tilt, and more.

See the Pointer Events specification for more information.

Visual Viewport API

WebKit added support for the Visual Viewport API, that allows webpages to detect the part of the page that is visible to the user, taking zooming and the onscreen keyboard into account. Developers can use this API to move content out of the way of the onscreen keyboard. This is useful for a floating overlay, a custom completion list popup, or a custom-drawn caret in a custom editing area.

See the Visual Viewport API specifications for more information.

Programmatic Paste

WebKit also brings new support for programmatic paste in Safari for iOS and iPadOS with document.execCommand('paste'). When a page triggers programmatic paste within scope of a user gesture, a callout bar with the option to paste is provided. When the call out is tapped it will grant access to the clipboard and proceed with the paste. For paste operations where the contents of the clipboard share the same origin as the page triggering the programmatic paste, WebKit allows the paste immediately with no callout bar.

Learn more in the Document.execCommand() reference on MDN.

Accelerated Scrolling on iOS and iPadOS

Accelerated scrolling the main frame has always been available with WebKit on iOS. In addition, developers could use a CSS property called -webkit-overflow-scrolling to opt-in to fast scrolling for overflow scroll. None of that is necessary with iPadOS and iOS 13. Subframes are no longer extended to the size of their contents and are now scrollable, and overflow: scroll; and iframe always get accelerated scrolling. Fast scrolling emulation libraries are no longer needed and -webkit-overflow-scrolling: touch; is a no-op on iPad. On iPhone, it still has the side-effect of creating a CSS stacking context on scrollable elements. Developers will want to test their content to see how hardware accelerated scrolling everywhere affects it and remove unnecessary workarounds.

Performance Improvements

This release brings performance improvements that reduced the initial rendering time for webpages on iOS, and reduced load time up to 50% for webpages on watchOS. Reduced memory use by JavaScript in Safari, web views, and non-web clients that use JSContext. WebKit also achieved better graphics rendering performance showing up to a 10% improvement in the MotionMark graphics performance benchmark score.

Intelligent Tracking Prevention

The latest update to Intelligent Tracking Prevention enhances prevention of cross-site tracking through the abuse of link-decoration. The updates in ITP 2.3 as part of this release of Safari add new countermeasures. In addition to the 24-hour cookie expiry from ITP 2.2, non-cookie website data like LocalStorage will be marked for deletion if the page is navigated to from a classified domain to a landing URL with a query string or fragment identifier. Deletion happens after seven days of Safari use without user interaction on the website. Beyond link decoration, Intelligent Tracking Prevention will also downgrade document.referrer to the referrer’s eTLD+1 if the referrer has link decoration when the user was navigated from a classified domain.

For details on Intelligent Tracking Prevention updates, see the “Intelligent Tracking Prevention 2.3” blog post and our collection of other privacy related blog posts.

FIDO2-compliant USB Security Keys

Safari 13 on macOS has support for FIDO2-compliant USB security keys through the Web Authentication standard. Security keys can hold device bounded public key credentials that are associated with specific internet accounts. This allows users to add an additional layer of protection to their accounts by utilizing security keys as a second factor to authenticate. Not just that, but Web Authentication also prevents phishing. Since user agents are arbitrating the entire authentication process and the public key credentials can never leave their bounded security keys, it’s impossible for phishing sites to get users’ targeted credentials.

More Privacy and Security Improvements

Building on the strength of privacy and security in WebKit, users will have additional protections with sandbox hardening on iOS and macOS, and navigation protection from third-party iframes.

Developers will now need to call DeviceMotionEvent.requestPermission() or DeviceOrientationEvent.requestPermission() to prompt the user for permission before getting access to the events, on in Safari or Safari View Controller on iOS and iPadOS.

Apple Pay in WKWebView

In iOS 13, webpages loaded in WKWebView can now accept Apple Pay. In order to protect the security of Apple Pay transactions in WKWebView, Apple Pay cannot be used alongside of script injection APIs such as WKUserScript or evaluateJavaScript(_:completionHandler:).

If these APIs are invoked before a webpage uses Apple Pay, Apple Pay will be disabled. If a webpage uses Apple Pay before evaluateJavaScript(_:completionHandler:) is invoked, the completion handler will be called with a non-nil NSError. These restrictions are reset every time the top frame is navigated.

Media Improvements

Media improvements in WebKit improve both compatibility and capability for developers. Support for the decodingInfo() method of the Media Capabilities API allows developers to check for supported codecs, efficiently supported codecs, and optional codec features including new alpha transparency. WebKit now supports transparency in video with an alpha channel that works for all supported video formats.

In Safari on macOS, WebKit added the ability for users to share their screen with others natively, using web technologies, without the need for any plug-ins. SFSafariViewController gained WebRTC support for the navigator.mediaDevices property of the Media and Streams API.

Dark Mode for iOS and iPadOS

Last year WebKit added dark mode for the web to Safari on macOS Mojave. This year, WebKit brings the same support to style web content that matches the system appearance in Safari on iOS and iPadOS.

WebKit.org blog posts page shown in light modeWebKit.org blog posts page shown in dark mode

Learn how it works and how to add support to your web content in the blog posts on “Dark Mode Support in WebKit” and “Dark Mode in Web Inspector”.

Improved Home Screen Web Apps on iOS and iPadOS

Support for websites saved to the home screen have been polished to work more like native apps. The changes focused on better multitasking support, improved login flow to work in-line without switching to Safari, support for Apple Pay, and improved reliability for remote Web Inspector.

Safari WebDriver for iOS

Support for Safari WebDriver on iOS 13. Control via WebDriver is exposed to developers via the /usr/bin/safaridriver executable, which hosts a driver that handles REST API requests sent by WebDriver test clients. In order to run WebDriver tests on an iOS device, it must be plugged into a macOS host that has a new enough version of safaridriver. Support for hosting iOS-based WebDriver sessions is available in safaridriver included with Safari 13 and later. Older versions of safaridriver do not support iOS WebDriver sessions.

If you’ve never used safaridriver on macOS before, you’ll first need to run safaridriver --enable and authenticate as an administrator. Then, you’ll need to enable Remote Automation on every device that you intend to use for WebDriver. To do this, toggle the setting in the Settings app under Safari → Advanced → Remote Automation.

With the introduction of native WebDriver support in Safari on iOS 13, it’s now possible to run the same automated tests of desktop-oriented web content on desktop and mobile devices equally. Safari’s support comes with new, exclusive safeguards to simultaneously protect user security and privacy and also help you write more stable and consistent tests. You can try out Safari’s WebDriver support today by installing a beta of macOS Catalina and iOS 13.

You can learn more about iOS support for Safari WebDriver by reading the “WebDriver is Coming to Safari in iOS 13”.

Web Inspector Improvements

Web Inspector adds tools that bring new insights to web content during development. This release also includes many tooling refinements with more capabilities and a better debugging experience. Among the changes, Web Inspector has improved performance for debugging large, complex websites.

A new CPU Usage Timeline is available in the Timelines Tab that provides developers with insight into power efficiency through CPU usage. This helps developers analyze and improve the power efficiency of their web content. The Timelines Tab has also been updated to support importing and exporting of recorded timeline data using a JSON file format. The portability of timeline data makes it possible to share recordings with other developers, or use the data in custom tools.

Read more in the “CPU Timeline in Web Inspector” blog post. For more tips on developing power efficient web content, you can also read the “How Web Content Can Affect Power Usage” blog post.

This release introduces a new Audit Tab to run tests against web content with results that can be easily imported and exported. The Audit Tab includes a built-in accessibility audit for web content and allows developers to create their own audits for custom checks throughout the web content development process.

You can read more in the blog posts for “Audits in Web Inspector” and “Creating Web Inspector Audits” .

When an iOS or iPadOS device with Web Inspector enabled in Safari’s Advanced Settings is connected to a macOS device running Safari, Web Inspector will offer a new Device Settings menu. The Device Settings menu allows overriding developer-related Safari settings such as the User-Agent string when Web Inspector is connected to the device.

Read more about this in the “Changing Page Settings on iOS Using Web Inspector” blog post.

The Elements Tab includes a new Changes sidebar to keep track of CSS changes made in the Styles sidebar, making it easier to capture all of the changes made and re-incorporate them into production code. In the Network Tab, certificates and TLS settings are now available to review in the Security pane of the resources view.

Feedback

These improvements are available to users running watchOS 6, iOS 13, iPadOS, macOS Catalina, macOS Mojave 10.14.6 and macOS High Sierra 10.13.6. These features were also available to web developers with Safari Technology Preview releases. Changes in this release of Safari were included in the following Safari Technology Preview releases: 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89. Download the latest Safari Technology Preview release to stay on the forefront of future web features. You can also use the WebKit Feature Status page to watch for changes to your favorite web platform features.

We love hearing from you. Send a tweet to @webkit or @jonathandavis to share your thoughts on this release, and any features you were hoping for that didn’t make it. If you run into any issues, we welcome your bug reports for Safari, or WebKit bugs for web content issues.

December 20, 2019 06:00 PM

December 18, 2019

Release Notes for Safari Technology Preview 97

Surfin’ Safari

Safari Technology Preview Release 97 is now available for download for macOS Catalina and macOS Mojave. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 251627-252823.

Resize Observer

  • Enabled Resize Observer by default (r251822)

WebAuthn

  • Added UI with instructions for authenticating with a security key while authenticating
  • Added support for legacy Google NFC Titan security keys (r252297)

Web Animations

  • Added support for AnimationEvent.pseudoElement (r251840)
  • Fixed retargeted transitions targeting accelerated properties that do not stop the original transition (r252540)
  • Fixed the easing property for a CSS transition effect (r251657)
  • Fixed the transform property always none for getKeyframes() output of a CSS Animation (r251839)
  • Fixed getKeyframes() to return the right timing function for declarative animations (r251649)

Web Inspector

  • Elements
    • Added support for multiline CSS property values (r252523)
    • Added support for highlighting nodes that match CSS rules with pseudo-selectors (r252436)
    • Added color picker support for P3 color space (r252168)
    • Fixed an issue where copying multiple DOM nodes would only copy the last selected DOM node (r252615)
    • Fixed aqua and fuchsia not being detected as CSS colors (r252448)
    • Outlined sRGB-safe areas on the P3 color picker (r252747)
  • Sources
    • Added a context menu item to reveal the local override when a resource is loaded from it (r252495)
    • Added support for matching local overrides based on URL pattern in addition to exact match (r252614)
    • Changed to prefer non-blackboxed scripts when showing a source code location link (r253087)
    • Changed to fire Node Removed breakpoints whenever the DOM node is removed from the main DOM tree, not just when it’s removed from it’s parent (r251871)
    • Fixed “Toggle Visibility” context menu item to work for elements inside a shadow tree (r252026)
    • Made the default content of the Inspector Bootstrap Script a comment that explains how it works (r251921)
    • Moved the “Local Override…” creation item from the Breakpoints section options menu to the Create Resource menu (r252517)
    • Made call frames from blackboxed scripts visually distinct (r252745)
  • Timelines
    • Added a marker for when a stop was requested (r252199)
    • Added a timeline that shows information about any recorded CSS animation/transition (r251959)
    • Labeled ResizeObserver callbacks in the JavaScript & Events timeline (r251832)
  • Layers
    • Enabled the Layers Tab by default (r252063)
  • Console
    • Defaulted to focusing the console prompt if no other content is focused after opening Web Inspector (r251958)
    • Fixed an issue where the saved result value was still being shown after navigating (r253000)
  • Settings
    • Enabled line wrapping by default (r251918)

Rendering

  • Fixed image flashing with transform: rotate animation (r252486)
  • Implemented accelerated video-to-texture upload path for ANGLE backend for WebGL (r252741)

Back-Forward Cache

  • Added site-specific back-forward cache quirk to work around an issue on vimeo.com (r252301)
  • Fixed WebAnimation to never prevent entering the back-forward cache (r251742)
  • Fixed PaymentRequest and PaymentResponse to not prevent entering the back-forward cache (r252338)
  • Fixed UserMediaRequest to not prevent entering the back-forward cache (r251746)
  • Made MediaStream and MediaStreamTrack back-forward cache friendly (r252337)

SVG

  • Added the orient property of the interface SVGMarkerElement (r252444)
  • Added percentage support for fill-opacity, stroke-opacity, stop-opacity, and flood-opacity (r251696)
  • Changed properties that take <position> to not accept 3 values (r251668)
  • Disabled SVG shapes should not be hit (r252069)
  • Fixed SVGGeometryElement.getPointAtLength to clamp its argument (r251877)
  • Fixed opacity to always serialize as a number (r251828)

Clipboard API

  • Added some infrastructure to resolve ClipboardItems into pasteboard data for writing (r252315)
  • Added support for Clipboard.readText() (r252627)

CSS

  • Added support for the Q unit (r251662)
  • Changed CSS Transitions and CSS Animations properties to treat unit-less 0 as an invalid value for times (r251658)
  • Fixed CSS grid line name positions after auto repeat with no line names (r251965)
  • Fixed -webkit-font-smoothing: none not antialiasing subsequent elements (r252622)
  • Fixed ::before and ::after elements not filling their grid cell when the container has display: contents (r251780)
  • Fixed calc() serialization to match the specifications (r253079)
  • Implemented the CSS clamp() function (r253105)

Remote Playback API

  • Enabled Remote Playback API by default (r251784, r251737)
  • Ensured the MediaRemote callback always called (r252331)

Media

  • Batched multiple EME key requests into one request and response (r251895)

JavaScript

  • Added BigInt support for ++ and -- (r252680)
  • Fixed Intl.DateTimeFormat to return resolvedOptions in the correct order (r251815)
  • Optimized Promise runtime functions (r251671)
  • Implement String.prototype.replaceAll (r252683)

Picture-in-Picture Web API

  • Enabled the Picture-in-Picture API by default (r251925, r251745, r251797)
  • Added support for the :picture-in-picture CSS pseudo-class for video elements in picture-in-picture mode (r252330)
  • Fixed picture-in-picture events to fire when entering or exiting the picture-in-picture mode (r252240)

WebAssembly

  • Created a WebAssembly interpreter (r251886)
  • Support WebAssembly.Global (r253074)

Web API

  • Added fullscreen style support for reddit.com (r251827)
  • Changed the file input to fire an input event before the change event (r252768)
  • Changed hidden framesets to provide default edgeInfo value (r251680)
  • Fixed <input type="range">.setAttribute("value") to update the value (r251718)
  • Fixed some captcha images rendering as a blank white space (r252353)
  • Fixed content sometimes disappearing for a <video> with controls and clipping (r252070)
  • Fixed a bug where focusing a shadow host which delegates focus will properly skip inner shadow hosts that delegate focus (r252537)
  • Fixed getComputedStyle returning auto for zIndex property even after it has been set on non-positioned elements (r252724)
  • Fixed very slow tile rendering due to event region painting in Google Docs spreadsheets (r252419)
  • Fixed notification permissions not getting remembered for origins without a port (r251709)
  • Fixed VeryHigh priority loads (r252431)

December 18, 2019 07:30 PM

December 12, 2019

Nikolas Zimmermann: CSS 3D transformations & SVG

Igalia WebKit

As mentioned in my first article, I have a long relationship with the WebKit project, and its SVG implementation. In this post I will explain some exciting new developments and possible advances, and I present some demos of the state of the art (if you cannot wait, go and watch them, and come back for the details). To understand why these developments are both important and achievable now though, we’ll have to first understand some history.

By zimmermann@kde.org (Nikolas Zimmermann) at December 12, 2019 12:00 AM

December 10, 2019

Preventing Tracking Prevention Tracking

Surfin’ Safari

This blog post covers enhancements to Intelligent Tracking Prevention (ITP) included in Safari on iOS and iPadOS 13.3, Safari 13.0.4 on macOS Catalina, Mojave, and High Sierra.

Tracking Prevention as a Tracking Vector

Any kind of tracking prevention or content blocking that treats web content differently based on its origin or URL risks being abused itself for tracking purposes if the set of origins or URLs provide some uniqueness to the browser and webpages can detect the differing treatment.

To combat this, tracking prevention features must make it hard or impossible to detect which web content and website data is treated as capable of tracking. We have devised three ITP enhancements that not only fight detection of differing treatment but also improve tracking prevention in general.

Origin-Only Referrer For All Third-Party Requests

ITP now downgrades all cross-site request referrer headers to just the page’s origin. Previously, this was only done for cross-site requests to classified domains.

As an example, a request to https://images.example that would previously contain the referrer header “https://store.example/baby/strollers/deluxe-stroller-navy-blue.html” will now be reduced to just “https://store.example/”.

All Third-Party Cookies Blocked on Websites Without Prior User Interaction

ITP will now block all third-party requests from seeing their cookies, regardless of the classification status of the third-party domain, unless the first-party website has already received user interaction.

The Storage Access API Takes the Underlying Cookie Policy Into Consideration

Safari’s original cookie policy restricts third-parties from setting cookies unless they already have set cookies as first-party. It is still in effect underneath ITP.

As of this ITP update, the Storage Access API takes Safari’s cookie policy into consideration when handling calls to document.hasStorageAccess().

Now a call to document.hasStorageAccess() may resolve with false for one of two reasons:

  1. Because ITP is blocking cookies and explicit storage access has not been granted.
  2. Because the domain doesn’t have cookies and thus the cookie policy is blocking cookies.

Developers have asked for this change because previously document.hasStorageAccess() could resolve with true but the iframe still couldn’t set cookies because of the cookie policy. This is a rare case in practice because Safari’s cookie policy has been in effect for a decade and a half and almost all websites that want to use cookies as third-party will also set cookies as first-party.

Absence of Cookies In Third-Party Requests Does Not Reveal ITP Status

A reasonable question to ask is can an attacker reveal ITP status for a domain outside their control by making a third-party request to the domain and checking for side effects of whether cookies were sent or not?

Safari’s default cookie policy requires a third-party to have “seeded” its cookie jar as first-party before it can use cookies as third-party. This means the absence of cookies in a third-party request can be due to ITP blocking existing cookies or the default cookie policy blocking cookies because the user never visited the website, the website’s cookies have expired, or because the user or ITP has explicitly deleted the website’s cookies.

Thus, the absence of cookies in a third-party request outside the attacker’s control is not proof that the third-party domain is classified by ITP.

Thanks To Google

We’d like to thank Google for sending us a report in which they explore both the ability to detect when web content is treated differently by tracking prevention and the bad things that are possible with such detection. Their responsible disclosure practice allowed us to design and test the changes detailed above. Full credit will be given in upcoming security release notes.

December 10, 2019 06:30 PM

December 08, 2019

Philippe Normand: HTML overlays with GstWPE, the demo

Igalia WebKit

Once again this year I attended the GStreamer conference and just before that, Embedded Linux conference Europe which took place in Lyon (France). Both events were a good opportunity to demo one of the use-cases I have in mind for GstWPE, HTML overlays!

As we, at Igalia, usually have a …

By Philippe Normand at December 08, 2019 02:00 PM

December 04, 2019

Manuel Rego: Web Engines Hackfest 2020: New dates, new venue!

Igalia WebKit

Igalia is pleased to announce the 12th annual Web Engines Hackfest. It will take place on May 18-20 in A Coruña, and in a new venue: Palexco. You can find all the information, together with the registration form, on the hackfest website: https://webengineshackfest.org/2020/.

Mission and vision

The main goal behind this event is to have a place for people from different parts of the web platform community to meet together for a few days and talk, discuss, draft, prototype, implement, etc. on different topics of interest for the whole group.

There are not many events where browser implementors from different engines can sit together and talk about their last developments, their plans for the future, or the controversial topics they have been discussing online.

However this is an event not only for developers, other roles that are part of the community, like people working on standards, are welcomed to the event.

It’s really nice to have people from different backgrounds and working on a variety of things around the web, to reach better solutions, enlighten the conversations and draft higher quality conclusions during the discussions.

We believe the combination of all these factors make the Web Engines Hackfest an unique opportunity to push forward the evolution of the web.

2020 edition

We realized that autumn is usually full of browser events (TPAC, BlinkOn, WebKit Contributors Meeting, … just to name a few), and most of the people coming to the hackfest are also attending some of them. For that reason we thought it would be a good idea to move the event from fall to spring, in order to better accommodate everyone’s schedules and avoid unfortunate conflicts or unnecessary hard choices. So next year the hackfest will happen on May from Monday 18th to Wednesday 20th (both days included).

At this stage the event is becoming popular and during the past three years we have been around 60-70 people. Igalia office has been a great venue for the hackfest during all this time, but on the last occasions we were using it as its full capacity. So this time we decided to move the hackfest to a new venue, which will allow us to grow to 100 or more participants, let’s see how things go. The venue would be Palexco, a lovely conferences building in A Coruña port, which is very close to the city center. We really hope you like the new place and enjoy it.

New venue: Palexco (picture by Jose Luis Cernadas Iglesias) New venue: Palexco (picture by Jose Luis Cernadas Iglesias)

Having more people and the new venue bring us lots of challenges but also new possibilities. So we’re changing a little bit the format of the event, we’ll have a first day in a more regular conference fashion (with some talks and lighting talks) but also including some space for discussions and hacking. And then the last 2 days will be more the usual unconference format with a bunch of breakout sessions, informal discussions, etc. We believe the conversations and discussions that happen during the hackfest are one of the best things of the event, and we hope this new format will work well.

Join us

Thanks to the changes on the venue, the event is no longer invitation-only (as it used to be). We’ll be still sending the invitations to the people usually interested on the hackfest, but you can already register by yourself just filling the registration form.

Soon we will open a call for papers for the talks, stay tuned! We’ll also have room for ligthing talks, so people attending can take advantage of them to explain their work and plans on the event.

Last but not least, Arm, Google and Igalia will be sponsoring 2020 edition, thank you very much! We hope more companies join the trend and help us to arrange the event with their support. If your company is willing to sponsor the hackfest, please don’t hesitate to contact us at hackfest@webengineshackfest.org.

Some historical information

Igalia has been organizing and hosting this event since 2009. Back then, the event was called the “WebKitGTK+ Hackfest”. The WebKitGTK+ project was, on those days, in early stages. There was lots of work to do around the project, and a few people (11 to be specific) decided to work together for a whole week to move the project forward. The event was really successful and it was happening on a similar fashion for 5 years.

On 2014 we decided to make broader the scope of the event and not restrict it to people working only on WebKitGTK+ (or WebKit), but open it to members from all parts of the web platform community (including folks working on other engines like Blink, Servo, Gecko). We changed the name to “Web Engines Hackfest”, we got a very positive response and the event has been running on yearly since then, growing more and more every year.

And now we’re looking forward to 2020 edition, in a new venue and with more people than ever. Let’s hope everything goes great.

December 04, 2019 11:00 PM

November 25, 2019

Nikolas Zimmermann: Back in town

Igalia WebKit

Welcome to my blog!

Finally I’m back after my long detour to physics :-)

Some of you might know that my colleague Rob Buis and me founded the ksvg project a little more than 18 years ago (announcement mail to kfm-devel) and met again after many years in Galicia last month.

By zimmermann@kde.org (Nikolas Zimmermann) at November 25, 2019 12:00 AM

November 21, 2019

Release Notes for Safari Technology Preview 96

Surfin’ Safari

Safari Technology Preview Release 96 is now available for download for macOS Catalina and macOS Mojave. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 251210-251627.

Web Animations

  • Enabled the Web Animations JavaScript API by default (r251237)

WebAuthN

  • Implemented AuthenticatorCancel (r251295)

SVG

  • Added auto behavior to the width and height properties of the SVG <image> element (r251378)
  • Added bounding-box keyword to pointer-events (r251444)
  • Fixed SVGElement to conform with SVG2 (r251499)
  • Loaded event must be fired only for the SVG structurally external elements and the outermost SVG element (r251290)
  • Removed the viewTarget property of SVGViewElement (r251461)

Web API

  • Added strictness to request’s Content-Type (r251490)
  • Changed XMLHttpRequest.responseXML url to be the HTTP response URL (r251542)
  • Changed String.prototype.matchAll to throw on non-global regex (r251483)
  • Fixed a bad is array assertion JSON.parse (r251394)
  • Fixed setting border-radius on <video> element clipping the top and left sections of the video (r251385)
  • Ignored document.open or document.write after the active parser has been aborted (r251506)
  • Made requestIdleCallback suspendable (r251258)

CSS

  • Added content-box and stroke-box to the transform-box property (r251252)
  • Added support for gradients using stops with multiple positions (r251474)
  • Fixed a crash when parsing gradients with multi-position color stops (r251437)

Clipboard API

  • Implemented ClipboardItem.getType() (r251377)
  • Implemented navigator.clipboard.read() (r251279)

CSS Shadow Parts

  • Changed :part rules to be able to override the style attribute (r251285)

JavaScript

  • Removed wasmAwareLexicalGlobalObject (r251529)

Picture-in-Picture API

  • Implemented EnterPictureInPictureEvent support (r251530)
  • Added runtime logging for the Picture-in-Picture API (r251458)

Media

  • Added support for callbacks for manifest events (r251626)

Service Worker

  • Fixed MP4 video element broken with Service Worker (r251594)

Back-Forward Cache

  • Prevented putting pages that have not reached the non-visually empty layout milestone into the back-forward cache (r251275)
  • Fixed Notification to not prevent entering the back-forward cache (r251528)
  • Fixed AudioContext to not prevent entering the back-forward cache (r251537)
  • Fixed FetchResponse to not prevent entering the back-forward cache (r251545)
  • Fixed XMLHttpRequest to not prevent entering the back-forward cache (r251366)

Web Inspector

  • Elements
    • Added clickable icons for each CSS rule that distinguish the rule’s type (r251624)
    • Fixed $0 being shown for the wrong node when selecting elements in a user agent shadow tree (r251302)
    • Fixed the selection color dimmed when inside a shadow tree (r251254)
    • Replaced color wheel with square HSB color picker (r251487)
  • Sources
    • Fixed the content of the function definition popover sometimes getting cut off (r251446)
    • Changed the function/object preview popover to keep the name sticky to the top (r251466)
    • Provided a way to inject “bootstrap” JavaScript into the page as the first script executed (r251531)

WebDriver

  • Fixed the Element Click endpoint triggering a click at an incorrect y-coordinate (r251948)

November 21, 2019 06:00 PM

November 13, 2019

Manuel Rego: Web Engines Hackfest 2019

Igalia WebKit

A month ago Igalia hosted another edition of the Web Engines Hackfest in our office in A Coruña. This is my personal summary of the event, obviously biased as I’m part of the organization.

Talks

During the event we arranged six talks about a variety range of topics:

Emilio Cobos during his talk at the Web Engines Hackfest 2019 Emilio Cobos during his talk at the Web Engines Hackfest 2019

Web Platform Tests (WPT)

Apart from the talks, the main and most important part of the hackfest (at least from my personal point of view) are the breakout sessions and discussions we organize about different interest topics.

During one of these sessions we talked about the status of things regarding WPT. WPT is working really fine for Chromium and Firefox, however WebKit is still lagging behind as synchronization is still manual there. Let’s hope things will improve in the future on the WebKit side.

We also highlighted that the number of dynamic tests on WPT are less than expected, we discarded issues with the infrastructure and think that the problems are more on the side of people writing the tests, that somehow forget to cover cases when things changes dynamically.

Apart from that James Graham put over the table the results from the last MDN survey, which showed interoperability as one of the most important issues for web developers. WPT is helping with interop but despite of the improvements on that regard this is still a big problem for authors. We didn’t have any good answer about how to fix that, in my case I shared some ideas that could help to improve things at some point:

  • Mandatory tests for specs: This is already happening for some specs like HTML but not for all of them. If we manage to reach a point where every change on a spec comes with a test, probably interoperability on initial implementations will be much better. It’s easy to understand why this is not happening as people working on specs are usually very overloaded.
  • Common forum to agree on shipping features: This is a kind of utopia, as each company has their own priorities, but if we had a place were the different browser vendors talk in order to reach an agreement about when to ship a feature, that would make web author’s lifes much easier. We somehow managed to do that when we shipped CSS Grid Layout almost simultaneously in the different browsers, if we could repeat that success story for more features in the future that would be awesome.

Debugging tools

One of the afternoons we did a breakout session related to debugging tools.

First Christian Biesinger showed us JdDbg which is an amazing tool to explore data structures in the web browser (like the DOM, layout or accessibility trees). All the information is updated live while you debug your code, and you can access all of them on a single view very comfortably.

Afterwards Emilio Cobos explained how to use the reverse debugger rr. With this tool you can record a bug and then replay it as many times as you need going back and forward in time. Also Emilio showed how to annotate all the output so you can go directly to that moment in time, or how to randomize the execution to help caught race conditions. As a result of this explanation we got a bug fixed in WebKitGTK+.

Other

About MathML Fred’s talk finished sending the intent-to-implement mail to blink-dev officially announcing the beginning of the upstreaming process. Since then a bunch of patches have already landed behind a runtime flag, you can follow the progress on Chromium issue #6606 if you’re interested.

On the last day a few of us even attended the CSS Working Group confcall during the hackfest, which worked as a test for Igalia office’s infrastructure thinking on the face-to-face meeting we’ll be hosting next January.

People attending the CSSWG confcall (from left to right: Oriol, Emilio, fantasai, Christian and Brian) People attending the CSSWG confcall (from left to right: Oriol, Emilio, fantasai, Christian and Brian)

As a side note, this time we arranged a guided city tour around A Coruña and, despite of the weather, people seemed to have enjoyed it.

Acknowledgements

Thanks to everyone coming, we’re really happy for the lovely feedback you always share about the event, you’re so kind! ☺

Of course, kudos to the speakers for the effort working on such a nice set of interesting talks. 👏

Last, but not least, big thanks to the hackfest sponsors: Arm, Google, Igalia and Mozilla. Your support is critical to make this event possible, you rock. 🎸

Web Engines Hackfest 2019 sponsors: Google and Igalia Web Engines Hackfest 2019 sponsors: Arm, Google, Igalia and Mozilla

See you all next year. Some news about the next edition will be announced very soon, stay tuned!

November 13, 2019 11:00 PM

November 08, 2019

WebGPU and WSL in Web Inspector

Surfin’ Safari

In Safari Technology Preview release 91, beta support was added for the WebGPU API and WSL. In Safari Technology Preview release 94, support for showing WebGPU devices, as well as all associated render/compute pipelines and <canvas> elements, was added to Web Inspector inside the Canvas Tab.

Just like WebGL shader programs, all render/compute pipelines are editable, and any changes will have an immediate effect. Compute pipelines, as they only have one shader module, are shown in a single text editor. Render pipelines, since they have both vertex and fragment shader modules, are shown in side-by-side text editors, one for each shader module. In the case that both the vertex and fragment shader modules are shared, however, they are shown as a single text editor, just like it is for a compute pipeline.

Try editing any of the WebGPU pipelines in our gallery of WebGPU samples using Web Inspector. We’ll be keeping that page updated with the latest demos. Many thanks to Austin Eng for making the demo used in the video above.

Let us know what you think! Send feedback on Twitter (@webkit, @jonathandavis, @dcrousso) or by filing a bug.

Note: Learn more about Web Inspector from the Web Inspector Reference documentation.

November 08, 2019 01:21 AM

October 30, 2019

Release Notes for Safari Technology Preview 95

Surfin’ Safari

Safari Technology Preview Release 95 is now available for download for macOS Catalina and macOS Mojave. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 250947-251210.

Shadow DOM

  • Added support for ShadowRoot.delegateFocus (r251043)

Images

  • Added image/apng as a supported mime type for images (r251182)
  • Fixed a bug with filter outsets that caused black lines on images on wikipedia (r251119)

WebRTC

  • Removed unified plan runtime flag (r250969)

Clipboard API

  • Implemented getType() for ClipboardItems created from bindings (r251134)
  • Refactored custom pasteboard writing codepaths to handle multiple items (r251117)
  • Refactored Pasteboard item reading functions (r250950)
  • Supported writing multiple PasteboardCustomData with SharedBuffers to the pasteboard (r251100)
  • Added support for programmatic paste requests on macOS (r250973)

Picture-in-Picture Web API

  • Implemented HTMLVideoElement.requestPictureInPicture() and Document.exitPictureInPicture() (r251160)

Service Workers

  • Changed to reject a response body promise in the case of a failure happening after the HTTP response (r251101)
  • Prevented timeout for a load intercepted by a Service Worker that receives a response (r250985)

Back-Forward Cache

  • Fixed pages frequently failing to enter the back-forward cache due to frames with a quick redirect (r251019)
  • Fixed back-forward cache after doing a Favorites navigation (r251049)
  • Fixed clearing website data for a given session to not shut down cached processes for other sessions (r251048)
  • Fixed DOMCacheStorage to not prevent pages from entering the back-forward cache (r250965)

Web Inspector

  • Sources
    • Enabled the new Sources Tab by default, which merges the Debugger Tab and Resources Tab into a single UI (r250991)
    • Added support for automatically creating an image or font local override when dragging content over a non-overridden resource (r251024, r251144)
  • Debugger
    • Added support for pattern-based script blackboxing by URL in the Settings Tab (r251039)
    • Prevented source mapped resources from being blackboxed (r251170)
  • Elements
    • Included a filter option in the Computed details sidebar for condensing all related longhand properties into their respective shorthand properties (r251038)

October 30, 2019 05:00 PM

October 28, 2019

Adrián Pérez de Castro: The Road to WebKit 2.26: a Six Month Retrospective

Igalia WebKit

Now that version 2.26 of both WPE WebKit and WebKitGTK ports have been out for a few weeks it is an excellent moment to recap and take a look at what we have achieved during this development cycle. Let's dive in!

  1. New Features
  2. Security
  3. Cleanups
  4. Releases, Releases!
  5. Buildbot Maintenance
  6. One More Thing

New Features

Emoji Picker

The GTK emoji picker has been integrated in WebKitGTK, and can be accessed with Ctrl-Shift-; while typing on input fields.

Your browser does not support WebM videos (see an animated GIF instead). GNOME Web showing the GTK emoji picker.

Data Lists

WebKitGTK now supports the <datalist> HTML element (reference), which can be used to list possible values for an <input> field. Form fields using data lists are rendered as an hybrid between a combo box and a text entry with type–ahead filtering.

Your browser does not support WebM videos (see an animated GIF instead). GNOME Web showing an <input> entry with completion backed by <datalist>.

WPE Renderer for WebKitGTK

The GTK port now supports reusing components from WPE. While there are no user-visible changes, with many GPU drivers a more efficient buffer sharing mechanism—which takes advantage of DMA-BUF, if available—is used for accelerated compositing under Wayland, resulting in better performance.

Packagers can disable this feature at build time passing -DUSE_WPE_RENDERER=OFF to CMake, which could be needed for systems which cannot provide the needed libwpe and WPEBackend-fdo libraries. It is recommended to leave this build option enabled, and it might become mandatory in the future.

In-Process DNS Cache

Running a local DNS caching service avoids doing queries to your Internet provider’s servers when applications need to resolve the same host names over and over—something web browsers do! This results in faster browsing, saves bandwidth, and partially compensates for slow DNS servers.

Patrick and Carlos have implemented a small cache inside the Network Process which keeps in memory a maximum of 400, valid for 60 seconds. Even though it may not seem like much, this improves page loads because most of the time the resources needed to render a page are spread across a handful of hosts and their cache entries will be reused over and over.

Promotional image of the “Gone in 60 Seconds” movie. This image has nothing to do with DNS, except for the time entries are kept in the cache.

While it is certainly possible to run a full-fledged DNS cache locally (like dnsmasq or systemd-resolved, which many GNU/Linux setups have configured nowadays), WebKit can be used in all kinds of devices and operating systems which may not provide such a service. The caching benefits all kinds of systems, with embedded devices (where running an additional service is often prohibitive) benefiting the most, and therefore it is always enabled by default.

Security

Remember Meltdown and Spectre? During this development cycle we worked on mitigations against side channel attacks like these. They are particularly important for a Web engine, which can download and execute code from arbitrary servers.

Subprocess Sandboxing

Both WebKitGTK and WPE WebKit follow a multi-process architecture: at least there is the “UI Process”, an application that embeds WebKitWebView widget; the “Web Process” (WebKitWebProcess, WPEWebProcess) which performs the actual rendering, and the “Network Process” (WebKitNetworkProcess, WPENetworkProcess) which takes care of fetching content from the network and also manages caches and storage.

Patrick Griffis has led the effort to add support in WebKit to isolate the Web Process from the rest of the system, running it with restricted access to the rest of the system. This is achieved using Linux namespaces—the same underlying building blocks used by containerization technologies like LXC, Kubernetes, or Flatpak. As a matter of fact, we use the same building blocks as Flatpak: Bubblewrap, xdg-dbus-proxy, and libseccomp. This not only makes it more difficult for a website to snoop on other processes' data: it also limits potential damage to the rest of the system caused by maliciously crafted content, because the Web Process is where most of which it is parsed and processed.

This feature is built by default, and using it in applications is only one function call away.

PSON

Process Swap On (cross-site) Navigation is a new feature which makes it harder for websites to steal information from others: rendering of pages from different sites always takes place in different processes. In practice, each security origin uses a different Web Process (see above) for rendering, and while navigating from one page to another new processes will be launched or terminated as needed. Chromium's Site Isolation works in a similar way.

Unfortunately, the needed changes ended up breaking a few important applications which embed WebKitGTK (like GNOME Web or Evolution) and we had to disable the feature for the GTK port just before its 2.26.0 release—it is still enabled in WPE WebKit.

Our plan for the next development cycle is keep the feature disabled by default, and to provide a way for applications to opt-in. Unfortunately it cannot be done the other way around because the public WebKitGTK API has been stable for a long time and we cannot afford breaking backwards compatibility.

HSTS

This security mechanism helps protect websites against protocol downgrade attacks: Web servers can declare that clients must interact using only secure HTTPS connections, and never revert to using unencrypted HTTP.

During the last few months Claudio Saavedra has completed the support for HTTP Strict Transport Security in libsoup—our networking backend—and the needed support code in WebKit. As a result, HSTS support is always enabled.

New WebSockets Implementation

The WebKit source tree includes a cross-platform WebSockets implementation that the GTK and WPE ports have been using. While great for new ports to be able to support the feature, it is far from optimal: we were duplicating network code because libsoup also implements them.

Now that HSTS support is in place, Claudio and Carlos decided that it was a good moment to switch libsoup's implementation so WebSockets can now also benefit from it. This also made possible to provide the RFC-7692 permessage-deflate extension, which enables applications to request compression of message payloads.

To ensure that no regressions would be introduced, Claudio also added support in libsoup for running the Autobahn 🛣 test suite, which resulted in a number of fixes.

Cleanups

During this release cycle we have deprecated the single Web Process mode, and trying to enable it using the API is a no-op. The motivation for this is twofold: in the same vein of PSON and sanboxing, we would rather not allow applications to make side channel attacks easier; not to mention that the changes needed in the code to accommodate PSON would make it extremely complicated to keep the existing API semantics. As this can potentially be trouble for some applications, we have been in touch with packagers, supporting them as best as we can to ensure that the new WebKitGTK versions can be adopted without regressions.

Another important removal was the support for GTK2 NPAPI browser plug-ins. Note that NPAPI plug-ins are still supported, but if they use GTK they must use version 3.x—otherwise they will not be loaded. The reason for this is that GTK2 cannot be used in a program which uses GTK3, and vice versa. To circumvent this limitation, in previous releases we were building some parts of the WebKit source code twice, each one using a different version of GTK, resulting in two separate binaries: we have only removed the GTK2 one. This allowed for a good clean up of the source tree, reduced build times, and killed one build dependency. With NPAPI support being sunsetted in all browsers, the main reason to keep some degree of support for it is the Flash plug-in. Sadly its NPAPI version uses GTK2 and it does not work starting with WebKitGTK 2.26.0; on the other hand, it is still possible to run the PPAPI version of Flash through FreshPlayerPlugin if needed.

Releases, Releases!

Last March we released WebKitGTK 2.24 and WPE WebKit 2.24 in sync, and the same for the current stable, 2.26. As a matter of fact, most releases since 2.22 have been done in lockstep and this has been working extremely well.

Hannibal Smith, happy about the simultaneous releases.

Both ports share many of their components, so it makes sense to stabilize and prepare them for a new release series at the same time. Many fixes apply to both ports, and the few that not hardly add noise to the branch. This allows myself and Carlos García to split the effort of backporting fixes to the stable branch as well—though I must admit that Carlos has often done more.

Buildroot ♥ WebKit

Those using Buildroot to prepare software images for various devices will be happy to know that packages for the WPE WebKit components have been imported a while ago into the source tree, and have been available since the 2019.05 release.

Two years ago I dusted off the webkitgtk package, bringing it up to the most recent version at the time, keeping up with updates and over time I have been taking care of some of its dependencies (libepoxy, brotli, and woff2) as well. Buildroot LTS releases are now receiving security updates, too.

Last February I had a great time meeting some of the Buildroot developers during FOSDEM, where we had the chance of discussing in person how to go about adding WPE WebKit packages to Buildroot. This ultimately resulted in the addition of packages libwpe, wpebackend-fdo, wpebackend-fdo, and cog to the tree.

My plan is to keep maintaining the Buildroot packages for both WebKit ports. I have also a few improvements in the pipeline, like enabling the sandboxing support (see this patch set) and usage of the WPE renderer in the WebKitGTK package.

Buildbot Maintenance

Breaking the Web is not fun, so WebKit needs extensive testing. The source tree includes tens of thousands of tests which are used to avoid regressions, and those are ran on every commit using Buildbot. The status can be checked at build.webkit.org.

Additionally, there is another set of builders which run before a patch has had the chance of being committed to the repository. The goal is to catch build failures and certain kinds of programmer errors as early as possible, ensuring that the source tree is kept “green”—that is: buildable. This is the EWS, short for Early Warning System, which trawls Bugzilla for new—or updated—patches, schedules builds with them applied, and adds a set of status bubbles in Bugzilla next to them. Igalia also contributes with EWS builders

EWS bot bubbles as shown in Bugzilla For each platform the EWS adds a status bubble after trying a patch.

Since last April there is an ongoing effort to revamp the EWS infrastructure, which is now using Buildbot as well. Carlos López has updated our machines recently to Debian Buster, then I switched them to the new EWS at ews-build.webkit.org. This is based on Buildbot as well, which brings niceties in the user interface like being able to check the status for the GTK and for the WPE WebKit port conveniently in realtime. Most importantly, this change has brought the average build time from thirteen minutes down to eight, making the “upload patch, check EWS build status” cycle shorter for developers.

Big props to Aakash Jain, who has been championing all the EWS improvements.

One More Thing

Finally, I would like to extend our thanks to everybody who has contributed to WebKit during the 2.26 development cycle, and in particular to the Igalia Multimedia team, who have been hard at work improving our WebRTC support and the GStreamer back-end 🙇.

October 28, 2019 12:30 AM

October 16, 2019

Release Notes for Safari Technology Preview 94

Surfin’ Safari

Safari Technology Preview Release 94 is now available for download for macOS Mojave and macOS Catalina. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 250329-250947.

CSS Shadow Parts

Web Animations

  • Fixed removing an element to only cancel its declarative animations (r250335)

Storage Access API

  • Changed document.hasStorageAccess() to return true when the cookie policy allows access and false otherwise, for third parties not blocked by ITP (r250431, r250589)

WebRTC

  • Changed to allow suspending RTCPeerConnection when not connected (r250726)

Media

  • Updated MediaDevices to require a secure context (r250551)

JavaScript

  • Changed toExponential, toFixed, and toPrecision to allow arguments up to 100 (r250389)

CSS Grid

  • Preserved auto repeat() in getComputedStyle() for non-grids (r250715)

Web API

  • Accepted two values in the overflow shorthand (r250849)
  • Allowed using WebGL 2 when USE_ANGLE=1 (r250740)
  • Changed the default statusText of Response to an empty string (r250787)
  • Changed CSS ellipse() to accept 0 or 2 <shape-radius> (r250653)
  • Changed Service Worker Fetch events to time out (r250852)
  • Corrected clip-path <geometry-box> mapping (r250778)
  • Changed Fetch API no-CORs check to take into account same-origin (r250515)
  • Changed radio button groups to be scoped by shadow boundaries (r250708)
  • Fixed a newly inserted element to get assigned to a named slot if slot assignments had already happened (r250709)
  • Fixed AbortSignal to always emit the abort signal (r250727)
  • Fixed JSON.parse to correctly handle array proxies (r250860)
  • Made table’s clientWidth and clientHeight include its border sizes (r250553)
  • Updated attachShadow to support attaching a shadow root to a main element (r250770)
  • Updated Fetch data URL HEAD request to result in empty response body (r250822)
  • Updated radial gradients to reject negative radii (r250730)
  • Updated ImageBitmap to be serializable (r250721)

Web Inspector

  • Elements
    • Fixed issue where properties were always shown as invalid if they didn’t match the selected node (r250633)
  • Resources
    • Fixed issue where newlines were being unexpectedly added inside template string expressions (r250544)
    • Include local resource overrides in the Open Resource dialog (r250407)
  • Debugger
    • Prevent blackboxing of scripts that haven’t finished loading or failed to load (r250813)
  • Canvas
    • Made it more obvious that the cards in the Overview are clickable (r250859)
    • Show “No Preview Available” instead of an empty preview for WebGPU devices (r250858)
    • Support editing of WebGPU render pipelines that use the same shader module for vertex and fragment (r250874)
    • Fixed issue where clicking on the Overview path component didn’t work (r250855)
    • Dark Mode: Minor dark mode style fixes (r250533, r250854)
  • Settings
    • Enable the image transparency grid by default and create a checkbox for it (r250814)

WebDriver

  • Fixed an issue that prevented sudo safaridriver --enable from working correctly

back-forward Cache

  • Allowed pages served over HTTPS with Cache-Control: no-store header to enter the back-forward cache (r250437)
  • Allowed pages using EventSource to enter the back-forward cache (r250761)
  • Allowed pages using FontFaceSet to enter the back-forward cache (r250693)
  • Allowed pages using IDBIndex to enter the back-forward cache (r250754)
  • Added basic back-forward cache support for RTCPeerConnection (r250379)
  • Changed IDBTransaction and IDBObjectStore to not prevent a page from entering the back-forward cache (r250531)
  • Fixed pages that frequently fail to enter the back-forward cache due to pending loads (r250414)
  • Fixed pages using WebGLRenderingContext to enter the back-forward cache (r250464)
  • Fixed pages with Web Workers to enter the back-forward cache (r250527)
  • Fixed pages using PendingImageBitmap to enter the back-forward cache (r250782)
  • Fixed ServiceWorkerContainer to never prevent a page from entering the back-forward cache (r250758)
  • Fixed XMLHttpRequest sometimes preventing pages from entering the back-forward cache (r250678)
  • Fixed IDBRequest to not prevent a page from entering the back-forward cache (r250425)
  • Fixed provisional and scheduled loads in subframes to not prevent a page from entering the back-forward cache (r250686)
  • Fixed RTCDataChannel to not prevent entering back-forward cache except if in an open state (r250573)
  • Made fixes to allow youtube.com to enter the back-forward cache on macOS (r250935)
  • Improved Service Worker support for back-forward cache (r250378)

IndexedDB

  • Added size estimate for key path when estimating task size (r250666)
  • Fixed wrapping CryptoKeys for IndexedDB during serialization (r250811)
  • Included size of index records in size estimate of put/add task (r250936)
  • Updated size to actual disk usage only when estimated increase is bigger than the space available (r250937)

October 16, 2019 05:00 PM

October 02, 2019

Release Notes for Safari Technology Preview 93

Surfin’ Safari

Safari Technology Preview Release 93 is now available for download for macOS Mojave and the macOS Catalina beta. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 249750-250329.

Resource Timing

  • Updated to report performance entries with all HTTP status codes (r250167)

SVG

  • Added auto behavior for rx and ry to the SVG <ellipse> and <rect> elements (r250147)
  • Fixed SVG <animateMotion> to reset the element to its first animation frame if its fill is set to "remove" (r249974)
  • Fixed SMIL animations of SVG <view> element having no effect (r249843)

Web API

  • Added support for sync-xhr feature policy (r250288)
  • Changed to use the same parser for <meta http-equiv="refresh"> and Refresh HTTP header (r249792)
  • Fixed Node.replaceChild()‘s pre-replacement validation order (r249821)
  • Fixed parsing Access-Control-Expose-Headers correctly (r249946)
  • Fixed preserving Content-Type to be preserved on responses created from DOMCache (r249945)
  • Fixed Date.prototype.toJSON to properly execute (r249861)
  • Fixed HTMLVideoElement with a broken poster image to take a square dimension (r250100)
  • Fixed a case where the Intersection Observer intersection ratio becomes larger than 1 (r249845)
  • Fixed cropped dropdown shadow for <datalist> (r250260)
  • Fixed blocking insecure WebSocket URL connection attempts by Workers on secure pages (r250300)
  • Fixed posting a message to a redundant Service Worker should fail silently instead of throwing (r249781)
  • Improved CSP inheritance semantics (r250255)
  • Provided a prototype for AR QuickLook to trigger processing in the originating page (r249855)
  • Removed gopher from the list of special schemes in URLParser (r249941)

Web Inspector

  • Elements
    • Prevented showing the Changes details sidebar panel when selecting text nodes (r249788)
  • Resources
  • Canvas
    • Added GPUDevice content previews and device configuration (r249786)
    • Added a list of any GPURenderPipeline/GPUComputePipeline, including the content of any GPUShaderModule, for the associated GPUDevice (r250258)
  • Console
    • Added a “Show More” button to see more than 100 items in arrays / collections (r250087)
    • Decreased the amount of horizontal space used by autocompletion bubbles (r249848)
    • Improved autocompletion typing performance by avoiding global forced layouts (r249863)
  • Miscellaneous
    • Changed to allow undocked Web Inspector windows to be dragged around by the title text (r250063)

Accessibility

  • Exposed misspelling ranges for editable content to accessibility clients. (r249893)

Apple Pay

  • Added support for telling websites why a session was cancelled (r250048)
  • Cleaned up handling of summary items and payment method updates (r250179)

JavaScript

  • Added missing syntax errors for await in function parameter default expressions (r249925)

WebGPU

  • Ensured structs and arrays with pointers as fields are disallowed (r249787)
  • Removed null from the standard library (r249794)

Web Authentication

  • Added support for more than two FIDO protocol versions (r249927)

WebDriver

  • Fixed a bug that caused safaridriver —enable to not take effect immediately in some cases.

October 02, 2019 05:00 PM

Paulo Matos: A Brief Look at the WebKit Workflow

Igalia WebKit

As I learn about the workflow for contributing to JSC (the JavaScript Compiler) in WebKit, I took a few notes as I went along. However, I decided to write them as a post in the hope that they are useful for you as well. If you use git, a Unix based system, and want to start contributing to WebKit, keep on reading.

More…

By Paulo Matos at October 02, 2019 03:56 PM

September 23, 2019

Intelligent Tracking Prevention 2.3

Surfin’ Safari

Note: Read about past updates to this technology in other blog posts about Intelligent Tracking Prevention, the Storage Access API, and ITP Debug Mode.

Intelligent Tracking Prevention (ITP) version 2.3 is included in Safari on iOS 13, the iPadOS beta, and Safari 13 on macOS for Catalina, Mojave, and High Sierra.

Enhanced Prevention of Tracking Via Link Decoration

Our previous release, ITP 2.2, focused specifically on the abuse of so-called link decoration for the purposes of cross-site tracking. With ITP 2.2, when a webpage is navigated to from a domain classified by ITP and the landing URL has a query string or fragment, the expiry of persistent client-side cookies created on that page is 24 hours.

Unfortunately, we see continued abuse of link decoration, so ITP 2.3 takes two new steps to combat this.

Capped Lifetime For All Script-Writeable Website Data

Since ITP 2.2, several trackers have announced their move from first-party cookies to alternate first-party storage such as LocalStorage. ITP 2.3 counteracts this in the following way:

  1. website.example will be marked for non-cookie website data deletion if the user is navigated from a domain classified with cross-site tracking capabilities to a final URL with a query string and/or a fragment identifier, such as website.example?clickID=0123456789.
  2. After seven days of Safari use without the user interacting with a webpage on website.example, all of website.example’s non-cookie website data is deleted.

Together with ITP’s capped expiry of client-side cookies, this change removes trackers’ ability to use link decoration combined with long-term first-party website data storage to track users. Put differently, ITP 2.3 caps the lifetime of all script-writeable website data after a navigation with link decoration from a classified domain.

The reason why we cap the lifetime of script-writable storage is simple. Site owners have been convinced to deploy third-party scripts on their websites for years. Now those scripts are being repurposed to circumvent browsers’ protections against third-party tracking. By limiting the ability to use any script-writeable storage for cross-site tracking purposes, ITP 2.3 makes sure that third-party scripts cannot leverage the storage powers they have gained over all these websites.

document.referrer Downgraded To eTLD+1

Our research has found that trackers, instead of decorating the link of the destination page, decorate their own referrer URL and read the tracking ID through document.referrer on the destination page.

ITP 2.3 counteracts this by downgrading document.referrer to the referrer’s eTLD+1 if the referrer has link decoration and the user was navigated from a classified domain. Say the user is navigated from social.example to website.example and the referrer is https://sub.social.example/some/path/?clickID=0123456789. When social.example’s script on website.example reads document.referrer to retrieve and store the click ID, ITP will make sure only https://social.example is returned.

For further reading on misuse of the referrer and changes coming to browsers in general, see the WHATWG Fetch issue Limit the length of the Referer header.

Updates To the Storage Access API

Developer Enhancement Requests

Developers have asked us for two changes to the Storage Access API, and we’re happy to provide them in Safari on iOS 13 beta, iPadOS beta, and macOS Catalina beta:

  • Only consume the user gesture (tap or click) when the user explicitly denies access, i.e. when the user is prompted and picks “Don’t allow.” Previously the gesture was also consumed when the promise was rejected without a user prompt, i.e. when the requesting domain was classified by ITP and had not received user interaction as first-party website the last 30 days of Safari use. This meant the user had to tap or click again to be shown a popup to log in to the third-party service.
  • Make document.hasStorageAccess() return true when ITP is off. Developers were confused when document.hasStorageAccess() returned false but ITP was off. Now it returns true in that case. Note that a returned true does not guarantee that the third-party can set cookies since Safari’s default cookie policy is to deny cookies for third-parties if they don’t already have cookies.

User Enhancement Request

Users have asked us to cap the number of times they can get asked for storage access by a specific piece of embedded web content.

Some services are requesting storage access on every click or tap, regardless of previous interactions with the user. To counter such repeated prompting, WebKit’s implementation of the Storage Access API will now automatically reject the request for storage access for documents where the user has picked “Don’t Allow” in the prompt twice. The reason why we don’t cap it to a single prompt is that the user may change their mind when they see the result of picking “Don’t Allow” the first time.

We will restrict the API further if we get reports of continued over prompting.

ITP Debug Mode In Safari on macOS Catalina

Safari 13 on macOS includes ITP Debug Mode. It’s been available in Safari Technology Preview for quite some time but now it’s available in regular Safari too so that you can debug your websites with the same Safari your customers are using.

Below is a step-by-step guide on how to use ITP Debug Mode. Note that both menu placement and naming has changed from the earlier experimental feature.

If you prefer to use Console:

  1. Launch the Console app.
  2. Click the Action menu → Include Info Messages.
  3. Filter on “ITPDebug” without the quotes.

If you prefer to use Terminal:

log stream -info | grep ITPDebug

Now you’re ready to enable ITP Debug Mode and see log output.

  1. Enable the Develop menu through Safari Preferences → Advanced → “Show Develop menu in menu bar.”
  2. Click “Intelligent Tracking Prevention Debug Mode” in the Develop menu.
  3. When you’re done, disable ITP Debug Mode through the same Develop menu item or by quitting Safari. Don’t leave ITP Debug Mode on when you’re not using it since it logs potentially sensitive information about your browsing. (Logging is done on the ephemeral INFO level.)

With ITP Debug Mode enabled, you will see ITP messages in the log as you browse the web. Whenever ITP decides to schedule deletion of website data it will indicate in the log “all data” or “all but cookies” to tell you whether it’s a regular deletion of all website data or the new capped lifetime of all non-cookie website data as explained above.

The domain 3rdpartytestwebkit.org is permanently classified with tracking capabilities in ITP Debug Mode, so at minimum you should see that domain show up in your log.

Classifying A Custom Domain For Testing

With ITP Debug Mode and User Defaults, you can manually set a custom domain as permanently classified with tracking capabilities. Here’s how you achieve this for a domain called website.example:

  1. Open System Preferences, click Security & Privacy, and unlock the padlock to be able to make changes.
  2. Pick the Full Disk Access category to the left.
  3. Use the + button to add the Terminal application to the list and make sure its checkbox is ticked.
  4. Open Terminal and execute the following command: defaults write com.apple.Safari ITPManualPrevalentResource website.example
  5. Go back to Security & Privacy in System Preferences and untick the checkbox for Terminal to not allow it permanent full disk access.

A Note On HttpOnly Cookies

Our blog post on ITP 2.1 provided guidance on how to protect cookies. We specifically encourage the use of Secure and HttpOnly cookies.

Since publishing that post, we’ve seen some confusion regarding the term HttpOnly cookie, sometimes mistakingly shortened to just “HTTP cookie.” Some say any cookie set by a server is an HttpOnly cookie. That is incorrect. The server needs to add the HttpOnly attribute in the Set-Cookie response header to make a cookie be HttpOnly, like so:

Set-Cookie: ExampleSessionID=a66e30012cc49846; path=/; HttpOnly

Adding the HttpOnly attribute provides these two security and privacy protections:

  • HttpOnly cookies are not exposed to JavaScript. This means tracking scripts or cross-site scripting attacks on the website cannot read and leak the contents of those cookies.
  • HttpOnly cookies are not copied into the web content process in WebKit. This means they are out of reach for speculative execution attacks that could otherwise steal the contents of those cookies.

September 23, 2019 05:00 PM

September 18, 2019

Release Notes for Safari Technology Preview 92

Surfin’ Safari

Safari Technology Preview Release 92 is now available for download for macOS Mojave and the macOS Catalina beta. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 249190-249750.

JavaScript

  • Fixed Math.round() which produced a wrong result for value prior to 0.5 (r249597)
  • Made Promise implementation faster (r249509)

WebGPU

  • Fixed matrices to have correct alignment (r249214)
  • Implemented GPUUncapturedErrorEvent (r249539)
  • Implemented SampleLevel(), SampleBias(), and SampleGrad() in WSL (r249453)
  • Updated several interface and enum names to match specifications (r249601)

SVG

  • Fixed fragment-only URL url(#fragment) to be resolved against the current document regardless of the HTML <base> element (r249416)
  • Fixed SMIL animations of SVG <view> element (r249730)
  • Made a change to get the value of the href attribute or the xlink:href attribute of SVG animation elements to get the animation target element (r249216)

Images

  • Changed to respect EXIF orientations by default when images are rendered (r249364)

Web API

  • Fixed copying and pasting two paragraphs with a newline between them resulting in a stray paragraph with newline inside (r249307)
  • Fixed cancelled transitions on Google image search leaving content with opacity: 0 sometimes (r249511)
  • Fixed document.fonts.ready resolving too quickly (r249295)
  • Fixed responseXML for XMLHttpRequest in some cases returning null if a valid Content-Type ends with +xml (r249361)
  • Made tabIndex IDL attribute reflect its content attribute (r249237)
  • Updated HTMLImageElement::decode() to return a resolved promise for decoding non-bitmap images (r249367)
  • Updated geolocation.watchPosition() and geolocation.getCurrentPosition() to return PERMISSION_DENIED when the context is not secure (r249207)

Service Workers

  • Added missing origin check for Service-Worker-Allowed header (r249733)
  • Added support for postMessage buffering between the Service Worker and window (r249629)
  • Dropped support for registration resurrection (r249627)

WebRTC

  • Added support to RTCDataChannel.send(Blob) (r249710)
  • Fixed audio sometimes failing to be captured in WebRTC (r249715)

IndexedDB

  • Changed to cache prepared SQLiteStatement in SQLiteIDBCursor to improve performance (r249729)
  • Changed to use the SQL COUNT statement for count operations to improve performance (r249583)
  • Updated the size of the database when the database operation is completed (r249333)

Web Inspector

  • Network
    • Provided a way to view XML, HTML, and SVG resource responses as a DOM tree (r249451)
  • Debugger
    • Added support for async event listener stack traces in Workers (r249315)
    • Added support for event breakpoints in Worker contexts (r249305)
    • Allow script resources to be blackboxed, which will prevent the debugger from pausing in that script (r249450)
  • Resources
    • Provide a way to override the content of resources loaded over the network with local content in Web Inspector (r249504)
    • Fixed issue where links to CSS resources didn’t map to the right line after pretty printing if the line is after a multiline comment (r249596)
    • Fixed issue where the closing } of nested @media weren’t indented (r249607)
  • Dark Mode
    • Fixed jarring white box-shadows in the Overview Timeline View in dark mode (r249655)
  • Miscellaneous
    • Fixed import file pickers sometimes not importing (r249248)

Accessibility

  • Fixed children cache to be re-computed if the tab index is removed (r249534)

Security

  • Disabled TLS 1.0 and TLS 1.1 in WebSockets (r249684)

September 18, 2019 05:30 PM

Michael Catanzaro: Epiphany Technology Preview Users: Action Required

Igalia WebKit

Epiphany Technology Preview has moved from https://sdk.gnome.org to https://nightly.gnome.org. The old Epiphany Technology Preview is now end-of-life. Action is required to update. If you installed Epiphany Technology Preview prior to a couple minutes ago, uninstall it using GNOME Software and then reinstall using this new flatpakref.

Apologies for this disruption.

The main benefit to end users is that you’ll no longer need separate remotes for nightly runtimes and nightly applications, because everything is now hosted in one repo. See Abderrahim’s announcement for full details on why this transition is occurring.

By Michael Catanzaro at September 18, 2019 02:19 PM

September 12, 2019

WebGPU and WSL in Safari

Surfin’ Safari

WebGPU is a new API being developed by Apple and others in the W3C which enables high-performance 3D graphics and data-parallel computation on the Web. This API represents a significant improvement over the existing WebGL API in both performance and ease of use. Starting in Safari Technology Preview release 91, beta support is available for WebGPU API and WSL, our proposal for the WebGPU shading language.

A few months ago we discussed a proposal for a new shading language called Web High-Level Shading Language, and began implementation as a proof of concept. Since then, we’ve shifted our approach to this new language, which I will discuss a little later in this post. With help from our friends at Babylon.js we were able to adapt a demo to work with WebGPU and Web Shading Language (WSL):

You can see the demo in action with Safari Technology Preview 91 or later, and with the WebGPU experimental feature enabled.1 The demo utilizes many of the best features of WebGPU. Let’s dig in to see what make WebGPU work so well, and why we think WSL is a great choice for WebGPU.

WebGPU JavaScript API

Just like WebGL, the WebGPU API is accessed through JavaScript because most Web developers are already familiar with working in JavaScript. We expect to create a WebGPU API that is accessed through WebAssembly in the future.

Pipeline State Objects and Bind Groups

Most 3D applications render more than a single object. In WebGL, each of those objects requires a collection of state-changing calls before that object could be rendered. For example, rendering a single object in WebGL might look like:

gl.UseProgram(program1);
gl.frontFace(gl.CW);
gl.cullFace(gl.FRONT);
gl.blendEquationSeparate(gl.FUNC_ADD, gl.FUNC_MIN);
gl.blendFuncSeparate(gl.SRC_COLOR, gl.ZERO, gl.SRC_ALPHA, gl.ZERO);
gl.colorMask(true, false, true, true);
gl.depthMask(true);
gl.stencilMask(1);
gl.depthFunc(gl.GREATER);
gl.drawArrays(gl.TRIANGLES, 0, count);

On the other hand, rendering a single object in WebGPU might look like:

encoder.setPipeline(renderPipeline);
encoder.draw(count, 1, 0, 0);

All the pieces of state in the WebGL example are wrapped up into a single object in WebGPU, named a “pipeline state object.” Though validating state is expensive, with WebGPU it is done once when the pipeline is created, outside of the core rendering loop. As a result, we can avoid performing expensive state analysis inside the draw call. Also, setting an entire pipeline state is a single function call, reducing the amount of “chatting” between Javascript and WebKit’s C++ browser engine.

Resources have a similar story. Most rendering algorithms require a set of resources in order to draw a particular material. In WebGL, each resource would be bound one-by-one, leading to code that looks like:

gl.bindBufferRange(gl.UNIFORM_BUFFER, 0, materialBuffer1, 0, size1);
gl.bindBufferRange(gl.UNIFORM_BUFFER, 1, materialBuffer2, 0, size2);
gl.activeTexture(0);
gl.bindTexture(gl.TEXTURE_2D, materialTexture1);
gl.activeTexture(1);
gl.bindTexture(gl.TEXTURE_2D, materialTexture2);

However, in WebGPU, resources are batched up into “bind groups.” When using WebGPU, the behavior of the above code is represented as simply:

encoder.setBindGroup(0, bindGroup);

In both of these examples, multiple objects are gathered up together and baked into a hardware-dependent format, which is when the browser performs validation. Being able to separate object validation from object use means the application author has more control over when expensive operations occur in the lifecycle of their application.

Run-time Performance

We expect WebGPU to perform faster and handle larger workloads than WebGL. To measure performance, we adopted the test harness from our 2D graphics benchmark MotionMark and wrote two versions of a performance test that had a simple yet realistic workload, one in WebGL and the other in WebGPU.

The test measures how many triangles with different properties can be drawn while maintaining 60 frames per second. Each triangle renders with a different draw call and bind group; this mimics the way many games render objects (like characters, explosions, bullets, or environment objects) in different draw calls with different resources. Because much of the validation logic in WebGPU is performed during the creation of the bind group instead of inside the draw call, both of these calls execute much faster than the equivalent calls in WebGL.

Web Shading Language

The WebGPU community group is actively discussing which shading language or languages should be supported by the specification. Last year, we proposed a new shading language for WebGPU. We are pleased to announce that a beta version of this language is available in Safari Technology Preview 91.

Because of community feedback, our approach toward designing the language has evolved. Previously, we designed the language to be source-compatible with HLSL, but we realized this compatibility was not what the community was asking for. Many wanted the shading language to be as simple and as low-level as possible. That would make it an easier compile target for whichever language their development shop uses.

There are many Web developers using GLSL today in WebGL, so a potential browser accepting a different high level language, like HLSL, wouldn’t suit their needs well. In addition, a high-level language such as HLSL can’t be executed faithfully on every platform and graphics API that WebGPU is designed to execute on.

So, we decided to make the language more simple, low-level, and fast to compile, and renamed the language to Web Shading Language to match this pursuit.2

The name change reflects a shift in our approach, but the major tenets of WSL have not changed. It’s still a language that is high-performance. It’s still text-based, which integrates well with existing web technologies and culture. WSL’s semantics still bake in safety and portability. It still works great as a compile target. As expected for every web technology, the language is well-defined with a robust specification. And because the WebGPU Community Group owns the language, it matures in concert with the WebGPU API.

In an upcoming post, we’ll talk more about the technical details of this language update. Here, let’s discuss further about WSL’s performance characteristics.

Wire Size

The Babylon.js demo above demonstrates an area where WSL really shines. The shaders in the demo use complex physically-based algorithms to draw with the highest realism possible. Babylon.js creates these complex shading algorithms by stitching together snippets of shader source code at run-time. This way, Babylon’s shaders are tuned at run-time to the specific type of content the application contains. Since WSL is a textual language like GLSL, this kind of workflow can be accomplished simply with string concatenation.

Building shaders by string concatenation and compiling them directly is a dramatic improvement over languages which cannot do this kind of manipulation easily, like bytecode-based languages. For these other languages, generation of shaders at run-time requires the execution of an additional compiler, written in JavaScript or WebAssembly, to compile the generated source text into the bytecode form.

This extra step slows down performance in two ways. First, the execution of the additional compiler takes time. Second, the compiler source would have to be served with the rest of the web page’s resources, which increases the page size and lengthens the loading time of the web page. On the other hand, since WSL’s format is textual, there’s no additional compile step; you can just write it and run it!

As a point of comparison, let’s take the Babylon.js demo shown above. Babylon.js renders scenes by joining GLSL shader snippets at runtime. When using WSL, Babylon.js would simply serve the WSL snippets just like they’re serving GLSL snippets today. To do the same with SPIR-V, Babylon.js would serve their shader snippets along with a compiler that compiles these textual snippets down to SPIR-V. Chrome recently announced their support for WebGPU with SPIR-V using the same demo. Here is a comparison of the gzipped wire sizes for the shaders as WSL, and that of the shaders as GLSL and the SPIR-V compiler:

The top bar shows the wire size if Babylon.js used WSL directly. The middle bar shows the wire size of the GLSL source, plus the compiler that Babylon.js uses to compile their snippets into SPIR-V. The bottom bar shows the wire size if Babylon.js simply served the results of their SPIR-V compilation. Because of the dynamic nature of Babylon.js’s shaders, they assemble their shaders at runtime and compile them dynamically, so they require the SPIR-V compiler. However, other websites’ shaders may be more static, in which case they wouldn’t need the compiler and could serve the SPIR-V directly, like in the bottom bar in the graph above.

Compilation Performance

We’ve heard from developers that shader compilation performance is extremely important. One way the community group addressed this was designing WebGPU to support asynchronous compilation, which avoids blocking successive rendering commands. Asynchronous compilation isn’t a magic bullet, though. If the next rendering command uses a shader currently being compiled, that rendering command must wait for the compilation complete. So, we’ve put significant effort into making WSL compilation times fast.

The WebGPU community created a compute shader to demonstrate the flocking characteristics of “boids,” and we turned that sample into a compiler performance benchmark. Here we compare the compilation time of WSL with the run-time of the GLSL compiler to SPIR-V running as a WebAssembly library:

As you can see, we’re continuing to optimize compilation performance, and it’s been increasing both throughout and after STP 91’s release.

Give WebGPU a try!

We’re very excited to have implemented a beta version of WebGPU and WSL in the latest version of Safari Technology Preview. Try it out and let us know how well it works for you! We’re interested in feedback so we can evolve the WebGPU and WSL language to better suit everyone’s needs.

And be sure to check out our gallery of WebGPU samples. We’ll be keeping this page updated with the latest demos. Many thanks to Sebastien Vandenberghe and David Catuhe from Babylon.js for their help with Babylon.js in Safari.

For more information, you can contact me at mmaxfield@apple.com or @Litherum, or you can contact our evangelist, Jonathan Davis.


1 To enable WebGPU beta support, in Develop menu, select Experimental Features > WebGPU.
2 We still pronounce it “whistle”, though.

September 12, 2019 05:00 PM

September 08, 2019

Michael Catanzaro: WebKit Vulnerabilities Facilitate Human Rights Abuses

Igalia WebKit

Chinese state actors have recently abused vulnerabilities in the JavaScriptCore component of WebKit to hack the personal computing devices of Uighur Muslims in the Xinjiang region of China. Mass digital surveillance is a key component of China’s ongoing brutal human rights crackdown in the region.

This has resulted in a public relations drama that is largely a distraction to the issue at hand. Whatever big-company PR departments have to say on the matter, I have no doubt that the developers working on WebKit recognize the severity of this incident and are grateful to Project Zero, which reported these vulnerabilities and has previously provided numerous other high-quality private vulnerability reports. (Many other organizations deserve credit for similar reports, especially Trend Micro’s Zero Day Initiative.)

WebKit as a project will need to reassess certain software development practices that may have facilitated the abuse of these vulnerabilities. The practice of committing security fixes to open source long in advance of corresponding Safari releases may need to be reconsidered.

Sadly, Uighurs should assume their personal computing devices have been compromised by state-sponsored attackers, and that their private communications are not private. Even if not compromised in this particular incident, similar successful attacks are overwhelmingly likely in the future.

By Michael Catanzaro at September 08, 2019 05:32 PM

September 04, 2019

Release Notes for Safari Technology Preview 91

Surfin’ Safari

Safari Technology Preview Release 91 is now available for download for macOS Mojave and the macOS Catalina beta. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 248705-249190.

Security

  • Disabled legacy TLS 1.0 and TLS 1.1 (r249019)

JavaScript API

  • Added a public API for unhandled promise rejections (r249058)
  • Added support for hashbang in ESNext (r248826)
  • Implemented optional chaining in ESNext (r248829)
  • Implemented StaticRange constructor (r249079)
  • Fixed Date.prototype.toJSON to not throw an exception if toISOString returns an object (r248876)
  • Fixed more missing exception checks in String.prototype (r248716)
  • Fixed a bad error message when for-await-of is used in a non-async function (r248711)
  • Fixed ProxyObject to not allow access to its target’s private properties (r248796)
  • Updated the Promise constructor to check argument before `Construct?“ (r248787)
  • Updated Promise.prototype.finally to accept non-promise objects (r248793)

JavaScript Performance

  • Changed to avoid looking up the join function each time Array.prototype.toString is called (r248906)
  • Ensured x?.y ?? z is fast (r249117)

Media

  • Fixed firing webkitpresentationmodechanged twice when exiting picture-in-picture (r249141)
  • Updated to stop MediaDevices timer when stopping MediaDevices (r248853)
  • Fixed removing fullscreen element in a requestAnimationFrame() callback after requestFullscreen() leaving fullscreen in an inconsistent state (r249147)
  • Disabled devices should not be taken into account when searching for a capture device (r249154)

Web API

  • Started to expose Geolocation interfaces (GeolocationPosition, GeolocationPositionError, and GeolocationCoordinates) on the global Window object (r249066)
  • Fixed Emoji with variation selectors to be rendered in emoji style, not text style (r248815)
  • Changed SVG elements to become focusable when focus and key event listeners are added (r248983)
  • Changed the default tab index of output and fieldset to be -1 (r248914)

Rendering

  • Fixed drawing an animated image to a canvas via drawImage to draw the first frame (r249162)

Pointer Events

  • Fixed removing the capture element preventing future pointer events from being dispatched on macOS (r248855)

WebDriver

  • Fixed mouse buttons to be correctly printed in SimulatedInputDispatcher logging (r248715)

Web Inspector

  • Elements
    • Fixed issue where “Copy Rule” menu item copied commented out properties incorrectly (r249089)
    • Changed to automatically enable the event listener when setting a breakpoint on it in the Node details sidebar (r248765)
    • Changed the DOM tree to always be LTR, even in RTL mode (r248991)
  • Network
    • Changed the Headers pane to always be LTR, even in RTL mode (r248889)
  • Resources
    • Added syntax highlighting for more CSS media queries (r248810)
    • Added syntax highlighting for JavaScript BigInt (r248898, r248908)
    • Added pretty printing support for more modern JavaScript language features (r248760, r248785, r248922, r248923)
    • Fixed a CodeMirror issue where tabs were still used even when the “Prefer indent using” setting is set to “Spaces” (r248739)
  • Debugger
    • Added a global breakpoint for pausing in the next microtask (r248894)
    • Fixed an issue where the Assertion Failures breakpoint didn’t work when inspecting a JSContext (r248891)
  • Console
    • Implemented queryHolders Command Line API function (r248925)
    • Updated console.dir to start out expanded when given an object (r249034)
    • Created additional Command Line API functions for other console methods (r249078)
    • Changed the Console to always be LTR, even in RTL mode (r248766)
  • Sources (Experimental)
    • Fixed an issue where the gear icon would move to the second line when the navigation sidebar is narrow (r248818)
    • Fixed an issue where the “No Filter Results” message was displayed on top of all of the other content, preventing any interaction (r248737)
    • Gave Origins their own icon in the navigation sidebar (r248912)
    • Moved the resource type scope bar to be next to the filter in the navigation sidebar (r248916, r248940)
    • Provided a way to create an arbitrary Inspector Style Sheet (r248753)
  • Layers (Experimental)
    • Fixed the background of the 3D area not updating when transitioning to/from Dark Mode (r248735)

WebGPU

  • Added unary plus (r248756)
  • Changed enums to not be shadowed by local variables (r248812)
  • Fixed WebGPU layers not showing up sometimes (r248879)
  • Implemented GPUErrors for and relax GPUBuffer validation rules (r249183)
  • Made operator== native and added bool matrices (r248795)
  • Updated matrices to have constructors that take a flattened list of scalars (r248754)
  • Updated vertex shader and fragment shader to be able to come from two different programs (r248993)

September 04, 2019 05:00 PM

August 27, 2019

How Web Content Can Affect Power Usage

Surfin’ Safari

Users spend a large proportion of their online time on mobile devices, and a significant fraction of the rest is users on untethered laptop computers. For both, battery life is critical. In this post, we’ll talk about factors that affect battery life, and how you, as a web developer, can make your pages more power efficient so that users can spend more time engaged with your content.

What Draws Power?

Most of the energy on mobile devices is consumed by a few major components:

  • CPU (Main processor)
  • GPU (Graphics processing)
  • Networking (Wi-Fi and cellular radio chips)
  • Screen

Screen power consumption is relatively constant and mostly under the user’s control (via screen on-time and brightness), but the other components, the CPU, GPU and networking hardware have high dynamic range when it comes to power consumption.

The system adapts the CPU and GPU performance based on the current tasks being processed, including, of course, rendering web pages that the user is interacting with in their web browser and other apps using web content. This is done through turning some components on or off, and by changing their clock frequency. In broad terms, the more performance that is required from the chips, the lower their power-efficiency. The hardware can ramp up to high performance very quickly (but at a large power cost), then rapidly go back to a more efficient low-power state.

General Principles for Good Power Usage

To maximize battery life, you therefore want to reduce the amount of time spent in high-power states, and let the hardware go back to idle as much as possible.

For web developers, there are three states of interaction to think about:

  • When the user is actively interacting with the content.
  • When the page is the frontmost, but the user is not interacting with it.
  • When the page is not the frontmost content.

Efficient user interaction

Obviously it’s good to expend power at times when the user is interacting with the page. You want the page to load fast and respond quickly to touch. In many cases, the same optimizations that reduce time to first paint and time to user interactive will also reduce power usage. However, be cautious about continuing to load resources and to run script after the initial page load. The goal should be to get back to idle as fast as possible. In general, the less JavaScript that runs, the more power-efficient the page will be, because script is work on top of what the browser has already done to layout and paint the page.

Once the page has loaded, user interactions like scrolling and tapping will also ramp up the hardware power (mainly the CPU and GPU), which again makes sense, but make sure to go back to idle as soon as the user stops interacting. Also, try to stay on the browser “fast paths” — for example, normal page scrolling will be much more power-efficient than custom scrolling implemented in JavaScript.

Drive idle power usage towards zero

When the user is not interacting with the page, try to make the page use as little power as possible. For example:

  • Minimize the use of timers to avoid waking up the CPU. Try to coalesce timer-based work into a few, infrequent timers. Lots of uncoordinated timers which trigger frequent CPU wake-ups are much worse than gathering that work into fewer chunks.
  • Minimize continually animating content, like animated images and auto-playing video. Be particularly vigilant to avoid “loading” spinner GIFs or CSS animations that continually trigger painting, even if you can’t see them. IntersectionObserver can be used to runs animations only when they are visible.
  • Use declarative animations (CSS Animations and Transitions) where possible. The browser can optimize these away when the animating content is not visible, and they are more efficient than script-driven animation.
  • Avoid network polling to obtain periodic updates from a server. Use WebSockets or Fetch with a persistent connection, instead of polling.

A page that is doing work when it should be idle will also be less responsive to user interaction, so minimizing background activity also improves responsiveness as well as battery life.

Zero CPU usage while in the background

There are various scenarios where a page becomes inactive (not the user’s primary focus), for instance:

  • The user switches to a different tab.
  • The user switches to a different app.
  • The browser window is minimized.
  • The browser window is visible but is not the focused window.
  • The browser window is behind another window.
  • The space the window is on is not the current space.

When a page becomes inactive, WebKit automatically takes steps to save power:

In addition, WebKit takes advantage of features provided by the operating system to maximize efficiency:

  • on iOS, tabs are completely suspended when possible.
  • on macOS, tabs participate in App Nap, which means that the web process for a tab that is not visually updating gets lower priority and has its timers throttled.

However, pages can trigger CPU wake-ups via timers (setTimeout and setInterval), messages, network events, etc. You should avoid these when in the background as much as possible. There are two APIs that are useful for this:

  • Page Visibility API provides a way to respond to a page transitioning to be in the background or foreground. This is a good way to avoid updating the UI while the page is in the background, then using the visibilitychange event to update the content when the page becomes visible.
  • blur events are sent whenever the page is no longer focused. In that case, a page may still be visible but it is not the currently focused window. Depending on the page, it can be a good idea to stop animations.

The easiest way to find problems is Web Inspector’s Timelines. The recording should not show any event happening while the page is in the background.

Hunting Power Inefficiencies

Now that we know the main causes of power use by web pages and have given some general rules about creating power-efficient content, let’s talk about how to identify and fix issues that cause excessive power drain.

Scripting

As mentioned above, modern CPUs can ramp power use from very low, when the device is idle, to very high to meet the demands of user interaction and other tasks. Because of this, the CPU is a leading cause of battery life variance. CPU usage during page loading is a combination of work the browser engine does to load, parse and render resources, and in executing JavaScript. On many modern web pages, time spent executing JavaScript far exceeds the time spent by the browser in the rest of the loading process, so minimizing JavaScript execution time will have the biggest benefits for power.

The best way to measure CPU usage is with Web Inspector. As we showed in a previous post, the timeline now shows the CPU activity for any selected time range:

To use the CPU efficiently, WebKit distributes work over multiple cores when possible (and pages using Workers will also be able to make use of multiple cores). Web Inspector provides a breakdown of the threads running concurrently with the page’s main thread. For example, the following screenshot shows the threads while scrolling a page with complex rendering and video playback:

When looking for things to optimize, focus on the main thread, since that’s where your JavaScript is running (unless you’re using Workers), and use the “JavaScript and Events” timeline to understand what’s triggering your script. Perhaps you’re doing too much work in response to user or scroll events, or triggering updates of hidden elements from requestAnimationFrame. Be cognizant of work done by JavaScript libraries and third party scripts that you use on your page. To dig deeper, you can use Web Inspector’s JavaScript profiler to see where time is being spent.

Activity in “WebKit Threads” is mostly triggered by work related to JavaScript: JIT compilation and garbage collection, so reducing the amount of script that runs, and reducing the churn of JavaScript objects should lower this.

Various other system frameworks invoked by WebKit make use of threads, so “Other threads” include work done by those; the largest contributor to “Other thread” activity is painting, which we’ll talk about next.

Painting

Main thread CPU usage can also be triggered by lots of layout and painting; these are usually triggered by script, but a CSS animation of a property other than transform, opacity and filter can also cause them. Looking at the “Layout and Rendering” timeline will help you understand what’s causing activity.

If the “Layout and Rendering” timeline shows painting but you can’t figure out what’s changing, turn on Paint Flashing:

This will cause those paints to be briefly highlighted with a red overlay; you might have to scroll the page to see them. Be aware that WebKit keeps some “overdraw” tiles to allow for smooth scrolling, so paints that are not visible in the viewport can still be doing work to keep offscreen tiles up-to-date. If a paint shows in the timeline, it’s doing actual work.

In addition to causing power usage by the CPU, painting usually also triggers GPU work. WebKit on macOS and iOS uses the GPU for painting, and so triggering painting can cause significant increases in power. The additional CPU usage will often show under “Other threads” in the CPU Usage timeline.

The GPU is also used for <canvas> rendering, both 2D canvas and WebGL/WebGPU. To minimize drawing, don’t call into the <canvas> APIs if the canvas content isn’t changing, and try to optimize your canvas drawing commands.

Many Mac laptops have two GPUs, an “integrated” GPU which is on the same die as the CPU, and is less powerful but more power-efficient, and a more powerful, but more power-hungry “discrete” GPU. WebKit will default to using the integrated GPU by default; you can request the discrete GPU using the powerPreference context creation parameter, but only do this if you can justify the power cost.

Networking

Wireless networking can affect battery life in unexpected ways. Phones are the most affected due to their combination of powerful radios (the WiFi and cellular network chips) with a smaller battery. Unfortunately, measuring the power impact of networking is not easy outside of a lab, but can be reduced by following some simple rules.

The most direct way to reduce networking power usage is to maximize the use of the browser’s cache. All of the best practices for minimizing page load time also benefit the battery by reducing how long the radios need to be powered on.

Another important aspect is to group network requests together temporally. Any time a new request comes, the operating system needs to power on the radio, connect to the base station or cell tower, and transmit the bytes. After transmitting the packets, the radio remains powered for a small amount of time in case more packets are transmitted.

If a page transmits small amounts of data infrequently, the overhead can become larger than the energy required to transmit the data:

Networking Power Overhead of transmitting 2 packets with a delay between them

Such issues can be discovered from Web Inspector in the Network Requests Timeline. For example, the following screenshot shows four separate requests (probably analytics) being sent over several seconds:

Sending all the requests at the same time would improve network power efficiency.

Conclusion

Webpages can be good citizens of battery life.

It’s important to measure the battery impact in Web Inspector and drive those costs down. Doing so improves the user experience and battery life.

The most direct way to improve battery life is to minimize CPU usage. The new Web Inspector provides a tool to monitor that over time.

To achieve great battery life the goals are:

  • Drive CPU usage to zero in idle
  • Maximize performance during user interaction to quickly get back to idle

If you have any questions, feel free to reach me, Joseph Pecoraro, Devin Rousso, and of course Jon Davis.

August 27, 2019 05:00 PM

August 21, 2019

Release Notes for Safari Technology Preview 90

Surfin’ Safari

Safari Technology Preview Release 90 is now available for download for macOS Mojave and the macOS Catalina beta. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 248024-248705.

Web API

  • Fixed ping loads to not prevent page caching (r248265)
  • Prevented autofocus for cross-origin iframes (r248491)
  • Prevented navigations of frames about to get replaced by the result of evaluating javascript: URLs (r248410)
  • Updated Element.outerHTML to link missing attribute prefixes in some cases in HTML documents (r248042)
  • Updated the wrapper for navigator.geolocation to not become GC-collectable once its frame is detached (r248276)

Media

  • Fixed an issue where muted <video> elements could block the display from sleeping (r248387)

WebRTC

  • Fixed incorrect this in negotiationneeded event (r248267)

WebGPU

Web Inspector

  • Elements
    • Added a way to disable or set a breakpoint on all event listeners for a given DOM node or event type in the Node details sidebar panel (r248052)
    • Added showing @supports CSS groupings in the Styles details sidebar panel (r248602)
    • Added experimental quick-action icon buttons to each CSS rule in the Styles details sidebar panel (r248202)
    • Added display of radius values in Box Model section of the Computed details sidebar panel (r248328)
    • Fixed an issue where CSS variable swatches were not shown for var() with a fallback in the Styles details sidebar panel (r248279)
    • Fixed some RTL issues in the Computed details sidebar panel (r248390, r248311)
    • Moved psuedo-selector rules before inherited rules in the Styles details sidebar panel (r248204)
    • Moved the Box Model section to the top of the Computed details sidebar panel (r248683)
  • Resources
    • Fixed brotli-compressed resources to correctly show as being compressed in the Resources details sidebar (r248284)
    • Fixed to properly handle CSS comments with an escape character when pretty printing (r248197)
  • Debugger
    • Added a global breakpoint for “All Events” which will pause whenever any event listener is about to be fired (r248201)
  • Timelines
    • Made Heap Snapshots searchable (r248198)
    • Fixed an issue where Develop > Start Timeline Recording didn’t work when focused on a detached Web Inspector window (r248177)
  • Console
    • Changed to always show all navigation items in the header area of the split console (r248180)
    • Fixed issue where the execution context picker didn’t update when switching to the inferred context from auto (r248196)
    • Provided a way to set an alias for previous console evaluation values (e.g. $0, $1, …, $99) in case the inspected page has variables with the same name (r248287)
    • Renamed queryObjects console command line API to queryInstances for clarity (r248434)
    • Supported console.screenshot with dataURL strings (r248688)
  • Overlay
    • Changed to show page width and height information (r248053)
  • Settings
    • Added an Engineering pane to expose useful settings for WebKit engineers (r248391)

Bug Fixes

  • Fixed dragging an image from Safari to Notes to correctly appear (r248166)

August 21, 2019 08:00 PM

August 14, 2019

Announcing the WebKit Tracking Prevention Policy

Surfin’ Safari

Today we are publishing the WebKit Tracking Prevention Policy, covering:

  • What types of tracking WebKit will prevent.
  • When other tracking countermeasures come into play such as limiting capabilities and informed user consent.
  • How WebKit handles unintended impact of our tracking prevention.

We’d like to thank Mozilla for their anti-tracking policy which served as inspiration for ours.

Please send us your comments or questions about this policy to @webkit on Twitter.

August 14, 2019 10:44 PM

August 07, 2019

Release Notes for Safari Technology Preview 89

Surfin’ Safari

Safari Technology Preview Release 89 is now available for download for macOS Mojave and the macOS Catalina beta. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 247433-248024.

JavaScript

  • Implemented nullish coalescing with the ?? operator for ESNext (r247819)

Web API

  • Added referrerpolicy attribute support for <script> elements (r247509)
  • Changed window.openDatabase to be overridable (r247434)
  • Fixed an IndexedDB error where the starting version change transaction may be neglected (r247649)
  • Fixed the ability to unbold selected text when the system font is used (r247439)
  • Made storing cross-origin top-level prefetches in HTTP cache optional (r247860)
  • Moving right by word boundary right before an object element followed by a <br> element hangs (r247881)
  • Removed support for beforeload on link=prefetch (r247481)

Compatibility

  • Fixed Daring Fireball long press highlights that were unnecessarily inflated due to false illegibility (r247792)
  • Fixed contextual menu actions for YouTube videos (r247901)

Accessibility

  • Exposed the aria-label attribute for <video> elements (r247891)

Media

  • Enabled a WebRTC debug mode without encryption (r247438)
  • Fixed support for FLAC-in-MP4 (r247934)

Web Inspector

  • Added support for console.screenshot with detached (not in main DOM tree) <img> and <picture> elements (r247814)
  • Added support for console.screenshot with ImageData and ImageBitmap (r247812)
  • Added a “Show Grid” navigation item for the Images collection in the Resources tab (r248004)
  • Added an indicator/separator around items in the Images collection in the Resources tab (r248019)
  • Add special case for about:blank resources to show “Resource has no content” message (r247747)
  • Fixed display of application/xml content for XHR requests (r247533)
  • Fixed issues toggling multiple breakpoints when they’re collapsed into one line (r247639)
  • Fixed Command-X (⌘X) to cut selected properties in the Styles sidebar (r247760)
  • Made the Changes panel in the Elements Tab render with LTR text direction (r247492)

WebGPU

  • Implemented errors for GPURenderPipeline creation (r247764)
  • Added descriptive error messages in WHLSL (r247834)
  • Changed checker to setError() when a property access node can’t commit its base type in WHLSL (r247676)
  • Changed to return the zero-value enum in the enum-from-integer constructor when the integer is not a valid enum value in WHLSL (r247675)
  • Made enums work in WHLSL (r247666)
  • Updated matrix memory layout to match HLSL by laying out columns linearly in WHLSL (r247468)

August 07, 2019 05:00 PM

August 05, 2019

Philippe Normand: Review of the Igalia Multimedia team Activities (2019/H1)

Igalia WebKit

This blog post takes a look back at the various Multimedia-related tasks the Igalia Multimedia team was involved in during the first half of 2019.

GStreamer Editing Services

Thibault added support for the OpenTimelineIO open format for editorial timeline information. Having many editorial timeline information formats supported by OpenTimelineIO reduces …

By Philippe Normand at August 05, 2019 01:30 PM

August 02, 2019

Changing Page Settings on iOS Using Web Inspector

Surfin’ Safari

If you’ve ever used Web Inspector before, chances are you’ve used (or are at least familiar with) the Develop menu. It holds action items and toggles for various settings of the browser, like whether local files (e.g. URLs beginning with file://) can be loaded or whether CSS is applied to each page.

All of these items apply to the entire browser, meaning that if you Disable Styles on one page, every other page will be affected.

Additionally, these items have no effect when using Web Inspector to inspect a remote target, like an iOS device or simulator. Checking Disable Styles in the Develop menu will not have any affect on the remote target.

In order to support this development workflow, Web Inspector has added a device settings menu that allows these settings to be toggled per-page when being remotely inspected.

Clicking on the device settings menu icon will show a popover with many of the same settings as the Develop menu.

Since these settings apply per-page and only on the remote target, the corresponding actions in the Develop menu are disabled, as they have no effect on a remote target:

  • Disable Images
  • Disable Styles
  • Disable JavaScript
  • Disable Site-specific Hacks
  • Disable Cross-Origin Restrictions
  • WebRTC
    • Allow Media Capture on Insecure Sites
    • Disable ICE Candidate Restrictions
    • Use Mock Capture Devices

Along these lines, the device settings menu is only shown when using Web Inspector to inspect a remote target.

Device settings are not preserved between Web Inspector sessions. Closing Web Inspector (or disconnecting the inspected device) will cause all previously set device settings for the inspected page to reset.

Device settings are preserved across navigations, however, so long as Web Inspector stays open/connected.

User Agent

The first item in the device settings menu is the User Agent editor. It contains a list of common user agents, as well as an option to input a custom user agent (Other…).

Each time the User Agent is modified, the inspected page will automatically reload so that the new User Agent is applied.

Disable Toggles

Each of these toggles, when checked, disables a specific piece of functionality in the inspected page.

  • Images will prevent any not-yet loaded images from loading, but will have no effect on any already loaded images.
  • Styles will immediately disable all CSS on the page, including inline <style>s and any style DOM attributes.
  • JavaScript will cause the page to ignore any future JavaScript from being run, including new <script> elements (the underlying resource isn’t even requested) and callbacks for previously added event listeners.
  • Site-specific Hacks controls whether workarounds are made by WebKit to support compatibility on certain sites.
    • A list of these sites can be found in Source/WebCore/page/Quirks.cpp.
    • If you develop a site that is found in that list, we strongly encourage developing and testing with Site-specific Hacks disabled.
  • Cross-Origin Restrictions controls whether CORS rules/restrictions are active for any future network requests.

WebRTC Toggles

These toggles focus specifically on functionality related to WebRTC.

  • Allow Media Capture on Insecure Sites will allow WebRTC media capture to be used/tested on insecure (e.g. non-https) pages for any future calls of getUserMedia.
  • Disable ICE Candidate Restrictions will prevent host ICE candidates from being filtered for new connection attempts.
  • Use Mock Capture Devices will replace all capture devices with a mock “Bip-Bop” device for any future calls of getUserMedia.
  • Disable Encryption will cause future connections to be established and future streams (from those connections) to be transmitted without any form of encryption.

Feedback

You can try out changing device settings with iOS 12.2 or later. Let us know how it works for you. Send feedback on Twitter (@webkit, @dcrousso) or by filing a bug.

August 02, 2019 05:56 AM

July 24, 2019

Release Notes for Safari Technology Preview 88

Surfin’ Safari

Safari Technology Preview Release 88 is now available for download for macOS Mojave and the macOS Catalina beta. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 247047-247433.

JavaScript

  • Enabled Intl.PluralRules and Intl.NumberFormatToParts by default (r247247)

WebRTC

  • Registered a MediaStreamTrack as media producer only if it is a capture track (r247208, r247382)

Web API

  • Fixed XHR CORS requests getting logged twice on the server (r247276)

Pointer Events

  • Updated to respect pointer capture when dispatching mouse boundary events and updating :hover (r247148)

Rendering

  • Changed to avoid extra backing store on elements with overflow: scroll and visibility: hidden (r247420)
  • Changed to trigger a compositing update when a video element is changing (r247187)

Accessibility

  • Added accessibility announcement notifications to show, dismiss and selection change for the datalist suggestions view (r247418)
  • Exposed HTML datalist accessibility (r247295)
  • Enhanced support of aria-haspopup per ARIA 1.1 specification. (r247071)
  • Implemented support for ARIA roles: insertion, deletion, subscript, superscript, and time (r247349)
  • Fixed ignored role="presentation" on HTML <table> elements for VoiceOver (r247330)

WebGL

  • Hooked up WebGL’s back buffer in ANGLE backend on macOS (r247315)

WebGPU

  • Added most of the remainder of the standard library (r247174)
  • Implemented GPUError and error scopes (r247366)
  • Made the destructor of VariableDeclaration non-virtual (r247124)
  • Optimized the lexer (r247171)
  • Removed the phase resolveCallsInFunctions (r247170)

Web Inspector

  • Added support for pasting copied DOM nodes in the Elements tab (r247054)
  • Fixed dismissing a blank property to no longer cause the rule to appear in the Changes panel (r247406)
  • Fixed an issue where unbalanced quotes or parenthesis weren’t displayed as properly closed after editing values (r247196)
  • Updated descendant DOM breakpoints be enabled, disabled, or deleted from a collapsed parent (r247053)

Bug Fixes

  • Fixed paste from Notes into Excel 365 spreadsheet (r247222)
  • Fixed video playback on xfinity.com/stream on macOS Catalina (r247213)

July 24, 2019 05:00 PM

July 10, 2019

Release Notes for Safari Technology Preview 87

Surfin’ Safari

Safari Technology Preview Release 87 is now available for download for macOS Mojave and the macOS Catalina beta. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 246691-247047.

Web API

  • Changed openDatabase to return an empty object when WebSQL is disabled (r246707)
  • Added an experimental behavior to prevent a 5 second delay for initial paint on pages that are using Google’s anti-flicker optimization when content blockers are enabled (r246764)

Web Sockets

  • Added support for sending blob messages when using web sockets platform APIs (r246964)

Payment Request

  • Changed to set state to Closed when show() is called during an active session (r246863)

Experimental Web Shading Language (WHLSL)

Rendering

  • Fixed incorrect clipping with overflow: scroll inside overflow: hidden with border-radius (r246845)
  • Fixed the preview of a <picture> element to match the element bounds (r246695)

WebGPU

  • Made WebGPU enabled only on Mojave and later (r246888)

Web Inspector

  • Implemented console.countReset (r246850)
  • Implemented console.timeLog (r246798)
  • Added additional demo audits for other WebInspectorAudit functions (r247042)
  • Enable the Show Shadow DOM navigation item by default in the Elements tab (r246821)

Known Issues

  • Open tabs are blank on initial launch after upgrading to Safari Technology Preview 87 and loading all tabs from the last session. To correct this behavior, reload open tabs or relaunch Safari Technology Preview.

July 10, 2019 05:00 PM

July 08, 2019

WebDriver is Coming to Safari in iOS 13

Surfin’ Safari

As anyone who has developed a mobile-friendly site can tell you, mobile browsers and desktop browsers are different. As device capabilities continue to evolve and users move more of their browsing to mobile browsers, web developers are facing an increasing need to write automated tests for the mobile web in the same way that desktop-oriented site content is tested.

Starting in iOS 13, Safari now includes native support for the W3C WebDriver standard. This feature builds on existing support for WebDriver in desktop Safari, first introduced in Safari 10 and macOS Sierra. This blog post explains how to get started with WebDriver for iOS, how to port your existing desktop-oriented tests, and a few details regarding safeguards and changes in behavior.

Getting Started

Control via WebDriver is exposed to developers via the /usr/bin/safaridriver executable, which hosts a driver that handles REST API requests sent by WebDriver test clients. In order to run WebDriver tests on an iOS device, it must be plugged into a macOS host that has a new enough version of safaridriver. Support for hosting iOS-based WebDriver sessions is available in safaridriver included with Safari 13 and later. Older versions of safaridriver do not support iOS WebDriver sessions.

If you’ve never used safaridriver on macOS before, you’ll first need to run safaridriver --enable and authenticate as an administrator. Then, you must enable Remote Automation on every device that you intend to use for WebDriver. To do this, toggle the setting in Settings → Safari → Advanced → Remote Automation.

Remote Automation Setting on iOS

Lastly, to run tests on the device, you’ll need to plug it into the macOS machine, trust the host, and ensure that it is unlocked when you try to start the WebDriver session.

Since safaridriver is capable of starting WebDriver sessions using Safari on the macOS host or Safari on an attached iOS device, it needs a little extra hint to know that you want to use a device or a desktop machine. To facilitate this, safaridriver supports many new capabilities that may be provided with a new session request. The most important new capability to use is ‘platformName’: ‘ios’; this tells the driver you want an iOS host instead of a macOS host. You can also specify the host(s) you wish to use by specifying device type, device UDID, device name, OS version, OS build, and browser version via standardized and extension capabilities. Lastly, safaridriver supports running WebDriver tests on iOS Simulator devices whose runtime is iOS 13 or later. To use a simulator for testing, add the capability ‘safari:useSimulator’: true. For a full list of supported capabilities and their semantics, please consult the safaridriver(1) man page by executing man safaridriver.

Safeguards

While it’s awesome to be able to write automated tests that run in Safari, a REST API for browser automation introduces a potential vector for malicious software. To support a valuable automation API without sacrificing a user’s privacy or security, Safari’s driver implementation introduces extra safeguards that further restrict browsing in WebDriver sessions. These safeguards also double as extra insurance against “flaky tests” by providing enhanced test isolation. Some of these safeguards are described below.

Just like on macOS, Safari on iOS isolates WebDriver tests by using a separate set of windows, tabs, preferences, and persistent storage. When a WebDriver session is active, existing tabs are hidden and a distinctively-colored WebDriver window is shown instead. Automation windows are easy to recognize by their orange Smart Search field. Similar to browsing in a Private Browsing window, WebDriver tests that run in an Automation window always start from a clean slate and cannot access Safari’s normal browsing history, AutoFill data, or other sensitive information. Aside from the obvious privacy benefits, this restriction also helps to ensure that tests are not affected by a previous test session’s persistent state such as local storage or cookies. When the WebDriver session terminates, the orange window goes away, preexisting tabs are restored, and any state accumulated during the WebDriver session is destroyed.

As a WebDriver test is executing in an Automation window, any attempts to interact with the window or web content could derail the test by unexpectedly changing the state of the page. To prevent this from happening, Safari installs a “glass pane” over the Automation window while the test is running. This blocks any stray interactions (mouse, keyboard, resizing, and so on) from affecting the Automation window. If a running test gets stuck or fails, it’s possible to interrupt the running test by tapping on the screen and choosing to end the session. When an automation session is interrupted, the test’s connection to the browser is permanently severed and cannot be resumed. Notifications and system gestures (i.e., slide in from top/bottom) are also suppressed while WebDriver tests are running in order to avoid flakiness.

Glass Pane

A non-goal of this release is to support general-purpose device automation via the WebDriver API. The W3C WebDriver API focuses on automating interactions with web content, and that is also the focus of the WebDriver support added in iOS 13. Other means exist to change system-level configuration with the XCTest Framework, and are complimentary to WebDriver. There are other ways in which web content interacts with the rest of iOS, and most of these are suppressed in order to provide a stable testing environment. Safari for iOS does not allow WebDriver-initiated navigations to be handled outside of Safari. In other words, clicking a tel:// link will not offer to place a phone call, and clicking an app store link will not redirect the user to the App Store. Similarly, non-Safari content displayed by the system–such as update dialogs, app notifications, incoming calls, etc.–are suppressed while a WebDriver session is active. Lastly, the software keyboard is suppressed when focusing text fields in WebDriver sessions. To simulate text input with WebDriver, use the existing Perform Actions endpoint or Element Send Keys endpoint.

Porting Tests

A major goal of iOS 13 is to make it easier to develop and consume desktop-like web experiences on mobile devices. Towards that end, a major goal for WebDriver on iOS is to make it easy to port existing desktop-oriented WebDriver tests to work with Desktop-class browsing in Safari on iPadOS. As much as we’d like to treat macOS and iOS devices the same in our tests, there are important differences that affect how you may want to use WebDriver.

The first notable difference is handling of user input, namely touch and clicks. In this initial version of WebDriver on iOS, simulated input sources include keyboard, mouse, and single-finger touches. The Perform Actions command and Element Click command synthesize touches using mouse inputs. If your existing test uses mouse clicks, double, clicks, and drags, safaridriver converts these interactions into equivalent single finger touches; no code needs to be changed. Multiple touch input sources (i.e., fingers) are not supported in this release. We experimented with offering this via the existing Perform Actions endpoint, but found that it was difficult to author WebDriver commands that performed well-known gestures such as flick, pinch, pan, and so on. This is partly due to the low-level nature of the Actions API, and partly due to the timing-sensitive nature of gesture recognizers used by Safari and WebKit. Apple is working within the W3C WebDriver working group to define a more expressive, intuitive, and stable API for performing gestures in WebDriver tests.

A less surprising difference between macOS and iOS WebDriver is that endpoints to manipulate the OS window are generally not supported by Safari on iOS. Specifically, the Set Window Rect, Minimize Window, and Maximize Window endpoints will return “unsupported operation”. For more detailed information concerning which endpoints are supported, see the W3C WebDriver Commands reference.

Summary

With the introduction of native WebDriver support in Safari on iOS 13, it’s now possible to run the same automated tests of desktop-oriented web content on desktop and mobile devices equally. Safari’s support comes with new, exclusive safeguards to simultaneously protect user security and privacy and also help you write more stable and consistent tests. You can try out Safari’s WebDriver support today by installing a beta of macOS Catalina and iOS 13.

If you encounter unexpected behavior in Safari’s WebDriver support, or have other suggestions for improvement, please file a bug at https://bugreport.apple.com/. For quick questions or feedback, email web-evangelist@apple.com or tweet @webkit on Twitter. Happy testing!

July 08, 2019 05:00 PM

June 28, 2019

Michael Catanzaro: On Version Numbers

Igalia WebKit

I’m excited to announce that Epiphany Tech Preview has reached version 3.33.3-33, as computed by git describe. That is 33 commits after 3.33.3:

Epiphany about dialog displaying the version number

I’m afraid 3.33.4 will arrive long before we  make it to 3.33.3-333, so this is probably the last cool version number Epiphany will ever have.

I might be guilty of using an empty commit to claim the -33 commit.

I might also apologize for wasting your time with a useless blog post, except this was rather fun. I await the controversy of your choice in the comments.

By Michael Catanzaro at June 28, 2019 04:07 PM

June 26, 2019

Release Notes for Safari Technology Preview 86

Surfin’ Safari

Safari Technology Preview Release 86 is now available for download for macOS Catalina betas and macOS Mojave. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 246093-246691.

Pointer Events

  • Added support for chorded button interactions (r246103, r246149)
  • Updated to fire pointerout and pointerleave events after firing pointercancel (r246122)

JavaScript

  • Added support for String.prototype.matchAll (r246567))
  • Changed to throw a TypeError exception if Proxy’s set trap returns falsy value (r246346)
  • Fixed JSON.parse to throw a syntax error when called without arguments (r246162)

Web Assembly

  • Added support for Anyref tables, Table.get and Table.set for Anyref only (r246139)
  • Added support for multiple tables (r246571)
  • Added support for Table.size, grow and fill instructions (r246577)
  • Fixed float64 “select” on ARM64 devices (r246134)

WebRTC

  • Updated mediaDevices.enumerateDevices() to list the system default audio devices with deviceId as “default” (r246215)

Web API

  • Fixed IntersectionObserver rootMargin detection failure when root is an element (r246432)
  • Updated to avoid generating a new XSLT-based document when already changing the document. (r246182)

CSS

  • Implemented tab-size with units (r246193)
  • Included touch-action in the computed styles list (r246314)

Security

  • Added wildcard to Access-Control-Allow-Methods and Access-Control-Allow-Headers (r246238)
  • Changed blob URLs to inherit CSP policy from their parent (r246277)
  • Changed data URLs to inherit their CSP policy from their parent (r246129)

Experimental Web Shading Language (WHLSL)

  • Enabled compute shaders (r246427)
  • Implemented array references (r246394)
  • Implemented loop expressions (r246121)
  • Implemented out-of-bounds and nullptr behavior (r246438)
  • Supported matrices (r246579)

WebGPU

  • Removed GPUBuffer.setSubData and implemented GPUDevice.createBufferMapped (r246217)

Web Inspector

  • Included JavaScript call trees when exporting or importing Timeline recordings (r246292)
  • Improved the performance of the CSS pretty printer by using a Worker (r246178)
  • Split the General panel in the Settings tab into sub panels (r246510)
  • Ensured that indent type and size settings are respected everywhere that there’s a CodeMirror editor (r246419)
  • Added support for respecting case sensitive or regex global settings when searching in a resource (r246502)
  • Changed to ensure that adding a DOM, Event, or URL breakpoint is enabled globally (r246523)
  • Added support for multiline keys when using “Copy Path to Property” (r246271)
  • Fixed an issue where longhand CSS properties that were overridden by shorthands didn’t have a strikethrough (r246223)
  • Fixed an issue where query parameters in the Network tab would truncate if the value contained a = (r246559)
  • Fixed an issue where the Inspector Style Sheet was missing when grouping resources by path (r246509)

June 26, 2019 05:00 PM

June 21, 2019

A New Bytecode Format for JavaScriptCore

Surfin’ Safari

In revision r237547 we introduced a new bytecode format for JavaScriptCore (JSC). The goals of the new format were to improve memory usage and allow the bytecode to be cached on disk, while the previous format was optimized for interpreter throughput at the cost of memory usage.

In this post, we will start with a quick overview of JSC’s bytecode, key aspects of the old bytecode format and the optimizations it enabled. Next, we will look into the new format and how it affects interpreter execution. Finally, we will look at the impact of the new format on memory usage and performance and how this rewrite improved type safety in JavaScriptCore.

Background

Before JSC executes any JavaScript code, it must lex, parse and generate bytecode for it. JSC has 4 tiers of execution:

  • Low Level Interpreter (LLInt): the start-up interpreter
  • Baseline JIT: a template JIT
  • DFG JIT: a low-latency optimizing compiler
  • FTL JIT: a high-throughput optimizing compiler

Execution starts by interpreting the bytecode, at the lowest tier, and as the code gets executed more it gets promoted to a higher tier. This is described in a lot more details in this blog post about the FTL.

The bytecode is the source of truth throughout the whole engine. The LLInt executes the bytecode. The baseline is a template JIT, which emits snippets of machine code for each bytecode instruction. Finally, the DFG and FTL parse the bytecode and emit DFG IR, which is then run through an optimizing compiler.

Because the bytecode is the source of truth, it tends to stay alive in memory throughout the whole program execution. In JavaScript-heavy websites, such as Facebook or Reddit, the bytecode is responsible for 20% of the overall memory usage.

The Bytecode

To make things more concrete, let’s look at a simple JavaScript program, learn how to inspect the bytecode generated by JSC and how to interpret the bytecode dump.

// double.js
function double(a) {
    return a + a;
}
double(2);

If you run the above program with jsc -d double.js, JSC will dump all the bytecode it generates to stderr. The bytecode dump will contain the bytecode generated for double:

[   0] enter
[   1] get_scope          loc4
[   3] mov                loc5, loc4
[   6] check_traps
[   7] add                loc7, arg1, arg1, OperandTypes(126, 126)
[  13] ret                loc7

Each line starts with the offset of the instruction in brackets, followed by the opcode name and its operands. Here we can see the operands loc for local variables, arg for function arguments and OperandTypes, which is metadata about the predicted types of the arguments.

Old Bytecode Format

The old bytecode format had a few issues that we wanted to fix:

  • It used too much memory.
  • The instruction stream was writable, which prevented memory-mapping the bytecode stream.
  • It had optimizations that we no longer benefited from, such as direct threading.

In order to better understand how we addressed these issues in the new format, we need a basic understanding of the old bytecode format. In the old format, instructions could be in one of two forms: unlinked, which is compact and optimized for storage and linked, which is inflated and optimized for execution, containing memory addresses of runtime objects directly in the instruction stream.

Unlinked Instructions

The instructions were encoded using variable-width encoding. The opcode and each operand took as little space as possible, ranging from 1 to 5 bytes. Take the add instruction from the program above as an example, it would take 6 bytes: one for the opcode (add), one for each of the registers (loc7, arg1 and arg1 again) and two bytes for the operand types.

Unlinked instructions

Linking / Linked Instructions

Before executing the bytecode it needed to be linked. Linking inflated all the instructions, making the opcode and each of the operands pointer-sized. The opcode was replaced by the actual pointer to the implementation of the instruction and the profiling-related metadata replaced with the memory address of the newly allocated profiling data structures. The add from above took 40 bytes to represent:

Linked instructions

Execution

The bytecode is executed by the LLInt, which is written in a portable assembly called offlineasm. Here are a few notes about offlineasm that may help understand the code snippets that will follow:

  • Temporary registers are t0-t5, argument registers are a0-a3 and return registers are r0 and r1. For floating point, the equivalent registers are ft0-ft5, fa0-fa3 and fr. cfp and sp are special registers that hold the call frame and stack pointer respectively.
  • Instructions have one of the following suffixes: b for byte, h for 16-bit word, i for 32-bit word, q for 64-bit word and p for pointer (either 32- or 64-bit depending on the architecture).
  • Macros are lambda expressions that take zero or more arguments and return code. Macros may be named or anonymous and can be passed as arguments to other macros.

If you want to learn more about offlineasm, LowLevelInterpreter.asm is the main offlineasm file in JSC and it contains a more in depth explanation of the language at the top.

The old bytecode format had two big advantages related to interpreter throughput: direct threading and inline caching.

Direct Threading

The linked bytecode had the actual pointer to the offlineasm implementation of the instruction in place of the opcode, which made executing the next instruction as simple as advancing the program counter (PC) by the size of the current instruction and making an indirect jump. That is illustrated in the following offlineasm code:

macro dispatch(instructionSize)
    addp instructionSize * PtrSize, PC
    jmp [PC]
end

Notice that dispatch is a macro, this way the code is duplicated at the end of every instruction. This is very efficient, since it’s just an addition plus an indirect branch. The duplication reduces branch prediction pollution since we don’t have all instructions jumping to a common label and sharing the same indirect jump to the next instruction.

Inline Caching

Since the instruction stream is writable and all the arguments are pointer-sized, we can store metadata in the instruction stream itself. The best example of this is for the get_by_id instruction, which is emitted when we load from an object in JavaScript.

object.field

This emits a get_by_id for loading the property field from object. Since this is one of the most common operations in JavaScript, it’s crucial that it be fast. JSC uses inline caching as a way of speeding this up. The way this was done in the interpreter was by reserving space in the instruction stream to cache metadata about the load. More specifically, we recorded the StructureID of the object we are loading from and the memory offset from which we have to load the value. The LLInt implementation of get_by_id looked like the following:

_llint_op_get_by_id:
    // Read operand 2 from the instruction stream as a signed integer,
    // i.e. the virtual register of `object`
    loadisFromInstruction(2, t0)

    // Load `object` from the stack and its structureID
    loadConstantOrVariableCell(t0, t3, .opGetByIdSlow)
    loadi JSCell::m_structureID[t3], t1

    // Read the cached StructureID and verify that it matches the object's
    loadisFromInstruction(4, t2)
    bineq t2, t1, .opGetByIdSlow

    // Read the PropertyOffset of `field` and load the actual value from it
    loadisFromInstruction(5, t1)
    loadPropertyAtVariableOffset(t1, t3, t0)

    // Read the virtual register where the result should be stored and store
    // the value to it
    loadisFromInstruction(1, t2)
    storeq t0, [cfr, t2, 8]
    dispatch(constexpr op_get_by_id_length)

.opGetByIdSlow
    // Jump to C++ slow path
    callSlowPath(_llint_slow_path_get_by_id)
    dispatch(constexpr op_get_by_id_length)

New Bytecode Format

When designing the new bytecode, we had two major goals: it should be more compact and easily cacheable on disk. With these two goals, we expected significant improvements on memory usage and set ourselves to improve runtime performance through caching. This shaped how we encoded instructions as well as runtime metadata.

The first and biggest change is that we no longer have a separate linked encoding for execution. This immediately means that the bytecode can no longer be direct threaded, since the address of the instruction could not be stored to disk, as it changes with every program invocation. Removing this optimization was a deliberate choice of the design.

In order to make the single format suitable for both storage and execution, each instruction can be encoded as either narrow or wide.

Narrow Instructions

In a narrow instruction, the opcode and its operands each take 1-byte. Here’s the add instruction again, this time as a narrow instruction in the new format:

Narrow

Ideally, all instructions would be encoded as narrow, but not all operands fit into a single byte. If any of the operands (or the opcode for that matter) requires more than 1 byte, the whole instruction is promoted to wide.

Wide Instructions

A wide instruction consists of a special 1-byte opcode, op_wide, followed by a series of 4-byte slots for the original opcode and each of its arguments.

Wide Instructions

An important thing to notice here is that it is not possible to have a single wide operand – if one operand overflows, the whole instruction becomes wide. This is an important compromise, because even though it might seem wasteful at first, it makes the implementation much simpler: the offset of any given operand is the same regardless of the instruction being narrow or wide. The only difference is whether the instruction stream is treated as an array of 4-byte values or 1-byte values.

Linking / Metadata Table

The other fundamental part of the new bytecode is the metadata table. During linking, instead of constructing a new instruction stream, we initialize a side table with all the writable data associated with any given instruction. Conceptually, the table is 2-dimensional, its first index is the opcode for the instruction, e.g. add, which points to a monomorphic array of metadata entries for that specific instruction. For example, we have a struct called OpAdd::Metadata to hold the metadata for the add instruction, so accessing metadataTable[op_add] will result in a pointer of type OpAdd::Metadata*. Additionally, each instruction has a special operand, metadataID, which is used as the second index in the metadata table.

Metadata

For compactness, the metadata table is laid out in memory as a single block of memory. Here’s a representation of what the table above would actually look like in memory.

Metadata Memory Table

At the start of the table is the header, an array containing an integer for each opcode representing the offset from the start of the table where the metadata for that opcode starts. The next region is the payload, which contains the actual metadata for each opcode.

Execution

The biggest changes to execution are that the interpreter is now indirect threaded, and for any writable or runtime metadata, we need to read from the metadata table. Another interesting aspect is how wide instructions are executed.

Indirect Threading

The process of mapping from opcode to instruction address, which used to be done as part of linking in the old format, is now part of the interpreter dispatch. This means that extra loads are required, and the dispatch macro now looks like the following:

macro dispatch(instructionSize)
    addp instructionSize, PC
    loadb [PC], t0
    leap _g_opcodeMap, t1
    jmp [t1, t0, PtrSize]
end

Metadata Table

The same kind of “inline caching”† still applies to get_by_id, but given that we need to write the metadata at runtime and the instruction stream is now read-only, this data needs to live in the metadata table.

Loads from the metadata table are a bit costly, since the CodeBlock needs to be loaded from the call frame, the metadata table from the CodeBlock, the array for the current opcode from the table and then finally the metadata entry from the array based on the current instruction’s metadataID.

† I write “inline caching” in quotes here since it’s not technically inline anymore.

Wide Instruction Execution

The execution of wide instructions is surprisingly simple: there are two functions for each instruction, one for the narrow version and another for the wide. We take advantage of offlineasm to generate the two functions from a single implementation. (e.g. the implementation of op_add will automatically generate its wide version, op_add_wide.)

By default, the narrow version of the instruction is executed, until the special instruction op_wide is executed. All it does is read the next opcode, and dispatch to its wide version, e.g. op_add_wide. Once the wide opcode is done, it goes back to dispatching a narrow instruction, since every wide instruction needs the 1-byte op_wide prefix.

Memory Usage

The new bytecode format uses approximately 50% less memory, which means a reduction of 10% in overall memory usage for JavaScript-heavy websites, such as Facebook or Reddit.

Here’s a chart comparing the bytecode size for reddit.com, apple.com, facebook.com and gmail.com.

Bytecode size charts for reddit.com, apple.com, facebook.com and gmail.com

Notice that in the After case the Metadata has been merged with Linked to represent the memory used by the metadata table and Unlinked represents the actual bytecode instructions. The reason is that a significant part of what lives in the new metadata used to live in the old linked bytecode. Otherwise, the comparison would look skewed, since the whole Linked bar would be gone and the Metadata would seem to have doubled in size, when in reality it was part of the data from Linked that moved into Metadata.

Performance

As discussed earlier, indirect threading increases the overhead of the interpreter dispatch. However, given the average complexity of the bytecode instructions in JSC we did not expect the dispatch overhead to play a meaningful role in the overall performance of the interpreter. This is further amortized by the multiple JIT tiers. We profiled purely switching from direct to indirect threading in CLoop, the C++ backend for LLInt, and the results were neutral even with the JITs disabled.

Loads from the metadata table are costly, involving a large chain of loads. In order to reduce this chain, we pin a callee save register in the interpreter to hold the pointer to the metadata table at all times. Even with this optimization, it takes three loads to access the metadata entry. This was a 10% interpreter slowdown, but was neutral when running with all JIT tiers enabled.

Bonus: Type Safety

Changing the bytecode format required changes across the whole engine, so we decided to take this as an opportunity to improve the bytecode-related infrastructure, increasing the type safety, readability and maintainability of the code.

To support all this changes, first we needed to change how instructions were specified. Originally, instructions were declared in a JSON file, with their respective name and length (the number of operands plus one, for the opcode). Here is how the add instruction was declared:

{ "name": "op_add", "length": 5 }

When accessing the instruction and its operands from C++ the code would look roughly like this:

SLOW_PATH_DECL(slow_path_add)
{
    JSValue lhs = OP_C(2).jsValue();
    JSValue rhs = OP_C(3).jsValue();
    ...
}

Notice that operands were accessed by their offset: OP_C was a macro that would access the Instruction* pc argument (hidden here by the SLOW_PATH_DECL macro) at the given offset. However, the type of the data at the specified offset is unknown, so the result would have to be cast to the desired type. This is error-prone and makes the code harder to understand. The only way to learn which operands an instruction had (and their types) was to look for usages of it and look at variable names and type casts.

With the new infrastructure, when declaring an instruction, a name and type must be given for each of the operands, as well as declaring all the data that will be stored in the metadata table.

op :add,
    args: {
        dst: VirtualRegister,
        lhs: VirtualRegister,
        rhs: VirtualRegister,
        operandTypes: OperandTypes,
    },
    metadata: {
        arithProfile: ArithProfile,
    }

With this extra information, it is now possible to generate a fully typed struct for each instruction, which results in safer C++ code:

SLOW_PATH_DECL(slow_path_add)
{
   OpAdd bytecode = pc->as<OpAdd>();
   JSValue lhs = GET_C(bytecode.m_lhs);
   JSValue rhs = GET_C(bytecode.m_rhs);
   ...
}

In the example above, we first need to convert the generic Instruction* pc to the instruction we want to access, which will perform a runtime check. If the opcode matches, it returns an instance of the generated struct, OpAdd, which includes the fields m_lhs and m_rhs, both of type VirtualRegister as specified in our instruction declaration.

The extra information also allowed us to replace a lot of mechanical code with safer, generated code. For example, we auto generate all the type safe code for writing instructions to the instruction stream, which automatically does the narrow/wide fitting. We also generate all the code to dump the instructions for debugging.

Conclusion

JavaScriptCore has a new bytecode format that uses 50% less memory on average and is available on Safari 12.1 and Safari Technology Preview. The work on the caching API for the new bytecode is already underway in the WebKit repository, and anyone interested is welcome to follow along and contribute on bugs.webkit.org! You can also get in touch with me on Twitter with any questions or comments.

June 21, 2019 05:00 PM

June 14, 2019

Michael Catanzaro: An OpenJPEG Surprise

Igalia WebKit

My previous blog post seems to have resolved most concerns about my requests for Ubuntu stable release updates, but I again received rather a lot of criticism for the choice to make WebKit depend on OpenJPEG, even though my previous post explained clearly why there are are not any good alternatives.

I was surprised to receive a pointer to ffmpeg, which has its own JPEG 2000 decoder that I did not know about. However, we can immediately dismiss this option due to legal problems with depending on ffmpeg. I also received a pointer to a resurrected libjasper, which is interesting, but since libjasper was removed from Ubuntu, its status is not currently better than OpenJPEG.

But there is some good news! I have looked through Ubuntu’s security review of the OpenJPEG code and found some surprising results. Half the reported issues affect the library’s companion tools, not the library itself. And the other half of the issues affect the libmj2 library, a component of OpenJPEG that is not built by Ubuntu and not used by WebKit. So while these are real security issues that raise concerns about the quality of the OpenJPEG codebase, none of them actually affect OpenJPEG as used by WebKit. Yay!

The remaining concern is that huge input sizes might cause problems within the library that we don’t yet know about. We don’t know because OpenJPEG’s fuzzer discards huge images instead of testing them. Ubuntu’s security team thinks there’s a good chance that fixing the fuzzer could uncover currently-unknown multiplication overflow issues, for instance, a class of vulnerability that OpenJPEG has clearly had trouble with in the past. It would be good to see improvement on this front. I don’t think this qualifies as a security vulnerability, but it is certainly a security problem that would facilitate discovering currently-unknown vulnerabilities if fixed.

Still, on the whole, the situation is not anywhere near as bad as I’d thought. Let’s hope OpenJPEG can be included in Ubuntu main sooner rather than later!

By Michael Catanzaro at June 14, 2019 02:43 PM

June 12, 2019

Release Notes for Safari Technology Preview 85

Surfin’ Safari

Safari Technology Preview Release 85 is now available for download for macOS Catalina betas and macOS Mojave. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS.

This release covers WebKit revisions 245618-246093.

Web Authentication

  • Added support for Attestation Conveyance Preference (r245638)

Pointer Events

  • Updated to ensure that capturing data managed by the PointerCaptureController gets cleared upon navigation (r245809, r246031)
  • Changed compatibility mouse events to be able to be prevented while the pointer is pressed (r245699)
  • Exposed navigator.maxTouchPoints (r246070)
  • Changed to only allow pointer capture if the pointer is in the active buttons state (r246074)
  • Changed to not prevent mouseover, mouseout, mouseenter, and mouseleave events while the pointer is down (r245695)
  • Changed toElement and fromElement on PointerEvent to be null (r245867)
  • Changed mouseenter and pointerenter events to be fired from the bottom up (r246061)

Editing

  • Fixed inserting a newline in contenteditable add one character, not two (r245912, r245980)

Media

  • Added an option to mute audio capture automatically when page is not visible (r246002)
  • Allowed for resizing camera video feeds to very small resolutions (r246049)
  • Fixed createAnswer() SDP Rejected by setLocalDescription() (r245829)
  • Fixed playing one video at a time for multiple videos with audio, and the autoplay and playinline attributes (r245712)
  • Changed to hide MediaCapabilities.encodingInfo() when the platform does not support it. (r245636)

CSS

  • Changed precedence for min-width to always win over max-width (r245966)
  • Fixed font-optical-sizing to apply the correct variation value (r245672)
  • Updated to include searching sub-selectors when determining the property whitelist for selector (r245664)
  • Updated to preserve CSS Grid repeat() notation when serializing declared values (r245798)

Web API

  • Fixed bounding client rect for an inline element in Intersection Observer (r245642)
  • Implemented feature policy self, none, and * parsing (r245625)
  • Implemented imagesrcset and imagesizes attributes on link rel=preload (r246045)
  • Implemented Promise.allSettled (r245869)
  • Fixed programmatic scroll in right-to-left overflow with async scrolling enabled (r245771)
  • Added support for preferred presentation size when pasting an image (r245637)
  • Made the computed width of non-replaced inline return computed style (r245768)

JavaScript

  • Implemented support for Numeric Separators (r245655)
  • Implemented opwide16 and opwide32 and introduced 16-bit version bytecode (r245906)
  • Fixed InferredValue to not be a JSCell (r246073)
  • Reduced metadata footprint (r245658)
  • Changed createListFromArrayLike to throw a type error if the value is not an object (r245675)

WebAssembly

  • Added support for anyref in globals (r245765)

Web GPU

  • Updated vertex buffers and Input State API (r245905)

WHLSL

  • Enforced variable lifetimes (r245945)
  • Implemented property resolver (r245680, r245722)
  • Improved the speed of parsing and lexing the standard library (r246052)

Web Inspector

  • Exposed a way to get the current Audit version from within an Audit (r245909)
  • Allowed arbitrary JSON data to be returned as part of the result of an Audit (r245914)
  • Changed CSS modifications to be shared for rules that match multiple elements (r245991)
  • Updated the debugger navigation sidebar to always reveal the active call frame when hitting a breakpoint (r246026)
  • Moved the overlay rulers to the opposite vertical or horizontal side if they intersect the highlighted nodes so that no content is obstructed (r245728)
  • Added a setting to show overlay rulers and guides whenever element selection is enabled (r245730)

June 12, 2019 08:00 PM

June 10, 2019

Javier Fernández: A new terminal-style line breaking with CSS Text

Igalia WebKit

The CSS Text 3 specification defines a module for text manipulation and covers, among a few other features, the line breaking behavior of the browser, including white space handling. I’ve been working lately on some new features and bug fixing for this specification and I’d like to introduce in this posts the last one we made available for the Web Platform users. This is yet another contribution that came out the collaboration between Igalia and Bloomberg, which has been held for several years now and has produced many important new features for the Web, like CSS Grid Layout.

The feature

I guess everybody knows the white-space CSS property, which allows web authors to control two main aspects of the rendering of a text line: collapsing and wrapping. A new value break-spaces has been added to the ones available for this property, which allows web authors to emulate a terminal-like line breaking behavior. This new value operates basically like pre-wrap, but with two key differences:

  • any sequence of preserved white space characters takes up space, even at the end of the line.
  • a preserved white space sequence can be wrapped at any character, moving the rest of the sequence, intact, to the line bellow.

What does this new behavior actually mean ? I’ll try to explain it with a few examples. Lets start with a simple but quite illustrative demo which tries to emulate a meteorology monitoring system which shows relevant changes over time, where the gaps between subsequent changes must be preserved:



 #terminal {
  font: 20px/1 monospace;
  width: 340px;
  height: 5ch;
  background: black;
  color: green;
  overflow: hidden;
  white-space: break-spaces;
  word-break: break-all;
 }


    

Another interesting use case for this feature could be a logging system which should preserve the text formatting of the logged information, considering different window sizes. The following demo tries to describe this such scenario:



body { width: 1300px; }
#logging {
  font: 20px/1 monospace;
  background: black;
  color: green;

  animation: resize 7s infinite alternate;

  white-space: break-spaces;
  word-break: break-all;
}
@keyframes resize {
  0% { width: 25%; }
  100% { width: 100%; }
}

Hash: 5a2a3d23f88174970ed8 Version: webpack 3.12.0 Time: 22209ms Asset Size Chunks Chunk Names pages/widgets/index.51838abe9967a9e0b5ff.js 1.17 kB 10 [emitted] pages/widgets/index img/icomoon.7f1da5a.svg 5.38 kB [emitted] fonts/icomoon.2d429d6.ttf 2.41 kB [emitted] img/fontawesome-webfont.912ec66.svg 444 kB [emitted] [big] fonts/fontawesome-webfont.b06871f.ttf 166 kB [emitted] img/mobile.8891a7c.png 39.6 kB [emitted] img/play_button.6b15900.png 14.8 kB [emitted] img/keyword-back.f95e10a.jpg 43.4 kB [emitted] . . .

Use cases

In the demo shown before there are several cases that I think it’s worth to analyze in detail.

A breaking opportunity exists after any white space character

The main purpose of this feature is to preserve the white space sequences length even when it has to be wrapped into multiple lines. The following example tries to describe this basic use case:



.container {
  font: 20px/1 monospace;
  width: 5ch;
  white-space: break-spaces;
  border: 1px solid;
}

XX XX

The example above shows how the white space sequence with a length of 15 characters is preserved and wrapped along 3 different lines.

Single leading white space

Before the addition of the break-spaces value this scenario was only possible at the beginning of the line. In any other case, the trailing white spaces were either collapsed or hang, hence the next line couldn’t start with a sequence of white spaces. Lets consider the following example:



.container {
  font: 20px/1 monospace;
  width: 3ch;
  white-space: break-spaces;
  border: 1px solid;
}

XX XX

Like when using pre-wrap, the single leading space is preserved. Since break-spaces allows breaking opportunities after any white space character, we break after the first leading white space (” |XX XX”). The second line can be broken after the first preserved white space, creating another leading white space in the next line (” |XX | XX”).

However, lets consider now a case without such first single leading white space.



.container {
  font: 20px/1 monospace;
  width: 3ch;
  white-space: break-spaces;
  border: 1px solid;
}

XXX XX

Again, it s not allowed to break before the first space, but in this case there isn’t any previous breaking opportunity, so the first space after the word XX should overflow (“XXX | XX”); the next white space character will be moved down to the next line as preserved leading space.

Breaking before the first white space

I mentioned before that the spec states clearly that the break-space feature allows breaking opportunities only after white space characters. However, it’d be possible to break the line just before the first white space character after a word if the feature is used in combination with other line breaking CSS properties, like word-break or overflow-wrap (and other properties too).



.container {
  font: 20px/1 monospace;
  width: 4ch;
  white-space: break-spaces;
  overflow-wrap: break-word;
  border: 1px solid;
}

XXXX X

The two white spaces between the words are preserved due to the break-spaces feature, but the first space after the XXXX word would overflow. Hence, the overflow-wrap: break-word feature is applied to prevent the line to overflow and introduce an additional breaking opportunity just before the first space after the word. This behavior causes that the trailing spaces are moved down as a leading white space sequence in the next line.

We would get the same rendering if word-break: break-all is used instead overflow-wrap (or even in combination), but this is actualy an incorrect behavior, which has the corresponding bug reports in WebKit (197277) and Blink (952254) according to the discussion in the CSS WG (see issue #3701).

Consider previous breaking opportunities

In the previous example I described a combination of line breaking features that would allow breaking before the first space after a word. However, this should be avoided if there are previous breaking opportunities. The following example is one of the possible scenarios where this may happen:



.container {
  font: 20px/1 monospace;
  width: 4ch;
  white-space: break-spaces;
  overflow-wrap: break-word;
  border: 1px solid;
}

XX X X

In this case, we could break after the second word (“XX X| X”), since overflow-wrap: break-word would allow us to do that in order to avoid the line to overflow due to the following white space. However, white-space: break-spaces only allows breaking opportunities after a space character, hence, we shouldn’t break before if there are valid previous opportunities, like in this case in the space after the first word (“XX |X X”).

This preference for previous breaking opportunities before breaking the word, honoring the overflow-wrap property, is also part of the behavior defined for the white-space: pre-wrap feature; although in that case, there is no need to deal with the issue of breaking before the first space after a word since trailing space will just hang. The following example uses just the pre-wrap to show how previous opportunities are selected to avoid overflow or breaking a word (unless explicitly requested by word-break property).



.container {
  font: 20px/1 monospace;
  width: 2ch;
  white-space: pre-wrap;
  border: 1px solid;
}

XX
overflow-wrap:
break-word
word-break:
break-all

In this case, break-all enables breaking opportunities that are not available otherwise (we can break a word at any letter), which can be used to prevent the line to overflow; hence, the overflow-wrap property doesn’t take any effect. The existence of previous opportunities is not considered now, since break-all mandates to produce the longer line as possible.

This new white-space: break-spaces feature implies a different behavior when used in combination with break-all. Even though the preference of previous opportunities should be ignored if we use the word-break: break-all, this may not be the case for the breaking before the first space after a word scenario. Lets consider the same example but using now the word-break: break-all feature:



.container {
  font: 20px/1 monospace;
  width: 4ch;
  white-space: break-spaces;
  overflow-wrap: break-word;
  word-break: break-all;
  border: 1px solid;
}

XX X X

The example above shows that using word-break: break-all doesn’t produce any effect. It’s debatable whether the use of break-all should force the selection of the breaking opportunity that produces the longest line, like it happened in the pre-wrap case described before. However, the spec states clearly that break-spaces should only allow breaking opportunities after white space characters. Hence, I considered that breaking before the first space should only happen if there is no other choice.

As a matter of fact, specifying break-all we shouldn’t considering only previous white spaces, to avoid breaking before the first white space after a word; the break-all feature creates additional breaking opportunities, indeed, since it allows to break the word at any character. Since break-all is intended to produce the longest line as possible, this new breaking opportunity should be chosen over any previous white space. See the following test case to get a clearer idea of this scenario:



.container {
  font: 20px/1 monospace;
  width: 4ch;
  white-space: break-spaces;
  overflow-wrap: break-word;
  word-break: break-all;
  border: 1px solid;
}

X XX X

Bear in mind that the expected rendering in the above example may not be obtained if your browser’s version is still affected by the bugs 197277(Safari/WebKit) and 952254(Chrome/Blink). In this case, the word is broken despite the opportunity in the previous white space, and also avoiding breaking after the ‘XX’ word, just before the white space.

There is an exception to the rule of avoiding breaking before the first white space after a word if there are previous opportunities, and it’s precisely the behavior the line-break: anywhere feature would provide. As I said, all these assumptions were not, in my opinion, clearly defined in the current spec, so that’s why I filed an issue for the CSS WG so that we can clarify when it’s allowed to break before the first space.

Current status and support

The intent-to-ship request for Chrome has been approved recently, so I’m confident the feature will be enabled by default in Chrome 76. However, it’s possible to try the feature in older versions by enabling the Experimental Web Platform Features flag. More details in the corresponding Chrome Status entry. I want to highlight that I also implemented the feature for LayoutNG, the new layout engine that Chrome will eventually ship; this achievement is very important to ensure the stability of the feature in future versions of Chrome.

In the case of Safari, the patch with the implementation of the feature landed in the WebKit’s trunk in r244036, but since Apple doesn’t announce publicly when a new release of Safari will happen or which features it’ll ship, it’s hard to guess when the break-spaces feature will be available for the web authors using such browser. Meanwhile, It’s possible to try the feature in the Safari Technology Preview 80.

Finally, while I haven’t see any signal of active development in Firefox, some of the Mozilla developers working on this area of the Gecko engine have shown public support for the feature.

The following table summarizes the support of the break-spaces feature in the 3 main browsers:

Chrome Safari Firefox
Experimental M73 STP 80 Public support
Ship M76 Unknown Unknown

Web Platform Tests

At Igalia we believe that the Web Platform Tests project is a key piece to ensure the compatibility and interoperability of any development on the Web Platform. That’s why a substantial part of my work to implement this relatively small feature was the definition of enough tests to cover the new functionality and basic use cases of the feature.

white-space overflow-wrap word-break
pre-wrap-008
pre-wrap-015
pre-wrap-016
break-spaces-003
break-spaces-004
break-spaces-005
break-spaces-006
break-spaces-007
break-spaces-008
break-spaces-009
break-word-004
break-word-005
break-word-006
break-word-007
break-word-008
break-all-010
break-all-011
break-all-012
break-all-013
break-all-014
break-all-015

Implementation in several web engines

During the implementation of a browser feature, even a small one like this, it’s quite usual to find out bugs and interoperability issues. Even though this may slow down the implementation of the feature, it’s also a source of additional Web Platform tests and it may contribute to the robustness of the feature itself and the related CSS properties and values. That’s why I decided to implement the feature in parallel for WebKit (Safari) and Blink (Chrome) engines, which I think it helped to ensure interoperability and code maturity. This approach also helped to get a deeper understanding of the line breaking logic and its design and implementation in different web engines.

I think it’s worth mentioning some of these code architectural differences, to get a better understanding of the work and challenges this feature required until it reached web author’s browser.

Chrome/Blink engine

Lets start with Chrome/Blink, which was especially challenging due to the fact that Blink is implementing a new layout engine (LayoutNG). The implementation for the legacy layout engine was the first step, since it ensures the feature will arrive earlier, even behind an experimental runtime flag.

The legacy layout relies on the BreakingContext class to implement the line breaking logic for the inline layout operations. It has the main characteristic of handling the white space breaking opportunities by its own, instead of using the TextBreakIterator (based on ICU libraries), as it does for determining breaking opportunities between letters and/or symbols. This design implies too much complexity to do even small changes like this, especially because is very sensible in terms of performance impact. In the following diagram I try to show a simplified view of the classes involved and the interactions implemented by this line breaking logic.

The LayoutNG line breaking logic is based on a new concept of fragments, mainly handled by the NGLineBreaker class. This new design simplifies the line breaking logic considerably and it’s highly optimized and adapted to get the most of the TextBreakIterator classes and the ICU features. I tried to show a simplified view of this new design with the following diagram:

In order to describe the work done to implement the feature for this web engine, I’ll list the main bugs and patches landed during this time: CR#956465, CR#952254, CR#944063,CR#900727, CR#767634, CR#922437

Safari/WebKit engine

Although as time passes this is less probable, WebKit and Blink still share some of the layout logic from the ages prior to the fork. Although Blink engineers have applied important changes to the inline layout logic, both code refactoring and optimizations, there are common design patterns that made relatively easy porting to WebKit the patches that implemented the feature for the Blink’s legacy layout. In WebKit, the line breaking logic is also implemented by the BreakingContext class and it has a similar architecture, as it’s described, in a quite simplified way, in the class diagram above (it uses different class names for the render/layout objects, though) .

However, Safari supports for the mac and iOS platforms a different code path for the line breaking logic, implemented in the SimpleLineLayout class. This class provides a different design for the line breaking logic, and, similar to what Blink implements in LayoutNG, is based on a concept of text fragments. It also relies as much as possible into the TextBreakIterator, instead of implementing complex rules to handle white spaces and breaking opportunities. The following diagrams show this alternate design to implement the line breaking process.

This SimpleLineLayout code path in not supported by other WebKit ports (like WebKitGtk+ or WPE) and it’s not available either when using some CSS Text features or specific fonts. There are other limitations to use this SimpleLineLayout codepath, which may lead to render the text using the BreakingContext class.

Again, this is the list of bugs that were solved to implement the feature for the WebKit engine: WK#197277, WK#196169, WK#196353, WK#195361, WK#177327, WK#197278

Conclusion

I hope that at this point these 2 facts are clear now:

  • The white-space: break-spaces feature is a very simple but powerful feature that provides a new line breaking behavior, based on unix-terminal systems.
  • Although it’s a simple feature, on the paper (spec), it implies a considerable amount of work so that it reaches the browser and it’s available for web authors.

In this post I tried to explain in a simple way the main purpose of this new feature and also some interesting corner cases and combinations with other Line Breaking features. The demos I used shown 2 different use cases of this feature, but there are may more. I’m sure the creativity of web authors will push the feature to the limits; by then, I’ll be happy to answer doubts, about the spec or the implementation for the web engines, and of course fix the bugs that may appear once the feature is more used.

Igalia logo
Bloomberg logo

Igalia and Bloomberg working together to build a better web

Finally, I want to thank Bloomberg for supporting the work to implement this feature. It’s another example of how non-browser vendors can influence the Web Platform and contribute with actual features that will be eventually available for web authors. This is the kind of vision that we need if we want to keep a healthy, open and independent Web Platform.

By jfernandez at June 10, 2019 08:11 PM

June 06, 2019

What’s New in the Payment Request API for Apple Pay

Surfin’ Safari

Since announcing last April that WebKit supports the W3C Payment Request API for Apple Pay, we’ve been hard at work adding even more features to the API. From support for phonetic names in the Apple Pay payment method to new standard features like the paymentmethodchange event and PaymentResponse.retry(), WebKit’s Payment Request implementation is now more capable than ever. Anything you could’ve done in the Apple Pay JS API can now be done directly with the Payment Request API using the Apple Pay payment method.

Let’s take a closer look at some of the improvements we’ve made to Payment Request in the past year.

The paymentmethodchange event

When the user selects a card in the Apple Pay payment sheet, WebKit now dispatches the paymentmethodchange event to your PaymentRequest object. WebKit populates the event’s methodDetails attribute with an ApplePayPaymentMethod dictionary, which you can use to determine information about the card such as its type.

Listening for paymentmethodchange is optional, but if you do, be sure to call the event’s updateWith() method with a promise for updated payment details.

const request = new PaymentRequest(...);
request.onpaymentmethodchange = (event) => {
    // Compute new details based on event.methodDetails
    const detailsUpdatePromise = computeDetails(event.methodDetails);
    event.updateWith(detailsUpdatePromise);
};

Whether or not you listen for the paymentmethodchange event, WebKit continues to support payment details modifiers, a declarative way to vary payment details based on the user’s selected card. See last year’s blog post for more information about modifiers.

Requesting Phonetic Names

The Apple Pay payment method now supports requesting a phonetic spelling of your customer’s name as part of shipping and billing contact information. Here’s an Apple Pay payment method that requests a shipping contact containing a localized name, a phonetic name, and a postal address:

const applePayMethod = {
    supportedMethods: "https://apple.com/apple-pay",
    data: {
        ...,
        requiredShippingContactFields: ["name", "phoneticName", "postalAddress"],
    },
};

When your customer authorizes payment, WebKit provides the phonetic name as part of the ApplePayPayment dictionary that’s stored in the PaymentResponse‘s details attribute.

const request = new PaymentRequest([applePayMethod], ...);
request.show().then((response) => {
    const shippingContact = response.details.shippingContact;
    const phoneticFamilyName = shippingContact.phoneticFamilyName;
    const phoneticGivenName = shippingContact.phoneticGivenName;
});

Reporting Errors

When handling contact information, you might encounter an error that needs to be resolved by the customer before they can proceed with the transaction. Common examples include invalid addresses, unserviceable addresses, and missing contact information.

So that you can report these errors to your customers, WebKit now supports Payment Request’s fine-grained error reporting capabilities in the PaymentDetailsUpdate dictionary. You can specify shippingAddressErrors for errors with shipping address attributes, payerErrors for errors with names, email addresses, and phone numbers, and paymentMethodErrors for errors specific to the selected payment method. The Apple Pay payment method treats paymentMethodErrors as a sequence of ApplePayError objects.

Here’s how you might report an error for an invalid postal code:

const request = new PaymentRequest([applePayMethod], details, { requestShipping: true });
request.onshippingaddresschange = (event) => {
    // Compute new details based on shippingAddress
    const detailsUpdate = computeDetails(request.shippingAddress);

    // Check for an invalid postal code
    if (!isValid(request.shippingAddress.postalCode))
        detailsUpdate.shippingAddressErrors = { postalCode: "Invalid postal code" };

    event.updateWith(detailsUpdate);
};

When you report errors, WebKit displays them in the Apple Pay payment sheet and prompts your customer to make corrections — perhaps by editing their shipping address or selecting from a list of stored address. Here’s what the shipping address error we specified above looks like on an iPhone.

When your customer taps on the erroneous shipping address, the Apple Pay sheet displays your custom error message. If they decide to edit their address, the fields you flagged with errors (postalCode, in this case) are highlighted to help guide them through making corrections.

Once the user finishes making corrections, WebKit fires the shippingaddresschange event again. If you are satisfied with the corrections, you can proceed with the transaction by calling the updateWith() method again without specifying errors.

When handling errors in shippingaddresschange, keep in mind that WebKit redacts some parts of the shipping contact before payment authorization. The rules vary by country, but in general, WebKit reveals only the contact information necessary to calculate tax and shipping costs. Once the customer authorizes payment, WebKit reveals all the requested contact information.

Retrying Authorizations

While many errors can be detected when handling events like shippingaddresschange, the redaction rules mentioned above mean that some errors can only be detected once the user has authorized payment. For instance, you might require an email address to send a purchase confirmation, only to find after authorization that the customer did not provide a valid one.

To handle these cases, WebKit now implements PaymentResponse‘s retry() method, allowing you to ask the user to retry the payment after correcting for errors.

When you detect an error in a PaymentResponse, you can call retry() with a PaymentValidationErrors dictionary. Like in PaymentDetailsUpdate, you can specify shippingAddress errors, payer errors, and paymentMethod errors. When you call retry(), it returns a promise that resolves once the PaymentResponse is re-authorized.

Here’s how you might handle an invalid email address:

const handleResponse = (response) => {
    if (!isValid(response.payerEmail)) {
        response.retry({
            payer: { email: "Invalid email address" },
        }).then(() => { handleResponse(response) });
        return;
    }
    response.complete("success");
};

const request = new PaymentRequest([applePayMethod], ...);
request.show().then(response => handleResponse(response));

Keep in mind that retry() is for errors that can be corrected by the user. If you detect a fatal error — for example, the card was declined or an item is no longer in stock — call PaymentResponse‘s complete() method with a PaymentComplete value of 'fail'. The Apple Pay payment sheet will display an appropriate error to the user and then dismiss itself.

Availability

These new Payment Request features are available starting in Safari 12.1 on macOS, Safari on iOS 12.2, and Safari Technology Preview.

Feedback

We’ve received some great developer feedback so far, and we look forward to more of your bug reports and feature requests. If you find a Payment Request bug in WebKit, please report it at bugs.webkit.org. On Twitter, you can reach the WebKit team at @webkit, or our web technologies evangelist Jonathan Davis at @jonathandavis.

June 06, 2019 05:25 PM

CPU Timeline in Web Inspector

Surfin’ Safari

Web Inspector now includes a new CPU usage timeline that lets developers measure a page’s CPU usage, estimate its energy impact, and more easily investigate sources of script execution that may be contributing to poor energy utilization.

Energy Efficiency

Web applications for productivity and entertainment are commonplace. For many users, popular websites are often left open in the foreground of their browser or inactive in a background tab for extended periods of time. The activity on all of these pages may contribute to power drain. Many of the devices used to browse the web are battery powered. To all of those users, battery life matters!

Ensuring web applications make the best use of limited resources is a difficult problem. Having a powerful set of tools makes it easier for developers to investigate and identify performance issues. Many different variables affect energy use on web pages, but the most prominent is CPU activity. Monitoring CPU usage can provide an overall estimate of the energy efficiency of a web page and can highlight cause for concern if activity is too high.

CPU Timeline

The new CPU usage timeline is enabled by default. Open the Timelines tab of Web Inspector and you will see a new CPU timeline in the overview at the top. Select the timeline to show its detail view, which including a breakdown of the main thread activity, an energy impact rating, and more.

New CPU Usage Timeline New CPU Usage Timeline New CPU Usage Timeline New CPU Usage Timeline

The CPU timeline overview shows CPU usage samples taken every 500ms. You can use these samples to visualize activity spikes, and how they correlate with networking, layout and rendering, and JavaScript. To see a detailed overview of CPU usage in a specific section of a recording, click and drag to make a time range selection.

Main Thread Breakdown

The first section of the detail view is a breakdown of the work being performed on the main thread. While the total CPU usage across all threads is important, the work performed on the main thread is particularly important for web content.

CPU Timeline Main Thread Breakdown

Principal tasks such as evaluating script, painting, layout, and style resolution are performed on the main thread. The chart allows you to quickly see a comparison of where time is spent in each of these categories.

The main thread is primarily active in order to service user interaction events such as scrolling and clicking or tapping on a page. This means that the main thread may have periods of inactivity. The number inside the chart shows the approximate number of milliseconds the main thread was active. This number may be much smaller than the total selected time range in the overview.

Different interactions produce different main thread workloads. For example, scrolling the page will increase the time spent painting, resizing the window will cause additional layouts, and typing into a form may execute scripts.

Energy Impact

The Energy Impact section provides an estimate of the battery drain caused by the page. This is based off of the average CPU usage for the selected time range.

Extended periods of high CPU utilization will show “High” energy impact, since sustained high CPU usage will drain power and have a noticeable effect on battery life. When making use of this section there are two important scenarios to consider and measure:

  • Idle recordings – An idle page can be one sitting in the foreground not being interacted with by the user, or a background tab that is still alive. It is important that idle pages use as little energy and CPU as possible. Recordings of idle pages should strive for a “Low” rating (less than 3% average CPU usage).
     
  • Interactive recordings – Recordings that include interactivity such as navigating, scrolling, clicking, typing, etc. should expect higher CPU usage because the page is responding to the user. Periods of interactivity should strive to maintain a “Medium” rating (less than 30% average CPU usage) and avoid “High” if possible.

  • While WebKit does work to throttle timers and limit the impact of background tabs or obscured content, idle pages may still perform costly work. Likewise foreground pages that are constantly performing script, such as timers or excessive event handling, may drain battery without any obvious indication.

    It is important to select a sufficiently large time range so that a single spike in CPU activity doesn’t dominate the average. Page load for example will briefly use very high CPU activity as the initial cost to download and render the page is high. A short time range selection during page load, or during a period of user interaction with the page, may show a high Energy Impact but may not drain the battery if the usage is not sustained. Normal browsing scenarios have idle periods between loads while users read or more slowly interact with page content. A selection range of 15 seconds or more will produce a more realistic estimate of battery drain than a short selection.

    Per-Thread Details

    The center of the detail view includes a larger CPU usage graph broken down by thread. The legend shows the different colors for the main thread, worker threads, and other threads. Hover the graph and you get the exact values for the nearest sample. Below the graph is the Main Thread indicator strip. This shows each of the individual samples of main thread activity. Clicking on this strip will take you directly to the event happening at that time in the event’s associated timeline.

    For an even more detailed breakdown by individual threads and thread groups you can expand the lower Threads section. While the main thread and worker threads will be individually graphed, the remaining threads are grouped. WebKit threads include any threads performing work known to WebKit such as Garbage Collection, JIT compilation, bmalloc scavanger, and more. CPU usage on any remaining threads are put into an unclassified group.

    Statistics and Sources

    These sections provide more detailed insight into the activity that occurred on the page within the selected time range.

    The Statistics section enumerates different kinds of activities that occured on the page and their individual frequencies. Activity includes network requests and entry into script via timers, events, or observer callbacks. The values are sorted by frequency so you can immediately see the most commonly occurring activities which may warrant investigation.

    The Sources section pinpoints and aggregates the sources of entries into script. This includes timer installations, event handlers, and observer handlers. Values are again sorted by frequency so you can immediately see the hottest entry points. For timers, we include the function name where the timer was installed. This makes it easy to detect and investigate unexpected timer registrations.

    The two sections work together to make a powerful debugging tool. Categories of script entries can be selected from the Statistics section and the Sources section will filter to show the corresponding script entry points. For example, by selecting requestAnimationFrame the Sources list will immediately filter down to show only the animation timer installations. With one more click you can then jump to the associated code and place breakpoints for further debugging.

    Timeline Import & Export

    Web Inspector now supports importing and exporting timeline recordings. Now when you capture a recording of a performance issue you can export the recording to share with others, attach to a bug report, or just save for later analysis.

    Import and export works across all of the timelines. This makes it useful for all kinds of performance analysis, not just the new CPU timeline. If you captured a recording with an interesting JavaScript and Events profile or one with JavaScript Allocations heap snapshots showing a memory leak, export and import will work great in each of these cases.

    Feedback

    You can try out the new CPU Usage Timeline in the latest Safari Technology Preview. Let us know how it works for you. Stay tuned for upcoming blog posts on best practices for energy efficient web content. Send feedback on Twitter (@webkit, @JosephPecoraro) or by filing a bug.

    June 06, 2019 02:00 PM

    June 05, 2019

    Safari Technology Preview 84, with Safari 13 Features, is Now Available

    Surfin’ Safari

    Safari Technology Preview Release 84 is now available for download for macOS Mojave. With this release, Safari Technology Preview is now available for betas of macOS Catalina. If you already have Safari Technology Preview installed, you can update from the Software Update pane of System Preferences.

    This release covers the same revisions of WebKit from Safari Technology Preview 83, but includes new Safari and WebKit features that will be present in Safari 13. The following Safari 13 features are new to Safari Technology Preview 84:

    Refreshed Favorites Design. The Favorites page has been visually refreshed, and now includes Show More and Show Less actions.

    Switch to Tab from Smart Search Field. The Smart Search Field now offers switching to an already-open tab when a search query matches the title or URL of an open tab.

    Warnings for Weak Passwords. When signing into a website with a weak password, Safari will prompt you to visit the website in a new tab to upgrade the password to an Automatic Strong Password. Safari uses the well-known URL for changing passwords (/.well-known/change-password), allowing websites to take users directly to their change password pages. The password list in Safari Preferences has also been updated to flag weak passwords.

    Many more WebKit features in Safari 13 are present in this release of Safari Technology Preview and have been in past releases. You can read more about these changes in What’s New in Safari 13 Beta.

    June 05, 2019 05:00 PM

    Creating Web Inspector Audits

    Surfin’ Safari

    This post is a followup to the Audits in Web Inspector post, and explains the capabilities of and how to write an audit.

    Test Case Format

    The actual test that is run in the page is just a stringified JavaScript function. async functions are allowed, as well as non-async functions that return a Promise. The only requirement is that the return value of the function (or Promise) conforms to the following rules:

    • Returning true/false will translate to a Pass/Fail result with no additional data.
    • Returning a string value will translate to the corresponding result (e.g. "pass" is a Pass result) with no additional data.
    • Returning an object gives the most flexibility, as it allows for additional data other than the audit’s result to be displayed in the Audit tab. Result levels are first retrieved from the "level" value (if it exists), and are translated in the same way as if a string was returned (e.g. "pass" is a Pass result). Alternatively, using any result string as a key with a value of true will have the same effect (e.g. "pass": true).

    There are five result levels:

    • "pass" corresponds to a Pass result, which is where everything was as it should be.
    • "warning" corresponds to a Warning result, which is a “soft pass” in that there was nothing wrong, but there were things that should be changed.
    • "fail" corresponds to a Fail result, which is an indication that something is not as it should be.
    • "error" corresponds to an Error result, which occurs when the JavaScript being run threw an error.
    • "unsupported" corresponds to an Unsupported result, which is a special case that can be used to indicate when the data being tested isn’t supported by the current page (e.g. missing some API).

    There are also three additional pieces of data that can be returned within the result object and have a dedicated specialized interface:

    • domNodes, which is an array of DOM nodes that will be displayed in the Audit tab much the same as if they were logged to the console.
    • domAttributes, which is an array of strings, each of which will be highlighted if present on any DOM nodes within domNodes.
    • errors, which is an array of Error objects and can be used as a way of exposing errors encountered while running the audit.
      • If this array has any values, the result of the audit is automatically changed to Error.
      • If an error is thrown while running an audit, it will automatically be added to this list.

    For custom data, you can add it to the result object and it will display in the Audit tab, so long as it’s JSON serializable and doesn’t overlap with any of the items above.

    Container Structure

    Web Inspector Audits follow a simple and highly flexible structure in the form of JSON objects. These objects fall into two main categories: tests and groups.

    The format for a test is as follows:

    {
        "type": "test-case",
        "name": "...",
        "test": "<stringified JavaScript function>"
    }
    

    The format for a group is as follows:

    {
        "type": "test-group",
        "name": "...",
        "tests": [...]
    }
    

    In this case, the values inside tests can be both individual test cases, or additional groups.

    Both tests and groups also support a number of optional properties:

    • description is a basic string value that is displayed in the Audit tab as a way of providing more information about that specific audit, such as what it does or what it’s trying to test.
    • supports can be used as an alternative to feature-checking, in that it prevents the audit from even being run unless the number value matches Web Inspector’s Audit version number, which can be found at the bottom of the Web Inspector window when in edit mode in the Audit tab. At the time of writing this post, the current version is 3.
    • setup is similar to a test case’s test value, except that it only has an effect when supplied for a top-level audit. The idea behind this value is to be able to share code between all audits in a group, as it is executed before the first audit in a group.

    Specially Exposed Data

    Since audits are run from Web Inspector, it’s possible for additional information to be exposed to each test function being executed. Much of this data is already exposed to Web Inspector, but was never accessible via JavaScript in any way.

    The information is exposed via a WebInspectorAudit object that is passed to each test function. Note that this object is shared between all test under a given top-level audit, which is defined as an audit with no parent. As such, attaching data to this object to be shared between test is accepted and encouraged.

    Version

    Accessing Web Inspector’s Audit version number from within a test is as simple as getting the value of WebInspectorAudit.Version.

    Resources

    The following all relate to dealing with resources loaded by the page, and are held by WebInspectorAudit.Resources.

    • getResources() will return a list of objects, each corresponding to a specific resource loaded by the inspected page, identified by a string url, string mimeType, and audit-specific id.
    • getResourceContent(id) will return an object with the string data and boolean base64Encoded contents of the resource with the given audit-specific id.

    DOM

    The following all relate to dealing with the DOM tree, and are held by WebInspectorAudit.Resources.

    • hasEventListeners(node[, type]) returns true/false depending on whether the given DOM node has any event listeners, or has an event listener for the given type (if specified).

    Accessibility

    The following all relate to dealing with the accessibility tree, and are held by WebInspectorAudit.Accessibility. Further information can be found in the WAI-ARIA specification.

    • getElementsByComputedRole(role[, container]) returns an array of DOM nodes that match the given role that are children of the container DOM node (if specified) or the main document.
    • getActiveDescendant(node) returns the active descendant of the given DOM node.
    • getMouseEventNode(node) returns the DOM node that would handle mouse events which is or is a child of the given DOM node.
    • getParentNode(node) returns the parent DOM node of the given DOM node in the accessibility tree.
    • getChildNodes(node) returns an array of DOM nodes that are children of the given DOM node in the accessibility tree.
    • getSelectedChildNodes(node) returns an array of currently selected DOM nodes that are children of the given DOM node in the accessibility tree.
    • getControlledNodes(node) returns an array of DOM nodes that are controlled by the given DOM node.
    • getFlowedNodes(node) returns an array of DOM nodes that are flowed to from the given DOM node.
    • getOwnedNodes(node) returns an array of DOM nodes that are owned by the given DOM node.
    • getComputedProperties(node) returns an object that contains various accessibility properties for the given DOM node. Since HTML allows for “incorrect” values in markup (e.g. an invalid value for an attribute), the following properties use the computed value determined by WebKit:
      • busy is a boolean related to the aria-busy attribute.
      • checked is a string related to the aria-checked attribute.
      • currentState is a string related to the aria-current attribute.
      • disabled is a boolean related to the aria-disabled attribute.
      • expanded is a boolean related to the aria-expanded attribute.
      • focused is a boolean that indicates whether the given node is focused.
      • headingLevel is a number related to the aria-level attribute and the various HTML heading elements.
      • hidden is a boolean related to the aria-hidden attribute.
      • hierarchicalLevel is a number related to the aria-level attribute.
      • ignored is a boolean that indicates whether the given node is currently being ignored in the accessibility tree.
      • ignoredByDefault is a boolean that indicates whether the given node is always ignored by the accessibility tree.
      • invaludStatus is a string related to the aria-invalid attribute.
      • isPopUpButton is a boolean related to the aria-haspopup attribute.
      • label is a string related to the aria-label attribute
      • liveRegionAtomic is a boolean related to the aria-atomic attribute.
      • liveRegionRelevant is an array of strings related to the aria-relevant attribute.
      • liveRegionStatus is a string related to the aria-live attribute.
      • pressed is a boolean related to the aria-pressed attribute.
      • readonly is a boolean related to the aria-readonly attribute.
      • required is a boolean related to the aria-required attribute.
      • role is a string related to the role attribute.
      • selected is a boolean related to the aria-selected attribute.

    Getting Started

    As mentioned in the Audits in Web Inspector post, the Demo Audit and Accessibility audits that are included as part of Web Inspector can be used as good starter templates for the formatting and structure of a Web Inspector audit.

    Alternatively, the eslint.json audit, which runs ESLint against every *.js resource that was loaded from the main origin of the inspected page, can serve as a more complex example of an async audit that makes use of some of the above specially exposed data.

    Feedback

    The Audit tab was added in Safari Technology Preview 75. Is there a workflow that isn’t supported, or a piece of data that isn’t exposed, that you’d like to use when writing/running audits? Do you have an idea for a default audit that should be included in Web Inspector? Let us know! Please feel free to send us feedback on Twitter (@dcrousso or @jonathandavis) or by filing a bug.

    June 05, 2019 02:41 PM

    Audits in Web Inspector

    Surfin’ Safari

    Often when creating web pages, there are particular rules or guidelines that are set by the site author, or by an included library/framework. In those cases, it can be tedious to find any cases where those rules/guidelines may be broken, assuming one even has the inclination to do so. As a simple example, remembering to set a title or alt on <img> is often forgotten, but it’s important to include them as it makes it easier for accessibility tools to understand and/or navigate the page.

    For those repeated cases, where one wants to check if the page conforms to an expectation, we’ve created a new tool, which we call Web Inspector Audits, that can help simplify that process.

    Audit Tab

    Web Inspector Audits can be found in the new Audit tab, represented as a tree of audits, each of which can be run against the inspected page. An audit can be either a group of other audits (e.g. a test group), or an individual test case.

    Each audit can have one of five different result levels, each denoted by a different icon.

    • Passed represents a result where everything was as it should be.
    • Warning is a “soft pass” in that there was nothing wrong, but there were things that should be changed.
    • Failed is an indication that something is not as it should be.
    • An Error result occurs when the JavaScript being run threw an error.
    • Unsupported is a special case that can be used to indicate when the data being tested isn’t supported by the current page (e.g. missing some API).

    Audits are also able to be disabled, to allow for more control as to what is actually run. This is done by clicking the Edit button at the bottom of the navigation sidebar.

    Default Audits

    Web Inspector itself also provides a few audits that are included by default and cannot be deleted (only disabled).

    Demo Audit is a basic exposé of the functionality/interface of audits, and can be used as a starter template for writing your own audits.

    Accessibility includes a handful of test cases that check for DOM accessibility best practices on the inspected page, such as the example mentioned at the beginning of this post, in accordance with the WAI-ARIA specification. The Accessibility audit serves as a great starting point for making sure that your page is accessible.

    Import and Export

    When we first began thinking about the concept of audits, one thing that was forefront in our minds was that both the audit and its result should be portable. This was the primary reason that we chose to use JSON as the format for audits, as well as their results.

    All audits are able to be imported and exported from any Web Inspector, regardless of whether that particular audit is able to be run on the current page (see the Unsupported explanation above). Any imported audits are persistent across Web Inspector sessions, meaning you can import an audit while inspecting one page, open Web Inspector on another page, and be able to run that same audit on the new page without having to re-import. This makes it easy for you to share audits you’ve used (or even written) with other developers out in the world.

    Audit results are similarly importable/exportable, but they are not preserved across sessions. This way, if you run an audit and encounter any non-Passed result, you can export it and share the data with those around you, possibly helping you find the right way to act on it.

    Demo

    The simplest way to see how Web Inspector Audits work is to simply run them on any page.

    As a demonstration of a more advanced usage, eslint.json is an audit that runs ESLint against every *.js resource that was loaded from the main origin of the inspected page. If you’re interested in editing this audit, or even writing your own, please see the upcoming post about creating Web Inspector Audits.

    Feedback

    The Audit tab was added in Safari Technology Preview 75. Is there a workflow that isn’t supported, or a piece of data that isn’t exposed, that you’d like to use when writing/running audits? Do you have an idea for a default audit that should be included in Web Inspector? Let us know! Please feel free to send us feedback on Twitter (@dcrousso or @jonathandavis) or by filing a bug.

    June 05, 2019 02:40 PM

    May 29, 2019

    Release Notes for Safari Technology Preview 83

    Surfin’ Safari

    Safari Technology Preview Release 83 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 245007-245618.

    Web Authentication

    • Enabled WebAuthN by default on macOS (r245589)
    • Changed to cancel the pending request when a new request is made (r245043)
    • Changed to return InvalidStateError to sites whenever authenticators return such error (r245262)

    Pointer Events

    • Fixed isPrimary property of pointercancel events to match previous events for that pointer (r245020)
    • Fixed calling preventDefault() on pointerdown to prevent “compatibility” mouse events (r245585)

    Storage Access API

    • Changed to only consume the user gesture when the user explicitly denies access and made document.hasStorageAccess() return true when the feature is off (r245025)

    Rendering

    • Implemented backing-sharing in compositing layers, allowing overlap layers to paint into the backing store of another layer (r245170)
    • Changed layers painting into shared backing to contribute to overlap (r245502)
    • Changed to repaint when the set of backing-sharing layers changes (r245220)
    • Fixed rendering of backing-sharing layers with transforms (r245205)
    • Fixed layer bounds for sharing layers that paint with transforms (r245208)
    • Fixed layer-related flashing with composited overflow: scroll (r245602)
    • Fixed overflow: scroll that becomes non-scrollable to stop being composited (r245212)
    • Fixed content disappearing when the scroller hosting a shared layer becomes non-scrollable (r245206)
    • Fixed rendering issues when layer contents become non-opaque (r245207)

    Service Workers

    • Changed to terminate a service worker instance when its SWServer is destroyed (r245200)
    • Changed a service worker process to app nap when all of its clients app nap (r245299)

    CSS

    • Implemented line-break: anywhere (r245275)
    • Implemented a modern “clearfix” with display: flow-root (r245494)
    • Implemented page-break-* and -webkit-column-break-* as legacy-shorthands. (r245276)
    • Fixed font-optical-sizing applying the wrong variation value (r245598)
    • Updated CSS grid when changing auto repeat type (r245295)
    • Updated to use max size to compute auto repeat CSS grid tracks (r245279)

    WebRTC

    • Defined a media buffering policy (r245039)
    • Allowed sequential playback of media files when initial playback started with a user gesture (r245467)
    • Fixed video stream freezing when the front camera orientation changes (r245033)

    WebDriver

    • Fixed the in-view center point for elements larger than the viewport (r245320)

    Web API

    • Changed to preserve DOM selection after clicking a button that hides itself on mousedown (r245238)
    • Limited the number of prefetches of a given page (r245171)

    Web Inspector

    • Changed the user gesture toggle to also force a user interaction flag (r245366)
    • Fixed the colors in the network table waterfall container for Dark Mode (r245484)
    • Fixed context menu items in the DOM tree when not clicking directly on the node representation in the Elements tab (r245495)
    • Fixed Storage tab crashes when adding new local storage or session storage entries (r245535)
    • Fixed the CPU timeline and Memory timeline bars sometimes incorrectly drawing and jumping around while scrolling (r245498)
    • Made it easier to switch to a DOM node in the Elements tab from a returned DOM node in the Audit tab (r245497)

    May 29, 2019 05:00 PM

    May 22, 2019

    Privacy Preserving Ad Click Attribution For the Web

    Surfin’ Safari

    A typical website is made of numerous components coming from a wide variety of sources. Many of the sources that make up a website are opaque to the user, and some third-party resources are designed to identify and track users as they browse the web, often in order to retarget ads and measure ad campaign effectiveness.

    The combination of third-party web tracking and ad campaign measurement has led many to conflate web privacy with a web free of advertisements. We think that’s a misunderstanding. Online ads and measurement of their effectiveness do not require Site A, where you clicked an ad, to learn that you purchased something on Site B. The only data needed for measurement is that someone who clicked an ad on Site A made a purchase on Site B.

    Today we are presenting a new technology to allow attribution of ad clicks on the web while preserving user privacy. We used the following principles as we designed this technology:

    • Users should not be uniquely identified across websites for the purposes of ad click attribution. This means the combined data of an ad click and a conversion should not be attributable to a single user at web scale. To achieve this, our design has the following properties:
      • Up to 64 ad campaigns can be measured in parallel per website where ads are placed by an advertiser. This low number means ad campaign IDs cannot be turned into user identifiers.
      • Up to 64 conversion events can be distinguished on the advertiser’s own website. This means conversion IDs are also restricted from being turned into user identifiers.
    • Only websites that users visit should be involved in measuring ad clicks and conversions. This means that opaque third-parties should not receive ad click attribution reports and we enforce it by requiring that the ad link is part of a first-party webpage and by only reporting on which first-party website a conversion happened.
    • The browser should act on behalf of the user and do its best to preserve privacy while reporting on ad click attribution. We achieve this by:
      • Sending attribution reports in a dedicated Private Browsing Mode even though the user is in regular browsing mode.
      • Disallowing data like cookies for reporting purposes.
      • Delaying reports randomly between 24 and 48 hours.
      • Not supporting Privacy Preserving Ad Click Attribution at all when the user is in Private Browsing Mode.
    • The browser vendor should not learn about the user’s ad clicks or conversions. For this reason, we designed the feature to do all of its work on-device. The browser vendor does not see any of the ad click attribution data.

    Critically, our solution avoids placing trust in any of the parties involved — the ad network, the merchant, or any other intermediaries — and dramatically limits the entropy of data passed between them to prevent communication of a tracking identifier.

    Ad Click Attribution in a Nutshell

    Here’s a simple example of ad click attribution:
    An online store runs an ad on a search engine website. If a user clicks the ad and eventually buys something, both the online store and the search engine website where the ad was placed want to know; they want the purchase to be attributed to the ad click so that the store knows where to focus their advertising budget. Such attribution is used for measurement of which ads are effective.

    Traditional, Privacy-Invasive Ad Click Attribution

    Traditionally, ad click attribution has been done through the use of cookies and so-called “tracking pixels.” Here’s an illustration of how this works:

    Privacy invading ad click attribution

    The illustration above shows the user John:

    1. Searching for “grill” on search.example,
    2. Clicking an ad which takes him to shop.example, and
    3. Finally adding a $90 grill to a shopping cart.

    Following each action on shop.example, shop.example fires a tracking pixel (a request for an invisible image) to search.example to report progress toward a purchase.

    In browsers without appropriate privacy protections, search.example will identify John through his cookies every time shop.example fires such a tracking pixel to search.example. This pervasive technology allows search.example to learn everything John does on shop.example and all other websites that fire similar tracking pixels. Even worse, all these pixels fire regardless of whether John has clicked an ad or not.

    Needless to say, tracking pixels that carry cookies enable sites such as search.example to build up a huge profile of people’s interests, purchasing power, habits, age, et cetera. We refer to this as cross-site tracking and Safari prevents it from happening through the WebKit feature Intelligent Tracking Prevention (ITP).

    As more and more browsers acknowledge the problems of cross-site tracking, we should expect privacy-invasive ad click attribution to become a thing of the past.

    Privacy Preserving Ad Click Attribution

    We propose a modern way of doing ad click attribution that doesn’t allow for cross-site tracking of users but does provide a means of measuring the effectiveness of online ads. It is built into the browser itself and runs on-device which means that the browser vendor does not get to see what ads are clicked or which purchases are made.

    Privacy Preserving Ad Click Attribution has three steps:

    1. Store ad clicks. This is done by the page hosting the ad at the time of an ad click.
    2. Match conversions against stored ad clicks. This is done on the website the ad navigated to as a result of the click. Conversions do not have to happen right after a click and do not have to happen on the specific landing page, just the same website.
    3. Send out ad click attribution data. This is done by the browser after a conversion matches an ad click.

    Let’s go through these steps in detail and which steps we’ve taken to preserve the user’s privacy.

    Step 1: Store Ad Clicks

    Anchor elements, often referred to as links, now support two new, optional attributes called adDestination and adCampaignID.

    As shown in the illustration below, adDestination is the domain the ad click is navigating the user to, and adCampaignID is the identifier of the ad campaign.

    Store ad clicks

    If the user clicks the ad link on search.example, the browser will follow the navigation, through potential redirects, to make sure that the user actually lands on shop.example. If so, the browser stores the ad click, comprising the following data (presented here in plain English): The user clicked shop.example’s ad campaign 55 on search.example.

    Here are the important privacy aspects of this step:

    • The link needs to be an element on the first-party website (the main frame), not a link in an iframe. This is to meet user expectations and to be able to provide control to the user. Users can only be expected to understand which first-party website they clicked an ad on and which first-party website they made a purchase on. We also think it’s important that the first-party website that serves the ad is the one attributed for the performance of the ad campaign.
    • Neither search.example nor shop.example can read the stored ad click data or detect that it exists.
    • The browser only stores ad clicks for a limited time. In WebKit’s implementation that is seven days.
    • The entropy of the ad campaign ID needs to be properly restricted to not become a cross-site tracking vector. WebKit’s implementation allows a value between 0 and 63, i.e. a maximum of 64 shop.example ad campaigns running in parallel on search.example.

    Step 2: Match Conversions Against Stored Ad Clicks

    To achieve ad click attribution, the browser needs to be able to match conversions with stored ad clicks. What are conversions?

    • Adding an item to the shopping cart is a conversion.
    • Signing up for a new service is a conversion.
    • Entering shipping or payment information is a conversion.
    • Pulling the trigger and actually buying something is a conversion.

    Matching conversions to ad clicks allows shop.example to understand that a specific ad campaign may be effective in getting customers to add items to their shopping carts but something in the checkout flow throws them off.

    How does Privacy Preserving Ad Click Attribution detect a conversion and match it with a stored ad click? It makes use of the legacy tracking pixels!

    Match conversions ad clicks

    In the illustration above, an existing request to the existing tracking pixel is redirected by search.example on its own server infrastructure to a well-known location in order to signal to the browser that this is in fact a conversion happening. Note that privacy protections such as ITP will typically make sure that no cookies are sent in this request.

    The path parameter “20” at the end of the well-known location is the conversion data. This gives shop.example an opportunity to say something about the conversion such as where in the sales funnel the customer is, what the value of the conversion is, what time of day it is, or whatever they decide is relevant for them.

    The redirect to the well-known location may also include an optional priority parameter which indicates the importance of this particular conversion in the case of multiple conversions matching the same stored ad click.

    Here are the important privacy aspects of this step:

    • Neither search.example nor shop.example know whether there is any stored ad click data to be matched against.
    • Neither search.example nor shop.example are told by the browser whether there was a match or not.
    • The entropy of the ad conversion data needs to be properly restricted to not become a cross-site tracking vector. WebKit’s implementation allows a value between 0 and 63, i.e. six bits to distinguish conversion events. As mentioned earlier, shop.example decides what goes into these bits. For instance, they may spend two bits on monetary value in four buckets: {less than $10, between $10 and $50, between $51 and $200, above $200}.

    We expect to also implement a JavaScript API to send this information to the .well-known location to remove the requirement for tracking pixels but we’d like to openly discuss what should go into that API since it is much more forward looking than retrofitting existing tracking pixels.

    Step 3: Send Out Ad Click Attribution Data

    Now we come to the third and final step — the browser reports that a conversion happened for a user that had previously clicked an ad.

    Send Ad Click Attribution Data

    Once the browser has matched a conversion against a stored ad click, it sets a timer, randomized between 24 and 48 hours. When that timer fires, the browser makes an ephemeral, stateless POST request to the same well-known location. In our example, the request would go to https://search.example/.well-known/ad-click-attribution/20/55, with the referrer request header set to https://shop.example.

    In plain English this report would say: 24 to 48 hours ago, some user who previously clicked shop.example’s ad campaign 55 on search.example, converted with data 20 on shop.example.

    Once the ephemeral, stateless POST request goes out, the stored ad click is consumed and cannot be converted further. This is in part why we have the minimum delay of 24 hours. During that delay, shop.example has the opportunity to signal further conversions, for instance down a sales funnel, and only the most important conversion will be sent in the POST request. The importance is controlled through the optional priority parameter in the conversion redirect, as mentioned above.

    Here are the important privacy aspects of this step:

    • Neither search.example nor shop.example know that an attribution request has been scheduled.
    • The 24–48 hour delay makes sure a conversion that happens shortly after an ad click will not allow for speculative profiling of the user by search.example. The randomness in the delay makes sure that the request does not in itself reveal when during the day the conversion happened. If shop.example wants time of day data, they will have to spend some of their six bits of conversion data on it.
    • The ephemeral, stateless request makes sure the request is not associated with state built up through other browsing. Ephemeral in this sense is referred to as Private Browsing in Safari.
    • The well-known location allows for a simple rule if Content Blockers wants to block such conversion reporting.

    Privacy Considerations

    For ad click attribution to happen, some bits of data about what happened across two websites need to be sent. Today’s practice of ad click attribution has no practical limit on the bits of data, which allows for full cross-site tracking of users using cookies. This is privacy invasive and thus we are obliged to prevent such ad click attribution from happening in Safari and WebKit.

    But by keeping the entropy of attribution data low enough, we believe the reporting can be done in a privacy preserving way.

    Here is a summary of our privacy considerations for Privacy Preserving Ad Click Attribution:

    • Only links served on first-party pages should be able to store ad click attribution data. This ensures that users have a chance of understanding how Privacy Preserving Ad Click Attribution works.
    • Neither the website where the ad click happens nor the website where the conversion happens should be able to see whether ad click data has been stored, has been matched, or is scheduled for reporting.
    • Ad clicks should only be stored for a limited time, such as a week. Users cannot be expected to understand that a purchase they make today is attributed to an ad click they made months ago.
    • The entropy of both ad campaign ID and conversion data needs to be restricted to a point where this data cannot be repurposed for cross-site tracking of users. We propose six bits each for these two pieces of data, or values between 0 and 63.
    • Ad click attribution requests should be delayed randomly between 24 to 48 hours. This makes sure that a conversion that happens shortly after an ad click will not allow for speculative cross-site profiling of the user. The randomness in the delay makes sure the request does not in itself reveal when during the day the conversion happened.
    • The browser should not guarantee any specific order in which multiple ad click attribution requests are sent, since the order itself could be abused to increase the entropy and allow for cross-site tracking of users.
    • The browser should use an ephemeral session (a.k.a. private or incognito mode) to make ad click attribution requests.
    • The browser should not use or accept any credentials such as cookies, client certificates, or Basic Authentication in ad click attribution requests or responses.
    • The browser should offer a way to turn ad click attribution on and off. We intend to have the default setting to be on to encourage websites to move to this technology and abandon general cross-site tracking.
    • The browser should not enable ad click attribution in private/incognito mode.

    Try It Out In Safari Technology Preview!

    We’re happy to offer Privacy Preserving Ad Click Attribution as an experimental feature in Safari Technology Preview 82+.

    First, enable the Develop menu, then go to the Experimental Features submenu.

    Experimental Features menu

    There you’ll find “Ad Click Attribution” which enables the feature itself, and “Ad Click Attribution Debug Mode” which enables debug logging for developers and shortens the 24–48 hour delay to a static one minute delay, also for use by developers.

    Debugging the Link Attributes

    A cross-site anchor element that wants to push ad click attribution data into the browser looks like this:
    <a href="https://some.site.example" addestination="https://shop.example" adcampaignid="55">

    To debug such elements, you use the Web Inspector’s console with the “Preserve Log” setting enabled. Here are a few examples of console warnings you may see if there’s something wrong with your attribution attributes:

    Both adcampaignid and addestination need to be set for Ad Click Attribution to work.

    Ad Click Attribution is only supported in the main frame.

    This tells you the anchor element is not part of the main frame.

    addestination can not be the same site as the current website. This technology is meant for cross-site attribution of ad clicks. There is no need for it within the same website.

    Debugging Storage of Ad Clicks

    For debugging anything beyond the anchor element, you need to use the system log (syslog). Here’s how you achieve that:

    1. Enable Ad Click Attribution Debug Mode in the Develop–>Experimental Features submenu.
    2. In your macOS Terminal, run: log stream -info | grep AdClickAttribution.

    Now if you click a cross-site element with adDestination and adCampaignID attributes, you should expect to see the following in your syslog:
    Storing an ad click.

    Debugging Conversions

    A conversion is signaled through a same-site HTTP redirect to /.well-known/ad-click-attribution/[a decimal value between 0 and 63 representing the conversion data]. Same-site here means search.example needs to be the server redirecting to https://search.example/.well-known/ad-click-attribution/. The reason for this is that search.example should be in control of when stored ad clicks on its site are consumed. Note that the conversion redirect is done as a subresource on shop.example so we don’t mean same-site as the main frame.

    Once you do such a redirect, the syslog might feature one of the following error messages:

    Conversion was not accepted because the HTTP redirect was not same-site. This is the requirement mentioned above, i.e. it has to be search.example redirecting to search.example/.well-known/ad-click-attribution/.

    Conversion was not accepted because it was requested in an HTTP redirect that is same-site as the first-party. Again, this technology is meant for cross-site attribution of ad clicks. There is no need for it within the same website.

    Conversion was not accepted because the URL's protocol is not HTTPS or the URL contains one or more of username, password, query string, and fragment. The request to the well-known location has to be HTTPS and cannot contain a username, password, query string, or fragment.

    Conversion was not accepted because the URL path did not start with /.well-known/ad-click-attribution/.

    Conversion was not accepted because the conversion data could not be parsed or was higher than the allowed maximum of 63.

    Conversion was not accepted because the URL path contained unrecognized parts. This is a catch-all error message for when the URL has unrecognized path elements or is not of the correct length.

    Detecting Successful Conversions

    If you got everything right in the redirect to the well-known location, you should see the following message in the syslog:
    Got a conversion with conversion data: 20 and priority: 0.

    Here you see the priority parameter. It is a way for the server to signal how important a particular conversion is so that the browser can report the most important one. Take the sales funnel example. There, multiple conversions will happen in succession: add to shopping cart, enter shipping info, enter payment info, and finalize purchase. Most likely, the finalized purchase is the conversion that should be reported together with the ad campaign ID. Priority can be 0 to 63, higher means higher priority, and the priority value is only used for internal bookkeeping, i.e. not sent in any request.

    Henceforth, lets assume the redirect is done with conversion data 20 and priority 12, like so:
    https://search.example/.well-known/ad-click-attribution/20/12

    Now, if there’s a stored ad click that matches this conversion, you’ll see detailed conversion information in the syslog:

    Converted a stored ad click with conversion data: 20 and priority: 12. This is when a previously unconverted ad click is converted.

    Re-converted an ad click with a new one with conversion data: 20 and priority: 12 because it had higher priority. This is for when there’s a conversion of higher priority that matches an already scheduled conversion request. The ad click is kept, re-converted with the high priority conversion, and scheduled for reporting.

    Replaced a previously converted ad click with a new one with conversion data: 20 and priority: 12 because it had higher priority. This is for when there is a different ad click (the user may have clicked more than one ad) with a scheduled conversion request but with lower priority. The newly converted ad click with higher priority replaces the old one.

    Finally, you’ll see the scheduling of the report request in the syslog:
    Setting timer for firing conversion requests to the debug mode timeout of 60 seconds where the regular timeout would have been 111003 seconds. This is special-cased for Ad Click Attribution Debug Mode. Instead of the 24 to 48 hour delay, there’s only a 60 second delay. The log message tells what the real delay would have been, in this case ≈31 hours.

    Receiving Conversion Reports

    When the scheduled timer fires, an HTTP POST request is made to ./well-known/ad-click-attribution/[conversion data]/[ad campaign ID], effectively reporting that a conversion happened for a user that previously clicked an associated advertisement. In our example, this request would go to:
    https://search.example/.well-known/ad-click-attribution/20/55
    … with the referrer request header set to:
    https://shop.example/

    When this request is about to go out, you’ll see the following syslog entry:
    About to fire an attribution request for a conversion.

    If something went wrong with the request, you’ll see it in the syslog:
    Received error: [error message] for ad click attribution request.

    Where To Send Feedback and Bug Reports

    Privacy Preserving Ad Click Attribution is in the early stage of being proposed as a standard through the W3C Web Platform Incubator Community Group (WICG). Please join the discussion and file issues to discuss how this technology fits with your use cases.

    If you find that the experimental feature in Safari Technology Preview doesn’t work as explained, please file a WebKit bug at https://bugs.webkit.org and CC John Wilander.

    For technical inquiries on Privacy Preserving Ad Click Attribution, you’ll find me on Twitter: @johnwilander

    May 22, 2019 01:00 PM

    May 15, 2019

    Debugging Media in Web Inspector

    Surfin’ Safari

    Media elements, such as <video> and <audio>, can sometimes be tricky to debug due to the sheer flexibility of the available functionality. Content can be loaded in a variety of different ways, such as a direct stream from a single file, byte-range requests from a single file, or multipart streams from different sources. In the past it was difficult to reconcile how content made the journey to being displayed on a page, as well as how it interacted with the other content around it. In macOS Mojave 10.14.4 we’ve added special functionality that makes it easier to understand how media content relates to network and JavaScript activity, specifically how that activity can stall or even prevent playback.

    Correlating Media Events and Network Activity

    Sometimes media playback may become stalled due to network activity, such as if a byte range request takes too long to load. Previously there was no easy way to identify these issues solely from Web Inspector. Now, however, Web Inspector provides an option called “Group Media Requests” that organizes network activity based on the related top-level media element. This makes it easier to keep track of related resources, such as byte range requests or items that are part of a media playlist.

    Since media elements aren’t resources it doesn’t make sense to have the usual “waterfall”. Instead, we’ve created a special timeline that keeps a log of playback related events fired by the media element, such as loadstart or stalled, as well as some basic state about the media element, such as whether it’s paused. A dashed line represents “paused”, a solid line represents “playing”, and no line represents “stopped”. If a <video> element enters fullscreen a grey area is shown indicating the time in which that <video> remained in fullscreen mode. On macOS <video> elements can also be rendered in a special way, called Power Efficient Video Playback, which is indicated by a green area.

    Identifying Media Issues Caused by JavaScript

    Sometimes, however, issues may arise due to incorrect JavaScript usage or layout issues. For these cases, we’ve also added the same event information to a new timeline in the Timelines tab called Media, as well as in the general timeline overview.

    Similar to the Network tab, correlating media events with nearby script activity can be a quick way to diagnose issues in situations where JavaScript causes the wrong activity to occur. In addition to this dedicated Media timeline all media events are also shown in the general timeline overview.

    Diagnosing issues with Power Efficient Video Playback

    Whenever a <video> element is about to render a frame on macOS, it makes a decision about whether to use the computer’s CPU/GPU or the display itself to composite the frame. In cases where the display can be used for frame compositing, it is often more power efficient. If your website contains full-screen video content, you should strive to follow these guidelines to ensure that your <video> is able to use display compositing as often as possible.

    • The <video> must be playing.
    • The <video> must be full-screen.
    • The <video> must not be occluded in any way (e.g. not showing controls or any content “above” it).

    Keep in mind that these are ways in which the web content itself can prevent power efficient video playback. There are other reasons that aren’t controllable from web content as to why a video may not enter a power efficient playback state, such as being connected to an external display or the device itself not supporting it.

    WebDriver

    Starting in macOS Mojave 10.14.4, WebDriver is able to access the total count of display composited frames which can be used for continuous integration testing.

    let video = document.querySelector("video");
    let videoPlaybackQuality = video.getVideoPlaybackQuality();
    videoPlaybackQuality.displayCompositedVideoFrames
    

    Each time a frame is display composited, that count will increment, so if the count does not increment, the <video> is not using display compositing.

    If you develop websites where the main content is a <video>, consider adopting these guidelines to provide a better experience for your users.

    Feedback

    These enhancements are available starting in macOS Mojave 10.14.4 and iOS 12.2. Is there additional information you’d like us to visualize with respect to media elements? Are there other elements that fit into this “category” for which you’d like to see similar instrumentation? Let us know! Please feel free to send us feedback on Twitter (@dcrousso or @jonathandavis) or by filing a bug.

    May 15, 2019 06:00 PM

    Release Notes for Safari Technology Preview 82

    Surfin’ Safari

    Safari Technology Preview Release 82 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 244684-245007.

    Web API

    • Enabled Fetch Keep Alive by default (r244931)
    • Changed to resolve the percentage values of inset properties against proper box. (r244906)
    • Changed to use the normal loading path for ping loads (r244700)
    • Fixed propertyRegistry() to be overridden for SVGFEFloodElement and SVGFEMergeElement (r244690)
    • Fixed programmatic scrolling with async overflow scroll (r244947)
    • Marked U+01C0 as a homograph of U+006C (r244903)
    • Updated WebKit to rely more on ICU’s UTF-8 support (r244828)

    JavaScript

    • Fixed crash when opening the inspector when using already in-flight generators and async functions (r244915)
    • Changed setting a frame src attribute to not run a JavaScript URL synchronously (r244892)

    Media

    • Changed WebAudio Node JS wrappers to not be collected if events can be fired (r244977)
    • Updated getDisplayMedia to require a user gesture (r244749)
    • Updated XMLHttpRequest to propagate user gestures for media playback (r244823)

    Storage

    • Made a change to stop IndexedDB transactions to release locked database files when network process is ready to suspend (r244687)
    • Updated to terminate Service Worker processes that use too much CPU (r244979)
    • Updated to terminate unresponsive Service Worker processes (r244936)

    CSS

    • Fixed font-weight: 1000 to be parsed successfully (r244817)
    • Fixed transform sometimes being left in a bad state after an animation (r244800)

    Accessibility

    • Fixed the hit-point of a link that spans multiple lines (r244983)

    Web Inspector

    • Updated support for dragging a node to the console to log the node (r244946)
    • Added support for recording actions performed on WebGL 2 contexts (r244908)

    Web Driver

    • Used a more informative key to indicate automation availability (r244791)

    Web Authentication

    • Added a check to require a call to WebAuthN from a focused document (r244938)

    WebGPU

    • Moved gpu API entry point from window to navigator (r244777)

    Browser Changes

    • Changed unchecking “Allow websites to ask for permission to send notifications” to also disable prompting for HTML5 notifications

    May 15, 2019 05:00 PM

    May 10, 2019

    Dark Mode in Web Inspector

    Surfin’ Safari

    Web Inspector on macOS Mojave now supports Dark Mode.

    Web Inspector: Light/Dark color schemes

    Dark Mode in Web Inspector was introduced in Safari Technology Preview last year. This article highlights implementation details which could be helpful for anyone adapting Dark Mode for their websites or web views.


    How do I enable Dark Mode for Web Inspector?

    Web Inspector matches macOS appearance. To enable Dark Mode in macOS Mojave, go to “System Preferences → General” and select “Appearance: Dark” (see How to use Dark Mode on your Mac for more details). Note that it’s not possible to enable Dark Mode only for Web Inspector.

    Dark Mode in Web Inspector is supported in macOS Mojave and later.

    Why Dark Mode isn’t supported on macOS 10.13 High Sierra and below?
    Web Inspector uses native form controls. macOS Mojave has a dark version of each form control (text fields, buttons, checkboxes, drop-downs, radio buttons, and etc.).

    Dark form controls on dark background and light form controls on light background in macOS Mojave

    macOS 10.13 High Sierra and below don’t support Dark Mode and only have light version of form controls. Without the proper form control styles, Dark Mode experience would be inconsistent so we decided not to do it.


    Resources tab: light/dark color schemes

    We tried to preserve syntax highlighting colors of the light theme. To keep text readable, we increased luminosity — by increasing brightness and sometimes changing the hue.

    Network tab: light/dark color schemes

    For some cases, like the bars in Network tab, we didn’t have to change anything. The same colors are readable in both light and dark themes.

    Console tab

    SVG and currentColor

    Instead of maintaining a separate set of icons for Dark Mode, we changed the colors of the existing icons with CSS wherever possible.

    SVG Icons

    Most of Web Inspector icons are SVG files included inline in HTML:

    <svg style="color: blue">
        <use xlink:href="Images/Paint.svg#root"/>
    </svg>
    

    Paint.svg:

    <svg>
        <path fill="currentColor" d="..."/>
    </svg>
    

    currentColor is the color of the parent element. In this example, it’s blue.

    currentColor only works for inline SVGs. It doesn’t work for SVGs included via <img> element or CSS (e.g. via background-image or content). For these cases, we used CSS filters to invert the colors.

    CSS filters

    The very first iteration of Dark Mode in Web Inspector was a single CSS rule:

    body {
      filter: invert()
    }
    

    Inverting colors rotated the hue by 180 degrees. Notice how the selected color changed from blue to orange. We fixed that by rotating the hue another 180 degrees:

    body {
       filter: invert() hue-rotate(180deg)
    }
    

    This had several issues. The colors didn’t exactly match the Dark Mode Human Interface Guidelines. The dark shadows turned light.

    We ended up not using this filter for body, but we used it for icons included via CSS (e.g. via background-image or content), where currentColor didn’t work.

    CSS variables

    Implementing Dark Mode took over 1,000 lines of CSS. We used CSS variables extensively to reduce code duplication.

    Previously, we had variable names such as --text-color-dark-gray and --text-color-light-gray. In Dark Mode, they would have to be inverted to provide desired contrast with the background: --text-color-dark-gray would become light gray, and --text-color-light-gray would become dark gray.

    We started using semantic names such as --text-color-secondary instead of --text-color-dark-gray, and --text-color-tertiary instead of --text-color-light-gray.

    Contributing

    Please report bugs regarding Dark Mode in Web Inspector on webkit.org/new-inspector-bug. Make sure to include “Dark Mode” in the title.

    If you’re interested in contributing or have any questions, please stop by the #webkit-inspector IRC channel.

    May 10, 2019 05:00 PM

    May 07, 2019

    Dark Mode Support in WebKit

    Surfin’ Safari

    With the introduction of Dark Mode in macOS Mojave last year, web developers have been asking for support in Safari to style web content that matches the system appearance. With the Safari 12.1 update in macOS 10.14.4, dark mode support in WebKit has arrived.

    WebKit.org Getting Started page shown in light modeWebKit.org Getting Started page shown in dark mode

    Dark Mode Behaviors

    By default there are no behavior changes with how pages look when a user is in dark mode. This preserves the standard assumptions web designers have had for almost thirty years — that a page defaults to a white background and black text. To change these defaults in dark mode would be a web compatibility nightmare.

    However, this leaves a large area of the screen with potentially bright content in dark mode. For simple content, an app could transform colors in the document for dark mode. This is what Mail does in macOS Mojave — it displays simple email messages with a dark mode interpretation.

    Not all web content is simple. For this reason Safari and WebKit do not auto-darken web content — documents will need to opt-in to dark mode. The main way to signal that your content supports dark mode it to adopt the new color-scheme style property, specified in this proposal.

    You would use this inherited property in your stylesheet like this:

    :root {
        color-scheme: light dark;
    }
    

    Specifying the values light and dark on the root element lets the engine know both modes are supported by the document. This changes the default text and background colors of the page to match the current system appearance. Also standard form controls, scrollbars, and other named system colors change their look automatically. Without this declaration, it would not be safe for the engine to use dark form controls or a dark color scheme, since many documents are designed with an assumed light color scheme.

    For example, this simple page, with color-scheme: light dark specified, will be entirely ready to use in both appearances.

    Example page in light modeExample page in dark mode

    These default background and text colors can be different values when using the high contrast accessibility mode or with different browser and system versions. You should not assume they will always be the same color values.

    The color-scheme style property is also supported on style rules that select specific elements in the document, not just the root element. This is useful if you have content or components that need a specific color scheme, specifically content with form controls, since dark mode form controls would not look correct on light backgrounds.

    There is also <meta name="color-scheme">, specified in this proposal. This meta tag declaration allows apps like Mail to know early on in parsing which color scheme to use for the document, as it can affect the auto-darkening transformations that get applied to the document’s colors. Declaring <meta name="color-scheme" value="light dark"> in a rich email message lets Mail know it supports its own styling for dark mode. You can also specify "light only", informing Mail that it should not transform your light color scheme email message.

    Note: The color-scheme property and meta tag name were renamed from supported-color-schemes in Safari Technology Preview 81. The old name is supported by WebKit, Safari, and Mail in macOS 10.14.4. This is a developing standard that could change more in the future.

    Styling For Dark Mode

    Defining color-scheme will get you going for simple content. For most web content, you will need to adopt the prefers-color-scheme media query, specified in this proposal, to style elements with custom colors or images. You can use this media query anywhere media queries are supported, such as in <picture> elements or window.matchMedia() for script triggers.

    The best way to deploy a dark and light color scheme in your documents is to utilize CSS variables. Then you can easily specify the colors in one place with the media query, and use those variables throughout your stylesheets. When the media query matches, the variables will change wherever they are used — auto switching with any appearance change.

    Here is an example using the media query to specify color values with CSS variables:

    :root {
        color-scheme: light dark;
        --special-text-color: hsla(60, 100%, 50%, 0.5);
        --border-color: black;
    }
    
    @media (prefers-color-scheme: dark) {
        :root {
            --special-text-color: hsla(60, 50%, 70%, 0.75);
            --border-color: white;
        }
    }
    
    .special {
        color: var(--special-text-color);
        border: 1px solid var(--border-color);
    }
    

    Images and Dark Mode

    For most hero images, they will still look great in dark mode — they might even pop off the page more than ever! As mentioned before, you can use the prefers-color-schemes media query in <picture> elements or style rules to select a different image if you have a dark mode version.

    For our simple example page, we can select a different hero image of the Mojave desert:

    <picture>
        <source srcset="mojave-night.jpg" media="(prefers-color-scheme: dark)">
        <img src="mojave-day.jpg">
    </picture>
    
    Example page in light modeExample page in dark mode

    You might also have images for controls or other interactive elements that you will need to update for dark mode. Many sites today have adopted vector images for these glyphs, such as SVG or web fonts. Fonts will naturally adapt to your dark mode text colors, but SVG images will need some additional styling to look correct. One trick you can use with SVG is the currentColor keyword to have embedded SVGs use the current text color — thus automatically adapting to your dark mode text colors. You can also use CSS variables to style colors inside of SVG, just like your other elements.

    For specific images that you can’t style any other way, you can use a CSS invert() filter. If the image also has color, you might consider using invert() hue-rotate(180deg) to moderately preserve the image intent. In any case you should not use an overall filter on broad sections of content, since that can have high memory and performance costs. Using direct styling of colors is the best approach for color accuracy and performance.

    Debugging Dark Mode

    Web Inspector now includes a navigation bar button in the Elements tab to toggle dark or light mode, depending on your current overall system preference. This allows you to quickly test pages in both appearances without having to toggle the whole system appearance in System Preferences. The Styles sidebar updates to reflect the current matching rules, as well as any CSS color variables.

    Dark Mode Web Inspector Toggle

    Examples

    As you might have noticed, if you are using Safari on macOS 10.14.4, the WebKit.org site already supports dark mode. Since we added support a while ago in Safari Technology Preview, a number of other sites have also adopted dark mode. Here are some examples you can check out today:

    • Twitter — The social networking service.
    • Emojipedia — Home of emoji meanings.
    • NSHipster — A journal of the overlooked bits in Objective-C, Swift, and Cocoa.
    • CSS DB — A database of staged CSS features.
    • Stuff & Nonsense — A design studio, specializing in creative design for digital products and web.

    May 07, 2019 05:00 PM

    May 01, 2019

    Release Notes for Safari Technology Preview 81

    Surfin’ Safari

    Safari Technology Preview Release 81 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 244110-244684.

    Dark Mode

    • Renamed supported-color-schemes to color-scheme (r244408)
    • Standardized the <meta name="color-scheme"> separator (r244413)

    JavaScript

    • Added support for incremental bytecode cache updates (r244143)

    WebRTC

    • Allowed the MediaSource API to be enabled via website policy (r244197)
    • Prevented restarting WebRTC stats timer if backend is stopped (r244632)
    • Supported RTCDataChannel blob binaryType (r244196)
    • Added support for parsing FairPlayStreaming PSSH boxes. (r244439)

    Media

    • Updated AudioSession route sharing policy (r244223)
    • Fixed layout issues occuring when entering full screen mode (r244545)

    Layout

    • Added @page margin support (r244202)
    • Fixed synthesized baseline for CSS Grid and CSS Flex (r244213)

    Accessibility

    • Removed deprecated Accessibility Object Model events (r244582)

    Web API

    • Changed requestAnimationFrame to execute before the next frame (r244182)
    • Changed visited link hash to be computed only once (r244642)
    • Fixed link clicks in PDFs to not send referrer headers (r244405)

    Storage

    • Changed to clear cache storage structures before removing all related files (r244162)
    • Delayed initialization of quota users until the first quota request (r244112)
    • Fixed blob type not getting stored correctly in IndexedDB when IDBObjectStore has autoIncrement and keyPath options (r244436)

    Security

    • Changed to Ignore X-Frame-Options header when frame-ancestors CSP directive is present (r244589)
    • Implemented remembering device orientation permission for the duration of the browsing session (r244382)

    Web Inspector

    • Added line and column numbers with source location when logging CSP messages to the console (r244563)
    • Added support for drag-and-drop to import files in the Canvas and Audit tabs (r244278)
    • Added support for drag-and-drop to import HAR archives in the Network tab (r244576)
    • Added support to show Worker imported scripts in the Open Quickly dialog even if Web Inspector is opened after the Worker was created (r244363)
    • Changed to prevent starting property selection when pressing the right mouse button in the Styles sidebar (r244616)
    • Changed to show a resource in the Debugger even if an extension script throws a parse error (r244398)
    • Changed to include the number of Network Requests in the Statistics section of the CPU Usage timeline (r244411)
    • Ensured that object previews with Promises don’t add unnecessary catch handlers that prevent rejectionhandled events from being fired in the Console (r244312)
    • Ensured that tracking stops for the CPU Usage Timeline when disconnecting Web Inspector (r244478)
    • Fixed an issue where navigating to a resource sometimes shows a different resource (r244412)
    • Made starting and stopping a recording more consistent and reliable in the Timelines tab (r244195)
    • Provided a more obvious way to return to the overview when viewing imported recordings in the Canvas tab (r244560)
    • Updated to preserve multiple selection in the Elements tab when switching tabs (r244154)
    • Updated to disable special breakpoints when they are deleted in the Sources tab (r244274)

    WebDriver

    • Added hooks to make it possible to easily run WPT WebDriver web server by itself (r244492)
    • Fixed JavaScript evaluation failures if a dialog is shown (r244230)
    • Fixed enter fullscreen error handling code (r244236)
    • Fixed the Set Cookie endpoint to correctly set subdomain cookies (r244281)

    Web GPU

    • Implemented support for indexed drawing (r244147)
    • Prevented narrowing conversions during Metal function calls on 32-bit platforms (r244235)

    Privacy Preserving Ad Click Attribution

    • Added the privacy preserving Ad Click Attribution API as an experimental feature (r244581)

    May 01, 2019 08:30 PM

    April 24, 2019

    Intelligent Tracking Prevention 2.2

    Surfin’ Safari

    Note: Read about past updates to this technology in other blog posts about Intelligent Tracking Prevention, and the Storage Access API.

    The beta releases of iOS 12.3 and Safari on macOS Mojave 10.14.5 include an updated version of Intelligent Tracking Prevention (ITP). For purposes of developer communication, we’ll refer to it as ITP 2.2.

    Tracking Via Link Decoration Caps Client-Side Cookies to 1 Day of Storage

    As of ITP 2.2, persistent cookies set through document.cookie are capped to one day of storage when both of the following conditions are met:

    1. A domain classified with cross-site tracking capabilities was responsible for navigating the user to the current webpage.
    2. The final URL of the navigation mentioned above has a query string and/or a fragment identifier.

    The rest of this blog post explores this in detail.

    What Is Cross-Site Tracking Via Link Decoration?

    Let us first explain the reason for this change. Intelligent Tracking Prevention was designed to protect user privacy by preventing cross-site tracking. Since ITP 1.0 was launched, we have seen increased usage of URL query strings for cross-site tracking purposes.

    social.example link decoration to news.example, blog.example, and shop.example

    In the illustration above, the social network website social.example is adding a “click ID” as a query parameter to every outgoing link. Each of these click IDs are connected to the real user ID “Jane Doe” in social.example’s database.

    Once Jane lands on the destination page, the social network uses JavaScript to read the click ID in the URL query string and store it in a first-party cookie. How does the social network gain scripting powers on the destination website? The destination website can import scripts from social networks or ad networks to integrate a feature, and these scripts can be augmented to read, store, and report back incoming click IDs.

    From this point, every time Jane happens to visit news.example, blog.example, or shop.example, social.example’s script reads the click ID first-party cookie, sends it to its server, and identifies her. Now social.example can enrich its profile of Jane by tracking her activities on these websites and they can serve her individually targeted ads on these websites, all because she once navigated there through a link on social.example.

    This shows how click IDs can effectively work like user IDs for cross-site tracking.

    Further Consequences of Cross-Site Tracking Through Link Decoration

    There are two additional consequences of link decoration that are concerning for user privacy and for web compatibility.

    User IDs are Transferred When Users Share Links

    When tracking information is added to links, it can be used to connect people and devices when the links are shared. Jane Doe in the example above may want to share the news.example link with a coworker. Through link decoration, her social.example user ID is now copied into her coworker’s web browser and can be used to connect her with her coworker on social.example’s platform.

    Some Websites Fail to Load When They Receive Unknown Query Parameters

    Many websites simply fail to load when they receive query parameters that their servers don’t recognize. In one case we came across, a link to a comedy website was shared on a popular social network and the website rendered blank because of the social network’s link decoration. The following comments ensued:

    • Commenter 1: “The link is broken.”
    • Poster: “Sorry about that. Here’s the link again …”
    • Commenter 1: “That link is broken too.”
    • Commenter 2: “Yeah, the social network messes with the links. You have to use a link shortener.”

    Exactly When Should I Expect Cookies to Expire?

    Here’s an example of when ITP 2.2 will cap cookie persistence to one day:

    1. The website social.example has been classified by ITP as having cross-site tracking capabilities. How this classification works is explained in detail in the original ITP blog post and the ITP 2.0 blog post.
    2. The user clicks a link on social.example.
    3. The click results in a cross-site navigation that lands on shop.example and the landing URL has a query string and/or a fragment identifier.
    4. The shop.example webpage sets a persistent cookie through document.cookie. This cookie will now have a maximum expiry of one day.

    What About Ad Click Attribution?

    You might recall from our original ITP blog post that we mentioned link decoration as a way to achieve ad attribution in navigations. Such attribution should serve the destination site information about what ad was clicked and on which site, not information about who the user is.

    The kind of link decoration described above is being used for cross-site tracking of users.

    What If My Website Uses Query Strings or Fragments?

    Just like with cookies and other web storage, there are legitimate and privacy-infringing uses of query strings and fragments, and cross-site trackers try to make their activities indistinguishable from good use. We are obliged to prevent cross-site tracking for Safari users.

    It has been suggested that ITP should simply remove any link decoration when the click source has cross-site tracking capabilities. We opted not to do so because of legitimate use and the risk of breaking website compatibility (see the comedy example above).

    Note though, the one-day cap on document.cookie storage only applies if your site was navigated to by another site with cross-site tracking capabilities, and only for the current webpage.

    Developers, Here’s What You Can Do About It

    We have discussed with developers how cross-site tracking through link decoration came about. After all, user IDs are stored in websites’ first-party cookies — a storage space they manage.

    Our impression is that many developers never understood it was happening; in many cases, changes to third-party JavaScript embedded on websites introduced link decoration without web developers’ knowledge. However, once developers know that their websites are leveraged for cross-site tracking purposes, they can choose to filter out trackers’ link decoration.

    We hope that web developers will join us in better protecting user privacy while concurrently creating the best user experiences on the web.

    April 24, 2019 05:00 PM

    April 17, 2019

    Release Notes for Safari Technology Preview 80

    Surfin’ Safari

    Safari Technology Preview Release 80 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 243538-244110.

    WebGPU

    • Updated setBlendColor, setViewport, setScissorRect in GPURenderPassEncoder (r244093)
    • Replaced unsigned longs in WebGPU with uint64_t (r243658)
    • Standardized WebGPU object reference counting and creation logic (r243563)
    • Removed WebMetal experimental feature in favor of WebGPU (r243666)

    Web API

    • Implemented ResizeObserver (r243643)
    • Added support for “noreferrer” window feature to window.open() (r243705)
    • Added support for <object>.contentWindow (r243638)
    • Changed window.closed to true immediately when close() is invoked (r243661)
    • Changed to close the service worker database on network process suspension (r244097)
    • Changed Fetch to allow used body replacement in Request constructor (r243757)
    • Fixed HTML fragment serialization to not strip whitespace from URL attribute values (r243821)
    • Made someWindow.frames, someWindow.self, someWindow.window always return someWindow, even without a browsing context (r243669)
    • Removed conditional parsing of <noembed> content in the HTML parser (r243782)
    • Fixed the loadstart event for XMLHttpRequestUpload to be correctly initialized (r243765)
    • Fixed getBoundingClientRect returning an empty rect on a collapsed range (r243635)
    • Fixed the select element not showing a popup if the element lost focus while the popup was previously shown (r243601)
    • Pasting a table from Confluence strips table cell content (r243653)
    • Updated to ensure resetting the storage quota takes into account third party origins (r243806)

    SVG Animation

    • Fixed SVG Animation (SMIL) on <text> or <tspan> to work correctly on the second run (r243780)

    Media

    • Added support for muting screen capture and camera/microphone independently (r243899)

    CSS

    • Implemented white-space: break-spaces value (r244036)
    • Removed functionality for -apple-trailing-word (r243819)
    • Allowed FontFace names which start with a number (r243637)

    Accessibility

    • Prevented <svg> elements with labels and no accessible contents from getting exposed as empty AXGroups (r244029)
    • Changed to automatically compute accessibility labels for Apple Pay buttons (r244061)

    Web Inspector

    • Added support for showing WebGPU contexts in the Canvas Tab (r243763)
    • Added support for showing the resource initiator in the summary of the headers in the Network Tab (r243614)
    • Adjusted the energy impact thresholds in the CPU Usage Timeline (r243704)
    • Ensured that the Script Profiler debugging thread is not counted as part of the page usage in the CPU Usage Timeline (r243679)
    • Disabling a breakpoint for a specific event listener no longer removes it from the list of breakpoints (r243722)
    • Prevented breakpoints for attribute modifications from firing when breakpoints are disabled (r243719)
    • Prevented single clicks from following links in text editors that are not read-only (r243826)

    Safari Extensions

    • Legacy Safari Extensions (.safariextz files) are no longer supported. Safari App Extensions and Content Blockers, which can take advantage of powerful native APIs and frameworks as well as web technologies, can be distributed with apps in the App Store or from developers’ websites. You can learn more at developer.apple.com/safari/extensions/.

    April 17, 2019 05:00 PM

    April 11, 2019

    Link Click Analytics and Privacy

    Surfin’ Safari

    We’ve recently received questions on what we refer to as link click analytics, and specifically an internal setting for disabling the Ping attribute for anchor elements.

    Privacy By Default

    WebKit always strives for privacy by default. To name three unique such efforts – we partition third-party data storage and ServiceWorkers by default, we partition HTTP caches by default, and our Intelligent Tracking Prevention (ITP) feature is on by default in Safari. To the best of our knowledge, no other browser on the market offers similar protections.

    However, there are cases when users want even stronger privacy guarantees and are willing to trade some functionality or web compatibility for it. Two such examples are Private Browsing and Content Blockers.

    Let’s have a look at how Safari and WebKit’s privacy features play into link click analytics.

    What Is Link Click Analytics?

    The goal of link click analytics is to report to a web server that a navigational link click happened and that the user is leaving the webpage. Such auditing can be used for first-party web analytics as well as third-party cross-site tracking. The latter is where ITP comes in.

    How Can Websites Do Link Click Analytics?

    There are several ways for websites to do link click analytics. The ones we see in use today are:

    • Synchronous XHR (XMLHttpRequest).
    • Asynchronous XHR or Fetch, with a delay.
    • First-party bounce tracking.
    • The Beacon API.
    • The Ping attribute.

    Let’s go through the details of these techniques.

    Synchronous XHR

    Synchronous XHR is the synchronous version of XMLHttpRequest. Such synchronous calls often cause hangs on the web since they block the webpage while it waits for the server’s response. Therefore, web browsers are actively trying to remove the API.

    In the context of link click analytics, a site that is willing to inconvenience users might do something like the following (please don’t use this technology):

    window.addEventListener("unload", function(event) {
      let xhr = new XMLHttpRequest(),
          data = captureTrackingData(event);
    
      xhr.open("post", "/log", false); // 'false' implies synchronous.
      xhr.send(data);
    });
    

    The code above triggers when the user clicks a link and the current webpage unloads. The synchronous XHR blocks the navigation until it’s done which delays the navigation significantly. For users, this is perceived as poor performance.

    Asynchronous XHR or Fetch, With a Delay

    Another popular way to do link click analytics is through asynchronous XHR or Fetch, not in the least because they allow for cross-site requests to third-party trackers.

    While not blocking all execution on the webpage, this introduces an artificial delay to the navigation which users experience as poor performance. We’ve seen between 100 and 350 ms delays in such link click analytics scripts, and even some that do a busy loop while waiting. This is what a delayed navigation with asynchronous XHR looks like:

    const clickTime = 350; // Milliseconds.
    
    theLink.addEventListener("click", function(event) {
      event.preventDefault(); // Cancel the user's link click.
    
      let xhr = new XMLHttpRequest(),
          data = captureTrackingData(event);
    
      xhr.open("post", "/log", true); // 'true' implies asynchronous.
      xhr.send(data);
      setTimeout(function() {
        window.location.href = theLink.getAttribute("href");
      }, clickTime); // Navigate properly after 350 ms.
    });
    

    First Party Bounce Tracking

    You might recall from our earlier blog posts that ITP 2.0 detects first-party bounce trackers and classifies them as any other kind of cross-site tracker.

    Say the user clicks on a news.example link on the social.example website. Instead of navigating them straight to their destination news.example, they are rapidly navigated through trackerOne.example and trackerTwo.example before reaching news.example.

    social.example → trackerOne.example → trackerTwo.example → news.example

    This is a way to let trackerOne and trackerTwo do link click analytics and the increased load time is again bad for performance.

    The Beacon API

    While not specifically built for link click analytics, the Beacon API is a way of sending arbitrary analytics and/or tracking data without affecting the user experience. This is how Beacon can be used for link click analytics:

    window.addEventListener("unload", function(event) {
      let data = captureTrackingData(event);
      navigator.sendBeacon("https://tracker.example/", data);
    });
    

    Beacon requests are guaranteed to be initiated before the page unloads but do not block the webpage or delay the navigation.

    The Ping Attribute

    Ping is an attribute on anchor elements, popularly referred to as just links. The purpose of Ping is link click analytics, plain and simple. Here’s how it’s used:

    <a href="https://news.example" ping="https://tracker.example/going-to-news-example">Read the news</a>
    

    The Ping request to tracker.example above does not block or delay the navigation to news.example.

    What Can WebKit Do About this?

    As can be seen above, websites have several ways to go about logging or tracking a user’s clicks. The first three — synchronous XHR, Fetch with delay, and first-party bounce tracking — all hurt performance and make the web experience worse. The latter two — Beacon and Ping — still log clicks but do so without hurting performance.

    Just turning off the Ping attribute or the Beacon API doesn’t solve the privacy implications of link click analytics. Instead, it creates an incentive for websites to adopt tracking techniques that hurt the user experience. In effect, the choice between supporting Ping and not is not one of privacy, rather it is a choice between a good user experience and a bad one.

    So our approach is to have ITP block cookies and downgrade the referrer header (see the section on Origin-Only Referrer in our ITP 2.0 blog post) for all the link click analytics techniques listed when the request goes to a third-party domain classified with cross-site tracking capabilities. ITP also cleans up website data for first-party bounce trackers.

    The distinction we’re making here is between analytics of link clicks in general versus third-party analytics of link clicks tied to individual users. The latter is what ITP prevents and what we think is the right balance for on-by-default privacy protections.

    What Can Users Do About This?

    For users who want to fully block third-party link click analytics, WebKit and Safari supports Content Blockers. The effect of such load blocking may include blocking of ads or third-party widgets. If you want to install a Content Blocker, check out the App Store where you’ll find plenty of offerings. If you are a developer who wants to build a Content Blocker, see Apple Developer Documentation.

    A Final Note On Hidden Feature Flags

    As a detail of its implementation, WebKit makes use of the User Defaults mechanism on Apple platforms. These flags or pieces of configuration data are not exposed in Safari’s menus or in Safari Settings. Instead, they are used to control the inner workings of features to, for instance, enable quality assurance testing.

    Say WebKit decides to obsolete synchronous XHR altogether on Apple platforms. Such a change might be put behind a User Defaults flag so that it’s easy for engineers to assess whether a reported issue stems from the obsoleted synchronous XHR, or something else.

    Until recently, Safari supported an internal User Defaults flag to disable support for the Ping attribute. It was never our intention to surface this flag as a customer setting. We think it’s misguided to offer users the ability to disable web-facing features if doing so doesn’t disable or prevent the ends of that technology. Instead, Intelligent Tracking Prevention and Content Blockers offer users different levels of support for categorically affecting link click analytics.

    April 11, 2019 10:00 PM

    April 08, 2019

    Philippe Normand: Introducing WPEQt, a WPE API for Qt5

    Igalia WebKit

    WPEQt provides a QML plugin implementing an API very similar to the QWebView API. This blog post explains the rationale behind this new project aimed for QtWebKit users.

    Qt5 already provides multiple WebView APIs, one based on QtWebKit (deprecated) and one based on QWebEngine (aka Chromium). WPEQt aims to provide …

    By Philippe Normand at April 08, 2019 10:20 AM