May 18, 2023

WebKit Features in Safari 16.5

Surfin’ Safari

Today we are pleased to share what new’s in WebKit for Safari 16.5.

After the massive, web-developer-feature-packed release of Safari 16.4, this version focuses predominately on polishing existing features and fixing bugs. Safari 16.5 does contain a handful of new features including CSS Nesting, :user-valid and :user-invalid, support for pre-orders and deferred payments with Apple Pay, and an update to Lockdown Mode.

CSS Nesting

Safari 16.5 includes support for CSS Nesting. A long-time favorite feature of pre-processors, nesting allows web developers to write code like this:

.foo {
  color: green;
 .bar {
    font-size: 1.4rem;
  }
}

Learn all about CSS Nesting — including the current limitation that prevents the use of an element selector (like article) without including a symbol (like &) before the element — in Try out CSS Nesting today.

The CSS Working Group is currently working on a very promising idea for removing the requirement that every nested selector begin with a symbol. Hopefully sometime in the future, that limitation will no longer exist.

CSS user valid and invalid pseudo-classes

The pseudo-classes :user-valid and :user-invalid are now supported in Safari 16.5.

For years, :valid and :invalid could be used to target an input field or other form field based on whether it is in a valid or invalid state. But there’s been one problem — as soon as a user starts typing, the content is evaluated and declared invalid if it doesn’t match. For example, an email address is marked :invalid as the first half is typed, before the @ is present. This has led to developers using JavaScript for this use case instead.

The new :user-invalid and :user-valid pseudo-classes are much smarter about when they are triggered, removing the need for JS. Compare :invalid to :user-invalid in a browser with support to see the difference.

Compare :invalid to :user-invalid in Safari 16.5 or Firefox 88 or later.

These new pseudo-classes are especially helpful when combined with :has() to provide powerful tools for styling anything on the page, like the label of a form field.

Apple Pay

Apple Pay on the web now supports pre-orders and deferred payments.

Lockdown mode

Lockdown mode now disables WebCodecs API.

Lockdown mode is an optional, extreme protection that’s designed for the very few individuals who, because of who they are or what they do, might be personally targeted by some of the most sophisticated digital threats. Most people are never targeted by attacks of this nature.

Bug Fixes

WebKit for Safari 16.5 provides multiple bug fixes and feature polish.

CSS

  • Fixed Scroll to Text Fragment sometimes scrolling to the top after reloading the page.
  • Fixed support for x resolution unit in calc().
  • Fixed reflecting trimmed block-start, block-end, inline-start, and inline-end margins for grid or flex items in computed styles.
  • Fixed the top offset of self collapsing children at the end of a block container with block-end margin trim.
  • Fixed triggering layout when changing margin-trim value.
  • Fixed increasing column-count above 2 not updating the layout.
  • Fixed CSS custom properties not applying to an SVG use element’s shadow tree.
  • Fixed new CSS property unexpectedly dropped from an empty CSS rule when tabbing through or editing a selector.
  • Made -webkit-image-set() an alias of image-set().

Editing & Forms

  • Fixed hairline on the selection of bidi text.
  • Fixed photo library picker showing videos for accept="image/*".

JavaScript

  • Updated digital display in Intl.DurationFormat to match spec changes.

Layout

  • Fixed text wrapping for bidi text when line-breaking.

Media

  • Fixed non-audible AudioContext preventing the audio session to change from play-and-record after stopping capture.
  • Fixed handling video streams containing a CodecDelay value that caused an audible pop at the beginning of video playback.
  • Fixed video freezing in a video conference when removing AirPods Pro during the call.

Scrolling

  • Fixed snapping to the last snap position when performing layout when scroll snapping occurs with a physical mouse wheel.
  • Fixed pinch-to-zoom when toggling on and off scroll snapping.
  • Fixed scroll snapping jumping to the previous page when swiping to the next page.
  • Fixed scroll snapping to work with a physical scroll wheel on a mouse.

Rendering

  • Fixed form controls rendering.
  • Fixed visual updates for content: counter() when position: absolute is set.
  • Fixed an unexpected visible first frame of a transform animation when !important style overrides the animated value.

Web API

  • Fixed filling metadata headers for preflight requests.
  • Fixed OffscreenCanvas WebGL to fire the context lost event.
  • Fixed getFileHandle() to return a TypeMismatchError on unexpected entry type.

Web Apps

  • Fixed “Untitled” label on the back to previous app button when opening a web app via a link.

Web Assembly

  • Fixed WASM SIMD breaking WebP decoding applications.

Web Inspector

  • Added initial support for color-mix CSS values.
  • Fixed element ::backdrop rules showing up without a backdrop.
  • Fixed “Selected element” console entry filling an entire row.
  • Fixed an issue causing the mini console to always opens when choosing “Inspect Element”, even if it was previously closed.

Feedback

We love hearing from you. Send a tweet to @webkit to share your thoughts on Safari 16.5. Find us on Mastodon at @jensimmons@front-end.social and @jondavis@mastodon.social. If you run into any issues, we welcome your feedback on Safari UI, or your WebKit bug report about web technologies or Web Inspector. Filing issues really does make a difference.

Download the latest Safari Technology Preview to stay at the forefront of the web platform and to use the latest Web Inspector features. You can also read the Safari 16.5 release notes.

Updating to Safari 16.5

Safari 16.5 is available for macOS Ventura, macOS Monterey, macOS Big Sur, iPadOS 16, and iOS 16. Update to Safari 16.5 on macOS Monterey or macOS Big Sur by going to System Preferences → Software Update → More info, and choosing to update Safari. Or update on macOS Ventura, iOS or iPadOS, by going to Settings → General → Software Update.

May 18, 2023 05:59 PM

May 17, 2023

Release Notes for Safari Technology Preview 170

Surfin’ Safari

Learn about the latest web technology updates in Safari Technology Preview: CSS, Layout, JavaScript, Media, Popover, and Accessibility.

Overview

Safari Technology Preview Release 170 is now available for download for macOS Monterey 12.3 or later and macOS Ventura. If you already have Safari Technology Preview installed, you can update it in the Software Update pane of System Preferences on macOS Monterey, or System Settings under General → Software Update on macOS Ventura.

This release includes WebKit changes between: 263290@main…263537@main.

CSS

  • Fixed :dir pseudo-class to invalidate after removing dir content attribute from the document element (263357@main)
  • Fixed computing the percentage height of table cell replaced children where the table cell has a horizontal scrollbar (263318@main)
  • Fixed shorthand flow relative margin and padding values to resolve for individual CSS property values (263372@main, 263391@main)
  • Fixed trimming nested self-collapsing children at block-end (263439@main)
  • Fixed trimmed block-end margins for block containers to be reflected in the computed style in a horizontal writing mode (263398@main)
  • Fixed content at block-start edge to have their trimmed margins reflected in the computed style (263412@main)
  • Fixed the CSS hypot() function sometimes returning the result squared (263351@main)

Forms

  • Fixed the value not updating on <meter> and <progress> elements (263473@main)

Layout

  • Fixed unexpected content wrapping when percent padding is present (263535@main)
  • Fixed sibling flex items sometimes showing clamped content (263360@main)
  • Fixed incorrect damage line index sometimes causing double inline items (263455@main)
  • Fixed content getting truncated too early caused by subpixel flooring (263428@main)
  • Fixed text-overflow: ellipsis truncating the text incorrectly in right-to-left (263418@main)

JavaScript

  • Expanded existing property names caching onto Reflect.ownKeys() and Object.getOwnPropertySymbols() (263441@main)
  • Integrated inlined megamorphic access in DFG and FTL (263300@main)
  • Optimized Object.assign with empty object (263444@main)
  • Simplified and optimized the JSON parser (263416@main)

Media

  • Fixed content briefly zooming in when exiting picture-in-picture (263350@main)
  • Fixed AirPlay sometimes failing (263534@main)

Popover

  • Fixed handling of popovers moved between documents after beforetoggle event (263449@main)
  • Implemented focus navigation scopes for popovers (263532@main)

Accessibility

  • Fixed aria-flowto for display: contents elements (263425@main)
  • Fixed display: contents elements never returning any selected AX children despite having them (263339@main)
  • Fixed the accessible name getting computed incorrectly for display: contents elements that rely on labels or captions (263379@main)
  • Fixed form controls to be focusable when tabindex attribute is set (263527@main)

May 17, 2023 08:59 PM

May 11, 2023

A quick introduction to the WPE WebKit Project

Surfin’ Safari

As mentioned in a previous post here and also in the related post from the WPE WebKit blog, the WPE project is a port of WebKit which, at the time of this writing, is responsible for bringing WebKit to millions of embedded devices around the world: you can find it in set-top-boxes, cars, cooking machines, and smart home appliances, to name a few examples.

While there is already some information about the design and architecture of WPE on the wpewebkit.org website, the team behind WPE at Igalia has recently started a series of blog posts to explain different aspects of WPE in a more personal way by having different members of the team talk about specific areas they are most familiar with.

As a result, the first installment of the series is a high-level overview of the WPE project where Claudio Saavedra talks briefly about the architecture of the project and how it’s designed in terms of several modular components that enable integrators to quickly build performant, lightweight, and reliable Web browsers optimized for their specific devices in a flexible way.

Therefore, we would like to invite you to check out his blog post for a better idea of how the different pieces of the WPE WebKit project fit together and how the project is perfect to accomodate different needs and use cases. This is very important not only to the world of embedded devices, which benefits directly of this flexibility, but also to the whole WebKit ecosystem in general by providing a broader use base for our beloved Web engine.

Last but not least, feel free to check regularly the WPE WebKit blog for new installments of the series, where articles on different topics will be published in the future.

May 11, 2023 04:00 PM

May 05, 2023

Introducing WebKit Documentation

Surfin’ Safari

Screenshot of WebKit documentation using a light theme in Safari

Today, we are excited to announce docs.webkit.org, a new documentation home for the WebKit developer community. WebKit is a massive open source project containing many subsystems, and coming up to speed as a new contributor or diving into a new subsystem can be quite a daunting endeavor. This documentation’s focus is on exposing the internals of the web engine in an easy to understand matter.

To date, we have used the wiki features of Trac as our primary documentation tool, and while it has served us well it no longer measures up to more modern systems. Our new documentation system is built upon MkDocs, with theming designed to match WebKit.org’s visual style. This approach provides numerous benefits:

  • Built-in search helps you quickly locate articles.
  • Documentation can be read offline and online.
  • Documents are authored in Markdown, allowing members to easily contribute.
  • Theming matches your system appearance (Light/Dark mode).
  • Documentation is hosted on GitHub, allowing developers to submit PRs to make edits and review changes.

When visiting docs.webkit.org, for the first time, you will see several improvements. To the left of the article, there is an accordion style menu to navigate between pages. On the right, there is a table of contents to help you easily zoom through longer documents. Above that is a search bar along with a link to our GitHub documentation page. Every article contains an edit button to bring you directly to the source code of the article located on GitHub. In addition, headers within the article can be easily linked to to allow easier sharing of information among colleagues.

Members are welcome to contribute in our new Documentation project repository. Much of the relevant documentation from past systems has already been ported over, but if you notice any missing information please submit a PR.

May 05, 2023 05:35 PM

May 03, 2023

Release Notes for Safari Technology Preview 169

Surfin’ Safari

Safari Technology Preview Release 169 is now available for download for macOS Monterey 12.3 or later and macOS Ventura. If you already have Safari Technology Preview installed, you can update it in the Software Update pane of System Preferences on macOS Monterey, or System Settings under General → Software Update on macOS Ventura.

This release includes WebKit changes between: 262762@main…263289@main.

Web Inspector

  • Console Tab
    • Fixed internal properties not getting greyed-out in object previews (262924@main)
    • Fixed non-enumerable properties appearing as though they’re internal (262923@main)
    • Fixed timestamps (263261@main)
    • Fixed repeated logs sometimes having the wrong timestamp (263265@main)
    • Fixed timestamps causing objects to be shown on a separate line (263263@main)
  • Sources Tab
    • Added support for ES2022 Private Fields when inspecting and logging JavaScript objects (262882@main)

CSS

  • Implemented overflow-block and overflow-inline media query features (263088@main)
  • Implemented the from-font value for font-size-adjust (262800@main)
  • Fixed parsing of nested at-rules with CSSOM (263028@main)
  • Fixed background-size to not accept unit-less lengths (262873@main)
  • Fixed cross-size width of a table flex-item inside inline-flex with column flex-direction (263001@main)
  • Fixed text-emphasis marks to not render if there is no emphasized character (262997@main)
  • Fixed computed style to reflect trimmed block-start, block-end, inline-end, and inline-start margins for grid items in horizontal writing-mode (262967@main, 263002@main, 263008@main, 263006@main)
  • Fixed rebuilding the rule set from the original (not resolved) selector (263026@main)
  • Fixed rendering issue with checkbox in flexbox layout (263052@main)

Scroll Snap

  • Fixed snapping to the last snap position when performing layout when scroll snapping occurs with a phyisical mouse wheel (263108@main)
  • Fixed scrolling with a physical mouse wheel to not always animate to the closest snap point (263071@main)
  • Fixed updating the tracking for currently snapped boxes on each scroll-related snap and re-snapping after layout (263097@main)

JavaScript

Popover

  • Renamed :open to :popover-open and removed :closed (262764@main)

SVG

Accessibility

  • Fixed aria-activedescendant to work for display: contents elements (263163@main)
  • Fixed aria-grabbed and aria-dropeffects to work for display: contents elements (263249@main)
  • Fixed CSS speak-as, AXAccessKey, aria-owns, and URL AX APIs to work for display: contents elements (263205@main)
  • Fixed isSelected AX APIs for some types of display: contents elements (263014@main)
  • Fixed properly exposing lists that have display: contents list items (262889@main)
  • Fixed conveying focus movement when using aria-activedescendant to set the active cell within a grid (263189@main)
  • Fixed computing the wrong accessibility clickpoint for display: contents links and headings (263287@main)

Media

  • Fixed to end a muted microphone track if its device disappears (263132@main)
  • Fixed a default camera whose facingMode is unknown to be selected by getUserMedia if there is no facingMode constraint (263022@main)

Rendering

  • Fixed <audio> element counter-scaling causing overlapping media elements when page scale is less than 1 (262952@main)

Web API

  • Added customElements.getName method (263281@main)
  • Added missing service worker content filter check (262972@main)
  • Fixed createImageBitmap using ImageData to respect the premultiply flag (263137@main)
  • Fixed data: URL base64 handling to always match atob() (262976@main)
  • Fixed postMessage support to occur when creating service worker clients (262818@main)
  • Fixed HTML fast parser failing to parse complex HTML entities (262856@main)
  • Fixed missing underline after the first character in contenteditable (262914@main)
  • Fixed window.open from an iframe not exiting fullscreen for parent document (263284@main)

May 03, 2023 09:01 PM

April 25, 2023

Badging for Home Screen Web Apps

Surfin’ Safari

Along with the many other features for web apps on iOS and iPadOS 16.4, WebKit now includes support for the W3C’s Badging API.

A badged web application icon in the iOS dock showing the number 15.

This straightforward API allows web developers to badge the icon of a web app. This article explores the various features of the Badging API and how to use it, and our ongoing collaboration with W3C’s Web Applications Working Group to improve the specification.

Home Screen Web App Availability

In iOS and iPadOS 16.4, the Badging API is available exclusively for web apps the user has added to their home screen. You won’t find the API exposed to websites in Safari or other browsers, or in any app that uses WKWebView.

You can easily check for the availability of the API using feature detection for the setAppBadge function:

if ('setAppBadge' in navigator) {
  // API is available...
}  

Badges and Notifications

On iOS and iPadOS, badging has always been closely tied to notifications.

Apps built for macOS, iOS, and iPadOS are free to set their badge counts through platform specific APIs whenever they like. The user must grant the app permission to display notifications before the badge will appear.

Similarly, web apps are free to call setAppBadge() whenever they like, but the badge will only appear if the user has granted notifications permission.

To request notification permission, call Notification.requestPermission() as part of a user activation (such as a button click):

<button onclick="requestNotificationPermission()">
  Request notifications permission
</button>

<script>
  async function requestNotificationPermission() {
    const permission = await Notification.requestPermission();
    if (permission === 'granted') {
      // You can now use the Badging API
    }
  }
</script>

iOS and iPadOS will show this system prompt to the user for permission:

A browser permission prompt for asking the user to allow web notifications.

You can also check if a user has previously granted notifications permission by using the Permissions API:

async function checkNotificationPermission() {
  const permissionStatus = await navigator
    .permissions
    .query({ name: 'notifications' });

  switch (permissionStatus.state) {
    case 'granted':
      // You can use the Badging API
      break;
    case 'denied':
      // The user has denied the permission
      break;
    default:
      // The user has not yet granted or denied the permission
      await requestNotificationPermission();
      break;
  }
}
checkNotificationPermission();

Note that even though the user may have granted notifications permission, they remain in control of that permission through iOS or iPadOS Settings. As some users find badges distracting, they might leave the notifications permission enabled, but choose not to show the badge.

To retain user privacy, we never expose this user preference to the web app.

Setting and Clearing a Badge From a Web App

App badges represent the number of items requiring the user’s attention (e.g., “you have 5 unread messages“). What the number means is application dependent.

An application badge icon showing a badge with the number 5.

To update the application icon badge, pass a positive number to the navigator.setAppBadge() method:

async function setBadge(count) {
  if ('setAppBadge' in navigator) {
    try {
      await navigator.setAppBadge(count);
    } catch (error) {
      console.error('Failed to set app badge:', error);
    }
  }
}

// Set the badge count to 5
setBadge(5);

You can clear the badge by using navigator.clearAppBadge().

async function clearBadge() {
  if ('clearAppBadge' in navigator) {
    try {
      await navigator.clearAppBadge();
    } catch (error) {
      console.error('Failed to clear app badge:', error);
    }
  }
}

// Clear the badge
clearBadge();

Alternatively, calling navigator.setAppBadge(0) is equivalent to calling navigator.clearAppBadge().

Using the API From a Service Worker

In addition to being exposed to window objects, the Badging API is exposed in Web Worker contexts.

This makes it particularly useful for apps that support Web Push as it is trivial to update your application badge while your Service Worker handles a push event.

// Function to determine the badge count based on the event data
function determineBadgeCount(data) {
  // Process the data to compute the badge count
}

self.addEventListener('push', (event) => {
  let promises = [];

  if ('setAppBadge' in self.navigator) {
    const badgeCount = determineBadgeCount(event.data);
    // Promise to set the badge
    const promise = self.navigator.setAppBadge(badgeCount);
    promises.push(promise);
  }

  // Promise to show a notification
  promises.push(self.registration.showNotification("You've got mail!"));

  // Finally...
  event.waitUntil(Promise.all(promises));
});

When requesting a push subscription you must promise that all pushes will be user visible events. This means that each push event you process must result in a user visible notification being displayed with a call to self.registration.showNotification()

You are encouraged to update your application badge when handling a push event, but a badge update by itself does not fulfill the “user visible” requirement; Keep showing those notifications!

Security Restrictions for Third-Party

When the user added your web app to their Home Screen, they were indicating their trust in you as a first party.

JavaScript that calls navigator.setAppBadge() in an attempt to update the badge count must come from a frame that is the same-origin as your top-level document. Calls to navigator.setAppBadge() from cross-origin frames have no effect.

Evolving the Badging API at the W3C

We’re collaborating with the W3C Web Applications Working Group and its members, particularly with Microsoft who have partnered with us as co-editors of the Badging API specification. This collaboration allows us to create a more consistent, secure, and privacy-preserving API across browsers and operating systems. The Badging API specification was previously edited by folks from Google. We’re excited to work with the W3C membership to continue its evolution.

April 25, 2023 06:00 PM

April 19, 2023

Release Notes for Safari Technology Preview 168

Surfin’ Safari

Learn about the latest web technology updates in Safari Technology Preview: Web Inspector, CSS, Rendering, Web Animations, JavaScript, Lockdown Mode, Media, Popover, Web API, and Accessibility.

Overview

Safari Technology Preview Release 168 is now available for download for macOS Monterey 12.3 or later and macOS Ventura. If you already have Safari Technology Preview installed, you can update it in the Software Update pane of System Preferences on macOS Monterey, or System Settings under General → Software Update on macOS Ventura.

This release includes WebKit changes between: 262125@main…262761@main.

Web Inspector

  • Network tab
    • Fixed cleared items reappearing with Preserve Log enabled (262603@main)
  • Sources tab
    • Fixed unresponsiveness after inspecting photos (262409@main)
  • Graphics tab
    • Added support for OffscreenCanvas in Canvas related operations (262388@main)

CSS

  • Added CSS Nesting serialization support for CSSOM (262177@main)
  • Added support for @supports font-format() (262305@main)
  • Implemented two-value syntax of font-size-adjust (262309@main, 262374@main)
  • Implemented CSSOM insertRule() on StyleRule (262394@main)
  • Fixed color() function parsing to reject missing components (262098@main)
  • Fixed to not show cursor: pointer on unclickable <area> (262559@main)
  • Fixed UA styles incorrectly applying to elements with other namespaces such as SVG (262053@main)
  • Fixed hit testing for an ::after with transform-style: preserve-3d and a negative z-index (262728@main)
  • Fixed content: counter() not updating visually if position: absolute is set (262269@main)
  • Fixed unknown function parsing in @supports rule (262308@main)
  • Fixed @counter-style to stop allowing redefining certain predefined styles (262038@main)
  • Fixed @counter-style extends system to always extend first symbol for fixed system (262264@main)
  • Fixed triggering layout on margin-trim style change (262423@main)
  • Fixed trimmed block-start, block-end, inline-start, and inline-end margins for flex items in horizontal writing mode to reflect in computed style (262700@main, 262081@main, 262663@main, 262708@main)

Rendering

  • Fixed statically positioned out-of-flow box location when display type changes from block to inline-block (262042@main)
  • Fixed incorrectly positioned out-of-flow box when a layout boundary is present (262470@main)
  • Fixed siblings layout if an adjacent float no longer affects them (262481@main)
  • Fixed vertical-align correctness (262506@main)
  • Fixed MathML element in display: flex not getting repainted on a content change (262674@main)
  • Fixed statically positioned out-of-flow box with anonymous sibling flex item inside display: flex (262341@main)

Web Animations

  • Fixed first frame of transform animation visible when !important style overrides the animated value (262327@main)

JavaScript

  • Added ImplementationVisibility to Wasm::Callee (262191@main)
  • Aligned RegExp V Flags Syntax errors with V8 (262017@main)
  • Applied the new display computation for digital in Intl.DurationFormat (262682@main)
  • Fixed Paren Context allocation and use with Duplicate Named Capture groups (262239@main)

Lockdown Mode

Media

  • Fixed videos not playing if opened in full screen on first play (262654@main)

Popover

  • Fixed event.preventDefault() to not cancel popover light dismiss (262283@main)
  • Fixed changing popover attribute during beforetoggle when showing a popover to throw an exception (262026@main)
  • Implemented “check and possibly close popover stack” algorithm (262440@main)

Web API

  • Implemented URL.canParse() (262072@main)
  • Fixed HTML Comments after </body> placed at the bottom of the <body> contents (262222@main)
  • Fixed innerHTML escaping <, >, &, and nbsp inside noembed, noframes, iframe and plaintext (262285@main)
  • Fixed innerHTML serialization to not have a special handling for javascript: URLs (262267@main)
  • Fixed sliced blobs ending up with the wrong Content-Type in Fetch (262583@main)
  • Fixed Offscreen Canvas not respecting size set via CSS (262039@main)
  • Fixed smooth keyboard scrolling with Page Up and Page Down (262466@main)

Accessibility

  • Changed to not include the password input value in aria-labelledby description (262433@main)
  • Fixed aria-owns attribute for the radio role (262566@main)
  • Fixed VoiceOver not reading entered text in text fields (262126@main)

April 19, 2023 09:17 PM

April 05, 2023

Release Notes for Safari Technology Preview 167

Surfin’ Safari

Safari Technology Preview Release 167 is now available for download for macOS Monterey 12.3 or later and macOS Ventura. If you already have Safari Technology Preview installed, you can update it in the Software Update pane of System Preferences on macOS Monterey, or System Settings under General → Software Update on macOS Ventura.

This release includes WebKit changes between: 261248@main…262124@main.

Web Inspector

  • Elements tab
    • Fixed editing CSS properties inside rules with nested rules causing the inner nested rules to be deleted (261329@main)
    • Fixed filtering styles to also match CSS at-rule identifiers (261804@main)
    • Fixed new CSS property unexpectedly getting dropped from an empty CSS rule (261861@main)
    • Fixed adding a new CSS rule in the Styles sidebar not appearing to add the rule on first attempt (261883@main)
  • Sources tab
    • Added pretty-printing support for various modern JavaScript syntax, including optional chaining, private class members, and optional assignment operators (261748@main)

CSS

  • Implemented CSS text-transform with multiple values (261419@main)
  • Added offsets into shape-outside Shapes (261331@main)
  • Added support for top-level & selector for CSS Nesting (261739@main)
  • Added support for counter() with @counter-style (261985@main)
  • Changed to apply basic font properties as font variation settings (261566@main)
  • Fixed CSS @imports in HTML missing quote marks getting mistakenly hidden from the Preload Scanner (261254@main)
  • Fixed calculating the CSSFilter geometry and clipping (261827@main)
  • Fixed border-image-repeat: round to match other browsers (261903@main)
  • Updated margin box for trimmed block-end boxes in block container and adjust the position of self-collapsing children (261750@main)

JavaScript

  • Added ClassSetCharacter syntax tests for RegExp v flag and fix issues found (261746@main)
  • Added more tests for RegExp v flag and fix issues found (261714@main)
  • Added ProxyObjectHas IC to optimize “has” trap (261628@main)
  • Aligned error message for non-callable ProxyObject’s “get” trap with its counterparts (261627@main)
  • Optimized Function.prototype.bind (261825@main)
  • Optimized Function#bind (261993@main)

WebAssembly

  • Added support for anyref behind flag (261711@main)
  • Implemented cast operations behind flag (261445@main)
  • Implemented eqref and ref.eq behind flag (261663@main)
  • Implemented initial minimal JS API for Wasm GC behind flag (261544@main)

Media

  • Fixed video in picture-to-picture snaps to incorrect size (261383@main)
  • Fixed a page with one document doing capture and another playing correctly handle remote commands (261414@main)
  • Fixed readyState to change back to HAVE_METADATA when removing a sample at the current playtime (261955@main)
  • Fixed SourceBuffer.buffered to return the same object if it’s not modified (261848@main)
  • Fixed video’s readyState incorrectly switching between HAVE_CURRENT_DATA and HAVE_METADATA (262112@main)
  • Skipped initial MSE buffering rate computation (261328@main)

Popover

  • Avoided conflicting interactions in the top layer (261317@main)
  • Made element.togglePopover() more interoperable (261386@main, 261436@main)
  • Implemented popover focusing steps (261400@main)
  • Implemented popovertarget & popovertargetaction attributes (261346@main)
  • Added an exception when calling dialog.show() on an open popover (261351@main)

Rendering

  • Fixed mixed characters in right-to-left SVG text (261495@main)
  • Fixed margin-top getting ignored on elements with zero-height (and clear set) if they appear after floating elements (261926@main)

Web API

  • Added support for Apple Pay in cross-origin iframes with allow=payment attribute (262616@main)
  • Implemented Priority Hints (261689@main)
  • Changed to allow quota to be set based on disk space (261840@main)
  • Changed range.extractContents() to abort early if there’s a doctype in the range (261342@main)
  • Changed to include FetchMetadata on preflight requests (261587@main)
  • Fixed some Scroll To Text Fragment URLs not finding existing text on the page (261302@main)
  • Fixed the <summary> element not focusable with tabindex (261497@main)
  • Fixed problems changing multiple state <select> element to single state (261380@main)
  • Fixed Cross-Origin-Embedder-Policy incorrectly blocking iframes on cache hit (261924@main)
  • Fullscreen window size is incorrect for non-video elements (261904@main)
  • Optimized the HTML parser entity names table by omitting semicolons (261734@main)
  • Implemented the Response.json static method (261960@main)

Accessibility

  • Added support for code ARIA role (261640@main)
  • Implemented ‘generic’ role mapping (261894@main)
  • Fixed standalone spin buttons to be directly incrementable and decrementable (261396@main)
  • Fixed form controls taking the AX text of an ancestor label over their own inner text (261843@main)

MathML

  • Fixed not ignoring out-of-flow boxes while processing invalid MathML content (261841@main)

SVG

  • Fixed text transformation not starting on initial render (261408@main)

April 05, 2023 08:50 PM

April 03, 2023

Carlos García Campos: WebKitGTK accelerated compositing rendering

Igalia WebKit

Initial accelerated compositing support

When accelerated compositing support was added to WebKitGTK, there was only X11. Our first approach was quite simple, we sent the web view widget Xwindow ID to the web process to be used as rendering target using GLX. This was very efficient, but soon we realized it broke the GTK rendering model so it was not possible to use a web view inside a GtkOverlay, for example, to show status messages on top. The solution was to use a redirected Xcomposite window in the web process, and use its ID as the render target using GLX. The pixmap ID of the redirected Xcomposite window was sent to the UI process to be painted in the web view widget using a Cairo Xlib surface. Since the rendering happens in the web process, this approach required to use Xdamage to monitor when the redirected Xcomposite window was updated to schedule a web view redraw.

Wayland support

To support accelerated compositing under Wayland we initially added a nested Wayland compositor running in the UI process. The web process connected to the nested Wayland compositor and created a surface to be used as the rendering target using EGL. The good thing about this approach compared to the X11 one, is that we can create an EGLImage from Wayland buffers and use a GDK GL context to paint the contents in the web view. This is more efficient than X11 because we can use OpenGL both in web and UI processes.
WPE, when using the fdo backend, uses the same approach of running a nested Wayland compositor, but in a more efficient way, using DMABUF instead of Wayland buffers when available. So, we decided to use libwpe in the GTK port only for rendering under Wayland, and eventually remove our Wayland compositor implementation.
Before the removal of the custom Wayland compositor we had all these possible combinations:

  • UI Process
    • X11: Cairo Xlib surface
    • Wayland: EGL
  • Web Process
    • X11: GLX using redirected Xwindow
    • Wayland (nested Wayland compositor): EGL using Wayland surface
    • Wayland (libwpe): EGL using libwpe to get the Wayland surface

To reduce a bit the differences, and to make it easier to support WebGL with ANGLE we decided to change X11 to prefer EGL if possible, falling back to GLX only if EGL failed.

GTK4

GTK4 was released and we added support for it. The fact that GTK4 uses GL by default should make the rendering more efficient in accelerated compositing mode. This is definitely true under Wayland, because we are using a GL context already, so we just keep passing a texture to GTK to paint the contents in the web view. However, in the case of X11 we still have a Cairo Xlib surface that GTK paints into a Cairo image surface to be uploaded to the GPU. With GTK4 now we have two more combinations in the UI process side X11 + GTK3, X11 + GTK4, Wayland + GTK3 and Wayland + GTK4.

Reducing all the combinations to (almost) one: DMABUF

All these combinations to support the different platforms made it quite difficult to maintain, every time we get a bug report about something not working in accelerated compositing mode we have to figure out the combination actually used by the reporter, GTK3 or GTK4? X11 or Wayland? using EGL or GLX? custom Wayland compositor or libwpe? driver? version? etc.

We are already using DMABUF in WebKit for different things like WebGL and media rendering, so we thought that we could also use it for sharing the rendered buffer between the web and UI processes. That would be a more efficient solution but it would also drastically reduce the amount of combinations to maintain. The web process always uses the surfaceless platform, so it doesn’t matter if it’s under Wayland or X11. Then we create a surfaceless context as the render target and use EGL and GBM APIs to export the contents as a DMABUF buffer. The UI process imports the DMABUF buffer using EGL and GBM too, to be passed to GTK as a texture that is painted in the web view.

This theoretically recudes all the previous combinations to just one (note that we removed GLX support entirely, making EGL a requirement for accelerated compositing), but there’s a problem under X11: GTK3 doesn’t support EGL on X11 and GTK4 defaults to EGL but falls back to GLX if it doesn’t find an EGL config that perfectly matches the screen visual. In my system it never finds that EGL config because mesa doesn’t expose any 32 bit depth config. So, in the case of GTK3 we have to manually download the buffer to CPU and paint normally using Cairo, but in the case of GTK4 + GLX, GTK uploads the buffer again to be painted using GLX. I don’t think it’s possible to force GTK to use EGL from the API, but at least you can use GDK_DEBUG=gl-egl.

WebKitGTK 2.41.1

WebKitGTK 2.41.1 is the first unstable release of this cycle and already includes the DMABUF support that is used by default. We encourage everybody to try it out and provide feedback or report any issue. Please, export the contents of webkit://gpu and attach it to the bug report when reporting any problem related to graphics. To check if the issue is a regression of the DMABUF implementation you can use WEBKIT_DISABLE_DMABUF_RENDERER=1 to use the WPE renderer or X11 instead. This environment variable and the WPE render/X11 code will be eventually removed if DMABUF works fine.

WPE

If this approach works fine we plan to use something similar for the WPE port and get rid of the nested Wayland compositor there too.

By carlos garcia campos at April 03, 2023 08:57 AM

March 27, 2023

WebKit Features in Safari 16.4

Surfin’ Safari

Today, we’re thrilled to tell you about the many additions to WebKit that are included in Safari 16.4. This release is packed with 135 new web features and over 280 polish updates. Let’s take a look.

You can experience Safari 16.4 on macOS Ventura, macOS Monterey, macOS Big Sur, iPadOS 16, and iOS 16. Update to Safari 16.4 on macOS Monterey or macOS Big Sur by going to System Preferences → Software Update → More info, and choosing to update Safari. Or update on macOS Ventura, iOS or iPadOS, by going to Settings → General → Software Update.

Web Push on iOS and iPadOS

iOS and iPadOS 16.4 add support for Web Push to web apps added to the Home Screen. Web Push makes it possible for web developers to send push notifications to their users through the use of Push API, Notifications API, and Service Workers.

Deeply integrated with iOS and iPadOS, Web Push notifications from web apps work exactly like notifications from other apps. They show on the Lock Screen, in Notification Center, and on a paired Apple Watch. Focus provides ways for users to precisely configure when or where to receive Web Push notifications — putting users firmly in control of the experience. For more details, read Web Push for Web Apps on iOS and iPadOS.

Improvements for Web Apps

WebKit on iOS and iPadOS 16.4 adds support for the Badging API. It allows web app developers to display an app badge count just like any other app on iOS or iPadOS. Permission for a Home Screen web app to use the Badging API is automatically granted when a user gives permission for notifications.

To support notifications and badging for multiple installs of the same web app, WebKit adds support for the id member of the Web Application Manifest standard. Doing so continues to provide users the convenience of saving multiple copies of a web app, perhaps logged in to different accounts separating work and personal usage — which is especially powerful when combined with the ability to customize Home Screen pages with different sets of apps for each Focus.

iOS and iPadOS 16.4 also add support so that third-party web browsers can offer “Add to Home Screen” in the Share menu. For the details on how browsers can implement support, as well more information about all the improvements to web apps, read Web Push for Web Apps on iOS and iPadOS.

We continue to care deeply about both the needs of a wide-range of web developers and the everyday experience of users. Please keep sending us your ideas and requests. There’s more work to do, and we couldn’t be more excited about where this space is headed.

Web Components

Web Components is a suite of technologies that together make it possible to create reusable custom HTML elements with encapsulated functionality. Safari 16.4 improves support for Web Components with several powerful new capabilities.

Safari 16.4 adds support Declarative Shadow DOM, allowing developers to define shadow DOM without the use of JavaScript. And it adds support for ElementInternals, providing the basis for improved accessibility for web components, while enabling custom elements to participate in forms alongside built-in form elements.

Also, there’s now support for the Imperative Slot API. Slots define where content goes in the template of a custom element. The Imperative Slot API allows developers to specify the assigned node for a slot element in JavaScript for additional flexibility.

CSS

Safari 16.4 adds support for quite a few new CSS properties, values, pseudo-classes and syntaxes. We are proud to be leading the way in several areas to the future of graphic design on the web.

Margin Trim

The margin-trim property can be used to eliminate margins from elements that are abutting their container. For example, imagine we have a section element, and inside it we have content consisting of an h2 headline and several paragraphs. The section is styled as a card, with an off-white background and some padding. Like usual, the headline and paragraphs all have top and bottom margins — which provide space between them. But we actually don’t want a margin above the first headline, or after the last paragraph. Those margins get added to the padding, and create more space than what’s desired.

Here’s an example of a simple card, with headline and paragraph text. On the left is the result. On the right, the same exact layout, but with the container’s padding marked in green, and the children’s margins marked in orange. Note there are margins above the first line of text, and the below the last line of text.

Often web developers handle this situation by removing the top margin on the headline with h2 { margin-block-start: 0 } and the bottom margin on the last paragraph with p:last-child { margin-block-end: 0 } — and hoping for the best. Problems occur, however, when unexpected content is placed in this box. Maybe another instance starts with an h3, and no one wrote code to remove the top margin from that h3. Or a second h2 is written into the text in the middle of the box, and now it’s missing the top margin that it needs.

The margin-trim property allows us to write more robust and flexible code. We can avoid removing margins from individual children, and instead put margin-trim: block on the container.

section {
  margin-trim: block;
}
The same simple content card example, but with margin-trim applied. Note that there is no longer a margin above the first line of text or below the last.

This communicates to the browser: please trim away any margins that butt up against the container. The rule margin-trim: block trims margins in the block direction, while margin-trim: inline trims margins in the inline direction.

Try this demo for yourself in Safari 16.4 or Safari Technology Preview to see the results.

Typography

Safari 16.4 also adds support for the new line height and root line height units, lh and rlh. Now you can set any measurement relative to the line-height. For example, perhaps you’d like to set the margin above and below your paragraphs to match your line-height.

p {
  font-size: 1.4rem;
  line-height: 1.2;
  margin-block: 1lh;
}

The lh unit references the current line-height of an element, while the rlh unit references the root line height — much like em and rem.

Safari 16.4 adds support for font-size-adjust. This CSS property provides a way to preserve the apparent size and readability of text when different fonts are being used. While a web developer can tell the browser to typeset text using a specific font size, the reality is that different fonts will render as different visual sizes. You can especially see this difference when more than one font is used in a single paragraph. In the following demo, the body text is set with a serif font, while the code is typeset in a monospace font — and they do not look to be the same size. The resulting differences in x-height can be quite disruptive to reading. The demo also provides a range of font fallback options for different operating systems, which introduces even more complexity. Sometimes the monospace font is bigger than the body text, and other times it’s smaller, depending on which font family is actually used. The font-size-adjust property gives web developers a solution to this problem. In this case, we simply write code { font-size-adjust: 0.47; } to ask the browser to adjust the size of the code font to match the actual glyph size of the body font.

Open this demo in Safari 16.4, Safari Technology Preview or Firefox to see font-size-adjust in action.

To round out support for the font size keywords, font-size: xxx-large is now supported in Safari 16.4.

Pseudo-classes

Safari 16.4 also adds support for several new pseudo-classes. Targeting a particular text direction, the :dir() pseudo-class lets you define styles depending on whether the language’s script flows ltr (left-to-right) or rtl (right-to-left). For example, perhaps you want to rotate a logo image a bit to the left or right, depending on the text direction:

img:dir(ltr) { rotate: -30deg; }
img:dir(rtl) { rotate: 30deg; }

Along with unprefixing the Fullscreen API (see below), the CSS :fullscreen pseudo-class is also now unprefixed. And in Safari 16.4, the :modal pseudo-class also matches fullscreen elements.

Safari 16.4 adds :has() support for the :lang pseudo-class, making it possible to style any part of a page when a particular language is being used on that page. In addition, the following media pseudo-classes now work dynamically inside of :has(), opening up a world of possibilities for styling when audio and video are in different states of being played or manipulated — :playing, :paused, :seeking, :buffering, :stalled, :picture-in-picture, :volume-locked, and :muted. To learn more about :has(), read Using :has() as a CSS Parent Selector and much more.

Color

Safari 16.4 adds support for Relative Color Syntax. It provides a way to specify a color value in a much more dynamic fashion. Perhaps you want to use a hexadecimal value for blue, but make that color translucent — passing it into the hsl color space to do the calculation.

section { background: hsl(from #1357a6 h s l / 0.5); }

Or maybe you want to define a color as a variable, and then adjust that color using a mathematical formula in the lch color space, telling it to cut the lightness (l) in half with calc(l / 2), while keeping the chroma (c) and hue (h) the same.

:root { 
    --color: green; 
}
.component {
    --darker-accent: lch(from var(--color) calc(l / 2) c h);
}

Relative Color Syntax is powerful. Originally appearing in Safari Technology Preview 122 in Feb 2021, we’ve been waiting for the CSS Working Group to complete its work so we could ship. There isn’t documentation on MDN or Can I Use about Relative Color Syntax yet, but likely will be soon. Meanwhile the Color 5 specification is the place to learn all about it.

Last December, Safari 16.2 added support for color-mix(). Another new way to specify a color value, the functional notation of color-mix makes it possible to tell a browser to mix two different colors together, using a certain color space.

Safari 16.4 adds support for using currentColor with color-mix(). For example, let’s say we want to grab whatever the current text color might be, and mix 50% of it with white to use as a hover color. And we want the mathematical calculations of the mixing to happen in the oklab color space. We can do exactly that with:

:hover {
    color: color-mix(in oklab, currentColor 50%, white);
}

Safari 16.2 also added support for Gradient Interpolation Color Spaces last December. It allows the interpolation math of gradients — the method of determining intermediate color values — to happen across different color spaces. This illustration shows the differences between the default sRGB interpolation compared to interpolation in lab and lch color spaces:

Safari 16.4 adds support for the new system color keywords. Think of them as variables which represent the default colors established by the user, browser, or OS — defaults that change depending on whether the system is set to light mode, dark mode, high contrast mode, etc. For instance, Canvas represents the current default background color of the HTML page. Use system color keywords just like other named colors in CSS. For example, h4 { color: FieldText; } will style h4 headlines to match the default color of text inside form fields. When a user switches from light to dark mode, the h4 color will automatically change as well. Find the full list of system colors in CSS Color level 4.

Media Queries Syntax Improvements

Safari 16.4 adds support for the syntax improvements from Media Queries level 4. Range syntax provides an alternative way to write out a range of values for width or height. For example, if you want to define styles that are applied when the browser viewport is between 400 and 900 pixels wide, in the original Media Query syntax, you would have written:

@media (min-width: 400px) and (max-width: 900px) {
  ...
}

Now with the new syntax from Media Queries level 4, you can instead write:

@media (400px <= width < 900px) {
  ...
}

This is the same range syntax that’s been part of Container Queries from its beginning, which shipped in Safari 16.0.

Media Queries level 4 also brings more understandable syntax for combining queries using boolean logic with and, not, and or. For example:

@media (min-width: 40em), (min-height: 20em) {
  @media not all and (pointer: none) { 
    ... 
  }
}

Can instead be greatly simplified as:

@media ((min-width: 40em) or (min-height: 20em)) and (not (pointer: none)) {
  ... 
}

Or, along with the range syntax changes, as:

@media ((40em < width) or (20em < height)) and (not (pointer: none)) {
  ...
}

Custom Properties

Safari 16.4 adds support for CSS Properties and Values API with support for the @property at-rule. It greatly extends the capabilities of CSS variables by allowing developers to specify the syntax of the variable, the inheritance behavior, and the variable initial value — similar to how browser engines define CSS properties.

@property --size {
  syntax: "<length>";
  initial-value: 0px;
  inherits: false;
}

With @property support, developers can to do things in CSS that were impossible before, like animate gradients or specific parts of transforms.

Web Animations

Safari 16.4 includes some additional improvements for web animations. You can animate custom properties. Animating the blending of mismatched filter lists is now supported. And Safari now supports KeyframeEffect.iterationComposite.

Outline + Border Radius

Until now, if a web developer styled an element that had an outline with a custom outline-style, and that element had curved corners, the outline would not follow the curve in Safari. Now in Safari 16.4, outline always follows the curve of border-radius.

CSS Typed OM

Safari 16.4 adds support for CSS Typed OM, which can be used to expose CSS values as typed JavaScript objects. Input validation for CSSColorValues is also supported as part of CSS Typed OM. Support for Constructible and Adoptable CSSStyleSheet objects also comes to Safari 16.4.

HTML

Safari 16.4 now supports lazy loading iframes with loading="lazy". You might put it on a video embed iframe, for example, to let the browser know if this element is offscreen, it doesn’t need to load until the user is about to scroll it into view.

<iframe src="videoplayer.html" title="This Video" 
        loading="lazy" width="640" height="360" ></iframe>

By the way, you should always include the height and width attributes on iframes, so browsers can reserve space in the layout for it before the iframe has loaded. If you resize the iframe with CSS, be sure to define both width and height in your CSS. You can also use the aspect-ratio property to make sure an iframe keeps it’s shape as it’s resized by CSS.

iframe { 
    width: 100%; 
    height: auto; 
    aspect-ratio: 16 / 9; 
}

Now in Safari 16.4, a gray line no longer appears to mark the space where a lazy-loaded image will appear once it’s been loaded.

Safari 16.4 also includes two improvements for <input type="file">. Now a thumbnail of a selected file will appear on macOS. And the cancel event is supported.

JavaScript and WebAssembly

Safari 16.4 brings a number of useful new additions for developers in JavaScript and WebAssembly.

RegExp Lookbehind makes it possible to write Regular Expressions that check what’s before your regexp match. For example, match patterns like (?<=foo)bar matches bar only when there is a foo before it. It works for both positive and negative lookbehind.

JavaScript Import Maps give web developers the same sort of versioned file mapping used in other module systems, without the need for a build step.

Growable SharedArrayBuffer provided a more efficient mechanism for growing an existing buffer for generic raw binary data. And resizable ArrayBuffer allows for resizing of a byte array in JavaScript.

In WebAssembly, we’ve added support for 128-bit SIMD.

Safari 16.4 also includes:

  • Array.fromAsync
  • Array#group and Array#groupToMap
  • Atomics.waitAsync
  • import.meta.resolve()
  • Intl.DurationFormat
  • String#isWellFormed and String#toWellFormed
  • class static initialization blocks
  • Symbols in WeakMap and WeakSet

Web API

Safari 16.4 adds support for quite a few new Web API. We prioritized the features you’ve told us you need most.

Offscreen Canvas

When using Canvas, the rendering, animation, and user interaction usually happens on the main execution thread of a web application. Offscreen Canvas provides a canvas that can be rendered off screen, decoupling the DOM and the Canvas API so that the <canvas> element is no longer entirely dependent on the DOM. Rendering can now also be transferred to a worker context, allowing developers to run tasks in a separate thread and avoid heavy work on the main thread that can negatively impact the user experience. The combination of DOM-independent operations and rendering of the main thread can provide a significantly better experience for users, especially on low-power devices. In Safari 16.4 we’ve added Offscreen Canvas support for 2D operations. Support for 3D in Offscreen Canvas is in development.

Fullscreen API

Safari 16.4 now supports the updated and unprefixed Fullscreen API on macOS and iPadOS. Fullscreen API provides a way to present a DOM element’s content so that it fills the user’s entire screen, and to exit fullscreen mode once it’s unneeded. The user is given control over exiting fullscreen mode through various mechanisms, include pressing the ‘Esc’ key on the keyboard, or performing a downwards gesture on touch-enabled devices. This ensures that the user always has the ability to exit fullscreen whenever they desire, preserving their control over the browsing experience.

Screen Orientation API

Along with the Fullscreen API we’ve added preliminary support for Screen Orientation API in Safari 16.4, including:

  • ScreenOrientation.prototype.type returns the screen’s current orientation.
  • ScreenOrientation.prototype.angle returns the screen’s current orientation angle.
  • ScreenOrientation.prototype.onchange event handler, which fires whenever the screen changes orientation.

Support for the lock() and unlock() methods remain experimental features for the time being. If you’d like to try them out, you can enable them in the Settings app on iOS and iPadOS 16.4 via Safari → Advanced → Experimental Features → Screen Orientation API (Locking / Unlocking).

Screen Wake Lock API

The Screen Wake Lock API provides a mechanism to prevent devices from dimming or locking the screen. The API is useful for any application that requires the screen to stay on for an extended period of time to provide uninterrupted user experience, such as a cooking site, or for displaying a QR code.

User Activation API

User Activation API provides web developers with a means to check whether a user meaningfully interacted with a web page. This is useful as some APIs require meaningful “user activation”, such as, a click or touch, before they can be used. Because user activation is based on a timer, the API can be used to check if document currently has user activation as otherwise a call to an API would fail. Read The User Activation API for more details and usage examples.

WebGL Canvas Wide Gamut Color

WebGL canvas now supports the display-p3 wide-gamut color space. To learn more about color space support, read Improving Color on the Web, Wide Gamut Color in CSS with Display-P3, and Wide Gamut 2D Graphics using HTML Canvas.

Compression Streams API

Compression Streams API allows for compressing and decompressing streams of data in directly in the browser, reducing the need for a third-party JavaScript compression library. This is handy if you need to “gzip” a stream of data to send to a server or to save on the user’s device.

And more

Safari 16.4 also includes many other new Web API features, including:

  • Reporting API
  • Notification API in dedicated workers
  • Permissions API for dedicated workers
  • Service Workers and Shared Workers to the Permissions API
  • gamepad.vibrationActuator
  • A submitter parameter in the FormData constructor
  • COEP violation reporting
  • COOP/COEP navigation violation reporting
  • Fetch Initiator
  • Fetch Metadata Request Headers
  • importing compressed EC keys in WebCrypto
  • loading scripts for nested workers
  • non-autofill credential type for the autocomplete attribute
  • revoking Blob URLs across same-origin contexts
  • isComposing attribute on InputEvent
  • termination of nested workers
  • transfer size metrics for first parties in ServerTiming and PerformanceResourceTiming
  • KeyframeEffect.iterationComposite
  • WEBGL_clip_cull_distance

Images, Video, and Audio

Last fall, Safari 16 brought support for AVIF images to iOS 16, iPadOS 16 and macOS Ventura. Now with Safari 16.4, AVIF is also supported on macOS Monterey and macOS Big Sur. Updates to our AVIF implementation ensure animated images and images with film grain (noise synthesis) are now fully supported, and that AVIF works inside the <picture> element. We’ve also updated our AVIF implementation to be more lenient in accepting and displaying images that don’t properly conform to the AVIF standard.

Safari 16.4 adds support for the video portion of Web Codecs API. This gives web developers complete control over how media is processed by providing low-level access to the individual frames of a video stream. It’s especially useful for applications that do video editing, video conferencing, or other real-time processing of video.

Media features new to Safari 16.4 also include:

  • Improvements to audio quality for web video conferencing
  • Support for a subset of the AudioSession Web API
  • Support for AVCapture virtual cameras
  • Support for inbound rtp trackIdentifier stat field
  • Support for VTT-based extended audio descriptions
  • Support to allow a site to provide an “alternate” URL to be used during AirPlay

WKWebView

WKPreferences, used by WKWebView on iOS and iPadOS 16.4, adds a new shouldPrintBackgrounds API that allows clients to opt-in to including a pages’s background when printing.

Developer Tooling

Inspectable WebKit and JavaScriptCore API

Across all platforms supporting WKWebView or JSContext, a new property is available called isInspectable (inspectable in Objective-C) on macOS 13.4 and iOS, iPadOS, and tvOS 16.4. It defaults to false, and you can set it to true to opt-in to content being inspectable using Web Inspector, even in release builds of apps.

Develop Menu > Patrick's iPhone > Example App

When an app has enabled inspection, it can be inspected from Safari’s Develop menu in the submenu for either the current computer or an attached device. For iOS and iPadOS, you must also have enabled Web Inspector in the Settings app under Safari > Advanced > Web Inspector.

To learn more, read Enabling the Inspection of Web Content in Apps.

WebDriver

When automating Safari 16.4 with safaridriver, we now supports commands for getting elements inside shadow roots, as well as accessibility commands for getting the computed role and label of elements. When adding a cookie with safaridriver, the SameSite attribute is now supported. Improvements have also been made to performing keyboard actions, including better support for modifier keys behind held and support for typing characters represented by multiple code points, including emoji. These improvements make writing cross-browser tests for your website even easier.

Web Inspector

Typography Tooling

Web Inspector in Safari 16.4 adds new typography inspection capabilities in the Fonts details sidebar of the Elements Tab.

Warnings are now shown for synthesized bold and oblique when the rendering engine has to generate these styles for a font that doesn’t provide a suitable style. This may be an indicator that the font file for a declared @font-face was not loaded. Or it may be that the specific value for font-weight or font-style isn’t supported by the used font.

A variable font is a font format that contains instructions on how to generate, from a single file, multiple style variations, such as weight, stretch, slant, optical sizing, and others. Some variable fonts allow for a lot of fine-tuning of their appearance, like the stroke thickness, the ascender height or descender depth, and even the curves or roundness of particular glyphs. These characteristics are expressed as variation axes and they each have a custom value range defined by the type designer.

The Fonts details sidebar now provides interactive controls to adjust values of variation axes exposed by a variable font and see the results live on the inspected page allowing you to get the font style that’s exactly right for you.

Tooling for Conditionals in CSS

The controls under the new User Preference Overrides popover in the Elements Tab allow you to emulate the states of media features like prefers-reduced-motion and prefers-contrast to ensure that the web content you create adapts to the user’s needs. The toggle to emulate the states of prefers-color-scheme, which was previously a standalone button, has moved to this new popover.

The Styles panel of the Elements Tab now allows editing the condition text for @media, @container and @supports CSS rules. This allows you to make adjustments in-context and immediately see the results on the inspected page. Here’s a quick tip: edit the condition of @supports to its inverse, like @supports not (display: grid), to quickly check your progressive enhancement approach to styling and layout.

Badging HTML Elements

New badges for elements in the DOM tree of the Elements Tab join the existing badges for Grid and Flex containers. The new Scroll badge calls out scrollable elements, and the new Events badge provides quick access to the event listeners associated with the element when clicked. And a new Badges toolbar item makes it easy to show just the badges you are interested in and hide others.

And more

Changes to Web Inspector in Safari 16.4 also include:

  • Elements Tab: Improved visual hierarchy of the Layout sidebar.
  • Elements Tab: Added support for nodes that aren’t visible on the page to appear dimmed in the DOM tree.
  • Console Tab: Added support for console snippets.
  • Sources Tab: Added showing relevant special breakpoints in the Pause Reason section.
  • Sources Tab: Added support for inline breakpoints.
  • Sources Tab: Added support for symbolic breakpoints
  • Network Tab: Added a Path column.
  • Network Tab: Added alphabetic sorting of headers.
  • Network Tab: Added support for per-page network throttling.
  • Network Tab: Added using the Shift key to highlight the initiator or initiated resources.
  • Graphics Tab: Added OpenGL object IDs in the Canvas inspector.
  • Settings Tab: Added a setting to turn off dimming nodes that aren’t visible on the page.
  • Added support for function breakpoints and tracepoints.

Safari Web Extensions

Enhancements to Declarative Net Request

Safari is always working on improving support for declarativeNetRequest, the declarative way for web extensions to block and modify network requests. In Safari 16.4, several enhancements have been added to the API:

  • The declarativeNetRequest.setExtensionActionOptions API can be used to configure whether to automatically display the action count (number of blocked loads, etc.) as the extension’s badge text.
  • The modifyHeaders action type has been added to rewrite request and response headers. This action requires granted website permissions for the affected domains and the declarativeNetRequestWithHostAccess permission in the manifest.
  • The redirect action type now requires the declarativeNetRequestWithHostAccess permission in the manifest.
  • The requestDomains condition has been added to only match requests when the domain matches one from the array. If the condition is omitted, the rule is applied to requests from all domains.
  • The MAX_NUMBER_OF_DYNAMIC_AND_SESSION_RULES property has been added to check the maximum number of combined dynamic and session rules an extension can add. The current limit is set at 5,000 rules.

These enhancements give developers more options to customize their content blocking extensions and provide users with better privacy protection.

SVG Icon Support in Web Extensions

Safari 16.4 now supports SVG images as extension and action icons, giving developers more options for creating high-quality extensions. This support brings Safari in line with Firefox, allowing for consistent experiences across platforms. The ability to scale vector icons appropriately for any device means developers no longer need multiple sizes, simplifying the process of creating polished and professional-looking extensions.

Dynamic Content Scripts

Safari 16.4 introduces support for the new scripting.registerContentScript API, which enables developers to create dynamic content scripts that can be registered, updated, or removed programmatically. This API augments the static content scripts declared in the extension manifest, providing developers with more flexibility in managing content scripts and enabling them to create more advanced features for their extensions.

Toggle Reader Mode

The tabs.toggleReaderMode API has been added to Safari 16.4, which enables extensions to toggle Reader Mode for any tab. This function is particularly useful for extensions that want to enhance the user’s browsing experience by allowing them to focus on the content they want to read. By using this API, developers can create extensions that automate the process of enabling Reader Mode for articles, making it easier and more convenient for users to read online content.

Session Storage

The storage.session API, now supported in Safari 16.4, enables extensions to store data in memory for the duration of the browser session, making it a useful tool for storing data that takes a long time to compute or is needed quickly between non-persistent background page loads. This API is particularly useful for storing sensitive or security-related data, such as decryption keys or authentication tokens, that would be inappropriate to store in local storage. The session storage area is not persisted to disk and is cleared when Safari quits, providing enhanced security and privacy for users.

Background Modules

Developers can now take advantage of modules in background service workers and pages by setting "type": "module" in the background section of the manifest. This allows for more organized and maintainable extension code, making it easier to manage complex codebases. By setting this option, background scripts will be loaded as ES modules, enabling the use of import statements to load dependencies and use the latest JavaScript language features.

Safari Content Blockers

Safari 16.4 has added support for :has() selectors in Safari Content Blocker rules. This is a powerful new addition to the declarative content blocking capabilities of Safari, as it allows developers to select and hide parent elements that contain certain child elements. Its inclusion in Safari Content Blocker rules opens up a whole new range of possibilities for content blocking. Now developers can create more nuanced and precise rules that can target specific parts of a web page, making it easier to block unwanted content while preserving the user’s browsing experience. This is yet another example of Safari’s commitment to providing a secure and private browsing experience for its users while also offering developers the tools they need to create innovative and effective extensions.

New Restrictions in Lockdown Mode

Lockdown Mode is an optional, extreme protection that’s designed for the very few individuals who, because of who they are or what they do, might be personally targeted by some of the most sophisticated digital threats. Most people are never targeted by attacks of this nature.

If a user chooses to enable Lockdown mode on iOS 16.4, iPadOS 16.4, or macOS Ventura 13.3, Safari will now:

  • Disable binary fonts in the CSS Font Loading API
  • Disable Cache API
  • Disable CacheStorage API
  • Disable ServiceWorkers
  • Disable SVG fonts
  • Disable the WebLocks API
  • Disable WebSpeech API

More Improvements

Safari 16.4 now supports dark mode for plain text files. It has support for smooth key-driven scrolling on macOS. And it adds prevention of redirects to data: or about: URLs.

Bug Fixes

In addition to the 135 new features, WebKit for Safari 16.4 includes an incredible amount work polishing existing features. We’ve heard from you that you want to know more about the many fixes going into each release of Safari. We’ve done our best to list everything that might be of interest to developers, in this case, 280 of those improvements:

CSS

  • Fixed -webkit-mask-box-image: initial to set the correct initial value.
  • Fixed -webkit-radial-gradient parsing accidentally treating several mandatory commas as optional.
  • Fixed ::placeholder to not support writing-mode, direction, or text-orientation.
  • Fixed @supports to not work if not, or, or and isn’t followed by a space.
  • Fixed background-repeat not getting correctly exposed through inline styles.
  • Fixed baseline-shift to allow length or percentage, but not numbers.
  • Fixed contain: inline-size for replaced elements.
  • Fixed CSSPerspective.toMatrix() to throw a TypeError if its length is incompatible with the px unit.
  • Fixed cx, cy, x, and y CSS properties to allow length or percentage, but not numbers.
  • Fixed filter: blur on an absolutely positioned image losing overflow: hidden.
  • Fixed font-face to accept ranges in reverse order, and reverse them for computed styles.
  • Fixed font-style: oblique must allow angles equal to 90deg or -90deg.
  • Fixed font-style: oblique with calc() to allow out-of-range angles and clamp them for computed style.
  • Fixed font-weight to clamp to 1 as a minimum.
  • Fixed font shorthand to reject out-of-range angles for font-style.
  • Fixed font shorthand to reset more longhand properties.
  • Fixed overflow-x: clip causing a sibling image to not load.
  • Fixed overflow: clip not working on SVG elements.
  • Fixed stroke-dasharray parsing to align with standards.
  • Fixed stroke-width and stroke-dashoffset parsing to align with standards.
  • Fixed text-decoration-thickness property not repainting when changed.
  • Fixed allowing calc() that combines percentages and lengths for line-height.
  • Fixed an issue where using box-sizing: border-box causes the calculated aspect-ratio to create negative content sizes.
  • Fixed an issue with a monospace font on a parent causing children with a sans-serif font using rem or rlh units to grow to a larger size.
  • Fixed behavior of cursor: auto over links.
  • Fixed buttons with auto width and height to not set intrinsic margins.
  • Fixed calculating block size to use the correct box-sizing with aspect ratio.
  • Fixed cells overflowing their contents when a table cell has inline children which change writing-mode.
  • Fixed clipping perspective calc() values to 0.
  • Fixed font shorthand to not reject values that happen to have CSS-wide keywords as non-first identifiers in a font family name.
  • Fixed hit testing for double-click selection on overflowing inline content.
  • Fixed honoring the content block size minimum for a <fieldset> element with aspect-ratio applied.
  • Fixed incorrectly positioned line break in contenteditable with tabs.
  • Fixed invalidation for class names within :nth-child() selector lists.
  • Fixed omitting the normal value for line-height from the font shorthand in the specified style, not just the computed style.
  • Fixed pseudo-elements to not be treated as ASCII case-insensitive.
  • Fixed rejecting a selector argument for :nth-of-type or :nth-last-of-type.
  • Fixed serialization order for contain.
  • Fixed strings not wrapped at zero width spaces when word-break: keep-all is set.
  • Fixed supporting <string> as an unprefixed keyframe name.
  • Fixed the :has() pseudo-selector parsing to be unforgiving.
  • Fixed the font-face src descriptor format to allow only specified formats, others are a parse error.
  • Fixed the tz component not accounting for zoom when creating a matrix3d() value.
  • Fixed the computed value for stroke-dasharray to be in px.
  • Fixed the effect of the writing-mode property not getting removed when the property is removed from the root element.
  • Fixed the position of text-shadow used with text-combine-upright.
  • Fixed the title of a style element with an invalid type to never be added to preferred stylesheet set.
  • Fixed the transferred min/max sizes to be constrained by defined sizes for aspect ratio.
  • Fixed the user-agent stylesheet to align hidden elements, abbr, acronym, marquee, and fieldset with HTML specifications.
  • Fixed to always use percentages for computed values of font-stretch, never keywords.
  • Fixed to not require whitespace between of and the selector list in :nth-child or :nth-last-child.

CSS API

  • Fixed CSS.supports returning false for custom properties.
  • Fixed CSS.supports whitespace handling with !important.
  • Fixed forgiving selectors to not be reported as supported with CSS.supports("selector(...)").
  • Fixed getComputedStyle() to return a function list for the transform property.
  • Fixed linear-gradient keyword values not getting converted to their rgb() equivalents for getComputedStyle().

Content Security Policy

  • Fixed updating the Content Security Policy when a new header is sent as part of a 304 response.

Forms

  • Fixed <input type="submit">, <input type="reset">, and <input type="button"> to honor font-size, padding, height, and work with multi-line values.
  • Fixed firing the change event for <input type="file"> when a different file with the same name is selected.
  • Fixed preventing a disabled <fieldset> element from getting focus.
  • Fixed the :out-of-range pseudo class matching for empty input[type=number].

JavaScript

  • Fixed Array.prototype.indexOf constant-folding to account for a non-numeric index.
  • Fixed Intl.NumberFormat useGrouping handling to match updated specs.
  • Fixed Intl.NumberFormat ignoring maximumFractionDigits with compact notation.
  • Fixed String.prototype.includes incorrectly returning false when the string is empty and the position is past end of the string.
  • Fixed toLocaleLowerCase and toLocaleUpperCase to throw an exception on an empty string.

HTML

  • Fixed aligning the parsing of <body link vlink alink> to follow standards.
  • Fixed <legend> to accept more display property values than display: block.

Intelligent Tracking Prevention

  • Fixed user initiated cross-domain link navigations getting counted as Top Frame Redirects.

Images

  • Fixed some display issues with HDR AVIF images.
  • Fixed the accept header to correctly indicate AVIF support.

Lockdown Mode

  • Fixed common cases of missing glyphs due to custom icon fonts.

Media

  • Fixed enumerateDevices may return filtered devices even if page is capturing.
  • Fixed MediaRecorder.stop() firing an additional dataavailable event with bytes after MediaRecorder.pause().
  • Fixed duplicate timeupdate events.
  • Fixed limiting DOMAudioSession to third-party iframes with microphone access.
  • Fixed MSE to not seek with no seekable range.
  • Fixed mute microphone capture if capture fails to start because microphone is used by a high priority application.
  • Fixed not allowing text selection to start on an HTMLMediaElement.
  • Fixed only requiring a transient user activation for Web Audio rendering.
  • Fixed screen capture to fail gracefully if the window or screen selection takes too long.
  • Fixed switching to alternate <source> element for AirPlay when necessary.
  • Fixed the local WebRTC video element pausing after bluetooth audioinput is disconnected.
  • Fixed trying to use low latency for WebRTC HEVC encoder when available.
  • Fixed unmuting a TikTok video pauses it.
  • Fixed WebVTT styles not applied with in-band tracks.

Rendering

  • Ensured negative letter-spacing does not pull content outside of the inline box
  • Fixed <div> with border-radius not painted correctly while using jQuery’s .slideToggle().
  • Fixed border-radius clipping on composited layers.
  • Fixed box-shadow to paint correctly on inline elements.
  • Fixed box-shadow invalidation on inline boxes.
  • Fixed calculating the width of an inline text box using simplified measuring to handle fonts with Zero Width Joiner, Zero Width Non-Joner, or Zero Width No-Break Space.
  • Fixed clearing floats added dynamically to previous siblings.
  • Fixed clipping the source image when the source rectangle is outside of the source image in canvas.
  • Fixed CSS keyframes names to not allow CSS wide keywords.
  • Fixed elements with negative margins not avoiding floats when appropriate.
  • Fixed floating boxes overlapping with their margin boxes.
  • Fixed HTMLImageElement width and height to update layout to return styled dimensions not the image attributes.
  • Fixed ignoring nowrap on <td nowrap="nowrap"> when an absolute width is specified.
  • Fixed incorrect clipping when a layer is present between the column and the content layer.
  • Fixed incorrect static position of absolute positioned elements inside relative positioned containers.
  • Fixed layout for fixed position elements relative to a transformed container.
  • Fixed layout overflow rectangle overflows interfering with the scrollbar.
  • Fixed negative shadow repaint issue.
  • Fixed preventing a focus ring from being painted for anonymous block continuations.
  • Fixed recalculating intrinsic widths in the old containing block chain when an object goes out of flow.
  • Fixed rendering extreme border-radius values.
  • Fixed specified hue interpolation method for hues less than 0 or greater than 360.
  • Fixed tab handling in right-to-left editing.
  • Fixed text selection on flex and grid box items.
  • Fixed the position and thickness of underlines to be device pixel aligned.
  • Fixed transforms for table sections.
  • Fixed transition ellipsis box from “being a display box on the line” to “being an attachment” of the line box.
  • Fixed unexpected overlapping selection with tab in right-to-left context.
  • Fixed updating table rows during simplified layout.
  • Fixed: improved balancing for border, padding, and empty block content.

Safari Web Extensions

  • Extensions that request the unlimitedStorage permission no longer need to also request storage.
  • Fixed browser.declarativeNetRequest namespace is now available when an extension has the declarativeNetRequestWithHostAccess permission.
  • Fixed isUrlFilterCaseSensitive declarativeNetRequest rule condition to be false by default.
  • Fixed tabs.onUpdated getting called on tabs that were already closed.
  • Fixed background service worker failing to import scripts.
  • Fixed content scripts not injecting into subframes when extension accesses the page after a navigation.
  • Fixed CORS issue when doing fetch requests from a background service worker.
  • Fixed declarativeNetRequest errors not appearing correctly in the extension’s pane of Safari Settings.
  • Fixed display of extension cookie storage in Web Inspector. Now the extension name is shown instead of a UUID.
  • Fixed declarativeNetRequest rules not loading when an extension is turned off and then on.
  • Fixed result of getMatchedRules() to match other browsers.
  • Fixed browser.webNavigation events firing for hosts where the extension did not have access.
  • Removed Keyboard Shortcut conflict warnings for browser.commands when there are multiple commands without keyboard shortcuts assigned.

Scrolling

  • Fixed overscroll-behavior: none to prevent overscroll when the page is too small to scroll.

SVG

  • Fixed <svg:text> to not auto-wrap.
  • Fixed preserveAspectRatio to stop accepting defer.
  • Fixed SVG.currentScale to only set the page zoom for a standalone SVG.
  • Fixed svgElement.setCurrentTime to restrict floats to finite values.
  • Fixed applying changes to fill with currentColor to other colors via CSS.
  • Fixed changes to the filter property getting ignored.
  • Fixed CSS and SVG filters resulting in a low quality, pixelated image.
  • Fixed focusability even when tab-to-links is enabled for <svg:a>.
  • Fixed handling animation freezes when repeatDur is not a multiple of dur.
  • Fixed making sure computed values for baseline-shift CSS property use px unit for lengths.

Tables

  • Fixed not forcing display: table-cell, display: inline-table, display: table, and float: none on table cell elements when in quirks mode.
  • Fixed removing the visual border when the table border attribute is removed.

Text

  • Fixed font-optical-sizing: auto having no effect in Safari 16.
  • Fixed directionality of the <bdi> and <input> elements to align with HTML specifications.
  • Fixed handling an invalid dir attribute to not affect directionality.
  • Fixed the default oblique angle from 20deg to 14deg.
  • Fixed the handling of <bdo>.
  • Fixed the order of how @font-palette-values override-colors are applied.

Web Animations

  • Fixed @keyframes rules using an inherit value to update the resolved value when the parent style changes.
  • Fixed Animation.commitStyles() triggering a mutation even when the styles are unchanged.
  • Fixed Animation.startTime and Animation.currentTime setters support for CSSNumberish values.
  • Fixed baseline-shift animation.
  • Fixed baselineShift inherited changes.
  • Fixed commitStyles() failing to commit a relative line-height value.
  • Fixed getKeyframes() serialization of CSS values for an onkeyframe sequence.
  • Fixed rotate: x and transform: rotate(x) to yield the same behavior with SVGs.
  • Fixed word-spacing to support animating between percentage and fixed values.
  • Fixed accounting for non-inherited CSS variables getting interpolated for standard properties on the same element.
  • Fixed accumulating and clamping filter values when blending with "none".
  • Fixed accumulation support for the filter property.
  • Fixed additivity support for the filter property.
  • Fixed animation of color list custom properties with iterationComposite.
  • Fixed blend transform when iterationComposite is set to accumulate.
  • Fixed blending to account for iterationComposite.
  • Fixed Calculating computed keyframes for shorthand properties.
  • Fixed composite animations to compute blended additive or accumulative keyframes for in-between keyframes.
  • Fixed computing the keyTimes index correctly for discrete values animations.
  • Fixed CSS animations participation in the cascade.
  • Fixed custom properties to support interpolation with a single keyframe.
  • Fixed filter values containing a url() should animate discretely.
  • Fixed interpolating custom properties to take iterationComposite into account.
  • Fixed jittering when animating a rotated image.
  • Fixed keyframes to be recomputed if a custom property registration changes.
  • Fixed keyframes to be recomputed if the CSS variable used is changed.
  • Fixed keyframes to be recomputed when bolder or lighter is used on a font-weight property.
  • Fixed keyframes to be recomputed when a parent element changes value for a custom property set to inherit.
  • Fixed keyframes to be recomputed when a parent element changes value for a non-inherited property set to inherit.
  • Fixed keyframes to be recomputed when the currentcolor value is used on a custom property.
  • Fixed keyframes to be recomputed when the currentcolor value is used.
  • Fixed opacity to use unclamped values for from and to keyframes with iterationComposite.
  • Fixed running a transition on an inherited CSS variable getting reflected on a standard property using that variable as a value.
  • Fixed seamlessly updating the playback rate of an animation.
  • Fixed setting iterationComposite should invalidate the effect.
  • Fixed setting the transition-property to none does not disassociate the CSS Transition from owning the element.
  • Fixed the composite operation of implicit keyframes for CSS Animations to return "replace".
  • Fixed the timing model for updating animations and sending events.
  • Fixed updating timing to invalidate the effect.

Web API

  • Fixed -webkit-user-select: none allowing text to be copied to clipboard.
  • Fixed contentEditable caret getting left aligned instead of centered when the :before pseudo-element is used.
  • Fixed Cross-Origin-Embedder-Policy incorrectly blocking scripts on cache hit.
  • Fixed CSSRule.type to not return values greater than 15.
  • Fixed document.open() to abort all loads when the document is navigating.
  • Fixed document.open() to remove the initial about:blank-ness of the document.
  • Fixed Element.querySelectorAll not obeying element scope with ID.
  • Fixed FileSystemSyncAccessHandle write operation to be quota protected.
  • Fixed getBoundingClientRect() returning the wrong value for <tr>, <td>, and its descendants for a vertical table.
  • Fixed HTMLOutputElement.htmlFor to make it settable.
  • Fixed queryCommandValue("stylewithcss") to always return an empty string.
  • Fixed StorageEvent.initStorageEvent() to align with HTML specifications.
  • Fixed textContent leaving dir=auto content in the wrong direction.
  • Fixed -webkit-user-select: initial content within -webkit-user-select: none should be copied
  • Fixed WorkerGlobalScope.isSecureContext to be based on the owner’s top URL, not the owner’s URL.
  • Fixed a bug where mousedown without mouseup in a frame prevents a click event in another frame.
  • Fixed a sometimes incorrect location after exiting mouse hover.
  • Fixed accepting image/jpg for compatibility.
  • Fixed adding a non-breaking space, instead of a plain space, when it is inserted before an empty text node.
  • Fixed behavior of nested click event on a label element with a checkbox.
  • Fixed BroadcastChannel in a SharedWorker when hosted in a cross-origin iframe.
  • Fixed calculation of direction for text form control elements with dir="auto".
  • Fixed canvas fallback content focusability computation.
  • Fixed deleting a button element leaving the button’s style in a contenteditable element.
  • Fixed disconnected <fieldset> elements sometimes incorrectly matching :valid or :invalid selectors.
  • Fixed dragging the mouse over a -webkit-user-select: none node can begin selection in another node.
  • Fixed ensuring nested workers get controlled if matching a service worker registration.
  • Fixed errors caught and reported for importScripts().
  • Fixed escaping “&” in JavaScript URLs for innerHTML and outerHTML.
  • Fixed EventSource to stop allowing trailing data when parsing a retry delay.
  • Fixed Fetch Request object to keep its Blob URL alive.
  • Fixed filled text on a canvas with a web font refreshing or disappearing.
  • Fixed find on page failing to show results in PDFs.
  • Fixed firing an error event when link preload fails synchronously.
  • Fixed form submissions to cancel JavaScript URL navigations.
  • Fixed handing the onerror content attribute on body and frameset elements.
  • Fixed handling opaque origin Blob URLs.
  • Fixed handling text documents to align to modern HTML specifications.
  • Fixed handling the onerror content attribute on <body> and <frameset> elements.
  • Fixed HTMLTemplateElement to have a shadowRootMode attribute.
  • Fixed including alternate stylesheets in document.styleSheets.
  • Fixed incorrect caret movement in some right-to-left contenteditable elements.
  • Fixed incorrect color for videos loaded in a canvas.
  • Fixed incorrect image srcset candidate chosen for <img> cloned from <template>.
  • Fixed incorrectly ignored X-Frame-Options HTTP headers with an empty value.
  • Fixed lazy loading images sometimes not loading.
  • Fixed link elements to be able to fire more than one load or error event.
  • Fixed loading Blob URLs with a fragment from opaque, unique origins.
  • Fixed maintaining the original Content-Type header on a 303 HTTP redirect.
  • Fixed module scripts to always decode using UTF-8.
  • Fixed MouseEventInit to take movementX and movementY.
  • Fixed not dispatching a progress event when reading an empty file or blob using the FileReader API.
  • Fixed not replacing the current history item when navigating a cross-origin iframe to the same URL.
  • Fixed overriding the mimetype for an XHR.
  • Fixed parsing of negative age values in CORS prefetch responses.
  • Fixed pasting of the first newline into text area.
  • Fixed preventing selection for generated counters in ordered lists.
  • Fixed Safari frequently using stale cached resources despite using Reload Page From Origin.
  • Fixed scheduling a navigation to a Blob URL to keep the URL alive until the navigation occurs.
  • Fixed sending Basic authentication via XHR using setRequestHeader() when there is an existing session.
  • Fixed setting style="" to destroy the element’s inline style.
  • Fixed setting the tabIndex of a non-focusable HTMLElement.
  • Fixed system colors not respecting inherited color-scheme values.
  • Fixed textarea placeholder text not disappearing when text is inserted without a user gesture.
  • Fixed the event.keyIdentifier value for F10 and F11 keys.
  • Fixed the click event to not get suppressed on textarea resize.
  • Fixed the computed value for the transform property with SkewY.
  • Fixed the initialization of color properties.
  • Fixed timing of ResizeObserver and IntersectionObserver to match other browsers.
  • Fixed toggling a details element when a summary element receives a click().
  • Fixed updating Text node children of an option element to not reset the selection of the select element.
  • Fixed using NFC Security Key on iOS.
  • Fixed using WebAuthn credentials registered on iOS 15 if iCloud Keychain is disabled.
  • Fixed WebAuthn sending Attestation as None when requested as Direct.
  • Fixed XHR aborting to align with standards specification
  • Fixed XHR error events to return 0 for loaded and total.
  • Fixed: Made all FileSystemSyncAccessHandle methods synchronous.
  • Fixed: Removed the precision="float" attribute on <input type="range">.

WebGL

  • Fixed video textures set to repeat.

Web Inspector

  • Fixed “Inspect Element” not highlighting the element.
  • Fixed capturing async stack traces for queueMicrotask.
  • Fixed clicking coalesced events in the timeline selecting the wrong event.
  • Fixed event breakpoints to support case-insensitive and RegExp matching.
  • Fixed slow search with a lot of files in the Open Resource dialog.
  • Fixed sorting prefixed properties below non-prefixed properties in the Computed panel of the Elements Tab.
  • Fixed the always empty Attributes section in the Node panel of the Elements Tab.
  • Fixed the Computed Tab scrolling to the top when a <style> is added to the page.
  • Fixed URL breakpoints to also pause when HTML attributes are set that trigger loads.

WebDriver

  • Fixed “Get Element Rect” to not round to integer values.
  • Fixed automation sessions terminating during navigation.
  • Fixed click element failing on iPad when Stage Manager is disabled.
  • Fixed HTTP GET requests with a body failing.
  • Fixed the Shift modifier key not applying to typed text.

Feedback

We love hearing from you. Send a tweet to @webkit to share your thoughts on Safari 16.4. Find us on Mastodon at @jensimmons@front-end.social and @jondavis@mastodon.social. If you run into any issues, we welcome your feedback on Safari UI, or your WebKit bug report about web technology or Web Inspector. Filing issues really does make a difference.

Download the latest Safari Technology Preview to stay at the forefront of the web platform and to use the latest Web Inspector features. You can also read the Safari 16.4 release notes.

March 27, 2023 05:30 PM

March 23, 2023

Release Notes for Safari Technology Preview 166

Surfin’ Safari

Safari Technology Preview Release 166 is now available for download for macOS Monterey 12.3 or later and macOS Ventura. If you already have Safari Technology Preview installed, you can update it in the Software Update pane of System Preferences on macOS Monterey, or System Settings under General → Software Update on macOS Ventura.

This release includes WebKit changes between: 260849@main…261247@main.

Web Inspector

  • Added input fields for editing variation axes values in Fonts sidebar panel (261162@main)

CSS

JavaScript

  • Implemented RegExp v flag with set notation and properties of strings (261188@main)
  • Enabled new WASM baseline JIT (BBQ) for increased performance (261153@main)
  • Inlined Proxy [[Set]] trap in DFG / FTL (261058@main)
  • Made C++ to JS calls faster (260858@main)

Popover

  • Enabled the popover attribute (261193@main)
  • Implemented [popover=auto] and light dismiss behavior (261093@main)

Media

  • Fixed picture-in-picture video snapping to incorrect size (261383@main)

MSE

  • Changed to only fire durationchange after parsing the media buffer (261029@main)

Web API

  • Added support for preconnect via HTTP early hints (261079@main)
  • Added Cancel, Unknown, and Clear keycodes (261008@main)
  • Added selection API that works across shadow boundaries (261021@main)
  • Added support for largeBlob extension for the local authenticator (260958@main)
  • Adjusted text input scrollWidth and scrollHeight to include padding and any whitespace added by decorations (261121@main)
  • Fixed translation for shadow DOM content (261096@main)
  • Fixed translation to treat a floating list item as its own paragraph (261114@main)
  • Fixed Speech Recognition API terminating after one utterance or a short time (260886@main)
  • Fixed window.onload getting repeatedly re-executed when changing the URL fragment during onload (260860@main)
  • Improved support for prioritized HTTPS navigations (261022@main)
  • Stripped tab and newline from Location, URL, <a>, and <area>‘s protocol setter (261017@main)

Accessibility

  • Fixed input[type=date] individual fields getting announced as “group” (261123@main)
  • Fixed the wrong role displayed for input in Web Inspector (260868@main)

March 23, 2023 07:07 PM

March 20, 2023

Enabling the Inspection of Web Content in Apps

Surfin’ Safari

Web Inspector is a powerful tool that allows you to debug the layout of web pages, step through JavaScript, read messages logged to the console, and more. In Safari on macOS, you can use Web Inspector to inspect web pages, extensions, and service workers. iOS and iPadOS allow inspection of the same content as macOS, with the addition of Home Screen web apps.

Web content and JavaScript is used for various purposes in apps, from providing UI from a webpage to enabling apps to be scriptable. Previously, Web Inspector supported inspecting developer-provisioned apps built directly from Xcode for local development, meaning developers could debug this content so long as the app is installed for development. However, released versions of apps had no way to inspect dynamic web content or scripts, leaving developers and users to have to resort to more complicated workflows to get information that would otherwise be made available by Web Inspector. Now, this same functionality is available through an API on WKWebView and JSContext.

How do I enable inspection?

Across all platforms supporting WKWebView or JSContext, a new property is available called isInspectable (inspectable in Objective-C). It defaults to false, and you can set it to true to opt-in to content being inspectable. This decision is made for each individual WKWebView and JSContext to prevent unintentionally making it enabled for a view or context you don’t intend to be inspectable. So, for example, to make a WKWebView inspectable, you would:

Swift
let webConfiguration = WKWebViewConfiguration()
let webView = WKWebView(frame: .zero, configuration: webConfiguration)
webView.isInspectable = true
Objective-C
WKWebViewConfiguration *webConfiguration = [WKWebViewConfiguration new];
WKWebView *webView = [[WKWebView alloc] initWithFrame:CGRectZero configuration:webConfiguration];
webView.inspectable = YES;

For JSContext, matching API is available, with the addition of C API for developers using JSGlobalContextRef:

Swift
let jsContext = JSContext()
jsContext?.isInspectable = true
Objective-C
JSContext *jsContext = [JSContext new];
jsContext.inspectable = YES;
C
JSGlobalContextRef jsContextRef = JSGlobalContextCreate(NULL);
JSGlobalContextSetInspectable(jsContextRef, true);

The inspectable property can be changed at any point during the lifetime of your WKWebView or JSContext. Disabling inspection while Web Inspector actively inspects the content will automatically close Web Inspector, and no further information about the content will be available.

Once you’ve enabled inspection for your app, you can inspect it from Safari’s Develop menu in the submenu for either your current computer or an attached device. For iOS and iPadOS, you must also have enabled Web Inspector in the Settings app under Safari > Advanced > Web Inspector. You do not need to enable Web Inspector for simulators; it is always enabled. Learn more about enabling Web Inspector…

Develop Menu > Patrick's iPhone > Example App

When should I consider making content inspectable?

A common situation in which you may want the content of WKWebView to be inspectable is in an in-app web browser. The browser shows ordinary web content that would be inspectable when loaded in Safari. It can be beneficial both for the app developer, as well as web authors, to be able to inspect content in these views, as the size of the view may not match that of Safari’s, or the app developer may be injecting script into the view to provide integration with their app.

Web content is often dynamic, delivered by a server—not in the app—and easily changed over time. Unfortunately, not all issues can or will get debugged by folks with access to a developer-provisioned copy of your app.

JSContext can also enable scripting in an app whereby the customer provides the scripts to augment the app. Without the ability for a release version of your app to adopt inspectability, your customers may have no way to debug the scripts they have written. It makes it harder for customers to use this functionality of your app.

Provide readable names for inspectable JSContexts

Unlike WKWebView, which automatically gets a name based on the page currently loaded in the view, every JSContext with inspectable enabled will be listed as JSContext in Safari’s Develop menu. We recommend providing a unique, human-readable name for each inspectable JSContext to make it easier for you and your customers to determine what the JSContext represents. For example, if your app runs different pieces of JavaScript on behalf of the user, you should give each JSContext a name based on what runs inside the context.

API is available to set the user-visible name of a JSContext:

Swift
let jsContext = JSContext()
jsContext?.name = "Context name"
Objective-C
JSContext *jsContext = [JSContext new];
jsContext.name = @"Context name";
C
JSGlobalContextRef jsContextRef = JSGlobalContextCreate(NULL);
`JSGlobalContextSetName`(jsContextRef, JSStringCreateWithUTF8CString("Context name"));

Working with older versions of macOS and iOS

For apps linked against an SDK before macOS 13.3 and iOS 16.4 WKWebViews and JSContexts will continue to follow the previous behavior of always being inspectable when built for debugging from Xcode.

Apps that support older versions of macOS and iOS while linked against the most recent SDK will not get the previous behavior of all content being inspectable in debug builds to avoid confusion about what will and will not be inspectable by customers. Apps targeting older OS versions but linking against the new SDK can use this new API conditionally on OS versions that support it. To conditionally guard usage of the API:

Swift
if #available(macOS 13.3, iOS 16.4, tvOS 16.4, *) {
    webView.isInspectable = true
}
Objective-C
if (@available(macOS 13.3, iOS 16.4, tvOS 16.4, *))
    webView.inspectable = YES;

You can learn more about guarding usage of new API on developer.apple.com.

Feedback

As you explore this new API, please help us by providing feedback if you encounter problems. For issues using this new API, please file feedback from your Mac, iPhone, or iPad. Feedback Assistant will collect the information needed to help us understand what’s happening. For any issues you may experience with Web Inspector itself once inspecting your app’s content, please file a bug on bugs.webkit.org.

Also, we love hearing from you. You can find us on Mastodon at @patrickangle@hachyderm.io, @jensimmons@front-end.social, and @jondavis@mastodon.social.

Note: Learn more about Web Inspector from the Web Inspector Reference documentation.

March 20, 2023 06:54 PM

March 14, 2023

Víctor Jáquez: Review of Igalia Multimedia activities (2022)

Igalia WebKit

We, Igalia’s multimedia team, would like to share with you our list of achievements along the past 2022.

WebKit Multimedia

WebRTC

Phil already wrote a first blog post, of a series, on this regard: WebRTC in WebKitGTK and WPE, status updates, part I. Please, be sure to give it a glance, it has nice videos.

Long story short, last year we started to support Media Capture and Streams in WebKitGTK and WPE using GStreamer, either for input devices (camera and microphone), desktop sharing, webaudio, and web canvas. But this is just the first step. We are currently working on RTCPeerConnection, also using GStreamer, to share all these captured streams with other web peers. Meanwhile, we’ll wait for the second episode of Phil’s series 🙂

MediaRecorder

We worked in an initial implementation of MediaRecorder with GStreamer (1.20 or superior). The specification goes about allowing a web browser to record a selected stream. For example, a voice-memo or video application which could encode and upload a capture of your microphone / camera.

Gamepad

While WebKitGTK already has Gamepad support, WPE lacked it. We did the implementation last year, and there’s a blog post about it: Gamepad in WPEWebkit, with video showing a demo of it.

Capture encoded video streams from webcams

Some webcams only provide high resolution frames encoded in H.264 or so. In order to support these resolutions with those webcams we added the support for negotiate of those formats and decode them internally to handle the streams. Though we are just at the beginning of more efficient support.

Flatpak SDK maintenance

A lot of effort went to maintain the Flatpak SDK for WebKit. It is a set of runtimes that allows to have a reproducible build of WebKit, independently of the used Linux distribution. Nowadays the Flatpak SDK is used in Webkit’s EWS, and by many developers.

Among all the features added during the year we can highlight added Rust support, a full integrity check before upgrading, and offer a way to override dependencies as local projects.

MSE/EME enhancements

As every year, massive work was done in WebKit ports using GStreamer for Media Source Extensions and Encrypted Media Extensions, improving user experience with different streaming services in the Web, such as Odysee, Amazon, DAZN, etc.

In the case of encrypted media, GStreamer-based WebKit ports provide the stubs to communicate with an external Content Decryption Module (CDM). If you’re willing to support this in your platform, you can reach us.

Also we worked in a video demo showing how MSE/EME works in a Raspberry Pi 3 using WPE:

WebAudio demo

We also spent time recording video demos, such as this one, showing WebAudio using WPE on a desktop computer.

GStreamer

We managed to merge a lot of bug fixes in GStreamer, which in many cases can be harder to solve rather than implementing new features, though former are more interesting to tell, such as those related with making Rust the main developing language for GStreamer besides C.

Rust bindings and GStreamer elements for Vonage Video API / OpenTok

OpenTok is the legacy name of Vonage Video API, and is a PaaS (Platform As a Service) to ease the development and deployment of WebRTC services and applications.

We published our work in Github of Rust bindings both for the Client SDK for Linux and the Server SDK using REST API, along with a GStreamer plugin to publish and subscribe to video and audio streams.

GstWebRTCSrc

In the beginning there was webrtcbin, an element that implements the majority of W3C RTCPeerConnection API. It’s so flexible and powerful that it’s rather hard to use for the most common cases. Then appeared webrtcsink, a wrapper of webrtcbin, written in Rust, which receives GStreamer streams which will be offered and streamed to web peers. Later on, we developed webrtcsrc, the webrtcsink counterpart: an element which source pads push streams from web peers, such as another browser, and forward those Web streams as GStreamer ones in a pipeline. Both webrtcsink and webrtcsrc are written in Rust.

Behavior-Driven Development test framework for GStreamer

Behavior-Driven Development is gaining relevance with tools like Cucumber for Java and its domain specific language, Gherkin to define software behaviors. Rustaceans have picked up these ideas and developed cucumber-rs. The logical consequence was obvious: Why not GStreamer?

Last year we tinkered with GStreamer-Cucumber, a BDD to define behavior tests for GStreamer pipelines.

GstValidate Rust bindings

There have been some discussion if BDD is the best way to test GStreamer pipelines, and there’s GstValidate, and also, last year, we added its Rust bindings.

GStreamer Editing Services

Though not everything was Rust. We work hard on GStreamer’s nuts and bolts.

Last year, we gathered the team to hack GStreamer Editing Services, particularly to explore adding OpenGL and DMABuf support, such as downloading or uploading a texture before processing, and selecting a proper filter to avoid those transfers.

GstVA and GStreamer-VAAPI

We helped in the maintenance of GStreamer-VAAPI and the development of its near replacement: GstVA, adding new elements such as the H.264 encoder, the compositor and the JPEG decoder. Along with participation on the debate and code reviewing of negotiating DMABuf streams in the pipeline.

Vulkan decoder and parser library for CTS

You might have heard about Vulkan has now integrated in its API video decoding, while encoding is currently work-in-progress. We devoted time on helping Khronos with the Vulkan Video Conformance Tests (CTS), particularly with a parser based on GStreamer and developing a H.264 decoder in GStreamer using Vulkan Video API.

You can check the presentation we did last Vulkanised.

WPE Android Experiment

In a joint adventure with Igalia’s Webkit team we did some experiments to port WPE to Android. This is just an internal proof of concept so far, but we are looking forward to see how this will evolve in the future, and what new possibilities this might open up.

If you have any questions about WebKit, GStreamer, Linux video stack, compilers, etc., please contact us.

By vjaquez at March 14, 2023 10:45 AM

March 08, 2023

Release Notes for Safari Technology Preview 165

Surfin’ Safari

Safari Technology Preview Release 165 is now available for download for macOS Monterey 12.3 or later and macOS Ventura. If you already have Safari Technology Preview installed, you can update it in the Software Update pane of System Preferences on macOS Monterey, or System Settings under General → Software Update on macOS Ventura.

This release includes WebKit changes between: 260164@main…260848@main.

Web Inspector

  • Added support for color-mix CSS values in the Styles details sidebar of the Elements tab (260332@main)
  • Added setting to always show rulers when highlighting elements (260416@main)

CSS

  • Added support for text-transform: full-size-kana (260307@main)
  • Added support for margin-trim for floats in block containers that contain only block boxes (260318@main)
  • Added support for x units in calc() function (260678@main)
  • Added support to image-set() for resolution and type as optional arguments (260796@main)
  • Fixed preserve-3d not being applied to pseudo elements. (260324@main)
  • Fixed opacity not applying to dialog element ::backdrop pseudo-class (260556@main)
  • Fixed the background to not propagate when content: paint is set on the body or root element (260766@main)
  • Fixed table-layout: fixed not being applied when width is max-content (260501@main)
  • Fixed font-optical-sizing: auto having no effect (260447@main)

JavaScript

Layout

  • Fixed accounting of margins in multi-column layout before forced breaks (260510@main)
  • Fixed floats with clear to not be placed incorrectly (260674@main)

Media

  • Fixed SourceBuffer.timestampOffset not behaving correctly with webm content (260822@main)
  • Fixed HDR data to no longer be clipped in AVIF images (260512@main)

Forms

  • Fixed resetting the value of an input type=file to null to make the input invalid (260688@main)
  • Fixed minlength/maxlength attributes to rely on code units instead of grapheme clusters (260838@main)

Web Animations

  • Added support for the length property of CSSKeyframesRule (260400@main)
  • Changed animation of mask-image to be discrete (260756@main)
  • Fixed custom properties not being treated as valid in the shorthand animation property (260759@main)
  • Fixed transition-property: all not applying to custom properties (260384@main)

WebCrypto

  • Fixed Secure Curves not having a namedCurve property (260599@main)

WebGL

  • Fixed restored WebGL context not being visible until layout (260693@main)

Loading

  • Fixed lazily loaded frames to get a contentWindow/contentDocument as soon as they get inserted into the document (260713@main)
  • Fixed frames to not be lazily loaded if they have an invalid or about:blank URL (260612@main)

Web API

Accessibility

  • Fixed aria-errormessage to not be exposed when aria-invalid is false (260545@main)
  • Fixed text associated with various types of elements not being exposed (260521@main)
  • Fixed invalid summary elements to not be exposed as interactive (260546@main)
  • Fixed some inputs not being treated as invalid despite being rendered as such (260544@main)

Web Extensions

  • Fixed Content Blocker API ignoring some CSS selectors with uppercase letters (260638@main)

March 08, 2023 10:03 PM

March 07, 2023

WPE WebKit Blog: Integrating WPE: URI Scheme Handlers and Script Messages

Igalia WebKit

Most Web content is designed entirely for screen display—and there is a lot of it—so it will spend its life in the somewhat restricted sandbox implemented by a web browser. But rich user interfaces using Web technologies in all kinds of consumer devices require some degree of integration, an escape hatch to interact with the rest of their software and hardware. This is where a Web engine like WPE designed to be embeddable shines: not only does WPE provide a stable API, it is also comprehensive in supporting a number of ways to integrate with its environment further than the plethora of available Web platform APIs.

Integrating a “Web view” (the main entry point of the WPE embedding API) involves providing extension points, which allow the Web content (HTML/CSS/JavaScript) it loads to call into native code provided by the client application (typically written in C/C++) from JavaScript, and vice versa. There are a number of ways in which this can be achieved:

  • URI scheme handlers allow native code to register a custom URI scheme, which will run a user provided function to produce content that can be “fetched” regularly.
  • User script messaging can be used to send JSON messages from JavaScript running in the same context as Web pages to an user function, and vice versa.
  • The JavaScriptCore API is a powerful solution to provide new JavaScript functionality to Web content seamlessly, almost as if they were implemented inside the Web engine itself—akin to NodeJS C++ addons.

In this post we will explore the first two, as they can support many interesting use cases without introducing the additional complexity of extending the JavaScript virtual machine. Let’s dive in!

Intermission

We will be referring to the code of a tiny browser written for the occasion. Telling WebKit how to call our native code involves creating a WebKitUserContentManager, customizing it, and then associating it with web views during their creation. The only exception to this are URI scheme handlers, which are registered using webkit_web_context_register_uri_scheme(). This minimal browser includes an on_create_view function, which is the perfect place to do the configuration:

static WebKitWebView*
on_create_view(CogShell *shell, CogPlatform *platform)
{
g_autoptr(GError) error = NULL;
WebKitWebViewBackend *view_backend = cog_platform_get_view_backend(platform, NULL, &error);
if (!view_backend)
g_error("Cannot obtain view backend: %s", error->message);

g_autoptr(WebKitUserContentManager) content_manager = create_content_manager(); /** NEW! **/
configure_web_context(cog_shell_get_web_context(shell)); /** NEW! **/

g_autoptr(WebKitWebView) web_view =
g_object_new(WEBKIT_TYPE_WEB_VIEW,
"user-content-manager", content_manager, /** NEW! **/
"settings", cog_shell_get_web_settings(shell),
"web-context", cog_shell_get_web_context(shell),
"backend", view_backend,
NULL);
cog_platform_init_web_view(platform, web_view);
webkit_web_view_load_uri(web_view, s_starturl);
return g_steal_pointer(&web_view);
}
What is g_autoptr? Does it relate to g_steal_pointer? This does not look like C!

In the shown code examples, g_autoptr(T) is a preprocessor macro provided by GLib that declares a pointer variable of the T type, and arranges for freeing resources automatically when the variable goes out of scope. For objects this results in g_object_unref() being called.

Internally the macro takes advantage of the __attribute__((cleanup, ...)) compiler extension, which is supported by GCC and Clang. GLib also includes a convenience macro that can be used to define cleanups for your own types.

As for g_steal_pointer, it is useful to indicate that the ownership of a pointer declared with g_autoptr is transferred outside from the current scope. The function returns the same pointer passed as parameter and resets it to NULL, thus preventing cleanup functions from running.

The size has been kept small thanks to reusing code from the Cog core library. As a bonus, it should run on Wayland, X11, and even on a bare display using the DRM/KMS subsystem directly. Compiling and running it, assuming you already have the dependencies installed, should be as easy as running:

cc -o minicog minicog.c $(pkg-config cogcore --libs --cflags)
./minicog wpewebkit.org

If the current session kind is not automatically detected, a second parameter can be used to manually choose among wl (Wayland), x11, drm, and so on:

./minicog wpewebkit.org x11

The full, unmodified source for this minimal browser is included right below.

Complete minicog.c source (Gist)

/*
 * SPDX-License-Identifier: MIT
 *
 * cc -o minicog minicog.c $(pkg-config wpe-webkit-1.1 cogcore --cflags --libs)
 */
 
#include <cog/cog.h>
 
static const char *s_starturl = NULL;
 
static WebKitWebView*
on_create_view(CogShell *shell, CogPlatform *platform)
{
    g_autoptr(GError) error = NULL;
    WebKitWebViewBackend *view_backend = cog_platform_get_view_backend(platform, NULL, &error);
    if (!view_backend)
        g_error("Cannot obtain view backend: %s", error->message);
 
    g_autoptr(WebKitWebView) web_view =
        g_object_new(WEBKIT_TYPE_WEB_VIEW,
                     "settings", cog_shell_get_web_settings(shell),
                     "web-context", cog_shell_get_web_context(shell),
                     "backend", view_backend,
                     NULL);
    cog_platform_init_web_view(platform, web_view);
    webkit_web_view_load_uri(web_view, s_starturl);
    return g_steal_pointer(&web_view);
}
 
int
main(int argc, char *argv[])
{
    g_set_application_name("minicog");
 
    if (argc != 2 && argc != 3) {
        g_printerr("Usage: %s [URL [platform]]\n", argv[0]);
        return EXIT_FAILURE;
    }
 
    g_autoptr(GError) error = NULL;
    if (!(s_starturl = cog_uri_guess_from_user_input(argv[1], TRUE, &error)))
        g_error("Invalid URL '%s': %s", argv[1], error->message);
 
    cog_modules_add_directory(COG_MODULEDIR);
 
    g_autoptr(GApplication) app = g_application_new(NULL, G_APPLICATION_DEFAULT_FLAGS);
    g_autoptr(CogShell) shell = cog_shell_new("minicog", FALSE);
    g_autoptr(CogPlatform) platform =
        cog_platform_new((argc == 3) ? argv[2] : g_getenv("COG_PLATFORM"), &error);
    if (!platform)
        g_error("Cannot create platform: %s", error->message);
 
    if (!cog_platform_setup(platform, shell, "", &error))
        g_error("Cannot setup platform: %s\n", error->message);
 
    g_signal_connect(shell, "create-view", G_CALLBACK(on_create_view), platform);
    g_signal_connect_swapped(app, "shutdown", G_CALLBACK(cog_shell_shutdown), shell);
    g_signal_connect_swapped(app, "startup", G_CALLBACK(cog_shell_startup), shell);
    g_signal_connect(app, "activate", G_CALLBACK(g_application_hold), NULL);
 
    return g_application_run(app, 1, argv);
}

URI Scheme Handlers

“Railroad” diagram of URI syntax URI syntax (CC BY-SA 4.0, source), notice the “scheme” component at the top left.

A URI scheme handler allows “teaching” the web engine how to handle any load (pages, subresources, the Fetch API, XmlHttpRequest, …)—if you ever wondered how Firefox implements about:config or how Chromium does chrome://flags, this is it. Also, WPE WebKit has public API for this. Roughly:

  1. A custom URI scheme is registered using webkit_web_context_register_uri_scheme(). This also associates a callback function to it.
  2. When WebKit detects a load for the scheme, it invokes the provided function, passing a WebKitURISchemeRequest.
  3. The function generates data to be returned as the result of the load, as a GInputStream and calls webkit_uri_scheme_request_finish(). This sends the stream to WebKit as the response, indicating the length of the response (if known), and the MIME content type of the data in the stream.
  4. WebKit will now read the data from the input stream.

Echoes

Let’s add an echo handler to our minimal browser that replies back with the requested URI. Registering the scheme is straightforward enough:

static void
configure_web_context(WebKitWebContext *context)
{
webkit_web_context_register_uri_scheme(context,
"echo",
handle_echo_request,
NULL /* userdata */,
NULL /* destroy_notify */);
}
What are “user data” and “destroy notify”?

The userdata parameter above is a convention used in many C libraries, and specially in these based on GLib when there are callback functions involved. It allows the user to supply a pointer to arbitrary data, which will be passed later on as a parameter to the callback (handle_echo_request in the example) when it gets invoked later on.

As for the destroy_notify parameter, it allows passing a function with the signature void func(void*) (type GDestroyNotify) which is invoked with userdata as the argument once the user data is no longer needed. In the example above, this callback function would be invoked when the URI scheme is unregistered. Or, from a different perspective, this callback is used to notify that the user data can now be destroyed.

One way of implementing handle_echo_request() could be wrapping the request URI, which is part of the WebKitURISchemeRequest parameter to the handler, stash it into a GBytes container, and create an input stream to read back its contents:

static void
handle_echo_request(WebKitURISchemeRequest *request, void *userdata)
{
const char *request_uri = webkit_uri_scheme_request_get_uri(request);
g_print("Request URI: %s\n", request_uri);

g_autoptr(GBytes) data = g_bytes_new(request_uri, strlen(request_uri));
g_autoptr(GInputStream) stream = g_memory_input_stream_new_from_bytes(data);

webkit_uri_scheme_request_finish(request, stream, g_bytes_get_size(data), "text/plain");
}

Note how we need to tell WebKit how to finish the load request, in this case only with the data stream, but it is possible to have more control of the response or return an error.

With these changes, it is now possible to make page loads from the new custom URI scheme:

Screenshot of the minicog browser loading a custom echo:// URI It worked!

Et Tu, CORS?

The main roadblock one may find when using custom URI schemes is that loads are affected by CORS checks. Not only that, WebKit by default does not allow sending cross-origin requests to custom URI schemes. This is by design: instead of accidentally leaking potentially sensitive data to websites, developers embedding a web view need to consciously opt-in to allow CORS requests and send back suitable Access-Control-Allow-* response headers.

In practice, the additional setup involves retrieving the WebKitSecurityManager being used by the WebKitWebContext and registering the scheme as CORS-enabled. Then, in the handler function for the custom URI scheme, create a WebKitURISchemeResponse, which allows fine-grained control of the response, including setting headers, and finishing the request instead with webkit_uri_scheme_request_finish_with_response().

Note that WebKit cuts some corners when using CORS with custom URI schemes: handlers will not receive preflight OPTIONS requests. Instead, the CORS headers from the replies are inspected, and if access needs to be denied then the data stream with the response contents is discarded.

In addition to providing a complete CORS-enabled custom URI scheme example, we recommend the Will It CORS? tool to help troubleshoot issues.

Further Ideas

Once we have WPE WebKit calling into our custom code, there are no limits to what a URI scheme handler can do—as long as it involves replying to requests. Here are some ideas:

  • Allow pages to access a subset of paths from the local file system in a controlled way (as CORS applies). For inspiration, see CogDirectoryFilesHandler.
  • Package all your web application assets into a single ZIP file, making loads from app:/... fetch content from it. Or, make the scheme handler load data using GResource and bundle the application inside your program.
  • Use the presence of a well-known custom URI to have a web application realize that it is running on a certain device, and make its user interface adapt accordingly.
  • Provide a REST API, which internally calls into NetworkManager to list and configure wireless network adapters. Combine it with a local web application and embedded devices can now easily get on the network.

User Script Messages

While URI scheme handlers allow streaming large chunks of data back into the Web engine, for exchanging smaller pieces of information in a more programmatic fashion it may be preferable to exchange messages without the need to trigger resource loads. The user script messages part of the WebKitUserContentManager API can be used this way:

  1. Register a user message handler with webkit_user_content_manager_register_script_message_handler(). As opposed to URI scheme handlers, this only enables receiving messages, but does not associate a callback function yet.
  2. Associate a callback to the script-message-received signal. The signal detail should be the name of the registered handler.
  3. Now, whenever JavaScript code calls window.webkit.messageHandlers.<name>.postMessage(), the signal is emitted, and the native callback functions invoked.
Haven't I seen postMessage() elsewhere?

Yes, you have. The name is the same because it provides a similar functionality (send a message), it guarantees little (the receiver should validate messages), and there are similar restrictions in the kind of values that can be passed along.

It’s All JavaScript

Let’s add a feature to our minimal browser that will allow JavaScript code to trigger rebooting or powering off the device where it is running. While this should definitely not be functionality exposed to the open Web, it is perfectly acceptable in an embedded device where we control what gets loaded with WPE, and that exclusively uses a web application as its user interface.

Pepe Silvia conspiracy image meme, with the text “It's all JavaScript” superimposed Yet most of the code shown in this post is C.

First, create a WebKitUserContentManager, register the message handler, and connect a callback to its associated signal:

static WebKitUserContentManager*
create_content_manager(void)
{
g_autoptr(WebKitUserContentManager) content_manager = webkit_user_content_manager_new();
webkit_user_content_manager_register_script_message_handler(content_manager, "powerControl");
g_signal_connect(content_manager, "script-message-received::powerControl",
G_CALLBACK(handle_power_control_message), NULL);
return g_steal_pointer(&content_manager);
}

The callback receives a WebKitJavascriptResult, from which we can get the JSCValue with the contents of the parameter passed to the postMessage() function. The JSCValue can now be inspected to check for malformed messages and determine the action to take, and then arrange to call reboot():

static void
handle_power_control_message(WebKitUserContentManager *content_manager,
WebKitJavascriptResult *js_result, void *userdata)
{
JSCValue *value = webkit_javascript_result_get_js_value(js_result);
if (!jsc_value_is_string(value)) {
g_warning("Invalid powerControl message: argument is not a string");
return;
}

g_autofree char *value_as_string = jsc_value_to_string(value);
int action;
if (strcmp(value_as_string, "poweroff") == 0) {
action = RB_POWER_OFF;
} else if (strcmp(value_as_string, "reboot") == 0) {
action = RB_AUTOBOOT;
} else {
g_warning("Invalid powerControl message: '%s'", value_as_string);
return;
}

g_message("Device will %s now!", value_as_string);
sync(); reboot(action);
}

Note that the reboot() system call above will most likely fail because it needs administrative privileges. While the browser process could run as root to sidestep this issue—definitely not recommended!—it would be better to grant the CAP_SYS_BOOT capability to the process, and much better to ask the system manager daemon to handle the job. In machines using systemd a good option is to call the .Halt() and .Reboot() methods of its org.freedesktop.systemd1 interface.

Now we can write a small HTML document with some JavaScript sprinkled on top to arrange sending the messages:

<html>
<head>
<meta charset="utf-8" />
<title>Device Power Control</title>
</head>
<body>
<button id="reboot">Reboot</button>
<button id="poweroff">Power Off</button>
<script type="text/javascript">
function addHandler(name) {
document.getElementById(name).addEventListener("click", (event) => {
window.webkit.messageHandlers.powerControl.postMessage(name);
return false;
});
}
addHandler("reboot");
addHandler("poweroff");
</script>
</body>
</html>

The complete source code for this example can be found in this Gist.

Going In The Other Direction

But how can one return values from user messages back to the JavaScript code running in the context of the web page? Until recently, the only option available was exposing some known function in the page’s JavaScript code, and then using webkit_web_view_run_javascript() to call it from native code later on. To make this more idiomatic and allow waiting on a Promise, an approach like the following works:

  1. Have convenience JavaScript functions wrapping the calls to .postMessage() which add an unique identifier as part of the message, create a Promise, and store it in a Map indexed by the identifier. The Promise is itself returned from the functions.
  2. When the callback in native code handle messages, they need to take note of the message identifier, and then use webkit_web_view_run_javascript() to pass it back, along with the information needed to resolve the promise.
  3. The Javascript code running in the page takes the Promise from the Map that corresponds to the identifier, and resolves it.

To make this approach a bit more palatable, we could tell WebKit to inject a script along with the regular content, which would provide the helper functions needed to achieve this.

Nevertheless, the approach outlined above is cumbersome and can be tricky to get right, not to mention that the effort needs to be duplicated in each application. Therefore, we have recently added new API hooks to provide this as a built-in feature, so starting in WPE WebKit 2.40 the recommended approach involves using webkit_user_content_manager_register_script_message_handler_with_reply() to register handlers instead. This way, calling .postMessage() now returns a Promise to the JavaScript code, and the callbacks connected to the script-message-with-reply-received signal now receive a WebKitScriptMessageReply, which can be used to resolve the promise—either on the spot, or asynchronously later on.

Even More Ideas

User script messages are a powerful and rather flexible facility to make WPE integrate web content into a complete system. The provided example is rather simple, but as long as we do not need to pass huge amounts of data in messages the possibilities are almost endless—especially with the added convenience in WPE WebKit 2.40. Here are more ideas that can be built on top of user script messages:

  • A handler could receive requests to “monitor” some object, and return a Promise that gets resolved when it has received changes. For example, this could make the user interface of a smart thermostat react to temperate updates from a sensor.
  • A generic handler that takes the message payload and converts it into D-Bus method calls, allowing web pages to control many aspects of a typical Linux system.

Wrapping Up

WPE has been designed from the ground up to integrate with the rest of the system, instead of having a sole focus on rendering Web content inside a monolithic web browser application. Accordingly, the public API must be comprehensive enough to use it as a component of any application. This results in features that allow plugging into the web engine at different layers to provide custom behaviour.

At Igalia we have years of experience embedding WebKit into all kinds of applications, and we are always sympathetic to the needs of such systems. If you are interested collaborating with WPE development, or searching for a solution that can tightly integrate web content in your device, feel free to contact us.

March 07, 2023 12:00 AM

February 22, 2023

Release Notes for Safari Technology Preview 164

Surfin’ Safari

Safari Technology Preview Release 164 is now available for download for macOS Monterey 12.3 or later and macOS Ventura. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Monterey, or System Settings under General → Software Update on macOS Ventura.

This release includes WebKit changes between: 259549@main…260164@main.

Web Inspector

  • Elements tab
    • Added showing grid and flex overlays when in element selection and highlighting elements (259989@main, 260061@main)
    • Prevented showing ::backdrop rules for elements without a backdrop (259894@main)
  • Sources tab
    • Added experimental feature to enable aggressive limits on the length of lines that are formatted for sources (259603@main)

CSS

  • Fixed dynamically setting the width of tables with fixed layout and auto width (260143@main)
  • Improved serialization of mask and background properties (260157@main)
  • Made -webkit-image-set() an alias of image-set() (259994@main)
  • Made margin-trim trim collapsed margins at block-start and block-end sides (259734@main)

JavaScript

ResizeObserver

  • Fixed the initial last reported size of ResizeObservation (259673@main)

Rendering

  • Fixed content truncation when text-overflow is ellipsis (259850@main)
  • Fixed table cells, rows, sections or column (groups) to support margins (259955@main)
  • Fixed the margin for summary on details for right-to-left mode (260063@main)
  • Fixed inline text boxes containing Zero Width Joiner, Zero Width Non-Joiner, or Zero Width No-Break Space characters to not use simplified text measuring (259618@main)

Web Animations

  • Fixed animating two custom property list values with mismatching types to use a discrete animation (259557@main)
  • Fixed the animation of color list custom properties when iterationComposite is incorrect (259761@main)
  • Fixed composite of implicit keyframes for CSS Animations to be replace (259739@main)
  • Fixed keyframes to be recomputed if a custom property registration changes (259737@main)
  • Fixed keyframes to be recomputed when bolder or lighter is used on a font-weight property (259740@main)
  • Fixed keyframes to be recomputed when a parent element changes value for a custom property set to inherit (259812@main)
  • Fixed keyframes to be recomputed when a parent element changes value for a non-inherited property set to inherit (259645@main)
  • Fixed keyframes to be recomputed when the currentcolor value is used on color related properties (259736@main)
  • Fixed keyframes to be recomputed when the currentcolor value is used on a custom property (259808@main)
  • Fixed line-height to not transition from the default value to a number (260028@main)
  • Fixed animations without a browsing context to be idle (260101@main)
  • Fixed an @keyframes rule using an inherit value to update the resolved value when the parent style changes (259631@main)
  • Fixed non-inherited custom property failing to inherit from parent when inherit is set (259809@main)

WebAuthn

  • Fixed conditional passkey requests not cancelling correctly after AbortController.abort() (259754@main)

Media

  • Fixed distorted audio after getUserMedia when playing with AudioWorkletNode (259964@main)
  • Fixed getDisplayMedia to not build a list of every screen and window (259969@main)

HTTP

  • Enabled Clear-Site-Data HTTP header support (259970@main)
  • Added support for Clear-Site-Data: "executionContext" (259940@main)

Editing

  • Turned on the feature to make selection return a live range from getRangeAt and throw errors as specified (259904@main)
  • Fixed incorrect text caret placement when right-to-left text starts with whitespace (259868@main)

Web API

  • Added optional submitter parameter to FormData constructor (259558@main)
  • Added canvas.drawImage support for SVGImageElement (259869@main)
  • Implemented focus fixup rule so that focused elements are blurred when they are no longer focusable due to style changes (260067@main)
  • Fixed <link> elements with media queries that do not match to not block visually first paint
    (259963@main)
  • Fixed a Fetch bug with empty header values in Headers objects with “request-no-cors” guard (260066@main)
  • Fixed caret move by line when padding-top is set (259906@main)
  • Fixed individually paused or playing animations not being effected by Play All Animations and Pause All Animations (259971@main)
  • Fixed find on page failing to show results in PDFs in Safari (259655@main)
  • Fixed navigation within an iframe not exiting fullscreen for a parent iframe element (260024@main)
  • Fixed scrolling away from and back to an individually playing animation causing it to be incorrectly paused (259910@main)

Safari Web Extensions

  • Fixed Cross-Origin-Resource-Policy blocking fetch from extensions (259976@main)

February 22, 2023 10:52 PM

February 16, 2023

Philippe Normand: WebRTC in WebKitGTK and WPE, status updates, part I

Igalia WebKit

Some time ago we at Igalia embarked on the journey to ship a GStreamer-powered WebRTC backend. This is a long journey, it is not over, but we made some progress …

By Philippe Normand at February 16, 2023 08:30 PM

Web Push for Web Apps on iOS and iPadOS

Surfin’ Safari

Today marks the release of iOS and iPadOS 16.4 beta 1, and with it comes support for Web Push and other features for Home Screen web apps.

iPhone Lock Screen showing a notification arriving

Today also brings the first beta of Safari 16.4. It’s a huge release, packed with over 135 features in WebKit — including RegExp lookbehind assertions, Import Maps, OffscreenCanvas, Media Queries Range Syntax, @property, font-size-adjust, Declarative Shadow DOM, and much more. We’ll write all about these new WebKit features when Safari 16.4 is released. Meanwhile, you can read a comprehensive list of new features and fixes in the Safari 16.4 release notes.

But let’s set Safari aside and talk about Home Screen web apps on iOS and iPadOS.

Since the first iPhone, users could add any website to their Home Screen — whether it’s a brochure site, a blog, a newspaper, an online store, a social media platform, a streaming video site, productivity software, an application for creating artwork, or any other type of website. For the last ten years, users of Safari on iOS and iPadOS could do this by tapping the Share button to open the Share menu, and then tapping “Add to Home Screen”. The icon for that website then appears on their Home Screen, where a quick tap gets them back to the site.

Web developers have the option to create a manifest file (with its display member set to standalone or fullscreen) and serve it along with their website. If they do, that site becomes a Home Screen web app. Then, when you tap on its icon, the web app opens like any other app on iOS or iPadOS instead of opening in a browser. You can see its app preview in the App Switcher, separate from Safari or any other browser.

Web Push for Web Apps added to the Home Screen

Now with iOS and iPadOS 16.4, we are adding support for Web Push to Home Screen web apps. Web Push makes it possible for web developers to send push notifications to their users through the use of Push API, Notifications API, and Service Workers all working together.

A web app that has been added to the Home Screen can request permission to receive push notifications as long as that request is in response to direct user interaction — such as tapping on a ‘subscribe’ button provided by the web app. iOS or iPadOS will then prompt the user to give the web app permission to send notifications. Once allowed, the user can manage those permissions per web app in Notifications Settings — just like any other app on iPhone and iPad.

The notifications from web apps work exactly like notifications from other apps. They show on the Lock Screen, in Notification Center, and on a paired Apple Watch.

This is the same W3C standards-based Web Push that was added in Safari 16.1 for macOS Ventura last fall. If you’ve implemented standards-based Web Push for your web app with industry best practices — such as using feature detection instead of browser detection — it will automatically work on iPhone and iPad.

Web Push on iOS and iPadOS uses the same Apple Push Notification service that powers native push on all Apple devices. You do not need to be a member of the Apple Developer Program to use it. Just be sure to allow URLs from *.push.apple.com if you are in control of your server push endpoints.

To learn more about how to setup Web Push, read the article Meet Web Push on webkit.org, or watch the WWDC22 session video Meet Web Push.

Focus support

Notifications are a powerful tool, but it’s easy for people to get into situations where they are overwhelmed by too many of them. Notifications for Home Screen web apps on iPhone and iPad integrate with Focus, allowing users to precisely configure when or where to receive them. For users who add the same web app to their Home Screen on more than one iOS or iPadOS device, Focus modes automatically apply to all of them.

Badging API

Home Screen web apps on iOS and iPadOS 16.4 now support the Badging API. Just like any app on iOS and iPadOS, web apps are now able to set their badge count. Both setAppBadge and clearAppBadge change the count while the user has the web app open in the foreground or while the web app is handling push events in the background — even before permission to display the count has been granted.

Permission to display the badge on the app icon is granted in exactly the same way as other apps on iOS and iPadOS. Once a user gives permission to allow notifications, the icon on the Home Screen will immediately display the current badge count. Users can then configure permissions for Badging in Notifications Settings, just like any other app on iOS or iPadOS.

Manifest ID

WebKit for iOS and iPadOS 16.4 adds support for the id member from the Web Application Manifest standard. It’s a string (in the form of a URL) that acts as the unique identifier for the web application, intended to be used by an OS in whatever way desired. iOS and iPadOS use the Manifest ID for the purpose of syncing Focus settings across multiple devices.

iOS has supported multiple installs of the same web app since the very beginning. We believe the ability for people to install any web app more than once on their device can be useful — providing additional flexibility to support multiple accounts, separate work vs personal usage, and more.

When adding a web app to the Home Screen, users are given the opportunity to change the app’s name. iOS and iPadOS 16.4 combine this name with the Manifest ID to uniquely identify the web app. That way, a user can install multiple copies of the web app on one device and give them different identities. For example, notifications from “Shiny (personal)” can be silenced by Focus while notifications from “Shiny (work)” can be allowed. If the user gives their favorite website the same name on multiple devices, Focus settings on one device will sync and apply to the others as well.

Third-party browser support for Add to Home Screen

In iOS and iPadOS 16.4, third-party browsers can now offer their users the ability to add websites and web apps to the Home Screen from the Share menu.

Applications on iOS and iPadOS present the Share menu by creating a UIActivityViewController with an array of activityItems. For “Add to Home Screen” to be included in the Share menu the following must be true:

  1. The application has the com.apple.developer.web-browser managed entitlement
  2. A WKWebView is included in the array of activityItems
  3. The WKWebView is displaying a document with an HTTP or HTTPS URL
  4. If the device is an iPad, it must not be configured as a Shared iPad

As described above, after a user adds to Home Screen, any website with a Manifest file that sets the display member to standalone or fullscreen will open as a web app when a user taps its icon. This is true no matter which browser added the website to the Home Screen.

If there is no manifest file configured to request web app behavior (and no meta tag marking the site as web app capable), then that website will be saved as a Home Screen bookmark. Starting in iOS and iPadOS 16.4, Home Screen bookmarks will now open in the user’s current default browser.

New Fallback Icon

Web developers usually provide icons to represent their website throughout the interface of a browser. If icons for the Home Screen are not provided, previously iOS and iPadOS would create an icon from a screenshot of the site. Now, iOS and iPadOS 16.4 will create and display a monogram icon using the first letter of the site’s name along with a color from the site instead.

To provide the icon to be used for your website or web app, list the icons in the Manifest file — a capability that’s been supported since iOS and iPadOS 15.4. Or you can use the long-supported technique of listing apple-touch-icons in the HTML document head. (If you do both, apple-touch-icon will take precedence over the Manifest-declared icons.)

New Web API for Web Apps

Besides Web Push, Badging API, and Manifest ID, many of the other new features in Webkit for iOS and iPadOS 16.4 are of particular interest to web app developers focusing on Home Screen web apps. These include:

See the release notes for Safari 16.4 for the full list of features.

Feedback

Are you seeing a bug? That’s to be expected in a beta. Please help us such squash bugs before iOS and iPadOS 16.4 are released to the public by providing feedback from your iPhone or iPad. Feedback Assistant will collect all the information needed to help us understand what’s happening.

Also, we love hearing from you. You can find us on Mastodon at @jensimmons@front-end.social, @bradeeoh@mastodon.social and @jondavis@mastodon.social. Or send a tweet to @webkit to share your thoughts on these new features.

February 16, 2023 06:30 PM

February 15, 2023

The User Activation API

Surfin’ Safari

As a web developer, you’ve probably noticed that certain APIs only work if an end-user clicks or taps on an HTML element. For example, if you try to run the following code in Safari’s Web Inspector, it will result in an error:

await navigator.share({ text: "hi" });
NotAllowedError: The request is not allowed by the user agent or 
the platform in the current context, possibly because the user denied permission.

This error happens when code is not run as a direct result of the end-user clicking or tapping on an HTML element (e.g., a <button>).

Having code that runs as a result of an end-user action is what the HTML specification refers to as “user activation”. There are a large number of APIs on the web that depend on user activation. Common ones include:

  • window.open()
  • navigator.share()
  • navigator.wakelock.request()
  • PaymentRequest.prototype.show()
  • and there are many, many more…

So what constitutes a “user activation”?

The HTML spec defines the following events as “activation triggering user events”:

Together, this list effectively constitutes “user activation”. You’ll note the list of events above is really small. It’s restricted so that certain calls to APIs can only happen as a result of those very distinct end-user actions. This prevents end-users from being accidentally (or deliberately!) spammed with popup windows or other intrusive browser dialogs.

Now that we know about these special events, we can now write code to take into account user activation:

button.addEventListener("click", async () => {
   // This works fine...
   await navigator.share({text: "hi"});
});

So even though we are not specifically listening for a "mousedown" event, we know that the activation triggering event has occurred, so our code can run without throwing any errors.

End-user protections

Now you might be wondering, can one run execute multiple commands that require user activation insingle or multiple event listeners? Consider the following code sample:

button.addEventListener("click", async () => {

   // This works fine...
   await navigator.share({text: "hi"});

   // This will fail...
   window.open("https://example.com");
});

button.addEventListener("click", async () => {
   // This will now fail too...
   window.open("https://example.com");
});

But why do the calls to window.open() fail there? To understand that, we need to delve deeper into how browsers handle user activation under the hood.

Meet “transient” and “sticky” activation

When an “activation triggering user event” occurs what actually happens is that the browser starts an internal timer specifically tied to a browser tab. This timer is not directly exposed to the web page and runs for a short time (a few seconds, maybe). Each browser engine can determine how much time is allocated and it can change for a number of reasons (i.e., it’s deliberately not observable by JavaScript!). It’s designed to give your code enough time to perform some particular task (e.g., it could process some image data and then call navigator.share() to share the image with another application).

In HTML, this timer is called transient activation. And while this timer is running, that browser window “has transient activation”. HTML also defines a concept called sticky activation. This simply means that the web page has had transient activation at some point in the past.

Although rare, some APIs (e.g., Web Audio) use sticky activation to perform some actions.

Now, the above doesn’t explain why window.open() failed above. To understand why, we need to now discuss what HTML calls “activation-consuming APIs”.

APIs that “consume” the user activation

As the name suggests, “activation-consuming APIs” consume the user activation. That is, when those APIs are called, they effectively reset the transient activation timer, so the web page no longer has transient activation.

This behavior is why window.open() fails above: calling navigator.share() consumed the user activation, meaning that window.open() no longer had transient activation (so it fails).

A list of common APIs that consume transient activation in WebKit:

  • Web Notification’s requestPermission() method.
  • Payment Request: the show() method.
  • And, as we have already discussed, Web Share’s share() method.

This list is not exhaustive, and new APIs are being added to the web all the time that either rely on or consume transient activation.

As a point of interest: not all APIs consume the user activation. Some only require transient activation but won’t consume it. That allows multiple asynchronous operations dependent on user activation to take place. Otherwise, it would require the user to click or press on a button over and over again to complete a task, which would be quite annoying for them.

Scope of transient activation

A really useful thing to know about transient activation is that it’s scoped to the entire window (or browser tab)! That means that, so long as all iframes on a page are same-origin, they all have transient activation. However, for security reasons, cross-origin iframes will not have transient activation.

Transient activation across all same origin iframes

For third-party iframes to have transient activation, a user must explicitly activate an HTML element inside the third-party iframe. However, once they activate an element then transient activation propagates to the parent and to any iframes that are same origin to iframe where the activation took place:

Activation propagating to parent frame, to other iframes that match the third-party iframe

Security Note: you can (and should!) restrict what capacities third-party iframes have access to by setting the allow= and/or sandbox= attributes, as needed.

The UserActivation API

To assist developers with dealing with user activation, the HTML standard introduces a simple API to check if a page has transient and/or sticky activation.

  • navigator.userActivation.isActive:
    Returns true when the window has transient activation.
  • navigator.userActivation.hasBeenActive:
    Returns true if the window has had transient activation in the past (i.e., “sticky activation”).

So for example, you can do something like:

if (navigator.userActivation.isActive) {
    await navigator.share({text: "hi"})
}

Limitations and ongoing standards work

There are two significant limitations with the current user activation model that standards folks are still grappling with.
Firstly, consider the following case, where a file takes too long to download and the transient activation timer runs out:

button.onclick = () => {
    // Slow network + really big file
    const image = await fetch("really-big-file");

    // Oh no!!! transient activation expired! 😢
    navigator.share({files: [image]});
}   

There are ongoing discussions at the WHATWG and W3C about how we might address the problem above. Unfortunately, we don’t yet have a solution, but naturally we need some means to extend the transient activation so the code above doesn’t fail.

Secondly, there are legitimate use cases for enabling transient activation in a third-party iframe from a first-party document (e.g., to allow a third-party to process a request for payment). There is ongoing discussions to see if there is some means to safely enable third-party iframes to also have transient activation in special cases.

Automation and testing

To help developers deal with tricky edge-cases that could arise from the transient activation unexpectedly expiring, WebKit has been working with other browser vendors to allow the user activation to be consumed via Web Driver.

Conclusion

Web APIs being gated on user activation helps keep the user safe from annoying intrusions, like multiple popup windows or notification spam, while allowing developers to do the right thing in response to user interaction. The UserActivation API can help you determine if it’s OK to call a function that depends on user activation.

You can try out the User Activation API in Safari Technology Preview release 160 or later.

February 15, 2023 05:24 PM

February 13, 2023

Cathie Chen: How does ResizeObserver get garbage collected in WebKit?

Igalia WebKit

ResizeObserver is an interface provided by browsers to detect the size change of a target. It would call the js callback when the size changes. In the callback function, you can do anything including delete the target. So how ResizeObserver related object is managed?

ResizeObserver related objects

Let’s take a look at a simple example.
<div id="target"></div>
<script>
{
  var ro = new ResizeObserver( entries => {});
  ro.observe(document.getElementById('target'));
} // end of the scope
</script>
  • ro: a JSResizeObserver which is a js wrapper of ResizeObserver,
  • callback: JSResizeObserverCallback,
  • entries: JSResizeObserverEntry,
  • and observe() would create a ResizeObservation,
  • then document and the target.
So how these objects organized? Let’s take a look at the code.
ResizeObserver and Document
It needs Document to create ResizeObserver, and store document in WeakPtr<Document, WeakPtrImplWithEventTargetData> m_document;.
On the other hand, when observe(), m_document->addResizeObserver(*this), ResizeObserver is stored in Vector<WeakPtr<ResizeObserver>> m_resizeObservers;.
So ResizeObserver and Document both hold each other by WeakPtr.
ResizeObserver and Element
When observe(), ResizeObservation is created, and it is stored in ResizeObserver by Vector<Ref<ResizeObservation>> m_observations;.
ResizeObservation holds Element by WeakPtr, WeakPtr<Element, WeakPtrImplWithEventTargetData> m_target.
On the other hand, target.ensureResizeObserverData(), Element creates ResizeObserverData, which holds ResizeObserver by WeakPtr, Vector<WeakPtr<ResizeObserver>> observers;.
So the connection between ResizeObserver and element is through WeakPtr.

Keep JSResizeObserver alive

Both Document and Element hold ResizeObserver by WeakPtr, how do we keep ResizeObserver alive and get released properly?
In the example, what happens outside the scope? Per [1],
Visit Children – When JavaScriptCore’s garbage collection visits some JS wrapper during the marking phase, visit another JS wrapper or JS object that needs to be kept alive.
Reachable from Opaque Roots – Tell JavaScriptCore’s garbage collection that a JS wrapper is reachable from an opaque root which was added to the set of opaque roots during marking phase.
To keep JSResizeObserver itself alive, use the second mechanism “Reachable from Opaque Roots”, custom isReachableFromOpaqueRoots. It checks the target of m_observations, m_activeObservationTargets, and m_targetsWaitingForFirstObservation, if the targets containsWebCoreOpaqueRoot, the JSResizeObserver won’t be released. Note that it uses GCReachableRef, which means the targets won’t be released either. The timeline of m_activeObservationTargets is from gatherObservations to deliverObservations. And the timeline of m_targetsWaitingForFirstObservation is from observe() to the first time deliverObservations. So JSResizeObserver won’t be released if the observed targets are alive, or it has size changed observations not delivered, or it has any target not delivered at all.

ResizeObservation

ResizeObservation is owned by ResizeObserver, so it will be released if ResizeObserver is released.

Keep `JSCallbackDataWeak* m_data` in `JSResizeObserverCallback` alive

Though ResizeObserver hold ResizeObserverCallback by RefPtr, it is a IsWeakCallback.

JSCallbackDataWeak* m_data; in JSResizeObserverCallback does not keep align with JSResizeObserver.
Take a close look at JSCallbackDataWeak, there is JSC::Weak<JSC::JSObject> m_callback;.

To keep JSResizeObserver itself alive, ResizeObserver using the first mechanism “Visit Children”.
In JSResizeObserver::visitAdditionalChildren, it adds m_callback to Visitor, see:
void JSCallbackDataWeak::visitJSFunction(Visitor& visitor)
{
    visitor.append(m_callback);
}

JSResizeObserverEntry

Like JSResizeObserver and callback, JSResizeObserverEntry would make sure the target and contentRect won’t be released when it is alive.
void JSResizeObserverEntry::visitAdditionalChildren(Visitor& visitor)
{
    addWebCoreOpaqueRoot(visitor, wrapped().target());
    addWebCoreOpaqueRoot(visitor, wrapped().contentRect());
}
ResizeObserverEntry is RefCounted.
class ResizeObserverEntry : public RefCounted<ResizeObserverEntry>
It is created in ResizeObserver::deliverObservations, and passed to the JS callback, if JS callback doesn’t keep it, it will be released when the function is finished.
[1] https://github.com/WebKit/WebKit/blob/main/Introduction.md#js-wrapper-lifecycle-management

By cchen at February 13, 2023 02:33 PM

January 19, 2023

WPE WebKit Blog: Status of the new SVG engine in WebKit

Igalia WebKit

figure { margin: 0; } figure > figure { border: 1px #cccccc solid; padding: 4px; } figcaption { background-color: #cccccc; color: black; padding: 1px; text-align: center; margin-bottom: 4px; }

In the previous posts of this series, various aspects of the WPE port architecture were covered. Besides maintaining and advancing the WPE port according to our customers’ needs, Igalia also participates in the development of the WebCore engine itself, which is shared by all WebKit ports. WebCore is the part of the browser engine that does the heavy lifting: it contains all functionality necessary to load, parse, lay out, and paint Web content.

Since late 2019, Igalia has been working on a new SVG engine, dubbed Layer-Based SVG Engine (LBSE), that will unify the HTML/SVG rendering pipelines in WebCore. This will resolve long-standing design issues of the “legacy” SVG engine and unlock a bunch of new exciting possibilities for Web developers to get creative with SVG. Hardware-accelerated compositing, driven by CSS transform animations, 3D perspective transformations for arbitrary SVG elements, CSS z-index support for all SVG graphics elements, and proper coverage rectangle computations and repaints are just a few highlights of the capabilities the future SVG engine will offer.

In this article, an overview is given about the problems that LBSE aims to solve, and the importance of a performant, well-integrated SVG engine especially for the embedded market. Finally, the current upstreaming status is summarized including an outlook for the year 2023.

LBSE in a nutshell

Before diving into the technical topics, let’s take a few minutes to recap the motivations behind the LBSE work, and explain the importance of a well-integrated, performant SVG engine in WebKit, especially for the embedded market.

Motivation

Many of our customers build products that utilize a Linux-powered embedded device, typically using non-x86 CPUs, custom displays with built-in input capabilities (e.g., capacitive touchscreens) often without large amounts of memory or even permanent storage. The software stack for these devices usually consists of a device-specific Linux distribution, containing the proprietary network, GPU, and drivers for the embedded device - the vendor-approved “reference distribution”.

No matter what type of product is built nowadays, many of them need an active Internet connection, to e.g. update their software stack and access additional information. Besides the UI needed to control the product, a lot of additional dialogs, wizards and menus have to be provided to be able to alter the devices’ “system settings”, such as date/time information, time zones, display brightness, WiFi credentials, Bluetooth settings, and so on.

A variety of toolkits exist that assist in writing GUI applications for embedded devices, with a few open-source projects on the market, as well as commercial products providing closed-source, proprietary solutions, that specifically target the embedded market and are often optimized for specific target device families, e.g. certain ARM processors / certain GPUs.

If the need arises, not only to communicate with the Internet but also to display arbitrary Web content, WPE comes into play. As presented in the first post in this series, the flexible and modular WPE architecture makes it an ideal choice for any product in the embedded market that needs Web browsing abilities. The GLib/C-based WPE public APIs allow for customization of the browsing engine and its settings (react on page load/input events, inject custom JS objects, modify style sheets, etc.) and allow the embedder to control/monitor all relevant Web browsing-related activities.

With a full-fledged Web engine at hand, one might ponder if it is feasible to replace the whole native GUI stack with a set of Web pages/applications, and only use WPE to paint the UI in full-screen mode, thus migrating away from native GUI applications — following the trend in the desktop market. The number of organizations migrating native GUI applications into Web applications is rapidly increasing, since there are compelling reasons for Web apps: “write once, use everywhere”, avoiding vendor lock-in, easy/reliable deployment and update mechanisms, and efficient test/development cycles (local in-browser testing!).

Due to the sheer capabilities of the Web platform, it has grown to an environment in which any kind of application can be developed – ranging from video editing applications, big data processing pipelines to 3D games, all using JS/WebAssembly in a browser, presented using HTML5/CSS. And as an important bonus: in 2023, it’s much easier to find and attract talented Web developers and designers that are fluent in HTML/CSS/JS, than those that are comfortable designing UI applications in proprietary, closed-source C/C++ frameworks.

A long-term customer, successfully using WPE in their products, had very similar thoughts and carried out a study, contracting external Web designers to build a complete UI prototype using Web technology. The mock-up made extensive use of SVG2, embedded inline into HTML5 documents or via other mechanisms (CSS background-image, etc.). The UI fulfilled all expectations and worked great in Blink and WebKit-based browsers, delivering smooth animations. On the target device, however, the performance was too slow, far away from usable. A thorough analysis revealed that large parts of the Web page were constantly repainted, and layout operations were repeated for every frame when animations were active. The accumulated time to display a new frame during animations was in the order of a few milliseconds on desktop machines, but took 20-25 milliseconds on the target device, making smooth 60 FPS animations impossible.

The poor performance is not the result of shortcomings in the WPE port of WebKit: when replacing the aforementioned animated SVG document fragments with HTML/CSS “equivalents” (e.g. simulating SVG circles with CSS border-radius tricks) the performance issue vanisheed. Why? SVG lacks support for a key feature called accelerated compositing, which has been available for HTML/CSS since its introduction more than a decade ago. This compositing heavily relies on the Layer Tree, which is unaware of SVG. Extending the Layer Tree implementation to account for SVG is the main motivation for LBSE.

If you are unfamiliar with the concepts of Render Tree and Layer Tree, you might want to read the “Key concepts” section of an earlier LBSE design document, which provides an overview of the topic.

Prototyping

The LBSE effort began in October 2019 as a research project, to find out an ideal design for the SVG Render Tree, that allows SVG to re-use the existing Layer Tree implementation with minimal changes. The aim for LBSE is to share as much code as possible with the HTML/CSS implementation, removing the need for things like SVG specific clipping/masking/filter code and disjoint HTML counterparts for the same operations.

After an extensive phase of experimentation, two abandoned approaches, and a long time spent on regression fixing, the LBSE prototype was finally finished after almost two years of work. It passed all 60k+ WebKit layout tests and offered initial support for compositing, 3D transformations, z-index, and more. The intent was to prove that we can reach feature parity with the legacy SVG engine and retrieve the very same visual results, pixel-by-pixel (except for progressions of LBSE). Shortly after the finalization, the prototype was presented during the WebKit contributors meeting in 2021.

As the name “prototype” indicates, LBSE was not ready for integration into WebKit at this point. It replaced the old SVG engine with a new one, resulting in a monolithic patch exceeding 650 KB of code changes. External contributions generally demand small patches, with ChangeLogs, tests, etc. – no conscientious reviewer in any company would approve a patch replacing a core component of a browser engine in one shot. Splitting up into small pieces is also not going to work, since SVG needs to be kept intact upstream all the time. Duplicating the whole SVG engine? Not practicable either. With that problem in mind, a fruitful discussion took place with Apple during and after the WebKit contributors meeting: a realistic upstreaming strategy was defined - thanks Simon Fraser for suggesting a pragmatic approach!

The idea is simple: bootstrap LBSE in parallel to the legacy SVG engine. Upstream LBSE behind a compile-time flag and additionally a runtime setting. This way the LBSE code is compiled by the EWS bots during upstreaming (rules out bit-rot) and we gain the ability to turn LBSE on, selectively, from our layout tests – very useful during early bootstrapping. For WebKit, that strategy is the best – for LBSE another major effort is necessary: moving from a drop-in replacement approach to a dual-stack SVG engine: LBSE + legacy built into the same WebKit binaries. At least the timing was good since a split-up into small pieces was needed anyhow for upstreaming. Time to dissect the huge branch into logical, atomic pieces with proper change logs.

Before we jump to the upstreaming status, one question should be answered, that came up during the WebKit contributors meeting and also during various discussions: why don’t you just fix the existing SVG engine and instead propose a new one - isn’t that too risky for Web compatibility?

Why don’t you fix the existing SVG engine?

LBSE logo

There was no initial intention to come up with a new SVG engine. During LBSE development it became apparent how much SVG-specific code can be erased when unifying certain aspects with HTML/CSS. After carrying out the integration work, layout/painting and hit-testing work fundamentally different than before. Since that time, LBSE is labeled as a “new SVG engine”, even though the SVG DOM tree part remained almost identical. Web compatibility will improve with LBSE: a few long-standing, critical interop issues with other browser vendors are solved in LBSE. Therefore, there are no concerns regarding Web compatibility risks from our side.

To answer the initial question, whether it is possible to fix the existing SVG engine to add layer support without adding a “new” SVG engine in parallel? Short answer: no.

In the following section, it is shown that adding support for layers implies changing the class hierarchy of the SVG render tree. All SVG renderers need to inherit from RenderLayerModelObject – a change like this cannot be split up easily into small, atomic pieces. Improving the design is difficult if there’s a requirement to keep the SVG engine working all the time upstream: all patches in that direction end up being large as many renderers have to be changed at the same time. Having distinct, LBSE-only implementations of SVG renderers, independent of the legacy engine, leaves a lot of freedom to strive for an optimal design, free of legacy constraints, and avoids huge patches that are impossible to review.

Let’s close the introduction and review the upstreaming status, and discuss where we stand today.

Upstreaming progress

Planning

To unify the HTML/CSS and SVG rendering pipelines there are two possible paths to choose from: teach the Layer Tree about the SVG Render Tree and its rendering model, or vice-versa. For the latter path, the HTML/CSS-specific RenderLayer needs to split into HTML/SVG subclasses and a base class, that is constructible from non-RenderLayerModelObject-derived renderers. The layer management code currently in RenderLayerModelObject would need to move into another place, and so forth. This invasive approach can potentially break lots of things. Besides that danger, many places in the layer/compositing system would need subtle changes to account for the specific needs of SVG (e.g. different coordinate system origin/convention).

Therefore the former route was chosen, which requires transforming the SVG render tree class hierarchy, such that all renderers that need to manage layers derive from RenderLayerModelObject. Using this approach support, for SVG can be added to the layer/compositing system in a non-invasive manner, with only a minimum of SVG-specific changes. The following class hierarchy diagrams illustrate the planned changes.

Legacy design (click to enlarge) Visualization of the legacy SVG render tree class hierarchy in WebCore LBSE design (click to enlarge) Visualization of the LBSE SVG render tree class hierarchy in WebCore

The first graph shows the class hierarchy of the render tree in the legacy SVG engine: RenderObject is the base class for all nodes in the render tree. RenderBoxModelObject is the common base class for all HTML/CSS renderers. It inherits from RenderLayerModelObject, potentially allowing HTML renderers to create layers. For the SVG part of the render tree, there is no common base class shared by all the SVG renderers, for historical reasons.

The second graph shows only the SVG renderers of the LBSE class hierarchy. In that design, all relevant SVG renderers may create/destroy/manage layers, via RenderLayerModelObject. More information regarding the challenges can be found in the earlier LBSE design document.

Report

The upstreaming work started in December 2021, with the introduction of a new layer-aware root renderer for the SVG render subtree: RenderSVGRoot. The existing RenderSVGRoot class was renamed to LegacyRenderSVGRoot (as well as any files, comments, etc.) and all call sites and build systems were adapted. Afterward, a stub implementation of a layer-aware RenderSVGRoot class was added and assured that the new renderer is created for the corresponding SVG DOM element if LBSE is activated.

That process needs to be repeated for all SVG renderers that have substantially changed in LBSE and thus deserve an LBSE-specific upstream implementation. For all other cases, in-file #if ENABLE(LAYER_BASED_SVG_ENGINE) ... #endif blocks will be used to encapsulate LBSE-specific behavior. For example, RenderSVGText / RenderSVGInlineText are almost identical in LBSE downstream, compared to their legacy variants; thus, they are going to share the renderer implementation between the legacy SVG engine and LBSE.

The multi-step procedure was repeated for RenderSVGModelObject (the base class for SVG graphics primitives), RenderSVGShape, RenderSVGRect, and RenderSVGContainer. Core functionality such as laying out children of a container, previously hidden in SVGRenderSupport::layoutChildren() in the legacy SVG engine, now lives in a dedicated class: SVGContainerLayout. Computing the various SVG bounding boxes - object/stroke/decorated bounding box - is precisely specified in SVG2 and got a dedicated implementation as the SVGBoundingBoxComputation class, instead of fragmenting the algorithms all over the SVG render tree as in the legacy SVG engine.

By February 2022, enough functionality was in place to construct the LBSE render tree for basic SVG documents, utilizing nested containers and rectangles as leaves. While this doesn’t sound exciting at all, it provided an ideal environment to implement support for SVG in the RenderLayer-related code - before converting all SVG renderers to LBSE, and before implementing painting in the SVG renderers.

Both RenderLayer and RenderLayerBacking query CSS geometry information such as border box, padding box, or content box from their associated renderer, which is expected to be a RenderBox in many places. This is incorrect for SVG: RenderSVGModelObject inherits from RenderLayerModelObject, but not from RenderBox since it doesn’t adhere to the CSS box model. Various call sites cast the associated renderer to RenderBox to call e.g. borderBoxRect() to retrieve the border box rectangle. There are similar accessors in SVG to query the geometry, but there is no equivalent of a border box or other CSS concetps in SVG. Therefore, we extended RenderSVGModelObject to provide a CSS box model view of an SVG renderer, by offering methods such as borderBoxRectEquivalent() or visualOverflowRectEquivalent() that return geometry information in the same coordinate system using the same conventions as their HTML/CSS counterparts.

We also refactored RenderLayer to use a proxy method - rendererBorderBoxRect() - that provides access to the borderBoxRect() for HTML and the borderBoxRectEquivalent() for SVG renderers, and the same fix to RenderLayerBacking. With these fixes in place, support to position and size SVG layers and to compute overflow information could be added – both pre-conditions to enable painting.

By March 2022, LBSE was able to paint basic SVG documents - a major milestone for the bootstrapping process, demonstrating that the layer painting code was functional for SVG. It was time to move on to transformations: implementing RenderSVGTransformableContainer (e.g. <g> elements with a non-identity transform attribute or CSS transform property) and CSS/SVG transform support for all other graphics primitives, utilizing the RenderLayer-based CSS Transform implementation. As preparation, the existing code was reviewed and cleaned up: transform-origin computation was decoupled from CTM computation (CTM = current transformation matrix, see CSS Transforms Module Level 1) and transform-box computations were unified in a single place.

In April 2022, 2D transforms were enabled and became fully functional a few weeks later. Besides missing compositing support upstream, downstream work showed that enabling 3D transforms for SVG required fixing a decade-old bug that made the computed perspective transformation dependent on the choice of transform-origin. That became apparent when testing the layer code with SVG, which uses different default values for certain transform-related CSS properties than HTML does: transform-box: view-box and transform-origin: 0 0 are the relevant defaults for SVG, referring to the top-left corner of nearest SVG viewport vs. the center of the element in HTML.

By May 2022, the legacy SVG text rendering code was altered to be usable for LBSE as well. At this point, it made sense to run layout tests using LBSE. Previously most tests were expected to fail, as most either utilize text, paths, or shapes, and sometimes all three together. LBSE render tree text dumps (dumping the parsed render tree structure in a text file) were added for all tests in the LayoutTests/svg subdirectory, as well as a new pixel test baseline (screenshots of the rendering as PNGs), generated using the legacy SVG engine, to verify that LBSE produces pixel-accurate results. All upcoming LBSE patches are expected to change the expected layout test result baseline, and/or the TestExpectations file, depending on the type of patch. This will ease the reviewing process a lot for future patches.

To further proceed, a test-driven approach was used to prioritize the implementation of the missing functionality. At that time, missing viewBox support for outer <svg> elements was causing many broken tests. The effect of the transformation induced by the viewBox attribute, specified on outer <svg> elements, cannot be implemented as an additional CSS transformation applied to the outermost <svg> element, as that would affect the painted dimensions of the SVG document, which are subject to the CSS width/height properties and the size negotiation logic only. The viewBox attribute is supposed to only affect the visual appearance of the descendants, by establishing a new local coordinate system for them. The legacy SVG engine manually handled the viewBox-induced transformation in various places throughout LegacyRenderSVGRoot, to only affect the painting of the descendants and not e.g. the position/dimension of the border surrounding the <svg>, if the CSS border property is specified. In LBSE, transformations are handled on RenderLayer-level and not in the renderers anymore.

By July 2022, after testing different approaches, a proper solution to add viewBox support was upstreamed. The chosen solution makes use of another CSS concept that arises in the context of generated content: “anonymous boxes”. The idea is to wrap the direct descendants of RenderSVGRoot in an anonymous RenderSVGViewportContainer (“anonymous” = no associated DOM element) and apply the viewBox transformation as a regular CSS transformation on the anonymous renderer. With that approach, LBSE is left with just a single, unified viewBox implementation, without error-prone special cases in RenderSVGRoot, unlike the legacy SVG engine which has two disjoint implementations in LegacyRenderSVGViewportContainer and LegacyRenderSVGRoot.

After the summer holidays, in August 2022, the next major milestone was reached: enabling compositing support for arbitrary SVG elements, bringing z-index support, hardware-accelerated compositing and 3D transforms to SVG. This time all lessons learned from the previous LBSE prototypes were taken into account, resulting in a complete compositing implementation, that works in various scenarios: different transform-box / transform-origin combinations, inline SVG enclosed by absolute/relative positioned CSS boxes and many more, all way more polished than in the “final” LBSE prototype.

The aforementioned patch contained a fix for a long-standing bug (“Composited elements appear pixelated when scaled up using transform”), that made composited elements look blurry when scaling up with a CSS transform animation. The so-called “backing scale factor” of the associated GraphicLayers (see here for details about the role of GraphicLayer in the compositing system) never changes during the animation. Therefore, the rendered image was scaled up instead of re-rendering the content at the right scale. LBSE now enforces updates of that scale factor, to avoid blurry SVGs. The fix is not activated yet for HTML as that requires more thought - see the previously-linked bug report for details.

With all the new features in place and covered by tests, it was time to finish the remaining SVG renderers: RenderSVGEllipse, RenderSVGPath and RenderSVGViewportContainer (for inner <svg> elements), RenderSVGHiddenContainer, RenderSVGImage, and RenderSVGForeignObject. A proper <foreignObject> implementation was lacking in WebKit for 15+ years, due to the fundamental problem that the layer tree was not aware of the SVG subtree. The LBSE variant of RenderSVGForeignObject looks trivial, yet offers a fully compatible <foreignObject> implementation - for the first time without issues with non-static positioned content as a direct child of <foreignObject>, at least a few weeks later after it landed.

Returning to the test-driven approach, the next best target to fix was text rendering, which was working but not pixel-perfect. The legacy SVG engine takes into account the transformation from the text element up to the topmost renderer when computing the effective “on-screen” font size used to select a font for drawing/measuring, during layout time. LBSE needed a way to calculate the CTM for a given SVG renderer, up to a given ancestor renderer (or root), taking into account all possible transformation scenarios, including CSS transform, translate, rotate, SVG transform attribute, shifts due to transform-origin, perspective transformations, and much more. The same functionality is required to implement getCTM() / getScreenCTM().

By the end of August 2022, SVGLayerTransformComputation was added that re-used the existing mapLocalToContainer() / TranformState API to obtain the CTM. The CTM construction and ancestor chain walk - to accumulate the final transformation matrix - is performed by mapLocalToContainer() and no longer needs a special, incomplete SVG approach: the existing general approach now works for SVG too.

September 2022 was mostly devoted to bug fixes related to pixel-snapping. Outermost <svg> elements were not always enforcing stacking contexts and failed to align to device pixels. All other elements behaved fine with respect to pixel snapping (not applied for SVG elements) unless compositing layers were active. In that case, a RenderLayerBacking code path was used that unconditionally applied pixel-snapping - avoid that for SVG.

By October 2022 LBSE could properly display SVGs embedded into HTML host documents via <object> elements – the size negotiation logic failed to take into account the LBSE-specific renderers before. CSS background-image / list-image / HTML <img> / etc. were fixed as well. Zooming and panning support were implemented and improved compared to the legacy engine. Along the way an important bug was fixed, one that other browsers had already fixed back in 2014. The bug caused percentage-sized documents (e.g. width: 100%; height: 100%) that also specify a viewBox to always keep the document size, regardless of the zoom level. Thus, upon zooming, only the stroke width enlarged, but not the boundaries of the document, and thus scrollbars never appeared.

Over the following weeks, text-related issues had to be fixed, which were responsible for a bunch of the remaining test issues. Transformed text did not render, which turned out to be a simple mistake. More tests were upstreamed, related to compositing and transformations. More test coverage revealed that transform changes were not handled consistently – it took a period of investigation to land a proper fix. SVG transform / SMIL <animateMotion> / SMIL <animateTransform> / CSS transform changes are now handled consistently in LBSE, leading to proper repaints, as expected.

Transformation support can be considered complete and properly handled both during the painting and layout phases. Dynamic changes at runtime are correctly triggering invalidations. However, the Web-exposed SVG DOM API that allows querying the transformation matrices of SVG elements, such as getCTM() and getScreenCTM(), was still missing. By November 2022 a complete implementation was upstreamed, that utilized the new SVGLayerTransformComputation class to construct the desired transformation matrices. This way the same internal API is used for painting/layout/hit-testing and implementing the SVG DOM accessors.

By December 2022 LBSE was in a good shape: most important architectural changes were upstreamed and the most basic features were implemented. The year closed with a proposed patch that will avoid re-layout when an element’s transform changes. The legacy SVG engine always needs a re-layout if transform changes, as the size of each ancestor can depend on the presence of transformations on the child elements – a bad design decision two decades ago that LBSE will resolve. Only repainting should happen, but no layouts, in LBSE.

Let’s move on to 2023, and recap what’s still missing in LBSE.

Next steps

Besides fixing all remaining test regressions (see LayoutTests/platform/mac-ventura-wk2-lbse-text/TestExpectations) “SVG resources” are missing in LBSE. That includes all “paint servers” and advanced painting operations: there is no support for linear/radial gradients, no support for patterns, and no support for clipping/masking and filters.

From the painting capabilities, LBSE is still in a basic shape. However, this was intentional, since a lot of the existing code for SVG resource handling is no longer needed in LBSE. Clipping/masking and filters will be handled via RenderLayer, reusing the existing HTML/CSS implementations. Temporary ImageBuffers are no longer needed for clipping, and thus there is no need to cache the “per client” state in the resource system (e.g. re-using the cached clipping mask for repainting). This will simplify the implementation of the “SVG resources” a lot.

Therefore the first task in 2023 is to implement clipping, then masking, gradients, patterns, and as the last item, filters, since they require a substantial amount of refactoring in RenderLayerFilters. Note that these implementations are already complete in LBSE downstream and do not need to be invented from scratch. The first patches in that direction should be up for review by February 2023.

After all “SVG resources” are implemented in LBSE, feature parity is almost there and performance work will follow afterward. WebKit has a golden rule to never ship a performance regression; therefore, LBSE needs to be at least as fast in the standard performance tests, such as MotionMark, before it can replace the legacy engine. Currently, LBSE is slower than the legacy engine with respect to static rendering performance. Quoting numbers does not help at present, since the problem is well understood and will be resolved in the following months.

LBSE currently creates more RenderLayer objects than necessary: for each renderer, unconditionally. This is a great stress test of the layer system, and helpful for bootstrapping, but the associated overhead and complexity are simply not necessary for many cases, and actively hurt performance. LBSE already outperforms the legacy SVG engine whenever animated content is viewed, if it benefits from the hardware acceleration in LBSE.

2023 will be an exciting year, and hopefully brings LBSE to the masses, stay tuned!

Demos

“A picture is worth a thousand words”, so we’d like to share with you the videos shown during the WebKit contributors meeting in 2022 that demo the LBSE capabilities. Be sure to check them out so you can get a good picture of the state of the work. Enjoy!

  1. Accelerated 2D transforms (Tiger)

  2. Accelerated 3D transform (Tiger)

  3. Transition storm (Tiger)

  4. Vibrant example

Final thoughts

We at Igalia are doing our best to fulfill the mission and complete the LBSE upstreaming as fast as possible. In the meanwhile, let us know about your thoughts:

  • What would you do with a performant, next-level SVG engine?
  • Any particular desktop / embedded project that would benefit from it?
  • Anything in reach now, that seemed impossible before with the given constraints in WebKit?

Thanks for your attention! Be sure to keep an eye on our “Upstreaming status” page at GitHub to follow LBSE development.

January 19, 2023 12:00 AM

January 17, 2023

Manuel Rego: 10 years ago

Igalia WebKit

Once upon a time…

10 years ago I landed my first WebKit patch 🎂:

Bug 107275 - [GTK] Implement LayoutTestController::addUserScript

That was my first patch to a web engine related project, and also my first professional C++ patch. My previous work at Igalia was around web applications development with PHP and Java. At that time, 2013, Igalia had been already involved on web rendering engines for several years, and the work around web applications was fading out inside the company, so moving to the other side of the platform looked like a good idea. 10 years have passed and lots of great things have happened.

Since that first patch many more have come mostly in Chromium/Blink and WebKit; but also in WPT, CSS, HTML and other specs; and just a few, sadly, in Gecko (I’d love to find the opportunity to contribute there more at some point). I’ve became committer and reviewer in both Chromium and WebKit projects. I’m also member of the CSS Working Group (though not very active lately) and Blink API owner.

During these years I have had the chance to attend several events, spoke at a few conferences, got some interviews. I’ve been also helping to organize the Web Engines Hackfest since 2014 and a CSSWG face-to-face meeting at Igalia headquarters in A Coruña in January 2020 (I have so great memories of the last dinner).

These years have allowed me to meet lots of wonderful people from which I’ve learnt, and still learn every day, many things. I have the pleasure to work with amazing folks on a daily basis in the open. Every new feature or project in which I have to work, it’s again a new learning experience, and that’s something I really like about this kind of work.

In this period I’ve seen Igalia grow a lot in the web platform community, to the point that these days we’re the top world consultancy on the web platform, with an important position in several projects and standards bodies. We’re getting fairly well know in this ecosystem, and we’re very proud about having achieved that.

Looking back, I’m so grateful for the opportunity given by Igalia. Thanks to the amazing colleagues and the community and how they have helped me to contribute work on the different projects. Also thanks to the different customers that have allowed me to work upstream and develop my career around web browsers. 🙏 These days the source code I write (together with many others) is being used daily by millions of people, that’s totally unbelievable if you stop for a minute to think about it.

Looking forward to the next decade of involvement in the web platform. See you all on the web! 🚀

January 17, 2023 11:00 PM

Amanda Falke: Tutorial: Building WPE WebKit for Raspberry Pi 3

Igalia WebKit

A lightning guide for building WPE WebKit with buildroot

This tutorial will be for getting “up and running” with WPE WebKit using a Raspberry Pi 3 using a laptop/desktop with Linux Fedora installed. WPE WebKit has many benefits; you may read here about why WPE WebKit is a great choice for embedded devices. WPE WebKit has minimal dependencies and it displays high-quality animations, WebGL and videos on embedded devices.

WebPlatformForEmbedded is our focus; for this tutorial, we’ll be building WPE WebKit with buildroot to build our image, so, make sure to clone the buildroot repository.

You will need:

Raspberry pi 3 items:

  • A raspberry pi 3.
  • A microSD card for the pi. (I usually choose 32GB microSD cards).
  • An external monitor, extra mouse, and extra keyboard for our rpi3, separately from the laptop.
  • An HDMI cable to connect the rpi3 to its own external monitor.

Laptop items:

  • Linux laptop/desktop.
    • This tutorial will be based on Fedora, but you can use Debian or any distro of your choice.
  • You also need a way to interface the microSD card with your laptop. You can get an SD Adapter for laptops that have an SD Card adapter slot, or you can use an external SD Card adapter interface for your computer.
    • This tutorial will be based on having a laptop with an SD card slot, and hence an SD Card Adapter will work just fine.

Items for laptop to communicate with rpi3:

  • An ethernet cable to connect the rpi3 to your laptop.
  • You need some way to get ethernet into your laptop. This is either in the form of an ethernet port on your laptop (not likely), or an adapter of some sort (likely a USB adapter).

Steps: High level overview

This is a high level overview of the steps we will be taking.

  1. Partition the blank SD card.
  2. In the buildroot repository, make <desired_config>.
  3. Run the buildroot menuconfig with make menuconfig to set up .config file.
  4. Run make to build sdcard.img in buildroot/output dir; change .config settings as needed.
  5. Write sdcard.img file to the SD card.
  6. Connect the rpi3 to its own external monitor, and its own mouse and keyboard.
  7. Connect the rpi3 to the laptop using ethernet cable.
  8. Put the SD card into the rpi3.
  9. Setup a shared ethernet connection between the laptop and rpi to get the IP address of rpi.
  10. ssh into the rpi and start WPE WebKit.

Steps: Detailed overview/sequence

1. Partition the blank SD card using fdisk. Create a boot partition and a root partition.

  • Note: this is only needed in case the output image format is root.tar. If it’s sdcard.img, then that is dumped directly to the sdcard, that image is already partitioned internally.
  • If you’re unfamiliar with fdisk, this is a good tutorial.

2. In the buildroot repository root directory, make the desired config.

  • Make sure to clone the buildroot repository.
  • Since things change a lot over time, it’s important to note the specific buildroot commit this tutorial was built on, and that this tutorial was built on January 12th, 2023. It is recommended to build from that commit for consistency to ensure that the tutorial works for you.
  • We are building the cog 2.28 wpe with buildroot. See the build options from the buildroot repository.
  • Run make list-defconfigs to get a list of configurations.
  • Copy raspberrypi3_wpe_2_28_cog_defconfig and run it: make raspberrypi3_wpe_2_28_cog_defconfig.
  • You will quickly get output which indicates that a .config file has been written in the root directory of the buildroot repository.

3. Run the buildroot menuconfig with make menuconfig to set up .config file.

  • Run make menuconfig. You’ll see options here for configuration. Go slowly and be careful.
  • Change these settings. Help menus are available for menuconfig, you’ll see them displayed on the screen.
Operation in menuconfig Location Value
ENABLE Target packages -> Filesystem and flash utilities dosfstools
ENABLE Target packages -> Filesystem and flash utilities mtools
ENABLE Filesystem images ext2/3/4 root filesystem
SET VALUE Filesystem images -> ext2/3/4 root filesystem -> ext2/3/4 variant ext4
DISABLE Filesystem images initial RAM filesystem linked into linux kernel

4. Run make to build sdcard.img in buildroot/output dir; change .config settings as needed.

  • Run make. Then get a coffee as the build and cross-compilation will take awhile.

  • In reality, you may encounter some errors along the way, as cross-compilation can be an intricate matter. This tutorial will guide you through those potential errors.
  • When you encounter errors, you’ll follow a “loop” of sorts:

    Run make -> encounter errors -> manually edit .config file -> -> remove buildroot/output dir -> run make again until sdcard.img is built successfully.

  • If you encounter CMake errors, such as fatal error: stdlib.h: No such file or directory, compilation terminated, and you have a relatively new version of CMake on your system, the reason for the error may be that buildroot is using your local CMake instead of the one specified in the buildroot configuration.
    • We will fix this error by setting in .config file: BR2_FORCE_HOST_BUILD=y. Then remove buildroot/output dir, and run make again.
  • If you encounter error such as path/to/buildroot/output/host/lib/gcc/arm-buildroot-linux-gnueabihf/9.2.0/plugin/include/builtins.h:23:10: fatal error: mpc.h: No such file or directory:
  • then we can fix this error by changing Makefile in ./output/build/linux-rpi-5.10.y/scripts/gcc-plugins/Makefile, by adding -I path/to/buildroot/output/host/include in plugin_cxxflags stanza. Then, as usual, remove buildroot/output dir, and run make again.

5. Write sdcard.img file to the SD card.

  • At this point after the make process, we should have sdcard.img file in buildroot/output/images directory.
  • Write this file to the SD card.
  • Consider using Etcher to do so.

6. Connect the rpi3 to its own external monitor, and its own mouse and keyboard.

  • We’ll have separate monitor, mouse and keyboard all connected to the raspberry pi so that we can use it independently from the laptop.

7. Connect the rpi3 to the laptop using ethernet cable.

8. Put the SD card into the rpi3.

9. Setup a shared ethernet connection between the laptop and rpi to get the IP address of rpi.

In general, one of the main problems for connecting via ssh to the Raspberry Pi is to know the IP address of the device. This is very simple with Raspbian OS; simply turn on the raspberry pi and edit configurations to enable ssh, often over wifi.

This is where the ethernet capabilities of the raspberry pi come in.

Goal: To find syslog message DHCPACK acknowledgement and assignment of the IP address after setting up shared connection between raspberry pi and the laptop.

Throughout this process, continually look at logs. Eventually we will see a message DHCPACK which will likely be preceded by several DHCP handshake related messages such as DHCP DISCOVER, REQUEST etc. The DHCPACK message will contain the IP address of the ethernet device, and we will then be able to ssh into it.

    1. Tail the syslogs of the laptop. On Debian distributions, this is often /var/log/syslog. Since we are using Fedora, we’ll be using systemd's journald with the journactl command:
      • sudo journalctl -f
      • Keep this open in a terminal window.
      • You can also come up with a better solution like grepping logs, if you like, or piping output of stdout elsewhere.
    1. In a second terminal window, open up the NetworkManager.
      • Become familiar with existing devices prior to powering on the raspberry pi, by running nmcli.
    1. Power on the raspberry pi. Watch your system logs.
      • Syslogs will detail the raspberry pi’s name.
    1. Look for that name in NetworkManager nmcli device.
      • Using NetworkManager nmcli, set up shared connection for the ethernet device.
      • Setting up a shared connection is as simple as nmcli connection add type ethernet ifname $ETHERNET_DEVICE_NAME ipv4.method shared con-name local
      • This is a good tutorial for setting up a shared connection with NetworkManager.
    1. Once the connection is shared, syslogs will show a DHCPACK message acknowledging the ethernet device and its IP address. (You may need to power cycle the rpi to see this message, but it will happen).

10. ssh into the rpi and start WPE WebKit.

  • Now that we have the IP address of the raspberry pi, we can ssh into it from the laptop: ssh root@<RPI3_IP_ADDRESS>. (The default password is ‘root’. You can also add your user public key to /root/.ssh/authorized_keys on the pi. You can simplify this process by creating an overlay/root/.ssh/authorized_keys on your computer and by specifying the path to the overlay directory in the BR2_ROOTFS_OVERLAY config variable. That will copy everything in the overlay dir to the image.)
  • After that, export these env variables WPE_BCMRPI_TOUCH=1 and WPE_BCMRPI_CURSOR=1 to enable keyboard and mouse control.
    • Why: Recall that generally WPE WebKit is for embedded devices, such as kioks, or set top boxes requiring control with a remote control or similar device or touch interaction. We are exporting these environment variables so that we can “test” WPE WebKit with our separate mouse and keyboard for our raspberry pi without the need for a touch screen or special hardware targets, or a Wayland compositor such as weston. If this piques your curiosity, please see the WPE WebKit FAQ on Wayland.
  • Start WPE WebKit with cog: cog "http://www.igalia.com/"
  • A browser will launch in the external monitor connected to the raspberry pi 3, and we can control the browser with the raspberry pi’s mouse and keyboard!

That’s all for now. Feel free to reach out in the support channels for WPE on Matrix.

WPE’s Frequently Asked Questions

By Amanda Falke at January 17, 2023 08:03 AM

January 06, 2023

WPE WebKit Blog: Success Story: WPE in digital signage

Igalia WebKit

WPE WebKit in digital signage WPE

Digital signage web rendering players have many advantages and are a trend nowadays. They allow to use HTML5 for composing the UI, provisioning and scheduling new contents to the screens from the cloud is simple and effortless, they provide a robust environment with an automatic crash recovery system, etc. They are also a great choice for digital signage kiosk deployments.

WPE WebKit is an excellent option to use as web rendering engine for Linux-based digital signage players, specially for the low-end market, where WPE Webkit allows to achieve great graphics and rendering performance in the less powerful devices like the ones based on the Raspberry Pi. As a result, WPE WebKit is naturally compatible with the hardware of the main digital signage manufactures that rely on these kind of lower-powered devices.

January 06, 2023 12:00 AM