July 17, 2024

Release Notes for Safari Technology Preview 199

Surfin’ Safari

Safari Technology Preview Release 199 is now available for download for macOS Sequoia beta and macOS Sonoma. If you already have Safari Technology Preview installed, you can update it in System Settings under General → Software Update.

This release includes WebKit changes between: 280287@main…280674@main.

Apple Pay

Resolved Issues

  • Fixed arbitrary 8 digit limit on a line item’s total amount. (280584@main) (112078798)

CSS

New Features

  • Added support for (prefers-contrast: custom). (280383@main) (103658875)

Resolved Issues

  • Fixed computed values for text-box-edge. (280413@main) (117643064)
  • Fixed: Improved large Grid performance. (280473@main) (130728344)

Editing

Resolved Issues

  • Fixed an issue where input method editing would sporadically drop the composition range. (280291@main) (130020224)

JavaScript

New Features

  • Added support for Uint8Array.prototype.toBase64 and Uint8Array.prototype.toHex. (280654@main) (129045737)

Resolved Issues

  • Fixed Intl.DurationFormat for numeric and 2-digit. (280460@main) (130279541)

Media

Resolved Issues

  • Fixed missing picture-in-picture when leaving Safari with picture-in-picture enabled from video viewer. (279877@main) (124376686)
  • Fixed a hang in WebAudio. (280604@main) (130531570)

Rendering

Resolved Issues

  • Fixed support for border, padding, and margin on mfrac and mspace elements in MathML. (280665@main) (131119823)

Text

Resolved Issues

  • Fixed disclosure counter styles to consider writing-mode. (280433@main) (130468537)

WebRTC

Resolved Issues

  • Fixed RTCAudioPlayoutStats to match web specifications. (280461@main) (130767124)

July 17, 2024 11:31 PM

July 16, 2024

Private Browsing 2.0

Surfin’ Safari

When we invented Private Browsing back in 2005, our aim was to provide users with an easy way to keep their browsing private from anyone who shared the same device. We created a mode where users do not leave any local, persistent traces of their browsing. Eventually all other browsers shipped the same feature. At times, this is called “ephemeral browsing.”

We baked in cross-site tracking prevention in all Safari browsing through our cookie policy, starting with Safari 1.0 in 2003. And we’ve increased privacy protections incrementally over the last 20 years. (Learn more by reading Tracking Prevention in Webkit.) Other popular browsers have not been as quick to follow our lead in tracking prevention but there is progress.

Apple believes that users should not be tracked across the web without their knowledge or their consent. Entering Private Browsing is a strong signal that the user wants the best possible protection against privacy invasions, while still being able to enjoy and utilize the web. Staying with the 2005 definition of private mode as only being ephemeral, such as Chrome’s Incognito Mode, simply doesn’t cut it anymore. Users expect and deserve more.

So, we decided to take Private Browsing further and add even more protection beyond the normal Safari experience. Last September, we added a whole new level of privacy protections to Private Browsing in Safari 17.0. And we enhanced it even further in Safari 17.2 and Safari 17.5. Plus, when a user enables them, all of the new safeguards are available in regular Safari browsing too.

With this work we’ve enhanced web privacy immensely and hope to set a new industry standard for what Private Browsing should be.

Enhanced Private Browsing in a Nutshell

These are the protections and defenses added to Private Browsing in Safari 17.0:

  • Link Tracking Protection
  • Blocking network loads of known trackers, including CNAME-cloaked known trackers
  • Advanced Fingerprinting Protection
  • Extensions with website or history access are off by default

In addition, we added these protections and defenses in all browsing modes:

  • Capped lifetime of cookies set in responses from cloaked third-party IP addresses
  • Partitioned SessionStorage
  • Partitioned blob URLs (starting in Safari 17.2)

We also expanded Web AdAttributionKit (formerly Private Click Measurement) as a replacement for tracking parameters in URL to help developers understand the performance of their marketing campaigns even under Private Browsing.

Screenshot of Private Browsing in SafariPrivate Browsing in Safari

However, before we dive into these new and enhanced privacy protections, let’s first consider an important aspect of these changes: website compatibility risk.

The Risk of Breaking Websites and How We Mitigate It

There are many ideas for how to protect privacy on the web, but unfortunately many of them may break the user’s experience. Like security protections in real life, a balance must be struck. The new Private Browsing goes right up to the line, attempting to never break websites. But of course there is a risk that some parts of some sites won’t work. To solve this, we give users affordances to reduce privacy protections on a per-site basis. Such a change in privacy protections is only remembered while browsing within a site. This option is a last resort when a web page is not usable due to the privacy protections.

Reload menu with Reload Reducing Privacy Protections selectedReload Reducing Privacy Protections

All of the new privacy protections in Private Browsing are also available in regular browsing. On iOS, iPadOS and visionOS go to Settings > Apps > Safari > Advanced > Advanced Tracking and Fingerprinting Protection and enable “All Browsing”. On macOS go to Safari > Settings > Advanced and enable “Use advanced tracking and fingerprinting protection”:

Safari Advanced Settings with Use advanced tracking and fingerprinting protection in all browsing from Safari Advanced Settings

Let’s now walk through how these enhancements work.

Link Tracking Protection

Safari’s Private Browsing implements two new protections against tracking information in the destination URL when the user navigates between different websites. The specific parts of the URL covered are query parameters and the fragment. The goal of these protections is to make it more difficult for third-party scripts running on the destination site to correlate user activity across websites by reading the URL.

Let’s consider an example where the user clicks a link on clickSource.example, which takes them to clickDestination.example. The URL looks like this:

https://clickDestination.example/article?known_tracking_param=123&campaign=abc&click_val=456

Safari removes a subset of query parameters that have been identified as being used for pervasive cross-site tracking granular to users or clicks. This is done prior to navigation, such that these values are never propagated over the network. If known_tracking_param above represents such a query parameter, the URL that’s used for navigation will be:

https://clickDestination.example/article?campaign=abc&click_val=456

As its name suggests, the campaign above represents a parameter that’s only used for campaign attribution, as opposed to click or user-level tracking. Safari allows such parameters to pass through.

Finally, on the destination site after a cross-site navigation, all third-party scripts that attempt to read the full URL (e.g. using location.search, location.href, or document.URL) will get a version of the URL that has no query parameters or fragment. In our example, this script-exposed value is simply:

https://clickDestination.example/article

In a similar vein, Safari also hides cross-site any document.referrer from script access in Private Browsing.

Web AdAttributionKit in Private Browsing

Web AdAttributionKit (formerly Private Click Measurement) is a way for advertisers, websites, and apps to implement ad attribution and click measurement in a privacy-preserving way. You can read more about it here. Alongside the new suite of enhanced privacy protections in Private Browsing, Safari also brings a version of Web AdAttributionKit to Private Browsing. This allows click measurement and attribution to continue working in a privacy-preserving manner.

Web AdAttributionKit in Private Browsing works the same way as it does in normal browsing, but with some limits:

  • Attribution is scoped to individual Private Browsing tabs, and transfers attribution across new tabs opened when clicking on links. However, attribution is not preserved through other indirect means of navigation: for instance, copying a link and pasting in a new tab. In effect, this behaves similarly to how Web AdAttributionKit works for Direct Response Advertising.
  • Since Private Browsing doesn’t persist any data, pending attribution requests are discarded when the tab is closed.

Blocking Network Loads of Known Trackers

Safari 17.0 also comes with an automatically enabled content blocker in Private Browsing, which blocks network loads to known trackers. While Intelligent Tracking Prevention has long blocked all third party cookies, blocking trackers’ network requests from leaving the user’s device in the first place ensures that no personal information or tracking parameters are exfiltrated through the URL itself.

This automatically enabled content blocker is compiled using data from DuckDuckGo and from the EasyPrivacy filtering rules from EasyList. The requests flagged by this content blocker are only entries that are flagged as trackers by both DuckDuckGo and EasyPrivacy. In doing so, Safari intentionally allows most ads to continue loading even in Private Browsing.

Private Browsing also blocks cloaked network requests to known tracking domains. They otherwise have the ability to save third party cookies in a first-party context. This protection requires macOS Sonoma or iOS 17. By cloaked we mean subdomains mapped to a third-party server via CNAME cloaking or third-party IP address cloaking. See also the “Defending Against Cloaked First Party IP Addresses” section below.

When Safari blocks a network request to a known tracker, a console message of this form is logged, and can be viewed using Web Inspector:

`Blocked connection to known tracker: tracker.example` 

Network Privacy Enhancements

Safari 15.0 started hiding IP addresses from known trackers by default. Private Browsing in Safari 17.0 adds the following protections for all users:

  • Encrypted DNS. DNS queries are used to resolve server hostnames into IP addresses, which is a necessary function of accessing the internet. However, DNS is traditionally unencrypted, and allows network operators to track user activity or redirect users to other servers. Private Browsing uses Oblivious DNS over HTTPS by default, which encrypts and proxies DNS queries to protect the privacy and integrity of these lookups.
  • Proxying unencrypted HTTP. Any unencrypted HTTP resources loaded in Private Browsing will use the same multi-hop proxy network used to hide IP addresses from trackers. This ensures that attackers in the local network cannot see or modify the content of Private Browsing traffic.

Additionally, for iCloud+ subscribers who have iCloud Private Relay turned on, Private Browsing takes privacy to the next level with these enhancements:

  • Separate sessions per tab. Every tab that the user opens in Private Browsing now uses a separate session to the iCloud Private Relay proxies. This means that web servers won’t be able to tell if two tabs originated on the same device. Each session is assigned egress IP addresses independently. Note that this doesn’t apply to parent-child windows that need a programmatic relationship, such as popups and their openers.
  • Geolocation privacy by default. Private Browsing uses an IP location based on your country and time zone, not a more specific location.
  • Warnings before revealing IP address. When accessing a server that is not accessible on the public internet, such as a local network server or an internal corporate server, Safari cannot use iCloud Private Relay. In Private Browsing, Safari now displays a warning requesting that the user consents to revealing their IP address to the server before loading the page.

Extensions in Private Browsing

Safari 17.0 also boosts the privacy of Extensions in Private Browsing. Extensions that can access website data and browsing history are now off by default in Private Browsing. Users can still choose to allow an extension to run in Private Browsing and gain all of the extension’s utility. Extensions that don’t access webpage contents or browsing history, like Content Blockers, are turned on by default in Private Browsing when turned on in Safari.

Advanced Fingerprinting Protection

With Safari and subsequently other browsers restricting stateful tracking (e.g. cross-site cookies), many trackers have turned to stateless tracking, often referred to as fingerprinting.

Types of Fingerprinting

We distinguish these types of fingerprinting:

  • Device fingerprinting. This is about building a fingerprint based on device characteristics, including hardware and the current operating system and browser. It can also include connected peripherals if they are allowed to be detected. Such a fingerprint cannot be changed by the user through settings or web extensions.
  • Network and geographic position fingerprinting. This is about building a fingerprint based on how the device connects to the Internet and any means of detecting its geographic position. It could be done by measuring roundtrip speeds of network requests or simply using the IP address as an identifier.
  • User settings fingerprinting. This is about reading the state of user settings such as dark/light mode, locale, font size adjustments, and window size on platforms where the user can change it. It also includes detecting web extensions and accessibility tools. We find this kind of fingerprinting to be extra hurtful since it exploits how users customize their web experience to fit their needs.
  • User behavior fingerprinting. This is about detecting recurring patterns in how the user behaves. It could be how the mouse pointer is used, how quickly they type in form fields, or how they scroll.
  • User traits fingerprinting. This is about figuring out things about the user, such as their interests, age, health status, financial status, and educational background. Those gleaned traits can contribute to a unique ID but also can be used directly to target them with certain content, adjust prices, or tailor messages.

Fingerprint Stability

A challenge for any tracker trying to create a fingerprint is how stable the fingerprint will be over time. Software version fingerprinting changes with software updates, web extension fingerprinting changes with extension updates and enablement/disablement, user settings change when the user wants, multiple users of the same device means behavior fingerprints change, and roaming devices may change network and geographic position a lot.

Fingerprinting Privacy Problem 1: Cross-Site Tracking

Fingerprints can be used to track the user across websites. If successful, it defeats tracking preventions such as storage partitioning and link decoration filtering.

There are two types of solutions to this problem:

  1. Make the fingerprint be shared among many users, so called herd immunity.
  2. Make the fingerprint unique per website, typically achieved via randomized noise injection.

Fingerprinting Privacy Problem 2: Per-Site User Recall

Less talked about is the fingerprinting problem of per-site user recall. Web browsers offer at least two ways for the user to reset their relationship with a website: Clear website data or use Private Browsing. Both make a subsequent navigation to a website start out fresh.

But fingerprinting defeats this and allows a website to remember the user even though they’ve opted to clear website data or use Private Browsing.

There are two types of solutions to this problem:

  1. Make the fingerprint be shared among many users, so called herd immunity.
  2. Make the fingerprint unique per website, and generate a new unique fingerprint for every fresh start.

Fingerprinting Privacy Problem 3: Per-Site Visitor Uniqueness

The ultimate anti fingerprinting challenge in our view is to address a specific user’s uniqueness when visiting a specific website. Here’s a simple example:

Having the locale setting to US/EN for American English may provide ample herd immunity in many cases. But what happens when a user with that setting visits an Icelandic government website or a Korean reading club website? They may find themselves in a very small “herd” on that particular website and combined with just a few more fingerprinting touch points they can be uniquely identified.

Addressing per-site visitor uniqueness is not possible in general by a browser unless it knows what the spread of visitors looks like for individual websites.

Fingerprinting Protections at a High Level

We view cross-site tracking and per-site user recall as privacy problems to be addressed by browsers.

Our approach:
Make the fingerprint unique per website, and generate a new unique fingerprint for every fresh start such as at website data removal.

Our tools:

  • Use multi-hop proxies to hide IP addresses and defend against network and geographic position fingerprinting.
  • Limit the number of fingerprintable web APIs whenever possible. This could mean altering the APIs, gating them behind user permissions, or not implementing them.
  • Inject small amounts of noise in return values of fingerprintable web APIs.

Fingerprinting Protection Details

Safari’s new advanced fingerprinting protections make it difficult for scripts to reliably extract high-entropy data through the use of several web APIs:

  1. To make it more difficult to reliably extract details about the user’s configuration, Safari injects noise into various APIs: namely, during 2D canvas and WebGL readback, and when reading AudioBuffer samples using WebAudio.
  2. To reduce the overall entropy exposed through other APIs, Safari also overrides the results of certain web APIs related to window or screen metrics to fixed values, such that fingerprinting scripts that call into these APIs for users with different screen or window configurations will get the same results, even if the users’ underlying configurations are different.

2D Canvas and WebGL

Many modern web browsers use a computer’s graphics processing unit (GPU) to accelerate rendering graphics. The Web’s Canvas API (2D Canvas) and WebGL API give a web page the tools it needs for rendering arbitrary images and complex scenes using the GPU, and analyzing the result. These APIs are valuable for the web platform, but they allow the web page to learn unique details about the underlying hardware without asking for consent. With Safari’s advanced fingerprinting protections enabled, Safari applies tiny amounts of noise to pixels on the canvas that have been painted using drawing commands. These modifications reduce the value of a fingerprint when using these APIs without significantly impacting the rendered graphics.

It’s important to emphasize that:

  1. This noise injection only happens in regions of the canvas where drawing occurs.
  2. The amount of noise injected is extremely small, and (mostly) should not result in observable differences or artifacts.

This strategy helps mitigate many of the compatibility issues that arise from this kind of noise injection, while still maintaining robust fingerprinting mitigations.

In Safari 17.5, we’ve bolstered these protections by additionally injecting noise when reading back data from offscreen canvas in both service workers and shared workers.

Web Audio

Similarly, when reading samples using the WebAudio API — via AudioBuffer.getChannelData() — a tiny amount of noise is applied to each sample to make it very difficult to reliably measure OS differences. In practice, these differences are already extremely minor. Typically due to slight differences in the order of operations when applying FFT or IFFT. As such, a relatively low amount of noise can make it substantially more difficult to obtain a stable fingerprint.

In Safari 17.5, we made audio noise injection more robust in the following ways:

  • The injected noise now applies consistently to the same values in a given audio buffer — this means a looping AudioSourceNode that contains a single high-entropy sample can’t be used to average out the injected noise and obtain the original value quickly.
  • Instead of using a uniform distribution for the injected noise, we now use normally-distributed noise. The mean of this distribution converges much more slowly on the original value, when compared to the average of the minimum and maximum value in the case of uniformly-distributed noise.
  • Rather than using a low, fixed amount of noise (0.1%), we’ve refactored the noise injection mechanism to support arbitrary levels of noise injection. This allows us to easily fine-tune noise injection, such that the magnitude of noise increases when using audio nodes that are known to reveal subtle OS or hardware differences through minute differences in sample values.

This noise injection also activates when using Audio Worklets (e.g. AudioWorkletNode) to read back audio samples.

Screen/Window Metrics

Lastly, for various web APIs that currently directly expose window and screen-related metrics, Safari takes a different approach: instead of the noise-injection-based mitigations described above, entropy is reduced by fixing the results to either hard-coded values, or values that match other APIs.

  • screen.width / screen.height: The screen size is fixed to the values of innerWidth and innerHeight.
  • screenX / screenY: The screen position is fixed to (0, 0).
  • outerWidth / outerHeight: Like screen size, these values are fixed to innerWidth and innerHeight.

These mitigations also apply when using media queries to indirectly observe the screen size.

Don’t Add Fingerprintable APIs to the Web, Like The Topics API

We have worked for many years with the standards community on improving user privacy of the web platform. There are existing web APIs that are fingerprintable, such as Canvas, and reining in their fingerprintability is a long journey. Especially since we want to ensure existing websites can continue to work well.

It is key for the future privacy of the web to not compound the fingerprinting problem with new, fingerprintable APIs. There are cases where the tradeoff tells us that a rich web experience or enhanced accessibility motivates some level of fingerprintability. But in general, our position is that we should progress the web without increasing fingerprintability.

A recent example where we opposed a new proposal is the Topics API which is now shipping in the Chrome browser. We provided extensive critical feedback as part of the standards process and we’d like to highlight a few pieces here.

The Topics API in a Nutshell

From the proposal:

// document.browsingTopics() returns an array of up to three topic objects in random order.
const topics = await document.browsingTopics();

Any JavaScript can call this function on a webpage. Yes, that includes tracker scripts, advertising scripts, and data broker scripts.

The topics come from a predefined list of hundreds of topics. It’s not the user who picks from these topics, but instead Chrome will record the user’s browsing history over time and deduce interests from it. The user doesn’t get told upfront which topics Chrome has tagged them with or which topics it exposes to which parties. It all happens in the background and by default.

The intent of the API is to help advertisers target users with ads based on each user’s interests even though the current website does not necessarily imply that they have those interests.

The Fingerprinting Problem With the Topics API

A new research paper by Yohan Beugin and Patrick McDaniel from University of Wisconsin-Madison goes into detail on Chrome’s actual implementation of the Topics API.

The authors use large scale real user browsing data (voluntarily donated) to show both how the 5% noise supposed to provide plausible deniability for users can be defeated, and how the Topics API can be used to fingerprint and re-identify users.

“We conclude that an important part of the users from this real dataset are re-identified across websites through only the observations of their topics of interest in our experiment. Thus, the real users from our dataset can be fingerprinted through the Topics API. Moreover, as can be seen, the information leakage and so, privacy violation worsen over time as more users are uniquely re-identified.” —Beugin and McDaniel, University of Wisconsin-Madison

The paper was published at the 2024 IEEE Security and Privacy Workshops (SPW) in May.

Further Privacy Problems With the Topics API

Re-identifying and tracking users is not the only privacy problem with the Topics API. There is also the profiling of users’ cross-site activity. Here’s an example using topics on Chrome’s predefined list.

Imagine in May 2024 you go to news.example where you are a subscriber and have provided your email address. Embedded on the website, dataBroker.example. The data broker has gleaned your email address from the login form and calls the Topics API to learn that you currently have these interests:

  • Flowers
  • Event & Studio Photography
  • Luxury Travel

In May 2026 you go to news.example where dataBroker.example calls the Topics API and is told that you now have these interests:

  • Children’s Clothing
  • Family Travel
  • Toys

Finally, in May 2029 you go to news.example where dataBroker.example calls the Topics API and is told that you have these interests:

  • Legal Services
  • Furnished Rentals
  • Child Care

You haven’t told any website with access to your email address anything that’s been going on in your family life. But the data broker has been able to read your shifting interests and store them in their permanent profile of you — while you were reading the news.

Now imagine what advanced machine learning and artificial intelligence can deduce about you based on various combinations of interest signals. What patterns will emerge when data brokers and trackers can compare and contrast across large portions of the population? Remember that they can combine the output of the Topics API with any other data points they have available, and it’s the analysis of all of it together that feeds the algorithms that try to draw conclusions about you.

We think the web should not expose such information across websites and we don’t think the browser, i.e. the user agent, should facilitate any such data collection or use.

Privacy Enhancements in Both Browsing Modes

Our defenses against cloaked third-party IP addresses and our partitioning of SessionStorage and blob URLs are enabled by default in both regular browsing and Private Browsing. Here’s how those protections work.

Defending Against Cloaked First Party IP Addresses

In 2020, Intelligent Tracking Prevention (ITP) gained the ability to cap the expiry of cookies set in third-party CNAME-cloaked HTTP responses to 7 days.

This defense did not mitigate cases where IP aliasing is used to cloak third party requests under first party subdomains. ITP now also applies a 7-day cap to the expiry of cookies in responses from cloaked third-party IP addresses. Detection of third-party IP addresses is heuristic, and may change in the future. Currently, two IP addresses are considered different parties if any of the following criteria are met:

  1. One IP address is IPv4, while the other is IPv6.
  2. If both addresses are IPv4, the length of the common subnet mask is less than 16 bits (half of the full address length).
  3. If both addresses are IPv6, the length of the common subnet mask is less than 64 bits (also half of the full address length).

Partitioned SessionStorage and Blob URLs

Websites have many options for how they store information over longer time periods. Session Storage is a storage area in Safari that is scoped to the current tab. When a tab in Safari is closed, all of the session storage associated with it is destroyed. Beginning in Safari 16.1 cross-site Session Storage is partitioned by first-party web site.

Similarly, Blobs are a storage type that allow websites to store raw, file-like data in the browser. A blob can hold almost anything, from simple text to something larger and more complex like a video file. A unique URL can be created for a blob, and that URL can be used to gain access to the associated blob, as long as the blob still exists. These URLs are often referred to as Blob URLs, and a Blob URL’s lifetime is scoped to the document that creates it. Beginning in Safari 17.2, cross-site Blob URLs are partitioned by first-party web site, and first-party Blob URLs are not usable by third parties.

Setting a New Industry Standard

The additional privacy protections of Private Browsing in Safari 17.0, Safari 17.2 and Safari 17.5 set a new bar for user protection. We’re excited for all Safari users and the web itself to benefit from this work!

Feedback

We love hearing from you! To share your thoughts on Private Browsing 2.0, find John Wilander on Mastodon at @wilander@mastodon.social or send a reply on X to @webkit. You can also follow WebKit on LinkedIn. If you run into any issues, we welcome your feedback on Safari UI (learn more about filing Feedback), or your WebKit bug report about web technologies or Web Inspector.

July 16, 2024 04:00 PM

July 12, 2024

Georges Stavracas: Profiling a web engine

Igalia WebKit

One topic that interests me endlessly is profiling. I’ve covered this topic many times in this blog, but not enough to risk sounding like a broken record yet. So here we are again!

Not everyone may know this but GNOME has its own browser, Web (a.k.a. Epiphany, or Ephy for the intimates). It’s a fairly old project, descendant of Galeon. It uses the GTK port of WebKit as its web engine.

The recent announcement that WebKit on Linux (both WebKitGTK and WPE WebKit) switched to Skia for rendering brought with it a renewed interest in measuring the performance of WebKit.

And that was only natural; prior to that, WebKit on Linux was using Cairo, which is entirely CPU-based, whereas Skia had both CPU and GPU-based rendering easily available. The CPU renderer mostly matches Cairo in terms of performance and resource usage. Thus one of the big promises of switching to Skia was better hardware utilization and better overall performance by switching to the GPU renderer.

A Note About Cairo

Even though nowadays we often talk about Cairo as a legacy piece of software, there’s no denying that Cairo is really good at what it does. Cairo can and often is extremely fast at 2D rendering on the CPU, specially for small images with simple rendering. Cairo has received optimizations and improvements for this specific use case for almost 20 years, and it is definitely not a low bar to beat.

I think it’s important to keep this in mind because, as tempting as it may sound, simply switching to use GPU rendering doesn’t necessarily imply better performance.

Guesswork is a No-No

Optimizations should always be a byproduct of excellent profiling. Categorically speaking, meaningful optimizations are a consequence of instrumenting the code so much that the bottlenecks become obvious.

I think the most important and practical lesson I’ve learned is: when I’m guessing what are the performance issues of my code, I will be wrong pretty much 100% of the time. The only reliable way to optimize anything is to have hard data about the behavior of the app.

I mean, so many people – myself included – were convinced that GNOME Software was slow due to Flatpak that nobody thought about looking at app icons loading.

Enter the Profiler

Thanks to the fantastic work of Søren Sandmann, Christian Hergert, et al, we have a fantastic modern system profiler: Sysprof.

Sysprof offers a variety of instruments to profile the system. The most basic one uses perf to gather stack traces of the processes that are running. Sysprof also supports time marks, which allow plotting specific events and timings in a timeline. Sysprof also offers extra instrumentation for more specific metrics, such as network usage, graphics, storage, and more.

  • Screenshot of Sysprof's callgraph viewCallgraph
  • Screenshot of Sysprof's flamegraphs viewFlamegraphs
  • Screenshot of Sysprof's mark chart viewMark chart
  • Screenshot of Sysprof's waterfall viewMark waterfall

All these metrics are super valuable when profiling any app, but they’re particularly useful for profiling WebKit.

One challenging aspect of WebKit is that, well, it’s not exactly a small project. A WebKit build can easily take 30~50min. You need a fairly beefy machine to even be able to build a debug build of WebKit. The debug symbols can take hundreds of megabytes. This makes WebKit particularly challenging to profile.

Another problem is that Sysprof marks require integration code. Apps have to purposefully link against, and use, libsysprof-capture to send these marks to Sysprof.

Integrating with Sysprof

As a first step, Adrian brought the libsysprof-capture code into the WebKit tree. As libsysprof-capture is a static library with minimal dependencies, this was relatively easy. We’re probably going to eventually remove the in-tree copy and switch to host system libsysprof-capture, but having it in-tree was enough to kickstart the whole process.

Originally I started sprinkling Sysprof code all around the WebKit codebase, and to some degree, it worked. But eventually I learned that WebKit has its own macro-based tracing mechanism that is only ever implemented for Apple builds.

Looking at it, it didn’t seem impossible to implement these macros using Sysprof, and that’s what I’ve been doing for the past few weeks. The review was lengthy but behold, WebKit now reports Sysprof marks!

Screenshot of Sysprof with WebKit marks highlighted

Right now these marks cover a variety of JavaScript events, layout and rendering events, and web page resources. This all came for free from integrating with the preexisting tracing mechanism!

This gives us a decent understanding of how the Web process behaves. It’s not yet complete enough, but it’s a good start. I think the most interesting data to me is correlating frame timings across the whole stack, from the kernel driver to the compositor to GTK to WebKit’s UI process to WebKit’s Web process, and back:

Screenshot of Sysprof with lots of compositor and GTK and WebKit marks

But as interesting as it may be, oftentimes the fun part of profiling is being surprised by the data you collect.

For example, in WebKit, one specific, seemingly innocuous, completely bland method is in the top 3 of the callgraph chart:

Screenshot of Sysprof showing the callgraph view with an interesting result highlighted

Why is WebCore::FloatRect::contains so high in the profiling? That’s what I’m investigating right now. Who guessed this specific method would be there? Nobody, as far as I know.

Once this is out in a stable release, anyone will be able to grab a copy of GNOME Web, and run it with Sysprof, and help find out any performance issues that only reproduce in particular combinations of hardware.

Next Plans

To me this already is a game changer for WebKit, but of course we can do more. Besides the rectangular surprise, and one particular slowdown that comes from GTK loading Vulkan on startup, no other big obvious data point popped up. Specially in the marks, I think their coverage is still fairly small compared to what it could have been.

We need more data.

Some ideas that are floating right now:

  • Track individual frames and correlate them with Sysprof marks
  • Measure top-to-bottom-to-top latency
  • Measure input delay
  • Integrate with multimedia frames

Perhaps this will allow us to make WebKit the prime web engine for Linux, with top-tier performance, excellent system integration, and more. Maybe we can even redesign the whole rendering architecture of WebKit on Linux to be more GPU friendly now. I can dream high, can’t I? 🙂

In any case, I think we have a promising and exciting time ahead for WebKit on Linux!

By Georges Stavracas at July 12, 2024 12:42 PM

July 03, 2024

Release Notes for Safari Technology Preview 198

Surfin’ Safari

Safari Technology Preview Release 198 is now available for download for macOS Sequoia beta and macOS Sonoma. If you already have Safari Technology Preview installed, you can update it in System Settings under General → Software Update.

This release includes WebKit changes between: 279855@main…280286@main.

CSS

New Features

  • Added support for the :active-view-transition pseudo-class. (280027@main) (129851076)

Forms

Resolved Issues

  • Fixed input elements to use the [value] as the first fallback step base. (280127@main) (107721910)

JavaScript

Resolved Issues

  • Fixed V128 values to be throwable to match the specification. (279989@main) (106030051)

Media

Resolved Issues

  • Fixed missing picture-in-picture when leaving Safari with picture-in-picture enabled from video viewer. (279877@main) (124376686)
  • Fixed the context menu to show “Exit Viewer” when using video viewer. (280128@main) (126300924)

Rendering

Resolved Issues

  • Fixed scrolling in a flexbox container. (279992@main) (129343181)

Safari Extensions

Resolved Issues

  • Fixed throwing an exception when calling browser.storage.set with undefined as a map value. (280265@main) (130282050)

SVG

Deprecations

  • Removed non-standard getTransformToElement from SVGGraphicsElement (280208@main) (122435702)

Web API

Resolved Issues

  • Fixed an issue when inserting writing suggestions into an editable display: grid container. (279986@main) (129366300)

Web Inspector

Resolved Issues

  • Fixed disappearing objects and disclosure triangles when Web Inspector is at narrow widths. (279894@main) (121861970)
  • Fixed the list of breakpoints in the Sources tab disappearing when Web Inspector is reloaded. (279884@main) (123641994)

WebRTC

Resolved Issues

  • Fixed RTCEncodedVideoFrame and RTCEncodedAudioFrame to match the WebIDL specification. (280198@main) (118607685)

July 03, 2024 10:47 PM

June 24, 2024

Release Notes for Safari Technology Preview 197

Surfin’ Safari

Safari Technology Preview Release 197 is now available for download for macOS Sequoia beta and macOS Sonoma. If you already have Safari Technology Preview installed, you can update it in System Settings under General → Software Update.

This release includes WebKit changes between: 279361@main…279854@main.

Canvas

Resolved Issues

  • Fixed a detached OffscreenCanvas to not transfer an ImageBuffer. (279775@main) (129270155)

CSS

Resolved Issues

  • Fixed custom counter styles disclosure-open and disclosure-closed to point to the correct direction in right-to-left. (279404@main) (109014745)

Forms

Resolved Issues

  • Fixed form controls drawing with an active appearance when the window is inactive. (279458@main) (127391198)

JavaScript

Resolved Issues

  • Fixed fractionalDigits of Intl.DurationFormat to be treated as at most 9 digits if it is omitted. (279632@main) (129145390)

Media

Resolved Issues

  • Fixed multiple cases of audio distortion occurring when using AudioWorklets. (279409@main) (128551401)

Rendering

Resolved Issues

  • Fixed min-content calculation for unstyled only-child inlines elements. (273633@main) (128348427)

  • Fixed ellipsis rendering multiple times when position: relative and top are used. (279645@main) (128394449)

  • Fixed a bug for inline elements inserted in reverse order after a block in a continuation. (279422@main) (128826228)

  • Fixed the flash of a page background-colored bar in the footer when the window is resized. (279562@main) (128940179)

Security

Resolved Issues

  • Fixed CORS bypass on private localhost domain using 0.0.0.0 host and mode “no-cors”. (279835@main) (125913679)

SVG

Resolved Issues

  • Fixed error handling for invalid filter primitive references. (279421@main) (104262208)

  • Fixed SVGLength to sync with the WebIDL specification. (279659@main) (129169603)

Web Inspector

Resolved Issues

  • Fixed Web Inspector to show nested workers. (279793@main) (108322385)

  • Fixed Accessibility inspector for switch controls to report “State: on/off” instead of “Checked: true/false”. (279772@main) (128952449)

WebDriver

Resolved Issues

  • Fixed retrieving titles containing multibyte characters. (279767@main) (123987149)

WebGL

Resolved Issues

  • Fixed OffscreenCanvas.transferToImageBuffer() to clear the WebGL drawing buffer. (279434@main) (126738038)

June 24, 2024 09:37 PM

June 12, 2024

Try out your website in the spatial web

Surfin’ Safari

Now that visionOS 2 is announced, it’s a great time to ensure your website works in Safari in visionOS. For the most part, there’s nothing special you need to do for the spatial web. It’s truly Safari, with the same WebKit engine and its extensive support for web standards.

You can see how Safari in visionOS works by watching the Apple Vision Pro introduction at the WWDC23 Keynote or the session Meet Safari for spatial computing. If you don’t have your own Apple Vision Pro, and you want to experience what it’s like in person, you can book a free demo of Apple Vision Pro at your local Apple Store (in available regions).

Let’s take a look at how to use Safari’s suite of developer tools to test, inspect, and debug your site. First, you’ll learn how to use Safari developer tools with the visionOS simulator. Then, you’ll get a walkthrough of pairing Safari and Web Inspector on your Mac with Apple Vision Pro. These instructions work with for visionOS 1.2 and visionOS 2 so you can debug even if you do not have an Apple developer account.


Develop with the visionOS simulator

The Mac is a powerful machine — powerful enough to run macOS, iOS, watchOS, tvOS, visionOS, and iPadOS at the same time. It’s here where app developers work on their apps, write code, and test their code in the appropriate Simulator.

iPhone simulator showing 9:41, watchOS simulator showing 9:41, and a visionOS simulator

You can test the website you are building in Safari inside Simulator on macOS. This lets you develop the site on your Mac, using your full development environment, with your favorite IDE, Terminal, and more. When you’re ready to test, use Web Inspector on Mac to inspect web pages running in its simulator.

Web Inspector to the left of a visionOS simulator

At WWDC24, Apple announced visionOS 2 beta available now to developers with an Apple Developer Program membership. Xcode 16 beta includes the visionOS 2 beta simulator with Safari 18 beta. Or test in visionOS 1.2 with Safari 17.5 by downloading Xcode 15 and installing its simulator.

Get setup

Here’s how to get started using the visionOS simulator:

  1. Download Xcode 15 from the Mac App Store. (It’s free.) Or download Xcode 15 or Xcode 16 beta from the Xcode website.
  2. Open Xcode. When asked which platforms you would like to develop for, enable visionOS. If you’d like to also install Simulator for iPhone and iPad, enable iOS.
    Xcode modal to selecct the components to get started with: macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0, Predictive Code Completion Model; Required additional components will also be installed. Dowload & Install button.
  3. The visionOS simulator will start downloading.
  4. Open the Simulator app on macOS. Xcode doesn’t need to be running to use Simulator.
  5. You will see a visionOS simulator named “Apple Vision Pro”.
  6. You can use the File menu anytime to create a new simulator or open existing simulators.
  7. Click on the Safari icon in the visionOS Home View to open Safari. Enter a URL in the navigation bar to go to a website.

Here’s how to get started using Safari’s developer tools on macOS:

  1. Open Safari on macOS.
  2. If you don’t see a Develop menu in the Menu Bar, you’ll need to enable Safari’s developer tools.
  3. Go to Safari > Settings > Advanced, and enable “Show features for web developers”.
  4. Now you can see “Apple Vision Pro (Simulator)” listed in the Develop menu.

Once you’ve completed this setup, you won’t need to repeat these steps again.

Open a web page in Safari in a visionOS simulator

You can use Safari in visionOS to navigate to a web page, but it’s likely you already have the page open in Safari on macOS. So instead, use this tip to quickly send web pages from macOS to the visionOS simulator:

  1. In Safari on macOS, go to the web page you’d like to test in the visionOS simulator.
  2. In the Safari menu bar, go to Develop > Open Page With > Apple Vision Pro.

The visionOS simulator comes to the front, and the web page opens in Safari. This also works for any number of other simulators for iPhone and iPad, which you can always install later.

Develop menu showing the Open Page With menu selected with Apple Vision Pro sub-menu option selected.

Learn more about using the visionOS simulator.

Inspect a visionOS simulator with Web Inspector

Web Inspector is instrumental when developing or troubleshooting web content in a simulator. To use it for a web page open in Safari in Apple Vision Pro Simulator. Click the Develop menu, select Apple Vision Pro (Simulator), and select the web page you want to inspect.

Develop menu showing Apple Vision Pro (Simulator) selected with a submenu showing web pages open in Safari in the simulator

Web Inspector opens connected to the page loaded in Safari in the visionOS simulator. This brings all of the powerful inspection and debugging tools of Web Inspector to your web content loaded in Safari in any simulator. It also works for connecting to web pages loaded in Safari on iPhone, or iPad simulators.

Preview WebXR and with Feature Flags

Apple announced WebXR support with visionOS 2. If you’re using visionOS 1.2 in Apple Vision Pro or in the visionOS simulator on your Mac, you can still experience WebXR by turning on support for the testable version. You can also preview other new features like the <model> element.

To enable support for WebXR or the <model> element, from the the Home View, go to Settings > Apps > Safari > Advanced > Feature Flags and enable the feature flags for “HTML element” or “WebXR”.


Develop with Apple Vision Pro

Apple Vision Pro is a great place to work on a website. Use Mac Virtual Display to bring your Mac web development workflows into Apple Vision Pro. Test your site in Safari in visionOS, while debugging in Safari’s Web Inspector on macOS.

A view from inside Apple Vision Pro where two giant windows float in a real office. The first is a Safari window with the Chess Garden website showing. The second is macOS, with multiple windows inside the Mac window — working on developing the Chess Garden website using Web Inspector and more.

Start a Mac Virtual Display session

Use Apple Vision Pro as a private, portable 4K display for your Mac with Mac Virtual Display. Learn how in Use your Mac with Apple Vision Pro.

Enable Web Inspector for Apple Vision Pro

To get started connecting Safari in Apple Vision Pro with Web Inspector on macOS, you’ll need to go through a one-time set-up process that ensures only your Mac and your Apple Vision Pro have this kind of access to each other.

  1. In Safari on macOS, if the Develop menu is missing, enable features for web developers.
  2. Make sure both Apple Vision Pro and your Mac are connected to the same network.
  3. On Apple Vision Pro, open Settings > Apps > Safari > Advanced, and enable Web Inspector. Then open Settings > General > Remote Devices, and choose your Mac to prepare visionOS for pairing.
  4. On Mac, open the Develop menu. You should see your Apple Vision Pro listed. Choose “Use for Development…”
  5. A six-digit PIN code will appear in visionOS. Type it into the form on macOS.

Now your devices are paired. You can reconnect anytime. In Safari on macOS, Apple Vision Pro will now appear in the Develop menu anytime your Apple Vision Pro is on and unlocked.

Alternatively, you can pair with Xcode.

Inspect content from macOS Safari

Now you can inspect a web page that’s running in Safari in Apple Vision Pro using Web Inspector in Safari on macOS:

  1. In Safari in visionOS, go to the web page you want to inspect.
  2. On macOS, go to the Develop menu. You’ll see your Apple Vision Pro listed.
  3. Select the page you want to inspect from the submenu.

A Web Inspector window will open on macOS. Any changes you make will appear in Safari in visionOS.

For more information, read Inspecting visionOS.

Learn more

Learn more about how to download Simulator, pair Apple Vision Pro with Mac, or use Web Inspector while creating content for visionOS by watching Rediscover Safari developer features or by reading Inspecting visionOS.

Note: Learn more about Web Inspector from the Web Inspector Reference documentation.

June 12, 2024 02:30 PM

June 10, 2024

News from WWDC24: WebKit in Safari 18 beta

Surfin’ Safari

The last year has been a great one for WebKit. After unveiling Safari 17 beta at WWDC23, we’ve shipped six releases of Safari 17.x with a total of 200 new web technologies. And we’ve been hard at work on multiple architectural improvement projects that strengthen WebKit for the long-term.

Now, we are pleased to announce WebKit for Safari 18 beta. It adds another 48 web platform features, as well as 18 deprecations and 174 bug fixes. Test it today on iOS 18 beta, iPadOS 18 beta, visionOS 2 beta, and macOS Sequoia beta.

WebXR

Safari 18 for visionOS 2 beta adds support for immersive WebXR. Now you can create fully immersive experiences and deliver them on the web to people using Apple Vision Pro. Safari on visionOS 2 beta supports immersive-vr sessions. WebXR scenes are displayed using hardware-accelerated graphics driven by WebGL.

A beautiful garden rendered in created graphics. There's a tree with bright red leaves. A blue sky full of puffy white clouds. Bright green grass, with a path leading by plants and garden sculpture. It's a world created in WebXR.Try out the Chess Garden demo to see WebXR in action.

Safari for visionOS 2 beta supports the new WebXR transient-pointer input mode. It lets you make the most of natural input on visionOS, and allow your users to interact with a look and a pinch.

We are in a rendered 3d environment, in a garden. We look at a chess board, with a real human hand lifting a rendered chess piece to make the next move in the game. A floating panel has two buttons reading Use transient-pointer in your WebXR experience to support interactions like this, where players use their hands to pinch and drag as they move chess pieces around.

If you want to animate a 3D model of the user’s hands, Safari for visionOS 2 beta also includes support for WebXR hand tracking. To ensure privacy, permission to allow hand tracking will be requested from users at the start of their WebXR session.

Learn all about WebXR on visionOS 2 beta by watching Build immersive web experiences with WebXR at WWDC24, available Wednesday June 12. And learn more about transient-pointer input mode by reading Introducing natural input for WebXR in Apple Vision Pro.

CSS

View Transitions

WebKit added support for the View Transitions API in Safari 18 beta. It provides an optimized browser API to animate elements from one state to another. Safari supports the CSS View Transitions Module Level 1 specification that adds new CSS properties and pseudo-elements for defining transition animations, along with a new browser API to start transition animations and react to different transition states. It works by capturing the current (old) state of the page and applying an animated transition to the new state. By default, the browser applies a cross-fade between the states.

Call the document.startViewTransition() method to initiate the capture. You can pass a callback function as the first argument to make DOM state changes between the old and new captures. The method returns a ViewTransition object which contains promises that can be used to track when the view transition starts or ends.

Once the states are captured, a pseudo-element tree is built which can be targeted with CSS, allowing you to modify the CSS animations used for the transitions. The animations out of the old page state and into the new page state can be modified via the ::view-transition-new(*) and ::view-transition-old(*) selectors. You can also ask the browser to independently track state changes for a specific element by naming it with the CSS view-transition-name property. You can then use the pseudo-element to customize animations for it.

.page-view {
    view-transition-name: page-view;
}
::view-transition-old(page-view) {
    animation: 500ms ease-in-out transition-out-animation;
}
::view-transition-new(page-view) {
    animation: 500ms ease-in-out transition-in-animation;
}

The example below demonstrates state management with tabbed navigation. Each tab view has a custom transition animation out and a subtly different animation in, while the tabs themselves rely on the default page transition.

See View Transitions in action in this demo.

Style Queries

WebKit for Safari 18 beta adds support for Style Queries when testing CSS Custom Properties. Similar to how developers can use Sass mixins, Style Queries can be used to define a set of reusable styles that get applied as a group.

Here, if the --background custom property is set to black, then certain styles will be applied — in this case to make the headline and paragraph text color white.

@container style(--background: black) {
  h2, h3, p {
    color: white;
  }
}

Don’t forget to pay attention the HTML structure. By default, Style Queries reference the styles on the direct parent element. You can create a different reference through the use of Container Query names.

currentcolor and system color keywords in Relative Color Syntax

Support for Relative Color Syntax shipped in Safari 16.4. It lets you define colors in a more dynamic fashion, creating a new color from an existing color. The value lch(from var(--color) calc(L / 2) C H) for instance uses the lch color space to take the variable --color and calculate a new color that’s half its lightness, calc(L / 2).

Now, starting in WebKit for Safari 18 beta, you can reference the currentcolor or a system color keyword as you define the new color. For example, this code will set the background color to be the same color as the text color, only 4 times lighter, as calculated in the oklch color space.

section { background: oklch(from currentcolor calc(L * 4) C H); }

See how `currentcolor` works with Relative Color Syntax in this demo.

Being able to reference system color keywords opens up another world of options. System colors are like variables that represent the default colors established by the OS, browser, or user — defaults that change depending on whether the system is set to light mode, dark mode, high contrast mode, etc. For example, canvas represents the current default background color of the HTML page, while fieldtext matches the color of text inside form fields. Find the full list of system colors in CSS Color level 4.

Relative Color Syntax lets you define dynamic connections between colors in your CSS, lessening the need to control color through variables in a tightly-regimented design system. Learn more about Relative Color Syntax by watching this portion of What’s new in CSS from WWDC23.

Animating display

WebKit for Safari 18 beta adds support for transition animation of the display property.

Many developers are excited to use @starting-style along with transition-behavior and display: none interpolation. WebKit for Safari 17.4 added general support for transition-behavior, including transition-behavior: allow-discrete. WebKit for Safari 17.5 added support for @starting-style, letting you define starting values for transitioning an element as it’s created (or re-created). Now in WebKit for Safari 18 beta, you can use these features together to transition the display property.

Shaping interaction regions on visionOS

As a web developer, you’re very familiar with how link styling works on the web. For decades you’ve been able to use CSS to style text-decoration, color and more for :link, :hover, :active, and :visited states. You’ve also been able to adjust the size of the invisible tap target through use of padding.

Apple Vision Pro adds a new dimension to how links work — tap targets are visible on visionOS. Anytime a user looks at an interactive element, it’s highlighted to let them know that it can be tapped. And you as a designer or developer can intentionally design how an interaction region looks. You may want to add padding, for instance, or even a rounded corner to the otherwise invisible box.

Now in Safari in visionOS 2 beta, when you use CSS clip-path to change the shape of tappable area of a link, the visible interaction region will change shape as well. Interactive UI elements built with SVG will also be highlighted with the proper shape. Learn more by watching Optimize for the spatial web at WWDC24, available Tuesday June 11.

Backdrop Filter

Originally shipped in Safari 9.0, backdrop filter provides a way to apply graphics effects to the content behind a particular element. You can apply backdrop-filter to a headline, for example, and everything behind the headline will be blurred, or have decreased saturation, or increased contrast. Any of the filter functions from SVG can be used — blur(), brightness(), contrast(), drop-shadow(), grayscale(), hue-rotate(), invert(), opacity(), saturate(), and sepia().

For many years, backdrop filter only worked in Safari. It was available when you prefixed the property with -webkit-backdrop-filter. Now, starting in Safari 18 beta, you don’t need the prefix. We also improved our implementation, fixing bugs and boosting interoperability.

See what’s possible with backdrop filter in this demo. Use the dropdown to switch filters.

This demo shows eight different filters and what you might do with each one alone. You can, of course, combine filters to create even more interesting results. With backdrop filter supported in Safari since 2015, Edge since 2018, Chrome since 2019, Samsung Internet since 2020, and Firefox since 2022, this is a great time to consider the kind of graphic design possibilities it enables.

safe in Flexbox

WebKit for Safari 18 beta adds support for the safe keyword for alignment in Flexbox. This provides a mechanism for refining how flex items overflow. Let’s look at an example of a simple navigation menu — a classic use of Flexbox.

<nav aria-label="main"><ul>
  <li><a href="/us">U.S.</a></li>
  <li><a href="/business">Business</a></li>
  <li><a href="/investigations">Investigations</a></li>
  <li><a href="/style">Style</a></li>
  <li><a href="/tech">Tech</a></li>
  <li><a href="/world">World</a></li>
</ul></nav>

The following CSS creates a simple layout that wraps when there’s not enough space on one line for the menu, while centering the items in the available space.

header nav {
  display: flex;
  flex-flow: wrap;
  gap: 1.5rem;
  justify-content: center; /* centers the items in the available space */
}
A simple menu of links, each represented by a word, laid out in two lines of centered text.

By default, justify-content: center will always keep the items centered, even when the content is overflowing the containing box. You might prefer, however, that the content not be centered when it overflows — being centered cuts off both the beginning and end of the word, making the content harder to understand when the overflow is not visible.

Diagram showing the difference between safe and default layout of the same menu, when the space for it is so narrow every word in on its own line, and some of the long words start to get chopped off. Compare justify-content: center without and with the safe keyword in this demo.

The safe keyword lets you change how alignment works when content overflows. The justify-content: safe center rule will instead start align any item that is overflowing, while continuing to center the items that are not overflowing.

If you want to override the safe keyword, you can use unsafe. The justify-content: unsafe center rule will do the same thing as justify-content: center. The unsafe keyword has been supported in WebKit for Safari for quite some time.

Content visibility

WebKit for Safari 18 beta adds support for content-visibility. This property controls whether or not an element renders its contents in a fashion that’s useful for making performance optimizations. It lets you communicate to the browser that certain portions of the page will likely be initially offscreen, and suggest they be omitted from layout and rendering. This can make the page load faster.

Web apps for Mac

Last year, we added support for web apps in macOS Sonoma. You can add any website to your dock — whether or not it was built with a Manifest file, Service Worker, or other technology to customize the web app experience. Go to the site in Safari, then File > Add to Dock… where you can customize the icon, change the name, and even clean up the URL. Then, just click on the web app icon in your Dock, and it will open as an app.

This year brings two improvements to web apps on Mac.

Opening links

macOS Sequoia beta adds support for opening links directly in web apps. Now, when a user clicks a link, if it matches the scope of a web app that the user has added to their Dock, that link will open in the web app instead of their default web browser.

For example, imagine you have added MDN Web Docs to the Dock. Then a colleague sends you a link to an MDN page in Messages, Mail, Slack, Discord, IRC, or any non-browser application on your Mac. Now when you click on that link, it will open in the MDN Web Docs web app instead of your default browser.

Clicking a link within a browser will maintain the current behavior. This feature only affects links opened elsewhere. (When a user is in Safari, clicking on a link that matches the scope of a web app that is added to Dock, they will see an “Open in web app” banner, unless they have previously dismissed the banner.)

By default, this behavior applies when the link matches the host of the web page used to create the web app. As a developer, you can refine this experience by defining the range of URLs that should open in the web app with the scope member in the web app manifest.

Extension support

Now you can personalize web apps on Mac with Safari Web Extensions and Content Blockers. Navigate to the web app’s Settings menu to access all your installed Content Blockers and Web Extensions. Any enabled in Safari will be on by default in the web app. Each web app is uniquely customizable, just like Safari profiles.

Safari Extensions

Safari 18 beta also adds support for Mobile Device Management of extension enabled state, private browsing state, and website access on managed devices. This means schools and businesses that manage iOS, iPadOS, or macOS devices can now include the configuration of Safari App Extensions, Content Blockers, and Web Extensions in their management.

Spatial media

One of the amazing experiences you can have on Apple Vision Pro is looking at spatial photos and panoramas. When you open the Photos app in visionOS, you see a montage of your photos. Tap an image, it appears alone in a floating frame in front of you, while the rest of the app disappears.

A family blows out candles on a birthday cake in a photo — that's floating in a frame in midair, in a living room. This is a still from the WWDC23 Keynote that introduced Apple Vision Pro. It's an example of how spatial photos work.

A spatial photo appears at just the right height and viewing angle to make it feel like you’ve gone back to a moment in time. A second tap of the UI breaks it out of the frame, as it grows and becomes even more immersive. Similarly, a panorama floats in a frame on first tap. Then on second tap of the UI, it expands to wrap all around you, creating a fully immersive experience.

Now in Safari 18 for visionOS 2 beta, you can use the Fullscreen API to create the same experience on the web. You can embed the photo in a web page, and provide the ability to tap. The photo will pop into a floating frame as the Safari window disappears. Then when the user taps on the spatial photo or panorama UI that visionOS provides, the photo will further expand to create a fully immersive experience. When they exit the image, the Safari window will return.

Let’s walk through how to support experiencing a spatial photo or panorama on the web using Fullscreen API. First, include the image on your web page using any of the techniques we’ve used on the web for years. Here, we can embed a flattened panoramic photo into the web page using simple HTML.

<img src="panorama.jpeg" class="go-fullscreen" alt="[description]">

Then using JavaScript, we’ll trigger .requestFullscreen() on tap. Perhaps like this.

document.querySelectorAll('.go-fullscreen').forEach(element => {
    element.addEventListener('click', async () => {
        await element.requestFullscreen();
    });
});

You could, of course, create your own UI for the user to tap, rather than making the entire photo the tap target.

Spatial images work just the same, although it’s likely we want to provide fallbacks for browsers that do not support HEIC files. We can do so with the picture element.

<picture>
    <source srcset="spatial.heic" type="image/heic">
    <source srcset="fallback.avif" type="image/avif">
    <img src="fallback.jpg" class="go-fullscreen" alt="[description]" >
</picture>

Spatial images are stereoscopic, with both a left and right channel. In Safari, when the image is embedded in the web page, the browser will show the left channel. And there’s no need to worry about providing a fallback of any sort for Safari on macOS, iOS, or iPadOS — the stereoscopic HEIC file works great.

This technique will also cause images to go fullscreen in any browser that supports Fullscreen API. Learn more about adding panorama and spatial photos to your websites by watching Optimize for the spatial web at WWDC24, available Tuesday June 11.

HTML

Writing Suggestions

At last year’s WWDC, Apple unveiled inline predictive text on iOS, iPadOS, macOS and more. It helps users input text faster by predicting what they might be typing and finishing the word, phrase or even a whole sentence when the user taps the space bar. Now, WebKit for Safari 18 beta on iOS, iPadOS, visionOS, macOS Sequoia and macOS Sonoma brings inline predictive text to the web.

While inline predictive text makes for a fantastic, personalized user experience, there might be specific situations on the web where it’s better to not have predictions. WebKit for Safari 18 beta on iOS, iPadOS, visionOS, macOS Sequoia and macOS Sonoma gives web developers the opportunity to disable inline predictions through the writingsuggestions attribute. By default, writing suggestions is set to true. You can turn off the capability by including the writingsuggestions="false" attribute on any type of text input field.

Try out the difference between having writing suggestions or not in this demo. Be sure to use a browser and operating system that provide suggestions of what to write, and have support for writingsuggestions.

Switch

WebKit for Safari on iOS 18 beta adds haptic feedback for <input type=checkbox switch>. This means, now when a user taps a switch control on iPhone, a single tap is felt — just like how toggling a switch feels in Settings app on iOS. Try this demo to see what it’s like.

Date and time inputs

WebKit for Safari 18 beta on macOS improves accessibility support for date and time input field types. Now <input type="date">, <input type="datetime-local">, and <input type="time"> elements work properly with VoiceOver.

ARIA

Usually elements have the labels they need, but sometimes there is no text label for a particular button or UI. In this situation, ARIA can be used to provide an accessible label. The aria-label attribute provides names of labels while aria-roledescription provides the description for the role of an element.

On very rare occasions, you may need to override aria-label or aria-roledescription to provide different names or descriptions specifically for braille. The aria-braillelabel and aria-brailleroledescription attributes provide such an ability. They exist to solve very specific needs, including educational contexts where the site needs to render the specific braille table dot pattern. If you do use braille-related ARIA attributes, be sure to test them using a braille reader. If in doubt, relying on the accessible name from content or aria-label / aria-roledescription is almost always the better user experience. WebKit has supported these ARIA attributes for years.

Now, WebKit for Safari 18 beta adds support for the ariaBrailleLabel and ariaBrailleRoleDescription element reflection properties. These make it possible to get and set the aria-braillelabel and aria-brailleroledescription ARIA attributes on DOM elements directly via JavaScript APIs, rather than by using setAttribute and getAttribute.

Media

Watch video without distractions in Viewer for Safari 18 beta on macOS.

A video playing in a Safari window, where the video is enlarged to fill almost all of the space. The rest of the web page content is mostly hidden behind a dark translucent overlay.

When you play in Viewer, the video fills the Safari window, while providing full access to system playback controls. Then it automatically enters picture-in-picture anytime you switch tabs, close the window, or occlude the web page with another window. Look for Video Viewer in the new page menu in Safari whenever you are on a web page with a prominent video element.

Video on visionOS

Safari for visionOS 2 beta adds support for docking fullscreen videos into the current Environment. Anytime a user is watching a video fullscreen, they can tap mountain symbol to enter a fully immersive experience. Turning the Digital Crown adjusts the immersion. Light emitted from the video reflects off of the Environment, while audio is rendered on a large soundstage.

Managed Media Source

WebKit for Safari 18 beta adds Workers support for both Managed Media Source (MMS) and Media Source Extensions (MSE). This can be especially helpful on complex websites that want to ensure continuous and smooth video playback even when other site activity (such as live commenting) causes a very busy main thread. You can see the performance difference in this demo.

WebRTC

WebKit for Safari 18 beta adds support for the WebRTC HEVC RFC 7789 RTP Payload Format. Previously, the WebRTC HEVC used generic packetization instead of RFC 7789 packetization. This payload format provides a new option for improving videoconferencing, video streaming, and delivering high-bitrate movies and TV shows.

WebKit for Safari 18 beta adds support for MediaStreamTrack processing in a dedicated worker. And it adds support for missing WebRTC stats.

Passkeys

Two years ago at WWDC22, we announced support for passkeys — a groundbreaking industry-standard way to login to websites and app services. Passkeys provide people with an extremely easy user experience, while delivering a profound increase in security. To learn more, watch Meet Passkeys or read Supporting passkeys.

WebKit for Safari 18 beta adds support for three new features as we continue to improve passkeys. First, Safari 18 beta adds support for using mediation=conditional for web authentication credential creation. This allows websites to automatically upgrade existing password-based accounts to use passkeys. Learn more by watching Streamline sign-in with passkey upgrades and credential managers at WWDC24, available on Tuesday, June 11.

Second, WebKit for Safari 18 beta adds support for using passkeys across related origins. This lets websites use the same passkey across a limited number of domains which share a credential backend.

And third, WebKit for Safari 18 beta adds support for the WebAuthn prf extension. It allows for retrieving a symmetric key from a passkey to use for the encryption of user data.

HTTPS

WebKit for Safari 18 beta adds support for secure HTTPS for all images, video, and audio by upgrading passive subresource requests in mixed content settings. This means that if some files for a website are served using HTTPS and some are served using HTTP (known as “mixed content”), all images and media will now be auto-upgraded to HTTPS, in adherence with Mixed Content Level 2.

JavaScript

WebKit for Safari 18 beta adds support for Unicode 15.1.0 characters in RegExp. Unicode 15.1 added 627 characters, bringing the total of characters to 149,813. Now, these new characters can be used in regular expressions.

WebKit for Safari 18 beta also adds support for the v flag with RegExp.prototype[Symbol.matchAll]. providing more powerful ways to match Unicode characters, as specified in the ECMAScript 2024 standard.

For example, you can now specify to only match on Latin characters, while avoiding matching on Cyrillic script characters.

const regex = /\p{sc=Latin}/v;
console.log(regex.test('A')); // true, 'A' is a Latin script character
console.log(regex.test('А')); // false, 'А' is a Cyrillic script character

Or split a string matching on Emojis.

"a 🥰 b 🥰".split(/[\p{Emoji}--\p{ASCII}]/v)// ["a ", " b ", ""]

Web API

WebKit for Safari 18 beta adds support for URL.parse(), a way to parse URLs which returns null rather than an exception when parsing fails.

// Before
let url = null;
try {
  url = new URL(input, base);
} catch(e) { }

// Now
const url = URL.parse(input, base);

WebKit for Safari 18 beta expands Declarative Shadow tree support by adding the shadowRootDelegatesFocus and shadowRootClonable IDL attributes to the <template> element. It also adds the shadowRootSerializable attribute and shadowRootSerializable IDL attribute to the <template> element, enabling those using Declarative Shadow roots to opt into making them serializable. Serializing can be done through the new getHTML() method that has been added at the same time.

WebKit for Safari 18 beta adds support for PopStateEvent’s hasUAVisualTransition, indicating whether the user agent has a visual transition in place for the fragment navigation.

WebKit for Safari 18 beta adds support for subresource integrity in imported module scripts, which gives cryptographic assurances about the integrity of contents of externally-hosted module scripts.

WebKit for Safari 18 beta adds support for the bytes() method to the Request, Response, Blob, and PushMessageData objects. This replaces the need for web developers to call arrayBuffer(), which can be difficult to use, and wraps the result in a Uint8Array . Calling bytes() is now the recommended way going forward when you need to access the underlying bytes of the data these objects represent.

WebKit for Safari 18 beta adds support for feature detecting text fragments by exposing document.fragmentDirective. Note that the returned object (a FragmentDirective) doesn’t provide any functionality, but it’s helpful if you need to know if Fragment Directives are supported by the browser.

Canvas

WebKit for Safari 18 beta adds support for the willReadFrequently context attribute for the getContext() method. It indicates whether or not a lot of read-back operations are planned. It forces the use of a software accelerated 2D or offscreen canvas, instead of hardware accelerated. This can improve performance when calling getImageData() frequently.

WebKit for Safari 18 beta extends 2D canvas support for currentcolor. It can now be used inside color-mix() or Relative Color Syntax. Here currentcolor will default to the computed color property value on the canvas element.

WebGL

WebKit for Safari 18 beta adds support for six new WebGL extensions:

  • EXT_texture_mirror_clamp_to_edge
  • WEBGL_render_shared_exponent
  • WEBGL_stencil_texturing
  • EXT_render_snorm
  • OES_sample_variables
  • OES_shader_multisample_interpolation

Web Inspector

WebKit for Safari 18 beta adds support for fuzzy search code completion in the Web Inspector’s CSS source editor.

WKWebView

WebKit for iOS 18 beta, iPadOS 18 beta, visionOS 2 beta, and macOS Sequoia beta adds support for two new API — the Writing Tools API and an API to control adaptive image glyph insertion. Learn more about these API by watching Get started with Writing Tools and Bring expression to your app with Genmoji at WWDC24, both available Tuesday June 11.

Apple Pay

WebKit for Safari 18 beta adds support for Apple Pay funds transfer.

Deprecations

While it’s rare to deprecate older technology from the web, there are occasions when it makes sense. We’ve been busy removing -webkit prefixed properties that were never standardized, aging media formats that were never supported in other browsers, and more. This helps align browser engines, improve interoperability, and prevent compatibility problems by reducing the possibility that a website depends on something that’s not a web standard.

Canvas

WebKit for Safari 18 beta removes support for OffscreenCanvasRenderingContext2D’s commit() method.

CSS

WebKit for Safari 18 beta deprecates support for a number of rarely used -webkit prefixed CSS pseudo-classes and properties — and even one -khtml prefixed property.

  • -webkit-alt and alt properties
  • :-webkit-animating-full-screen-transition pseudo-class
  • :-webkit-full-screen-ancestor pseudo-class
  • :-webkit-full-screen-controls-hidden pseudo-class
  • :-webkit-full-page-media pseudo-class
  • :-webkit-full-screen-document pseudo-class
  • :-khtml-drag pseudo-class

WebKit for Safari 18 beta also deprecates support for the resize: auto rule. Support for the resize property remains, just as it’s been since Safari 4. The values Safari continues to support include: none, both, horizontal, vertical, block, inline, plus the global values. Early versions of CSS Basic User Interface Module Level 3 defined auto, but it was later written out of the web standard.

WebKit for Safari 18 beta also deprecates support for non-standardize WEBKIT_KEYFRAMES_RULE and WEBKIT_KEYFRAME_RULE API in CSSRule.

Images

WebKit for Safari 18 beta removes support for the JPEG2000 image format. Safari was the only browser to ever provide support.

If you’ve been serving JPEG2000 files using best practices, then your site is using the picture element to offer multiple file format options to every browser. Safari 18 beta will simply no longer choose JPEG2000, and instead use a file compressed in JPEG XL, AVIF, WebP, HEIC, JPG/JPEG, PNG, or Gif — choosing the file that’s best for each user. Only one image will be downloaded when you use <picture>, and the browser does all the heavy lifting.

We have noticed that some Content Deliver Networks (CDN) use User Agent sniffing to provide one file to each UA, offering only JPEG2000 images to Safari — especially on iPhone and iPad. If you expect this might be happening with your site, we recommend testing in Safari 18 beta on both macOS Sequoia and iOS or iPadOS 18. If you see problems, contact your SaaS provider or change your image delivery settings to ensure your website provides fallback images using industry best practices.

If you notice a broken site, please file an issue at webcompat.com.

Media

WebKit for Safari 18 beta removes support for non-standard VTTRegion.prototype.track.

Storage

WebKit for Safari 18 beta removes the last bits of support for AppCache.

When AppCache first appeared in 2009, in Safari 4, it held a lot of promise as a tool for caching web pages for use offline. It was imagined as “HTML5 Application Cache” back when HTML itself was being further expanded to handle more use cases for web applications. A developer could create a simple cache manifest file with a list of files to be cached. Its simplicity looked elegant, but there was no mechanism for cache busting, and that made both developing a site and evolving the site over time quite frustrating. AppCache also had security challenges. So new web standards were created to replace it. Today, developers use Service Workers and Cache Storage instead.

WebKit deprecated AppCache with a warning to the Console in Safari 11.0. Then in 2021, we removed support for AppCache from Safari 15.0, with a few exceptions for third-party users of WKWebView. Now we are removing those exceptions. This change to WebKit will only affect the rare web content loaded in older third-party apps that have JavaScript code which relies on the existence of AppCache related interfaces.

SVG

WebKit for Safari 18 beta removes the SVGAnimateColorElement interface.

Web API

WebKit for Safari 18 beta removes support for four non-standard Web APIs:

  • KeyboardEvent.altGraphKey
  • AES-CFB support from WebCrypto
  • KeyboardEvent.prototype.keyLocation
  • HashChangeEvent’s non-standard initHashChangeEvent() method

Bug Fixes and more

In addition to all the new features, WebKit for Safari 18 beta includes work to polish existing features.

Accessibility

  • Fixed role assignment for <header> inside <main> and sectioning elements.
  • Fixed range input not firing an input event when incremented or decremented via accessibility APIs.
  • Fixed setting aria-hidden on a slot not hiding the slot’s assigned nodes.
  • Fixed comboboxes to expose their linked objects correctly.
  • Fixed time input accessibility by adding labels to subfields.
  • Fixed aria-hidden=true to be ignored on the <body> and <html> elements.
  • Fixed datetime values being exposed to assistive technologies in the wrong timezone.
  • Fixed time control accessibility by adding a label to the meridiem component.
  • Fixed wrong datetime value being exposed to assistive technologies for datetime-local inputs.
  • Fixed ignored CSS content property replacement text when it is an empty string.
  • Fixed the computed role for these elements: dd, details, dt, em, hgroup, option, s, and strong.
  • Fixed hidden elements targeted by aria-labelledby to expose their entire subtree text, not just their direct child text.
  • Fixed accessible name computation for elements with visibility: visible inside a container with visibility: hidden.
  • Fixed updating table accessibility text when its caption dynamically changes.
  • Fixed updating aria-describedby text after the targeted element changes its subtree.

Animations

  • Fixed the transition property to produce the shortest serialization.
  • Fixed the animation property to produce the shortest serialization.

Authentication

  • Fixed navigator.credentials.create() rejects with “NotAllowedError: Operation Failed” after a conditional UI request is aborted.
  • Fixed renaming DigitalCredential’s response attribute to data.
  • Fixed setting the cancel flag once the cancel completes regardless of a subsequent request occurring.

Canvas

  • Fixed drawImage(detachedOffscreenCanvas) to throw an exception.
  • Fixed OffscreenCanvas failing to render to the placeholder with nested workers.
  • Fixed losing the contents layer of the placeholder canvas of OffscreenCanvas when switching off the tab.
  • Fixed drawImage to not alter the input source or the destination rectangles.
  • Fixed toggling the visibility on a canvas parent undoing the effect of clearRect().
  • Fixed the Canvas drawImage() API to throw an exception when the image is in broken state.

CSS

  • Fixed setting white-space to a non-default value dynamically on a whitespace or a new line.
  • Fixed turning text-spacing properties into font properties.
  • Fixed custom counter styles extending disclosure-open and disclosure-closed to point to the correct direction in right-to-left.
  • Fixed backface-visibility to create a stacking context and containing block.
  • Fixed getComputedStyle() to work with functional pseudo-elements like ::highlight().
  • Fixed: Aliased :-webkit-full-screen pseudo-class to :fullscreen.
  • Fixed: Aliased :-webkit-any-link to :any-link and :matches() to :is().
  • Fixed getComputedStyle() pseudo-element parsing to support the full range of CSS syntax.
  • Fixed @supports to correctly handle support for some -webkit prefixed pseudo-elements that were incorrectly treated as unsupported.
  • Fixed updating media-query sensitive meta tags after style changes.
  • Fixed changing color scheme to update gradients with system colors or light-dark().
  • Fixed incorrect inline element size when using font-variant-caps: all-small-caps with font-synthesis.
  • Fixed :empty selector to work with animations.
  • Fixed preserving whitespace when serializing custom properties.
  • Fixed updating style correctly for non-inherited custom property mutations.
  • Fixed element removed by parent to end up losing the last remembered size.
  • Fixed an incorrect difference between implicit and explicit initial values for custom properties.
  • Fixed the contrast of Menu and MenuText system colors.
  • Fixed keeping the shorthand value for CSS gap as-is in serialized and computed values.
  • Fixed the style adjuster for @starting-style incorrectly invoking with a null element.
  • Fixed excluding -apple-pay-button from applying to any element that supports appearance: auto and is not a button.
  • Fixed missing color interpretation methods added to CSS color specifications.
  • Fixed hsl() and hsla() implementation to match the latest spec changes.
  • Fixed the implementation of rgb() and rgba() to match the latest spec.
  • Fixed the hwb() implementation to match the latest spec.
  • Fixed the remaining color types to be synced with the latest spec changes.
  • Fixed carrying analogous components forward when interpolating colors
  • Fixed applying the fill layer pattern for mask-mode.

Forms

  • Fixed displayed datalist dropdown to sync its options elements after a DOM update.
  • Fixed <select multiple> scrollbars to match the used color scheme.
  • Fixed updating the input value when selecting an <option> from a <datalist> element.
  • Fixed the value attribute not getting displayed in an input element with type="email" and the multiple attribute.
  • Fixed the iOS animation for <input type=checkbox switch>.
  • Fixed form controls drawing with an active appearance when the window is inactive.
  • Fixed constructed FormData object to not include entries for the image button submitter by default.

History

  • Fixed the properties of History to throw a SecurityError when not in a fully active Document.

HTML

  • Fixed “about:blank” document.referrer initialization.
  • Fixed parsing a self-closing SVG script element. It now successfully executes.

JavaScript

  • Fixed RegExp.prototype.@@split to update the following legacy RegExp static properties: RegExp.input, RegExp.lastMatch, RegExp.lastParen, RegExp.leftContext, RegExp.rightContext, and RegExp.$1, ... RegExp.$9.
  • Fixed String.prototype.replace to not take the fast path if the pattern is RegExp Object and the lastIndex is not numeric. (
  • Fixed spec compliance for Async / Await, Generators, Async Functions, and Async Generators.
  • Fixed async functions and generators to properly handle promises with throwing “constructor” getter.
  • Fixedreturn in async generators to correctly await its value.
  • Fixed Symbol.species getters to not share a single JS Function.
  • Fixed throwing a RangeError if Set methods are called on an object with negative size property.
  • Fixed eval() function from another realm to not cause a direct eval call.
  • Fixed eval() call with ...spread syntaxt to be a direct call.
  • Fixed try/catch to not intercept errors originated in [[Construct]] of derived class.
  • Fixed several issues:
    • direct eval() in a default value expression inside a rest parameter creates a variable in the environment of the function rather than the separate one of the parameters;
    • a ReferenceError is thrown when accessing a binding, which is defined inside rest parameter, in eval(), or a closure created in a default value expression of a preceding parameter, but only if there is a var binding by the same name;
    • a closure, created in the default value expression inside a rest parameter, is created in a different VariableEnvironment of the function than its counterparts in preceding parameters which causes the incorrect environment to be consulted when querying or modifying parameter names that are “shadowed” by var bindings.
  • Fixed TypedArray sorting methods to have a special-case for camparator returning false.
  • Fixed programming style for bitwise and in setExpectionPorts.
  • Fixed emitReturn() to load this value from arrow function lexical environment prior to the TDZ check.
  • Fixed NFKC normalization to work with Latin-1 characters.
  • Fixed parsing of private names with Unicode start characters.
  • Fixed instanceof to not get RHS prototype when LHS is primitive.
  • Fixed bracket update expression to resolve property key at most once.
  • Fixed bracket compound assignement to resolve the property key at most once.
  • Fixed Object.groupBy and Map.groupBy to work for non-objects.
  • Fixed Array.fromAsync to not call the Array constructor twice.
  • Fixed inconsistent output of Function.prototype.toString for accessor properties.
  • Fixed Set#symmetricDifference to call this.has in each iteration.
  • Fixed logical assignment expressions to throw a syntax error when the left side of the assignment is a function call.
  • Fixed throwing a syntax error for nested duplicate-named capturing groups in RegEx.
  • Fixed ArrayBuffer and SharedArrayBuffer constructor to check length before creating an instance.
  • Fixed Intl implementation to ensure canonicalizing “GMT” to “UTC” based on a spec update.
  • Fixed RegEx lookbehinds differing from v8.
  • Fixed fractionalDigits of Intl.DurationFormat to be treated as at most 9 digits if it is omitted.

Loading

  • Fixed navigator.cookieEnabled to return false when cookies are blocked.

Media

  • Fixed video sound coming from another window after changing tabs in the Tab Bar in visionOS.
  • Fixed playback for MSE videos on some sites.
  • Fixed allowing a video’s currentTime to be further than the gap’s start time.
  • Fixed broken audio playback for a WebM file with a Vorbis track.
  • Fixed sampleRate and numberOfChanges to be required and non-zero in a valid AudioEncoderConfig.
  • Fixed media elements appending the same media segment twice.
  • Fixedrejecting valid NPT strings if ‘hours’ is defined using 1 digit.
  • Fixed an issue where Safari audio may be emitted from the wrong window in visionOS.

Networking

  • Fixed upgrading inactive or passive subresource requests and fetches in would-be mixed security contexts to match standards.
  • Fixed incorrect Sec-Fetch-Site value for navigation of a nested document.
  • Fixed loading WebArchives with a non-persistent datastore.
  • Fixed Timing-Allow-Origin to not apply to an HTTP 302 response.

PDF

  • Fixed print buttons with a print action implementation.
  • Fixed Open in Preview for a PDF with a space in its name.

Rendering

  • Fixed Greek uppercase transforms failing for some characters.
  • Fixed resizing a <textarea> element with 1rem padding.
  • Fixed the color correctness of the color matrix filter.
  • Fixed backdrop-filter to apply to the border area of an element with a border-radius.
  • Fixed intrinsic inline size calculators to account for whitespace before an empty child with nonzero margins.
  • Fixed overlapping elements with flex box when height: 100% is applied on nested content.
  • Fixed incorrect grid item positioning with out-of-flow sibling.
  • Fixed break-word with a float discarding text.
  • Fixed min-content calculation for unstyled only-child inlines elements.
  • Fixed ellipsis rendering multiple times when position: relative and top are used.
  • Fixed a bug for inline elements inserted in reverse order after a block in a continuation.
  • Fixed the flash of a page background-colored bar in the footer when the window is resized.

Scrolling

  • Fixed the cursor not updating as content scrolls under it on some pages.

SVG

  • Fixed the SVG parser to interpret “form feed” as white space.
  • Fixed error handling for invalid filter primitive references.
  • Fixed displaying an SVG element inside a <switch> element.
  • Fixed SVG title to have display: none as the default UA style rule.
  • Fixed the UA stylesheet for links in SVGs to apply cursor: pointer matching standards.
  • Fixed returning the initial value for the SVG gradient stop-color if it is not rendered in the page.
  • Fixed the SVG marker segment calculations if the marker path consists of sub-paths.
  • Fixed SVGLength to sync with the WebIDL specification.

Web Animations

  • Fixed percentage transform animations when width and height are animated.
  • Fixed updating an animation when changing the value of a transform property while that property is animated with an implicit keyframe.
  • Fixed animating with color-mix.

Web API

  • Fixed cssTextsetter to change the style attribute when the serialization differs.
  • Fixed history.pushState() and history.replaceState() to ignore the title argument.
  • Fixed URL text fragment directives not fully stripped from JavaScript.
  • Fixed showPicker() method to trigger suggestions from a datalist.
  • Fixed lang attribute in no namespace to only apply to HTML and SVG elements.
  • Fixed unnecessarily unsetting the iframe fullscreen flag.
  • Fixed DOM Range to correctly account for CDATASection nodes.
  • Fixed getGamepads() to no longer trigger an insecure contexts warning.
  • Fixed inserting a <picture> element displaying the same image twice.
  • Fixed throwing exceptions in navigation methods if in a detached state.
  • Fixed a minor issue in URL’s host setter.
  • Fixed cloning of ShadowRoot nodes following a DOM Standard clarification.
  • Fixed GeolocationCoordinates to expose a toJSON() method.
  • Fixed GeolocationPosition to expose a toJSON() method.
  • Fixed setting CustomEvent.target when dispatching an event.
  • Fixed navigator.language only returning the system language in iOS 17.4.
  • Fixed: Removed presentational hints from the width attribute for <hr>.

Web Apps

  • Fixed resolving www. sub-domain for Associated Domains for all web apps.

Web Assembly

  • Fixed initialization of portable reference typed globals.

Web Inspector

  • Fixed font sizes in the Audits tab.
  • Fixed expanded sections of Storage to not collapse
  • Fixed CSS font property values marked !important not getting overridden when using the interactive editing controls.
  • Fixed an issue where the Web Inspector viewport might appear cut off.
  • Fixed runtimes to be aligned in the Audit tab.
  • Fixed remembering the message type selection in the Console tab.
  • Fixed autocomplete for the text-indent property suggesting prefixed properties instead of each-line or hanging.
  • Fixed background autocompletion suggestion to include repeating-conic-gradient.
  • Fixed console clearing unexpectedly when Web Inspector reopens
  • Fixed console code completion to be case-insensitive.
  • Fixed overflow: scroll elements to scroll as expected when highlighting an element from the DOM tree.
  • Fixed showing additional Safari tabs from an iOS device in the Develop menu.
  • Fixed Console and code editor completion not auto-scrolling the suggestion into view.
  • Fixed search in the DOM tree view unexpectedly chaning the text display.
  • Fixed clicking the “goto” arrow for computed CSS when “show independent Styles sidebar” is disabled.
  • Fixed inspectable tabs from Safari in the visionOS Simulator don’t appear in Developer menu on the host macOS.

Web Views

  • Fixed Gamepad API in WKWebView.
  • Fixed repainting HTML elements when their width or height change in legacy WebView.

WebRTC

  • Fixed VideoTrackGenerator writer to close when its generator track (and all its clones) are stopped.
  • Fixed WebRTC AV1 HW decoding on iPhone 15 Pro.
  • Fixed black stripes with screen sharing windows.
  • Fixed black stripes with getDisplayMedia captured windows when the window is resized.

Help us Beta Test

You can test Safari 18 beta by installing the beta of macOS 15, iOS 18, or iPadOS 18. Or, if you’d like, you can try out Safari 18 beta on macOS Sonoma or macOS Ventura by downloading the Safari 18 beta, once it’s available. (Sign in using a free Apple ID to download. Installing Safari 18 beta on macOS Sonoma or macOS Ventura will replace your existing version of Safari with no way to revert to an earlier version.) You can also help test many of these features in Safari Technology Preview.

Feedback

We love hearing from you. To share your thoughts on Safari 18 beta, find us on Mastodon at @jensimmons@front-end.social and @jondavis@mastodon.social. Or send a reply on X to @webkit. You can also follow WebKit on LinkedIn. If you run into any issues, we welcome your feedback on Safari UI (learn more about filing Feedback), or your WebKit bug report about web technologies or Web Inspector. If you notice a website that seems broken in Safari, but not in other browsers, please file a report at webcompat.com. Filing issues really does make a difference.

Download the latest Safari Technology Preview on macOS to stay at the forefront of the web platform and to use the latest Web Inspector features.

You can also find this information in the Safari 18 beta release notes.

June 10, 2024 08:00 PM

Web technology sessions at WWDC24

Surfin’ Safari

It’s time for WWDC24! Today brings announcements of the new web technology shipping in Safari on iOS 18 beta, iPadOS 18 beta, visionOS 2 beta and macOS 15 beta. Read our in-depth article about everything in WebKit for Safari 18 beta.

Three sessions at WWDC24 dive into the spatial web, passkey upgrades, and WebXR.

Screenshot of an experience in Apple Vision Pro, looking at a full-sized, rendered 3D bicycle sitting in the middle of a real office.

Optimize for the spatial web

Discover how to make the most of visionOS capabilities on the web. Explore recent updates like improvements to selection highlighting, and the ability to present spatial photos and panorama images in fullscreen. Learn to take advantage of existing web standards for dictation and text-to-speech with WebSpeech, spatial soundscapes with WebAudio, and immersive experiences with WebXR.

Coming Tuesday, June 11.


A slide from the presentation explaining visually how passwords are less secure and passkeys are most secure.

Streamline sign-in with passkey upgrades and credential managers

Learn how to automatically upgrade existing password-based accounts to use passkeys. We’ll share why and how to improve account security and ease of sign-in, information about new features available for credential manager apps, and how to make your app information shine in the new Passwords app.

Coming Tuesday, June 11.


A view from inside Apple Vision Pro where two giant windows float in a real office. The first is a Safari window with the Chess Garden website showing. The second is macOS, with multiple windows inside the Mac window — working on developing the Chess Garden website using Safari Web Inspector and more.

Build immersive web experiences with WebXR

Discover how WebXR empowers you to add fully immersive experiences to your website in visionOS. Find out how to build WebXR experiences that take full advantage of the input capabilities of visionOS, and learn how you can use Simulator to test WebXR experiences on macOS.

Coming Wednesday, June 12.

June 10, 2024 07:15 PM

June 06, 2024

Improving Web Accessibility with Web Platform Tests

Surfin’ Safari

Last year, a colleague introduced me to the Interop Accessibility project. I immediately resonated with its charge to “improve the state of accessibility across the entire web, regardless of which platform, browser, or assistive technology.” The project’s mission was compelling but it also represented, from my perspective, a transformative example of shifting accessibility to the level of web platform features.

After digging in further, I learned about Web Platform Tests (WPT), a cross-browser test suite essential to executing Interop Accessibility’s ambitious vision. WPT enables anyone to write a single automated test that runs in all major browser engines (i.e., Chromium, Gecko, and WebKit), thereby ensuring that web technologies such as HTML or ARIA (Accessible Rich Internet Applications) work as expected. It’s never been possible before this to write an accessibility test that runs in all browsers, on any system. Despite some limitations around mobile testing and operating system coverage, WPT holds great promise as a tool for realizing a more accessible and inclusive web!

As an example, the following hypothetical test asserts that the accessibility role of an <img> element is “image”:

<img data-expectedrole="image" ...>

For an inclusive user experience, it’s imperative that roles are accurately computed by the browser and in turn, properly exposed by platform accessibility APIs (HTML Accessibility API Mappings). Roles enable assistive technology users to understand the purpose of user interface elements (e.g., button, link, dialog) and with WPT, we can easily verify browser-calculated roles.

You may observe that our fictitious <img> example is missing a text alternative even though it’s required for accessibility. I can add this to test that the <img>’s textual description, supplied via the alt attribute, is properly exposed as its accessibility label:

<img alt="stack of fluffy pancakes" 
     data-expectedlabel="stack of fluffy pancakes"
     data-expectedrole="image" 
 ...
 >

In terms of the screen reader experience, we expect that this particular UI properly announces as “stack of fluffy pancakes, image”. Another hypothetical example showcasing label calculation for an image button:

<button data-expectedlabel="Order!" ...>
    <img alt="Order!" ...>
</button>

Here, the button’s nested content (the <img>’s alt specifically) provides the accessibility label for the button itself and the resulting announcement would be “Order!, button”. These examples demonstrate the simplicity and power of WPT: with a couple lines of markup, we’re able to quickly ascertain how web browsers expose the accessibility of any DOM element.

Note: for those interested in seeing more complex, real WPT examples: resolving ambiguity in label calculation that involves hidden nodes, investigating display: contents and accessible name calculation for aria-label .

Under the hood, when I run the above <img> test for example, a JavaScript Promise function asserts that the image’s accessibility role and label are calculated correctly by the browser. To this end, WPT uses testharness.js, a JavaScript framework for writing test cases, and WebDriver for obtaining browser-calculated accessibility metadata about DOM elements (at present, either an element’s programmatic label or role). Most WPT accessibility tests are also written with HTML markup and some data-* attributes (e.g., data-testname, data-expectedrole) that provide hooks for WPT test execution and reporting.

Tests can be run locally using CLI commands that invoke browser automation. One of the slicker aspects of WPT is its robust continuous integration (CI) infrastructure that runs tests in the WPT repo across latest versions of all major browser engines on a daily basis (and for pull requests)! Test results are stored in the WPT interop dashboard for easy viewing and to help browser engineers improve interoperability. A special thank you is definitely in order to all the people that keep the WPT CI servers running; their hard work provides a well-maintained, reliable, and stable WPT infrastructure.

I hope that the utility of these WPT accessibility tests is abundantly clear: when accessibility semantics (e.g., role) and labels work interoperably across browsers, and in accordance with developer intent, this is a victory for both web developers and users. Thinking more broadly, what if we could regularly test the accessibility behavior of any and all web platform features on the latest browsers in an automated fashion? How much time and effort could this save?

This is why WPT is so compelling: using a powerful, scaleable testing framework, we can produce easy-to-write tests that:

  • decrease frustration for code that should “just work”,
  • provide a more consistent, accessible experience for the entire web, including your users,
  • surface and eliminate cross-browser discrepancies and unexpected accessibility behaviors,
  • prevent future regressions of web platform accessibility bugs,
  • reduce accessibility QA efforts, and
  • allow you more time to focus efforts on other development priorities.

For the reasons above, it’s difficult to overstate the utility of interoperability and the ultimate benefit of WPT: it gets everyone collectively closer to the shared goal and vision of a reliable and inclusive web!

The value of writing WPT accessibility tests isn’t limited to ensuring correct cross-browser behavior or gauging browser conformance to accessibility specifications. WPT tests can also positively influence the very criteria they are written to test. For example, a colleague recently wrote a WPT test for accessible name node traversal and after investigation, we observed that all major browsers failed in an identical manner (which is unusual). We identified multiple specification ambiguities from this work which motivated fixes across browsers. The issue also spawned fruitful discussion on several related topics all in the pursuit of great web accessibility and a best-in-class user experience. Awesome stuff!

In addition to the value of WPT testing for web developers, further benefits include:

  • increased coordination between browser vendors
  • greater influence on browser vendors to support web technologies in interoperable ways
  • reduction of implementation gaps in accessibility standards and specifications
  • improved standardization of how code is interpreted by assistive technologies
  • concrete evidence of how browsers actually behave thus facilitating fixes, improved behavior and greater browser parity by browser engineers

It’s incredible to see the WPT accessibility testing grow over time with our efforts continually bolstered by more efficient infrastructure, broader testing coverage, and strong partnerships. While Apple has taken a lead role, it’s highly collaborative across partners such as Adobe, Hilton, Mozilla, Google, Igalia, Bocoup, etc. I’m amazed by the degree of diligence, technical knowledge and passion that is demonstrated in all aspects of this project. From both inside and outside of Apple, the deep focus and unwavering commitment towards the mission of web accessibility is infectious. To everyone involved: kudos and keep up the great work!

I’m excited for a future that includes greater WebDriver capabilities in inspecting browser-generated accessibility metadata, in addition to dwindling accessibility gaps across web browsers as a result of productive collaboration. I invite you, dear reader, to contribute to this future of a more interoperable, reliable, and accessible web by writing WPT tests. Please join us!

June 06, 2024 09:07 PM

June 05, 2024

Safari Technology Preview 196 Release Notes

Surfin’ Safari

Safari Technology Preview Release 196 is now available for download for macOS Sonoma and macOS Ventura. If you already have Safari Technology Preview installed, you can update it in System Settings under General → Software Update.

This release includes WebKit changes between: 278844@main…279360@main.

Canvas

Resolved Issues

  • Fixed toggling the visibility on a canvas parent undoing the effect of clearRect(). (279302@main) (128226178)

  • Fixed the Canvas drawImage() API to throw an exception when the image is in broken state. (279319@main) (128588063)

CSS

Resolved Issues

  • Fixed incorrect inline element size when using font-variant-caps: all-small-caps with font-synthesis. (279183@main) (121314557)

JavaScript

Resolved Issues

  • Fixed RegEx lookbehinds differing from v8. (278863@main) (127440248)

Web API

New Features

  • Added support for bytes() to Blob and PushMessageData. (279263@main) (128418858)

Web Inspector

New Features

  • Added support for fuzzy search code completion in the CSS source editor. (2791444@main) (125030691)

Resolved Issues

  • Fixed console code completion to be case-insensitive. (279195@main) (124544458)

WebGL

Resolved Issues

  • Fixed compiling shaders without variables that are arrays and start with underscore. (278880@main) (126944294)

WebRTC

Resolved Issues

  • Fixed a delay in getUserMedia promise resolution. (278851@main) (128112002)

June 05, 2024 11:46 PM

May 23, 2024

Patrick Griffis: Introducing the WebKit Container SDK

Igalia WebKit

Developing WebKitGTK and WPE has always had challenges such as the amount of dependencies or it’s fairly complex C++ codebase which not all compiler versions handle well. To help with this we’ve made a new SDK to make it easier.

Current Solutions

There have always been multiple ways to build WebKit and its dependencies on your host however this was never a great developer experience. Only very specific hosts could be “supported”, you often had to build a large number of dependencies, and the end result wasn’t very reproducable for others.

The current solution used by default is a Flatpak based one. This was a big improvement for ease of use and excellent for reproducablity but it introduced many challenges doing development work. As it has a strict sandbox and provides read-only runtimes it was difficult to use complex tooling/IDEs or develop third party libraries in it.

The new SDK tries to take a middle ground between those two alternatives, isolating itself from the host to be somewhat reproducable, yet being a mutable environment to be flexible enough for a wide range of tools and workflows.

The WebKit Container SDK

At the core it is an Ubuntu OCI image with all of the dependencies and tooling needed to work on WebKit. On top of this we added some scripts to run/manage these containers with podman and aid in developing inside of the container. It’s intention is to be as simple as possible and not change traditional development workflows.

You can find the SDK and follow the quickstart guide on our GitHub: https://github.com/Igalia/webkit-container-sdk

The main requirements is that this only works on Linux with podman 4.0+ installed. For example Ubuntu 23.10+.

In the most simple case, once you clone https://github.com/Igalia/webkit-container-sdk.git, using the SDK can be a few commands:

source /your/path/to/webkit-container-sdk/register-sdk-on-host.sh
wkdev-create --create-home
wkdev-enter

From there you can use WebKit’s build scripts (./Tools/Scripts/build-webkit --gtk) or CMake. As mentioned before it is an Ubuntu installation so you can easily install your favorite tools directly like VSCode. We even provide a wkdev-setup-vscode script to automate that.

Advanced Usage

Disposibility

A workflow that some developers may not be familiar with is making use of entirely disposable development environments. Since these are isolated containers you can easily make two. This allows you to do work in parallel that would interfere with eachother while not worrying about it as well as being able to get back to a known good state easily:

wkdev-create --name=playground1
wkdev-create --name=playground2

podman rm playground1 # You would stop first if running.
wkdev-enter --name=playground2

Working on Dependencies

An important part of WebKit development is working on the dependencies of WebKit rather than itself, either for debugging or for new features. This can be difficult or error-prone with previous solutions. In order to make this easier we use a project called JHBuild which isn’t new but works well with containers and is a simple solution to work on our core dependencies.

Here is an example workflow working on GLib:

wkdev-create --name=glib
wkdev-enter --name=glib

# This will clone glib main, build, and install it for us. 
jhbuild build glib

# At this point you could simply test if a bug was fixed in a different versin of glib.
# We can also modify and debug glib directly. All of the projects are cloned into ~/checkout.
cd ~/checkout/glib

# Modify the source however you wish then install your new version.
jhbuild make

Remember that containers are isoated from each other so you can even have two terminals open with different builds of glib. This can also be used to test projects like Epiphany against your build of WebKit if you install it into the JHBUILD_PREFIX.

To Be Continued

In the next blog post I’ll document how to use VSCode inside of the SDK for debugging and development.

May 23, 2024 04:00 AM

May 22, 2024

Release Notes for Safari Technology Preview 195

Surfin’ Safari

Safari Technology Preview Release 195 is now available for download for macOS Sonoma and macOS Ventura. If you already have Safari Technology Preview installed, you can update it in System Settings under General → Software Update.

This release includes WebKit changes between: 278097@main…278429@main.

Accessibility

Resolved Issues

  • Fixed updating table accessibility text when its caption dynamically changes. (278164@main) (127263464)

  • Fixed updating aria-describedby text after the targeted element changes its subtree. (278318@main) (127390465)

CSS

Resolved Issues

  • Fixed carrying analogous components forward when interpolating colors. (278379@main) (127170141)

Media

Resolved Issues

  • Fixed broken audio playback for a WebM file with a Vorbis track. (278665@main) (124880261)

Rendering

Resolved Issues

  • Fixed pseudo-elements not getting captured for View Transitions. (278123@main) (126964779)

  • Fixed View Transition rendering when the root element is not captured in the new snapshot. (278120@main) (127217265)

Scrolling

Resolved Issues

  • Fixed the cursor not updating as content scrolls under it on some pages. (278297@main) (122347347)

Web Extensions

Resolved Issues

  • Fixed Content Script CSS inconsistently overriding style attributes. (278387@main) (126916972)

Web Inspector

Resolved Issues

  • Fixed an issue where the Web Inspector viewport might appear cut off. (278208@main) (117272735)

  • Fixed overflow: scroll elements to scroll as expected when highlighting an element from the DOM tree. (278614@main) (124554999)

May 22, 2024 10:20 PM

May 21, 2024

WPE WebKit Blog: Update on the Layer Based SVG Engine (LBSE) in WebKit

Igalia WebKit

This blog entry gives an update on what we at Igalia have done on upstreaming and development of LBSE in WebKit in the last seven months. For an explanation of what LBSE is and how it is related to WPE, see this previous entry as a refresher.

Thanks to generous funding by Wix, which extensively uses SVG in their products and has a broad knowledge of the SVG peculiarities across browser engines, LBSE has made great progress in the past seven months. During this period, several advanced SVG painting features were implemented (e.g. clip-paths, masks, gradients, patterns), along with important performance improvements that expanded the new engine’s capabilities and stability. All this was possible thanks to Wix’s decision to address their problems by funding upstream work at the core of WebKit, instead of accepting the status-quo and implementing case-by-case fixes and workarounds on their end. As a result, WebKit-based browsers now benefit from the results of this fruitful collaboration, which we’ll try to explain in more detail in this blog post.

Project kick off and WebKit Contributors Meeting

In October 2023 we started the project mostly by thinking about the design and roadmap. We also did some general SVG bug fixing. For example, visual overflow computation for SVG renderers was corrected, which fixed quite a few SVG pixel tests. Various visual bugs were also fixed, such as unnecessary repainting when viewBox is used on <svg> elements, and incorrect clipping for outermost <svg> elements

Also in the same month, we attended the WebKit Contributors Meeting, where we presented a talk on the LBSE (slides are available here). The feedback on LBSE at the meeting was very positive. Giving the talk early on in the process actually helped us since we needed to have a good design in place.

Supporting SVG resources

The main focus in October 2023 was introducing the SVG resource concept, as already outlined in the WebKit Contributors Meeting talk. Thus, we started with adding a base SVG resource class: RenderSVGResourceContainer . This class was kept as simple as possible, with no support for resource invalidation or repainting logic. The main task of RenderSVGResourceContainer is to take care of registration so that the resource can be looked up by its clients.

For the first SVG resource to implement, we chose the SVG <clip-path> element, so we landed RenderSVGResourceClipper. To comply with the specification, the RenderSVGResourceClipper implementation produces 1-bit masks and uses a special rendering mode:

  • fill-opacity/stroke-opacity/opacity set to 1
  • masker/filter not applied when rendering the children
  • fill set to solid black and stroke set to none

The initial implementation did not use caching of ImageBuffers for clipping, but relied on Porter-Duff DestinationIn/SourceOver compositing operations to achieve the same effect, but faster. By integrating RenderSVGResourceClipper properly into RenderLayer, it aligned SVG clipping with HTML/CSS clipping.

Finally, the implementation prefers a pure clipping solution internally, as in legacy rendering, but for more complicated clip-paths (for example when the clip-path involves text content), a fallback to a mask is done.

Resource invalidation handling

After introducing the first SVG resource (RenderSVGResourceClipper), we noticed some issues with handling invalidations for it, such as adding to clip-path contents. In the legacy engine, invalidations have been handled through layouting. This caused various problems: for one, it could cause calling the setNeedsLayout method from within layout, which meant the invalidation chain depended on the DOM order.

In November 2023, an implementation landed that avoided using layout for resource invalidation. Instead, on dynamic updates, the style system is used to determine the appropriate action:

  • For non resources, changes that cause renderer geometry changes, like changing the x value of a <rect> element, still require a relayout. For visual changes not affecting geometry, like changing the fill color of a <rect>, a repaint action is enough.
  • For resources, the resource is invalidated/updated with the change and any of its clients are repainted using the new resource.

Support for masks

With improved support for SVG resource invalidation, in late November 2023 we were ready to upstream support for the next SVG resource, RenderSVGResourceMasker.

Like the support for clip-path, RenderSVGResourceMasker started out without caching image buffers and relied on creating temporary image buffers at rendering time. Mask content invalidations/changes were supported out of the box since we had improved resource invalidation handling (see above).

Support for gradients

In early January 2024, support for SVG gradients was upstreamed. Gradients are a kind of SVG resource that is a bit different to the previously implemented clipping paths and masks because it is a paint server, so a helper class for that called SVGPaintServerHandling and a base class RenderSVGResourcePaintServer were introduced. The main difference is in invalidation: paint servers simply need a repaint of all its clients on invalidation, whereas clipping paths/masks may need to do more work; i.e., masks underlying image buffers need to be updated before its clients can be repainted.

Support for patterns and markers

By the end of January 2024, support for SVG patterns was upstreamed. In the first implementation, no image buffer caching was implemented in order to keep things clean and simple. This implementation is different from the legacy implementation because the pattern contents are being rendered through pattern content layers (see RenderLayer::paintSVGResourceLayer). To make this work, RenderSVGResourcePattern has to set up the graphics context matrix correctly before calling paintSVGResourceLayer.

Around that same time, we implemented the next to last SVG resource on our TODO list, namely SVG markers.

SVG filters

In February 2024, Apple started work on supporting SVG filters in LBSE. A first iteration managed to fix a lot of the official SVG filter tests, but it turned out a filter regression had to be fixed first. Moreover, the initial work uncovered issues with the HTML/CSS filters implementation that need to be fixed in general. Finally, another reason why this support takes more time than some other features is that there is a strong requirement to make the support efficient in both memory usage and overall (re)painting speed. Still, the early results are very promising!

Cycle detection

It is quite easy in SVG to cause direct circular references:

<svg>
    <defs>
        <pattern id="p" xlink:href="#p" />
    </defs>
    ...
</svg>

It is also possible to cause indirect circular references:

<svg>
    <defs>
        <mask id="z" />
            <rect mask="url(#z)" />
        </mask>
    </defs>
    <ellipse mask="url(#z)" />
</svg>

The legacy engine solved this in an ad-hoc way in various places in the engine; it tried to break cycles before rendering, but still needed cycle protections in various places, since the solution was never unified or complete.

In February 2024 we provided a unified solution for LBSE by introducing SVGVisitedRendererTracking; see this commit for more. In the new approach, we don’t attempt to remove cycles, but detect them everywhere upon usage and stop processing in well-defined ways, all centralized in SVGVisitedRendererTracking.

Nested mask/pattern slowness

In April 2024, we addressed the slowness problems with nested masks/patterns. As an example, consider this for nested masks:

<svg>
    <defs>
        <mask id="z" />
            <rect id="1" mask="url(#y)" />
            <rect id="2" mask="url(#y)" />
            ...
        </mask>
        <mask id="y" />
            <rect id="1" mask="url(#x)" />
            <rect id="2" mask="url(#x)" />
            ...
        </mask>
        ...
    </defs>
    <ellipse mask="url(#z)" />
</svg>

For this example, the complexity can be increased at will by adding more masks and contents per mask.

The solution was twofold:

  • For masks, we realized bounding box calculations for a mask were not affected by masks used in the mask contents, so we could cut off bounding box calculations for nested masks.
  • For both masks and patterns, we added caching of image buffers per resource client so nested masks/patterns that are already encountered can reuse the image buffer cache.

See optimizations here for nested masks and patterns.

Next steps

For the short and mid-term, the plan is to make LBSE at least as good as legacy in regards to test coverage; i.e., all tests that pass in legacy should pass in LBSE. We have made a lot of progress over the last seven months just because of the amount of SVG resources that were implemented, but for example ,we will need to have SVG filters in place to pass this goal.

Another goal is to make sure LBSE passes all security requirements, as failing that would be a blocker to replacing the current engine. Fortunately, we are already taking this into account in several ways, such as adopting a lot of good smart pointer practices.

Finally, a big goal will be for LBSE to perform well on certain benchmarks like MotionMark, since WebKit has a golden rule to never ship a performance regression. So far there has not been an explicit focus on performance, and we know there are likely optimizations possible in RenderLayer usage, both in reducing the number of RenderLayer objects we create in certain situations as well as a possible reduction in complexity of RenderLayer for LBSE usage.

All in all, we are very pleased with the results and the progress we made in the last seven months. We at Igalia look forward to finishing the work to get the new engine in a shippable state in the near future!

May 21, 2024 12:00 AM

May 20, 2024

Frédéric Wang: Time travel debugging of WebKit with rr

Igalia WebKit

Introduction

rr is a debugging tool for Linux that was originally developed by Mozilla for Firefox. It has long been adopted by Igalia and other web platform developers for Chromium and WebKit too. Back in 2019, there were breakout sessions on this topic at the Web Engines Hackfest and BlinkOn.

For WebKitGTK, the Flatpak SDK provides a copy of rr, but recently I was unable to use the instructions on trac.webkit.org. Fortunately, my colleague Adrián Pérez suggested using a direct build without flatpak or the bubblewrap sandbox, and that indeed solved my problem. I thought it might be interesting to share this information with others, so I decided to write this blog post.

Disclaimer: The build instructions below may be imperfect, will likely become outdated, and are in any case not a replacement for the official ones for WebKitGTK development. Use them at your own risk!

CMake configuration

The approach that worked for me was thus to perform a direct build from my system. I came up with the following configuration step:

cmake -S. -BWebKitBuild/Release \
   -DCMAKE_BUILD_TYPE=Release \
   -DCMAKE_INSTALL_PREFIX=$HOME/WebKit/WebKitBuild/install/Release \
   -GNinja -DPORT=GTK -DENABLE_BUBBLEWRAP_SANDBOX=OFF \
   -DDEVELOPER_MODE=ON -DDEVELOPER_MODE_FATAL_WARNINGS=OFF \
   -DENABLE_TOOLS=ON -DENABLE_LAYOUT_TESTS=ON

where:

  • The -B option specifies the build directory, which is traditionnaly called WebKitBuild/ for the WebKit project.
  • CMAKE_BUILD_TYPE specifies the build type, e.g. optimized release builds (Release, corresponding to --release for the offical script) or debug builds with assertions (Debug, corresponding to --debug) 1.
  • CMAKE_INSTALL_PREFIX specifies the installation directory, which I place inside WebKitBuild/install/ 2.
  • The -G option specifies the build system generator. I used Ninja, which is the default for the offical script too.
  • -DPORT=GTK is for building WebKitGTK. I haven’t tested rr with other Linux ports.
  • -DENABLE_BUBBLEWRAP_SANDBOX=OFF was suggested by Adrián. The bubblewrap sandbox probably does not make sense without flatpak, so it should be safe to disable it anyway.
  • I extracted the other -D flags from the official script, trying to stay as close as possible to what it provides for WebKit development (being able to run layout tests, building the Tools/, ignoring fatal warnings, etc).

Needless to say, the advantage of using flatpak is that it automatically downloads and install all the required dependencies. But if you do your own build, you need to figure out what they are and perform the setup manually. Generally, this is straightforward using your distribution’s package manager, but there can be some tricky exceptions 3.

While we are still at the configuration step, I believe it’s worth sharing two more tricks for WebKit developers:

  • You can use -DENABLE_SANITIZERS=address to produce Asan builds or builds with other sanitizers.
  • You can use -DCMAKE_CXX_FLAGS="-DENABLE_TREE_DEBUGGING" in release builds if you want to get access to the tree debugging functions (ShowRenderTree and the like). This flag is turned on by default for debug builds.

Building and running WebKit

Once the configure step is successful, you can build and install WebKit using the following CMake command 2.

cmake --build WebKitBuild/Release --target install

When that operation completes, you should be able to run MiniBrowser with the following command:

LD_LIBRARY_PATH=WebKitBuild/install/Release/lib ./WebKitBuild/Release/bin/MiniBrowser

For WebKitTestRunner, some extra environment variables are necessary 2:

TEST_RUNNER_INJECTED_BUNDLE_FILENAME=$HOME/WebKit/WebKitBuild/Release/lib/libTestRunnerInjectedBundle.so LD_LIBRARY_PATH=WebKitBuild/install/Release/lib ./WebKitBuild/Release/bin/WebKitTestRunner filename.html

You can also use the official scripts, Tools/Script/run-minibrowser and Tools/Script/run-webkit-tests. They expect some particular paths, but a quick workaround is to use a symbolic link:

ln -s $HOME/WebKit/WebKitBuild $HOME/WebKit/WebKitBuild/GTK

Using rr for WebKit debugging

rr is generally easily installable from your distribution’s package manager. However, as stated on the project wiki page:

Support for the latest hardware and kernel features may require building rr from Github master.

Indeed, using the source has always worked best for me to avoid mysterious execution failures when starting the recording 4.

If you are not familiar with rr, I strongly invite you to take a look at the overview on the project home page or at some of the references I mentioned in the introduction. In any case, the first step is to record a trace by passing the program and arguments to rr. For example, to record a trace for MiniBrowser:

LD_LIBRARY_PATH=WebKitBuild/install/Debug/lib rr ./WebKitBuild/Debug/bin/MiniBrowser https://www.igalia.com/

After the program exits, you can replay the recorded trace as many times as you want. For hard-to-reproduce bugs (e.g. non-deterministic issues or involving a lot of manual steps), that means you only need to be able to record and reproduce the bug once and then can just focus on debugging. You can even turn off your machine after hours of exhausting debugging, then continue the effort later when you have more time and energy! The trace is played in a deterministic way, always using the same timing and pointer addresses. You can use most gdb commands (to run the program, interrupt it, and inspect data), but the real power comes from new commands to perform reverse execution!

Before coming to that, let’s explain how to handle programs with multiple processes, which is the case for WebKit and modern browsers in general. After you recorded a trace, you can display the pids of all recorded processes using the rr ps command. For example, we can see in the following output that the MiniBrowser process (pid 24103) actually forked three child processes, including the Network Process (pid 24113) and the Web Process (24116):

PID     PPID    EXIT    CMD
24103   --      0       ./WebKitBuild/Debug/bin/MiniBrowser https://www.igalia.com/
24113   24103   -9      ./WebKitBuild/Debug/bin/WebKitNetworkProcess 7 12
24115   24103   1       (forked without exec)
24116   24103   -9      ./WebKitBuild/Debug/bin/WebKitWebProcess 15 15

Here is a small debugging session similar to the single-process example from Chromium Chronicle #13 5. We use the option -p 24116 to attach the debugger to the Web Process and -e to start debugging from where it exited:

rr replay -p 24116 -e
(rr) break RenderFlexibleBox::layoutBlock
(rr) rc # Run back to the last layout call
Thread 2 hit Breakpoint 1, WebCore::RenderFlexibleBox::layoutBlock (this=0x7f66699cc400, relayoutChildren=false) at /home/fred/src-obj/WebKit/Source/WebCore/rendering/RenderFlexibleBox.cpp:420
(rr) # Inspect anything you want here. To find the previous Layout call on this object:
(rr) cond 1 this == 0x7f66699cc400
(rr) rc
Thread 2 hit Breakpoint 1, WebCore::RenderFlexibleBox::layoutBlock (this=0x7f66699cc400, relayoutChildren=false) at /home/fred/src-obj/WebKit/Source/WebCore/rendering/RenderFlexibleBox.cpp:420
420     {
(rr) delete 1
(rr) watch -l m_style.m_nonInheritedFlags.effectiveDisplay # Or find the last time the effective display was changed
Thread 4 hit Hardware watchpoint 2: -location m_style.m_nonInheritedFlags.effectiveDisplay

Old value = 16
New value = 0
0x00007f6685234f39 in WebCore::RenderStyle::RenderStyle (this=0x7f66699cc4a8) at /home/fred/src-obj/WebKit/Source/WebCore/rendering/style/RenderStyle.cpp:176
176     RenderStyle::RenderStyle(RenderStyle&&) = default;

rc is an abbreviation for reverse-continue and continues execution backward. Similarly, you can use reverse-next, reverse-step and reverse-finish commands, or their abbreviations. Notice that the watchpoint change is naturally reversed compared to normal execution: the old value (sixteen) is the one after intialization, while the new value (zero) is the one before initialization!

Restarting playback from a known point in time

rr also has a concept of “event” and associates a number to each event it records. They can be obtained by the when command, or printed to the standard output using the -M option. To elaborate a bit more, suppose you add the following printf in RenderFlexibleBox::layoutBlock:

@@ -423,6 +423,8 @@ void RenderFlexibleBox::layoutBlock(bool relayoutChildren, LayoutUnit)
     if (!relayoutChildren && simplifiedLayout())
         return;

+    printf("this=%p\n", this);
+

After building, recording and replaying again, the output should look like this:

$ rr -M replay -p 70285 -e # replay with the new PID of the web process.
...
[rr 70285 57408]this=0x7f742203fa00
[rr 70285 57423]this=0x7f742203fc80
[rr 70285 57425]this=0x7f7422040200
...

Each printed output is now annotated with two numbers in bracket: a PID and an event number. So in order to restart from when an interesting output happened (let’s say [rr 70285 57425]this=0x7f7422040200), you can now execute run 57425 from the debugging session, or equivalently:

rr replay -p 70285 -g 57425

Older traces and parallel debugging

Another interesting thing to know is that traces are stored in ~/.local/share/rr/ and you can always specify an older trace to the rr command e.g. rr ps ~/.local/share/rr/MiniBrowser-0. Be aware that the executable image must not change, but you can use rr pack to be able to run old traces after a rebuild, or even to copy traces to another machine.

To be honest, most the time I’m just using the latest trace. However, one thing I’ve sometimes found useful is what I would call the “parallel debugging” technique. Basically, I’m recording one trace for a testcase that exhibits the bug and another one for a very similar testcase (e.g. with one CSS property difference) that behaves correctly. Then I replay the two traces side by side, comparing them to understand where the issue comes from and what can be done to fix it.

The usage documentation also provides further tips, but this should be enough to get you started with time travel debugging in WebKit!

  1. RelWithDebInfo build type (which yields an optimized release build with debug symbols) might also be interesting to consider in some situations, e.g. debugging bugs that reproduce in release builds but not in debug builds. 

  2. Using an installation directory might not be necessary, but without that, I had trouble making the whole thing work properly (wrong libraries loaded or libraries not found).  2 3

  3. In my case, I chose the easiest path to disable some features, namely -DUSE_JPEGXL=OFF, -DUSE_LIBBACKTRACE=OFF, and -DUSE_GSTREAMER_TRANSCODER=OFF

  4. Incidentally, you are likely to get an error saying that perf_event_paranoid is required to be at most 1, which you can force using sudo sysctl kernel.perf_event_paranoid=1

  5. The equivalent example would probably have been to watch for the previous style change with watch -l m_style, but this was exceeding my hardware watchpoint limit, so I narrowed it down to a smaller observation scope. 

May 20, 2024 10:00 PM

May 13, 2024

WebKit Features in Safari 17.5

Surfin’ Safari

Happy May! It’s time for another release of Safari — our third significant update of 2024. With just a month until WWDC24 and the unveiling of what’s coming later this year, we are happy to get these 7 features and 22 bug fixes into the hands of your users today.

CSS

There are several exciting new CSS features in Safari 17.5, including text-wrap: balance, the light-dark() color function, and @starting-style, plus the ability to use feature queries with @import rules. Let’s look at how you can put each one to use.

Text wrap balance

On the web, with its flexible container widths, inconsistent lengths of content, and variation between browsers, it can feel impossible to avoid having text wrap in such a way that too few words end up all by themselves on a very short last line.

Very long text headline wrapping using the normal algorithm — which leaves a single word on the last line, all by itself

When type was set by hand, typographers would painstakingly avoid this undesirable result by manually moving content around. Over the decades, web developers have tried a series of different tricks to avoid orphans in CSS, in HTML, in JavaScript, and in content management systems. None work very well. The attempts usually feel hacky, laborious, and fragile.

To solve this and other frustrations, the CSS Working Group has defined three new options that you can use to change how text will wrap. You can switch from default wrapping to another style with text-wrap. WebKit for Safari 17.5 adds support for the first of these new options — balancing.

The text-wrap: balance rule asks the browser to “balance” the lines of text and make them all about the same length.

A very long headline wrapped using text-wrap: balance, so each of the three lines are the same length as each other — and none of them fill all the horizontal space available

You can see how now the text no longer fills the containing block — there’s a large amount of space on the right of the words. This is expected, and something you’ll want to think about as you decide when to use text-wrap: balance.

Where exactly each line of text will break when using text-wrap: balance may be slightly different in each browser. The CSS Text level 4 web standard leaves it up to each browser engine team to decide which algorithm they want to use in determining how exactly to wrap balanced text.

It can be computationally expensive for the browser to count characters and balance multiple lines of text, so the standard allows browsers to limit the number of lines that are balanced. Chromium browsers balance 6 or fewer lines, Firefox balances 10 or fewer, while Safari/WebKit balances an unlimited numbers of lines.

For now, Safari does not balance text if it’s surrounding a float or initial letter. And Safari disables the balancer if the content contains preserved tabs or soft hyphens.

Text wrap shorthands and longhands

The text-wrap property is actually a shorthand for two longhand properties: text-wrap-style and text-wrap-mode.

The text-wrap-mode property provides a mechanism for expressing whether or not text should wrap.

text-wrap-mode: wrap; /* initial value */
text-wrap-mode: nowrap;

The wrap value turns it on, and the nowrap value turns it off, just like the values for white-space. (In fact, text-wrap-mode is the newly introduced longhand of white-space.) WebKit added support for text-wrap-mode: wrap and nowrap in Safari 17.4.

The text-wrap-style property selects how to wrap. The initial value is auto — asking text to wrap in the way it has for decades. Or, you can choose a value to switch to another “style” of wrapping.

WebKit for Safari 17.5 adds support for text-wrap-style: balance, stable, and auto.

text-wrap-style: auto; /* initial value */
text-wrap-style: balance;
text-wrap-style: stable;

Of course, the text-wrap shorthand is a way to combine text-wrap-mode and text-wrap-style and declare them together. If you write text-wrap: balance it’s the same as text-wrap: wrap balance, meaning: “yes, please wrap, and when you do, please balance the text”.

Full support will eventually include three properties and six values. No browser supports everything yet, so be sure to look up support for the text-wrap, text-wrap-mode, and text-wrap-style properties, as well as the balance, pretty, stable, auto, wrap, and nowrap values.

The balance, pretty, and stable values will simply fall back to auto in browsers without support, so progressive enhancement is easy. You can use these values today, no matter how many of your users don’t yet have a browser with support. They will simply get auto-wrapped text, just like they would if you didn’t use text-wrap. Meanwhile, those users with support will get an extra boost of polish.

Dark mode and the light-dark() color function

More and more, users expect websites and web apps to support dark mode. Since Safari 12.1, the prefers-color-scheme media query has given you the ability to write code like this:

body {
  background: white;
  color: black;
}
@media (prefers-color-scheme: dark) {
  body {
    background: darkslategray;
    color: white;
  }
}

Or perhaps you’ve used variables to define colors for both light and dark mode at once, making it easier to use them everywhere.

:root {
  --background: white;
  --text: black;
}
@media (prefers-color-scheme: dark) {
  :root {
    --background: darkslategray;
    --text: white;
  }
}
body {
  background: var(--background);
  color: var(--text);
}

Well, now there’s a new option — the light-dark() function. It makes defining colors for dark mode even easier.

First, inform the browser you are providing a design for both light and dark modes with the color-scheme property. This prompts the browser to switch the default user agent styles when in dark mode, ensuring the form controls appear in dark mode, for example. It’s also required for light-dark() to work correctly.

:root {
  color-scheme: light dark;
}

Then, any time you define a color, you can use the light-dark() function to define the first color for light mode, and the second color for dark mode.

color: light-dark(black, white);
background-color: light-dark(white, darkslategray);

You can still use variables, if you’d like. Perhaps you want to structure your code like this.

:root {
  color-scheme: light dark;
  --background: light-dark(black, white);
  --text: light-dark(white, darkslategray);
}
body {
  background: var(--background);
  color: var(--text);
}

An often-asked question when learning about light-dark() is “does this only work for colors?” Yes, this function only works for colors. Use the prefers-color-scheme media query to define the rest of your color-scheme dependent styles.

Starting style

WebKit for Safari 17.5 adds support for @starting-style. It lets you define starting values for a particular element. This is needed to enable a transition when the element’s box is created (or re-created).

.alert {
  transition: background-color 2s;
  background-color: green;
  @starting-style {
    background-color: transparent;
  }
}

In the above example, the background-color will transition from transparent to green when the element is added to the document.

Many developers are excited to use @starting-style along with display: none interpolation. To do so, WebKit also needs to support animation of the display property, which has not yet shipped in Safari. You can test this use case today in Safari Technology Preview.

Features queries for importing CSS

WebKit for Safari 17.5 adds the supports() syntax to @import rules. Now you can conditionally import CSS files based on whether or not there’s support for a certain feature.

@import <url> supports(<feature>);

For example, you could load different stylesheets based on whether or not CSS Nesting is supported.

@import "nested-styles.css" supports(selector(&));
@import "unnested-styles.css" supports(not selector(&));

Or you could load certain CSS files when a browser does not have support for Cascade Layers. (Note that any @import rules with layer() will automatically be ignored in a browser without layer support.)

@import url("reset.css") layer(reset);
@import url("framework.css") layer(framework);
@import url("custom.css") layer(custom);

@import url("unlayered-fallback-styles.css") supports(not at-rule(@layer));

Or simply test for a feature. Here, these layout styles will only be loaded if Subgrid is supported.

@import url("layout.css") supports(grid-template-columns: subgrid);

WebCodecs

WebKit for Safari 17.5 adds support for AV1 to WebCodecs when an AV1 hardware decoder is available.

WebGL

WebKit for Safari 17.5 adds WebGL support for EXT_conservative_depth and NV_shader_noperspective_interpolation.

WKWebView

WKWebView adds support for logging MarketplaceKit errors to the JavaScript console. This will make errors easier to debug.

Bug Fixes and more

In addition to these new features, WebKit for Safari 17.5 includes work polishing existing features.

Accessibility

  • Fixed a bug preventing VoiceOver word echoing in some text fields. (122451549) (FB13592798)

Animations

  • Fixed flickering with multiple accelerated animations and direction changes. (117815004)

Authentication

  • Fixed excludeCredentials property being ignored during a passkey registration request. (124405037)

CSS

  • Fixed the proximity calculation for implicit @scope. (124640124)
  • Fixed the Grid track sizing algorithm logical height computation avoid unnecessary grid item updates. (124713418)
  • Fixed any @scope limit making the element out of scope. (124956673)

Forms

  • Fixed native text fields becoming invisible in dark mode. (123658326)
  • Fixed fallback native <select> rendering in dark mode. (123845293)

Media

  • Fixed scrolling for an element when a video element with pointer-events: none is placed over it. (118936715)
  • Fixed HTML5 <audio> playback to continue to the next media activity when in the background. (121268089) (FB13551577)
  • Fixed AV1 to decode in hardware on iPhone 15 Pro. (121924090)
  • Fixed audio distortion over internal speakers when streaming content in web browsers. (122590884)
  • Fixed firing loadeddata events for <audio> and <video> on page load. (124079735) (FB13675360)

Rendering

  • Fixed adjusting the size of the scrollable area when changing betwen non-overlay and overlay scrollbars. (117507268)
  • Fixed flickering when showing a layer on a painted background for the first time by avoiding async image decoding. (117533495)
  • Fixed line breaking before or between ruby sequences. (122663646)

Web API

  • Fixed mousemove events in an iframe when the mouse is clicked from outside the iframe and then moves into it while the button is held down. (120540148) (FB13517196)

Web Apps

  • Fixed several issues that caused Web Push to not show notifications when the web app or Safari was not already running. (124075358)

Web Inspector

  • Fixed info and debug buttons not appearing in the Console Tab until new console messages are displayed. (122923625)

WebRTC

  • Fixed WebCodecs to correctly use the VP9 hardware decoder. (123475343)
  • Fixed no incoming video in Teams VA. (124406255)
  • Fixed the camera pausing occasionally when torch is enabled. (124434403)

Updating to Safari 17.5

Safari 17.5 is available on iOS 17.5, iPadOS 17.5, macOS Sonoma 14.5, macOS Ventura, macOS Monterey and in visionOS 1.2.

If you are running macOS Ventura or macOS Monterey, you can update Safari by itself, without updating macOS. On macOS Ventura, go to  > System Settings > General > Software Update and click “More info…” under Updates Available.

To get the latest version of Safari on iPhone, iPad, or Apple Vision Pro, go to Settings > General > Software Update, and tap to update.

Feedback

We love hearing from you. To share your thoughts on Safari 17.5, find us on Mastodon at @jensimmons@front-end.social and @jondavis@mastodon.social. Or send a reply on X to @webkit. You can also follow WebKit on LinkedIn. If you run into any issues, we welcome your feedback on Safari UI, or your WebKit bug report about web technologies or Web Inspector. Filing issues really does make a difference.

Download the latest Safari Technology Preview on macOS to stay at the forefront of the web platform and to use the latest Web Inspector features.

You can also find this information in the Safari 17.5 release notes.

May 13, 2024 06:30 PM

April 16, 2024

Philippe Normand: From WebKit/GStreamer to rust-av, a journey on our stack’s layers

Igalia WebKit

In this post I’ll try to document the journey starting from a WebKit issue and ending up improving third-party projects that WebKitGTK and WPEWebKit depend on.

I’ve been working on WebKit’s GStreamer backends for a while. Usually some new feature needed on WebKit side would trigger work …

By Philippe Normand at April 16, 2024 08:15 PM

April 08, 2024

WPE WebKit Blog: WPE WebKit 2.44 highlights

Igalia WebKit

The WPE team released WPE WebKit 2.44 a few weeks ago. Let’s have a look at the most significant changes to the port for this release cycle.

DisplayLink is a WebCore feature that improves resource utilization and improves synchronization with vertical screen retrace. 2.44 adds an implementation of this feature for the WPE port that improves rendering performance.

Improved hardware-acceleration video decoding and rendering

When WebKit is using GStreamer 1.24 or newer, video playback can use the new support for DRM modifiers in the DMA-BUF sink. This improves video decoding and rendering, as it allows for zero-copy negotiation with the video decoders.

WebCodec API supported

WPE now supports the WebCodecs API, which allows web developers low-level access to video frames and audio chunks, a feature of importance for multimedia applications that need finer grain control over what gets played on the browser.

Other noteworthy changes

  • Support for the JPEG2000 image format has been removed. WebKit was the only major engine still supporting the format, which these days is rarely used. As a consequence, OpenJPEG is no longer a dependency. JPEG2000 should not be confused with JPEG-XL, which is still supported.
  • Support usage of libbacktrace, enabled by default at build time. This library provides quality stacktraces that can help developers and deployers more efficiently debug crashes in WPE-powered browsers.
  • Many memory and stability improvements, particularly on the multimedia backends.

For a complete list of changes, please check the releases page.

April 08, 2024 12:00 AM

March 20, 2024

Jani Hautakangas: Bringing WebKit back to Android.

Igalia WebKit

It’s been quite a while since the last blog post about WPE-Android, but that doesn’t mean WPE-Android hasn’t been in development. The focus has been on stabilizing the runtime and implementing the most crucial core features to make it easier to integrate new APIs into WPEView.

Main building blocks #

WPE-Android has three main building blocks:

  • Cerbero
    • Cross-platform build aggregator that is used to build WPE WebKit and all of its dependencies to Android.
  • WPEBackend-Android
    • Implements Android specific graphics buffer support and buffer sharing between WebKit UIProcess and WebProcess.
  • WPEView
    • Allows displaying web content in activity layout using WPEWebKit.

WPE-Android high-level design

What’s new #

The list of all work completed so far would be quite long, as there have been no official releases or public announcements, with the exception of the last blog post. Since that update, most of the efforts have been focused on ‘under the hood’ improvements, including enhancing stability, adding support for some core WebKit features, and making general improvements to the development infrastructure.

Here is a list of new features worth mentioning:

  • Based on WPE WebKit 2.42.1, the project was branched for the 2.42.x series. Future work will continue in the main branch for the next release.
  • Dropped support for 32-bit platforms (x86 and armv7). Only arm64-v8a and x86_64 are supported
  • Integration to Android main loop so that WPE WebKit GLib main loop is driven by Android main loop
  • Process-Swap On Navigation aka PSON
  • Added ASharedMemory support to WebKit SharedMemory
  • Hardware-accelerated multimedia playback
  • Fullscreen support
  • Cookies management
  • ANGLE based WebGL
  • Cross process fence insertion for composition synchronization with Surface Flinger
  • WebDriver support
  • GitHub Actions build bots
  • GitHub Actions WebDriver test bots

Demos #

WPEWebkit powered web view (WPEView) #

Demo uses WPE-Android MiniBrowser sample application to show basic web page loading and touch-based scrolling, usage of a simple cookie manager to clear the page’s date usage, and finally loads the popular “Aquarium sample” to show a smooth (60FPS) WebGL animation running thanks to HW acceleration support.

WebDriver #

Demo shows how to run WebDriver test with with emulator. Detailed instructions how to run WebDriver test can be found in README.md

Test requests a website through the Selenium remote webdriver. It then replaces the default behavior of window.alert on the requested page by injecting and executing a JavaScript snippet. After loading the page, it performs a click() action on the element that calls the alert. This results in the text ‘cheese’ being displayed right below the ‘click me’ link.

Test contains three building blocks:

  • WPE WebDriver running on emulator
  • HTTP server serving test.html web page
  • Selenium python test script executing the test

What’s next #

We have ambitious plans for the coming months regarding the development of WPE Android, focusing mainly on implementing additional features, stabilization, and performance improvements.

As for implementing new features, now that the integration with the WPE WebKit runtime has reached a more robust state, it’s time to start adding more of the APIs that are still missing in WPEView compared to other webviews on Android and to enable other web-facing features supported by WebKit. This effort, along with adding support for features like HTTP/2 and the remote Web Inspector, will be a major focus.

As for stabilization and performance, having WebDriver support will be very helpful as it will enable us to identify and fix issues promptly and thus help make WPE Android more stable and feature-complete. We would also like to focus on conformance testing compared to other web views on Android, which should help us prioritize our efforts.

The broader goal for this year is to develop WPE-Android into an easy-to-use platform for third-party projects, offering a compelling alternative to other webviews on the Android platform. We extend many thanks to the NLNet Foundation, whose support for WPE Android through a grant from the NGI Zero Core fund will be instrumental in helping us achieve this goal!

Try it yourself #

WPE-Android is still considered a prototype, and it’s a bit difficult to build. However, if you want to try it out, you can follow the instructions in the README.md file. Additionally, you can use the project’s issue tracker to report problems or suggest new features.

March 20, 2024 12:00 AM