March 20, 2019

Release Notes for Safari Technology Preview 78

Surfin’ Safari

Safari Technology Preview Release 78 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

This release covers WebKit revisions 242087-242836.

Pointer Events

  • Enabled support for Pointer Events by default (r242232)
  • Added support for Pointer Events on macOS (r242137)

Web Inspector

  • Enabled the CPU Usage timeline by default (r242741, r242739, r242567, r242243)
  • Added an Energy Impact section to the CPU Usage timeline (r242300)
  • Added Statistics and Sources sections to the CPU Usage timeline (r242562)
  • Added CPU Usage categorization per-thread and per-worker (r242466, r242104, r242242, r242740)
  • Added support for system accent color throughout the user interface (r242118, r242578, r242296)
  • Added node contextual menu items for event listeners sorted by node in the Elements tab (r242820)
  • Added support for shift-clicking a variable swatch to replace the text with the variable’s value in the Styles sidebar (r242602)
  • Added support for Control-Space to toggle the auto-completion menu in the Styles sidebar (r242218)
  • Added a go-to button for overridden CSS properties to jump to the effective property in the Styles sidebar (r242622)
  • Added the version number in the UI of the Audit tab (r242395)
  • Added export Canvas recording as HTML in the Canvas tab (r242809)
  • Changed to remove event listeners once a recording is ready in the Canvas tab (r242511)
  • Changed to stop showing the auto-stop UI when not inspecting a page in the Timelines tab (r242244)
  • Changed enabling “Stop when page loads” toggle to stop any active recording if the load event has already occurred in the Timelines tab (r242370)
  • Changed to allow image collections to be filtered by type in the Sources tab (r242829)
  • Changed breakpoints to be disabled when an audit is running (r242315)
  • Changed to not enable breakpoints when the source location changes in the Debugger tab (r242314)
  • Changed to highlight the node when hovering event listeners sorted by node in the Elements tab (r242590)
  • Fixed color swatches for valid inputs with leading or trailing whitespace in the Canvas tab (r242553)
  • Fixed DOM, URL, and Event breakpoints to grey out when all breakpoints are disabled in the Debugger tab (r242318)
  • Fixed the ability to select records in the bottom 16px of the timeline overview graph (r242510)
  • Fixed unreadable text in bezier curve editor numeric input fields in dark mode (r242151)
  • Fixed popover colors when transitioning to and from dark mode (r242144)
  • Fixed DOM event breakpoints firing when breakpoints are disabled (r242588)
  • Fixed frequent flashing of DOM node attributes in the Elements tab (r242322)
  • Fixed the keyboard shortcut for the Settings tab on non-US keyboards (r242821)
  • Fixed no results when opening to Search tab (r242105)
  • Improved handling past recordings in the Timelines tab (r242785)
  • Updated to only show changes for the given node in the CSS Changes sidebar (r242556)

WebDriver

  • Fixed string not getting terminated with a null character (r242352)

Web API

  • Adjusted XMLHttpRequest Content-Type handling (r242284)
  • Added support for the referrerpolicy attribute (r242534)
  • Implemented further CORS restrictions (r242786)

Accessibility

  • Added remote search support for keyboard focusable element search type (r242528)

Media

  • Removed HEVC as a codec requiring hardware support (r242357)

March 20, 2019 05:00 PM

March 19, 2019

Michael Catanzaro: Epiphany Technology Preview Upgrade Requires Manual Intervention

Igalia WebKit

Jan-Michael has recently changed Epiphany Technology Preview to use a separate app ID. Instead of org.gnome.Epiphany, it will now be org.gnome.Epiphany.Devel, to avoid clashing with your system version of Epiphany. You can now have separate desktop icons for both system Epiphany and Epiphany Technology Preview at the same time.

Because flatpak doesn’t provide any way to rename an app ID, this means it’s the end of the road for previous installations of Epiphany Technology Preview. Manual intervention is required to upgrade. Fortunately, this is a one-time hurdle, and it is not hard:

$ flatpak uninstall org.gnome.Epiphany

Uninstall the old Epiphany…

$ flatpak install gnome-apps-nightly org.gnome.Epiphany.Devel org.gnome.Epiphany.Devel.Debug

…install the new one, assuming that your remote is named gnome-apps-nightly (the name used locally may differ), and that you also want to install debuginfo to make it possible to debug it…

$ mv ~/.var/app/org.gnome.Epiphany ~/.var/app/org.gnome.Epiphany.Devel

…and move your personal data from the old app to the new one.

Then don’t forget to make it your default web browser under System Settings -> Details -> Default Applications. Thanks for testing Epiphany Technology Preview!

By Michael Catanzaro at March 19, 2019 06:39 PM

March 12, 2019

On the Road to WebRTC 1.0, Including VP8

Surfin’ Safari

Safari 11 was the first Safari version to support WebRTC. Since then, we have worked to continue improving WebKit’s implementation and compliance with the spec. I am excited to announce major improvements to WebRTC in Safari 12.1 on iOS 12.2 and macOS 10.14.4 betas, including VP8 video codec support, video simulcast support and Unified Plan SDP (Session Description Protocol) experimental support.

VP8 Video Codec

The VP8 video codec is widely used in existing WebRTC solutions. It is now supported as a WebRTC-only video codec in Safari 12.1 on both iOS and macOS betas. By supporting both VP8 and H.264, Safari 12.1 can exchange video with any other WebRTC endpoint. H.264 is the default codec for Safari because it is backed by hardware acceleration and tuned for real-time communication. This provides a great user experience and power efficiency. We found that, on an iPhone 7 Plus in laboratory conditions, the use of H.264 on a 720p video call increases the battery life by up to an hour compared to VP8. With H.264, VP8 and Unified Plan, Safari can mix H.264 and VP8 on a single connection. It is as simple as doing the following:

const localStream = await navigator.mediaDevices.getUserMedia({audio: true, video: true});
var pc = new RTCPeerConnection();
// Create a transceiver to send video.
pc.addTransceiver('video', { direction: 'sendonly'}).sender.replaceTrack(localStream.getVideoTracks()[0]);
// Create a transceiver to receive video.
pc.addTransceiver('video', { direction: 'recvonly'});
// Send the offer to the other party.
sendOffer(await pc.createOffer());
// Other party selects H.264 for the 1st transceiver and VP8 for the 2nd transceiver.

Video Simulcast

To further improve WebRTC support for multi-party video conferencing, simulcast is now supported for both H.264 and VP8. Kudos to the libwebrtc community, including Cosmo Software, for making great progress in that important area. Simulcast is a technique that encodes the same video content with different encoding parameters, typically different frame sizes and bit rates. This is particularly useful when the same content is sent to several clients through a central server, called an SFU. As the clients might have different constraints (screen size, network conditions), the SFU is able to send the most suitable stream to each client. Each individual encoding can be controlled using RTCRtpSender.setParameters on the sender side. Simulcast is currently activated using SDP munging. Future work should allow simulcast activation using RTCPeerConnection.addTransceiver, as per specification.

Unified Plan

WebRTC uses SDP as the format for negotiating the configuration of a connection. While previous versions of Safari used Plan B SDP only, Safari is now transitioning to the standardized version known as Unified Plan SDP. Unified Plan SDP can express WebRTC configurations in a much more flexible way, as each audio or video stream transmission can be configured independently.

If your website uses at most one audio and one video track per connection, this transition should not require any major changes. If your website uses connections with more audio or video tracks, adoption may be required. With Unified Plan SDP enabled, the support of the WebRTC 1.0 API, in particular the transceiver API, is more complete and spec-compliant than ever. To detect whether Safari uses Unified Plan, you can use feature detection:

function usesUnifiedPlan()
{
return RTCRtpTransceiver.prototype.hasOwnProperty("currentDirection”);
}

Unified Plan is an experimental feature that is currently turned on by default in Safari Technology Preview and turned off by default in Safari in iOS 12.2 and macOS 10.14.4 betas. This can be turned on using the Develop menu on macOS and Safari settings on iOS.

Additional API updates

Safari also comes with additional improvements, including better support of capture device selection, experimental support of the screen capture API, and deprecation of the WebRTC legacy API.

Web applications sometimes want to select the same capture devices used on a previous call. Device IDs will remain stable across browsing sessions as soon as navigator.mediaDevices.getUserMedia is successfully called once. Your web page can implement persistent device selection as follows:

let stream;
let deviceId = window.localStorage.getItem('preferredMicrophoneId');
const constraint = deviceId ? {audio : {deviceId: {exact: deviceId}}} : {audio: true};
try {
stream = await navigator.mediaDevices.getUserMedia(constraint);
} catch (e) {
// ...
}
deviceId = stream.getAudioTracks()[0].getSettings().deviceId;
window.localStorage.setItem('preferredMicrophoneId', deviceId);

Existing fingerprinting mitigations remain in place, including the filtering of information provided by navigator.mediaDevices.enumerateDevices as detailed in the blog post, “A Closer Look Into WebRTC”.

Initial support of the screen capture API is now available as an experimental feature in Safari Technology Preview. By calling navigator.mediaDevices.getDisplayMedia, a web application can capture the main screen on macOS.

Following the strategy detailed in “A Closer Look Into WebRTC”, the WebRTC legacy API was disabled by default in Safari 12.0. Support for the WebRTC legacy API is removed from iOS 12.2 and macOS 10.14.4 betas. Should your application need support of the WebRTC legacy API, we recommend the use of the open source adapter.js library as a polyfill.

Feedback

We always appreciate your feedback. Send us bug reports, or feel free to tweet @webkit on Twitter, or email web-evangelist@apple.com.

March 12, 2019 05:00 PM

March 06, 2019

Release Notes for Safari Technology Preview 77

Surfin’ Safari

Safari Technology Preview Release 77 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

This release covers WebKit revisions 241432-242087.

WebRTC

  • Fixed getUserMedia with an ideal deviceId constraint to always select the correct device (r241489)
  • Made navigator.mediaDevices SecureContext (r241602)

Web API

  • Fixed WebSocket to not fire events after being stopped (r241599)
  • Fixed Same-Site Lax cookies to be sent with cross-site redirect from a client-initiated load (r241918)
  • Updated the MIME type parser (r241863)

CSS Grid

  • Changed to handle indefinite percentages in fit-content() (r241746)

Accessibility

  • Fixed HTML5 <footer> tag to be read as an ARIA Landmark to VoiceOver (r242051)
  • Fixed incorrectly mapping the <meter> element to progressbar (r241989)

Media

  • Fixed setting fullscreen pseudo-classes outside of root when entering fullscreen from inside of a shadow root (r241484)
  • Changed inline WebVTT styles to start with ::cue (r241608)

Web Inspector

  • Added emulation of user gestures from Web Inspector console (r241633)
  • Added built-in accessibility tests to the Audit tab (r241639)
  • Added a macOS user-agent option in the device settings menu (r241870)
  • Added an experimental setting to merge Resources and Debugger tabs into a single Sources tab (r242049)
  • Changed how selected items are tracked in trees and tables to improve performance (r241652)
  • Changed Command-/ (⌘/) to toggle the edited property in the Styles sidebar (r242063)
  • Changed typing ; to not focus on the next property when there’s open parenthesis or open comment in the Styles sidebar (r241740)
  • Changed clicking a CSS property or selector to always select its text in the Styles sidebar (r241980)
  • Fixed all actions after an offscreen path modification getting marked as offscreen path errors in the Canvas tab (r241616)
  • Fixed an issue where taking a second Canvas recording wouldn’t select the Initial State of the new recording (r241601)
  • Fixed clearing the selection when dragging a selection outside of the selected element in the Console (r242078)
  • Fixed the dim appearance of commas in CSS selectors for dark mode (r241638)
  • Fixed the DOM Content Loaded blue marker for dark mode (r241975)
  • Fixed the Network Overview Graph segments box shadow for dark mode (r242072)
  • Fixed node-specific event breakpoints to get cleared or recalculated on navigation (r241874)
  • Fixed highlighting a node when hovering inside an object preview (r242076)
  • Fixed the navigation sidebar showing “No Search Results” when a slow search is in progress (r242080)
  • Fixed Scripts Timeline loading indicator margins in right-to-left mode (r241857)
  • Fixed valid values in style attributes getting reported as unsupported property values in the Styles sidebar (r241623)
  • Improved the Scripts timeline on-going-recording in dark mode (r241757)
  • Improved categorization of JavaScript Class instances in the Heap Snapshot instances view (r241787)
  • Provided a way to avoid a Timeline recording auto-stop on reload (r241750)
  • Provided a way to make searches case-sensitive or use a regular expression (r242018)

Bug Fixes

  • Fixed a bug that caused loading on some sites to hang in Safari Technology Preview (r241658)

Safari app Extensions

  • Made the window.safari object available in frames opened to safari-extension:// resources

March 06, 2019 09:00 PM

February 26, 2019

Frédéric Wang: Review of Igalia's Web Platform activities (H2 2018)

Igalia WebKit

This blog post reviews Igalia’s activity around the Web Platform, focusing on the second semester of 2018.

Projects

MathML

During 2018 we have continued discussions to implement MathML in Chromium with Google and people interested in math layout. The project was finally launched early this year and we have encouraging progress. Stay tuned for more details!

Javascript

As mentioned in the previous report, Igalia has proposed and developed the specification for BigInt, enabling math on arbitrary-sized integers in JavaScript. We’ve continued to land patches for BigInt support in SpiderMonkey and JSC. For the latter, you can watch this video demonstrating the current support. Currently, these two support are under a preference flag but we hope to make it enable by default after we are done polishing the implementations. We also added support for BigInt to several Node.js APIs (e.g. fs.Stat or process.hrtime.bigint).

Regarding “object-oriented” features, we submitted patches private and public instance fields support to JSC and they are pending review. At the same time, we are working on private methods for V8

We contributed other nice features to V8 such as a spec change for template strings and iterator protocol, support for Object.fromEntries, Symbol.prototype.description, miscellaneous optimizations.

At TC39, we maintained or developed many proposals (BigInt, class fields, private methods, decorators, …) and led the ECMAScript Internationalization effort. Additionally, at the WebAssembly Working Group we edited the WebAssembly JS and Web API and early version of WebAssembly/ES Module integration specifications.

Last but not least, we contributed various conformance tests to test262 and Web Platform Tests to ensure interoperability between the various features mentioned above (BigInt, Class fields, Private methods…). In Node.js, we worked on the new Web Platform Tests driver with update automation and continued porting and fixing more Web Platform Tests in Node.js core.

We also worked on the new Web Platform Tests driver with update automation, and continued porting and fixing more Web Platform Tests in Node.js core. Outside of core, we implemented the initial JavaScript API for llnode, a Node.js/V8 plugin for the LLDB debugger.

Accessibility

Igalia has continued its involvement at the W3C. We have achieved the following:

We are also collaborating with Google to implement ATK support in Chromium. This work will make it possible for users of the Orca screen reader to use Chrome/Chromium as their browser. During H2 we began implementing the foundational accessibility support. During H1 2019 we will continue this work. It is our hope that sufficient progress will be made during H2 2019 for users to begin using Chrome with Orca.

Web Platform Predictability

On Web Platform Predictability, we’ve continued our collaboration with AMP to do bug fixes and implement new features in WebKit. You can read a review of the work done in 2018 on the AMP blog post.

We have worked on a lot of interoperability issues related to editing and selection thanks to financial support from Bloomberg. For example when deleting the last cell of a table some browsers keep an empty table while others delete the whole table. The latter can be problematic, for example if users press backspace continuously to delete a long line, they can accidentally end up deleting the whole table. This was fixed in Chromium and WebKit.

Another issue is that style is lost when transforming some text into list items. When running execCommand() with insertOrderedList/insertUnorderedList on some styled paragraph, the new list item loses the original text’s style. This behavior is not interoperable and we have proposed a fix so that Firefox, Edge, Safari and Chrome behave the same for this operation. We landed a patch for Chromium. After discussion with Apple, it was decided not to implement this change in Safari as it would break some iOS rich text editor apps, mismatching the required platform behavior.

We have also been working on CSS Grid interoperability. We imported Web Platform Tests into WebKit (cf bugs 191515 and 191369 and at the same time completing the missing features and bug fixes so that browsers using WebKit are interoperable, passing 100% of the Grid test suite. For details, see 191358, 189582, 189698, 191881, 191938, 170175, 191473 and 191963. Last but not least, we are exporting more than 100 internal browser tests to the Web Platform test suite.

CSS

Bloomberg is supporting our work to develop new CSS features. One of the new exciting features we’ve been working on is CSS Containment. The goal is to improve the rendering performance of web pages by isolating a subtree from the rest of the document. You can read details on Manuel Rego’s blog post.

Regarding CSS Grid Layout we’ve continued our maintenance duties, bug triage of the Chromium and WebKit bug trackers, and fixed the most severe bugs. One change with impact on end users was related to how percentages row tracks and gaps work in grid containers with indefinite size, the last spec resolution was implemented in both Chromium and WebKit. We are finishing the level 1 of the specification with some missing/incomplete features. First we’ve been working on the new Baseline Alignment algorithm (cf. CSS WG issues 1039, 1365 and 1409). We fixed related issues in Chromium and WebKit. Similarly, we’ve worked on Content Alignment logic (see CSS WG issue 2557) and resolved a bug in Chromium. The new algorithm for baseline alignment caused an important performance regression for certain resizing use cases so we’ve fixed them with some performance optimization and that landed in Chromium.

We have also worked on various topics related to CSS Text 3. We’ve fixed several bugs to increase the pass rate for the Web Platform test suite in Chromium such as bugs 854624, 900727 and 768363. We are also working on a new CSS value ‘break-spaces’ for the ‘white-space’ property. For details, see the CSS WG discussions: issue 2465 and pull request. We implemented this new property in Chromium under a CSSText3BreakSpaces flag. Additionally, we are currently porting this implementation to Chromium’s new layout engine ‘LayoutNG’. We have plans to implement this feature in WebKit during the second semester.

Multimedia

  • WebRTC: The libwebrtc branch is now upstreamed in WebKit and has been tested with popular servers.
  • Media Source Extensions: WebM MSE support is upstreamed in WebKit.
  • We implemented basic support for <video> and <audio> elements in Servo.

Other activities

Web Engines Hackfest 2018

Last October, we organized the Web Engines Hackfest at our A Coruña office. It was a great event with about 70 attendees from all the web engines, thank you to all the participants! As usual, you can find more information on the event wiki including link to slides and videos of speakers.

TPAC 2018

Again in October, but this time in Lyon (France), 12 people from Igalia attended TPAC and participated in several discussions on the different meetings. Igalia had a booth there showcasing several demos of our last developments running on top of WPE (a WebKit port for embedded devices). Last, Manuel Rego gave a talk on the W3C Developers Meetup about how to contribute to CSS.

This.Javascript: State of Browsers

In December, we also participated with other browser developers to the online This.Javascript: State of Browsers event organized by ThisDot. We talked more specifically about the current work in WebKit.

New Igalians

We are excited to announce that new Igalians are joining us to continue our Web platform effort:

  • Cathie Chen, a Chinese engineer with about 10 years of experience working on browsers. Among other contributions to Chromium, she worked on the new LayoutNG code and added support for list markers.

  • Caio Lima a Brazilian developer who recently graduated from the Federal University of Bahia. He participated to our coding experience program and notably worked on BigInt support in JSC.

  • Oriol Brufau a recent graduate in math from Barcelona who is also involved in the CSSWG and the development of various browser engines. He participated to our coding experience program and implemented the CSS Logical Properties and Values in WebKit and Chromium.

Coding Experience Programs

Last fall, Sven Sauleau joined our coding experience program and started to work on various BigInt/WebAssembly improvements in V8.

Conclusion

We are thrilled with the web platform achievements we made last semester and we look forward to more work on the web platform in 2019!

February 26, 2019 11:00 PM

Miguel A. Gómez: Hole punching in WPE

Igalia WebKit

As you may (or may not) know, WPE (and WebKitGtk+ if the proper flags are enabled) uses OpengGL textures to render the video frames during playback.

In order to do this, WPE creates a playbin and uses a custom bin as videosink. This bin is composed by some GStreamer-GL components together with an appsink. The GL components ensure that the video frames are uploaded to OpenGL textures, while the appsink allows the player to get a signal when a new frame arrives. When this signal is emitted, the player gets the frame as a texture from the appsink and sends it to the accelerated compositor to be composed with the rest of the layers of the page.

This process is quite fast due to the hardware accelerated drawings, and as the video frames are just another layer that is composited, it allows them to be transformed and animated: the video can be scaled, rotated, moved around the page, etc.

But there are some platforms where this approach is not viable, maybe because there’s no OpenGL support, or it’s too slow, or maybe the platform has some kind of fast path support to take the decoded frames to the display. For these cases, the typical solution is to draw a transparent rectangle on the brower, in the position where the video should be, and then use some platform dependent way to put the video frames in a display plane below the browser, so they are visible through the transparent rectangle. This approach is called hole punching, as it refers to punching a hole in the browser to be able to see the video.

At Igalia we think that supporting this feature is interesting, and following our philosophy of collaborating upstream as much as possible, we have added two hole punching approaches to the WPE upstream trunk: GStreamer hole punch and external hole punch.

GStreamer hole punch

The idea behind this implementation is to use the existent GStreamer based MediaPlayer to perform the media playback, but replace the appsink (and maybe other GStreamer elements) with a platform dependant video sink that is in charge of putting the video frames on the display. This can be enabled with the -DUSE_GSTREAMER_HOLEPUNCH flag.

Of course, the current implementation is not complete cause the platform dependent bits need to be added to have the full functionality. What it currently does is to use a fakevideosink so the video frames are not shown, and draw the transparent rectangle on the position where the video should be. If you enable the feature and play a video, you’ll see the transparent rectangle and you’ll be able to hear the video sound (and even use the video controls as they work), but nothing else will happen.

In order to have the full functionality there are a couple of places in the code that need to be modified to create the appropriate platform dependend elements. These two places are inside MediaPlayerPrivateGStreamerBase.cpp, and they are the createHolePunchVideoSink() and setRectangleToVideoSink() functions.

GstElement* MediaPlayerPrivateGStreamerBase::createHolePunchVideoSink()
{
    // Here goes the platform-dependant code to create the videoSink. As a default
    // we use a fakeVideoSink so nothing is drawn to the page.
    GstElement* videoSink =  gst_element_factory_make("fakevideosink", nullptr);

    return videoSink;
}
static void setRectangleToVideoSink(GstElement* videoSink, const IntRect& rect)
{
    // Here goes the platform-dependant code to set to the videoSink the size
    // and position of the video rendering window. Mark them unused as default.
    UNUSED_PARAM(videoSink);
    UNUSED_PARAM(rect);
}

The first one, createHolePunchVideoSink() needs to be modified to create the appropriate video sink to use for the platform. This video sink needs to have some method that allows setting the position where the video frames are to be displayed, and the size they should have. And this is where setRectangleToVideoSink() comes into play. Whenever the transparent rectangle is painted by the browser, it will tell the video sink to render the frames to the appropriate position, and it does so using that function. So you need to modify that function to use the appropriate way to set the size and position to the video sink.

And that’s all. Once those changes are made the feature is complete, and the video should be placed exactly where the transparent rectangle is.

Something to take into account is that the size and position of the video rectangle are defined by the CSS values of the video element. The rectangle won’t be adjusted to fit the aspect ratio of the video, as that must be done by the platform video sink.

Also, the video element allows some animations to be performed: it can be translated and scaled, and it will properly notify the video sink about the animated changes. But, of course, it doesn’t support rotation or 3D transformations (as the normal video playback does). Take into account that there might be a small desynchronization between the transparent rectangle and the video frames size and position, due to the asynchronicity of some function calls.

Playing a video with GStreamer hole punch enabled.

External hole punch

Unlike the previous feature, this one doesn’t rely at all on GStreamer to perform the media playback. Instead, it just paints the transparent rectangle and lets the playback to be handled entirely by an external player.

Of course, there’s still the matter about how to synchronize the transparent rectangle position and the external player. There would be two ways to do this:

  • Implement a new WebKit MediaPlayerPrivate class that would communicate with the external player (through sockets, the injected bundle or any other way). WPE would use that to tell the platform media player what to play and where to render the result. This is completely dependant of the platform, and the most complex solution, but it would allow to use the browser to play content from any page without any change. But precisely as it’s completely platform dependant, this is not valid approach for upstream.
  • Use javascript to communicate with the native player, telling it what to play and where, and WPE would just paint the transparent rectangle. The problem with this is that we need to have control of the page to add the javascript code that controls the native player, but on the other hand, we can implement a generic approach on WPE to paint the transparent rectangle. This is the option that was implemented upstream.

So, how can this feature be used? It’s enabled by using the -DUSE_EXTERNAL_HOLEPUNCH flag, and what it does is add a new dummy MediaPlayer to WPE that’s selected to play content of type video/holepunch. This dummy MediaPlayer will draw the transparent rectangle on the page, according to the CSS values defined, and won’t do anything else. It’s up to the page owners to add the javascript code required to initiate the playback with the native player and position the output in the appropriate place under the transparent rectangle. To be a bit more specific, the dummy player will draw the transparent rectangle once the type has been set to video/holepunch and load() is called on the player. If you have any doubt about how to make this work, you can give a look to the video-player-holepunch-external.html test inside the ManualTests/wpe directory.

This implementation doesn’t support animating the size and position of the video… well, it really does, as the transparent rectangle will be properly animated, but you would need to animate the native player’s output as well, and syncing the rectangle area and the video output is going to be a challenging task.

As a last detail, controls can be enabled using this hole punch implementation, but they are useless. As WPE doesn’t know anything about the media playback that’s happening, the video element controls can’t be used to handle it, so it’s just better to keep them disabled.

Using both implementations together

You may be wondering, is it possible to use both implementations at the same time? Indeed it is!! You may be using the GStreamer holepunch to perform media playback with some custom GStreamer elements. At some point you may find a video that is not supported by GStreamer and you can just set the type of the video element to video/holepunch and start the playback with the native player. And once that video is finished, start using the GStreamer MediaPlayer again.

Availability

Both hole punch features will be available on the upcoming stable 2.24 release (and, of course, on 2.25 development releases). I hope they are useful for you!

By magomez at February 26, 2019 10:25 AM

February 21, 2019

Intelligent Tracking Prevention 2.1

Surfin’ Safari

Note: Read about past updates to this technology in other blog posts about Intelligent Tracking Prevention, and the Storage Access API.

The beta releases of iOS 12.2 and Safari 12.1 on macOS High Sierra and Mojave include an updated version of Intelligent Tracking Prevention (ITP). For purposes of developer communication, we’ll refer to it as ITP 2.1.

With this update we further reduce trackers’ ability to establish user identities across sites.

A Single Set of Cookies Per Site

Previous versions of ITP allowed domains that were classified with tracking capabilities to store partitioned cookies, i.e. additional sets of cookies keyed off of the top site. As of ITP 2.1, partitioned cookies are no longer supported and third-parties classified with cross-site tracking capabilities now have to use the Storage Access API to get any cookie access.

The revised timeline for classified domains’ cookies and website data looks like this:

There are multiple reasons for this change:

  • Simplify for developers. We’ve received reports of existing site behavior getting tripped up by partitioned cookies. For instance, one site had a script which if it concluded that the user was logged out, overwrote its cookies with blank values to make sure no remnant data persisted (except for the cookie names). When this site was rendered in a third-party iframe, this cookie overwriting would instead create a new set of blank cookies.
  • Lower the memory footprint. As of ITP 1.1, partitioned cookies are session cookies which means they are not persisted. By getting rid of partitioned cookies, the storage session doesn’t have to keep multiple sets of cookies in memory.
  • Simplify ITP adoption for other WebKit ports. Partitioned cookies require support in the HTTP layer below WebKit. By removing support for them, we increase ITP’s flexibility and can make it work on more platforms.

Client-Side Cookies Capped to 7 Days of Storage

Cookies can either be set in HTTP responses or through the document.cookie API, the latter sometimes referred to as client-side cookies. With ITP 2.1, all persistent client-side cookies, i.e. persistent cookies created through document.cookie, are capped to a seven day expiry.

The reasons for this change cover privacy, security, and performance:

  • Cross-site trackers have started using first-party sites’ own cookie jars for the purpose of persistent tracking. The first-party storage space is especially troublesome for privacy since all tracker scripts in the first-party context can read and write each other’s data. Say social.example writes a user tracking ID as a news.example first-party cookie. Now analytics.example, adnetwork.example, and video.example can leverage or cross pollinate that user tracking ID through their scripts on news.example.
  • Cookies available in document.cookie can be stolen by speculative execution attacks on memory. Therefore, they should not carry sensitive information such as credentials.
  • Cookies available in document.cookie can be stolen by cross-site scripting attacks. Again, therefore, they should not carry sensitive information such as credentials.
  • The proliferation of cookies slows down page and resource loads since cookies are added to every applicable HTTP request. Additionally, many cookies have high entropy values which means they cannot be compressed efficiently. We come across sites with kilobytes of cookies sent in every resource request.
  • There is a size limit on outgoing cookie headers for performance reasons, and websites risk hitting this limit when cross-site trackers add first-party cookies. We’ve investigated reports of news site subscribers getting spuriously logged out, and found that trackers were adding so many cookies that the news site’s legitimate login cookie got pushed out.

Mike West (Google) has provided related statistics on cookies.

Will This Change Log Users Out?

Authentication cookies should be Secure and HttpOnly to protect them against man-in-the-middle attacks, cross-site scripting attacks, and speculative execution attacks. Cookies created through document.cookie cannot be HttpOnly which means authentication cookies should not be affected by the lifetime cap. If they are, you need to set your authentication cookies in an HTTP response and mark them Secure and HttpOnly.

Implementation Details

  • Only cookies created through document.cookie are affected by this change.
  • The change covers cookies created in first-party contexts as well as in third-party iframes.
  • Session cookies are not affected, they remain session cookies.
  • Persistent cookies that have an expiry shorter than seven days keep their shorter expiry.

Verified Partitioned Cache

WebKit implemented partitioned caches more than five years ago. A partitioned cache means cache entries for third-party resources are double-keyed to their origin and the first-party eTLD+1. This prohibits cross-site trackers from using the cache to track users. Even so, our research has shown that trackers, in order to keep their practices alive under ITP, have resorted to partitioned cache abuse. Therefore, we have developed the verified partitioned cache.

When a partitioned cache entry is created for a domain that’s classified by ITP as having cross-site tracking capabilities, the entry gets flagged for verification. After seven days, if there’s a cache hit for such a flagged entry, WebKit will act as if it has never seen this resource and load it again. The new response is then compared to the cached response and if they match in the ways we care about for privacy reasons, the verification flag is cleared and the cache entry is from that point considered legitimate. However, if the new response does not match the cache entry, the old entry is discarded, and a new one is created with the verification flag set, and the verification process starts over.

ITP currently does this verification for permanent redirects since that’s where we see abuse today.

Removed Compatibility Fix for Popups

As mentioned in the ITP 2.0 blog post, we implemented a temporary compatibility fix for third-party popups, automatically forwarding storage access under the opener page. In ITP 2.1, we’ve removed the compatibility fix for popups that dismiss before they receive user interaction, i.e. dismiss before they receive a tap, click, or text entry.

Removed Support For the DNT (“Do Not Track”) Signal

Apple believes that privacy is a fundamental human right and that users should not be tracked across the web without their knowledge or their consent. Intelligent Tracking Prevention was enabled by default for all users when it was released in 2017, and prevented users from being tracked by third parties they didn’t engage with. ITP has been enhanced since then in response to trackers abusing web technologies to track users and additional on-by-default privacy functionality has been added to Safari, such as browser fingerprinting defenses.

The introduction of ITP 2.1 coincides with Safari removing support for the Do Not Track (DNT) signal. The DNT signal was an attempt by web stakeholders to offer users an off-by-default way to ask servers not to track them. Importantly, DNT did not offer technical enforcement to prevent tracking of users by websites. Apple supported the DNT project starting in 2011, but since then, the vast majority of websites unfortunately have not changed their behavior in response to the DNT signal for the users who elected to turn it on. Instead, online tracking and tracking techniques have become more pervasive and sophisticated in spite of the DNT project.

The DNT project recently ended without the publication of a standard, in part “because there has not been sufficient deployment of these extensions (as defined) to justify further advancement.” Given the lack of deployment of DNT and Safari’s on by default privacy protections such as ITP, Safari removed support for DNT so that users are not presented with a misleading and ineffective privacy control that, if anything, only offered additional browser fingerprinting entropy.

February 21, 2019 06:30 PM

February 20, 2019

Release Notes for Safari Technology Preview 76

Surfin’ Safari

Safari Technology Preview Release 76 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

This release covers WebKit revisions 240780-241432.

Dark Mode

  • Fixed overflow element scrollbar for dark mode content

WebRTC

  • Fixed RTCRtpSender.getCapabilities(“video”) before initial offer breaking VP8

Web API

  • Simplified and streamlined code that creates an appropriate document based on MIME type
  • Aligned with Fetch on data: URLs

CSS

  • Fixed scroll-snap-align to be honored on a child with non-visible overflow

Rendering

  • Added basic scrollability for async overflow scrolling
  • Fixed first paint getting blocked by async or deferred scripts
  • Fixed jumpy async overflow scrolling if the main thread is busy
  • Fixed rendering for async overflow scroll with border-radius

Payment Request

  • Updated to allow requiring a phonetic name for shipping contacts

Media

  • Updated inline WebVTT styles to start with ::cue
  • Updated to always stop ducking audio after audio capture ends

Web Inspector

  • Added the ability to Import and Export Heap Snapshots
  • Added context menu items to copy a resource’s HTTP request or response data
  • Added keyboard shortcut to the Export button tooltip in the Audit tab
  • Added icons for media event types
  • Fixed selection getting broken after deleting the selected node
  • Fixed the Memory timeline starting from zero when both CPU and Memory timeline are enabled
  • Fixed missing resource data for the document on reload in the Resources tab
  • Fixed timeline Detail views to reset properly when a new time range selection contains nothing in the Timelines tab
  • Fixed timeline graphs that have drawing issues with multiple discontinuities in the Timelines tab
  • Fixed timeline time range selection to show duration alongside start and end in the Timelines tab
  • Fixed clicking on an empty space in the overview to deselect any selected record bar in the Timelines tab
  • Reduced the time spent updating related resources in the Details sidebar of the Resources tab when loading a page with lots of resources
  • Updated the Styles sidebar to automatically close unbalanced quotes and parenthesis when editing values
  • Updated the Styles sidebar with an easier way to select a single line

Browser Changes

  • Added a “Reset All to Defaults” option in the Experimental Features menu of the Develop menu

February 20, 2019 06:00 PM

February 11, 2019

Víctor Jáquez: Review of Igalia’s Multimedia Activities (2018/H2)

Igalia WebKit

This is the first semiyearly report about Igalia’s activities around multimedia, covering the second half of 2018.

Great length of this report was exposed in Phil’s talk surveying mutimedia development in WebKitGTK and WPE:

WebKit Media Source Extensions (MSE)

MSE is a specification that allows JS to generate media streams for playback for Web browsers that support HTML 5 video and audio.

Last semester we upstreamed the support to WebM format in WebKitGTK with the related patches in GStreamer, particularly in qtdemux, matroskademux elements.

WebKit Encrypted Media Extensions (EME)

EME is a specification for enabling playback of encrypted content in Web bowsers that support HTML 5 video.

In a downstream project for WPE WebKit we managed to have almost full test coverage in the YoutubeTV 2018 test suite.

We merged our contributions in upstream, WebKit and GStreamer, most of what is legal to publish, for example, making demuxers aware of encrypted content and make them to send protection events with the initialization data and the encrypted caps, in order to select later the decryption key.

We started to coordinate the upstreaming process of a new implementation of CDM (Content Decryption Module) abstraction and there will be even changes in that abstraction.

Lighting talk about EME implementation in WPE/WebKitGTK in GStreamer Conference 2018.

WebKit WebRTC

WebRTC consists of several interrelated APIs and real time protocols to enable Web applications and sites to captures audio, or A/V streams, and exchange them between browsers without requiring an intermediary.

We added GStreamer interfaces to LibWebRTC, to use it for the network part, while using GStreamer for the media capture and processing. All that was upstreamed in 2018 H2.

Thibault described thoroughly the tasks done for this achievement.

Talk about WebRTC implementation in WPE/WebKitGTK in WebEngines hackfest 2018.

Servo/media

Servo is a browser engine written in Rust designed for high parallelization and high GPU usage.

We added basic support for <video> and <audio> media elements in Servo. Later on, we added the GstreamerGL bindings for Rust in gstreamer-rs to render GL textures from the GStreamer pipeline in Servo.

Lighting talk in the GStreamer Conference 2018.

GstWPE

Taking an idea from the GStreamer Conference, we developed a GStreamer source element that wraps WPE. With this source element, it is possible to blend a web page and video in a single video stream; that is, the output of a Web browser (to say, a rendered web page) is used as a video source of a GStreamer pipeline: GstWPE. The element is already merged in the gst-plugins-bad repository.

Talk about GstWPE in FOSDEM 2019

Demo #1

Demo #2

GStreamer VA-API and gst-MSDK

At last, but not the least, we continued helping with the maintenance of GStreamer-VAAPI and gst-msdk, with code reviewing and on-going migration of the internal library to GObject.

Other activities

The second half of 2018 was also intense in terms of conferences and hackfest for the team:


Thanks to bear with us along all this blog post and to keeping under your radar our work.

By vjaquez at February 11, 2019 12:52 PM

February 06, 2019

Release Notes for Safari Technology Preview 75

Surfin’ Safari

Safari Technology Preview Release 75 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

This release covers WebKit revisions 239991-240780.

WebRTC

  • Enabled Screen Capture (r240635)
  • Updated track sources to be unmuted whenever re-enabled after setDirection changes (r240152)
  • Updated Screen Capture to correctly handle rotation for local video playback (r240000)

Intelligent Tracking Prevention

  • Added the ability to enable and disable Intelligent Tracking Prevention Debug Mode in the Develop menu (r240129)

Web Authentication

  • Changed the nonce in the CTAP kInit command to weak random values (r240081, r240103)

Media

  • Allowed <video> elements not in the DOM to AirPlay (r240537)

Accessibility

  • Added support for returning relative frames for accessibility (r240219)
  • Changed the AXSelected attribute on RadioButton to not be settable (r240205)

Layout

  • Fixed the position of layers nested inside of composited overflow-scroll (r240271)
  • Updated animations to only trigger layer re-compositing when necessary (r240012)

Web API

  • Implemented PerformanceObserver.supportedEntryTypes (r240454)
  • Fixed a bug where sendBeacon to a previously-unvisited HTTPS domain would always fail (r240094)
  • Improved Fetch data: URL handling to align with standards (r240706)

WebDriver

  • Added support for running subtests (r240649)
  • Added support for simulating single touches to Automation.performInteractionSequence (r240554)
  • Updated Automation.computeElementLayout to return Visual Viewport-aware coordinates (r240195)
  • Updated POST commands to require a valid parameters object (r240648)

Web Inspector

  • Added Changes sidebar panel to Elements tab (r240559, 240699)
  • Added support for CSS Color 4 color syntaxes (r240550)
  • Added supports key to test/group for compatibility in the Audit tab (r240469)
  • Added a mechanism for Web Inspector to edit page settings on a remote target (r240540)
  • Added a mechanism for Web Inspector to edit page WebRTC settings on a remote target (r240644, r240664)
  • Added a mechanism for Web Inspector to edit the user agent of a remote target (r240549)
  • Added grouped CSS rules by resource to the Changes sidebar panel (r240741)
  • Added a CPU usage timeline in the Timelines tab (r240457)
  • Added new mechanisms for getting related Accessibility nodes and properties for a given node for audits in the Audit tab (r240277, r240309)
  • Changed the style of the device settings override popover content (r240737)
  • Changed the Elements tab to toggle visibility for all selected nodes (r240639)
  • Enabled computed style cascades by default in the Styles sidebar (r240691)
  • Excluded Debugger Threads from CPU usage values in Web Inspector (r240522)
  • FIxed a bug that prevented audits from being imported in the Audit tab (r240538)
  • Improved Dark Mode appearance within Memory timeline (r240518)
  • Improved invalid Audit and Recording JSON error messages (r240471)
  • Updated the Memory Timeline View to be responsive when resizing (r240763)
  • Updated the Audit tab to ensure that each test run creates its own injected objects (r240188)
  • Updated the Network waterfall column to redraw when adding or removing new columns (r240347)
  • Updated Web Inspector to show the uncaught exception view for unhandled promise rejections (r240499)

CSS

  • Updated CSS Grid to properly handle static positions of absolute-positioned inside grid items (r240333)
  • Implemented flow-relative inset properties (r240334)
  • Implemented flow-relative margin, padding and border shorthands (r240251)
  • Changed to reject unitless length quirk in the inset shorthand (r240588)

Browser Changes

  • Removed support for the expired Do Not Track standard to prevent potential use as a fingerprinting variable.

Bug Fixes

  • Fixed snapshots removed too late after swiping back on Twitter (r240542)
  • Fixed history navigations to twitter.com that were leading to a 403 HTTP error (r240750)
  • Fixed a loading hang that could occur on history navigation (r240725)
  • Updated the MIME-type parser (r240591)

February 06, 2019 06:00 PM

February 04, 2019

IntersectionObserver in WebKit

Surfin’ Safari

Web authors often need to know when a particular element on a page becomes visible. The element could be an ad or a video whose viewability we want to compute. Or we might want to defer loading an image until it is visible. A common way to solve this problem is using polling, periodically computing the position of an element with respect to the viewport. However, polling is inefficient, wastes battery life, and doesn’t work for cross-origin content. The IntersectionObserver API offers a better solution to this problem, and is now available in Safari Technology Preview, macOS 10.14.4 beta, and iOS 12.2 beta.

API Overview

Each IntersectionObserver has a set of target elements and observes the intersection of these targets with a particular root element or with the viewport. When the intersection of a target and root crosses a specified threshold, the observer invokes a callback. Here’s a simple example to get started:

var intersectionCount = 0;
function callback(entries) {
    entries.forEach((entry) => {
        if (entry.isIntersecting)
            intersectionCount++;
    });
}
var observer = new IntersectionObserver(callback);
observer.observe(target);

In the example above, we haven’t specified a root element so we’re observing intersections with the viewport. We pass the IntersectionObserver constructor the callback function we want it to use. We’ll get a callback as soon as any portion of the target intersects the viewport, and we’ll get another callback as soon as the target has no intersection with the viewport.

What if we want to know when all or most of an element is visible, not just any portion of it? We can specify an intersection threshold when constructing the observer:

observer = new IntersectionObserver(callback, { threshold: 0.95 } );

Now, we’ll get a callback once 95% of a target intersects the viewport and another callback once the intersection ratio drops below 95%.

Extending this idea, we can provide a list of thresholds that we’re interested in:

observer = new IntersectionObserver(callback, {
    threshold: [ 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1 ] });

Here’s a live demo that uses a list of thresholds. Scroll the target into view, and then slowly scroll it out of view and back into view, and notice observations getting logged as the intersection ratio crosses a threshold.

The same idea works for a target that’s inside an iframe (demo). In this case, there are two ways to scroll the target out of view — scrolling the window so that the iframe is out of view, and scrolling the iframe itself — and both will trigger IntersectionObserver callbacks.

Suppose we want advance notice that an element is about to become visible. The IntersectionObserver constructor lets us specify a margin by which to expand the root when computing intersections.

observer = new IntersectionObserver(callback, { rootMargin: '100px 0px' } );

When computing intersections, the viewport will be expanded by 100 pixels at the top and bottom, and will not be expanded on the left or right.

Using negative root margin values, we can shrink the root when computing intersections. For example, in this demo, we shrink the viewport by 50px on all four sides:

observer = new IntersectionObserver(callback, {
    threshold: [ 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1 ],
    rootMargin: '-50px'
});

Finally, we can specify a particular element to intersect with rather than the viewport:

observer = new IntersectionObserver(callback, { root: someElement } );

Let’s now take a look at the entries that get passed to the callback each time it is invoked. Suppose we want to to compute how long an element is at least 80% visible:

observer = new IntersectionObserver((entries) => {
    entries.forEach((entry) => {
        if (entry.isIntersecting)
            entry.target.startTime = entry.time;
        else if (entry.target.startTime)
            entry.target.duration = entry.time - entry.target.startTime;
    });
}, { threshold: 0.8 });

Each intersection observer entry is a dictionary containing details about the intersection between a particular target and the root. This includes a timestamp, the intersection rectangle, and an intersection ratio that represents the fraction of the target that’s contained within the intersection rectangle.

More examples

Lazy image load

Suppose we have a long page with many images. Rather than loading all these images at once, including those that are off-screen, we can start with low-resolution placeholder images, and defer loading high-resolution versions until an image starts to intersect (or, using rootMargin, is about to intersect) with the viewport. Here’s a demo illustrating this approach. To make it easier to see the high-resolution images getting loaded, the demo doesn’t use rootMargin, and waits 500ms after an image starts to intersect the viewport before switching to a high-res version.

Triggering an animation when an element becomes visible

To ensure that an animation only starts once the animated content is visible, we can trigger the animation inside an IntersectionObserver callback:

function intersectionCallback(entries) {
    for (let entry of entries) {
        if (entry.isIntersecting) {
            entry.target.style.transform = 'translateX(20px)';
            entry.target.style.transition = 'transform 2s';
        } else {
            entry.target.style.transform = 'none';
            entry.target.style.transition-property = 'none';
        }
    }
}

Every time a target element starts intersecting the viewport, we’ll start an animation to slide it to the left, and we’ll put the element back to its original position once it no longer intersects. This demo has a few examples of animations that get triggered by intersection observations.

Feedback

We’re excited to have IntersectionObserver available in Safari Technology Preview, macOS 10.14.4 beta, and iOS 12.2 beta! Please try it out and file bugs for any issues you run into.

February 04, 2019 07:45 PM

January 25, 2019

Removing Legacy SPDY Protocol Support

Surfin’ Safari

This is a guest post from Apple’s CFNetwork team about changes to the URLSession networking framework used by Safari and WebKit on Apple platforms. These changes may affect compatibility with your websites.

SPDY is a network protocol designed to improve the performance of web browsing and other HTTP-based communication by addressing many limitations of the HTTP/1.1 protocol. Safari gained support for the SPDY protocol in 2014 with Apple’s OS X Yosemite and iOS 8 releases. This was achieved by adding SPDY support to the Foundation networking URLSession API.

Many components of the SPDY protocol proved to be successful improvements over HTTP/1.1, so the Internet Engineering Task Force (IETF) drafted a new protocol specification, HTTP/2, based on many of the same ideas, along with other improvements. Safari and URLSession added support for the HTTP/2 protocol in 2015 with Apple’s OS X El Capitan and iOS 9 releases.

Unlike HTTP/2 and HTTP/1.1, the SPDY protocol is not specified by an Internet standard from the IETF. Rather, it is an experimental protocol that served as a stepping stone to HTTP/2. In fact, many browsers and servers that supported SPDY quickly migrated to HTTP/2, and furthermore, have since dropped support for SPDY entirely (e.g., Chrome, Firefox, and Cloudflare).

Support for the legacy SPDY protocol is deprecated in the recently released Apple Developer Betas of macOS Mojave 10.14.4 and iOS 12.2. In an upcoming release, legacy SPDY protocol support will be completely removed from Safari along with Apple’s URLSession (and deprecated NSURLConnection) API. At the same time, support for the legacy NPN TLS Extension will be removed. NPN, which was added for SPDY, also was never standardized and has been replaced by ALPN.

Removing legacy SPDY protocol support will have little impact to many users of Safari and other applications using URLSession. Servers that did add support for SPDY continued to support HTTP/1.1, so connections to these servers will continue to work by gracefully switching to using HTTP/1.1 (if upgrades to add HTTP/2 support have not occurred), but may suffer from a loss of performance in doing so. To help identify any unexpected cases where SPDY is still being used, a new deprecation log message is generated by URLSession once per process launch if SPDY is negotiated with a server:

WARNING: URLSession (or NSURLConnection) is using SPDY protocol, which is deprecated and will be removed in a future release

This log message is available starting in the Apple Developer Betas of macOS Mojave 10.14.4 and iOS 12.2 and will be found in the system log.

For many, removal of legacy SPDY support requires no action, as the URLSession API will automatically switch to the best protocol available. However, if you own or operate a web server that still supports the SPDY and/or NPN experimental technologies, we encourage upgrading to HTTP/2 now so that your clients can take advantage of its many benefits. If you have any systems which prevent transitioning from SPDY to HTTP/2, please let us know by contacting our Web Technologies Evangelist or by filing a bug report with details.

January 25, 2019 05:30 PM

January 23, 2019

Release Notes for Safari Technology Preview 74

Surfin’ Safari

Safari Technology Preview Release 74 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

This release covers WebKit revisions 239566-239991.

Fetch API

  • Implemented abortable fetch (r239644)

Web Animations

  • Changed to compute animation effect timing properties in batch (r239723)

Web Authentication

  • Changed the nonce in the CTAP kInit command to weak random values (r239852)
  • Imported U2F command and response converters from Chromium (r239665)
  • Added support for U2F HID Authenticators on macOS (r239752)

WebRTC

  • Enabled MDNS ICE candidate support by default (r239936)

Media

  • Changed <video> elements to not enter “paused” state when playing to the end over AirPlay (r239857)
  • Defined page media state flags for display capture (r239840)
  • Changed to deactivate audio session whenever possible (r239694)

CSS

Web API

  • Added an override of the session configuration for cookieAcceptPolicy (r239832)
  • Enabled visual viewport API by default (r239897)
  • Changed to no longer trigger a main frame load when doing a history navigation if the source and destination history items are clones
  • Fixed Bulgarian TLDs to not punycode-encode URLs with Bulgarian Cyrillic characters (r239967)
  • Fixed an issue where view is unresponsive for 5 seconds after swiping back on sites like quoteunquoteapps.com (r239720)
  • Fixed the parsed protocol of JavaScript URLs with embedded newlines and carriage returns to match the parsed protocol in Chrome and Firefox (r239642)
  • Fixed unwanted page navigation after showing and dismissing contextual menu with control-click (r239686)

Service Workers

  • Fixed Service Worker fetch handler that resulted in a bad referrer (r239749)
  • Updated Service Worker fetch to obey its referrer policy (r239659)

JavaScript

  • Optimized Object.prototype.toString (r239612)

Accessibility

  • Updated the accessibility string for the <hr> HTML tag to use “separator” instead of “rule” (r239613)

Web Inspector

  • Added a Setting toggle for Source Maps (r239635)
  • Added the ability to allow audits to be enabled or disabled in the Audits tab (r239858)
  • Added the ability for audits to determine whether a give node has event listeners when running an audit (r239987)
  • Added the ability for audits to query for all nodes with a given computed Accessibility role when running an audit (r239986)
  • Changed to automatically disable breakpoints when running an audit in the Audit tab (r239646)
  • Changed the Network tab to show secure connection details per-request (r239698)
  • Changed to use the save sheet instead of a dialog when clicking the “Export” button in the Network tab (r239647)
  • Extended XHR breakpoints to work with Fetch (r239703)
  • Fixed the color picker to accurately show color when the saturation value is not 100% (r239597)
  • Fixed support for “white” as a recognized color keyword (r239690)
  • Fixed text field and completion popover fonts to match in the Events breakpoint popover (r239953)
  • Fixed showing completions for events that start with “DOM” when typing uppercase “DOM” in Event breakpoints (r239955)
  • Fixed vertical alignment of group titles in the Settings tab (r239932)
  • Fixed read-only properties from being able to be deleted in the Styles sidebar (r239585)
  • Fixed pressing down key on an empty value field discarding the completion popover in the Styles sidebar (r239935)
  • Fixed selection becoming lost in the Styles sidebar when Web Inspector is blurred (r239588)
  • Fixed undo to prevent it reverting all changes at once in the Styles sidebar (r239762)
  • Implemented queryObjects Command Line API (r239583)
  • Included globalThis in default JavaScript completions (r239630)
  • Prevented starting a selection when clicking on a property that is being discarded in the Styles sidebar (r239766)

WebDriver

  • Changed element click and other commands to appropriately handle user prompts (r239610)
  • Updated the Create Window implementation to support the newest specification (r239988)

Storage

  • Fixed IndexedDB storage of Crypto keys in private browsing mode (r239746)
  • Fixed deleting IndexedDB databases to ensure open databases are not missed (r239960)
  • Made “Disable Web SQL” on by default (r239885)

Security

  • Fixed CSP violation reports to bypass CSP checks (r239634)

January 23, 2019 09:45 PM

January 09, 2019

Release Notes for Safari Technology Preview 73

Surfin’ Safari

Safari Technology Preview Release 73 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

This release covers WebKit revisions 239091-239566.

Web API

  • Fixed navigator.userAgent in service workers to correctly reflect the customUserAgent set by the client (r239534)
  • Fixed offsetLeft and offsetParent to adjust across shadow boundaries (r239313)
  • Fixed an incorrect value for key property in keydown and keyup events generated holding the Control (⌃) key (r239361)
  • Removed support for <meta http-equiv=set-cookie> (r239342)

JavaScript

  • Added support for globalThis (r239464)
  • Added support for BigInt in JSON.stringify (r239544)
  • Implemented support for BigInt ValueDiv into DFG (r239158)
  • Implemented DFG and FTL support for typeof in BigInt (r239141)
  • Implemented the “well-formed JSON.stringify” proposal (r239537)
  • Optimized Object.keys by caching key results (r239324)

CSS

  • Fixed percentages in relative offsets for CSS grid items (r239502)
  • Fixed thick overlines and line-throughs growing in the wrong direction (r239357)
  • Fixed CSS variables to work for colors in the border property (r239516)
  • Prevented attempting to animate invalid CSS properties (r239174)
  • Updated CSS Properties and Values API to use new cycle fallback behavior (r239365)
  • Updated repeated background images with zero size to display the background color (r239504)

Media

  • Changed EME MediaKeySystemConfiguration distinctiveIdentifier and persistentState to default to optional (r239373)
  • Changed inline WebVTT styles should override styles from Captions settings in System Preferences (r239181)
  • Updated PiP video to switch tabs before returning video to inline (r239447)
  • Changed playing a media element synchronously after setting srcObject to succeed without user gesture (r239189)
  • Fixed a flicker when exiting element-fullscreen (r239475)

WebRTC

  • Ensured the first video frame of a MediaStream is rendered (r239319)
  • Fixed the MediaStream calculation of width or height when constraints contain only one of them (r239163)
  • Ensured that an ended event fires on MediaStreamTrack when a USB camera is unplugged (r239531)
  • Made RTCRtpSender.setParameters activate specific encodings (r239252)
  • Made sure getSenders() and getReceivers() do not return closed transceiver senders or receivers (r239238)
  • Implemented non-timeslice mode encoding for MediaRecorder (r239145)
  • Made sure RTCRtpTransceiver.stopped is true when a remote description with the corresponding m section rejected is applied (r239192)

Web Animations

  • Changed Web Animations to ensure an animation’s finished state only occurs once when updating animations (r239269)

Web Authentication

  • Changed userPresence to always be true (r239523)

WebGPU

  • Implemented WebGPUBindGroupLayoutDescriptor and its supporting dictionaries (r239288)
  • Added basic implementation of vertex buffers with Metal Shading Language in WebGPU (r239139)

Web Inspector

  • Enabled the display of m3u8 content as text in the Network tab (r239343)
  • Ensured that localized strings in default audits respond to locale changes in the Audit tab (r239449)
  • Fixed clipping of “Clear Filters” button in short windows in the Audit tab (r239394)
  • Fixed descriptions of default tests in the Audit tab (r239433)
  • Provided localization support for the display of “percentage passed” results in the Audit tab (r239404)
  • Fixed autoformat for minified JavaScript (r239467)
  • Fixed alignment of path view in the Canvas tab (r239312)
  • Fixed initial focus state of the recording auto-capture input in the Canvas tab (r239420)
  • Fixed selected text color when Inspector window is inactive in dark mode (r239386)
  • Fixed colors of ThreadTreeElement status icon when hovered in dark mode (r239430)
  • Fixed background color of Type profiler popovers in dark mode (r239484)
  • Improved readability for tables containing object previews in dark mode (r239443)
  • Fixed which element gets selected after pressing an arrow key after undoing deletion of a DOM node in the Elements tab (r239405)
  • Fixed Charles Proxy errors when opening some HAR files exported from Safari (r239478)
  • Fixed disappearing reload button after changing more than one setting (r239179)
  • Fixed the spacing of “E” icon on the debugger dashboard (r239432)
  • Fixed the rendering of column sort controls for dark mode in the Layers tab (r239385)
  • Increased the default column width for some localizations in the Layers tab (r239345)
  • Fixed the alignment of experimental setting editors for some localizations in the Settings tab (r239521)
  • Fixed a problem in the Styles sidebar where focus was lost when the inspector is blurred (r239527)
  • Fixed shift-clicking a color-swatch to change formats in the Styles sidebar (r239413)
  • Fixed data corruption when toggling selected properties in the Styles sidebar (r239251)
  • Improved the usability of the Computed styles sidebar when the panel is narrow (r239399)
  • Made the “esc” key select the entire property when editing the name or value in the Styles sidebar (r239481)
  • Prevented table selection from becoming corrupted when deleting selected cookies (r239175)
  • Fixed labelling of Intersection Observer callbacks in the Timelines tab (r239397)

January 09, 2019 06:00 PM

January 08, 2019

Carlos García Campos: Epiphany automation mode

Igalia WebKit

Last week I finally found some time to add the automation mode to Epiphany, that allows to run automated tests using WebDriver. It’s important to note that the automation mode is not expected to be used by users or applications to control the browser remotely, but only by WebDriver automated tests. For that reason, the automation mode is incompatible with a primary user profile. There are a few other things affected by the auotmation mode:

  • There’s no persistency. A private profile is created in tmp and only ephemeral web contexts are used.
  • URL entry is not editable, since users are not expected to interact with the browser.
  • An info bar is shown to notify the user that the browser is being controlled by automation.
  • The window decoration is orange to make it even clearer that the browser is running in automation mode.

So, how can I write tests to be run in Epiphany? First, you need to install a recently enough selenium. For now, only the python API is supported. Selenium doesn’t have an Epiphany driver, but the WebKitGTK driver can be used with any WebKitGTK+ based browser, by providing the browser information as part of session capabilities.

from selenium import webdriver

options = webdriver.WebKitGTKOptions()
options.binary_location = 'epiphany'
options.add_argument('--automation-mode')
options.set_capability('browserName', 'Epiphany')
options.set_capability('version', '3.31.4')

ephy = webdriver.WebKitGTK(options=options, desired_capabilities={})
ephy.get('http://www.webkitgtk.org')
ephy.quit()

This is a very simple example that just opens Epiphany in automation mode, loads http://www.webkitgtk.org and closes Epiphany. A few comments about the example:

  • Version 3.31.4 will be the first one including the automation mode.
  • The parameter desired_capabilities shouldn’t be needed, but there’s a bug in selenium that has been fixed very recently.
  • WebKitGTKOptions.set_capability was added in selenium 3.14, if you have an older version you can use the following snippet instead
from selenium import webdriver

options = webdriver.WebKitGTKOptions()
options.binary_location = 'epiphany'
options.add_argument('--automation-mode')
capabilities = options.to_capabilities()
capabilities['browserName'] = 'Epiphany'
capabilities['version'] = '3.31.4'

ephy = webdriver.WebKitGTK(desired_capabilities=capabilities)
ephy.get('http://www.webkitgtk.org')
ephy.quit()

To simplify the driver instantation you can create your own Epiphany driver derived from the WebKitGTK one:

from selenium import webdriver

class Epiphany(webdriver.WebKitGTK):
    def __init__(self):
        options = webdriver.WebKitGTKOptions()
        options.binary_location = 'epiphany'
        options.add_argument('--automation-mode')
        options.set_capability('browserName', 'Epiphany')
        options.set_capability('version', '3.31.4')

        webdriver.WebKitGTK.__init__(self, options=options, desired_capabilities={})

ephy = Epiphany()
ephy.get('http://www.webkitgtk.org')
ephy.quit()

The same for selenium < 3.14

from selenium import webdriver

class Epiphany(webdriver.WebKitGTK):
    def __init__(self):
        options = webdriver.WebKitGTKOptions()
        options.binary_location = 'epiphany'
        options.add_argument('--automation-mode')
        capabilities = options.to_capabilities()
        capabilities['browserName'] = 'Epiphany'
        capabilities['version'] = '3.31.4'

        webdriver.WebKitGTK.__init__(self, desired_capabilities=capabilities)

ephy = Epiphany()
ephy.get('http://www.webkitgtk.org')
ephy.quit()

By carlos garcia campos at January 08, 2019 05:22 PM

December 19, 2018

Release Notes for Safari Technology Preview 72

Surfin’ Safari

Safari Technology Preview Release 72 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

This release covers WebKit revisions 238607-239091.

Web Authentication

  • Added support for CTAP HID authenticators on macOS (r238729)

CSS Painting API

  • Added CSS Painting API support for multiple worklets. (r239067)
  • Changed the CSS Painting API to pass this to the paint callback, and repaint when properties change. (r238686)
  • Updated the CSS Painting API to allow image values in inputProperties (r238872)
  • Updated the CSS Painting API to scale the display list when drawing (r238839)

Web API

  • Changed CSS subresource loads to not be observable from resource timing if the stylesheet is opaque (r238698)
  • Changed overflow scrolling layers to be self-painting (r238725)
  • Changed to prevent computing animated values when there is no relevant animation (r239090)
  • Fixed XMLHttpRequest to prevent removing spaces from content-types before processing (r239040)
  • Fixed HEAD requests becoming changed to GET requests after 301, 302, and 303 redirects (r238891)
  • Fixed document.lastModified to work for non-rendered documents (r238628)
  • Fixed IndexedDB with binary data (Uint8Array) and autoIncrement key in store (r238677)
  • Modernized version checks for same site cookie support (r238614)
  • Restricted the DeviceMotion and DeviceOrientation APIs to secure contexts (r239087)

WebRTC

  • Added support for firing the devicechange event when more capture device information is revealed when getUserMedia is granted by user (r238796)
  • Fixed the devicechange event to not fire in frames that can’t access capture devices (r239006)
  • Changed scaled video frames to be resized in letterbox mode (r239016)
  • Fixed DataChannels created asynchronously to open properly (r239064)
  • Fixed calling getUserMedia in a link that was opened with target="_blank" to work properly the second time (r238873)
  • Fixed the case of a sender created through addTransceiver whose track is populated using addTrack (r238845)

Web Inspector

  • Enabled selection of multiple properties by default in the Styles sidebar (r238649)
  • Fixed $0 being shown for all selected elements in the Elements tab (r238859)
  • Fixed jumping from Computed to Styles to correctly select the property in the Elements tab (r238762)
  • Fixed the overlay bounds rulers to match the element when the page is scrolled (r239079)
  • Fixed disclosure triangles for RTL to be flipped and aligned right (r238711)
  • Fixed the selection of properties of read-only rules in the Styles sidebar (r238813)
  • Updated the selected DOM node scope highlight to improve contrast in the Elements tab (r238671)
  • Updated ⌘A to select all visible nodes in the Elements tab (r238858)
  • Fixed “Add Default Audits” shown when there are no filter results in the Audit tab (r238831)
  • Fixed empty results folder shown after clearing filters in the Audit tab (r238648)
  • Fixed navigating while tests are running to properly stop and clear all results in the Audit tab (r238841)
  • Fixed the sidebar briefly showing “No Filter Results” before populating default audits in the Audit tab (r238842)
  • Updated to save the expanded state of test groups in the Audit tab (r238843)
  • Updated test tree elements to start out collapsed in the Audit tab (r238822)
  • Updated tests to support async operations in the Audit tab (r238850)

Web Share API

  • Fixed the share overlay to properly stick to the Safari window (r238941)

Media

  • Fixed the lifetime of the HTMLMediaElement to be properly handled in asynchronous actions (r238788)

JavaScript

  • Enabled .mjs content when loaded from file:// (r238673)
  • Added BigInt support for logic operations (r238861)
  • Implemented BigInt support for << and >> (r238790)

Service Workers

  • Fixed updating a service worker during a navigation load sometimes causing the load to fail (r238683)

December 19, 2018 06:00 PM

December 08, 2018

Philippe Normand: Web overlay in GStreamer with WPEWebKit

Igalia WebKit

After a year or two of hiatus I attended the GStreamer conference which happened in beautiful Edinburgh. It was great to meet the friends from the community again and learn about what’s going on in the multimedia world. The quality of the talks was great, the videos are published …

By Philippe Normand at December 08, 2018 02:09 PM

December 05, 2018

Release Notes for Safari Technology Preview 71

Surfin’ Safari

Safari Technology Preview Release 71 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

This release covers WebKit revisions 237919-238607.

Dark Mode

  • Enabled Dark Mode CSS Support by default in the Experimental Features menu (r238213)
  • Added experimental support for a supported-color-schemes CSS property (r238001)
  • Updated prefers-color-scheme media query matching to better match the draft spec (r238200)
  • Changed the default document background and text colors in dark mode and when dark is listed as a supported-color-schemes on the document or body element (r238212)
  • Added support for forcing color scheme (dark or light mode) in Web Inspector’s Elements tab (r238570)

Web Animations

  • Enabled Web Animations by default in the Experimental Features menu (r238095)

Intersection Observer

  • Enabled Intersection Observer by default in the Experimental Features menu (r238129)

Web Authentication

  • Added Web Authentication as an experimental feature with support for USB-based CTAP2 devices (r238466)

Web API

  • Fixed click event and touch event listeners on the body element (r237978)
  • Fixed input element gaining focus when a selectstart event listener on the document prevents the default action (r238409)
  • Fixed phantom focus events and blur events firing when clicking between text input fields with addEventListener (r238440)
  • Fixed Content-Type parameter values to allow empty quoted strings (r238124)
  • Changed CSS Painting API to pass size, arguments, and input properties to the paint callback (r237981)

SVG

  • Fixed incorrectly sized monospace text when an SVG is included as an <img> (r238447)
  • Fixed SVG <use> element inside a shadow tree to be able to reference an element in the same tree (r238524)
  • Fixed updating href on textPath to properly update its rendering (r238464)

CSS

  • Unprefixed text-decoration CSS3 properties (r238002)
  • Fixed normal-flow-only flex items to correctly respect z-index (r238058)
  • Fixed overlays with -webkit-overflow-scrolling:touch to become scrollable after added text makes it taller (r238266)

Rendering

  • Made compositing updates incremental (r238090)
  • Changed to update the transform of a composited layer when the layer also needs repainted (r238170)
  • Fixed the caret disappearing at the end of a password field when the caps lock indicator is shown (r238522)
  • Fixed invisible dotted underlines that skip descenders (r237948)
  • Added support for dynamic pseudo-classes on elements with display: contents (r238097)
  • Fixed positioned text underlines that can look like a strike-through (r237955)

Intelligent Tracking Prevention

  • Enabled a cap on partitioned cache max age (r237949)

WebRTC

  • Required <iframe allow="display"> for an iframe to use getDisplayMedia (r238091)
  • Added support for RTCRtpCodecParameters.sdpFmtpLine (r238172)
  • Added support for transportStats and peerConnectionStats (r238171)
  • Enabled setting RTCRtpTransceiver.direction (r237974)
  • Fixed RTCPeerConnection.getTransceivers to always expose all transceivers (r238150)

Web Inspector

  • Added support for the tab bar to automatically add a new Audit tab for running page audits when enabled in Web Inspector Experimental settings (r238500)
  • Added default tests to the Audit tab (r238334)
  • Added support for drag-and-drop for importing Audits in the Audits tab and Recordings in the Canvas tab (r238198)
  • Changed to not start the Canvas tab with auto-capture enabled (r238169)
  • Added support for the copy keyboard shortcut and context menu to the Cookies table (r238569)
  • Fixed the Debugger Popover to work when paused in a Worker (r238003)
  • Fixed Elements tab to allow selecting and deleting multiple DOM nodes (r238602)
  • Fixed missing minimum sizes in Network graphs that could cause no graph to be shown (r238168)
  • Added secure certificate details per-request in the Network tab (r238122)
  • Added a button to show the system certificate dialog in the Network tab (r238350)
  • Fixed a potential issue for watch expressions that never show results (r238005)
  • Added support for ⌘A (Command-A) to select all properties in the Styles sidebar (r238135)
  • Added support for ⇧Click (Shift-click) on a property to extend the selection in the Styles sidebar (r238201)
  • Fixed inline swatches when multiple properties selection is enabled in the Styles sidebar (r238120)
  • Changed Pressing ⇥ (Tab) or Enter to start editing the focused property in the Styles sidebar (r238096)
  • Added support to allow expanding properties to show list of overridden values in the Computed sidebar (r238589)
  • Fixed the experimental Computed sidebar in Dark Mode (r238594)
  • Updated tables to recalculate scrollable height when resized (r238203)
  • Updated tables to support Select All ⌘A (Command-A) (r238140)
  • Updated tables to support extending the row selection with the ⇧ (Shift) key (r238121)
  • Updated tables with no selection to select the first or last row when pressing the down or up arrow key (r238139)
  • Added a Media timeline in the Timelines tab (r238484)
  • Updated TreeOutline to re-use multiple-selection logic from Table (r238599)
  • Updated TreeOutline to prevent selecting the first child of an expanded element when pressing the right arrow key (r238305)
  • Added a button to clear filters with “No Filter Results” in the Navigation sidebar (r238502)
  • Made the “Reload Web Inspector” button more reliable when toggling experimental features (r238378)

CSS Grid

  • Fixed absolute positioned child getting incorrectly sized when using auto-fit, generating spurious collapsed tracks (r238491)
  • Fixed align-self:center and position:sticky to work together (r238551)
  • Fixed grid container sizing under min-content height (r238488)
  • Implemented Baseline Alignment for grid items (r238457)

Media

  • Fixed frame re-ordering preventing iframes from getting enqueued (r238086)
  • Enabled external playback for video in element fullscreen (r238327)
  • Fixed SourceBuffer to prevent throwing an error when appending a second init segment after changeType() (r238054)

Payment Request

  • Fixed canMakePayment() to not consider serialized payment method data (r238042)

JavaScript

  • Added BigInt support into ValueAdd (r237972)

December 05, 2018 06:00 PM

November 14, 2018

Release Notes for Safari Technology Preview 70

Surfin’ Safari

Safari Technology Preview Release 70 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

This release covers WebKit revisions 237630-237919.

Web Animations

  • Implemented getTiming() and updateTiming() (r237853)
  • Implemented Animation.updatePlaybackRate() (r237854)
  • Changed to not reset pending tasks when setting a null effect (r237856)
  • Made document.getAnimations() return declarative animations in the correct order (r237726)
  • Updated the API to allow the auto composite value (r237855)
  • Updated the Web Animations API to remove all of the ReadOnly interfaces (r237852)

Intersection Observer

  • Added support to allow cross-document intersection observing (r237737)
  • Fixed isIntersecting computation when 0 is not a threshold (r237798)
  • Updated to account for CSS zoom when computing client rects (r237862)

CSS

  • Added CSS Custom Properties API Support for syntax="*" and <length> with proper cycle handling (r237697)
  • Fixed <picture> container to update when the prefers-color-scheme media query changes (r237878)
  • Fixed CSS grid elements with justify-content:space-around that have extra whitespace (r237884)
  • Implemented text-underline-offset and text-decoration-thickness (r237903, r237835)

WebRTC

  • Added support for sender and receiver getCapabilities (r237911)
  • Added onremovetrack support for RTCPeerConnection removed tracks (r237908)
  • Changed MediaRecorder to fire dataavailable event when all tracks are ended and stop() is called (r237642, r237650, r237675)
  • Fixed calling sender.replaceTrack() twice producing a new transceiver and its corresponding m= section (r237910)
  • Fixed sender.replaceTrack() fails with InvalidStateError if the transceiver.direction is “inactive” (r237916)
  • Fixed RTCTrackEvent.streams should be SameObject (r237688)
  • Updated to prevent revealing device IDs until the user has granted permission to capture (r237643)
  • Updated enumerateDevices to not expose devices that are not available to getUserMedia (r237770)
  • Updated to prevent prompting the user again after denying a getDisplayMedia request (r237879)

Rendering

  • Fixed requestAnimationFrame causing bad location of position:fixed inside overflow:auto and iframe (r237754)
  • Fixed overlays with -webkit-overflow-scrolling:touch to become scrollable after added text makes it taller (r237849)

Web Inspector

  • Added low-power video playback enter and exit events in Timelines and Network waterfalls (r237669)
  • Added support to show save and restore stack for recorded 2D Canvases (r237808)
  • Added missing context menu items for links in the Styles sidebar (r237660)
  • Added support to toggle selected properties by pressing Space or ⌘/ (Command-/) in the Styles sidebar (r237692)
  • Added support for multiple selection and multiple deletion of cookie records (r237746)
  • Created a setting for auto-recording newly created contexts in the Canvas tab (r237670)
  • Implemented copying and deletion of multiple properties in the Styles sidebar (r237659)
  • Updated to capture changes to <canvas> (i.e., canvas size) that would affect the recorded context in the Canvas tab (r237777)

Storage

  • Added a storage limit for IndexedDB (r237700)
  • Changed IndexedDB to allow storing RTCCertificate (r237779)

Media

  • Fixed MediaSource.isTypeSupported('video/mp4; codecs="hvc1.1.6.L60.B0') getting improperly rejected (r237655)

Payment Request

  • Changed the errorFields of PaymentResponse.retry() to be optional (r237776)

Security UI

  • Added a warning in the Smart Search field when loading non-secure pages

November 14, 2018 06:00 PM

November 12, 2018

Web High Level Shading Language

Surfin’ Safari

This article is introducing a new graphics shading language for the Web named Web High Level Shading Language (WHLSL, pronounced “whistle”). The language is insprired by HLSL, the dominant shading language for graphics app developers. It extends HLSL for the Web platform to be safe and secure. It’s easy to read and write, and is well-specified using formal techniques.

Background

Over the past few decades, 3D graphics have changed significantly, and the APIs programmers use to write 3D applications have also changed accordingly. Five years ago, state-of-the-art graphics applications would use OpenGL to perform their rendering. However, the past few years have seen a shift in the 3D graphics industry toward newer, lower-level graphics frameworks that better match the behavior of real hardware. In 2014, Apple created the Metal framework, which lets iOS and macOS apps use the full power of the GPU. In 2015, Microsoft created Direct3D 12, a major update to Direct3D which allows for console-level efficiency for rendering and compute. In 2016, the Khronos Group published the Vulkan API, which is primarily used on Android, that offers similar advantages.

Just like how WebGL brought OpenGL to the Web, the Web community is pursuing bringing this type of new, low-level 3D graphics API to the platform. Last year, Apple established the WebGPU Community Group inside the W3C to standardize a new 3D graphics API which provides the benefits of these native APIs, but is also suitable for the Web environment. This new Web API is implementable on top of Metal, Direct3D, and Vulkan. All of the major browser vendors are participating and contributing to the standardization effort.

Each of these modern 3D graphics APIs uses shaders, and WebGPU is no different. Shaders are programs that take advantage of the specialized architecture of GPUs. In particular, GPUs are better than CPUs at heavy parallel numerical processing. To take advantage of both architectures, modern 3D apps use a hybrid design, using both the CPU and the GPU for different tasks. By leveraging the best traits of each, modern graphics APIs provide a powerful framework for developers to create complex, rich, and fast 3D apps. Apps designed for Metal use the Metal Shading Language, apps designed for Direct3D 12 use HLSL, and apps designed for Vulkan use SPIR-V or GLSL.

Language Requirements

Just like its native counterparts, WebGPU needs a shader language. This language needs to meet several requirements that make it well-tailored for the Web platform.

The language needs to be safe. No matter what the application does, the shader must only be able to read or write data from the Web page’s domain. Without this guarantee, a malicious website could run a shader that reads pixels out of other parts of your screen, even from native apps.

The language needs to be well-specified. The language specification has to be explicit about whether every single possible string of characters is a valid program or not. As with all other Web formats, a shading language for the Web must be precisely specified to guarantee interoperability between browsers.

The language also needs to be well-specified so that it can be used as a compilation target. Many rendering teams write shaders in their own custom in-house language, and then cross-compile to whichever language is necessary. For this reason, the language should have a reasonably small set of unambiguous grammar and type checking rules that compiler writers can reference when emitting this language.

This language needs to be translatable to Metal Shading Language, HLSL (or DXIL), and SPIR-V. This is because WebGPU is designed to work on top of Metal, Direct3D 12, and Vulkan, so the shaders need to be able to be represented in a form that each of those APIs can accept.

The language needs to be performant. The entire reason developers want to use the GPU in the first place is for performance. The compiler itself needs to run quickly, and programs produced by the compiler need to run efficiently on real GPUs.

This language needs to evolve with the WebGPU API. WebGPU features such as the binding model and tessellation model interact deeply with the shading language. Though it is feasible to have the language developed independently of the API, having the WebGPU API and shading language in the same forum ensures the goals are shared, and makes development more streamlined.

The language needs to be easy for a developer to read and write. There are two pieces to this: firstly, the language should be familiar to both GPU programmers and CPU programmers. GPU programmers are important clients because they have experience writing shaders. CPU programmers are important because GPUs are increasingly being used for purposes beyond rendering, including machine learning, computer vision, and neural networks. For them, the language should be compatible with familiar programming language concepts and syntax.

The second piece to this is that the language should be human-readable. The culture of the Web is one where anyone can start writing a webpage with just a text editor and a browser. This democratization of content is one of the Web’s greatest strengths. This culture has created a rich ecosystem of tools and inspectors, where tinkerers can investigate how any webpage works simply by View-Source. Having a single canonical human-readable language will greatly aid in community adoption of the WebGPU API.

All the major languages used on the web today are human-readable, with one exception. The WebAssembly Community Group expected that parsing a bytecode would be more performant than parsing a text language. However, that turned out not to be true; Asm.js, which is JavaScript source, is still faster than WebAssembly for many use cases.

Similarly, using a bytecode format such as WebAssembly doesn’t obviate the browser from needing to run optimization passes over the source code. Every major browser runs optimization passes on the bytecode prior to execution. Unfortunately, the desires of simpler compilers never ended up panning out.

There is active debate in the Community Group about whether or not this human-readable language should be the one that’s natively accepted by the API, but the group agrees that the language that shaders are written in should be easily readable and writable.

A New Language? Really?

While there are a number of existing languages, none have been designed with both the Web and modern graphics applications in mind, and none that address the requirements listed above. Before we describe WHLSL, let’s look at some existing languages.

Metal Shading Language is very similar to C++, which means it has all the power of bit-casts and raw pointers. It’s extremely powerful; the same source code can even be compiled for CPUs and GPUs. It’s extremely easy to port existing CPU-side code to Metal Shading Language. Unfortunately, all this power has some downsides. In Metal Shading Language, you could, for example, write a shader that casts a pointer to an integer, adds 17, casts it back to a pointer, and dereferences it. This is a security problem because it means the shader could access any resource that happens to be in the address space of the application, which is contrary to the Web’s security model. Theoretically, it could be possible to specify a dialect of Metal Shading Language that doesn’t have raw pointers, but pointers are so fundamental to the C and C++ languages that the result would be completely unfamiliar. C++ also heavily relies on undefined behavior, so any effort to fully specify each of C++’s numerous features would be unlikely to be successful.

HLSL is the supported language that Direct3D shaders written in. It’s currently the most popular realtime shading language in the world, and is therefore the most familiar language to graphics programmers. There are multiple implementations, but there is no formal specification, making it difficult to create consistent, interoperable implementations. Nonetheless, given HLSL’s ubiquity, it is valuable to adopt its syntax as much as possible in the design of WHLSL.

GLSL is the language used by WebGL, and was adopted by WebGL for the web platform. However, reaching cross-browser interoperability was extremely difficult due to incompatibilities in GLSL compilers. There remains a long tail of security and portability bugs with GLSL still being investigated. Also, GLSL is showing its age. It’s limited in that it doesn’t have pointer-like objects, or the ability to have variable length arrays. Its input and outputs are global variables with hardcoded names.

SPIR-V was designed to be a low-level universal intermediate format for the actual shading languages that developers would use. People do not author SPIR-V; they use a human-readable language, and then convert it into SPIR-V bytecode using a tool.

There are a few challenges with adopting SPIR-V for the web. First, SPIR-V was not written with security as a first principle, and it’s unclear whether it can be modified to satisfy the security requirements of the web. Forking the SPIR-V language means developers would have to recompile their shaders, possibly being forced to rewrite their source code anyway. Additionally, browsers would still be unable to trust incoming bytecode, and would be required to validate programs to make sure they are not doing anything insecure. And since Windows and macOS/iOS don’t support Vulkan, the incoming SPIR-V would still need to be translated/compiled into another language. Weirdly, this would mean on those two platforms, the starting point and the ending point are both human readable, but the bit in between is obfuscated with no benefit.

Second, a significant amount of the SPIR-V specification exists inside separate documents known as “execution environments.” A SPIR-V execution environment currently doesn’t exist for the Web, and without one of these execution environments, many critical pieces of SPIR-V are undefined, such as which of the over 50 optional capabilities are supported.

Third, many graphics applications such as Babylon.js require dynamically modifying shaders at runtime. Using a bytecode format means that these applications would have to include a compiler written in JavaScript that runs in the browser to produce the bytecode from the dynamically created shader. This would significantly increase the bloat of these sites and would lead to worse performance.

Though JavaScript is the canonical language for the Web, its properties make it a poor candidate for a shading language. One of its strengths is its flexibility, but this dynamism leads to many conditionals and divergent control flow, which GPUs are not designed to execute efficiently. It is also garbage-collected, which is a procedure that definitely isn’t well-suited for GPU hardware.

WebAssembly is another familiar possibility, but it also doesn’t map well to the architecture of GPUs. For example, WebAssembly assumes a single dynamically-sized heap, but GPU programs operate with access to multiple dynamically-sized buffers. There isn’t a high-performance way to map between the two models without recompiling.

Therefore, after a fairly exhaustive search for a suitable language, we couldn’t find one which adequately meets the requirements of the project. So, the Community Group is making a new language. Creating a new language is a large task, but we feel there is an opportunity to produce something new that uses modern programming language design principles and fulfills our requirements.

WHLSL

WHLSL is a new shading language that fits the Web platform. It’s being developed by the WebGPU Community Group at the W3C, and the group is working on a specification, a compiler, and a CPU-side interpreter to show correctness.

The language is based on HLSL, but simplifies and extends it. We’d really like existing HLSL shaders to just work as WHLSL shaders. Since WHLSL is a well-specified powerful and expressive shading language, some HLSL shaders will need tweaks to work, but as a result, WHLSL can guarantee safety and other benefits outlined above.

For example, here is an example vertex shader from Microsoft’s DirectX-Graphics-Samples repository. It works as a WHLSL shader without any changes:

VSParticleDrawOut output;
output.pos = g_bufPosVelo[input.id].pos.xyz;
float mag = g_bufPosVelo[input.id].velo.w / 9;
output.color = lerp(float4(1.0f, 0.1f, 0.1f, 1.0f), input.color, mag);
return output;

And here’s the associated pixel shader, which works as a WHLSL shader completely unmodified:

float intensity = 0.5f - length(float2(0.5f, 0.5f) - input.tex);
intensity = clamp(intensity, 0.0f, 0.5f) * 2.0f;
return float4(input.color.xyz, intensity);

Basics

Let’s talk about the language itself.

Just like in HLSL, the primitive data types are bool, int, uint, float, and half. Doubles are not supported because they don’t exist in Metal, and software emulation would be too slow. Bools don’t have a particular bit representation and thus cannot be present in shader inputs/outputs or resources. This same restriction is present in SPIR-V, and we’d like to be able to use OpTypeBool in the generated SPIR-V code. WHLSL also includes smaller integral types of char, uchar, short, and ushort, which are available directly in Metal Shading Language, can be specified in SPIR-V by specifying 16 in OpTypeFloat, and can be emulated in HLSL. Emulation of these types is faster than emulation of doubles because the types are smaller and their bit representation is less complicated.

WHLSL doesn’t provide C-style implicit conversions. We’ve found implicit conversions to be a common source of errors in shaders, and forcing the programmer to be explicit about where the conversions occur eliminates this often frustrating and mysterious class of bugs. This is a similar approach that languages such as Swift have taken. Additionally, a lack of implicit conversions keeps the specification and the compiler simple.

Just like in HLSL, there are vector types and matrix types such as float4 and int3x4. Rather than add a bunch of “x1” single-element vectors and matrices, we opted to keep the standard library simple, since a single-element vector is already representable as a scalar and a single-element-matrix is already representable as a vector. This is consistent with the desire to eliminate implicit conversions, and requiring an explicit conversion between float1 and float is cumbersome and needlessly verbose.

So, the following is a valid snippet of a shader:

int a = 7;
a += 3;
float3 b = float3(float(a) * 5, 6, 7);
float3 c = b.xxy;
float3 d = b * c;

I mentioned earlier that no implicit conversions are allowed, but you may have noticed in the above snippet, 5 is not written as 5.0. This is because literals are represented as a special type that can be unified with other numeric types. When the compiler sees the above code, it knows the multiplication operator requires the arguments to be the same type, and the first argument is clearly a float. So, when the compiler sees float(a) * 5 it says “well, I know the first argument is a float, so that means I must be using the (float, float) overload, so let’s unify the 5 with the second argument, and thus the 5 becomes a float.” This works even when both arguments are literals, because literals have a preferred type. So, 5 * 5 will get the (int, int) overload, 5u * 5u will get the (uint, uint) overload, and 5.0 * 5.0 will get the (float, float) overload.

One difference between WHLSL and C is that WHLSL zero-initializes all uninitialized variables at their declaration site. This prevents non-portable behavior across OSes and drivers, or even worse, reading whatever value happened to be there before your shader started executing. It also means that all constructible types in WHLSL have a zero-value.

Enums

Because they don’t incur any runtime cost and are extremely useful, WHLSL has native support for enums.

enum Weekday {
    Monday,
    Tuesday,
    Wednesday,
    Thursday,
    PizzaDay
}

The underlying type for an enum defaults to int, but you can override the type, e.g. enum Weekday : uint. Similarly, enum values can have an underlying value like Tuesday = 72. Because enums have defined types and values, they can therefore be used in buffers, and they can be casted between their underlying type and the enum type. When you want to refer to a value in code, you qualify it like Weekday.PizzaDay similar to how enum classes work in C++. This means that enum values don’t pollute the global namespace, and values of independent enums won’t collide.

Structs

Structs in WHLSL work similarly to HLSL and C.

struct Foo {
    int x;
    float y;
}

Simply designed, they avoid inheritance, virtual methods, and access control. It’s impossible to have a “private” member of a struct. Because structs don’t have access control, there is no need for structs to have member functions. Free functions can see every member of every struct.

Arrays

Like other shading languages, arrays are value types that are passed and returned from functions by value (aka “copy-in copy-out,” like regular scalars). You make one using the following syntax:

int[3] x;

Just like any variable declaration, this will zero-fill the contents of the array, and is therefore an O(n) operation. We wanted to put the brackets after the type instead of after the variable name for two reasons:

  1. Putting all type information in a single place makes the parser simpler (avoiding the clockwise/spiral rule)
  2. Avoiding ambiguity when multiple variables are declared in a single statement (e.g. int[10] x, y;)

One critical way we ensure safety of the language is performing bounds checking on every array access. There are a number of ways we make this potentially expensive operation efficient. Array indexes are uint, which reduce the check to a single comparison. Arrays are not sparsely implemented, and contain a length member which is available at compile-time, making access have near-zero cost.

Whereas arrays are value types, WHLSL achieves reference semantics using two other types: safe pointers and array references.

Safe Pointers

The first is the safe pointer. Some form of reference semantics, which is the behavior pointers allow for, are used in almost every CPU-side programming language. Including pointers in WHLSL will make it easier for developers to migrate existing CPU-side code to the GPU, thereby allowing for easy porting of things like machine learning, computer vision, and signal processing applications.

To satisfy the safety requirement, WHLSL uses safe pointers, which are guaranteed to either point to something valid or be null. Like C, you can create a pointer to an lvalue by using the & operator and can dereference one by using the * operator. Unlike C, you can’t index through a pointer as-if it were an array. You can’t cast it to and from a scalar value, and it doesn’t have a specific bit pattern representation. Therefore, it can’t exist in a buffer or as a shader input/output.

Just like in OpenCL and Metal Shading Language, the GPU has different heaps, or address spaces that values can exist within. WHLSL has 4 different heaps: device, constant, threadgroup, and thread. All reference types must be tagged with the address space they point into.

The device address space corresponds to the majority of memory on the device. This memory is readable and writable, and corresponds to Unordered Access Views in Direct3D and device memory in Metal Shading Language. The constant address space corresponds to a read-only region of memory, typically optimized for data being broadcast to every thread. As such, writing to an lvalue that exists in the constant address space is a compile error. Lastly, the threadgroup address space corresponds to a readable and writable region of memory that is shared between each thread in a threadgroup. It can only be used in compute shaders.

By default, values exist within the thread address space:

int i = 4;
thread int* j = &i;
*j = 7;
// i is now 7

Because all variables are zero-initialized, pointers are null-initialized. Therefore, the following is valid:

thread int* i;

Trying to dereference this pointer will cause either trapping or clamping, as described later.

Array References

Array references are similar to pointers, but they can be used with the subscript operator to access multiple elements in the array reference. Whereas arrays’ lengths are known at compile time and must be stated inside the type declaration, an array reference’s length is only known at runtime. Just like pointers, they must be associated with an address space, and they may be nullptr. Just like arrays, they are indexed using uints for single-comparison bounds checks, and they can’t be sparse.

They correspond to the OpTypeRuntimeArray type in SPIR-V and one of Buffer, RWBuffer, StructuredBuffer, or RWStructuredBuffer in HLSL. In Metal, it is represented as a tuple of a pointer and a length. Just like array accesses, all operations are checked against the array reference’s length. Buffers are passed into the entry points from the API via array references or pointers.

You can make an array reference from an lvalue by using the @ operator:

int i = 4;
thread int[] j = @i;
j[0] = 7;
// i is 7
// j.length is 1

Just as you might expect, using @ on pointer j creates an array reference that points to the same thing as j:

int i = 4;
thread int* j = &i;
thread int[] k = @j;
k[0] = 7;
// i is 7
// k.length is 1

Using @ on an array makes the array reference point to that array:

int[3] i = int[3](4, 5, 6);
thread int[] j = @i;
j[1] = 7;
// i[1] is 7
// j.length is 3

Functions

Functions look very similar to their C counterparts. For example, here is a function in the standard library:

float4 lit(float n_dot_l, float n_dot_h, float m) {
    float ambient = 1;
    float diffuse = max(0, n_dot_l);
    float specular = n_dot_l < 0 || n_dot_h < 0 ? 0 : n_dot_h * m;
    float4 result;
    result.x = ambient;
    result.y = diffuse;
    result.z = specular;
    result.w = 1;
    return result;
}

This example shows how similar WHLSL functions are to C: function declarations and calls (e.g. to max()) have similar syntax, arguments and parameters are matched up pairwise in order, and ternary expressions are supported.

Operators and Operator Overloading

However, something else is going on here, too. When the compiler sees n_dot_h * m, it doesn’t intrinsically know how to perform that multiplication. Instead, the compiler will turn that into a call to operator*(). Then, the specific operator*() is chosen via the standard function overload resolution algorithm. This is important because it means you can write your own operator*() function, and teach WHLSL how to multiply your own types.

This even works for operations like ++. Though pre- and post-increment have different behaviors, they both get overloaded to the same function: operator++(). Here’s an example from the standard library:

int operator++(int value) {
    return value + 1;
}

This operator will be called for both pre-increment and post-increment, and the compiler is smart enough to do the right thing with the result. This solves the problem that C++ runs into where those operators are distinct, and are differentiated using an extra dummy int argument. For post-increment, the compiler will emit code to save the value to an anonymous variable, call operator++(), assign the result, and use the saved value for further processing.

Operator overloading is used all over the language. It’s how vector and matrix multiplication is implemented. It’s how arrays are indexed. It’s how swizzle operators work. Operator overloading provides power and simplicity; the core language doesn’t have to know about each of these operations directly because they are implemented by overloaded operators.

Generated Properties

WHLSL doesn’t just stop at operator overloading, though. An earlier example included b.xxy where b is a float3. This is an expression that means “make a 3-element vector where the first two elements have the same value as b.x and the third element has the same value as b.y.” So it’s sort of like a member of the vector, except it isn’t actually associated with any storage; instead, it’s computed during the time it’s accessed. These “swizzle operators” are present in every realtime shading language, and WHLSL is no exception. The way they’re supported is by marking them as a generated property, like in Swift.

Getters

The standard library includes many functions of the following form:

float3 operator.xxy(float3 v) {
    float3 result;
    result.x = v.x;
    result.y = v.x;
    result.z = v.y;
    return result;
}

When the compiler sees a property access to a member that doesn’t exist, it can call the operator passing the object as the first argument. Colloquially, we call this a getter.

Setters

The same approach even works for setters:

float4 operator.xyz=(float4 v, float3 c) {
    float4 result = v;
    result.x = c.x;
    result.y = c.y;
    result.z = c.z;
    return result;
}

Using setters is very natural:

float4 a = float4(1, 2, 3, 4);
a.xyz = float3(7, 8, 9);

The implementation of the setter creates a copy of the object with the new data. When the compiler sees an assignment to a generated property, it calls the setter and assigns the result to the original variable.

Anders

A generalization of getters and setters is the ander, which works with pointers. It exists as a performance optimization, so setters don’t have to create a copy of the object. Here’s an example:

thread float* operator.r(thread Foo* value) {
    return &value->x;
}

Anders are more powerful than either getters or setters, because the compiler can use anders to implement either reads or assignments. When reading from a generated property via an ander, the compiler invokes the ander and then dereferences the result. When writing to it, the compiler invokes the ander, dereferences the result, and assigns to the result of that. Any user-defined type can have any combination of getters, setters, anders, and indexers; if the same type has an ander and either a getter or a setter, the compiler will prefer using the ander.

Indexers

But what about matrices? In most realtime shading languages, matrices aren’t accessed with members corresponding to their columns or rows. Instead, they are accessed using array syntax, e.g. myMatrix[3][1]. Vector types also usually have this kind of syntax. So how does this work? More operator overloading!

float operator[](float2 v, uint index) {
    switch (index) {
        case 0:
            return v.x;
        case 1:
            return v.y;
        default:
            /* trap or clamp, more on this below */
    }
}

float2 operator[]=(float2 v, uint index, float a) {
    switch (index) {
        case 0:
            v.x = a;
            break;
        case 1:
            v.y = a;
            break;
        default:
            /* trap or clamp, more on this below */
    }
    return v;
}

As you can see, indexing uses operators too, and can therefore be overloaded. Vectors get these “indexers” too, so myVector.x and myVector[0] are synonyms for each other.

Standard Library

We designed the standard library based on the Microsoft Docs describing the HLSL standard library. The WHLSL standard library mostly includes math operations, which work both on scalar values and element-wise on vectors and matrices. All the standard operators you would expect are defined, including logical and bitwise operations, like operator*() and operator<<(). All the swizzle operators, getters, and setters are defined for vectors and matrices, where applicable.

One of the design principles of WHLSL was to keep the language itself small so as much as possible could be defined in the standard library. Of course, not all the functions in the standard library can be expressed in WHLSL (like bool operator*(float, float)) but almost everything else is implemented in WHLSL. For example, this function is part of the standard library:

float smoothstep(float edge0, float edge1, float x) {
    float t = clamp((x - edge0) / (edge1 - edge0), 0, 1);
    return t * t * (3 - 2 * t);
}

Because the standard library is designed to match HLSL as much as possible, most of the functions in it are already present in HLSL directly. So a compilation of WHLSL’s standard library to HLSL would choose to omit these functions and instead use the built-in versions. This will happen, for instance, for all the vector/matrix indexers — the GPU should never actually see the code above; the code generation step in the compiler should use the intrinsic instead. However, different shading languages have different intrinsics, so every function is defined to allow for correctness testing. Similarly, WHLSL includes a CPU-side interpreter, which uses the WHLSL implementations of these functions when executing WHLSL programs. This is true for every WHLSL function including the texture sampling functions.

Not every function in HLSL’s standard library is present in WHLSL. For example, HLSL supports printf(). However, implementing such a function in Metal Shading Language or SPIR-V would be quite difficult. We included as many functions from the HLSL standard library as is reasonable in the Web environment.

Variable Lifetime

But if there are pointers in the language, how should we deal with use-after-free problems? For example, consider the following snippet:

thread int* foo() {
    int a;
    return &a;
}
…
int b = *foo();

In languages like C, this code has undefined behavior. So, one solution is for WHLSL to just forbid this kind of structure and throw a compilation error when it sees something like this. However, this would require tracking the values that every pointer could possibly point to, which is a difficult analysis in the presence of loops and function calls. Instead, WHLSL makes every variable behave as if it has global lifetime.

This means that this WHLSL snippet is completely valid and well-defined, for two reasons:

  1. Declaring a without an initializer will zero-fill it. Therefore, the value of a is well-defined. This zero-filling will occur each time foo() is called.
  • All variables have global lifetime (similar to C’s static keyword). Therefore, a never goes out of scope.

  • This global lifetime is only possible because recursion is disallowed (which is common for shading languages), which means there will not be any reentrancy problems. Similarly, shaders cannot allocate or free memory, so the compiler knows at compile-time every piece of memory that a shader could possibly access.

    So, for example:

    thread int* foo() {
        int a;
        return &a;
    }
    …
    thread int* x = foo();
    *x = 7;
    thread int* y = foo();
    // *x equals 0, because the variable got zero-filled again
    *y = 8;
    // *x equals 8, because x and y point to the same variable
    

    Most variables won’t need to truly be made global, so there isn’t a big impact on performance. If the compiler can prove that it is unobservable whether or not a particular variable actually has global lifetime, the compiler is free to keep the variable local. Because the pattern of returning pointers to locals is discouraged in other languages (indeed, many other shading languages don’t even have pointers), examples like this one will be relatively rare.

    Stages of Compilation

    WHLSL doesn’t make use of a preprocessor as other languages do. In other languages, the preprocessor’s primary purpose is to include multiple source files together. On the web, however, there is no direct file access, and usually the entire shader is presented in one downloaded resource. In many shading languages, the preprocessor is used to conditionally enable rendering features inside a large ubershader, but WHLSL allows for this use case by using specialization constants instead. Moreover, the many variants of preprocessors are incompatible in subtle ways, so the benefit of a preprocessor for WHLSL doesn’t outweigh the complexity of creating a specification for one.

    WHLSL is designed for a two-stage compilation. In our research, we’ve found that many 3D engines want to compile a large corpus of shaders, and each compilation includes large libraries of functions that are duplicated between the different compilations. Instead of compiling these support functions multiple times, a better solution is to compile the entire library once, and then allow a second stage to select which entry points from the library should be used together.

    This two-stage compilation means that as much of the compilation should be done in the first pass, so it isn’t run multiple times for families of shaders. This is the reason entry points in WHLSL are marked as either vertex, fragment, or compute. Letting the first stage of the compilation know which functions are entry points of which type lets more of the compilation occur in the first stage rather than the second stage.

    This second compilation stage also provides a convenient place to specify specialization constants. Recall that WHLSL doesn’t have a preprocessor, which is the traditional way for features to be enabled and disabled in HLSL. Engines often tailor a single shader to a particular situation by enabling a rendering effect or switching out a BRDF with the flip of a switch. The technique of including every rendering option in a single shader, and specializing the single shader based on which effects to enable, is so common it has a name: ubershaders. Instead of preprocessor macros, WHLSL programmers can use specialization constants, which work the same way as SPIR-V’s specialization constants. From the language’s point of view, they are just scalar constants. However, the values for these constants are supplied during this second compilation stage, making it super easy to configure the program at runtime.

    Because a single WHLSL program can include multiple shaders, the inputs and outputs to the shader aren’t represented by global variables in the way that other shading languages do it. Instead, the inputs and the outputs for a particular shader are associated with that shader itself. Inputs are represented as arguments to the shader’s entry point and outputs are represented as the return value of the entry point.

    The following shows how to describe a compute shader entry point:

    compute void ComputeKernel(device uint[] b : register(u0)) {
       …
    }
    

    Safety

    WHLSL is a safe language. This means that it is impossible to access information outside of a website’s origin. One of the ways WHLSL achieves this is by eliminating undefined behavior, as described above regarding uniformity.

    Another way WHLSL achieves safety is by performing bounds checks of array/pointer accesses. There are three ways these bounds checks may occur:

    1. Trapping. When a trap occurs in a program, the shader stage immediately exits, filling in 0s for all of the shader stage’s outputs. The draw call continues, and the next stage of the graphics pipeline gets run.
      Because trapping introduces new control flow, it has an effect on the uniformity of the program. Traps are issued inside bounds checks, which means they are necessarily present in non-uniform control flow. It may be okay for some programs that don’t use uniformity for anything, but in general this makes traps difficult to use.
    2. Clamping. Array index operations can clamp the index to the size of the array. This doesn’t involve new control flow, so it doesn’t have any affect on uniformity. It is even possible to “clamp” a pointer access or a zero-length array access by disregarding writes and returning 0s for reads. This is possible because the set of things you can do with a pointer in WHLSL is limited, so we can simply make each of those operations do some well-defined thing with a “clamped” pointer.
    3. Hardware and Driver Support. Some hardware and drivers already includes a mode where out-of-bounds accesses can’t happen. With this method, the mechanism by which the hardware forbids out-of-bounds accesses is implementation-defined. One example is the ARB_robustness OpenGL Extension. Unfortunately, WHLSL should be runnable on almost all modern hardware, and simply not enough APIs/devices support these kinds of modes.

    Whichever method the compiler uses, it should not affect the uniformity of the shader; in other words, it can’t turn an otherwise valid program into an invalid one.

    In order to determine the best behavior for bounds checks, we ran some performance experiments. We took some of the kernels used in the Metal Performance Shaders framework and made two new versions: one that uses clamping, and one that uses trapping. The kernels we picked were ones that do lots of array accesses: for example, multiplying large matrices. We ran this benchmark on a variety of devices at varying data sizes. We made sure that none of the traps were actually hit and none of the clamps actually had any effect, so we can be sure we were measuring the common case of a correctly-written program.

    We expected trapping to be generally faster, because redundant traps can be eliminated by the downstream compiler. However, we discovered that there wasn’t one clear winner. On some devices, trapping was significantly faster than clamping, and on other devices, clamping was significantly faster than trapping. These results show that the compiler should be able to choose which method is best for the particular device it’s being run on, rather than being forced to always choose one method.

    Chart of iPhone 6 vs iPhone X runtime scores

    Shader Signatures

    WHLSL supports a language feature of HLSL called “semantics.” They are used to identify variables between shader stages and from the WebGPU API. There are four types of semantics:

    • Built-in variables, e.g. uint vertexID : SV_VertexID
    • Specialization constants, e.g. uint numlights : specialized
    • Stage in/out semantics, e.g. float2 coordinate : attribute(0)
    • Resource semantics, e.g. device float[] coordinate : register(u0)

    As described above, WHLSL programs accept their inputs and outputs in the form of function parameters, not global variables.

    However, shaders often have multiple outputs. The most common example of this is the vertex shader passing multiple output values to the interpolator to be fed as inputs into the fragment shader.

    In order to accommodate this, the return value of a shader can be a struct, and the individual fields are treated independently. In fact, this works recursively — the struct can contain another struct, and its members are also treated independently. Nested structs are flattened, and all the fields which aren’t structs are gathered and treated as shader outputs.

    Shader parameters work the same way. An individual parameter can be a shader input, or it can be a struct with a collection of shader inputs. Structs can also contain other structs. Variables inside these structs are treated independently, as if they were additional parameters to the shader.

    After all these structs have been flattened into a set of inputs and a set of outputs, each item in the sets must have a semantic. Each built-in variable must have a particular type and must only be used in a particular shader stage. Specialization constants must only have simple scalar types.

    Stage in/out variables have the attribute semantic rather than the traditional HLSL semantics because many shaders pass around data that don’t match the canned semantics HLSL provides. In HLSL, it’s common to see generic data packed into the COLOR semantic, because COLOR is a float4 and the data fits inside a float4. Instead, the approach both SPIR-V and Metal Shading Language (via [[user(n)]]) take is to assign an identifier to each stage in/out variable, and use the assignments to match the variables between shader stages.

    Resource semantics should be familiar to HLSL programmers. WHLSL includes both resource semantics and address spaces, but both of these have different purposes. The address space of the variable is used to determine which cache and memory hierarchy it should be accessed within. The address space is necessary because it persists even through pointer operations; a device pointer can’t be set to point to a thread variable. In WHLSL, the resource semantic is only used to identify a variable from the WebGPU API. However, for consistency with HLSL, the resource semantic must “match” the address space of the variable it’s being put on. For example, you can’t put register(s0) on a texture. You can’t put register(u0) on a constant resource. Arrays in WHLSL don’t have address space (because they are value types, not reference types) so if an array appears as a shader argument, it is treated as if it was a device resource for the purposes of matching semantics.

    Just like Direct3D, WebGPU has a two-level binding model. Resource descriptors are aggregated into sets, and sets can be switched out in the WebGPU API. WHLSL matches HLSL by modeling this by an optional space parameter inside resource semantics: register(u0, space1).

    “Logical Mode” restrictions

    WHLSL is designed with the requirement that it is compilable to Metal Shading Language, SPIR-V, and HLSL (or DXIL). SPIR-V has many different operating modes, targeted by different embedding APIs. Specifically, we’re interested in the flavor of SPIR-V that Vulkan targets.

    This flavor of SPIR-V is a flavor of SPIR-V called Logical Addressing Mode. In SPIR-V Logical Mode, variables cannot have pointer type. Similarly, a pointer cannot be used in a Phi operation. The result of this is that each pointer must point to exactly one thing for all time; a pointer is simply a name for a value.

    Because WHLSL needs to be compilable to SPIR-V, WHLSL must not be more expressive than SPIR-V. Therefore, WHLSL has some restrictions to make it expressible in SPIR-V Logical Mode. These restrictions aren’t surfaced as an optional mode for WHLSL; instead, they’re part of the language itself. Eventually, we hope these restrictions can be lifted in a future version of the language, but until then, the language is restricted.

    These restrictions are:

    • Pointers and array references must not occur inside device, constant, or threadgroup memory
    • Pointers and array references must not occur inside arrays or array references
    • Pointers and array references must not be assigned outside of their initializer (in their declaration)
    • Functions that return pointers or array references must only have a single return point
    • Ternary expressions must not result in pointers

    With these restrictions, the compiler knows exactly what every pointer points to.

    But not so fast! Recall from above that thread variables have global lifetime, which means they behave as-if they were declared at the beginning of the entry point. What if the runtime gathered all these local variables together, sorted by type, and aggregated all the variables with the same type into arrays? Then, a pointer could simply be an offset into the appropriate array. A pointer can’t be recast to point to a different type in WHLSL, which means the appropriate array is determined statically by the compiler. Therefore, thread pointers don’t need to abide by the restrictions above. But, this technique doesn’t work for the pointers in the other address spaces; it only works for thread pointers.

    Resources

    WHLSL supports textures, samplers, and array references for buffers. Just like in HLSL, texture types in WHLSL look like Texture2D<float4>. The presence of these angle brackets don’t imply templates or generics; the language doesn’t have facilities for those (for simplicity). The only types that are allowed to have them are a finite set of built-in types. This design is a middle ground between allowing these types, which are present in HLSL, but also allowing further development of the language in a way that the Community Group can use the angle bracket characters.

    Depth textures are distinct from non-depth-textures because they are different types in Metal Shading Language, so the compiler needs to know which one to emit when it’s emitting Metal Shading Language. Because WHLSL doesn’t support member functions, texture sampling isn’t done like texture.Sample(…); instead, it’s done with free functions like Sample(texture, …).

    Samplers are not specialized; there is one sampler type for all use cases. You can use this sampler for both depth textures and non-depth textures. Depth textures support things like comparison operations in the sampler. If the sampler is configured to include a depth comparison and it’s used with a non-depth texture, the depth operation is ignored.

    The WebGPU API will automatically emit some resource barriers at particular places, which means the API needs to know which resources are used in a shader. Therefore, the “bindless” model of resources can’t be used. This means that all resources are listed as explicit inputs to a shader. Similarly, the API wants to know which resources are used for reading and which are used for writing; the compiler knows this statically by inspecting the program. There is no language-level support for “const” or a distinction between StructuredBuffer and RWStructuredBuffer because that information is already present in the program.

    Current Work

    The WebGPU community group is working on a formal language specification written with OTT that describes WHLSL with the same level of rigor that other Web languages employ. We’re also working on a compiler that can produce Metal Shading Language, SPIR-V, and HLSL. In addition, the compiler includes a CPU-side interpreter to show correctness of an implementation. Please try it out!

    Future Directions

    WHLSL is still nascent, and there is still a long way to go before the design of the language is complete. We would love to hear from you about your desires, concerns, and use cases! Please feel free to file issues in our GitHub repository about your ideas and thoughts!

    For the first proposal, we wanted to satisfy the constraints outlined at the beginning of this post, yet provide ample opportunity for expanding the language. One natural evolution of the language could add facilities for abstractions of types, like protocols or interfaces. WHLSL includes simple structs with no access control or inheritance. Other shading languages like Slang model type abstractions as a set of methods that must be present inside the struct. However, Slang runs into a problem where it is impossible to make an existing type adhere to a new interface. Once the struct is defined, you can’t add new methods to it; the curly brace has closed the struct forever. This problem is solved with extensions, similar to Objective-C or Swift, which can retroactively add methods into a struct after the struct has been defined. Java solved this problem by encouraging authors to add new classes, called adapters, which only exist to implement an interface, and plumb each call through to the implementing type.

    The WHLSL approach is much simpler; by using free functions instead of struct methods, we can use a system like Haskell’s type classes. Here, a type class defines a set of arbitrary functions that must exist, and a type adheres to the type class by implementing them. A solution like this could potentially be added to the language in the future.

    Wrapping Up

    This describes a new shading language named WHLSL that the W3C’s WebGPU Community Group owns. The goals of the language are satisfied by its familiar, HLSL-based syntax, safety guarantees, and simple, extensible design. As such, it represents the best-supported way to write shaders to be used in the WebGPU API. However, the WebGPU Community Group is unsure whether or not WHLSL programs should be supplied to the WebGPU API directly, or whether they should be compiled to an intermediate form before delivery to the API. Either way, WebGPU programmers should be writing in WHLSL because it fits best with the API.

    Please get involved! We’re doing this work on the WebGPU GitHub project. We’ve been working on a formal specification for the language, a reference compiler to emit Metal Shading Language and SPIR-V, and a CPU-side interpreter to validate correctness. We welcome everyone to try it out, and let us know how it goes!

    For more information, you can contact me at mmaxfield@apple.com or @Litherum, or you can contact our evangelist, Jonathan Davis.

    November 12, 2018 11:21 PM

    Michael Catanzaro: The GNOME (and WebKitGTK+) Networking Stack

    Igalia WebKit

    WebKit currently has four network backends:

    • CoreFoundation (used by macOS and iOS, and thus Safari)
    • CFNet (used by iTunes on Windows… I think only iTunes?)
    • cURL (used by most Windows applications, also PlayStation)
    • libsoup (used by WebKitGTK+ and WPE WebKit)

    One guess which of those we’re going to be talking about in this post. Yeah, of course, libsoup! If you’re not familiar with libsoup, it’s the GNOME HTTP library. Why is it called libsoup? Because before it was an HTTP library, it was a SOAP library. And apparently somebody thought that when Mexican people say “soap,” it often sounds like “soup,” and also thought that this was somehow both funny and a good basis for naming a software library. You can’t make this stuff up.

    Anyway, libsoup is built on top of GIO’s sockets APIs. Did you know that GIO has Object wrappers for BSD sockets? Well it does. If you fancy lower-level APIs, create a GSocket and have a field day with it. Want something a bit more convenient? Use GSocketClient to create a GSocketConnection connected to a GNetworkAddress. Pretty straightforward. Everything parallels normal BSD sockets, but the API is nice and modern and GObject, and that’s really all there is to know about it. So when you point WebKitGTK+ at an HTTP address, libsoup is using those APIs behind the scenes to handle connection establishment. (We’re glossing over details like “actually implementing HTTP” here. Trust me, libsoup does that too.)

    Things get more fun when you want to load an HTTPS address, since we have to add TLS to the picture, and we can’t have TLS code in GIO or GLib due to this little thing called “copyright law.” See, there are basically three major libraries used to implement TLS on Linux, and they all have problems:

    • OpenSSL is by far the most popular, but it’s, hm, shall we say technically non-spectacular. There are forks, but the forks have problems too (ask me about BoringSSL!), so forget about them. The copyright problem here is that the OpenSSL license is incompatible with the GPL. (Boring details: Red Hat waves away this problem by declaring OpenSSL a system library qualifying for the GPL’s system library exception. Debian has declared the opposite, so Red Hat’s choice doesn’t gain you anything if you care about Debian users. The OpenSSL developers are trying to relicense to the Apache license to fix this, but this process is taking forever, and the Apache license is still incompatible with GPLv2, so this would make it impossible to use GPLv2+ software except under the terms of GPLv3+. Yada yada details.) So if you are writing a library that needs to be used by GPL applications, like say GLib or libsoup or WebKit, then it would behoove you to not use OpenSSL.
    • GnuTLS is my favorite from a technical standpoint. Its license is LGPLv2+, which is unproblematic everywhere, but some of its dependencies are licensed LGPLv3+, and that’s uncomfortable for many embedded systems vendors, since LGPLv3+ contains some provisions that make it difficult to deny you your freedom to modify the LGPLv3+ software. So if you rely on embedded systems vendors to fund the development of your library, like say libsoup or WebKit, then you’re really going to want to avoid GnuTLS.
    • NSS is used by Firefox. I don’t know as much about it, because it’s not as popular. I get the impression that it’s more designed for the needs of Firefox than as a Linux system library, but it’s available, and it works, and it has no license problems.

    So naturally GLib uses NSS to avoid the license issues of OpenSSL and GnuTLS, right?

    Haha no, it uses a dynamically-loadable extension point system to allow you to pick your choice of OpenSSL or GnuTLS! (Support for NSS was started but never finished.) This is OK because embedded systems vendors don’t use GPL applications and have no problems with OpenSSL, while desktop Linux users don’t produce tivoized embedded systems and have no problems with LGPLv3. So if you’re using desktop Linux and point WebKitGTK+ at an HTTPS address, then GLib is going to load a GIO extension point called glib-networking, which implements all of GIO’s TLS APIs — notably GTlsConnection and GTlsCertificate — using GnuTLS. But if you’re building an embedded system, you simply don’t build or install glib-networking, and instead build a different GIO extension point called glib-openssl, and libsoup will create GTlsConnection and GTlsCertificate objects based on OpenSSL instead. Nice! And if you’re Centricular and you’re building GStreamer for Windows, you can use yet another GIO extension point, glib-schannel, for your native Windows TLS goodness, all hidden behind GTlsConnection so that GStreamer (or whatever application you’re writing) doesn’t have to know about SChannel or OpenSSL or GnuTLS or any of that sad complexity.

    Now you know why the TLS extension point system exists in GIO. Software licenses! And you should not be surprised to learn that direct use of any of these crypto libraries is banned in libsoup and WebKit: we have to cater to both embedded system developers and to GPL-licensed applications. All TLS library use is hidden behind the GTlsConnection API, which is really quite nice to use because it inherits from GIOStream. You ask for a TLS connection, have it handed to you, and then read and write to it without having to deal with any of the crypto details.

    As a recap, the layering here is: WebKit -> libsoup -> GIO (GLib) -> glib-networking (or glib-openssl or glib-schannel).

    So when Epiphany fails to load a webpage, and you’re looking at a TLS-related error, glib-networking is probably to blame. If it’s an HTTP-related error, the fault most likely lies in libsoup. Same for any other GNOME applications that are having connectivity troubles: they all use the same network stack. And there you have it!

    P.S. The glib-openssl maintainers are helping merge glib-openssl into glib-networking, such that glib-networking will offer a choice of GnuTLS or OpenSSL and obsoleting glib-openssl. This is still a work in progress. glib-schannel will be next!

    P.S.S. libcurl also gives you multiple choices of TLS backend, but makes you choose which at build time, whereas with GIO extension points it’s actually possible to choose at runtime from the selection of installed extension points. The libcurl approach is fine in theory, but creates some weird problems, e.g. different backends with different bugs are used on different distributions. On Fedora, it used to use NSS, but now uses OpenSSL, which is fine for Fedora, but would be a license problem elsewhere. Debian actually builds several different backends and gives you a choice, unlike everywhere else. I digress.

    By Michael Catanzaro at November 12, 2018 04:51 AM

    November 07, 2018

    Release Notes for Safari Technology Preview 69

    Surfin’ Safari

    Safari Technology Preview Release 69 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 237231-237630.

    CSS

    • Enabled Conic Gradients by default (r237402)

    Web API

    • Enabled the <datalist> element by default (r237245)
    • Added new image type for the CSS Paint API (r237276)
    • Changed the CSS Painting API to give a 2D rendering context (r237344)
    • Changed the CSS Painting API to parse paint() and store paint callbacks (r237243)
    • Changed Web Animations to not create a DocumentTimeline to suspend or resume animations (r237231)

    Intersection Observer

    • Handled zero-area intersections (r237284)

    Dark Mode

    • Changed to use a dark appearance scrollbar when the page background is dark or the document supports dark mode (r237466)

    JavaScript

    • Implemented BigInt support for ^ (r237296)

    Media

    • Added support for MediaKeyEncryptionScheme (r237268)
    • Updated to be able to adapt between H.264 streams with and without EditList (r237271)
    • Updated TextTrack cues to be updated more often than every 250ms (r237376)
    • Fixed timestampOffset to prevent introducing floating-point rounding errors to incoming samples (r237274)
    • Fixed the WebVTT region parameter and value to be separated by : (r237270)

    Web Inspector

    • Added corner rounding to the network timing blocks in the Network tab waterfall (r237432)
    • Changed to more aggressively snap timing blocks together in the Network tab (r237430)
    • Fixed CSP JSON request payload without a MIME-type to be pretty-printed (r237396)
    • Added fullscreen enter and exit events in the Timelines and Network waterfalls (r237431)
    • Fixed the Quick Open dialog to show named scripts that appear in the Debugger sidebar (r237327)
    • Fixed the Quick Open dialog for line and column to have a caret indicate the position (r237232)
    • Fixed malformed popovers for function source code (r237401)
    • Prevented the Canvas tab from listening for “space” key press when the tab is not visible (r237560)
    • Improved Canvas Recording loading speed when the “Frame” tree element is expanded (r237436)
    • Updated the Canvas tab to show a warning when the path moves offscreen (r237574)

    WebRTC

    • Changed MediaRecorder to fire a stop event when all tracks are ended (r237311)
    • Changed to handle MDNS resolution of candidates through libwebrtc directly (r237568)

    Payment Request

    • Implemented MerchantValidationEvent.methodName (r237521)
    • Implemented PaymentResponse.retry() (r237597)
    • Updated PaymentRequest.canMakePayment() to resolve to true whenever Apple Pay is available (r237594)

    Storage

    • Added a deprecation warning to the console for Web SQL (r237591)
    • Fixed iteration of cursors skipping records if updated or deleted in IndexedDB (r237590)

    November 07, 2018 06:00 PM

    Michael Catanzaro: Mesa Update Breaks WebKitGTK+ in Fedora 29

    Igalia WebKit

    If you’re using Fedora and discovered that WebKitGTK+ is displaying blank pages, the cause is a bad mesa update, mesa-18.2.3-1.fc29. This in turn was caused by a GCC bug that resulted in miscompilation of mesa.

    To avoid this bug, downgrade to mesa-18.2.2-1.fc29:

    $ sudo dnf downgrade mesa*

    You can also update to mesa-18.2.4-2.fc29, but this build has not yet reached updates-testing, let alone stable, so downgrading is easier for now. Another workaround is to run your application with accelerated compositing mode disabled, to avoid OpenGL usage:

    $ WEBKIT_DISABLE_COMPOSITING_MODE=1 epiphany

    On the bright side of things, from all the bug reports I’ve received over the past two days I’ve discovered that lots of people use Epiphany and notice when it’s broken. That’s nice!

    Huge thanks to Dave Airlie for quickly preparing the fixed mesa update, and to Jakub Jelenik for handling the same for GCC.

    By Michael Catanzaro at November 07, 2018 02:28 AM

    November 03, 2018

    Michael Catanzaro: WebKitGTK+ 2.22.2 and 2.22.3, Media Source Extensions, and YouTube

    Igalia WebKit

    Last month, I attended the Web Engines Hackfest (hosted by Igalia in A Coruña, Spain) and also the WebKit Contributors Meeting (hosted by Apple in San Jose, California). These are easily the two biggest WebKit development events of the year, and it’s always amazing to meet everyone in person yet again. A Coruña is an amazing city, and every browser developer ought to visit at least once. And the Contributors Meeting is a no-brainer event for WebKit developers.

    One of the main discussion points this year was Media Source Extensions (MSE). MSE is basically a way for browsers to control how videos are downloaded. Until recently, if you were to play a YouTube video in Epiphany, you’d notice that the video loads way faster than it does in other browsers. This is because WebKitGTK+ — until recently — had no support for MSE. In other browsers, YouTube uses MSE to limit the speed at which video is downloaded, in order to reduce wasted bandwidth in case you stop watching the video before it ends. But with WebKitGTK+, MSE was not available, so videos would load as quickly as possible. MSE also makes it harder for browsers to offer the ability to download the videos; you’ll notice that neither Firefox nor Chrome offer to download the videos in their context menus, a feature that’s been available in Epiphany for as long as I remember.

    So that sounds like it’s good to not have MSE. Well, the downside is that YouTube requires it in order to receive HD videos, to avoid that wasted bandwidth and to make it harder for users to download HD videos. And so WebKitGTK+ users have been limited to 720p video with H.264 and 480p video with WebM, where other browsers had access to 1080p and 1440p video. I’d been stuck with 480p video on Fedora for so long, I’d forgotten that internet video could look good.

    Unfortunately, WebKitGTK+ was quite late to implement MSE. All other major browsers turned it on several years ago, but WebKitGTK+ dawdled. There was some code to support MSE, but it didn’t really work, and was disabled. And so it came to pass that, in September of this year, YouTube began to require MSE to access any WebM video, and we had a crisis. We don’t normally enable major new features in stable releases, but this was an exceptional situation and users would not be well-served by delaying until the next release cycle. So within a couple weeks, we were able to release WebKitGTK+ 2.22.2 and Epiphany 3.30.1 (both on September 21), and GStreamer 1.14.4 (on October 2, thanks to Tim-Philipp Müller for expediting that release). Collectively, these releases enabled basic video playback with MSE for users of GNOME 3.30. And if you still use of GNOME 3.28, worry not: you are still supported and can get MSE if you update to Epiphany 3.28.5 and also have the aforementioned versions of WebKitGTK+ and GStreamer.

    MSE in WebKitGTK+ 2.22.2 had many rough edges because it was a mad rush to get the feature into a minimally-viable state, but those issues have been polished off in 2.22.3, which we released earlier this week on October 29. Be sure you have WebKitGTK+ 2.22.3, plus GStreamer 1.14.4, for a good experience on YouTube. Unfortunately we can’t provide support for older software versions anymore: if you don’t have GStreamer 1.14.4, then you’ll need to configure WebKitGTK+ with -DENABLE_MEDIA_SOURCE=OFF at build time and suffer from lack of MSE.

    Epiphany 3.28.1 uses WebKitSettings to turn on the “enable-mediasource” setting. Turn that on if your application wants MSE now (if it’s a web browser, it certainly does). This setting will be enabled by default in WebKitGTK+ 2.24. Huge thanks to the talented developers who made this feature possible! Enjoy your 1080p and 1440p video.

    By Michael Catanzaro at November 03, 2018 04:19 AM

    Michael Catanzaro: On WebKit Build Options (Also: How to Accidentally Disable Important Security Features!)

    Igalia WebKit

    When building WebKitGTK+, it’s a good idea to stick to the default values for the build options. If you’re building some sort of embedded system and really know what you’re doing, then OK, it might make sense to change some settings and disable some stuff. But Linux distros are generally well-advised to stick to the defaults to avoid creating problems for users.

    One exception is if you need to disable certain features to avoid newer dependencies when building WebKit for older systems. For example, Ubuntu 18.04 disables web fonts (ENABLE_WOFF2=OFF) because it doesn’t have the libbrotli and libwoff2 dependencies that are required for that feature to work, hence some webpages will display using subpar fonts. And distributions shipping older versions of GStreamer will need to disable the ENABLE_MEDIA_SOURCE option (which is missing from the below feature list by mistake), since that requires the very latest GStreamer to work.

    Other exceptions are the ENABLE_GTKDOC and ENABLE_MINIBROWSER settings, which distros do want. ENABLE_GTKDOC is disabled by default because it’s slow to build, and ENABLE_MINIBROWSER because, well, actually I don’t know why, you always want that one and it’s just annoying to find it’s not built.

    OK, but really now, other than those exceptions, you should probably leave the defaults alone.

    The feature list that prints when building WebKitGTK+ looks like this:

    --  ENABLE_ACCELERATED_2D_CANVAS .......... OFF
    --  ENABLE_DRAG_SUPPORT                     ON
    --  ENABLE_GEOLOCATION .................... ON
    --  ENABLE_GLES2                            OFF
    --  ENABLE_GTKDOC ......................... OFF
    --  ENABLE_ICONDATABASE                     ON
    --  ENABLE_INTROSPECTION .................. ON
    --  ENABLE_JIT                              ON
    --  ENABLE_MINIBROWSER .................... OFF
    --  ENABLE_OPENGL                           ON
    --  ENABLE_PLUGIN_PROCESS_GTK2 ............ ON
    --  ENABLE_QUARTZ_TARGET                    OFF
    --  ENABLE_SAMPLING_PROFILER .............. ON
    --  ENABLE_SPELLCHECK                       ON
    --  ENABLE_TOUCH_EVENTS ................... ON
    --  ENABLE_VIDEO                            ON
    --  ENABLE_WAYLAND_TARGET ................. ON
    --  ENABLE_WEBDRIVER                        ON
    --  ENABLE_WEB_AUDIO ...................... ON
    --  ENABLE_WEB_CRYPTO                       ON
    --  ENABLE_X11_TARGET ..................... ON
    --  USE_LIBHYPHEN                           ON
    --  USE_LIBNOTIFY ......................... ON
    --  USE_LIBSECRET                           ON
    --  USE_SYSTEM_MALLOC ..................... OFF
    --  USE_WOFF2                               ON

    And, asides from the exceptions noted above, those are probably the options you want to ship with.

    Why are some things disabled by default? ENABLE_ACCELERATED_2D_CANVAS is OFF by default because it is experimental (i.e. not great :) and requires CairoGL, which has been available in most distributions for about half a decade now, but still hasn’t reached Debian yet, because the Debian developers know that the Cairo developers consider CarioGL experimental (i.e. not great!). Many of our developers use Debian, and we’re not keen on having two separate sets of canvas bugs depending on whether you’re using Debian or not, so best keep this off for now. ENABLE_GLES2 switches you from desktop GL to GLES, which is maybe needed for embedded systems with crap proprietary graphics drivers, but certainly not what you want when building for a general-purpose distribution with mesa. Then ENABLE_QUARTZ_TARGET is for building on macOS, not for Linux. And then we come to USE_SYSTEM_MALLOC.

    USE_SYSTEM_MALLOC disables WebKit’s bmalloc memory allocator (“fast malloc”) in favor of glibc malloc. bmalloc is performance-optimized for macOS, and I’m uncertain how its performance compares to glibc malloc on Linux. Doesn’t matter really, because bmalloc contains important heap security features that will be disabled if you switch to glibc malloc, and that’s all you need to know to decide which one to use. If you disable bmalloc, you lose the Gigacage, isolated heaps, heap subspaces, etc. I don’t pretend to understand how any of those things work, so I’ll just refer you to this explanation by Sam Brown, who sounds like he knows what he’s talking about. The point is that, if an attacker has found a memory vulnerability in WebKit, these heap security features make it much harder to exploit and take control of users’ computers, and you don’t want them turned off.

    USE_SYSTEM_MALLOC is currently enabled (bad!) in openSUSE and SUSE Linux Enterprise 15, presumably because when the Gigacage was originally introduced, it crashed immediately for users who set address space (virtual memory allocation) limits. Gigacage works by allocating a huge address space to reduce the chances that an attacker can find pointers within that space, similar to ASLR, so limiting the size of the address space prevents Gigacage from working. At first we thought it made more sense to crash than to allow a security feature to silently fail, but we got a bunch of complaints from users who use ulimit to limit the address space used by processes, and also from users who disable overcommit (which is required for Gigacage to allocate ludicrous amounts of address space), and so nowadays we just silently disable Gigacage instead if enough address space for it cannot be allocated. So hopefully there’s no longer any reason to disable this important security feature at build time! Distributions should be building with the default USE_SYSTEM_MALLOC=OFF.

    The openSUSE CMake line currently looks like this:

    %cmake \
      -DCMAKE_BUILD_TYPE=Release \
      -DLIBEXEC_INSTALL_DIR=%{_libexecdir}/libwebkit2gtk%{_wk2sover} \
      -DPORT=GTK \
    %if 0%{?suse_version} == 1315
      -DCMAKE_C_COMPILER=gcc-7 \
      -DCMAKE_CXX_COMPILER=g++-7 \
      -DENABLE_WEB_CRYPTO=OFF \
      -DUSE_GSTREAMER_GL=false \
    %endif
    %if 0%{?suse_version} <= 1500
      -DUSE_WOFF2=false \
    %endif
      -DENABLE_MINIBROWSER=ON \
    %if %{with python3}
      -DPYTHON_EXECUTABLE=%{_bindir}/python3 \
    %endif
    %if !0%{?is_opensuse}
      -DENABLE_PLUGIN_PROCESS_GTK2=OFF \
    %endif
    %ifarch armv6hl ppc ppc64 ppc64le riscv64 s390 s390x
      -DENABLE_JIT=OFF \
    %endif
      -DUSE_SYSTEM_MALLOC=ON \
      -DCMAKE_EXE_LINKER_FLAGS="-Wl,--as-needed -Wl,-z,now -pthread" \
      -DCMAKE_MODULE_LINKER_FLAGS="-Wl,--as-needed -Wl,-z,now -pthread" \
      -DCMAKE_SHARED_LINKER_FLAGS="-Wl,--as-needed -Wl,-z,now -pthread"

    which all looks pretty reasonable to me: certain features that require “newer” dependencies are disabled on the old distros, and NPAPI plugins are not supported in the enterprise distro, and JIT doesn’t work on odd architectures. I would remove the ENABLE_JIT=OFF lines only because WebKit’s build system should be smart enough nowadays to disable it automatically to save you the trouble of thinking about which architectures the JIT works on. And I would also remove the -DUSE_SYSTEM_MALLOC=ON line to ensure users are properly protected.

    By Michael Catanzaro at November 03, 2018 03:29 AM

    October 24, 2018

    Release Notes for Safari Technology Preview 68

    Surfin’ Safari

    Safari Technology Preview Release 68 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 236793-237231.

    Web API

    • Enabled <input type="color"> support (r236942)
    • Fixed the Document and Window objects to lose their browsing context as soon as its iframe is removed from the document (r236862)
    • Fixed incorrect input.checked when parsing its children (r236795)
    • Fixed window.navigator and window.performance to not become null after the window loses its browsing context (r237185, r237209)
    • Changed to restrict browsing context lookup by name to frames that are related to one another (r237112)
    • Changed anchor target to be ignored on activation when the download attribute is set (r236987)
    • Changed target="_blank" on anchors to imply rel="noopener" (r237144)
    • Fixed incorrect garbage collection of JavaScript node wrappers held by MutationObserver and MutationRecord (r236850, r236801)

    WebRTC

    • Added VP8 support to WebRTC (r236821)
    • Added support for IceCandidate stats (r236963)
    • Added support for reporting “display composited video frames” through the VideoPlaybackQuality object (r236875)
    • Added support for RTCPeerConnection.generateCertificate (r237140)
    • Added support for RTCConfiguration.certificates (r237202)
    • Implemented error handler of MediaRecorder (r237106)

    Media

    • Added support for inline WebVTT styles (r237187)
    • Changed to use nanoseconds as MaximumTimeScale (r237208)
    • Changed to only report the supported WebGL version (r237018)

    CSS

    • Added prefers-color-scheme media query support for styling dark mode content (r237156)
    • Changed to resolve inset properties to computed style when there is over-constrainment (r236979)

    Rendering

    • Fixed slow tiling for CSS gradients (r237230)

    JavaScript

    • Added BigInt support for the bitwise | operator (r236901)

    Web Inspector

    • Added the ability to go directly from an event in the overview view to the specialized timeline for that event (r237195)
    • Added support for showing redirect requests in the Network and Timelines tabs (r236995)
    • Added table support for multiple selection and ⌘ Click (Command-Click) behavior (r236853)
    • Changed to use the iframe name attribute for FrameTreeElement (r236885)
    • Changed to allow multiple canvases to be recorded at the same time (r236952)
    • Created a special Network waterfall for media events (r237028)
    • Exposed Server Timing response headers in the Network tab (r237151)
    • Fixed the Canvas recording sidebar scroll position after switching tabs (r237196)
    • Fixed the color contrast of the disabled Record button in the Canvas tab (r236986)
    • Fixed capturing previously saved states and adding them to the recording payload in the Canvas tab (r237198)
    • Fixed previews to be removed when the parent view is hidden in the Canvas tab (r237090)
    • Fixed clicking the initiator link in the Network tab table to automatically switch to the Preview section (r236923)
    • Fixed dark mode contrast issues (r236953, r237085, r237150)
    • Fixed style editor warnings to not look like errors in dark mode (r237125)
    • Fixed unreadable text when hovering over CSS properties while holding the ⌘ (Command) key in dark mode (r237143)
    • Fixed the “goto arrow” color for the selected DOM breakpoint tree element in dark mode (r237078)
    • Fixed the detail view to be correctly shown after sorting the Network table (r237043)
    • Fixed the detail view reverting to “Response” when new requests are added in the Network tab (r237061)
    • Fixed the Open Resource dialog to show the path to the resource to disambiguate resources with the same name (r236918)
    • Fixed the toolbar getting hidden when Web Inspector is docked to side (r237131)
    • Fixed ⌃G (Control-G) to not wipe the line when jumping to the line in a CSS file (r237212)
    • Grouped media network entries by the node that triggered the request (r236927)
    • Indented all network entries when “Group by Node” is enabled (r237006)

    WebDriver

    • Fixed a bug where some key combinations such as ⌘A (Command-A) can cause the WebDriver session to hang (r236939)
    • Fixed a crash when a WebDriver session is terminated while waiting for simulated inputs to be handled (r236852)
    • Fixed a hang when creating a WebDriver session for Safari Technology Preview (macOS Mojave only).

    Payment Request

    • Changed to abort requests after details settle when the user cancels (r236922)

    Apple Pay

    • Fixed new shipping methods getting ignored when updating after the shippingaddresschange event (r237142)
    • Changed payment authorization results with ApplePayErrors to never be considered final (r237134)

    Web Animations

    • Fixed setting animation-name:none after a fill:forwards animation has completed to correctly revert to the unanimated style (r236809)

    WebCrypto

    • Changed ECDSA to be able to handle invalid signature inputs (r236820)

    October 24, 2018 05:00 PM

    October 21, 2018

    Manuel Rego: Igalia at TPAC 2018

    Igalia WebKit

    Just a quick update before boarding to Lyon for TPAC 2018. This year 12 igalians will be at TPAC, 10 employees (Álex García Castro, Daniel Ehrenberg, Javier Fernández, Joanmarie Diggs, Martin Robinson, Rob Buis, Sergio Villar, Thibault Saunier and myself) and 2 coding experience students (Oriol Brufau and Sven Sauleau). We will represent Igalia in the different working groups and breakout sessions.

    On top of that Igalia will have a booth in the solutions showcase where we’ll have a few demos of our last developments like: WebRTC, MSE, CSS Grid Layout, CSS Box Alignment, MathML, etc. Showing them in some low-end boards like the Raspebrry Pi using WPE an optimized WebKit port for embedded platforms.

    Thread by W3C Developers announcing my talk.

    In my personal case I’ll be attending the CSS Working Group (CSSWG) and Houdini Task Force meetings to follow the work Igalia has been doing on the implementation of different standards. In addition, I’ll be giving a talk about how to contribute to the evolution of CSS on the W3C Developer Meetup that happens on Monday. I’ll try to explain how easy is nowadays to provide feedback to the CSSWG and have some influence on the different specifications.

    Tweet by Daniel Ehrenberg about the Web Platform position.

    Last but not least, Igalia Web Platform Team is hiring, we’re looking for people willing to work on web standards from the implementation on the different browser engines, to the discussions with the standard bodies or the definition of test suites. If you’re attending TPAC and you want to work on a flat company focused on free software development, probably you are a good candidate to join us. Read the position announcement and don’t hesitate to talk to any of us there about that.

    See you at TPAC tomorrow!

    October 21, 2018 07:00 AM

    October 15, 2018

    Canvas Debugging

    Surfin’ Safari

    Ever since its introduction in 2004, the Canvas element and its associated APIs have been an incredible tool for generating specific visual effects on a webpage. As more technologies, such as WebGL, came into being, Canvas’ complexity grew, and more was expected of the developer. For many years, debugging Canvas issues was a process of adding console.log statements, or even overriding the prototype of the Canvas’ context to better understand when functions are being called. Today, we’re excited to tell you about our improvements to the debugging process for Canvas, specifically the ways in which we are providing in-depth insight into the cause and effect of every action, as well as exposing in-process information that was previously invisible.

    Canvas Tab

    Web Inspector now has a dedicated Canvas tab for debugging canvas contexts. This new tool makes it easier to diagnose performance and correctness problems in canvas drawing code without having to manually instrumenting your code. The rest of this post explains these new capabilities.

    Visualizing Canvases

    The Canvas tab overview shows all active canvas contexts, listing the type of drawing context, preview image, dimensions, and current memory usage for each canvas in the page.

    Canvas Tab

    At any point, clicking on the refresh button in the top right of either the “card” or the Canvas tab will refresh the preview image(s) so that you can stay up to date on what you are looking at from the page.

    Clicking on any Canvas will “drill down” into a more specific view, where any attached objects, such as Shader Programs, can be viewed and edited.

    Canvas View

    Recording and Replaying Canvases

    A big part of the Canvas debugging initiative was to provide a near-native exploration of exactly what happens to the Canvas. It’s often difficult to debug drawing code because so much of it occurs behind the scenes, as only a few actions actually cause changes (if any) to be drawn. 2D and WebGL canvases are now able to be recorded and visualized action-by-action as needed to debug any issues. Clicking the red circle next to the canvas preview (overview) or in the left sidebar (specific canvas) will start a recording. When you’ve reproduced the behavior you want to debug, click the button again to stop recording. It is also possible to take single-frame recordings by holding Shift when clicking. After some processing, the recording will become available for inspection.

    Recording View

    The recording view shows a list of all commands (left sidebar), the current state and backtrace (right sidebar), and the current output up to the selected command (center). Clicking on any action will apply it and everything prior to the preview, generating the same result as the one viewed in the actual page. Moving the slider will quickly jump between “visual” actions (highlighted in the left sidebar), which are those that can cause a visible change in the generated content.

    Debugging 2D Paths

    In the 2D Canvas world, there exists a concept called “path”. To give an analogy, imagine a painter using a pencil to sketch out a brush stroke before actually painting. For 2D Canvas, the “path” is this penciled line that can be used to draw more expressive shapes all in one go. Normally, like the pencil, this path is invisible until actually drawn/painted. This can lead to unintended results if the developer put the wrong value by accident.

    Recording Path

    Dashed lines represent any “move” commands. The most recent path action is drawn in red.

    Export and Import

    One problem we envisioned while building Canvas Recording was that a recording is only as good as what it was able to capture. If a developer encounters a bug on their computer, but their coworker doesn’t have an issue, then there’s no real benefit to being able to record the buggy Canvas, especially if they don’t know how the code works. Our solution is to make Canvas Recording fully portable by allowing any recording to be exported to a JSON file and imported into any other computer.

    All of the recording information is encoded into the JSON file, so loading it into any other computer will provide the exact same replay accuracy as on the origin computer. In order to minimize the size of a Canvas Recording, however, values are deduplicated and repeated keys are removed in favor of more compact arrays. According to our rough estimates, this decreased the average JSON size by about 30%.

    Shader Program Editor

    For more advanced Canvas usage, such as WebGL, it is often the case that much of the work is done in a Shader. In these cases, recording the Canvas might not be enough to determine the cause of a potential problem, as it might be due to an error in one of the Shaders used by the Canvas. In order to facilitate this, we have exposed all valid Shader Programs for the selected Canvas.

    The original source of the Shader Program is shown in a split view for the Vertex and Fragment shaders. Each of these is editable, and will have an immediate effect on the inspected page.

    In this way, bugs can be ironed out without having to edit the source of the page, or having to take additional recordings.

    Playground

    As a simple demo for debugging using Canvas Recording, this page is the one displayed in the screenshots above. It heavily utilizes canvas 2D paths, often times outside the bounds of the canvas. This is a good example of a time to optimize, as there’s no reason to draw something outside the visible space. Thanks to Canvas Recording, this issue is very apparent and has enough information to make a fix actionable.

    Feedback

    The Canvas tab is available in macOS High Sierra 10.13.4 and macOS Mojave. Is there a Canvas API that wasn’t covered here that you debug often? Let us know! Please feel free to send us feedback on Twitter (@jonathandavis) or by filing a bug.

    October 15, 2018 05:00 PM

    Deprecation of Legacy TLS 1.0 and 1.1 Versions

    Surfin’ Safari

    Transport Layer Security (TLS) is a critical security protocol used to protect web traffic. It provides confidentiality and integrity of data in transit between clients and servers exchanging (often sensitive) information. To best safeguard this data, it is important to use modern and more secure versions of this protocol. Specifically, applications should move away from TLS 1.0 and 1.1. Doing so provides many benefits, including:

    • Modern cryptographic cipher suites and algorithms with desirable performance and security properties, e.g., perfect forward secrecy and authenticated encryption, that are not vulnerable to attacks such as BEAST.
    • Removal of mandatory and insecure SHA-1 and MD5 hash functions as part of peer authentication.
    • Resistance to downgrade-related attacks such as LogJam and FREAK.

    Now is the time to make this transition. Properly configured for App Transport Security (ATS) compliance, TLS 1.2 offers security fit for the modern web. It is the standard on Apple platforms and represents 99.6% of TLS connections made from Safari. TLS 1.0 and 1.1 — which date back to 1999 — account for less than 0.36% of all connections. With the recent finalization of TLS 1.3 by the IETF in August 2018, the proportion of legacy TLS connections will likely drop even further. TLS 1.2 is also required for HTTP/2, which delivers significant performance improvements for the web.

    Therefore, we are deprecating support for TLS 1.0 and 1.1. Complete support will be removed from Safari in updates to Apple iOS and macOS beginning in March 2020. Firefox, Chrome, and Edge are also planning to drop TLS 1.0 and 1.1 support at that time. If you own or operate a web server that does not support TLS 1.2 or newer, please upgrade now. If you use legacy services or devices that cannot be upgraded, please let us know by contacting our Web Technologies Evangelist or by filing a bug report with details.

    October 15, 2018 04:00 PM

    October 12, 2018

    MotionMark 1.1

    Surfin’ Safari

    Two years ago, we released MotionMark, a benchmark for Web graphics. It runs a series of tests that exercise common ways of painting on the Web. We’ve used this benchmark to improve the performance of WebKit and underlying frameworks, and we’re happy to see that others have used the benchmark for their own performance investigations.

    Today we are announcing an update to the MotionMark benchmark, which we’re calling version 1.1.

    Test updates

    First, let’s do a quick review about the benchmark. MotionMark runs a suite of tests that measures a browser’s painting performance by exercising various techniques, including CSS, text, canvas, and SVG. Each test packages a small set of graphics primitives into a single rendering element, and renders as many of them as possible while maintaining 60 fps. That number represents the test’s score. For more details, you can read our introductory blog post.

    We want to expose MotionMark to as many aspects of the rendering engine as possible. In our first update to the benchmark, we’ve refreshed two tests in the suite.

    The presentation of the Multiply test, which utilizes DOM element painting, caps the maximum number of elements that can be drawn. MotionMark requires tests to be able to stress browsers such that they cannot maintain 60 fps, and some of Apple’s new hardware, like the iPhone Xs, were powerful enough to animate Multiply at the maximum complexity at 60 fps. So we had to make the test harder! We reduced the physical size of each element to fit more of them on the available space and increase that cap threefold. We also updated the ways that an element can be hidden during the test. Before the elements toggled their visibility using visibility: hidden; now they also do so via display: none and opacity: 0.

    In the original 1.0 suite, each rendering element in the Leaves test was an image to which translate and rotate transforms were applied. For 1.1 we use the width and height attributes to paint the images in a range of different sizes, and test blending by animating a fade using opacity.

    Leveraging MotionMark’s design for easier analysis

    MotionMark has enabled us to monitor our graphics performance in WebKit and underlying system frameworks we depend on like CoreAnimation, as well as find opportunities to progress performance. As our experience in regression analysis with this benchmark grew, we found a couple of processes that facilitated our efforts to isolate bottlenecks and verify performance gains.

    First, we split tests from the “main” suite into derivative versions based on specific techniques. For example, with the Multiply test update mentioned earlier, we created versions where all of the elements used only one of the CSS techniques for hiding an element. For the Design test we created a version that only contained Latin text, one that used East Asian characters, and one that used right-to-left and complex scripts. Isolating the techniques made it easier to determine which part of the test WebKit regressed, and magnified problems in traces for easier analysis.

    Second, we added a way to run a test more simply, with a fixed number of elements. The benchmark uses a ramping controller that varies the painting load such that it repeatedly crosses over the threshold where the browser can no longer render the scene at 60 fps. Analyzing Instruments traces for such a run can be difficult to track a regression, however, since the load is dynamic from frame to frame. MotionMark’s design allows us to plug in different kinds of controllers, and using one that simply rendered a scene at a static complexity facilitated A/B analyses across versions of WebKit and the OS. Subsequent fixes could then be easily verified using both the fixed and ramping controllers.

    Results

    Using MotionMark as a tool for tracking regressions and discovering ways to progress have been critical for us to improve graphics performance in WebKit. As a result, we’ve made Safari on macOS Mojave and iOS 12 on average over 20% faster on MotionMark 1.1 than in the previous release.

    MotionMark 1.1 Results

    In conclusion, MotionMark has been updated to version 1.1 to scale better on faster hardware and include more techniques for painting. We hope this benchmark continues to be a useful tool for browsers to optimize their engines for the modern Web.

    October 12, 2018 05:00 PM

    October 10, 2018

    Viewing Augmented Reality Assets in Safari for iOS

    Surfin’ Safari

    Safari on iOS 12 supports viewing 3D models and allows you to see them in Augmented Reality (AR). Supported assets use the Universal Scene Description format, or USDZ, developed by Pixar.

    Apple has made a gallery of examples for you to play with. This post will tell you how to author Web content to reference USDZ files so your users can experience AR.

    Serving USDZ content

    For Safari to recognize AR content it must be served over HTTP with the appropriate MIME-type. Safari is looking for model/vnd.pixar.usd.

    Since this is a new format, it’s unlikely that your Web server knows about USDZ. You may need to configure it to serve the appropriate header. For Apache, the configuration would look something like this:

    AddType model/vnd.pixar.usd usdz # All files ending in .usdz served as USD.
    

    Refer to your Web server documentation, or your Web Application Framework documentation, for information on how to set the Content-Type header.

    Note that the official MIME-type for USDZ was recently registered as model/vnd.usdz+zip. At the moment Safari does not recognize that type, but this post will be updated when it does.

    Linking to USDZ

    Once the content is served with the correct MIME-type, you can link to USDZ content in the normal manner.

    <a href="heart-tapback.usdz">iMessage Heart Tap-back</a>
    

    Here is such a link: iMessage Heart Tap-back

    Upon tapping that link, Safari on iOS will navigate to a page that shows a static thumbnail of the 3D asset. Tapping again on the thumbnail will open a live view where the user can pan and zoom, and switch into AR mode. Any animations embedded in the USDZ will play in the live view. The user can exit that view and then return to the previous page using the back button.

    In this way, linking to a USDZ file is just like linking to other files.

    In-place viewing of USDZ

    Navigating away from a page to get an AR experience might be disruptive to your interaction. There is a better way to show AR content by marking-up the link element in such a way that Safari knows in advance how the link should be processed.

    By adding rel="ar" to your a (link/anchor) element, Safari won’t navigate on tap. Instead it will jump directly into the live 3D view and, upon exit, returns to your page.

    Safari decorates such links with a 3D-cube badge in the top-right corner, to show the user there is an AR experience. You may have noticed this experience on the gallery mentioned earlier.
    There are some extra requirements for the link. We already mentioned the rel attribute. The link must also contain a single child that is either an img or picture element. For example:

    <a rel="ar" href="model.usdz">
        <img src="model-preview.jpg">
    </a>
    

    Obviously you can use whatever image you like inside the anchor.

    Feature Detection

    To detect support for AR, you can use the following JavaScript:

    const a = document.createElement("a");
    if (a.relList.supports("ar")) {
      // AR is available.
    }
    

    Other than Safari, the AR integration is available in SFSafariViewController clients. We’ve also received requests to add it to WKWebView.

    More information on iOS and AR can be found in WWDC 2018 Session 603 – Integrating Apps and Content with AR Quick Look. The section relevant to Safari begins about 16 minutes in. Other parts of that session discuss creating USDZ files. General documentation on Apple’s AR frameworks is available from the developer site.

    October 10, 2018 06:00 PM

    Release Notes for Safari Technology Preview 67

    Surfin’ Safari

    Safari Technology Preview Release 67 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 236195-236793.

    Pointer Events

    • Added PointerEvent in the Experimental Features menu (r236378)
    • Ensured PointerEvent is not visible when disabled (r236410)

    WebRTC

    • Added RTCCodecStats support (r236346)
    • Changed to allow IP mismatch for local connections on localhost (r236316)
    • Cleaned up change handling for RealtimeMediaSource settings (r236646)
    • Fixed video track clone to preserve original property (r236536)
    • Implemented sender and receiver getStats (r236207)
    • Updated constraints supported by getDisplayMedia (r236476)

    Media

    • Enabled H.264 Simulcast (r236792)
    • Enabled Modern Encrypted Media API by default in the Experimental Features menu (r236281)
    • Added WebM sanitization in EME (r236256, r236307)
    • Introduced the concept of CDMInstanceSession for EME (r236317)
    • Changed to use some tolerance when deciding whether a frame should be appended to the decode queue in MSE (r236258)
    • Fixed a comparison with an uninitialized greatestDecodeDuration in MSE (r236314)
    • Fixed an unwanted erased sample from the decode queue in MSE (r236566)
    • Fixed WebVTT cue alignment (r236531)
    • Updated the WebVTT parser according to new region syntax (r236265)

    CSS

    • Fixed item alignment next to collapsed tracks with gutters in CSS Grid (r236643)
    • Restricted the total combined size of backdrop filters (r236306)
    • Fixed CSS reference filter with feDisplacementMap buffer corruption on Retina displays (r236415)
    • Fixed updating feMorphology filter in CSS when the element moves (r236416)
    • Fixed a garbled rendering of an image when feConvolveMatrix is applied to it on a Retina display (r236412)
    • Started implementing CSS Custom Properties and Values (r236273)
    • Implemented initialValue support for CSS Custom Properties and Values API (r236379)
    • Made overflow:overlay a synonym for overflow:auto (r236341)
    • Fixed <form> in quirks mode to have margin-block-end:1em (r236673)

    JavaScript

    • Implemented BigInt support for bitwise & operation (r236637)
    • Optimized Array#indexOf in the C++ runtime (r236240)
    • Optimized Array#lastIndexOf in the C++ runtime (r236496)

    Web Animations

    • Fixed accelerated animations to get suspended (r236312, r236313)
    • Fixed endlessly calling DocumentTimeline::updateAnimations() (r236308)
    • Ensured renderers with accelerated animations have layers (r236501, r236670)

    Shadow DOM

    • Added the support for users to select content across shadow boundaries (r236519)
    • Added the support for copying content across shadow boundaries (r236785)

    Web API

    • Fixed custom elements in a reaction queue losing its JavaScript wrapper and becoming an HTMLUnknownElement (r236376)
    • Fixed elements enqueued in mutation observers losing its JavaScript wrappers (r236519)
    • Changed Image.proto to be a Function.prototype not an HTMLElement.prototype (r236769)
    • Fixed radio inputs and checkbox inputs to fire “click”, “input”, and “change” events in order when clicked (r236779)
    • Simplified authentication code (r236344)
    • Changed document.open() to not propagate URLs to non-fully active documents (r236550)
    • Changed document.open() to throw errors for cross-origin calls (r236613)
    • Changed to strip the fragment from the document URL during document.open() URL propagation (r236560)
    • Fixed no-op document.open() calls to ensure no side effects (r236433)
    • Dropped support for the cross-origin-window-policy header (r236623)

    Apple Pay

    • Added support for granular errors in PaymentDetailsUpdate (r236552)
    • Removed the “in-store” button type (r236615)

    Web Inspector

    • Fixed an issue that caused Web Inspector’s tabs to be hidden when docked to the side (r236411)
    • Fixed an issue that caused dock buttons to disappear when Web Inspector goes fullscreen (r236375)
    • The Debugger tab now has one unified section for all breakpoint types (r236540)
    • Fixed the light background on new watch expression popover in Dark Mode (r236532)
    • Fixed image resources without content getting shown when the Images folder is selected (r236596)
    • Removed the Legacy Style Editor (r236336)
    • Removed the Visual Style CSS details sidebar panel (r236298)
    • Reworked the Computed panel in the Styles sidebar panel to match the styles of the Rules panel (r236297)
    • Added an experimental setting for multi-property selection (r236706)
    • Changed to start editing property names and values on mouseup instead of mousedown in the Styles sidebar (r236780)

    Web Driver

    • Fixed an issue wherein sending a Tab key press could sometimes shift focus out of web content and hang the WebDriver session (r236774)
    • Fixed an issue that caused safaridriver to immediately abort on launch
    • Fixed an issue where the Perform Actions command failed if a keyboard input source specified a tick containing a “pause” value without a “key” value

    Storage

    • Fixed an WebSQL issue preventing the user from granting a quota increase if the JavaScript provides an expected usage value that is too low (r236348)

    October 10, 2018 05:00 PM

    October 09, 2018

    Manuel Rego: Web Engines Hackfest 2018

    Igalia WebKit

    One year more and a new edition of the Web Engines Hackfest was arranged by Igalia. This time it was the tenth edition, the first five ones using the WebKitGTK+ Hackfest name and another five editions with the new broader name Web Engines Hackfest. A group of igalians, including myself, have been organizing this event. It has been some busy days for us, but we hope everyone enjoyed it and had a great time during the hackfest.

    This was the biggest edition ever, we were 70 people from 15 different companies including Apple, Google and Mozilla (three of the main browser vendors). It seems the hackfest is getting more popular, several people attending are repeating in the next editions, so that shows they enjoy it. This is really awesome and we’re thrilled about the future of this event.

    Talks

    The presentations are not the main part of the event, but I think it’s worth to do a quick recap about the ones we had this year:

    • Behdad Esfahbod and Dominik Röttsches from Google talked about Variable Fonts and the implementation in Chromium. It’s always amazing to check the possibilities of this new technology.

    • Camille Lamy, Colin Blundell and Robert Kroeger from Google presented the Servicification effort in the Chromium project. Which is trying to modularize Chromium in smaller parts.

    • Žan Doberšek from Igalia gave an update on WPE WebKit. The port is now official and it’s used everyday in more and more low-end devices.

    • Thibault Saunier from Igalia complemented Žan’s presentation talking about the GStreamer based WebRTC implementation in WebKitGTK+ and WPE ports. Really cool to see WebRTC arriving to more browsers and web engines.

    • Antonio Gomes and Jeongeun Kim from Igalia explained the status of Chromium on Wayland and it’s way to become fully supported upstream. This work will help to use Chromium on embedded systems.

    • Youenn Fablet from Apple closed the event talking about Service Workers support on WebKit. This is a key technology for Progressive Web Apps (PWA) and is now available in all major browsers.

    The slides of the talks are available on the website and wiki. The videos will be published soon in our YouTube channel.

    Some pictures from Web Engines Hackfest 2018 Some pictures from Web Engines Hackfest 2018 (Flickr album)

    Other topics

    During the event there were breakout sessions about many different topics. In this section I’m going to talk about the ones I’m more interested on.

    • Web Platform Tests (WPT)

      This is a key topic to improve interoperability on the web platform. Simon Pieters started the session with an introduction to WPT just in case someone was not aware of the repository and how it works. For the rest of the session we discussed the status of WPT on the different browsers.

      Chromium and Firefox are doing an automatic two ways (import/export) synchronization process so the tests can be easily shared between both implementations. On the other side WebKit still has some kind of manual process over the table, neither import or export is totally automatic, there are some scripts that help with the process though.

      Apart from that, WPT is a first-class citizen in Chromium, and the encouraged way to do new developments. In Firefox it’s still not there, as the test suites are not run in all the possible configurations yet (but they’re getting there).

      Finally the WPT dashboard is showing results for the most recent unstable releases of the different browsers, which is really cool despite being somehow hidden on the UI: https://wpt.fyi/results/?label=experimental.

    • LayoutNG

      Christian Biesinger gave an introduction to LayoutNG project in Blink, where Google is rewriting Chromium’s layout engine. He showed the main ideas and concepts behind this effort and navigated the code showing some examples. According to Christian things are getting ready and LayoutNG could be shipping in the coming months for inline and block layout.

      On top of questions about LayoutNG, we briefly mentioned how other browsers are also trying to improve the layout code: Firefox with Servo layout and WebKit with Layout Formatting Context (LFC) aka Layout Reloaded. It seems quite clear that the current layout engines are getting to their limits and people are looking for new solutions.

    • Chromium downstream

      Several companies (Google included) have to maintain downstream forks Chromium with their own customizations to fit their particular use cases and hardware platforms.

      Colin Blundell was explaining how it was the process of maintaining the downstream version of Chrome for iOS. After trying many different strategies the best solution was rebasing their changes 2-3 times per day. That way the conflicts they had to deal with were much simpler to resolve, otherwise it was not possible for them to cope with all the upstream changes. Note that he mentioned that one (rotatory) full-time resource was required to perform this job in time.

      It was good to share the experiences of different companies that are facing very similar issues for this kind of work.

    Thank you very much

    Just to close this post, big thanks to all the people attending the event, without you the hackfest wouldn’t have any sense at all. People are key for this event where discussions and conversations are one of the main parts of it.

    Of course special acknowledgments to the speakers for the hard work they put on their lovely talks.

    Finally I couldn’t forget to thank the Web Engines Hackfest 2018 sponsors: Google and Igalia. Without their support this event won’t be possible.

    Web Engines Hackfest 2018 sponsors: Google and Igalia Web Engines Hackfest 2018 sponsors: Google and Igalia

    Looking forward for a new edition!

    October 09, 2018 10:00 PM

    September 26, 2018

    Release Notes for Safari Technology Preview 66

    Surfin’ Safari

    Safari Technology Preview Release 66 is now available for download for macOS Mojave and macOS High Sierra. If you already have Safari Technology Preview installed, you can update in the Software Update pane of System Preferences on macOS Mojave and from the Mac App Store’s Updates tab on macOS High Sierra. After updating to macOS Mojave, you may have to reinstall Safari Technology Preview.

    This release covers WebKit revisions 235669-236195.

    This release of Safari Technology Preview has a known issue where launching SafariDriver fails.

    Web Share API

    • Implemented the Web Share API (r235925)

    Conic Gradients

    • Added support for Conic Gradients (r235772)
    • Fixed angle interpretation for Conic Gradients (r235868)
    • Fixed color stop blending in Conic Gradients for stops past the first stop (r235999)

    WebRTC

    • Enabled WebRTC Unified Plan as an Experimental Feature (r236144)
    • Exposed RTCRtpSender.setParameters (r235714)
    • Introduced a backend for RTCRtpTransceiver (r235719)

    Shadow DOM

    • Fixed :first-child, :last-child, :nth-child, and :nth-of-type to work on shadow root’s children (r235917)
    • Fixed mouseenter and mouseleave events to get dispatched when there is a capturing event listener for a slot ancestor (r235865)

    CSS

    • Added support for double-position gradient color stop syntax (r236155)

    Web Inspector

    • Updated the bezier curve editor for Dark Mode (r235998)
    • Changed to generate CSSKeywordCompletions from backend values (r236091)
    • Fixed imported recordings that are unable to be viewed after navigation (r235937)
    • Fixed opening Web Inspector with a selected element that might immediately scroll that element off screen in the DOM Tree outline (r235996)
    • Changed to record actions performed on ImageBitmapRenderingContext (r236008)
    • Fixed Source View to scroll to show the line when a breakpoint is hit inside of a <script> in an HTML resource (r236028)
    • Fixed a Location popover triggering for a hidden Location column when clicking a row in the Script Events grid of the Timelines tab (r235997)

    Web API

    • Fixed XMLHttpRequest open() to throw a SYNTAX_ERR exception if the method is empty or the URL cannot be resolved (r235808)
    • Fixed overrideMimeType to not update the Content-Type header in the XMLHttpRequest response (r235844)

    Apple Pay

    • Updated to dispatch a paymentmethodchange event when the payment method changes (r235833)
    • Renamed the -apple-pay-button-type value checkout to check-out (r235754)

    JavaScript

    • Fixed Symbol.prototype.description to handle a null Symbol (r235712)

    Media

    • Added Media Capabilities API as an Experimental Feature (r235675)
    • Added support for HEVC codec types in Media Capabilities (r236094)
    • Fixed track.onmute not getting called for a remote MediaStreamTrack when its counter part track is removed from the peer connection (r236090)
    • Updated to include supported frame rates in video capture presets (r235760)
    • Simplified logic when changing RealtimeMediaSource settings (r235670)
    • Changed the middle value to center in WebVTT for consistency with CSS (r236143)

    Web Animations

    • Fixed a failure that occurred when interrupting an accelerated CSS transition on a composited element in-flight (r235843)
    • Fixed accelerated animations to correctly respect positive delay values (r235854, r236072)

    Web Assembly

    • Optimized JavaScript to Web Assembly call by removing Vector allocation (r235778)

    Password AutoFill

    • Automatically submit login forms when filling credentials with Password AutoFill
    • On MacBook Pro with Touch Bar, require authentication with Touch ID to fill passwords into web pages; this can be changed in Safari’s AutoFill preferences

    September 26, 2018 05:00 PM

    September 10, 2018

    Release Notes for Safari Technology Preview 65

    Surfin’ Safari

    Safari Technology Preview Release 65 is now available for download for macOS High Sierra and the beta of macOS Mojave. If you already have Safari Technology Preview installed, you can update from the Mac App Store’s Updates tab on macOS High Sierra and in the Software Update pane of System Preferences on macOS Mojave. This release covers WebKit revisions 235171-235669.

    Storage Access API

    • Made a change to maintain access through same-site navigations of an iframe (r235569)
    • Updated the Storage Access API prompt to show the eTLD+1s, not the full host names (r235209)

    Media

    • Fixed scrubbing video with Touch Bar on YouTube to prevent video playback freezing (r235334)
    • Fixed muted elements not updating their Now Playing status when unmuted (r235474)

    Apple Pay

    • Added Web Inspector auto-complete keywords for -apple-pay-button-style and -apple-pay-button-type (r235406)
    • Changed to allow $0 totals (r235318)
    • Changed PaymentRequest.show() to reject when an unsupported ApplePayRequest version is specified (r235342)
    • Introduced Apple Pay JS v4 (r235251)
    • Introduced new values for -apple-pay-button-type (r235349)

    Payment Request

    • Added an onpayerdetailchange event handler to PaymentResponse (r235625)
    • Implemented the MerchantValidationEvent constructor (r235600)
    • Removed PaymentAddress.languageCode (r235607)

    Shadow DOM

    • Changed to compose click events from click() (r235337)
    • Fixed the focus navigation order in slot fallback contents (r235191)

    Fetch API

    • Made a change to stop checking the Request.integrity value in no-CORS mode (r235174)

    Service Workers

    • Fixed undefined self.isSecureContext in Service Workers (r235234)

    Web API

    • Added getModifierState to MouseEvent (r235329)

    Web Inspector

    • Added auto-completion for event breakpoints (r235389)
    • Added support for breakpoints for timers and animation-frame events (r235248)
    • Fixed deleting multiple event breakpoints by repeatedly hitting delete (r235183)
    • Fixed the color picker to allow entering decimal numbers for opacity (r235273)
    • Fixed typos in some compositing reasons (r235257)
    • Fixed the Search bar being too narrow in some localizations (r235447)
    • Fixed console.inspect(sessionStorage) to correctly show the Session Storage content view if the Storage tab was previously unvisited (r235242)
    • Fixed console.log() to show the passed string when a certain format is used (r235452)
    • Fixed the JSContext Inspector to correctly show Scripts in the Resources tab (r235226)

    WebGL 2

    • Updated WebGL 2 implementation to handle READ_FRAMEBUFFER and default framebuffer specification conformance (r235417)

    Web Assembly

    • Updated Web Assembly to parse wasm modules in a streaming fashion (r235420)

    Safari extensions

    • Messages from a Safari App Extension to its content script are now delivered to the correct page after a back or forward navigation

    September 10, 2018 08:00 PM

    August 29, 2018

    Release Notes for Safari Technology Preview 64

    Surfin’ Safari

    Safari Technology Preview Release 64 is now available for download for macOS High Sierra and the beta of macOS Mojave. If you already have Safari Technology Preview installed, you can update from the Mac App Store’s Updates tab on macOS High Sierra and in the Software Update pane of System Preferences on macOS Mojave. This release covers WebKit revisions 234684-235171.

    Custom Elements

    • Fixed the custom element constructor to use HTMLElement in new.target‘s realm (r234957)
    • Fixed custom element to invalidate its style when :defined rule starts to apply (r234953)
    • Changed invoking custom element reactions after constructing and inserting a custom element (r234893)
    • Changed to perform a microtask checkpoint before creating a custom element (r234944)

    Web API

    • Added the content-length header to the Fetch safe-list (r234840)
    • Disallowed navigations when page cache updates the current document of the frame (r235121)
    • Fixed navigator.sendBeacon to work in pagehide callbacks (r234684)
    • Fixed <object> elements that change from hidden to visible to properly display its content if its URL has a fragment identifier (r234762)

    Media

    • Added Experimental Feature support for SourceBuffer.changeType() (r234940)
    • Fixed an issue causing video playback to use more power (r234743)
    • Skipped validation of ideal constraints when selecting matching getUserMedia devices (r234949)

    JavaScript

    • Added support for a “name” option for dedicated workers (r235159)
    • Added support for unhandled promise rejections in a Worker (r234846)
    • Changed Date.UTC to not return NaN with only year parameter (r234763)
    • Changed Array.prototype.sort to throw a TypeError if the parameter is a not callable object (r234716)

    Storage Access API

    • Changed to use eTLD+1s, not full host names, when granting storage access (r235145)

    Web Inspector

    • Added support for breakpoints on arbitrary event names (r234974)
    • Added Initiator information in the Network table (r234963)
    • Allowed breakpoints to be set for specific event listeners (r235103)
    • Allowed recording processing to be stopped midway in the Canvas tab (r235095)
    • Created icons for recordings and shaders in the preview tile of the Canvas tab (r235093)
    • Fixed console.log to prevent firing getters for deep properties (r234780)
    • Fixed the contrast of the error and warning text widget for the source code text editor in Dark Mode (r234736)
    • Fixed XHR content sometimes showing as an error even though the load succeeded (r234702)
    • Fixed ⌘G to work as expected when the find banner is focused (r235147)
    • Provided a default icon for all actions in the Canvas tab (r235141)
    • Prevented tables from centering rows when scrolling them into view (r234882)

    WebDriver

    • Changed top not handle prompts that appear while running scripts (r234792)
    • Included all capabilities in the new session response (r234837)
    • Stopped trying to set the caret when focusing a non-text element in the send keys command (r234838)

    CSS

    • Implemented flow-relative margin, padding, border and sizing properties (r234798)
    • Updated the behavior of percentage row tracks and gutters in CSS Grid (r234687))

    WebGL

    • Fixed WebGL contexts to be updated when the display configuration is changed (r235125)

    Bug Fix

    • Fixed color wells to appear pressed when presenting a color picker (r234788)

    Safari Push Notifications

    • A user gesture, such as a mouse click, is now required to request the user’s permission to send Safari Push Notifications

    Safari Reader

    August 29, 2018 05:00 PM

    August 15, 2018

    Release Notes for Safari Technology Preview 63

    Surfin’ Safari

    Safari Technology Preview Release 63 is now available for download for macOS High Sierra and the beta of macOS Mojave. If you already have Safari Technology Preview installed, you can update from the Mac App Store’s Updates tab on macOS High Sierra and in the Software Update pane of System Preferences on macOS Mojave. This release covers WebKit revisions 234197-234684.

    Known Issue

    • Adobe Flash content does not load on macOS Mojave

    Dark Mode

    • Fixed the appearance of the Smart Search Field in Dark mode on macOS Mojave

    Stability

    • Fixed a Safari Technology Preview application crash that could happen when waking one’s Mac from sleep (r234674)
    • Fixed a Safari Technology Preview application crash that could happen when navigating between web pages (234462)

    Custom Elements

    • Fixed document.open and document.write to throw errors while the HTML parser is synchronously constructing a custom element (r234680)
    • Implemented customElements.upgrade() (r234507)
    • Fixed erroneous garbage collection for properties set on window.customElements (r234578, r234585)
    • Changed innerHTML to not synchronously create a custom element (r234577)
    • Made HTML parser immediately execute custom elements reactions after synchronously constructing a custom element (r234608)
    • Fixed the crash that occurs when a custom element reaction callback throws an exception (r234539)

    Media

    • Prevented creating composited layers for renderers unless they are part of the fullscreen subtree (r234291)
    • Fixed switching tabs to not close the picture-in-picture window (r234264)

    Web Animations

    • Fixed accelerated animations to respect a positive delay value (r234279)
    • Fixed transitions added immediately after element creation to work as expected (r234250)
    • Fixed animation stopping with object-fit:contain on an animated 2D canvas (r234343)

    Web API

    • Fixed navigator.sendBeacon to work in pagehide event callbacks (r234684)
    • Fixed setting the table layout to fixed causing incorrect cell width calculations (r234630)

    Accessibility

    • Fixed returning nothing for various previous text marker APIs from one <div> or <p> node to another (r234275)

    Web Inspector

    • Fixed inconsistent background color for the JavaScript Debugger breakpoint editor in Dark Mode (r234219)
    • Fixed the contrast for the disabled breakpoints banner in Dark Mode (r234464)
    • Fixed a white background for the Search sidebar panel text field in Dark Mode (r234394)
    • Fixed the source code text editor thread indicator widget contrast in Dark Mode (r234465)
    • Included a full URL tooltip when hovering the name in the Network Tab (r234431)

    Internationalization API

    • Changed the Internationalization API to ignore encoding when parsing BCP 47 language tag from an ISO 15897 locale string passed via LANG (r234260)

    WebGL 2

    • Added support for compilation of GLSL ES version 300 shaders (r234380)

    SVG

    • Changed to allow href attribute without xlink on SVG elements (r234683)

    August 15, 2018 05:00 PM

    August 09, 2018

    Manuel Rego: Changes on CSS Grid Layout in percentages and indefinite height

    Igalia WebKit

    This is a blog post about a change of behavior on CSS Grid Layout related to percentage row tracks and gutters in grid containers with indefinite height. Igalia has just implemented the change in Chromium and WebKit, which can affect some websites out there. So here I am going to explain several things about how percentages work in CSS and all the issues around it, of course I will also explain the change we are doing in Grid Layout and how to keep your previous behavior in the new version with very simple changes.

    Sorry for the length but I have been dealing with these issues since 2015 (probably earlier but that is the date of the first commit I found about this topic), and I went too deep explaining the concepts. Probably the post has some mistakes, this topic is not simple at all, but it represents a kind of brain dump of my knowledge about it.

    Percentages and definite sizes

    This is the easy part, if you have an element with fixed width and height resolving percentages on children dimensions is really simple, they are just computed against the width or height of the containing block.

    A simple example:

    <div style="width: 500px; height: 200px; border: dashed thick black;">
      <div style="width: 50%; height: 75%; background: magenta;"></div>
    </div>

    Example of percentage dimensions in a containing block with definite sizes Example of percentage dimensions in a containing block with definite sizes

    Things are a bit trickier for percentage margins and paddings. In inline direction (width in horizontal writing mode) they work as expected and are resolved against the inline size. However in block direction (height) they are not resolved against the block size (as one can initially expect) but against the inline size (width) of the containing block.

    Again a very simple example:

    <div style="width: 500px; height: 200px; border: dashed thick black;">
      <div style="margin-left: 10%; margin-top: 10%;
                  height: 150px; background: magenta;"></div>
    </div>

    Example of percentage margins in a containing block with definite sizes Example of percentage margins in a containing block with definite sizes

    Note that there is something more here, in both Flexbox and Grid Layout specifications it was stated in the past that percentage margins and paddings resolve against their corresponding dimension, for example inline margins against inline axis and block margins against block axis.

    This was implemented like that in Firefox and Edge, but Chromium and WebKit kept the usual behavior of resolving always against inline size. So for a while the spec had the possibility to resolve them in either way.

    This was a source of interoperability issues between the different browsers but finally the CSS Working Group (CSSWG) resolved to keep the behavior for regular blocks also for flex and grid items. And both Firefox and Edge modified their behavior and all browsers have the same output nowadays.

    Percentages and indefinite sizes

    First question is, what is an indefinite size? The simple answer is that a definite size is a size that you can calculate without taking into account the contents of the element. An indefinite size is the opposite, in order to compute it you need to check the contents first.

    But then, what happens when the containing block dimensions are indefinite? For example, a floated element has indefinite width (unless otherwise manually specified), a regular block has indefinite height by default (height: auto).

    For heights this is very simple, percentages are directly ignored so they have no effect on the element, they are treated as auto.

    For widths it starts to get funny. Web rendering engines have two phases to compute the width of an element. A first one to compute the minimum and maximum intrinsic width (basically the minimum and maximum width of its contents), and a second one to compute the final width for that box.

    So let’s use an example to explain this properly. Before getting into that, let me tell you that I am going to use Ahem font in some examples, as it makes very easy to know the size of the text and resolve the percentages accordingly, so if we use font: 50px/1 Ahem; we know that the size of an X character is a square of 50x50 pixels.

    <div style="float: left; font: 50px/1 Ahem;
                border: solid thick black; background: magenta; color: cyan;">
      XX XXXXX
    </div>

    Example of intrisic width without constraints Example of intrisic width without constraints

    The browser first calculates the intrinsic width, as minimum it computes 250px (the size of the smallest word, XXXXX in this case), as maximum size it would be 400px (the size of the whole text without line breaking XX XXXXX). So after this phase the browser knows that the element should have a width between 250px and 400px.

    Then during layout phase the browser will decide the final size, if there are no constraints imposed by the containing block it will use the maximum intrinsic width (400px in this case). But if you have a wrapper with a 300px width, the element will have to use 300px as width. If you have a wrapper smaller than the minimium intrinsic width, for example 100px, the element will still use the minimum 250px as its size. This is a quick and dirty explanation, but I hope it is useful to get the general idea.

    Example of intrisic width width different constraints Example of intrisic width with different constraints

    In order to resolve percentage widths (in the indefinite width situations) the browser does a different thing depending on the phase. During intrinsic size computations the percentage width is ignored (treated as auto like for the heights). But in the layout phase the width is resolved against the intrinsic size computed earlier.

    Trying to summarize the above paragraphs, we can say that somehow the width is only indefinite while the browser is computing the intrinsic width of the element, afterwards during the actual layout the width is considered definite and percentages are resolved against it.

    So now let’s see an example of indefinite dimensions and percentages:

    <div style="float: left;
                border: solid thick black; background: magenta;">
      <div style="width: 50%; height: 50%; background: cyan;">Hello world!</div>
    </div>

    Example of percentage dimensions in a containing block with indefinite sizes Example of percentage dimensions in a containing block with indefinite sizes

    First the size of the magenta box is calculated based on its contents, as it has not any constraint it uses the maximum intrinsic width (the length of Hello world!). Then as you can see the width of the cyan box is 50% of the text length, but the height is the same than if we use height: auto (the default value), so the 50% height is ignored.

    Back-compute percentages

    For margins and paddings things work more or less the same, remember that all of them are resolved against the inline direction (so they are ignored during intrinsic size computation and resolved later during layout).

    But there is something special about this too. Nowadays all the browsers have the same behavior but that was not always the case, not so long time ago (before Firefox 61 which was released past June) things worked different in Firefox than the rest of browsers

    Again let’s go to an example:

    <div style="float: left; font: 50px/1 Ahem;
                border: solid thick black; background: magenta;">
      <div style="margin-left: 50%; height: 100px;
                  background: cyan; color: blue;">XXXXX</div>
    </div>

    Example of percentage margins in a containing block with indefinite sizes Example of percentage margins in a containing block with indefinite sizes

    In this example the size of the magenta box (the floated div) is the width of the text, 250px in this case. Then the margin is 50% of that size (125px), making that the size of the cyan box gets reduced to 125px too, which causes overflow.

    But for these cases (percentage width margins and paddings and indefinite width container) Firefox did something extra that was called back-compute percentages. For that it something similar to the following formula:

    Intrinsic width / (1 - Sum of percentages)

    Which for this case would be 250px / (1 - 0.50) = 500px. So it takes as intrinsic size of the magenta box 500px, and then it resolves the 50% margin against it (250px). Thanks to this there is no overflow, and the margin is 50% of the containing block size.

    Example of old Firefox behavior back-computing percentage margins Example of old Firefox behavior back-computing percentage margins

    This Firefox behavior seems really smart and avoid overflows, but the CSSWG discussed about it and decided to use the other behavior. The main reason is what happens when you are around 100% percentages, or if you go over that value. The size of the box starts to be quite big (with 90% margin it would be 2500px), and when you go to 100% or over it you cannot use that formula so it considers the size as infinity (basically the viewport size in this example) and there is discontinuity in how percentages are resolved.

    So after that resolution Firefox changed their implementation and removed the back-computing percentages logic, thus we have now interoperability in how percentage margins and paddings are resolved.

    CSS Grid Layout and percentages

    And now we arrive to CSS Grid Layout and how to resolve percentages in two places: grid tracks and grid gutters.

    Of course when the grid container has definite dimensions there are no problems in resolving percentages against them, that is pretty simple.

    As usual the problem starts with indefinite sizes. Originally this was not a controversial topic, percentages for tracks were behaving similar to percentage for dimensions in regular blocks. A percentage column was treated as auto for intrinsic size computation and later resolved against that size during layout. For percentage rows they were treated as auto. It does not mean that this is very easy to understand (actually it took me a while), but once you get it, it is fine and not hard to implement.

    But when percentage support was added to grid gutters the big party started. Firefox was the first browser implementing them and they decided to use the back-compute technique explained in the previous point. Then when we add support in Chromium and WebKit we did something different than Firefox, we basically mimic the behavior of percentage tracks. As browsers started to diverge different discussions appear.

    One of the first agreements on the topic was that both percentage tracks and gutters should behave the same. That invalidated the back-computing approach, as it was not going to work fine for percentage tracks as they have contents. In addition it was finally discarded even for regular blocks, as commented earlier, so this was out of the discussion.

    However the debate moved to how percentage row tracks and gutters should be resolved, if similar to what we do for regular blocks or if similar to what we do for columns. The CSSWG decided they would like to keep CSS Grid Layout as symmetric as possible, so making row percentages resolve against the intrinsic height would achieve that goal

    So finally the CSSWG resolved to modify how percentage row tracks and gutters are resolved for grid containers with indefinite height. The two GitHub issues with the last discussions are: #509 and #1921.

    Let’s finish this point with a pair of examples to understand the change better comparing the previous and new behavior.

    Percentage tracks:

    <div style="display: inline-grid; border: solid thick;
                grid-template-columns: 75%; grid-template-rows: 50%;">
      <div style="background: magenta;">Testing</div>
    </div>

    Example of percentage tracks in a grid container with indefinite sizes Example of percentage tracks in a grid container with indefinite sizes

    Here the intrinsic size of the grid container is the width and height of the text Testing, and then the percentages tracks are resolved against that size for both columns and rows (before that was only done for columns).

    Percentage gutters:

    <div style="display: inline-grid; grid-gap: 10%; border: solid thick;
                grid-template-columns: 200px 200px; grid-template-rows: 100px 100px;">
      <div style="background: magenta;"></div>
      <div style="background: cyan;"></div>
      <div style="background: yellow;"></div>
      <div style="background: lime;"></div>
    </div>

    Example of percentage gutters in a grid container with indefinite sizes Example of percentage gutters in a grid container with indefinite sizes

    In this example we can see the same thing, with the new behavior both the percentage column and row gaps are resolved against the intrinsic size.

    Change behavior for indefinite height grid containers

    For a while all browsers were behaving the same (after Firefox dropped the back-computing approach) so changing this behavior would imply some kind of risks, as some websites might be affected by that and get broken.

    For that reason we added a use counter to track how many websites where hitting this situation, using percentage row tracks in a indefinite height grid container. The number is not very high, but there is an increasing trend as Grid Layout is being adopted (almost 1% of websites are using it today).

    And then Firefox changed the behavior for percentage row gutters to follow the new text on the spec, so they are resolved against the intrinsic height (this happened in version 62). However it did not change the behavior for percentage row tracks yet.

    This was a trigger to retake the topic and go deeper on it, after analyzing it carefully and crafting a prototype implementation we sent an intent to implement and ship to blink-dev mailing list.

    The intent was approved, but we were requested to analyze the sites that were hitting the use counter. After checking 178 websites only 8 got broken due to this change, we contacted them to try to get them fixed explaining how to keep the previous behavior (more about this in next point). You can find more details about this research in this mail.

    Apart from that we added a deprecation message in Chromium 69, so if you have a website that is affected by this (it does not mean that it has to get broken but that it uses percentage row tracks in a grid container with indefinite height) you will get the following warning in the JavaScript console:

    [Deprecation] Percentages row tracks and gutters for indefinite height grid containers will be resolved against the intrinsic height instead of being treated as auto and zero respectively. This change will happen in M70, around October 2018. See https://www.chromestatus.com/feature/6708326821789696 for more details.

    Finally this week the patch has been accepted and merged in master, so since Chromium 70.0.3516 (current Canary) you will have the new behavior. Apart from that we also make the fix in WebKit that will be hopefully part of the next Safari releases.

    In addition Firefox and Edge developers have been notified and we have shared the tests in WPT as usual, so hopefully those implementations will get updated soon too.

    Update your website

    Yes this change might affect your website or not, even if you get the deprecation warning it can be the case that your website is still working perfectly fine, but in some cases it can break quite badly. The good news is that the solution is really straightforward.

    If you find issues in your website and you want to keep the old behavior you just need to do the following for grid containers with indefinite height:

    • Change percentages in grid-template-rows or grid-auto-rows to auto.
    • Modify percentages in row-gap or grid-row-gap to 0.

    With those changes your website will keep behaving like before. In most cases you will realize that the percentages were unneeded and were not doing anything useful for you, even you would be able to drop the declaration completely.

    One of these cases would be websites that have grid containers with just one single row of 100% height (grid-template-rows: 100%), many of the sites hitting the use counter are like this. All these are not affected by this change, unless the have extra implicit rows, but the 100% is not really useful at all there, they can simply remove the declaration.

    Another sites that have issues are the ones that have for example two rows that sum up 100% in total (grid-template-rows: 25% 75%). These percentages were ignored before, so the contents always fit in each of the rows. Now the contents might not fit in each row and the results might not be the desired ones. Example:

    <div style="display: grid; grid-template-rows: 25% 75%; border: solid thick;">
      <div style="background: magenta;">First<br>two lines</div>
      <div style="background: cyan;">Second</div>
    </div>

    Example of overlapping rows in the new behavior Example of overlapping rows in the new behavior

    The sites that were more broken usually have several rows and used percentages only for a few of them or for all. And now the rows overflow the height of the grid container and they overlap other content on the website. There were cases like this example:

    <div style="display: grid; grid-template-rows: 50%; border: solid thick;">
      <div style="background: magenta;">First</div>
      <div style="background: cyan; height: 200px;">Second</div>
      <div style="background: yellow; height: 100px;">Third</div>
    </div>

    Example of overflowing rows in the new behavior Example of overflowing rows in the new behavior

    Closing

    This topic has been a kind of neverending story for the CSSWG, but finally it seems we are reaching to an end. Let’s hope this does not get any further and things get settle down after all this time. We hope that this change is the best solution for web authors and everyone will be happy with the final outcome.

    As usual I could not forget to highlight that all this work has been done by Igalia thanks to Bloomberg sponsorship as part of our ongoing collaboration.

    Igalia and Bloomberg working together to build a better web Igalia and Bloomberg working together to build a better web

    Thanks for reading that long, this ended up being much more verbose and covering more topics than originally planned. But I hope it can be useful to understand the whole thing. You can find all the examples from this blog post in this pen feel free to play with them.

    And to finish this blog post I could only do it by quoting fantasai:

    this is why I hate percentages in CSS

    I cannot agree more with her. 😇

    August 09, 2018 10:00 PM

    August 07, 2018

    Manuel Rego: CSS Logical Properties and Values in Chromium and WebKit

    Igalia WebKit

    Since the beginning of the web we have been used to deal with physical CSS properties for different features, for example we all know how to set a margin in an element using margin-left, margin-right, margin-top and/or margin-bottom. But with the appearance of CSS Writing Modes features, the concepts of left, right, top and bottom have somehow lost their meaning.

    Imagine that you have some right-to-left (RTL) content on your website your left might be probably the physical right, so if you are usually setting margin-left: 100px for some elements, you might want to replace that with margin-right: 100px. But what happens if you have mixed content left-to-right (LTR) and RTL at the same time, then you will need different CSS properties to set left or right depending on that. Similar issues are present if you think about vertical writing modes, maybe left for that content is the physical top or bottom.

    CSS Logical Properties and Values is a CSS specification that defines a set of logical (instead of physical) properties and values to prevent this kind of issues. So when you want to set that margin-left: 100px independently of the direction and writing mode of your content, you can directly use margin-inline-start: 100px that will be smart enough. Rachel Andrew has a nice blog post explaining deeply this specification and its relevance.

    Example of 'margin-inline-start: 100px' in different combinations of directions and writing modes Example of margin-inline-start: 100px in different combinations of directions and writing modes

    Oriol Brufau, an active collaborator on the CSS Working Group (CSSWG), has been doing a Igalia Coding Experience implementing support for CSS Logical Properties and Values in Blink and WebKit. Maybe you were already aware of this as my colleague Frédéric Wang already talked about it in his last blog post reviewing the activities of Igalia Web Platform team in the past semester.

    Some history

    Chromium and WebKit have had support since a long time ago for some of the CSS logical properties defined by the spec. But they were not using the standard names defined in the specification but some -webkit- prefixed ones with different names.

    For setting the dimensions of an element Chromium and WebKit have properties like -webkit-logical-width and -webkit-logical-height. However CSS Logical defines inline-size and block-size instead. There are also the equivalent ones for minimum and maximum sizes too. These ones have been already unprefixed at the beginning of 2017 and included in Chromium since version 57 (March 2017). In WebKit they are still only supported using the prefixed version.

    But there are more similar properties for margins, paddings and borders in Chromium and WebKit that use start and end for inline direction and before and after for block direction. In CSS Logical we have inline-start and inline-end for inline direction and block-start and block-end for block direction, which are much less confusing. There was an attempt in the past to unprefix these properties but the work was abandoned and never completed. These ones were still using the -webkit- prefix so we decided to tackle them as the first task.

    The post has been only talking about properties so far, but the same thing applies to some CSS values, that is why the spec is called CSS Logical Properties and Values. For example a very well-known property like float has the physical values left and right. The spec defines inline-start and inline-end as the logical values for float. However these were not supported yet in Chromium and WebKit, not even using -webkit- prefixes.

    Firefox used to have some -moz- prefixed properties, but since Firefox 41 (September 2015) it is shipping many of the standard logical properties and values. Firefox has been using these properties extensively in its own tests, thus having them supported in Chromium will make easier to share them.

    At the beginning of this work, Oriol wrote a document in which explaining the implementation plan where you can check the status of all these properties in Chromium and Firefox.

    Unprefix existent properties

    We originally send an intent to implement and ship for the whole spec, actually not all the spec but the parts that the CSSWG considered ready to implement. But Chromium community decided it was better to split it in two parts:

    The work on the first part, making the old -webkit- prefixed properties to use the new standard names, has been already completed by Oriol and it is going to be included in the upcoming release of Chromium 69.

    In addition to the Chromium work Oriol has just started to do this on WebKit too. Work is on early stages here but hopefully things will move forward in parallel to the Chromium stuff.

    Adding support for the rest

    Next step was to add support for the new stuff behind an experimental flag. This work is ongoing and you can check the current status in the latest Canary enabling the Experimental Web Platform features flag.

    So far Oriol has added support for a bunch of shorthands and the flow-relative offset properties. You can follow the work in issue #850004 in Chromium bug tracker.

    We will talk more about this in a future blog post once this task is completed and the new logical properties and values are shipped.

    Tests!

    Of course testing is a key part of all these tasks, and web-platform-tests (WPT) repository plays a fundamental role to ensure interoperability between the different implementations. Like we have been doing in Igalia lately in all our developments we used WPT as the primary place to store all the tests related to this work.

    Oriol has been creating tests in WPT to cover all these features. Initial tests were based in the ones already available in Firefox and modified them to adapt to the rest of stuff that needs to be checked.

    Note that in Chromium all the sideways writing modes test cases are failing as there is no support for sideways in Chromium yet.

    Plans for the future

    As explained before, this is an ongoing task but we already have some extra plans for it. These are some of the tasks (in no particular order) that we would like to do in the coming months:

    • Complete the implementation of CSS Logical Properties and Values in Chromium. This was explained in the previous point and is moving forward at a good pace.
    • Get rid of usage of -webkit- prefixed properties in Chromium source code. Oriol has also started this task and is currently work in progress.
    • Deprecate and remove the -webkit- prefixed properties. It still too early for that but we will keep an eye on the metrics and do it once usage has decreased.
    • Implement it in WebKit too, first by unprefixing the current properties (which has been already started) and later continuing with the new things. It would be really nice if WebKit follows Chromium on this. Edge also has plans to add support for this spec, so that would make logical properties and values available in all the major browsers.

    Wrap up

    Oriol has been doing a good job here as part of his Igalia Coding Experience. Apart from all the new stuff that is landing in Chromium, he has also been fixing some related bugs.

    We have just started the WebKit tasks, but we hope all this work can be part of future Chromium and Safari releases in the short term.

    And that is all for now, we will keep you posted! 😉

    August 07, 2018 10:00 PM

    August 05, 2018

    ITP Debug Mode in Safari Technology Preview 62

    Surfin’ Safari

    One of the more frequent questions we get from developers regarding Safari’s Intelligent Tracking Prevention (ITP) is how to debug your website with it enabled, especially since it is enabled by default in Safari. Today we’re happy to present you with an experimental version of ITP Debug Mode. Please try it out and send us feedback.

    Enable ITP Debug Mode

    This is how you enable ITP Debug Mode in Safari Technology Preview 62:

    1. Click the Safari Technology Preview menu → Preferences → Advanced
    2. Enable the Develop menu by clicking the checkbox next to “Show Develop menu in the menu bar”
    3. Click the Develop menu → Experimental Features → ITP Debug Mode

    Disable ITP Debug Mode After Use

    ITP Debug Mode should only be used when actively debugging your website for Intelligent Tracking Prevention behaviors. Remember to disable ITP Debug Mode when you’re done debugging. Why?

    For privacy reasons, ITP Debug Mode logs on the INFO level to ensure domain names are not written to persistent log files. If you leave ITP Debug Mode enabled, you run the risk of having privacy-sensitive domain names included in your logs when running some system diagnostic tools, like capturing a sysdiagnose.

    Domains Classified by Intelligent Tracking Prevention

    When launching Safari Technology Preview, ITP Debug Mode logs which domains ITP has classified as having cross-site tracking abilities. To see these logs, do the following:

    1. Quit Safari Technology Preview
    2. Filter ITP Debug Mode log messages using the Console app or Terminal app
      1. If you prefer the Console app:
        1. Launch the Console app
        2. Click the Action menu → Include Info Messages
        3. Filter on “ResourceLoadStatisticsDebug” without the quotes
      2. If you prefer Terminal:
    log stream -info | grep ResourceLoadStatisticsDebug
    
    1. Launch Safari Technology Preview

    You will now see which domains ITP has already classified as having tracking abilities in one or more log statements, such as:

    About to block cookies in third-party contexts for: 3rdpartytestwebkit.org.
    

    3rdpartytestwebkit.org is set up to always be classified as a tracker in ITP Debug Mode and you can use it to test functionality such as the Storage Access API.

    Depending on how much browsing you’ve done since starting fresh (see the “Starting Fresh” section below), you’ll see anything from just 3rdpartytestwebkit.org to hundreds of domain names listed in these log statements.

    Further Logging as You Browse the Web

    OK, so you’ve enabled ITP Debug Mode, set up Console or Terminal to capture the right logs, launched Safari Technology Preview, and seen the initial ITP Debug Mode logs. From this point you will see new log statements whenever Intelligent Tracking Prevention makes a new decision.

    Such as when it purges website data:

    About to remove data records for tracker.example, another-tracker.example.
    

    When it classifies a new domain as having tracking abilities:

    About to block cookies in third-party contexts for: a-third-tracker.example.
    

    Or, when it prompts the user as a result of a call to the Storage Access API:

    About to ask the user whether they want to grant storage access to 3rdpartytestwebkit.org under news.example or not.
    

    Classifying A Custom Domain For Testing

    One specific request we received from developers was for the ability to manually set a custom domain as permanently classified with tracking abilities. With ITP Debug Mode and User Defaults, that is now possible. Open Terminal and execute the following command (in this example to set example.com as permanently classified):

    defaults write com.apple.SafariTechnologyPreview ResourceLoadStatisticsManualPrevalentResource example.com
    

    When you launch Safari Technology Preview with ITP Debug Mode enabled, you will now see this added log statement in Console or Terminal:

    Did set example.com as prevalent resource for the purposes of ITP Debug Mode.
    

    You can inspect the setting in Terminal like this:

    defaults read com.apple.SafariTechnologyPreview ResourceLoadStatisticsManualPrevalentResource
    

    And you can delete the setting like this:

    defaults delete com.apple.SafariTechnologyPreview ResourceLoadStatisticsManualPrevalentResource
    

    Starting Fresh

    When you’re debugging under Intelligent Tracking Prevention, it’s often useful to start fresh. You do that by deleting all Safari Technology Preview history. This not only deletes history and website data, but also resets ITP. Clearing your history will also sync with other devices over iCloud. To avoid that, we recommend creating a new user on your macOS system to use exclusively for debugging Intelligent Tracking Prevention. Be careful though. If you’re investigating an issue that is the result of real life browsing, resetting ITP may prevent you from reproducing the issue, since all of ITP’s data is deleted and the real life browsing that got you into the state with the issue may not be something you can replicate.

    In ITP Debug Mode, 3rdpartytestwebkit.org and any custom domain you’ve set up through User Defaults stay classified even when you delete history. This behavior provides for controlled, repeatable testing.

    Feedback and Bug Reports

    Please report bugs about ITP Debug Mode using the WebKit bug tracker at bugs.webkit.org, and send feedback to our evangelist Jon Davis. If you have technical questions on debugging with ITP Debug Mode you can find me on Twitter @johnwilander.

    August 05, 2018 05:00 PM

    August 01, 2018

    Release Notes for Safari Technology Preview 62

    Surfin’ Safari

    Safari Technology Preview Release 62 is now available for download for macOS High Sierra and the beta of macOS Mojave. If you already have Safari Technology Preview installed, you can update from the Mac App Store’s Updates tab on macOS High Sierra and in the Software Update pane of System Preferences on macOS Mojave. This release covers WebKit revisions 233728-234197.

    Known Issues

    • This release of Safari Technology Preview for macOS Mojave betas does not render text properly in the Smart Search Field when in Dark Mode.
    • This release of Safari Technology Preview for macOS Mojave betas does not have a usable WebDriver implementation; safaridriver hangs when processing the New Session command.

    Intelligent Tracking Prevention

    • Added an experimental ITP Debug Mode to be used only when actively debugging, not to remain enabled (r234080, r234108)

    JavaScript

    • Changed to canonical language tags in the Internationalization API (r234127)
    • Fixed the Generator and AsyncGeneratorMethod prototype (r233855)
    • Fixed the JSON.stringify replacer to use isArray instead of JSArray checks (r233918)
    • Fixed the iterator of Array.keys() to return the object in the correct order (r233740)
    • Fixed JavaScript URL to provide the correct result when frame is navigated (r233793)
    • Changed JSON.stringify to emit properties included in the prototype chain for entries in the second array argument (r233924)

    Media

    • Changed to require the document to be visible for fullscreen video (r233865)
    • Disabled all network caching for HLS streams. (r233738)
    • Fixed the transition for the first Picture-in-Picture from Fullscreen (r234051)
    • Changed HLS resources with remote sub-resources to taint media elements (r234055)
    • Improved WebGL selection of GPU driving the current display (r234074)

    Rendering

    • Fixed newly added float elements to trigger full layout on the block (r233767)

    Dark Mode

    • Improved the visibility of spelling and grammar markers in Dark Mode (r233814)

    Web API

    • Limited editing selections to within Shadow DOM boundaries (r233778)

    Web Inspector

    SVG

    • Implemented support for margin-box as reference box and box shape (r233886)
    • Added full support for -webkit-clip-path on SVG elements (r233835, r234136)

    Web Animations

    • Changed to discrete interpolation between font-styles with a keyword value (r233935)
    • Changed to discrete interpolation between lengths with an auto value (r233892)

    CSS

    • Added support for calc() in CSS Grid gutter properties (r234131)

    IndexedDB

    • Fixed the ability to remove the database file of a sub-frame (r233777)

    Accessibility

    • Fixed pressing tab to highlight items on a webpage when voiceover enabled (r234112)

    August 01, 2018 05:00 PM

    July 21, 2018

    Michael Catanzaro: On Flatpak Nightlies

    Igalia WebKit

    Here’s a little timeline of some fun we had with the GNOME master Flatpak runtime last week:

    • Tuesday, July 10: a bad runtime build is published.  Trying to start any application results in error while loading shared libraries: libdw.so.1: cannot open shared object file: No such file or directory. Problem is the library is present in org.gnome.Sdk instead of org.gnome.Platform, where it is required.
    • Thursday, July 12:  the bug is reported on WebKit Bugzilla (since it broke Epiphany Technology Preview)
    • Saturday, July 14: having returned from GUADEC, I notice the bug report and bisect the issue to a particular runtime build. Mathieu Bridon fixes the issue in the freedesktop SDK and opens a merge request.
    • Monday, July 16: Mathieu’s fix is committed. We now have to wait until Tuesday for the next build.
    • Tuesday, Wednesday, and Thursday: we deal with various runtime build failures. Each day, we get a new build log and try to fix whatever build failure is reported. Then, we wait until the next day and see what the next failure is. (I’m not aware of any way to build the runtime locally. No doubt it’s possible somehow, but there are no instructions for doing so.)
    • Friday, July 20: we wait. The build has succeeded and the log indicates the build has been published, but it’s not yet available via flatpak update
    • Saturday, July 21: the successful build is now available. The problem is fixed.

    As far as I know, it was not possible to run any nightly applications during this two week period, except developer applications like Builder that depend on org.gnome.Sdk instead of the normal org.gnome.Platform. If you used Epiphany Technology Preview and wanted a functioning web browser, you had to run arcane commands to revert to the last good runtime version.

    This multi-week response time is fairly typical for us. We need to improve our workflow somehow. It would be nice to be able to immediately revert to the last good build once a problem has been identified, for instance.

    Meanwhile, even when the runtime is working fine, some apps have been broken for months without anyone noticing or caring. Perhaps it’s time for a rethink on how we handle nightly apps. It seems likely that only a few apps, like Builder and Epiphany, are actually being regularly used. The release team has some hazy future plans to take over responsibility for the nightly apps (but we have to take over the runtimes first, since those are more important), and we’ll need to somehow avoid these issues when we do so. Having some form of notifications for failed builds would be a good first step.

    P.S. To avoid any possible misunderstandings: the client-side Flatpak technology itself is very good. It’s only the server-side infrastructure that is problematic here. Clearly we have a lot to fix, but it won’t require any changes in Flatpak.

    By Michael Catanzaro at July 21, 2018 02:12 PM

    July 18, 2018

    Release Notes for Safari Technology Preview 61

    Surfin’ Safari

    Safari Technology Preview Release 61 is now available for download for macOS High Sierra and the beta of macOS Mojave. If you already have Safari Technology Preview installed, you can update from the Mac App Store’s Updates tab on macOS High Sierra and in the Software Update pane of System Preferences on macOS Mojave. This release covers WebKit revisions 233256-233728.

    CSS

    • Fixed -webkit-clip-path offset for clipPath references (r233287)
    • Turned CSS Animation Triggers off by default (r233265)
    • Updated clip-path box mapping to unified box (r233302)

    Dark Mode

    • Made the focus ring color honor the system accent color (r233315)
    • Prevented inverting text color for selections when not in dark mode (r233532)

    Web API

    • Enabled Link Preload by default in the Experimental Features menu (r233263)
    • Fixed cookie creation time exposed to Objective-C (r233308)
    • Fixed find in page to find low (German) quotes (r233345)
    • Fixed return values for DOMMatrix.invertSelf() when used on a non-invertible matrix (r233628)
    • Implemented support for Element.toggleAttribute (r233475)
    • Improved window.event compliance: Should not be set when target is in shadow tree (r233489)
    • Promoted the Secure Context API from an experimental feature to always on (r233264)
    • Updated the Element API to use qualifiedName to comply with standards (r233545)

    Service Workers

    • Made fetch() use “same-origin” credentials by default (r233720)
    • Fixed fetching several times in a row (r233719)

    Media

    • Disabled autoplay when the element is suspended (r233485)
    • Fixed video flicker which sometimes happened when playing to AppleTV (r233435, r233535)
    • Changed to reject getUserMedia promise if capture fails (r233425)

    WebRTC

    • Fixed delay and video lag caused by enabling and disabling a MediStreamTrack (r233604)

    Web Assembly

    • Stopped using tracePoints in JavaScript/WASM entry (r233378)

    Web Inspector

    • Fixed copy from Search results content view (r233334)
    • Fixed the “Open Link” context menu action (r233316)
    • Fixed the resource search field in dark mode (r233608)
    • Fixed the Debugger content view to properly update when the left sidebar is collapsed (r233686)
    • Enabled control-dragging to pan the 3D render in the Layers inspector (r233695)

    Accessibility

    • Added a subrole for meter elements on macOS (r233607)
    • Fixed setValue on text controls to send out key events (r233525, r233580)

    July 18, 2018 05:20 PM

    July 09, 2018

    Frédéric Wang: Review of Igalia's Web Platform activities (H1 2018)

    Igalia WebKit

    This is the semiyearly report to let people know a bit more about Igalia’s activities around the Web Platform, focusing on the activity of the first semester of year 2018.

    Projects

    Javascript

    Igalia has proposed and developed the specification for BigInt, enabling math on arbitrary-sized integers in JavaScript. Igalia has been developing implementations in SpiderMonkey and JSC, where core patches have landed. Chrome and Node.js shipped implementations of BigInt, and the proposal is at Stage 3 in TC39.

    Igalia is also continuing to develop several features for JavaScript classes, including class fields. We developed a prototype implementation of class fields in JSC. We have maintained Stage 3 in TC39 for our specification of class features, including static variants.

    We also participated to WebAssembly (now at First Public Working Draft) and internationalization features for new features such as Intl.RelativeTimeFormat (currently at Stage 3).

    Finally, we have written more tests for JS language features, performed maintenance and optimization and participated to other spec discussions at TC39. Among performance optimizations, we have contributed a significant optimization to Promise performance to V8.

    Accessibility

    Igalia has continued the standardization effort at the W3C. We are pleased to announce that the following milestones have been reached:

    A new charter for the ARIA WG as well as drafts for ARIA 1.2 and Core Accessibility API Mappings 1.2 are in preparation and are expected to be published this summer.

    On the development side, we implemented new ARIA features and fixed several bugs in WebKit and Gecko. We have refined platform-specific tools that are needed to automate accessibility Web Platform Tests (examine the accessibility tree, obtain information about accessible objects, listen for accessibility events, etc) and hope we will be able to integrate them in Web Platform Tests. Finally we continued maintenance of the Orca screen reader, in particular fixing some accessibility-event-flood issues in Caja and Nautilus that had significant impact on Orca users.

    Web Platform Predictability

    Thanks to support from Bloomberg, we were able to improve interoperability for various Editing/Selection use cases. For example when using backspace to delete text content just after a table (W3C issue) or deleting a list item inside a content cell.

    We were also pleased to continue our collaboration with the AMP project. They provide us a list of bugs and enhancement requests (mostly for the WebKit iOS port) with concrete use cases and repro cases. We check the status and plans in WebKit, do debugging/analysis and of course actually submit patches to address the issues. That’s not always easy (e.g. when it is touching proprietary code or requires to find some specific reviewers) but at least we make discussions move forward. The topics are very diverse, it can be about MessageChannel API, CSSOM View, CSS transitions, CSS animations, iOS frame scrolling custom elements or navigating special links and many others.

    In general, our projects are always a good opportunity to write new Web Platform Tests import them in WebKit/Chromium/Mozilla or improve the testing infrastructure. We have been able to work on tests for several specifications we work on.

    CSS

    Thanks to support from Bloomberg we’ve been pursuing our activities around CSS:

    We also got more involved in the CSS Working Group, in particular participating to the face-to-face meeting in Berlin and will attend TPAC’s meeting in October.

    WebKit

    We have also continued improving the web platform implementation of some Linux ports of WebKit (namely GTK and WPE). A lot of this work was possible thanks to the financial support of Metrological.

    Other activities

    Preparation of Web Engines Hackfest 2018

    Igalia has been organizing and hosting the Web Engines Hackfest since 2009, a three days event where Web Platform developers can meet, discuss and work together. We are still working on the list of invited, sponsors and talks but you can already save the date: It will happen from 1st to 3rd of October in A Coruña!

    New Igalians

    This semester, new developers have joined Igalia to pursue the Web platform effort:

    • Rob Buis, a Dutch developer currently living in Germany. He is a well-known member of the Chromium community and is currently helping on the web platform implementation in WebKit.

    • Qiuyi Zhang (Joyee), based in China is a prominent member of the Node.js community who is now also assisting our compilers team on V8 developments.

    • Dominik Infuer, an Austrian specialist in compilers and programming language implementation who is currently helping on our JSC effort.

    Coding Experience Programs

    Two students have started a coding experience program some weeks ago:

    • Oriol Brufau, a recent graduate in math from Spain who has been an active collaborator of the CSS Working Group and a contributor to the Mozilla project. He is working on the CSS Logical Properties and Values specification, implementing it in Chromium implementation.

    • Darshan Kadu, a computer science student from India, who contributed to GIMP and Blender. He is working on Web Platform Tests with focus on WebKit’s infrastructure and the GTK & WPE ports in particular.

    Additionally, Caio Lima is continuing his coding experience in Igalia and is among other things working on implementing BigInt in JSC.

    Conclusion

    Thank you for reading this blog post and we look forward to more work on the web platform this semester!

    July 09, 2018 10:00 PM

    July 03, 2018

    Release Notes for Safari Technology Preview 60

    Surfin’ Safari

    Safari Technology Preview Release 60 is now available for download for macOS High Sierra and the beta of macOS Mojave. If you already have Safari Technology Preview installed, you can update from the Mac App Store’s Updates tab on macOS High Sierra and in the Software Update pane of System Preferences on macOS Mojave. This release covers WebKit revisions 232790-233256.

    Known Issues

    • Safari Technology Preview Release 60 will crash on launch on macOS Mojave Developer Beta 1. Users should upgrade to macOS Mojave Developer Beta 2 to avoid the crash.
    • After updating to Safari Technology Preview Release 60, the homepage preference and the Develop menu preference will be lost.

    Web Animations

    • Changed CSS Animations to take precedence over CSS Transitions (r232868)
    • Ensured animations are updated prior to requestAnimationFrame callbacks (r233140)
    • Implemented the starting of CSS Transitions according to the standards specifications (r232946)

    Dark Mode

    • Fixed input form controls with a white background in dark mode (r232806)
    • Fixed the white corner between vertical and horizon scrollbars in dark mode (r233116)

    Web Inspector

    • Fixed all non-Same-Site cookies getting marked as Same-Site Strict in the Storage tab (r233027)
    • Fixed the Box Model section to have dark background in dark mode (r233187)
    • Fixed color outline that was too dark in dark mode (r233156)
    • Fixed font guideline colors that were too bright in dark mode (r233152)
    • Fixed media query names that were unreadable in dark mode (r233154)
    • Fixed network headers colors are too dim in dark mode (r233153)

    Media

    • Fixed the AirPlay picker to use the correct theme in dark mode (r233214)

    CSS

    • Fixed CSS background-color style to no longer affect natively rendered text fields (r232799)
    • Exposed more semantic system colors (r232847)

    WebRTC

    • Changed RTCRtpSender.replaceTrack(null) to stop the real-time source but keep the track (r232956)

    Security

    • Changed CSP to apply checks before content blocker checks for network loads to match cache load behavior (r232849)
    • Validated Cross-Origin-Resource-Policy for resources cached in the MemoryCache (r232933)

    Plug-ins

    • Fixed plug-in process crashing that affected Flash on macOS Mojave betas (r232848)

    Intelligent Tracking Prevention

    • Improved classification of redirect collusion to a prevalent resource (r232850)

    WebDriver

    • Fixed key actions to support multiple pressed virtual keys (r233131)
    • Included correct key code with synthesized NSEvents used for keystrokes (r233149)

    Accessibility

    • Fixed focus to follow the text cursor when zoom is enabled (r232944)

    July 03, 2018 05:00 PM

    June 20, 2018

    Web Animations in WebKit

    Surfin’ Safari

    Over the last 8 months we have been working on adding support for Web Animations, a W3C standard offering Web developers a JavaScript API to create, query and controls animations. While there is work left to do to ship this experimental feature to the Web at large, we feel our implementation has matured enough that, with the release of Safari Technology Preview 59, we can turn Web Animations on by default for our Web developer audience.

    An Animation API for the Web

    Browser engines have supported various animation features for many years, CSS Transitions and CSS Animations being two widely-supported approaches to authoring efficient animations on the Web. While these features have proven popular, they become limited when developers try to integrate browser-implemented animations via JavaScript:

    • Creating a CSS Transition dynamically requires forcing or waiting for a style invalidation so start and end values can be specified
    • Creating CSS Animations dynamically requires @keyframe rules to be generated and inserted in a global stylesheet
    • Animations created via CSS are not represented via JavaScript and cannot be queried or controlled

    For instance, developers would have to resort to code such as this to slide an element 100 pixels to the right:

    const element = document.getElementById("my-element");
    
    // Set the start value and transition properties.
    element.style.transform = "translateX(0)";
    element.style.transitionProperty = "transform";
    element.style.transitionDuration = "1s";
    
    // Force a style invalidation.
    window.getComputedStyle(element);
    
    // Set the end value.
    element.style.transform = "translateX(100px)";
    

    This approach is not elegant as it forces a style invalidation that causes extra work rather than just letting the browser invalidate styles at the most appropriate time. And this is just one single transition, but what if another library in your webpage also needed to create a transition? This would multiply forced style invalidation for no good reason.

    The value of Web Animations lies in having a JavaScript API that preserves the ability to let the browser engine do the heavy lifting of running animations efficiently while enabling more advanced control of your animations. Using Web Animations, we can rewrite the code above with a single method call:¬…

    element.animate({ transform: ["translateX(0)", "translateX(100px)"] }, 1000);
    

    A single method call and you’re done! But that’s not all, now you can harness the power of Web Animations with a full-featured API to control this animation:

    // Obtain a reference to the animation we created above.
    const animation = element.getAnimations()[0];
    // Seek the animation to 500ms.
    animation.currentTime = 500;
    // Pause the animation.
    animation.pause();
    

    The Web Animations API is very powerful, for instance letting you get a list of all running animations on the document or an individual element, use promises to run code when an animation is ready to start or has completed, reverse an animation, etc.

    Integration with CSS

    The Web Animations specification goes further than specifying a JavaScript API. It provides a timing and animation model for web browsers to implement features such as CSS Transitions and CSS Animations. In fact, in WebKit, we’ve updated our implementation of these existing CSS technologies so that the same CSS Transitions and Animations you’ve been authoring for years are now represented as Web Animations objects in the browser. Using the getAnimations() method, you can obtain a reference to a CSSTransition or CSSAnimation object which are Animation subclasses. Then you can seek or pause a CSS transition running on an element just like we did above with an animation created using element.animate(). As a developer, you can think of CSS Transitions and Animations as a declarative layer above Web Animations.

    Help Wanted

    We’re very excited to be bringing the power of Web Animations to WebKit and enabling the technology in Safari Technology Preview 59. But there is still a fair bit of work ahead and we need your help to ensure we have a quality implementation before enabling the feature in Safari. We encourage you to try the new API and report issues that you find, bearing in mind that our current implementation is a bit behind the current state of the specification, and you can track all API changes with bug #186518.

    It’s also important to check your existing content using CSS Transitions and Animations for possible regressions. One way to see if a regression you might be seeing is caused by the new Web Animations implementation, try toggling “Web Animations and CSS Integration” under the DevelopExperimental Features menu and see if your page’s behavior differs.

    June 20, 2018 05:00 PM

    Release Notes for Safari Technology Preview 59

    Surfin’ Safari

    Safari Technology Preview Release 59 is now available for download for macOS High Sierra and the beta of macOS Mojave. If you already have Safari Technology Preview installed, you can update from the Mac App Store’s Updates tab on macOS High Sierra and in the Software Update pane of System Preferences on macOS Mojave. This release covers WebKit revisions 232108-232790.

    Known Issues

    • This release of Safari Technology Preview for macOS Mojave betas does not render text properly in the Smart Search Field when in Dark Mode
    • Users running this version of Safari Technology Preview on macOS Mojave Developer Beta 2 will need to login to websites when restarting the application, or may require logging in again on some websites when launching a new window

    Intelligent Tracking Prevention 2.0

    • Enabled the latest version of Safari’s privacy protection “Intelligent Tracking Prevention.” For technical details, see the WebKit blog post.

    Web Animations

    • Enabled Web Animations as an experimental feature by default (r232186)
    • Added support for handling relative length units such as em, vw, and vh (r232255)
    • Fixed a bug causing WebAnimation objects to never get destroyed (r232185)

    Payment Request

    • Removed currencySystem member (r232155)

    JavaScript

    • Added Symbol.prototype.description getter (r232404)
    • Implemented + and - unary operation for BigInt (r232232)
    • Implemented support for % operation for BigInt (r232295)
    • Implemented support for < and > relational operation for BigInt (r232273)
    • Implemented support for =< and >= relational operation for BigInt (r232386)
    • Implemented support for addition operations for BigInt (r232449)
    • Fixed the Array.prototype.concat fast case when a single argument is a Proxy object (r232261)
    • Fixed Date.parse() to properly handle input outside of ES spec limits (r232122)
    • Fixed Array.prototype.sort to reject null comparator (r232666)
    • Renamed Array#flatten to flat (r232226)

    Editing

    • Fixed not displaying spelling errors in the middle of an inserted paragraph (r232530)

    Storage Access API

    • Changed Storage Access API calls to handle the absence of an attached frame (r232584)

    Security

    • Enabled Cross-Origin-Resource-Policy by default (r232311)
    • Changed the NetworkCORSPreflightChecker to set the preflight request User-Agent header (r232470)
    • Changed Accept request header values to be more tightly checked in the case of a CORS load (r232728)
    • Fixed the Referrer-Policy response header to not be ignored (r232310)
    • Renamed Cross-Origin-Options HTTP header to Cross-Origin-Window-Policy (r232499)
    • Renamed Cross-Origin-Resource-Policy:same to same-origin (r232309)
    • Migrated From-Origin to Cross-Origin-Resource-Policy (r232217)

    Service Workers

    • Added Accept-Encoding:identity to Range requests (r232571)
    • Fixed ServiceWorker registration to store any script fetched through importScripts (r232516)
    • Fixed HTTP Header values validation to not be too strict (r232572)
    • Improved error messages when FetchEvent.respondWith has a rejected promise (r232739)
    • Prevented starting service worker fetch when there is substitute data (r232580)
    • Updated Fetch code to provide more useful exception messages (r232484)

    Compatibility

    • Fixed page reloading when viewing photos in Google Drive due to exceeding canvas memory limits (r232113)
    • Fixed memory management when zooming and scrolling on some websites (r232356)
    • Link drag image is inconsistently unreadable in dark mode (r232731)

    WebRTC

    • Added an option to restrict communication to localhost sockets (r232420)

    Web API

    • Changed PopStateEvent to not be cancelable by default (r232610)

    Media

    • Changed media elements outside of fullscreen to not be considered main content (r232300)
    • Changed to exit fullscreen when JavaScript alerts are presented (r232437)
    • Changed automatic picture-in-picture to use the main content heuristic (r232301)
    • Changed to stop playing in the background when automatic picture-in-picture is disabled (r232426)
    • Enabled subsampling for progressive JPEG images (r232177)
    • Fixed incorrectly sized captions in picture-in-picture mode (r232220)
    • Fixed regions outside of the fullscreen window becoming exposed during zoom operations (r232543)
    • Fixed fullscreen element clipping by an ancestor (r232208)

    Web Inspector

    • Added Same-Site cookie annotations (r232318)
    • Fixed copying a link address in the Elements tab (r232481)
    • Fixed popovers getting dismissed while attempting to move the cursor inside (r232189)
    • Fixed the tab picker becoming briefly visible when the TabBar is initially shown (r232524)
    • Prevented text in “Add New Class” from being auto-capitalized (r232518)

    Accessibility

    • Exposed the <link> rel attribute to voiceover (r232326)
    • Fixed VoiceOver to announce when a details element is expanded when using role group (r232285)
    • Fixed setValue on contenteditable to preserve whitespace (r232120, r232259)
    • Fixed VoiceOver to announce fieldset description from aria-describedby when focusing inputs (r232331)
    • Implemented support for new blockquote, caption, and paragraph ARIA roles (r232508)

    iCloud Keychain Password Manager

    • Added support for loading https://example.com/.well-known/change-password when a user indicates they’d like to change a password that is reused in Safari’s Passwords preferences, falling back to loading example.com if that load fails

    June 20, 2018 04:00 PM

    June 06, 2018

    Safari Technology Preview 58, with Safari 12 Features, is Now Available

    Surfin’ Safari

    Safari Technology Preview Release 58 is now available for download for macOS High Sierra. With this release, Safari Technology Preview is now available for betas of macOS Mojave. If you already have Safari Technology Preview installed on macOS High Sierra, you can update from the Mac App Store’s Updates tab.

    This release covers the same revisions of WebKit from Safari Technology Preview 57, but includes new Safari and WebKit features that will be present in Safari 12. The following Safari 12 features are new to Safari Technology Preview 58:

    Icons in Tabs. You can enable showing website icons in tabs in Safari’s Tabs preferences.

    Automatic Strong Passwords. Safari automatically generates strong, unique passwords when signing up for accounts or changing passwords on websites. New to Safari 12 and Safari Technology Preview 58, generated passwords can be customized using the passwordrules attribute. See Apple’s Password Rules Validation Tool for more information.

    Many more WebKit features in Safari 12 are present in this release of Safari Technology Preview and have been in past releases. You can read more about these changes and many others in What’s New in Safari 12.

    June 06, 2018 05:00 PM

    June 04, 2018

    Michael Catanzaro: Security vulnerability in Epiphany Technology Preview

    Igalia WebKit

    If you use Epiphany Technology Preview, please update immediately and ensure you have revision 3.29.2-26 or newer. We discovered and resolved a vulnerability that allowed websites to access internal Epiphany features and thereby exfiltrate passwords from the password manager. We apologize for this oversight.

    The unstable Epiphany 3.29.2 release is the only affected release. Epiphany 3.29.1 is not affected. Stable releases, including Epiphany 3.28, are also not affected.

    There is no reason to believe that the issue was discovered or exploited by any attackers, but you might wish to change your passwords if you are concerned.

    By Michael Catanzaro at June 04, 2018 11:00 PM

    Intelligent Tracking Prevention 2.0

    Surfin’ Safari

    Today we’re happy to bring you Intelligent Tracking Prevention 2.0, or ITP 2.0. It builds upon ITP 1.0, which we released last year, and ITP 1.1, which was released in March, adding the Storage Access API.

    Removal of the 24 Hour Cookie Access Window

    ITP 2.0, as opposed to earlier versions, immediately partitions cookies for domains determined to have tracking abilities. The previous general cookie access window of 24 hours after user interaction has been removed. Instead, authenticated embeds can get access to their first-party cookies through the Storage Access API. The API requires that the user interacts with the embedded content.

    0 days: Cookies are partitioned and not persisted in 3rd-part contexts. | 30 days: Existing cookies are purged. New cookies are blocked. | Days of use after the most recent interaction with the website or successful use of the Store Access API.
    Example

    video.example offers an ad-free subscription service and has many of its videos embedded on other websites. ITP classifies video.example as having tracking abilities and therefore partitions its cookies. This is a timeline of how ITP might work for video.example:

    1. Day 1: The user logs in to video.example which updates video.example’s user interaction timestamp.
    2. Day 22: The user clicks to watch a video.example clip on news.example and the embedded video calls the Storage Access API. ITP notes that the user has not previously granted video.example access to its first-party cookies on news.example and thus prompts the user. The user grants storage access and video.example’s user interaction timestamp is updated.
    3. Day 26: The user clicks to watch another video.example clip on news.example and the video.example embed calls the Storage Access API. ITP notes that the user has previously granted video.example access to its first-party cookies on news.example and thus provides access without a prompt and updates video.example’s user interaction timestamp.
    4. Day 55: The user interacts with video.example as first party website which updates video.example’s user interaction timestamp.
    5. Day 76: The user clicks to watch a video.example clip on blog.example and the video.example embed calls the Storage Access API. ITP notes that the user has not previously granted video.example access to its first-party cookies on blog.example and thus prompts the user. The user grants storage access and video.example’s user interaction timestamp is updated.
    6. Day 80-82: The user doesn’t use Safari which means that these three days do not count towards the 30 days before website data purge.
    7. Day 109: video.example’s cookies, website data, and granted storage access entries are purged as a result of 30 days of Safari usage without an update to video.example’s user interaction timestamp.
    Developer Advice

    If you are a provider of authenticated third-party content, you should adopt the Storage Access API. If your website relies on a third party domain for user authentication, your authentication provider should adopt the Storage Access API or transfer authentication tokens to you in URLs.

    User Prompt For the Storage Access API

    ITP 2.0 adds a prompt to WebKit’s implementation of the Storage Access API. If the user allows access, their choice is persisted. If the user declines, their choice is not persisted which allows them to change their mind if they at a later point tap a similar embedded widget that calls the Storage Access API.

    As an example, video.example may have an ad-free subscription service and have many of its videos embedded on other websites. ITP will thus classify video.example as having tracking abilities and partition its cookies. When the user taps or clicks to play a clip embedded on news.example, video.example can call the Storage Access API to check whether the user is a subscriber. The user will be prompted if they have not explicitly allowed access under news.example previously.

    Storage Access API promptStorage Access API Prompt

    This prompt provides users with a way to show intent (the tap/click enabling the API call) and provide consent (“Allow” in the prompt). If the user chooses “Allow,” their choice is persisted and subsequent calls to the Storage Access API by video.example embeds on news.example do not trigger a prompt. Instead a tap or click on the embed is enough for a successful API call. As always, ITP considers the eTLD+1 to be the “party,” so a user “Allow” for sub.video.example under sub.news.example will be valid for video.example and any of its subdomains under news.example or any of its subdomains.

    Successful use of the Storage Access API now counts as user interaction with the third-party and refreshes the 30 days of use before ITP purges the third-party’s website data. By successful use we mean the user was either prompted right now and chose “Allow,” or had previously chosen “Allow.” The fact that successful Storage Access API calls now count as user interaction allows users to stay logged into services they rarely use as first party but keep using as embedded third parties.

    Finally, we received developer feedback (thank you), saying it should be possible to do a popup should storage access be granted but it turns out the user is not logged in. Our original version of the Storage Access API would consume the user gesture and thus require another tap or click to do a login popup. Now, the third-party embed is allowed to do a popup in the resolve scope of the returned promise, like so:

    <script>
    function makeRequestWithUserGesture() {
      var promise = document.requestStorageAccess();
      promise.then(
        function () {
          // Storage access was granted.
          // Check whether the user is logged in.
          // If not, do a popup to log the user in.
        },
        function () {
          // Storage access was denied.
        }
      );
    }
    </script>
    <button onclick="makeRequestWithUserGesture()">Play video</button>
    
    Developer Advice

    We encourage you to adopt the Storage Access API if you provide authenticated embeds. The API’s prompt provides you with a way to extend your users’ authenticated sessions if ITP has classified your domain as having the ability to track the user. If ITP classifies your domain and you don’t adopt the API, your domain will be permanently blocked from accessing its first-party cookies in a third-party context.

    Temporary Compatibility Fix: Automatic Storage Access for Popups

    Many federated logins send authentication tokens in URLs or through the postMessage API, both of which work fine under ITP 2.0. However, some federated logins use popups and then rely on third-party cookie access once the user is back on the opener page. Some instances of this latter category stopped working under ITP 2.0 since domains with tracking abilities are permanently partitioned.

    Our temporary compatibility fix is to detect such popup scenarios and automatically forward storage access for the third party under the opener page. Since popups require user interaction, the third party could just as well had called the Storage Access API instead.

    Developer Advice

    If you provide federated login services, we encourage you to first call the Storage Access API to get cookie access and only do a popup to log the user in or acquire specific consent. The Storage Access API provides a better user experience without new windows and navigations. We’d also like to stress that the compatibility fix for popups is a temporary one. Longterm, your only option will be to call the Storage Access API.

    Protection Against First Party Bounce Trackers

    ITP 2.0 has the ability to detect when a domain is solely used as a “first party bounce tracker,” meaning that it is never used as a third party content provider but tracks the user purely through navigational redirects.

    Say the user clicks on a news.example link on the social.example website. Instead of navigating them straight to their destination news.example, they are rapidly navigated through trackerOne.example and trackerTwo.example before reaching news.example. Those two tracker domains can store information about the user’s browsing history in first party storage and cookies. ITP 2.0 detects such tracking behavior and treats those domains just like any other tracker, i.e. purges their website data.

    First-party cookie bounce diagram

    Protection Against Tracker Collusion

    Through our research, we found that cross-site trackers help each other identify the user. This is basically one tracker telling another tracker that “I think it’s user ABC,” at which point the second tracker tells a third tracker “Hey, Tracker One thinks it’s user ABC and I think it’s user XYZ.” We call this tracker collusion, and ITP 2.0 detects this behavior through a collusion graph and classifies all involved parties as trackers.

    Cross-site tracker collusion diagram

    In the graph above, as soon as trackerSix.example is classified by ITP, all the domains that have redirected to trackerSix.example get classified too. That’s trackerTwo.example, trackerThree.example, and trackerFive.example. Then domains that have redirected to those get classified too, which covers the last two—trackerOne.example and trackerFour.example.

    Developer Advice

    Avoid making unnecessary redirects to domains that are likely to be classified as having tracking ability.

    Origin-Only Referrer for Domains Without User Interaction

    ITP’s purging of website data does not prevent trackers from receiving the so called referrer header which reveals the webpage the user is currently on. In ITP 2.0, the referrer, if there is one, is downgraded to just the page’s origin for third party requests to domains that have been classified as possible trackers by ITP and have not received user interaction.

    Here’s an example of what we mean by this: Say the user visits https://store.example/baby-products/strollers/deluxe-navy-blue.html, and that page loads a resource from trackerOne.example. Before ITP 2.0, the request to trackerOne.example would contain the full referrer “https://store.example/baby/strollers/deluxe-stroller-navy-blue.html” which reveals a lot about the user to the third-party. With ITP 2.0, the referrer will be reduced to just “https://store.example/”.

    For further reading on this subject, see Mozilla’s research on origin-only referrers.

    FAQ

    Does ITP differentiate between my subdomains?
    No. ITP captures statistics and applies its rules for the effective top-level domain plus one, or eTLD+1. An eTLD is .com or .co.uk so an example of an eTLD+1 would be social.co.uk but not sub.social.co.uk (eTLD+2) or just co.uk (eTLD).

    Does eTLD mean public suffix?
    Yes. See the Public Suffix List.

    Is there a way for users to whitelist my domain to be excepted from ITP’s rules?
    No, there is no such whitelisting feature.

    How does ITP classify a domain’s tracking abilities?
    See the original ITP 1.0 blog post for details on the machine learning model.

    My domain is not a tracker, but it gets classified by ITP. Is that a bug?
    ITP does not rely on a centralized blacklist of known trackers. Instead, it captures per-domain (eTLD+1) statistics on-device and classifies each domain’s ability to track cross-site. If a domain has the ability to track the user cross-site, it’s subject to ITP’s cookie and website data rules.

    Is it enough for users to visit my website to keep its cookies from being purged if my domain gets classified by ITP?
    No, a mere visit does not suffice. The user has to interact with your website, meaning a tap, click, or form entry. In ITP 2.0, user granted access through the Storage Access API also counts as such user interaction.

    How do I reset ITP or the domains I’ve allowed storage access for?
    Clear your Safari history. Note that this gets rid of data that may be needed to reproduce an ITP issue. If you’re investigating a bug related to ITP, don’t clear your history before you’re done with that work.

    I need to debug my website in regards to ITP. Are there specific developer tools?
    We’re working on an ITP Debug Mode which will help you debug websites and capture data that’s useful when filing bugs on ITP. It will be announced on this blog.

    What is an authenticated widget, an authenticated embed, or authenticated third-party content?
    The web platform allows for powerful integration across websites. This enables services such as social media to create widgets that are to be embedded as third-party content on news sites and blogs, but it also enables cross-site tracking. Some widgets have to see the user as logged in to work. For example, if the user wants to comment on blog.example with their social.example account, the social.example commenting widget needs to see the user as logged in to their service. We refer to these widgets as authenticated and to enable them while disabling cross-site tracking, the widgets now have to ask for permission to see the user’s identity, per-site.

    How does the removal of the 24 hour cookie access window prevent cross-site tracking from social media etc?
    ITP 1.0 had a 24 hour window in which websites the user interacted with could use their cookies as in previous versions of Safari. This was a compatibility measure we took to enable federated logins (e.g. use social.example to login to news.example) and authenticated third-party content (e.g. use social.example to make a comment on blog.example). However, this also allowed social media and search engines that the user interacted with as first parties to track the user across websites while inside that 24 hour window. The mere existence of third-party content from these services was enough for them to see which webpages the user visited. For example, if a user used social.example and then, several hours later, visited news.example which had a social.example Like button or comment widget embedded on the site, social.example could track their browsing on news.example. In ITP 2.0 we restrict such third-party content to only be able to identify the user when they actually use the content, such as write a comment or play a video. This is also the point at which Safari will ask for the user’s permission (if the widget is asking for permission to see its cookies).

    Feedback and Bug Reports

    Please report bugs through bugs.webkit.org and send feedback to our evangelist Jon Davis. If you have technical questions on how the feature works you can find me on Twitter @johnwilander.

    June 04, 2018 07:00 PM

    May 30, 2018

    Release Notes for Safari Technology Preview 57

    Surfin’ Safari

    Safari Technology Preview Release 57 is now available for download for macOS High Sierra. If you already have Safari Technology Preview installed, you can update from the Mac App Store’s Updates tab. This release covers WebKit revisions 231553-232108.

    This version of Safari Technology Preview will no longer run on macOS Sierra. To continue testing or living on the latest enhancements to Safari and WebKit, please upgrade to macOS High Sierra.

    JavaScript

    • Added support for Intl.NumberFormat().formatToParts() (r231867)
    • Changed to throw SyntaxErrors for runtime generated regular expressions with errors (r231939)
    • Implemented BigInt support for *, /, and == operations (r231733, r231886, r231629)
    • Improved specification compliance for Internationalization APIs (r231740)
    • Improved the speed of Object.assign for final objects (r231687)

    Animations

    • Exposed Web Animations CSS integration as an experimental feature (r231798)
    • Fixed a bug where animation-play-state:paused causes very high cpu load because of a style invalidation loop (r231794)

    Storage Access API

    • Extended the lifetime of cookies on successful user approval (r231684)

    Security

    • Added initial support for the Cross-Origin-Options HTTP response header (r231622, r231654)
    • Changed Cross-Origin-Options:deny and Cross-Origin-Options:allow-postmessage to prevent getting navigated by cross-origin scripts (r231911)
    • Changed X-Frame-Options:SAMEORIGIN to check all ancestor frames (r231730)

    Media

    • Enabled the modern EME API by default (r231903)
    • Fixed media continuing to load after rendered invisible (e.g. removed from DOM; scrolled off screen) (r231817)
    • Improved NowPlaying using the element "title" attribute when available (r231866)

    Accessibility

    • Changed the accessiblility name provided for a node to simplify the whitespace when using innerText (r231627)
    • Excluded hidden nodes which are not directly referenced from participating in name or description content (r231620)
    • Ensured listbox and combobox roles embedded in labels participate in name calculation (r231778)
    • Exposed the primary screen height through the AX API (r231937)
    • Fixed a bug with VoiceOver causing <iframe> scrolling focus jumping (r231628)
    • Fixed VoiceOver to manually focus or read dialog paragraph description text inside the modal in role=dialog elements with aria-modal=true (r231720)

    WebDriver

    • Fixed a crash that happens if a browsing context’s cookies are requested prior to an initial navigation (r232031)
    • If the network or storage process crashes, terminate the automation session to avoid undefined behavior (r232028)
    • Automation.getBrowsingContext now returns the same window origin as window.screenX and window.screenY (r231769)

    Web Inspector

    • Added rulers and guides to the Canvas tab (r231819, r231881)
    • Changed the Canvas tab to not automatically select a recording when viewing a canvas (r231773)
    • Improved the placement logic for the element details popover that is shown during Inspect Element mode (r231838)
    • Calculating the visual effect of each Canvas action is now off by default for performance reasons. (r231981)

    May 30, 2018 05:00 PM

    May 27, 2018

    Michael Catanzaro: Thoughts on Flatpak after four months of Epiphany Technology Preview

    Igalia WebKit

    It’s been four months since I announced Epiphany Technology Preview — which I’ve been using as my main browser ever since — and five months since I announced the availability of a stable channel via Flatpak. For the most part, it’s been a good experience. Having the latest upstream development code for everything is wonderful and makes testing very easy. Any user can painlessly download and install either the latest stable version or the bleeding-edge development version on any Linux system, regardless of host dependencies, either via a couple clicks in GNOME Software or one command in the terminal. GNOME Software keeps it updated, so I always have a recent version. Thanks to this, I’m often noticing problems shortly after they’re introduced, rather than six months later, as was so often the case for me in the past. Plus, other developers can no longer complain that there’s a problem with my local environment when I report a bug they can’t reproduce, because Epiphany Technology Preview is a canonical distribution environment, a ground truth of sorts.

    There have been some rough patches where Epiphany Technology Preview was not working properly — sometimes for several days — due to various breaking changes, and the long time required to get a successful SDK build when it’s failing. For example, multimedia playback was broken for all of last week, due to changes in how the runtime is built. H.264 video is still broken, since the relevant Flatpak extension is only compatible with the 3.28 runtime, not with master. Opening files was broken for a while due to what turned out to be a bug in mutter that was causing the OpenURI portal to crash. I just today found another bug where closing a portal while visiting Slack triggered a gnome-shell crash. For the most part, these sorts of problems are expected by testers of unstable nightly software, though I’m concerned about the portal bugs because these affect stable users too. Anyway, these are just bugs, and all software has bugs: they get fixed, nothing special.

    So my impression of Flatpak is still largely positive. Flatpak does not magically make our software work properly in all host environments, but it hugely reduces the number of things that can go wrong on the host system. In recent years, I’ve seen users badly break Epiphany in various ways, e.g. by installing custom mimeinfo or replacing the network backend. With Flatpak, either of these would require an incredible amount of dedicated effort. Without a doubt, Flatpak distribution is more robust to user error. Another advantage is that we get the latest versions of OS dependencies, like GStreamer, libsoup, and glib-networking, so we can avoid the many bugs in these components that have been fixed in the years since our users’ LTS distros froze the package versions. I appreciate the desire of LTS distros to provide stability for users, but at the same time, I’m not impressed when users report issues with the browser that we fixed two years ago in one dependency or another. Flatpak is an excellent compromise solution to this problem: the LTS distro retains an LTS core, but specific applications can use newer dependencies from the Flatpak runtime.

    But there is one huge downside to using Flatpak: we lose crash reports. It’s at best very difficult — and often totally impossible — to investigate crashes when using Flatpak, and that’s frankly more important than any of the gains I mention above. For example, today Epiphany Technology Preview is crashing pretty much constantly. It’s surely a bug in WebKit, but that’s all I can figure out. The way to get a backtrace from a crashing app in flatpak is to use coredumpctl to manually dump the core dump to disk, then launch a bash shell in the flatpak environment and manually load it up in gdb. The process is manual, old-fashioned, primitive, and too frustrating for me by a lot, so I wrote a little pyexpect script to automate this process for Epiphany, thinking I could eventually generalize it into a tool that would be useful for other developers. It’s a horrible hack, but it worked pretty well the day I tested it. I haven’t seen it work since. Debuginfo seems to be constantly broken, so I only see a bunch of ???s in my backtraces, and how are we supposed to figure out how to debug that? So I have no way to debug or fix the WebKit bug, because I can’t get a backtrace. The broken, inconsistent, or otherwise-unreliable debuginfo is probably just some bug that will be fixed eventually (and which I half suspect may be related to our recent freedesktop SDK upgrade. Update: Alex has debugged the debuginfo problem and it looks like that’s on track to be solved), but even once it is, we’re back to square one: it’s still too much effort to get the backtrace, relative to developing on the host system, and that’s a hard problem to solve. It requires tools that do not exist, and for which we have no plans to create, or even any idea of how to create them.

    This isn’t working. I need to be able to effortlessly get a backtrace out of my application, with no or little more effort than running coredumpctl gdb as I would without Flatpak in the way. Every time I see Epiphany or WebKit crash, knowing I can do nothing to debug or investigate, I’m very sorely tempted to switch back to using Fedora’s Epiphany, or good old JHBuild. (I can’t promote BuildStream here, because BuildStream has the same problem.)

    So the developer experience is just not good, but set that aside: the main benefits of Flatpak are for users, not developers, after all. Now, what if users run into a crash, how can they report the bug? Crash reports are next to useless without a backtrace, and wise developers refuse to look at crash reports until a quality backtrace has been posted. So first we need to fix the developer experience to work properly, but even then, it’s not enough: we need an automatic crash reporter, along the lines of ABRT or apport, to make reporting crashes realistically-achievable for users, as it already is for distro-packaged apps. But this is a much harder problem to solve. Such a tool will require integration with coredumpctl, and I have not the faintest clue how we could go about making coredumpctl support container environments. Yet without this, we’re asking application developers to give up their most valuable data — crash reports — in order to use Flatpak.

    Eventually, if we don’t solve crash reporting, Epiphany’s experiment with Flatpak will have to come to an end, because that’s more important to me than the (admittedly-tremendous) benefits of Flatpak. I’m still hopeful that the ingenuity of the Flatpak community will find some solutions. We’ll see how this goes.

    By Michael Catanzaro at May 27, 2018 11:39 PM

    May 16, 2018

    Release Notes for Safari Technology Preview 56

    Surfin’ Safari

    Safari Technology Preview Release 56 is now available for download for macOS Sierra and macOS High Sierra. If you already have Safari Technology Preview installed, you can update from the Mac App Store’s Updates tab. This release covers WebKit revisions 230913-231553.

    This is the last release of Safari Technology Preview that will install and run on macOS Sierra. To continue testing or living on the latest enhancements to Safari and WebKit, please upgrade to macOS High Sierra.

    JavaScript

    • Implemented Intl.PluralRules (r231047)

    WebAssembly

    • Added support for stream APIs (r231194)

    Web API

    • Fixed document.open() event listener removal to be immediate (r231248)
    • Fixed DHTML drag operations to report the number of files in the operation (r231003)
    • Fixed window.postMessage(), window.focus(), and window.blur() unexpectedly throwing a TypeError (r231037)
    • Serialized font-variation-settings with double-quotes to match standards (r231165)
    • Stopped using the id of an <iframe> as fallback if its name attribute is not set (r231456)

    Security

    • Added support for the WHATWG proposed From-Origin:same and From-Origin:same-site response headers with nested frame origin checking as an off by default experimental feature (r230968)
    • Fixed CSP referrer for a document blocked due to a violation of its frame-ancestors directive (r231461)
    • Fixed CSP status-code for a document blocked due to a violation of its frame-ancestors directive (r231464)
    • Fixed CSP to pass the document’s referrer (r231445)
    • Fixed CSP to only notify Web Inspector to pause the debugger on the first policy to violate a directive (r231443)
    • Fixed a bug causing first-party cookies to be blocked on redirects (r231008)

    CSS

    • Fixed CSS filters which reference SVG filters to respect the color-interpolation-filters of the filter (r231473)
    • Fixed feTurbulence to render correctly on a Retina display (r231485)
    • Fixed shape-outside and filter styles occuring twice in the result of getComputedStyle (r230976)

    Rendering

    • Changed font collection fragment identifiers to use PostScript names (r231259)
    • Fixed selecting text on a webpage causing the text vanish (r231178)
    • Fixed hiding then showing an <object> of type image to ensure the underlying image is displayed (r231292)

    Media

    • Changed MediaStreams that are playing to allow removing some of its tracks (r231304)
    • Updated text track cue logging to include cue text (r231490)

    Web Inspector

    • Improved the user experience in Canvas tab to show progress bars while processing actions in a new recording (r231218)
    • Ensured that tabbing through the last section of rules in the Styles editor wraps back to the first section of rules (r231372)
    • Fixed Console drawer resizing when the console prompt has more than one line of code (r231527)
    • Fixed unsupported properties that sometimes don’t get warnings just after adding them (r231377)
    • Updated the Canvas tab to determine functions by looking at the prototype (r231368)

    May 16, 2018 05:00 PM

    May 02, 2018

    Release Notes for Safari Technology Preview 55

    Surfin’ Safari

    Safari Technology Preview Release 55 is now available for download for macOS Sierra and macOS High Sierra. If you already have Safari Technology Preview installed, you can update from the Mac App Store’s Updates tab. This release covers WebKit revisions 230521-230913.

    CSS

    • Added support for calc() in webkit-gradient and cross-fade (r230816)
    • Changed :active pseudo class to deactivate when using pressure sensitive trackpad (r230786)
    • Omitted the default value when serializing font-feature-settings (r230838)
    • Updated HSL and HSLA parsing to match CSS Color 4 (r230861)

    Web API

    • Enforced content-type for <script> tags (r230602)
    • Fixed input.webkitEntries to work as expected when a folder contains accented chars (r230639)
    • Setting event.returnValue to true is now a no-op (r230664)

    Rendering

    • Fixed animated GIFs with finite looping to loop the expected number of times (r230712)
    • Fixed the position of the caret in empty table cells (r230797)

    Accessibility

    • Added the accessibility events to the HTML attribute names so AOM works with DOM Level 1 events (r230890)
    • Fixed a hang when triggering an alert from an AOM increment event (r230782)

    JavaScript

    • Fixed function.prototype.caller to not return generator bodies (r230662)

    Web Driver

    • Fixed simulated mouse interactions to not be done until the associated DOM events have been dispatched (r230817)

    Web Inspector

    • Fixed selecting and copying text from a Network tab popover (r230701)

    Media

    • Added allSamplesInTrackEnqueued event (r230909)

    WebRTC

    • Fixed a WebRTC data channel issue for non-ASCII characters. (r230524)

    Security

    • Changed to use CSP and content blockers for sync XHR (r230810)

    Bug Fixes

    • Fixed an issue that would sometimes incorrectly rule out PDF as a mime type that can be showed (r230853)
    • Fixed an issue with Command-clicking web app links on icloud.com (r230721)

    May 02, 2018 05:00 PM

    April 27, 2018

    Visualizing Layers in Web Inspector

    Surfin’ Safari

    Recent releases of Safari Technology Preview contain a new experimental feature for Web Inspector: the Layers tab. Building upon the legacy Layers sidebar, this tab introduces a 3D visualization of the inspected page’s compositing layers, to provide developers with a more hands-on (and hopefully fun!) way to understand where layers are being generated and in what order they will render.

    Layers tab

    In this post, we’ll look at how the Layers tab can be used to find unexpected memory consumption or excessive repaints on a web page.

    A Whirlwind Introduction to Layers

    Before introducing any debugging tool, it’s important to clarify what we’re aiming to debug in the first place. At first glance, a layer visualization may appear quite similar to a DOM visualization (such as Firefox’s Tilt), yet compositing layers and DOM elements are conceptually different entities. While web developers are intimately familiar with the DOM tree as the structure of elements on a page, the way in which these elements end up rendered on screen tends to only be learned as a need arises.

    DOM elements are not just painted to the screen one by one. After the position and size of each element has been calculated, they are drawn onto a series of surfaces called layers. It is these layers which are then composited in a particular order to produce the resulting appearance of the web page. All pages have a root layer corresponding to the document itself, but any DOM element can cause a new layer to be created. Among the reasons for this are element type (e.g. <canvas>), presence of certain CSS properties (e.g. 3D transforms), or interaction with other layer-generating elements.

    Layers have a significant impact on rendering performance. Without layers, animating any element (e.g. sliding a header down) would mean having to wastefully repaint the whole page on every frame. If the element has its own layer, however, it may be possible to skip repainting altogether and animate by simply recompositing the layers on screen. Of course, the trade-off for this computational savings is memory cost. Creating too many layers—intentionally or otherwise—can have disastrous results on memory-constrained devices. Like any performance concern, finding the right balance is an empirical issue, so having a suitable debugging tool is crucial!

    Identifying Compositing Issues with the Layers Tab

    The Layers tab features two real-time representations of the layers on the page, a 3D visualization and a data table. We can use these in conjunction to discover potential performance problems.

    We might begin by exploring the visualization to understand each layer’s position, size, and rendering order. To navigate, use left-drag to rotate, right-drag to pan, and scroll to zoom. Clicking a layer in the visualization highlights the corresponding row in the layer table.

    We can then use the table to identify the costliest layers, sorting by Memory to prioritize memory consumption (default) or by Paints to prioritize repaint count. Selecting a row in the table displays information about why the layer exists as well as its composited and visible dimensions (visualized as an outline and a plane, respectively). This also recenters the visualization on the selection, clarifying how that layer fits into the larger picture.

    explore table

    At this point, perhaps we’ll take notice of an oddly-positioned layer nearby and jump to it accordingly. Or we could simply inspect each of the most expensive layers in turn. If a layer seems suspicious, we can click the arrow icon in its table row to switch to the Elements tab and examine the DOM element that generated it. And if that’s not enough to explain an unusually high repaint count, we can turn to the Timelines tab to figure out where those paint events are being triggered.

    Once we’ve identified our issue, it’s just a matter of making the appropriate HTML/CSS modifications!

    Feedback

    The Layers tab is available in the latest Safari Technology Preview. To enable it, visit the Experimental panel on Web Inspector’s Settings tab and check the “Enable Layers tab” setting. Give it a try and let us know what you think! Reach out at #webkit-inspector on Freenode IRC, to @webkit on Twitter, or by filing a bug.

    April 27, 2018 05:00 PM

    April 25, 2018

    Web Inspector Styles Sidebar Improvements

    Surfin’ Safari

    In Web Inspector that recently shipped with Safari 11.1 on macOS High Sierra, the Elements tab sidebar panels and the styles editor got a lot of attention from the Web Inspector team. We’ve re-written the styles editor to provide an editing experience more familiar to web developers, and rearranged the sidebar panels to improve access to the most used tools. The design refresh brings new behaviors and fine-tuning to enhance web developers’ ability to access and understand the elements they’re inspecting.

    Tabs Layout

    styles tabs before & afterBefore / After

    In the Elements tab, Styles and Computed are the most commonly used panels. We made them top-level tabs, so it takes a single click to switch between them.

    Styles Panel

    styles panel information density before & afterBefore / After

    The redesigned Styles panel of the same size now fits more data:

    • Selectors are no longer generated for style attributes
    • The default “Media: all” header is no longer shown
    • The icons were removed to save some horizontal space

    Syntax Highlighting

    styles syntax highlighting

    Property values are now black to make them easier to distinguish from property names. Strings are still red, but links are now blue.

    We added curly braces back so copying CSS would produce valid CSS. Curly braces, colons, and semicolons are light gray so they won’t distract from the content.

    Styles Editor

    We rewrote the styles editor from scratch. This is the first major overhaul of the styles editor since 2013. Instead of a free-form text editor, we changed to cell-based style editing.

    Styles tab and shift-tab behavior

    CSS property names and values are now separate editable fields. Pressing Tab (⇥) or Return navigates to the next editable field. Pressing Shift-Tab (⇧⇥) navigates to the previous editable field.

    Also, typing a colon (“:”) when focused on the property name focuses on the corresponding property value. Typing semicolon (“;”) at the end of the property value navigates to the next property.

    Styles add new property behavior

    To add a new property, you can click on the white space before or after an existing property. Pressing Tab (⇥) or Return when focused on the last property value also adds a new property.

    Styles remove property behavior

    To remove a property, you can remove either a property name or a property value.

    Styles up and down arrow behavior

    Completion suggestions display right away when focused on the value field. Completion values apply right away when selecting using Up and Down arrow keys.

    Styles more arrow key behaviors

    While editing a property field, Up and Down arrow keys now can increment and decrement values. You can change the interval by holding modifier keys:

    • Option (⌥): 0.1
    • Shift (⇧): 10
    • Command (⌘): 100

    Legacy Settings

    Legacy settings screen with the Legacy Style Editor setting

    The previous version of the styles editor is still available in the in Web Inspector settings, but it’s no longer maintained.

    The Visual Styles Panel never gained enough traction to remain in Elements tab by default. It is no longer maintained. Along with the Legacy Style Editor, the Visual Styles Panel can still be enabled in the Experimental settings.

    Contributing

    Please report bugs and feature requests regarding the new styles editor on webkit.org/new-inspector-bug. If you’re interested in contributing or have any questions, please stop by the #webkit-inspector IRC channel.

    Web Inspector is primarily written in HTML, JavaScript, and CSS, which means that web developers already have the skills needed to jump in and contribute a bug fix, enhancement or a new feature.

    April 25, 2018 05:00 PM

    April 18, 2018

    Release Notes for Safari Technology Preview 54

    Surfin’ Safari

    Safari Technology Preview Release 54 is now available for download for macOS Sierra and macOS High Sierra. If you already have Safari Technology Preview installed, you can update from the Mac App Store’s Updates tab. This release covers WebKit revisions 230029-230521.

    Clipboard API

    • Fixed copying a list from Microsoft Word to TinyMCE when mso-list is the first property (r230120)
    • Prioritized file promises over filenames during drag and drop (r230221)

    Beacon API

    • Fixed Beacon redirect responses to be CORS validated (r230495)

    Web API

    • Implemented createImageBitmap(Blob) (r230350)

    WebRTC

    • Added a special software encoder mode when a compression session is not using a hardware encoder and VCP is not active (r230451)
    • Added experimental support for MDNS ICE candidates in WebRTC data channel peer-to-peer connections (r230290, r230307)

    Web Inspector

    • Fixed the errors glyph to fully change to blue when active (r230372)
    • Tinted all pixels drawn by a shader program when hovering over a tree element in the Canvas Tab (r230127)

    April 18, 2018 05:00 PM

    April 15, 2018

    Manuel Rego: CSSWG F2F Berlin 2018

    Igalia WebKit

    Last week I was in Berlin for the CSS Working Group (CSSWG) face-to-face meeting representing Igalia, member of the CSSWG since last year. Igalia has been working on the open web platform for many years, where we help our customers with the implementation of different standards on the open source web engines. Inside the CSSWG we play the implementors role, providing valuable feedback around the specifications we’re working on.

    It was really nice to meet all the folks from the CSSWG there, it’s amazing to be together with such a brilliant group of people in the same room. And it’s lovely to see how easy is to talk with any of them, you all rock!

    CSSWG F2F Berlin 2018 by Rossen Atanassov CSSWG F2F Berlin 2018 by Rossen Atanassov

    This is a brief post about my highlights from there, of course totally subjective and focused on the topics I’m more interested.

    CSS Grid Layout

    We were discussing two issues of the current specification related to the track sizing algorithm and its behavior in particular cases. Some changes will be added in the specification to try to improve them and we’ll need to update the implementations accordingly.

    On top of that, we discussed about the Level 2 of the spec. It’s already defined that this next level will include the following features:

    • The awaited subgrids feature: There was the possibility of allowing subgrids in both axis (dual-axis) or only in one of them (per-axis), note that the per-axis approach covers the dual-axis if you define the subgrid in both axis.

      There are clear uses cases for the per-axis approach but the main doubt was about how hard it’d be to implement. Mats Palmgren from Mozilla posted a comment on the issue explaining that he has just created a prototype for the feature following the per-axis idea, so the CSSWG resolved to remove the dual-axis one from the spec.

    • And aspect-ratio controlled gutters: Regarding this topic, the CSSWG decided to add a new ar unit. We didn’t discuss anything more but we need to decide what we’ll do in the situations where there’s no enough free space to fulfill the requested aspect-ratio, should we ignore it or overflow in that case?

      Talking to Rachel Andrew about the issue, she was not completely sure of what would be the preferred option from the authors point of view. I’ve just added some examples to the issue so we can discuss about them there and gather more feedback, please share your thoughts.

    Tests

    This was a discussion I wanted to have with the CSSWG people in order to understand better the current situation and possible next steps for the CSSWG test suites.

    Just to add some context, the CSSWG test suites are now part of the web-platform-tests (WPT) repository. This repository is being used by most browser vendors to share tests, including tests for new CSS features. For example, at Igalia we’re currently using WPT test suites in all our developments.

    The CSSWG uses the CSS Test Harness tool which has a build system that adds some special requirements for the test suites. One of them causes that we need to duplicate some files in the repository, which is not nice at all.

    Several people in the CSSWG still rely on this tool mainly for two things:

    • Run manual tests and store their results: Some CSS features like media queries or scrolling are hard to automate when writing tests, so several specs have manual tests. Probably WebDriver can help to automate this kind of tests, maybe not all though.
    • Extract status reports: To verify that a spec fulfills the CR exit criteria, the current tooling has some nice reports, it also provides info about the test coverage of the spec.

    So we cannot get rid of the CSS Test Harness system at this point. We discussed about possible solutions but none of them were really clear, also note that the lack of funding for this kind of work makes it harder to move things forward.

    I still believe the way to go would be to improve the WPT Dashboard (wpt.fyi) so it can support the 2 features listed above. If that’s the case maybe the specific CSS Test Harness stuff won’t be needed anymore, thus the weird requirements for people working on the test suites will be gone, and there would be a single tool for all the tests from the different working groups.

    As a side note wpt.fyi needs some infrastructure improvements, for example Microfost was not happy as Ahem font (which is used a lot in CSS tests suites) is still not installed on the Windows virtual machines that extract test results for wpt.fyi.

    Floats, floats, floats

    People are using floats to simulate CSS Shapes on browsers that don’t have support yet. That is causing that some special cases related to floats happen more frecuently, and it’s hard to decide what’s the best thing to do on them.

    The CSSWG was discussing what would be the best solution when the non-floated content doesn’t fit in the space left by the floated elements. The problem is quite complex to explain, but imagine the following picture where you have several floated elements.

    An example of float layout An example of float layout

    In this example there are a few floated elements restricting the area where the content can be painted, if the browser needs to find the place to add a BFC (like a table) it needs to decide where to place it avoiding overlapping any other floats.

    There was a long discussion, and it seems the best choice would be that the browser tests all the options and if there’s no overlapping then puts the table there (basically Option 1 in the linked illustration). Still there are concerns about performance, so there’s still more work to be done here. As a result of this discussion a new CSS Floats specification will be created to describe the expected behavior in this kind of scenarios.

    Monica Dinculescu created a really cool demo to explain how float layout works, with the help of Ian Kilpatrick who knows it pretty well as he has been dealing with lots of corner cases while working in LayoutNG.

    TYPO Labs

    The members of the CSSWG were invited to the co-located TYPO Labs event. I attended on Friday when Elika (fantasai), Myles and Rossen gave a talk. It was nice to see that CSS Grid Layout was mentioned in the first talk of the day, as an useful tool for typographers. Variable fonts and Virtual Reality were clearly hot topics in several talks.

    Elika (fantasai), Myles and Rossen in the CSSWG talk at TYPO Labs Elika (fantasai), Rossen and Myles in the CSSWG talk at TYPO Labs

    It’s funny that the last time I was in Berlin was 10 years ago for a conference related to TYPO3, totally unrelated but with a similar name. 😄

    Other

    Some pictures of Berlin Some pictures of Berlin

    And that’s mostly all that I can remember now, I’m sure I’m missing many other important things. It was a fantastic week and I even find some time for walking around Berlin as the weather was really pleasant.

    April 15, 2018 10:00 PM

    April 12, 2018

    New WebKit Features in Safari 11.1

    Surfin’ Safari

    Update: This post has been updated to omit Offscreen Canvas which is not available in Safari 11.1 on macOS and Safari on iOS 11.3. Offscreen Canvas is available as an experimental feature in Safari Technology Preview releases.

    The recently shipped Safari 11.1 in macOS High Sierra and Safari on iOS 11.3 includes WebKit improvements that add support for a lot of exciting web platform technologies. These features offer new capabilities to developers to improve user-perceived performance of web content, it adds a new dynamic approach for offline experiences, and brings a standard approach to web payments that simplifies development. It also delivers redesigned developer tools in Web Inspector that improve overall usability and convenience.

    What follows is a short tour of all of the new features and improvements available with Safari 11.1.

    Service Worker API

    The Service Worker API enables scripts that run in the background to handle generalized processing for web pages. Service Worker scripts commonly work with other complimentary APIs, such as the Cache API, also included in this release, to store fetch requests and responses that to improve network performance or provide an offline experience. WebKit’s implementation respects user privacy partitioning Service Workers by the top level document origin to help prevent cross-site tracking.

    Read the “Workers at Your Service” blog post for more details about the WebKit implementation of Service Workers.

    Payment Request API for Apple Pay

    WebKit added support for the Payment Request API to perform Apple Pay transactions. This enables merchants to offer payment methods across multiple browsers using a standard API, greatly reducing the effort of supporting secure payments across different browsers and platforms.

    See Introducing the Payment Request API for Apple Pay for more details.

    Beacon API

    Beacon API schedules asynchronous requests to send before the page is unloaded and completes them without a blocking request. This is useful for sending diagnostics data about the web application without impacting perceived page load performance.

    See the Beacon specification for more information.

    Directory Upload & Clipboard API Updates

    The Clipboard API has a number of improvements focusing on a modernized implementation of the DataTransfer API. It adds support for dataTransfer.items, reading and writing of HTML content and custom MIME types, and fixed a number of bugs including copying & pasting images from native applications. WebKit also added support for uploading directories using DataTransfer.webkitGetAsEntry() and input.webkitdirectory, allowing interoperability with other browsers. Users can now upload an entire directory onto cloud storage and file sharing services such as iCloud or Dropbox.

    Read more about the improvements to the Clipboard API and Directory Upload API.

    Video as Image Assets

    Animated image formats are very popular, but they easily become large, bandwidth intensive file sizes. To address the performance impact, WebKit in Safari now supports loading H.264 encoded MP4 video with an HTML tag. This allows content authors to replace animated GIF files that are much larger than H.264 video files and require more processing power to display. Beyond the performance gains, this change also allows web content authors to use videos as a CSS background-image.

    Some detailed information about this change is available in the blog post Evolution of : Gif without the GIF by Colin Bendell.

    font-display

    Page authors have more control over font loading behavior using the newly support font-display descriptor. WebKit’s new default behavior will show invisible text for up to 3 seconds before switching to an installed font until the desired font finishes downloading. Using font-display, the page author can use the swap value to immediately show an installed font until the specified web font is downloaded. There is a fallback behavior to show an installed font for up to 3 seconds and if the desired font doesn’t load in that time, it will never be shown. Finally, the optional behavior will use the font if it is available in the browser’s cache. If it isn’t there, it skips using it at all. These provide a rich set of behaviors for web developers to control the loading and display experience that best suits the design of their web content.

    Find more details in the CSS Fonts Module Level 4 specification.

    HTML Image Decoding

    Content authors sometimes need a loaded, fully decoded image that is available immediately. Loading and decoding large images or lots of images can cause page performance to suffer. The new HTMLImageElement.decode() function asynchronously starts decoding an image and returns a Promise that resolves when the decoded image is available. With it, developers now have control over when loaded images are decoded, without tying up the main thread. Developers can also declaratively indicate an image should load asynchronously using the decoding attribute on an <img> element.

    <img decoding="async">
    
    <img decoding="sync">
    
    <img decoding="auto">
    

    For more information, read the WhatWG proposal.

    Passive Event Listeners

    To avoid causing poor scroll performance when attaching event handlers to touchstart, touchmove, or wheel events, developers can specify their event listener as passive if the listener does not call preventDefault(). This enables Safari to continue to provide high performance scrolling without waiting for the event listener to finish.

    More details on Passive Events Listeners can be found in the DOM specification.

    Storage Access API

    The new Storage Access API provides a solution to allow third-party embeds to request access to their first-party cookies when the user interacts with them in a world with Intelligent Tracking Prevention. This gives content providers the flexibility to provide embedded content, while offering users transparency about who they are sharing their information with.

    Find out more about the Storage Access API on the WebKit blog.

    Subresource Integrity

    WebKit has added support for Subresource Integrity which enables websites to validate the integrity of resources and reject them if something has changed outside the site owner’s control. An integrity check compares a hash provided by the web content author in an integrity attribute on a <script> or a <link> element with a computed hash of the delivered resource. When the hashes don’t match, instead of applying a stylesheet, or executing a script, a network error is returned. This helps web content authors reduce impact a compromised content delivery network (CDN).

    Read the Subresource Integrity specification to learn more.

    WebDriver Improvements

    Safari includes several improvements to the implementation of WebDriver. With Safari 11.1, WebDriver now supports file uploads and testing WebRTC functionality. In addition, several bug fixes improve the overall quality of the WebDriver implementation.

    Web Inspector Improvements

    Web Inspector has honed the design of several existing tools, and also adds a useful new tool to inspect HTML Canvas elements. The new Canvas tab allows web developers to understand the changes in recorded frames of canvas animations.

    The Styles sidebar in the Elements tab was reworked to use a different, but familiar model for editing style rules, properties and values. It also features improvements in navigating between different style views.

    Finally, the Network tab was redesigned, improving both the timeline of loading resources, and navigating to the network details of each resource such as content previews, headers, cookies, size and timing details.

    Feedback

    Most of these improvements are available to users running iOS 11.3 and macOS High Sierra 10.13.4, as well as macOS Sierra and OS X El Capitan. The Storage Access API is only available on iOS 11.3 and macOS High Sierra 10.13.4. These features were also available to web developers with Safari Technology Preview releases. Changes in this release of Safari were included in the following Safari Technology Preview releases: 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47. Download the latest Safari Technology Preview release to stay on the forefront of future web features. You can also use the WebKit Feature Status page to watch for changes to your favorite web platform features.

    We love hearing from you. Send a tweet to @webkit or @jonathandavis to share your thoughts on this release, and any features you were hoping for that didn’t make it. If you run into any issues, we welcome your bug reports for Safari, or WebKit bugs for web content issues.

    April 12, 2018 05:00 PM

    April 05, 2018

    Introducing the Payment Request API for Apple Pay

    Surfin’ Safari

    We’re pleased to announce that Safari 11.1 on macOS and Safari on iOS 11.3 now support the W3C Payment Request API for conducting Apple Pay transactions on the web.

    We first introduced Apple Pay on the Web in macOS Sierra and iOS 10. Apple Pay brought novel ease-of-use, security, and privacy to online transactions, so we’ve been encouraged to see it widely adopted by merchants in the years since. However, we recognize that merchants need to support multiple payment methods, and doing so comes at the cost of increased complexity. Payment Request aims to reduce this complexity by supporting payment methods across multiple browsers using a standard API.

    Today I’ll take you step-by-step through an Apple Pay transaction using Payment Request. If you’ve already worked with Apple Pay JS, this should look familiar to you. If you haven’t, this is a great place to get started!

    Prerequisites

    This post assumes that you have already registered a merchant identifier and set up your server environment to process Apple Pay transactions. You can find detailed instructions for doing this in the Apple Pay on the Web documentation.

    Showing Apple Pay Buttons

    Apple Pay transactions are always initiated by your customer, so you’ll need to display an Apple Pay button for them to tap or click. For a consistent visual appearance, WebKit provides built-in support for rendering Apple Pay buttons. Here’s how to display the most basic type of Apple Pay button:

    <button style="-webkit-appearance: -apple-pay-button;"></button>
    

    There are several other types of buttons available in various styles. Here’s a few that you might have seen while shopping in Safari:

    Apple Pay Buttons

    See Displaying Apple Pay Buttons for more information on the button types and styles available.

    According to the Apple Pay on the Web Human Interface Guidelines, you should display Apple Pay buttons whenever your customer is using a supported device. To check if your customer’s device supports Apple Pay, call ApplePaySession.canMakePayments():

    if (window.ApplePaySession && ApplePaySession.canMakePayments())
        // Show Apple Pay button.
    

    If your customer taps or clicks an Apple Pay button, you should always present the Apple Pay payment sheet. If they haven’t yet enrolled a payment card you can accept in Apple Pay, Safari prompts them to do so before continuing with your transaction. Since these rules are specific to Apple Pay, Apple Pay JS’s ApplePaySession API is required for this step. The remainder of the transaction can be conducted using standard Payment Request API.

    Constructing a PaymentRequest

    When your customer taps or clicks an Apple Pay button, you initiate the transaction by constructing a new PaymentRequest:

    var request = null;
    if (window.PaymentRequest)
        request = new PaymentRequest(methods, details, options);
    else
        // Consider using Apple Pay JS instead.
    

    The PaymentRequest constructor takes three arguments: the payment methods you support, the details to show to your customer (like shipping options and total amount), and any options you require.

    Payment Methods

    Payment methods represent the means by which you can accept payments from your customer using Payment Request. You specify the payment methods you accept as a sequence of PaymentMethodData dictionaries, each of which contains an identifier (supportedMethods) and associated data.

    To use Apple Pay with Payment Request, include it as a payment method. Apple Pay’s URL-based payment method identifier is "https://apple.com/apple-pay", and its associated data is an ApplePayRequest dictionary. Here’s what a PaymentMethodData dictionary for Apple Pay might look like:

    const applePayMethod = {
        supportedMethods: "https://apple.com/apple-pay",
        data: {
            version: 3,
            merchantIdentifier: "merchant.com.example",
            merchantCapabilities: ["supports3DS", "supportsCredit", "supportsDebit"],
            supportedNetworks: ["amex", "discover", "masterCard", "visa"],
            countryCode: "US",
        },
    };
    

    Safari only supports the Apple Pay payment method, but other browsers might support additional payment methods. When you specify multiple payment methods in a request, the browser decides which to present to your customer based on device availability and user preference.

    Payment Details

    Payment details are represented by the PaymentDetailsInit dictionary. It contains your transaction’s total amount, display items, shipping options, and payment method-specific modifiers (more on modifiers below). Here’s what valid payment details might look like for a $20 item plus tax and two shipping options:

    const paymentDetails = {
        total: {
            label: "My Merchant",
            amount: { value: "27.50", currency: "USD" },
        },
        displayItems: [{
            label: "Tax",
            amount: { value: "2.50", currency: "USD" },
        }, {
            label: "Ground Shipping",
            amount: { value: "5.00", currency: "USD" },
        }],
        shippingOptions: [{
            id: "ground",
            label: "Ground Shipping",
            amount: { value: "5.00", currency: "USD" },
            selected: true,
        }, {
            id: "express",
            label: "Express Shipping",
            amount: { value: "10.00", currency: "USD" },
        }],
    };
    

    You can choose a default shipping option by setting the selected attribute to true as we did above for "Ground Shipping". The total amount must not be negative, and when using Apple Pay, all payment amounts in your request must use the same ISO 4217 currency code. It is up to you to ensure the correctness of your payment details; Safari does not do any currency calculations on your behalf.

    What about modifiers?

    You can optionally include a sequence of modifiers in your payment details. Modifiers update your transaction’s display items and total when criteria you specify for a given payment method are satisfied. In Apple Pay, you can use modifiers to adjust the price based on the type of payment card selected in the Apple Pay payment sheet. For instance, the following modifier applies a $1.00 discount when your customer selects a debit card in Apple Pay:

    const debitModifier = {
        supportedMethods: "https://apple.com/apple-pay",
        data: { paymentMethodType: "debit" },
        total: {
            label: "My Merchant",
            amount: { value: "26.50", currency: "USD" },
        },
        additionalDisplayItems: [{
            label: "Debit Card Discount",
            amount: { value: "-1.00", currency: "USD" },
        }],
    };
    

    Modifiers provide some of the functionality present in the paymentmethodselected event from Apple Pay JS. See ApplePayModifier for more information.

    Payment Options

    Payment options are represented by the PaymentOptions dictionary. If you need to request your customer’s name, email address, or phone number – or request a certain type of shipping – you can do so here:

    const paymentOptions = {
        requestPayerName: true,
        requestPayerEmail: true,
        requestPayerPhone: true,
        requestShipping: true,
        shippingType: "shipping",
    };
    

    If you set requestShipping to true, the shipping options you specified in Payment Details are presented in the payment sheet for your customer to choose between. You receive the requested information once your customer authorizes payment.

    Exceptions

    Safari might raise an exception when constructing a new PaymentRequest. Exceptions can occur for the following reasons:

    • The frame is not in a secure context.
    • The frame is a cross-origin subframe.
    • No payment methods were specified.
    • A payment method identifier is invalid.
    • Calling JSON.stringify() on the payment method data failed.
    • Invalid currency amounts were specified (e.g., negative total or multiple currencies).

    canMakePayment() method

    Once you’ve constructed a PaymentRequest, you can ask it if your customer will be able to authorize a transaction given the payment methods you can accept. You do this by calling the canMakePayment() method, which returns a promise that resolves to either true or false. In Safari, when Apple Pay is one of the payment methods, canMakePayment() resolves to true only if your customer has an active card enrolled in Apple Pay. This is the equivalent of how ApplePaySession.canMakePaymentsWithActiveCard() behaves in Apple Pay JS.

    As we discussed in Showing Apple Pay Buttons, the Apple Pay Human Interface Guidelines require you to show an Apple Pay button whenever your customer is on supported hardware, whether or not they have an active card enrolled. Therefore, you should not hide Apple Pay buttons when canMakePayment() resolves to false. Always show an Apple Pay button if ApplePaySession.canMakePayments() returns true, and always present the Apple Pay payment sheet when your customer taps or clicks the button. Safari prompts your customer to enroll a payment card if they haven’t done so already before continuing with your transaction. For more information, see Human Interface Guidelines > Apple Pay on the Web.

    show() method

    When your customer taps or clicks an Apple Pay button, you should present the Apple Pay payment sheet. You do this by calling the show() method, which returns a promise that resolves to a PaymentResponse once your customer authorizes payment. The promise is rejected with an AbortError if your customer cancels the transaction.

    You can optionally call show() with a promise for a PaymentDetailsUpdate. Sometimes you might still be in the process of calculating payment details when your customer taps or clicks the Apple Pay button. In this case, you can construct a new PaymentRequest with placeholders for details, then call show() with a promise to provide up-to-date details later. When you resolve this promise, Safari displays the updated details in the Apple Pay payment sheet.

    Safari might reject the promise returned by show() with an exception. Exceptions can occur for the following reasons:

    • show() was not triggered by user activation (e.g., a tap or click).
    • The request has already been aborted.
    • An Apple Pay session is already active.
    • Payment method data is invalid (e.g., is missing required fields).

    abort() method

    If you need to abort the presented transaction, you can call the abort() method. When you do this, Safari dismisses the Apple Pay payment sheet and rejects the promise returned by show() with an AbortError. If the transaction has already been aborted, or show() has not yet been called, calling abort() throws an InvalidStateError.

    Merchant Validation

    Before Safari can present the Apple Pay payment sheet, you must acquire a payment session from Apple. This process is referred to as merchant validation.

    Soon after you call show(), Safari dispatches the merchantvalidation event to your PaymentRequest object. The event defines a validationURL attribute representing the Apple URL your server contacts to receive a payment session. You must call the event’s complete() method with a promise that you resolve with this payment session once you receive it.

    Here is what a merchantvalidation event handler might look like:

    request.onmerchantvalidation = function (event) {
        const sessionPromise = fetchPaymentSession(event.validationURL);
        event.complete(sessionPromise);
    };
    

    You can learn more about merchant validation from Requesting an Apple Pay Payment Session.

    Shipping Events

    Once you’ve received a merchant session, Safari presents the Apple Pay payment sheet to your customer. If you’ve requested shipping, your customer is able to select between your shipping options and provide a shipping address. When they make these selections in the payment sheet, Safari dispatches a shippingoptionchange or shippingaddresschange event to your PaymentRequest object.

    shippingoptionchange

    When the user selects a shipping option, Safari dispatches the shippingoptionchange event. In your event handler, you can determine the selected shipping option by checking the PaymentRequest‘s shippingOption attribute. To update the payment details based on the selected shipping option, call updateWith() on the event object with a promise that resolves to a PaymentDetailsUpdate.

    When requesting shipping with Apple Pay, you must always listen for shippingoptionchange and call updateWith() with a promise that resolves within 30 seconds, otherwise, the transaction will time out.

    shippingaddresschange

    When your customer selects a shipping address, Safari dispatches the shippingaddresschange event. In your event handler, you can determine the selected shipping address by checking the PaymentRequest‘s shippingAddress attribute. To update the payment details based on the selected shipping address, call updateWith() on the event object with a promise that resolves to a PaymentDetailsUpdate. If you are unable to ship to the selected address, you can provide an error message in your PaymentDetailsUpdate that Safari displays to your customer.

    When using Apple Pay, Safari might redact some details from the shipping address. For instance, in the United States, only city, state, 5-digit ZIP code, and country are provided. Safari provides the full, un-redacted shipping address once your customer authorizes payment.

    When requesting shipping with Apple Pay, you must always listen for shippingaddresschange and call updateWith() with a promise that resolves within 30 seconds, otherwise the transaction will time out.

    Handling Payment Authorization

    When your customer authorizes payment, Safari resolves the promise you received from calling show() with a PaymentResponse. Depending on what you requested in your PaymentOptions, the response might contain the selected shipping option, shipping address, name, email, and phone number of your customer.

    The response also contains the payment method identifier used to conduct the transaction (methodName), along with its associated details. When Apple Pay is the selected payment method, the associated details is an ApplePayPayment dictionary. ApplePayPayment contains the Apple Pay token you use to process the payment authorization. It also includes your customer’s billing and shipping contact information as ApplePayPaymentContacts if you required this in your ApplePayRequest.

    When you have finished processing the payment authorization, you call the complete() method on PaymentResponse to indicate the result of your processing. You can call complete() with a status of "success" or "failure". At this point, the Apple Pay payment sheet is dismissed.

    Demo

    You now have all the pieces you need to conduct an Apple Pay transaction using Payment Request. Here’s what an Apple Pay session might look like using Payment Request:

    async function applePayButtonClicked()
    {
        // Consider falling back to Apple Pay JS if Payment Request is not available.
        if (!window.PaymentRequest)
            return;
    
        try {
            const request = new PaymentRequest([applePayMethod], paymentDetails, paymentOptions);
    
            request.onmerchantvalidation = function (event) {
                // Have your server fetch a payment session from event.validationURL.
                const sessionPromise = fetchPaymentSession(event.validationURL);
                event.complete(sessionPromise);
            };
    
            request.onshippingoptionchange = function (event) {
                // Compute new payment details based on the selected shipping option.
                const detailsUpdatePromise = computeDetails();
                event.updateWith(detailsUpdatePromise);
            };
    
            request.onshippingaddresschange = function (event) {
                // Compute new payment details based on the selected shipping address.
                const detailsUpdatePromise = computeDetails();
                event.updateWith(detailsUpdatePromise);
            };
    
            const response = await request.show();
            const status = processResponse(response);
            response.complete(status);
        } catch (e) {
            // Handle errors
        }
    }
    

    Let’s see how this works in a live demo. If you are viewing this post on a device capable of Apple Pay, you should see an Apple Pay button below. Feel free to click it! Don’t worry, no matter what you do in the payment sheet, your card won’t be charged anything.

    Availability

    The Payment Request API is available in Safari 11.1 on macOS Sierra and macOS High Sierra, Safari on iOS 11.3, and Safari Technology Preview.

    More Information

    Apple provides comprehensive documentation for Apple Pay on the Web. Here are a few links you might find useful:

    Feedback

    We’d love to hear your feedback! If you find a Payment Request bug, please report it at bugs.webkit.org. On Twitter, you can reach the WebKit team at @webkit, or our web technologies evangelist Jonathan Davis at @jonathandavis.

    April 05, 2018 05:00 PM