safari ios vp9

HTML5 / Tizen / webOS SDK
Android / FireTV SDK
iOS / tvOS SDK
React Native SDK
Flutter SDK
Test Your Stream
Supported Platforms
Open Video UI
Sports Betting
Video streaming for smart TV
Low latency live streaming
Cross-platform playback
Playback on set-top boxes
High efficiency streaming
React Native OTT Apps
Flutter OTT Apps
Getting started
Platforms & SDKs
API reference
Sample apps
Customer Stories
Market Research Report 2024

The VP9 Breakthrough: VP9 Support Now Possible on Apple Devices and All Major Platforms

by Pieter-Jan Speelmans on April 19, 2023

Over the past week, during my early mornings at NAB, I’ve been exploring the potential of running the VP9 codec on all platforms. Despite my sleep-deprived, jet-lagged state, I’ve discovered some exciting news: it's now possible to use VP9 everywhere, making it possible to thank H.264 for its services and move on. Although I admit this article might come across as unorganized, my enthusiasm for the findings was too great to keep to myself.

AVC/h.264 has been a reliable codec for the past 19 years. Originally published on August 17, 2004, it remains the go-to choice for ensuring your content can play on any device. However, from an efficiency standpoint, it's outdated technology. Many improvements have been made since its inception.

The codecs that followed H.264, H.265/HEVC, and VP9, released on June 7 and 17, 2013, respectively, have yet to achieve widespread adoption. The main reason is that device coverage isn't as comprehensive, necessitating a dual codec approach that increases encoding and storage costs. In addition, H.265 is also plagued by licensing issues, making it unclear who needs to be paid to use it (and that’s not even mentioning the Sisvel VP9/AV1 claims).

The H.265 and VP9 generation of codecs boasts impressive gains in compression efficiency. One could wonder why they are not often used. While I prefer to stay away from the (almost religious) discussions on which codecs are best and what the exact efficiency gains are, H.265 and VP9 seem to have efficiency gains of around 40-50%. If you’re storing a large library of video or streaming huge quantities of it, it’s no surprise the potential cost reduction is massive. There will be an increase in encoding cost using a more modern codec (my numbers seem to indicate a 3x increase in compute requirements for VP9), but when you don’t have a long tail of rarely viewed and only briefly stored videos, this should be offset easily. Platform support has however always been a significant limitation, particularly for Apple devices such as iPhones with iOS.

Guided by some insights shared by industry friends, I discovered that HLS with VP9 has been enabled on iOS by Apple, although not publicly acknowledged. One of the clear hints can be found when analyzing YouTube’s traffic to iPhone devices. YouTube seems to stream HLS with VP9 to iPhone devices, playable in Safari's video element. While it’s unclear how long this has been going on, this observation triggered me big time.

Did Apple change its position on VP9? No, at least not publicly. Former TN2224, now known as the HTTP Live Streaming (HLS) Authoring Specification for Apple Devices , still mentions you should use h.264 and h.265 to operate on iOS. That’s where NAB is great. Over the past week, I’ve been able to discuss from various angles if this is just a fluke, or if we are seeing intentional behavior here. The only conclusion I can come to is that this is absolutely an intentional change. Given VP9 support in iOS is now a real thing, I see no reason to not leverage this.

Using this information as a guide, I dug deeper, discovering that VP9 in ISOBMFF/CMAF/MP4 containers works well with HLS in a native way. Additionally, VP9 support on iPadOS and macOS, though less straightforward than on iPhone, is still achievable with HLS. I was even able to implement this into our player product, resulting in stunning performance and the ability to play VP9 everywhere. To make things even better, I already implemented the HLS with AV1 combination in some forward compatibility for those platforms where AV1 is already enabled (Apple IS a part of AOM after all, and indications are good that support for this is coming as well, but I did not find a way to enable this on iOS… yet at least).

Furthermore, after running new builds of our player on different devices using our automated testing platform, I found that our current changes enable VP9 in HLS on Tizen and webOS. Indications are strong that we can extend this support to all older devices we support, including Tizen 2.3 (2015 TVs) and webOS 3.0 (2016 devices). It's not surprising that VP9 works flawlessly on other major platforms such as Android TV, FireTV, and Roku.

Based on my current observations, it's now possible to run HLS streams with VP9 in ISOBMFF on all major platforms supported by our player product. Given that we support nearly every relevant platform for the bulk of the streaming services out there, that’s pretty good. (Note to self, we should update Can I Use , which currently lists a support rate of "80.13% +17.21% = 97.34." where the 17.21% indicates only partial support on iOS for webRTC.)

Some further work I’m doing is validating all of this with DRM and perhaps pumping this into the HESP protocol to do VP9 in live with sub-second latency glass-to-glass. Indications are very good that this will work as well, and I expect to have an update here soon once I get the time to craft some streams.

I hope this article didn’t come over as too disorganized (but hey, I warned you at the start), but I must admit I’m quite excited about the findings here. Let’s thank H.264 for its great service over the past years and look towards finally using some more efficient encoding.

THEOplayer (45)
online streaming (40)
live streaming (35)
low latency (32)
video streaming (32)
new features (21)
THEO Technologies (19)
THEOlive (17)
best video player (17)
html5 player (16)
LL-HLS (15)
cross-platform (15)
online video (15)
SmartTV (12)
delivering content (12)
MPEG-DASH (11)
latency (11)
partnership (11)
Samsung (10)
awards (10)
content monetisation (10)
innovation (10)
Big Screen (9)
High Efficiency Streaming Protocol (9)
fast zapping (9)
video codec (9)
Ultra Low Latency (8)
advertising (8)
viewers expercience (8)
"content delivery" (7)
Adobe flash (7)
Online Advertising (7)
Streaming Media Readers' Choice Awards (7)
low bandwidth (7)
Efficiency (6)
interactive video (6)
sports streaming (6)
video content (6)
viewer experience (6)
Bandwidth Usage (5)
Deloitte (5)
ad revenue (5)
adaptive bitrate (5)
nomination (5)
reduce buffering (5)
release (5)
sports betting (5)
video monetization (5)
Encoding (4)
THEO Technologies Partner Success Team (4)
case study (4)
content encryption (4)
content protection (4)
fast 50 (4)
monetization (4)
nab show (4)
streaming media west (4)
support matrix (4)
AES-128 (3)
Cost Efficient (3)
HESP Alliance (3)
IBC trade show (3)
THEOplayer Partner Success Team (3)
Year Award (3)
client-side ad insertion (3)
content integration (3)
customer case (3)
customise feature (3)
dynamic ad insertion (3)
scalable (3)
video trends (3)
"network api" (2)
Amino Technologies (2)
Android TV (2)
CSI Awards (2)
Encryption (2)
LL-DASH (2)
Microsoft Silverlight (2)
Press Release (2)
React Native SDK (2)
Start-Up Times (2)
VPAID2.0 (2)
ad block detection (2)
ad blocking (2)
ads in HTML5 (2)
analytics (2)
android (2)
captions (2)
chromecast (2)
chromecast support (2)
clipping (2)
closed captions (2)
deloitte rising star (2)
fast500 (2)
frame accurate clipping (2)
frame accurate seeking (2)
metadata (2)
multiple audio (2)
playback speed (2)
plugin-free (2)
pricing (2)
seamless transition (2)
server-side ad insertion (2)
server-side ad replacement (2)
subtitles (2)
video publishers (2)
viewer engagement (2)
"smooth playback" (1)
360 Video (1)
Best of Show (1)
CEA-608 (1)
CEA-708 (1)
HLS stream (1)
Microsoft Azure Media Services (1)
Monoscopic (1)
NAB Show 2016 (1)
NetOn.Live (1)
Periscope (1)
React Native (1)
Real-time (1)
Scale Up of the Year award (1)
Seeking (1)
Stereoscopic (1)
Swisscom (1)
TVB Europe (1)
Tech Startup Day (1)
Telenet (1)
Uncategorized (1)
University of Manitoba (1)
User Interface (1)
Vivaldi support (1)
adblock detection (1)
apple tv (1)
autoplay (1)
facebook html5 (1)
faster ABR (1)
iGameMedia (1)
iOS SDK (1)
language localisation (1)
micro moments (1)
mobile ad (1)
nagasoft (1)
new web browser (1)
offline playback (1)
preloading (1)
program-date-time (1)
stream problems (1)
streaming media east (1)
support organization (1)
thumbnails (1)
use case (1)
video clipping (1)
video recording (1)
video trends in 2016 (1)
visibility (1)
vulnerabilities (1)
zero-day exploit (1)
January 2024 (1)
December 2023 (2)
September 2023 (1)
July 2023 (2)
June 2023 (1)
April 2023 (4)
March 2023 (2)
December 2022 (1)
September 2022 (4)
July 2022 (2)
June 2022 (3)
April 2022 (3)
March 2022 (1)
February 2022 (1)
January 2022 (1)
November 2021 (1)
October 2021 (3)
September 2021 (3)
August 2021 (1)
July 2021 (1)
June 2021 (1)
May 2021 (8)
April 2021 (4)
March 2021 (6)
February 2021 (10)
January 2021 (4)
December 2020 (1)
November 2020 (1)
October 2020 (1)
September 2020 (3)
August 2020 (1)
July 2020 (3)
June 2020 (3)
May 2020 (1)
April 2020 (3)
March 2020 (4)
February 2020 (1)
January 2020 (3)
December 2019 (4)
November 2019 (4)
October 2019 (1)
September 2019 (4)
August 2019 (2)
June 2019 (1)
December 2018 (1)
November 2018 (3)
October 2018 (1)
August 2018 (4)
July 2018 (2)
June 2018 (2)
April 2018 (1)
March 2018 (3)
February 2018 (2)
January 2018 (2)
December 2017 (1)
November 2017 (1)
October 2017 (1)
September 2017 (2)
August 2017 (3)
May 2017 (3)
April 2017 (1)
March 2017 (1)
February 2017 (1)
December 2016 (1)
November 2016 (3)
October 2016 (2)
September 2016 (4)
August 2016 (3)
July 2016 (1)
May 2016 (2)
April 2016 (4)
March 2016 (2)
February 2016 (4)
January 2016 (2)
December 2015 (1)
November 2015 (2)
October 2015 (5)
August 2015 (3)
July 2015 (1)
May 2015 (1)
March 2015 (2)
January 2015 (2)
September 2014 (1)
August 2014 (1)

Subscribe by email

Home New Posts Forum List Trending New Threads New Media Spy
WikiPost Latest summaries Watched WikiPosts
Support FAQ and Rules Contact Us

iOS/iPadOS 14 and VP9 support [Merged]

Thread starter EugW
Start date Jun 26, 2020
Tags 4kvideo ipad ipados14 youtube
Sort by reaction score
iPhone, iPad, and iPod Touch
iOS and iPadOS
Older iOS Versions

macrumors G5

Jun 26, 2020

Some questions about VP9 support in iOS 14 and iPadOS 14: 1. This is hardware VP9 support, right? 2. If hardware VP9, which hardware? How far back in terms of Ax chips is this VP9 support included? 3. If VP9 is not supported in hardware, is there software fallback? Do the iPad Air 2 and iPad mini 4 support VP9 in some form? 4. Is the VP9 support only with YouTube, or can other applications make use of this? 5. Does the VP9 support work in Safari and other browsers in video players (including embedded YouTube)? 6. Can you drag and drop VP9 in to iOS/iPadOS 14 video editors with full hardware decode support? 7. Is VP9 HDR supported? If so, is there automatic HDR to SDR dithering (for iDevices that need it)?

macrumors 68020

Jun 27, 2020

melancholy said: Yes Click to expand...

rui no onna

Contributor.

Jun 28, 2020

FaceTime is 720p max, though. We know that A9 doesn’t have HEVC hardware encode. Possible that those earlier models are just handling it via software. > 2. If hardware VP9, which hardware? How far back in terms of Ax chips is this VP9 support included? Here’s one data point: 10.2” iPad 7th gen (Apple A10, 3GB RAM, 2160x1620) still limited to 1080p on iPadOS 14 dev beta 1

macrumors newbie

rui no onna said: Here’s one data point: 10.2” iPad 7th gen (Apple A10, 3GB RAM, 2160x1620) still limited to 1080p on iPadOS 14 dev beta 1 Click to expand...

The Game 161

Macrumors nehalem.

vobele said: My 12.9 iPad Pro 2020 with iPadOS 14 dev beta 1 -> does still NOT play any YouTube videos in higher than FullHD My iPhone 11 Pro Max with iPadOS 14 dev beta 1 -> this one DOES play YouTube videos in higher than FullHD res. Anyone a clue for what reason? Thanks Click to expand...

how do you tested, in youtube app?

wortex said: how do you tested, in youtube app? Click to expand...

ok, so on my iphone xs max youtube 2k, on ipad pro 10.5 1080p, so iphone only

Yes, exactly. Strange they test it on smaller devices first. Hopefully it makes it‘s way into both final OS releases.

wortex said: ok, so on my iphone xs max youtube 2k, on ipad pro 10.5 1080p, so iphone only Click to expand...

rui no onna said: FaceTime is 720p max, though. We know that A9 doesn’t have HEVC hardware encode. Possible that those earlier models are just handling it via software. Click to expand...

rui no onna said: Hmm, maybe they enabled it on iPhones first? Click to expand...

Has anyone else noticed that most YouTube videos on iPadOS 14 can playback 2160p?

EugW said: I'd be shocked if the A8 iPhone 6 handled FaceTime HEVC encode in software. The chip is too slow, even at 720p. I'd guess the hardware is present in A9 too, just disabled via software. P.S. Some have postulated it's AV1 not VP9, but I'm not convinced about that since the official spec of AV1 only came out in 2018, and A10 came out in 2016. I can't rule that out though. Click to expand...

rui no onna said: As for AV1 vs VP9, do we actually have confirmed reports of YouTube 4K on a device with A10 chipset? On this thread, we have XS Max and 11 Pro Max. Click to expand...

EugW said: I mentioned above that it works on AppleTV 4K with A10X. Technically A10X didn't come out until 2017, but it would have to have been in development for years before that, and as mentioned the AV1 spec was not finalized until 2018. Click to expand...

macrumors 6502a

Jun 29, 2020

rui no onna said: FaceTime is 720p max, though. We know that A9 doesn’t have HEVC hardware encode. Possible that those earlier models are just handling it via software. > 2. If hardware VP9, which hardware? How far back in terms of Ax chips is this VP9 support included? Here’s one data point: 10.2” iPad 7th gen (Apple A10, 3GB RAM, 2160x1620) still limited to 1080p on iPadOS 14 dev beta 1 Click to expand...

Homme said: But now with the new OS, FaceTime now can do 1080p, depending on the Models who’s front facing camera can record in 1080p Click to expand...

macrumors regular

EugW said: Some questions about VP9 support in iOS 14 and iPadOS 14: 1. This is hardware VP9 support, right? 2. If hardware VP9, which hardware? How far back in terms of Ax chips is this VP9 support included? 3. If VP9 is not supported in hardware, is there software fallback? Do the iPad Air 2 and iPad mini 4 support VP9 in some form? 4. Is the VP9 support only with YouTube, or can other applications make use of this? 5. Does the VP9 support work in Safari and other browsers in video players (including embedded YouTube)? 6. Can you drag and drop VP9 in to iOS/iPadOS 14 video editors with full hardware decode support? 7. Is VP9 HDR supported? If so, is there automatic HDR to SDR dithering (for iDevices that need it)? Click to expand...

Birkan said: I think 4K on YouTube only works on HDR videos such as this one (see the screenshot below) using VP9.2 codec which is used for HDR videos on YouTube. I suspect that you probably need a HDR capable device to watch it. If your device is not HDR compatible, you won't be shown the option for it. Unfortunately, they do not appear to support normal SDR 4K videos on iOS or iPadOS yet. At least, I couldn't get them to work yet on my iPhone Xs running iOS14 DP1. Also, YouTube on Safari only goes up to 720p using H.264 so Safari doesn't seem to take advantage of this functionality yet. They recently added HDR media capabilities to Safari on v14 but they do not mention if it's only limited to HEVC. View attachment 929083 Click to expand...

rui no onna said: That video and Costa Rica 4K were two of the videos I tried to play on iPadOS 14 DB1. Tried on both a 10.2” iPad 7th gen as well as an iPad Pro 12.9 (2nd gen, 2017). Safari and YouTube app only allowed up to 1080p60. The iPad 10.2 doesn’t support HDR content but the iPP 12.9 should support HDR10 and Dolby Vision playback. Click to expand...

macrumors 68040

Cult of Mac has also written about 4K YouTube videos being available on iPhone/iPad too. They didn’t provide any new information or proof that it’s rolling out to all .4K videos though.

xxray said: I’m able to watch the Costa Rica video and this Peru video in 4K, 60fps, HDR on my iPhone XS Max. They’re both the VP9.2 codec. Not able to on my A10X iPad Pro 10.5” though that’s supposed to also be HDR compatible. View attachment 929139 Click to expand...

EugW said: How are you showing the specs in iOS? Or was that on a computer? Click to expand...

macrumors 65816

We have a 11” and 12.9” iPad Pro 2020 model running the beta as well as my iPhone 11 Pro Max. The 11 Pro Max has 4K youtube while logged into my YT Premium account. Weirdly, both iPads won’t show 4k video as an option if I’m logged into the same YT account, but if I log out, boom, there’s 4K. Weird. Looks gorgeous, especially on the 12.9“ iPP, and the iPads don’t seem to be breaking a sweat even with multitasking. I’d have to imagine this selective appearance is all beta bug based and that this is likely worthy of an official formal announcement in the fall.

Agora Releases VP9 Video Support for Safari

Agora is excited to be the first real-time video platform-as a-service (PaaS) provider to release full support for VP9 in browsers including Safari. Full VP9 support comes with the release of Web SDK 14.9.2. VP9 provides twice the quality of the VP8 codec for the same bitrate. In my 12 years of WebRTC expertise this is a truly impressive milestone for the industry.

What is VP9 and why is it important?

VP9 is a video coding format developed by Google. VP9 is the successor to VP8, which is currently the default for real-time video on the web. VP9 enables twice the compression of VP8 and is customized for video resolutions greater than 1080p (such as UHD).

VP9 is a significant advancement in the world of video codecs for several reasons:

Improved Compression: VP9 provides better compression than its predecessor (VP8) and is often compared favorably to HEVC/H.265 in terms of compression efficiency. Better compression means smaller file sizes without compromising video quality, which leads to faster streaming and reduced bandwidth usage.
Adaptive Streaming: VP9 is well-suited for adaptive bitrate streaming, such as YouTube’s Dynamic Adaptive Streaming over HTTP (DASH). This adaptability ensures that videos stream smoothly across various network conditions.
Support for 4K and Beyond: VP9 is designed to handle high-resolution video, making it an excellent choice for 4K streaming and even resolutions beyond that.
Broad Adoption: Major platforms, like YouTube, have adopted VP9 due to its efficiency, leading to a significant portion of internet traffic being encoded in VP9.
Power Efficiency: For mobile devices and other battery-powered gadgets, VP9 is designed to decode efficiently, conserving power and extending battery life.
Web Integration: Being a product from Google, VP9 has robust support in browsers, particularly in Chrome. This integration is essential for the web, especially for platforms that rely heavily on video content.

In summary, VP9 is a big deal because it represents a combination of cost-saving (due to being royalty-free), technological advancement (with improved compression and resolution support), and widespread industry adoption, all of which benefits both content creators and consumers.

Challenges with VP9 on Safari

While VP9 has been around since 2016, it has presented challenges with a lack of hardware support and Safari browser support. To this day, Safari does not provide network and compute adaptation other than to reduce the outgoing frame-rate and bit-rate. This means that a video stream coming from an iOS device will not look good on the receiving end when the bitrate drops or the device isn’t capable of encoding 720p video at 30fps.

Comparing VP9 to VP8

This comparison shows video coming from two identical iPhone 11s connected to the same Wi-Fi access point in a high packet loss office environment. On the left is VP8 and on the right is VP9. The bit-rate is limited to 1mbps on each device and the difference in quality is impressive.

Agora now offers full VP9 support for Safari

Agora has now solved the issues with VP9 in the Safari browser by providing dynamic resolution scaling on Safari 16.0 and later versions for desktop and mobile to adapt to the needs of a low-compute device or limited uplink. This allows the VP9 picture in Safari to remain stable with a high frame rate and no blockiness as seen in Google Chrome and other browsers.

VP9 is the latest innovation from Agora that ensures video quality is maintained in all environments. Real-time video quality is vital for ensuring a seamless user experience, fostering clear and effective communication, and reflecting professionalism. In contexts ranging from business meetings to telemedicine, high-quality video is essential for accuracy, engagement, and credibility.

As technology advances, users increasingly expect superior video quality as a standard. It’s essential to provide a seamless user experience that fosters clear and effective communication and reduces visual strain during prolonged usage. To find out more of Agora’s other innovations in real-time video quality, space: Revolutionizing Live Video Quality: Agora Unveils Next-Gen Enhancements .

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Safari 15.0 VP9 support

I am using macOS Catalina due to design preferences. But Safari has updated to version 15.0, which claims support for the VP9 codec, which allows you to watch 4K content on YouTube. For some reason, I do not have such an opportunity - the maximum available resolution is 1080p. This is weird as my friend uses an iMac on Catalina and has 4K. My device is a MacBook Pro 16" late 2019.

I would appreciate your reply.

MacBook Pro 16″, macOS 10.15

Posted on Oct 18, 2021 1:18 PM

iOS/iPadOS VP9 Codec support

VideoToolbox

I tested on iPhone 13 with iOS 16.5 and VTIsHardwareDecodeSupported(kCMVideoCodecType_VP9); returns false. Which Apple device supports VP9 hardware decoding ?

Search Configure Global Search

Enable VP9 playback for Safari versions that support it Closed, Resolved Public BUG REPORT Actions

Edit Related Tasks...
Create Subtask
Edit Parent Tasks
Edit Subtasks
Merge Duplicates In
Close As Duplicate
Edit Related Objects...
Edit Commits
Mute Notifications
Protect as security issue
Award Token
Flag For Later
VideoJS player (Active)
Kaltura player (Backlog)
MW-1.38-notes (1.38.0-wmf.17; 2022-01-10)

Description

Apple added support for both opus and vp9 a while ago, but messed up the feature detection for it.

https://bugs.webkit.org/show_bug.cgi?id=229799

Highly disappointing. But I think I found a workaround, at the very least for videoJS. Instead of testing for video/webm; codecs="vp9,opus" we can test for video/webm; codecs="vp9" AND MediaSource support of audio/webm; codecs="opus" and this would basically the detect the Safari bug and recognise our capabilities.

Related Objects

Event timeline.

Change 751440 had a related patch set uploaded (by TheDJ; author: TheDJ):

[mediawiki/extensions/TimedMediaHandler@master] VP9 native playback for Safari

https://gerrit.wikimedia.org/r/751440

Test wiki created on Patch demo by TheDJ using patch(es) linked to this task:

https://patchdemo.wmflabs.org/wikis/b2bf2642d5/w/

https://patchdemo.wmflabs.org/wikis/ae09cc7a96/w/

Test wiki on Patch demo by TheDJ using patch(es) linked to this task was deleted :

https://patchdemo.wmflabs.org/wikis/d0160181fa/w/

https://patchdemo.wmflabs.org/wikis/6299cb60c4/w/

https://patchdemo.wmflabs.org/wikis/700f209163/w/

https://patchdemo.wmflabs.org/wikis/925641d4ab/w/

Hmm. harder than I figured for videojs. Somehow we will have to intercept nativeSourceHandler.canPlayType() and patch that, because the loader uses it as well. For mwEmbed, I'm not sure yet, since I cannot modify the config of patch demo...

https://patchdemo.wmflabs.org/wikis/6aec6259da/w/

OK, that seems to have done the trick. https://patchdemo.wmflabs.org/wikis/6aec6259da/wiki/File:Sneak_Preview_-_Wikipedia_VisualEditor.webm

issues and thoughts:

might want to add a canPlayType on video/webm; just to be sure the container format works ?
no browser detection on mwembed
switching src in both mwembed and videojs seems to pause the video element. This is probably a Safari specific problem... requires some investigation.

https://patchdemo.wmflabs.org/wikis/b7e23928ad/w/

https://patchdemo.wmflabs.org/wikis/b7e23928ad/wiki/File:Sneak_Preview_-_Wikipedia_VisualEditor.webm

autoplay issue fixed (kinda surprising btw that Chrome doesn't block this audioplay attempt)
issue that remains is source switching
should do some regression testing on older safari or older macos... but who keeps those around ? :D

What do you think about this Brion ?

Oh god, Apple what are you doing... So I can take a VP9 video and just have it playback on Safari 15.2 on my MacBook Pro 2016.. but not NOT with the WTP 137. Nvmd, I forgot I had enabled the software VP9 decoder, in Safari's experimental settings.

The src switching issue seems to be because the load algorithm isn't being properly done (on either). After switching the source, i get 0 events. I'm expecting loadedmetadata, loadeddata, canplay and timeupdate. I only receive loadedmetadata and then its silent. I'm filing a ticket about this with Apple. OK, this is actually a Chrome deviance.. We load with preload=metadata and Safari actually respects that (Chrome does not it seems, or maybe only on certain mobile situations). This is legal of Chrome, the setting is only a HINT for the user-agent. Anyway, because of this, there is never a loadeddata event, only a loadedmetadata event. We can patch this is resolution-switcher.

https://patchdemo.wmflabs.org/wikis/dff1f911ba/w/

With fixed source switching: https://patchdemo.wmflabs.org/wikis/dff1f911ba/wiki/File:Sneak_Preview_-_Wikipedia_VisualEditor.webm

I suspect older macs (like mine) and iOS use a software VP8 decoder... and that's why it's only available to webRTC... they need it there because as a encoder/decoder its more common than h264 for the protocol. But for normal video support they didn't need that (h264 was very big), and they want to support it there only if they can offer hardware acceleration... This makes sense, as battery life tanks with software decoders.

Change 751440 merged by jenkins-bot:

[mediawiki/extensions/TimedMediaHandler@master] VP8 and VP9 native playback for Safari

Apple to unveil AI-enabled Safari browser alongside new operating systems

Safari getting new AI tools in iOS 18

The software — expected to debut as Safari 18 later in 2024 — is currently undergoing evaluation alongside internal builds of Apple's next-generation operating system updates, namely iOS 18 and macOS 15, according to people familiar with the matter. Should all of the new features make it to the release candidate stage, users will be treated to a new user interface (UI) for customizing popular page controls, a "Web eraser" feature, and AI-driven content summarization tools.

Intelligent Search - AI-enhanced browsing and text summarization

Engineers evaluating the latest builds of Safari 18 can find a toggle for the new page controls menu within the browser's address bar. The menu consolidates — and provides quick access to — old and new page control tools, among them being the browser's new "Intelligent Search" feature.

A screenshot of Arc Search on iPhone with a summary of an article from AppleInsider

This feature is not automatically enabled in test builds of Safari 18 and instead requires manual activation from the page controls menu. Once enabled, it's believed the tool leverages Apple's on-device AI technology — specifically the Ajax language-learning model — to identify topics and key phrases within a webpage for which to base the summarization.

In selecting key phrases, Apple's LLM software identifies sentences that provide explanations or describe the structure of objects, depending on the text in question. Words repeated within a text and key sentences are recognized as text topics.

These early indications suggest Apple is striving to deliver text summarization alongside Safari 18 later this year, though the exact implementation of this feature remains fuzzy.

Apple's text summarization features could be a response to rival generative AI tools, such as OpenAI's ChatGPT. But Apple's on-device models and technologies like Private Relay could make the experience much more secure for users. Intelligent Search is also likely to be the same AI browser assistant that one X user hinted at a couple of weeks back.

Web Eraser for better content-blocking

Also accessible from the new page controls menu is a feature Apple is testing called "Web Eraser." As its name would imply, it's designed to allow users to remove, or erase, specific portions of web pages, according to people familiar with the feature.

The feature is expected to build upon existing privacy features within Safari and will allow users to erase unwanted content from any webpage of their choosing. Users will have the option to erase banner ads, images, text or even entire page sections, all with relative ease.

A screenshot of 1Blocker's element blocking feature that removes selected parts of a webpage

The erasure is said to be persistent, rather than limited to a browsing session. This means that Safari will remember the changes even after the original tab or window has been closed.

When visiting a web page with previously erased content, Safari will inform the user that the page has been modified to reflect their desired changes. The browser will also give the user the option to revert changes and restore the webpage to its initial, unaltered state.

As for where Apple got the inspiration for Web Eraser, the company could have Sherlocked the feature from the third-party app 1Blocker. The application features a similar way of erasing ads, where users would tap ads to make them disappear.

Updated UI will move key tools into one easy location

The new page controls menu referenced throughout this article will attempt to offer Safari 18 users quick and easy access to several options previously located across different menus and the Share Sheet. Running on pre-release versions of macOS 15, for instance, the menu also pulls in the "Aa" menu common on existing iPadOS versions of the browser. These include zoom options, webpage settings for privacy controls, content blocking options, extension shortcuts, and access to the new AI and erasure tools.

A mockup image of desktop Safari with an iPad-like menu that includes AI and erase features.

These various options existed previously on macOS, but were found by navigating through the Safari settings menu. Having all of these controls in one central location in the Address Bar will make these options more discoverable for end users. Other options like on-page text search and reader mode are also in this menu.

This all suggests Apple intends to unite the iPadOS and macOS Safari UI paradigms when it unveils its new OS releases during its Worldwide Developers Conference in June.

AI-enhanced Visual Lookup feature could be available in 2025

Simultaneously, Apple is also working on a much more powerful visual search feature scheduled for integration sometime in 2025 that will allow users to obtain information on consumer products when browsing through images. People familiar with the tool say it's similar to the Visual Lookup feature, through which Siri can identify plants, pets and landmarks from photos.

An image of a black cat with big yellow eyes in Apple Photos with text below classifying it as a 'bombay'

Visual Search could be implemented as a system-wide feature. So, instead of surfacing search results in the Photos app, the user may be able to use Visual Search directly in Safari by selecting an image.

Given development of this tool is in the earlier stages, it remains unclear how Apple will ultimately go about implementing it. It also remains to be seen whether or not the feature will make it through internal testing.

The iPhone maker has expressed significant interest in artificial intelligence and more recently published research related to on-device language learning models. This goes hand in hand with widespread rumors about several other AI-powered software features, many of which are expected to make their debut later this year with iOS 18.

Another big year for Safari

From its inception, Safari was always meant to rival existing web browsers.

A Flickr login screen with password auto fill shown as a pop-up

Safari was originally added to macOS as a means of replacing Microsoft's Internet Explorer for Mac, which was the default browser. Over the years, the app made its way to the iPhone , and has since received various upgrades over the years.

More recent versions of Safari provide users with personalized web suggestions, and allow for greater privacy protection with features such as iCloud Private Relay.

This latest set of enhancements only serves to better position Safari against the competition, in an ever-increasing landscape of generative AI tools and third-party ad-blockers.

While Apple tests many such features during different development stages, it is important to remember that not all of them make it to release. Apple has been known to delay features or cancel them entirely, so there's always a chance we may not see these new Safari features after all.

Along with Safari 18 and its associated improvements, we should also see a revamped Calculator with Math Notes make its way to iPadOS 18. Freeform Scenes and Adaptive Voice Shortcuts are new OS features, which are also expected to debut later this year.

Apple is expected to introduce various AI-related enhancements as part of iOS 18 and macOS 15, both of which should be announced at the company's annual WWDC on June 10.

36 Comments

"As for where Apple got the inspiration for Web Eraser, the company could have Sherlocked the feature from the third-party app 1Blocker. The application features a similar way of erasing ads, where users would tap ads to make them disappear." They may also have been inspired by Simplified Page....

As long as I can turn it off/hide it they can knock themselves out.

Will the web eraser feature remove pop up videos, such as the ones that appear on this site?

I use the 'Reader' function in Safari quite a bit, but sometimes it doesn't work very well. Seems like something that AI could help make better.

Web ads suck but how else do you think your favorite websites will stay in business? Are you prepared to pay a subscription to every site? I doubt it…

Blowout deal: grab Apple's M2 Pro MacBook Pro for $1,599 today only

Apple's new Photos app will utilize generative AI for image editing

New iPad Air & iPad Pro models are coming soon - what to expect

New Apple Pencil may be called 'Apple Pencil Pro'

Siri for iOS 18 to gain massive AI upgrade via Apple's Ajax LLM

How to watch Apple's 'Let Loose' iPad Air & iPad Pro event

Featured deals.

Deals: Apple AirPods dip to $79, M3 iMac 24-inch drops to $1,149 at Amazon

Latest exclusives.

Apple's iOS 18 to streamline task management with unified events and reminders

An inside look at Apple's various internal iOS variants that aid development

Latest comparisons.

Apple headphone shootout: Beats Solo 4 vs AirPods Max

M3 15-inch MacBook Air vs M3 14-inch MacBook Pro — Ultimate buyer's guide

M3 MacBook Air vs M1 MacBook Air — Compared

How to add fonts in macOS Sonoma

Installing fonts in macOS Sonoma is almost as easy as downloading them. Here's how you can add them to your Mac.

Secret Apple 'Project ACDC' to pioneer AI chips for data centers

Amidst a high-stakes AI race, Apple is forging ahead with "Project ACDC," a secret initiative recently come to light that develops proprietary AI chips for data centers.

US Labor Board rules Apple illegally interrogated retail staff in NYC

The US Labor Board found that Apple conducted unauthorized interrogations of employees at the Apple World Trade Center retail store in New York City.

A new teaser on Apple's website could be indicative of some of the company's upcoming software plans, namely a new version of its ubiquitous Photos app that will tap generative AI to deliver Photoshop-grade editing capabilities for the average consumer, AppleInsider has learned.

iPhone 17 Pro rumored to get 12GB of RAM

A new rumor suggests the iPhone 17 Pro, due in 2025, will increase to 12GB of RAM and have a more narrow Dynamic Island cutout.

Lots of new smart home gear, cheap HomeKit sensors, & Matter 1.2 on HomeKit Insider

On this episode of the HomeKit Insider Podcast, we check out several new products and discuss the delay with vacuums in Matter 1.2.

AppleInsider has learned that code in the Japanese version of Apple's website reveals multiple mentions of an "Apple Pencil Pro," ahead of the expected release of a new model on May 7.

Apple Arcade rolls out five new games, including 'Rabbids: Legends of the Multiverse'

Apple Arcade is set to receive new titles in June, including the adventure game "Return to Monkey Island+" and the retro-style platformer "Tomb of the Mask."

Warren Buffett has sold a lot of Apple stock so far in 2024

Famed investor Warren Buffett cut his firm's Apple holdings by 13% in the first quarter of 2023, as analyst consensus was that the iPhone was seeing declining demand.

Latest Videos

Beats Solo 4 headphones review: Great audio quality and features

iPhone 16 Pro: what to expect from Apple's fall 2024 flagship phone

Latest reviews.

Unistellar Odyssey Pro review: Unlock pro-level astronomy with your iPhone from your backyard

Ugreen DXP8800 Plus network attached storage review: Good hardware, beta software

Espresso 17 Pro review: Magnetic & modular portable Mac monitor

a. Send us an email
b. Anonymous form
Buyer's Guide
Upcoming Products
Tips / Contact Us
Podcast Instagram Facebook Twitter Mastodon YouTube Notifications RSS Newsletter

New AI Features Reportedly Coming to Safari in iOS 18

Safari will gain a series of AI-powered features and UI enhancements in iOS 18 , AppleInsider reports .

An "Intelligent Search" browser assistant tool that leverages Apple's on-device AI technology to identify key topics and phrases on webpages for summarization purposes.
A "Web Eraser" tool that allows users to remove unwanted portions of webpages easily. Erasure is persistent, remaining even when revisiting a site unless changes are reverted.
A new, quick-access menu emerging from the address bar that consolidates page tools, bringing over some functions that currently sit in the Share Sheet and placing them alongside the new tools.

The iPadOS and macOS versions of Safari are also expected to align further. These new features are purportedly undergoing evaluation alongside internal builds of ‌iOS 18‌ and macOS 15 ahead of their unveiling at Apple's Worldwide Developer's Conference (WWDC) in June. Earlier this month, backend code on Apple's servers suggested that a new Safari browsing assistant is in the works, corroborating this report.

Farther in the future, Apple is said to be working on a more powerful version of Visual Look Up that allows users to obtain information on products when browsing through images. The feature is expected to be released sometime in 2025.

Last week, Bloomberg 's Mark Gurman said that ‌iOS 18‌ will "overhaul" many of Apple's built-in apps , including Notes, Mail, Photos , and Fitness. Apple's next-generation operating systems are almost certain to be previewed during Apple's WWDC keynote on June 10, and the updates should be widely released in the fall.

Get weekly top MacRumors stories in your inbox.

Top Rated Comments

A "Web Eraser" tool that allows users to remove unwanted portions of webpages easily. Erasure is persistent, remaining even when revisiting a site unless changes are reverted.

iOS 18 Rumored to Add New Features to These 16 Apps on Your iPhone

Report: Five Things Apple Won't Announce at Its Event Next Week

Rumor: iOS 18 to add new ‘Web Eraser’ and page summarization features to Safari

Apple is reportedly planning big updates to Safari as part of iOS 18 and macOS 15 later this year. According to a report on Tuesday, the changes will include a new AI-powered tool called “Intelligent Search” with page summarization, a “Web Eraser” feature, and more.

iOS 18: New Safari features

The report, which comes from Marko Zivkovic at Apple Insider , cautions that “exact implementation” of these features “remains fuzzy.” We’d treat these claims with skepticism for the time being.

According to Zivkovic, Apple is planning a redesigned controls menu for Safari’s address bar that “consolidates old and new page control tools” in a central location. One of the new tools is reportedly something called “Intelligent Search.”

This feature, which is not enabled by default in test builds of Safari 18, can be activated from the new menu in the address bar. From there, the tool will “identify topics and key phrases within a webpage” and generate a summary of the page. The report speculates that Apple is “striving to deliver text summarization alongside Safari 18 later this year,” but again, the specific details are “fuzzy.”

Arc from The Browser Company also combines a variety of AI features with web browsing, including page summarization options.

Safari 18 in iOS 18 and macOS 15 will also reportedly include something called “Web Eraser.” This feature is “designed to allow users to remove, or erase, specific portions of web pages,” according to Zivkovic.

Using the Web Eraser feature, users could choose to remove certain elements from a webpage like ads, images, and more. It’s a functionality already offered by certain third-party ad blockers, but Apple is apparently working on its own first-party implementation for iOS 18 and macOS 15.

Notably, any changes you make to a webpage would apply each time you visited that page. “This means that Safari will remember the changes even after the original tab or window has been closed,” Zivkovic says.