Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Archives:

Chrome’s New Audio Notifier

January 29th, 2014 by Multimedia Mike

Version 32 of Google’s Chrome web browser introduced this nifty feature:


Chrome audio notifier icon

When a browser tab has an element that is producing audio, the browser’s tab shows the above audio notification icon to inform the user. I have seen that people have a few questions about this, specifically:

  1. How does this feature work?
  2. Why wasn’t this done sooner?
  3. Are other browsers going to follow suit?

Short answers: 1) Chrome offers a new plugin API that the Flash Player is now using, as are Chrome’s internal media playing facilities; 2) this feature was contingent on the new plugin infrastructure mentioned in the previous answer; 3) other browsers would require the same infrastructure support.

Longer answers follow…

Plugin History
Plugins were originally based on the Netscape Plugin API. This was developed in the early 1990s in order to support embedding PDFs into the Netscape web browser. The NPAPI does things like providing graphics contexts for drawing and input processing, and mediate network requests through the browser’s network facilities.

What NPAPI doesn’t do is handle audio. In the early-mid 1990s, audio support was not a widespread consideration in the consumer PC arena. Due to the lack of audio API support, if a plugin wanted to play audio, it had to go outside of the plugin framework.


NPAPI plugin model

There are a few downsides to this approach:

So that last item hopefully answers the question of why it has been so difficult for NPAPI-supporting browsers to implement what seems like it would be simple functionality, like implementing a per-tab audio notifier.

Plugin Future
Since Google released Chrome in an effort to facilitate advancements on the client side of the internet, they have made numerous efforts to modernize various legacy aspects of web technology. These efforts include the SPDY protocol, Native Client, WebM/WebP, and something call the Pepper Plugin API (PPAPI). This is a more modern take on the classic plugin architecture to supplant the aging NPAPI:


PPAPI plugin model

Right away, we see that the job of the plugin writer is greatly simplified. Where was this API years ago when I was writing my API jungle piece?

The Linux version of Chrome was apparently the first version that packaged the Pepper version of the Flash Player (doing so fixed an obnoxious bug in the Linux Flash Player interaction with GTK). Now, it looks like Windows and Mac have followed suit. Digging into the Chrome directory on a Windows 7 installation:

AppData\Local\Google\Chrome\Application\[version]\PepperFlash\pepflashplayer.dll

This directory exists for version 31 as well, which is still hanging around my system.

So, to re-iterate: Chrome has a new plugin API that plugins use to access the audio API. Chrome knows when the API is accessed and that allows the browser to display the audio notifier on a tab.

Other Browsers
What about other browsers? “Mozilla is not interested in or working on Pepper at this time. See the Chrome Pepper pages.”

Posted in General | 6 Comments »

6 Responses

  1. Alex Says:

    The one important detail you are missing is that Pepper flash was the default on windows as of Chrome 21 and Mac as of Chrome 23 [1]. Pepper flash is old hat. Sure it made this possible but what else stood in the way?

    [1] http://helpx.adobe.com/adobe-connect/kb/google-chrome-browser-incompatibility-version.html

  2. Multimedia Mike Says:

    @Alex: Is it really fair to say that Pepper Flash is old hat given the frenetic pace at which Chrome moves? :-) Chrome 23 wasn’t really that long ago (I’m finding dates that indicate 23 came out a little over a year ago).

    To be fair, most of this post was educated guessing, based on my experience developing the Flash Player, and also developing my PPAPI-based game music player, which also exhibits the audio notifier.

  3. How does Chrome know which of your tabs is playing audio? — Tech News and Analysis Says:

    […] hacker Mike Melanson has the entire backstory to this in a fascinating little post on his website that also explains why this new approach is a lot more easier for developers. Check […]

  4. Ever Wondered How Chrome Knows Which Tab Is Playing Audio? EDM Live Says:

    […] hacker Mike Melanson explains that there are two ways media is presented online: Publishers use third party plugins like […]

  5. Alexander E. Patrakov Says:

    Technically, under Linux, it IS possible for the browser to mediate access to audio devices even under the old NPAPI. Here is how.

    1. Ship an ioplug-based ALSA plugin that sends audio to the browser.
    2. Make sure that the browser plugins use it.

    The second issue is solvable by putting a configuration snippet under /usr/share/alsa/alsa.conf.d/ in a matter similar to 51-pulseaudio-probe.conf. That is, write a function that determines whether the current process is a browser plugin container, reference it in the 99-browser-detect.conf. If it succeeds, load the configuration file that assign the ALSA plugin from (1) to the default PCM.

  6. Stephan Sokolow Says:

    Apparently the original Pepper API and modern PPAPI are quite different and the “Mozilla pulling out” part went something like this:

    Google: Hey, wanna work on a new browser plugin API with us?
    Mozilla: Sure. Let’s base it on the existing web APIs.
    Google: Sorry. We’ve already got this code and we’re not changing it.
    Mozilla: Well that API assumes Chrome-like browser internals. It’d be too much work to implement on Firefox.
    Google and Mozilla: Ok, never mind then.

    Source: https://lwn.net/Articles/588583/

Leave a Comment

Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.