![[ anti-audio rant here ]](https://image.slidesharecdn.com/html5audio-probablymaybeno-scottschiller-110624100457-phpapp01/85/Probably-Maybe-No-The-State-of-HTML5-Audio-Scott-Schiller-13-320.jpg)
![(As with any new technology, there are always a few “buts”…) [ enhance ]](https://image.slidesharecdn.com/html5audio-probablymaybeno-scottschiller-110624100457-phpapp01/85/Probably-Maybe-No-The-State-of-HTML5-Audio-Scott-Schiller-77-320.jpg)
![[ demos ] SoundManager 2](https://image.slidesharecdn.com/html5audio-probablymaybeno-scottschiller-110624100457-phpapp01/85/Probably-Maybe-No-The-State-of-HTML5-Audio-Scott-Schiller-103-320.jpg)
![A Glimpse Into The Future: Mozilla/Webkit Audio APIs [ demo? ]](https://image.slidesharecdn.com/html5audio-probablymaybeno-scottschiller-110624100457-phpapp01/85/Probably-Maybe-No-The-State-of-HTML5-Audio-Scott-Schiller-126-320.jpg)
![[ demo ] Turntable Prototype](https://image.slidesharecdn.com/html5audio-probablymaybeno-scottschiller-110624100457-phpapp01/85/Probably-Maybe-No-The-State-of-HTML5-Audio-Scott-Schiller-149-320.jpg)
![[ demo ] Turntable Prototype](https://image.slidesharecdn.com/html5audio-probablymaybeno-scottschiller-110624100457-phpapp01/85/Probably-Maybe-No-The-State-of-HTML5-Audio-Scott-Schiller-151-320.jpg)
![[ demo ] Turntable Prototype Layout/box model mode (RGBA outline and background colors)](https://image.slidesharecdn.com/html5audio-probablymaybeno-scottschiller-110624100457-phpapp01/85/Probably-Maybe-No-The-State-of-HTML5-Audio-Scott-Schiller-152-320.jpg)
![[ demo ] Turntable Prototype For fun, an alternate skin (background-color, etc.)](https://image.slidesharecdn.com/html5audio-probablymaybeno-scottschiller-110624100457-phpapp01/85/Probably-Maybe-No-The-State-of-HTML5-Audio-Scott-Schiller-153-320.jpg)
More Related Content
Similar to "Probably, Maybe, No: The State of HTML5 Audio" - Scott Schiller (20)
"Probably, Maybe, No: The State of HTML5 Audio" - Scott Schiller
- 1. “ Probably, Maybe, No”: The State Of HTML5 Audio
- 2. Hi. Scott Schiller (@schill) HTML Technician (front-end engineering) Photography / DJing / carnivorous plant enthusiast Y! Photos (2005-2007,) Flickr (present)
- 3. The Point / Things I’m Going To Talk Aboot
- 4. The Point / Things I’m Going To Talk Aboot Where we: ( were / are / going ) with web audio
- 5. How we got this far The Point / Things I’m Going To Talk Aboot
- 6. “ Show me something shiny ” The Point / Things I’m Going To Talk Aboot
- 7. Inform + inspire + excite -> go and make shiny things The Point / Things I’m Going To Talk Aboot
- 8. A (largely) silent web
- 9. Sound is one of the senses.
- 10. Yet, it seems to be largely missing from the web.
- 11. WTF?
- 13. [ anti-audio rant here ]
- 14. A Brief History of Web Audio
- 15. <bgsound="xfiles.mid" controls="console" loop="5"> <!-- inline player UI, often QuickTime --> A Brief History of Web Audio
- 16. <!-- autostart === auto-annoy, hidden evil --> <embed src="hamsterdance.wav" autostart=true loop=true volume=100 hidden=true> <noembed> <bgsound src="hamsterdance.wav"> </noembed> A Brief History of Web Audio
- 17. <!-- Adobe Flash embed code, 2011 --> <OBJECT classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" WIDTH="550" HEIGHT="400" id="myMovieName"> <PARAM NAME=movie VALUE="flashSoundPlayer.swf"> <PARAM NAME=quality VALUE=high> <PARAM NAME=bgcolor VALUE=#FFFFFF> <EMBED href="flashSoundPlayer.swf" quality=high bgcolor=#FFFFFF WIDTH="550" HEIGHT="400" NAME="myMovieName" ALIGN="" TYPE="application/x-shockwave-flash" PLUGINSPAGE="http://www.macromedia.com/go/getflashplayer"></EMBED> </OBJECT> A Brief History of Web Audio
- 18. The (Mis-)Truth Is Out There.
- 20. These are from search results. In 2011.
- 21. <BGSOUND> + <EMBED> Limitations
- 22. Slow. Buggy. Inconsistent. <BGSOUND> + <EMBED> Limitations
- 23. <embed> loads a plugin/helper binary (QT, RealPlayer, WMP) <BGSOUND> + <EMBED> Limitations
- 24. Parameters differ. loop=true vs. loop=5 <BGSOUND> + <EMBED> Limitations
- 25. <embed> sometimes scriptable via JS. Little/no API docs per plugin. (+buggy.) * <BGSOUND> + <EMBED> Limitations * Excluding Flash 8+, documented and scriptable via ExternalInterface.
- 26. Which brings us to…
- 28. Ah, the future. So shiny.
- 29. <audio src="yayhtml5.mp3" controls> new Audio('yayhtml5.mp3');
- 30. So simple, too!
- 32. Browser format support is varied.
- 33. (Shock, horror.)
- 34. So it’s not that simple.
- 35. <audio src="html5.mp3" controls> <!– for non-HTML5 browsers… --> Download <a href="html5.mp3">html5.mp3</a>. </audio>
- 36. Furthermore…
- 37. No single format is supported in all major browsers.* * Not even .WAV, as of March 2011. Boourns.
- 38. So it’s not really that simple.
- 39. At least two formats are required.
- 40. <audio controls> <source src="html5.mp3" /> <source src="html5.ogg" /> Download <a href="html5.mp3">html5.mp3</a> or <a href="html5.ogg">html5.ogg</a> </audio>
- 41. AND…
- 42. Your client wants it to work in IE 6.
- 43. (As with any new technology, there are always a few “buts”…)
- 44. If only <audio> were as simple as <img> .
- 45. This is unlikely, given technical, philosophical and legal complexities.
- 46. If only <audio> were as simple as <img> . The A/V Codec Fight
- 47. Format Wars
- 48. Mozilla + Google (OGG/WEB-M) vs. Apple + Microsoft (AAC/MP4/H.264) Format Wars
- 49. Web-M = “free”*, H.264 = IP/$$$ Format Wars * Mumble mumble something about MPEG-LA threatening Web-M, allegedly violating IP, etc.
- 50. Developer headaches, 2+ encodings, confusion Format Wars
- 51. And then there’s mobile.
- 52. “ Special snowflake” devices
- 53. Only one sound at a time
- 54. *Under most circumstances. iOS 4.2 locks this down pretty well. No auto-play allowed*
- 55. Pop-up blocker-style security model for Audio.play()
- 56. o nfinish()-> play() allowed, otherwise, fail
- 57. (As with any new technology, there are always a few “buts”…)
- 58. So it’s really not that simple.
- 59. Other Annoyances
- 60. Occasional Safari/OS X audio load failure.
- 61. "Safari" (Windows) = No HTML5 A/V support.
- 62. Download "Safari + QuickTime" = HTML5 A/V.
- 63. How is HTML5 Audio Implemented?
- 64. Multiple vendors targeting W3 spec How is HTML5 Audio Implemented?
- 65. "Plug-in" architecture (eg. Safari -> QT Plugin) How is HTML5 Audio Implemented?
- 66. Per spec, user should be able to install plugins How is HTML5 Audio Implemented?
- 67. XiphQT -> Safari (and iTunes) do OGG How is HTML5 Audio Implemented?
- 68. Chrome, Firefox /Mozilla include F/OSS-type libraries for playback How is HTML5 Audio Implemented?
- 69. IE 9: WMP /DirectAudio / ?? How is HTML5 Audio Implemented?
- 71. HTML5, like AJAX, is not “magical.” * Potential vs. Reality * Unless you’re in marketing, and then maybe it is.
- 72. The potential is there. Potential vs. Reality
- 73. We'll get there, eventually. Potential vs. Reality
- 74. For now, bugs and quirks will likely: Potential vs. Reality
- 75. For now, bugs and quirks will likely: * Enhance job security Potential vs. Reality
- 76. For now, bugs and quirks will likely: * Enhance job security * Enhance grey hair Potential vs. Reality
- 77. (As with any new technology, there are always a few “buts”…) [ enhance ]
- 78. Can I Use HTML5 Audio Today?
- 79. (Might as well) Go for it.
- 81. Use responsibly, tread carefully.
- 82. HTML5 Audio via JavaScript
- 83. Audio().canPlayType() = 'probably','maybe', 'no' * The most un-computer-like response I’ve ever seen. Reflects the reality of fragmented format support. HTML5 Audio via JavaScript * Spec changed 'no' to '', to be a non-truthy return value.
- 84. Audio().canPlayType('audio/mp3'); Audio().canPlayType('audio/mpeg; codecs="mp3"'); More detail should = “probably” HTML5 Audio via JavaScript
- 85. Audio can use HTTP partials, range requests. Arbitrary seek is a benefit. HTML5 Audio via JavaScript
- 86. A taste of events: canplay, load, ended, error, progress, timeupdate, waiting HTML5 Audio via JavaScript
- 87. And How About Flash?
- 88. Single-vendor/platform consistency advantage And How About Flash?
- 89. Overall, best non-HTML5 fallback for MP3/MP4 And How About Flash?
- 90. Is Flash Dead Yet?
- 91. No.
- 93. HTML5 Isn’t Going To Kill Flash?
- 94. No.
- 95. Flash remains a handy fallback for non-HTML5 UAs and is the de-facto method for “secure” RTMP streaming, etc.
- 96. So HTML5 And Flash Can Co-Exist…
- 97. CAN I USE BOTH AT THE SAME TIME TO PLAY AUDIO ON BOTH IE 6 AND THE iPHONE, DEPENDING ON WHICH TECHNOLOGY IS AVAILABLE?
- 98. Exactly. :)
- 99. This is why SoundManager 2 exists.
- 100. JavaScript API for sound SoundManager 2
- 101. HTML5 Audio() and/or Flash as needed (abstracted) ~10 KB (gzip) SoundManager 2
- 102. soundManager.onready(function() { soundManager.play( 'mySound', '/path/to/an.mp3’ ); }); JS talks to Flash (ExternalInterface) or HTML5 Audio(), depending on support. Transparent to developer. SoundManager 2
- 103. [ demos ] SoundManager 2
- 104. Why use an abstraction? SoundManager 2
- 105. Why use an abstraction? Support for both HTML4 + HTML5 SoundManager 2
- 106. Why use an abstraction? Support for both HTML4 + HTML5 Normalize HTML5 differences SoundManager 2
- 107. SO ULTIMATELY TO PLAY MP3/MP4 EVERYWHERE, YOU MUST HAVE FLASH?
- 108. Correct. It’s either that, or split formats.
- 109. THAT IS LAME!
- 110. Unfortunately, yes. LAME is also a great free MP3 encoder. ;)
- 111. Bonus Flash Bits
- 112. Flash audio API: Well-designed, overall Bonus Flash Bits
- 113. Revamped with Flash 8, extended in 9, moreso in 10 Bonus Flash Bits
- 114. Some conveniences/extras beyond HTML5 audio Bonus Flash Bits
- 115. Audio metadata access (waveform/spectrum) = nifty visualizations http://flic.kr/p/9btbyE Bonus Flash Bits
- 116. A Glimpse Into The Future
- 117. HTML5 audio: A good start A Glimpse Into The Future
- 118. Meets basics for sound apps (near-Flash parity) A Glimpse Into The Future
- 119. Pending: Bug finding, fixes, improvements and eventual stability A Glimpse Into The Future
- 120. What’s Next?
- 121. Mozilla Audio Data API / Webkit “Web Audio” API
- 122. Access/manipulate raw audio bytes during playback A Glimpse Into The Future: Mozilla/Webkit Audio APIs
- 123. Generate sound entirely from JavaScript A Glimpse Into The Future: Mozilla/Webkit Audio APIs
- 124. Mozilla released their API with Firefox 4 http://videos.mozilla.org/serv/blizzard/audio-slideshow/ https://wiki.mozilla.org/Audio_Data_API A Glimpse Into The Future: Mozilla/Webkit Audio APIs
- 125. Chrome 10+ beta with "Web Audio" enabled in about:flags http://chromium.googlecode.com/svn/trunk/samples/audio/o3d-webgl-samples/pool.html http://chromium.googlecode.com/svn/trunk/samples/audio/simple.html A Glimpse Into The Future: Mozilla/Webkit Audio APIs
- 126. A Glimpse Into The Future: Mozilla/Webkit Audio APIs [ demo? ]
- 127. Show me the shiny.
- 128. In late 2010, Technics announced that they were ceasing production of the DJ industry hallmark, the SL-1200 turntable.
- 129. 1972 - 2010
- 130. As a DJ / Web Developer, It seemed appropriate to take some sort of action.
- 131. So I started looking at CSS, which does boxes quite well.
- 133. CSS3’s border-radius finally lets us draw circles.
- 135. We can also apply background:url() and overflow:hidden …
- 137. Mozilla and Webkit give us -moz-transform and –webkit-transform:rotate()
- 140. <input type=“range” … />
- 141. <input type=“crossfader”> ? ;)
- 142. Prototype Goals
- 143. Recreate core turntable UI + behaviours in a browser. Prototype Goals
- 144. HTML5 audio should cover most sound requirements. Load, pause/play, seek etc. Prototype Goals
- 145. Hardware acceleration should keep it fast, too. Prototype Goals
- 146. Initially considered canvas, went with HTML + CSS. Layout advantage, fast render still possible. Prototype Findings
- 147. HTML5 audio works, despite browser quirks (and support issues.) The fix: Flash fallback. Prototype Findings
- 148. Heavily dependent on acceleration. transform:rotate3d() etc. Smooth in Webkit on OS X, some audio lag on Windows. Might be OK on IE 9. Acceleration is still “new.” Prototype Findings
- 149. [ demo ] Turntable Prototype
- 150. How about pitch bending, scratch, EQ effects etc.? (This is where Flash 10 / Audio Data APIs come in.) Turntable Prototype
- 151. [ demo ] Turntable Prototype
- 152. [ demo ] Turntable Prototype Layout/box model mode (RGBA outline and background colors)
- 153. [ demo ] Turntable Prototype For fun, an alternate skin (background-color, etc.)
- 154. The (Near) End
- 155. In Closing HTML5: Tons of potential. Bugs to be found + fixed. X-platform/device support. Audio Data APIs = shiny.
- 156. In Closing Use abstractions as needed to get similar experiences in more places. My hope is that SoundManager 2 becomes redundant over time.
- 157. “ You see, HTML5 audio is like a shark…”
- 159. In Closing Creative Commons-Licensed Photos "Butts" by arbyreed - http://flic.kr/p/jvCCh "Untitled" (facepalm) by Zack Sheppard - http://flic.kr/p/7gwXNt "20100216_0082_1600x1067" (hockey photo) by Les_Stockton - http://flic.kr/p/7DL6NU "Snow Phone" by WKeown - http://flic.kr/p/7Ef7Aa "Safari" by aditza121 - http://flic.kr/p/k7kkq "Holly (and shadow) mid jump" by Ali Smiles :) - http://flic.kr/p/5ZKWKY "I can't believe this illustration had no caption" by Ross Harmes - http://flic.kr/p/9jmjyt "Water Slide" by Ross Harmes - http://flic.kr/p/9jJRB7 fin.