Cloud Spin - building a photo booth with the Google Cloud Platform
0 likes•823 views
Bret McGowen discusses the challenges and solutions in coordinating the simultaneous photo capture using multiple phones at Google Cloud's DevFest. He elaborates on using Firebase for real-time syncing, video recording techniques, and Google Cloud services for scaling and processing the data efficiently. The presentation highlights a prototype approach, showcasing the importance of cloud computing in developing innovative applications quickly.
![@bretmcg @googlecloud @googlecloudspin
Object Change Notifications
Cloud
Storage Bucket
[myapp-input-bucket]
App Engine
Object Change Notification
Client
Add/update/remove object](https://image.slidesharecdn.com/cloudspin-bretmcgowen-160211192621/85/Cloud-Spin-building-a-photo-booth-with-the-Google-Cloud-Platform-38-320.jpg)
![from moviepy.video.io.VideoFileClip import VideoFileClip
import numpy
STD_DEV_SCALAR = 4
def calc_peak_time(clip):
audio_arr = clip.audio.to_soundarray(buffersize=1000)
frames = clip.fps * len(audio_arr) / clip.audio.fps
buckets = numpy.array_split(audio_arr, frames)
mean_arr = [numpy.sqrt(numpy.mean(bucket**2)) for bucket in buckets]
peaks = numpy.where(mean_arr > MEAN_SCALAR * numpy.mean(mean_arr))
if len(peaks) > 0 and len(peaks[0]) > 0:
return peaks[0][0] / float(clip.fps)
else:
return 0.0](https://image.slidesharecdn.com/cloudspin-bretmcgowen-160211192621/85/Cloud-Spin-building-a-photo-booth-with-the-Google-Cloud-Platform-59-320.jpg)
Cloud Spin - building a photo booth with the Google Cloud Platform
- 1. Cloud Spin Bret McGowen Google @bretmcg Learning about the cloud by dodging bullets, Matrix-style
- 2. Bret McGowen Developer Advocate, Google Google Cloud Platform New York, NY @bretmcg Howdy DevFest MN!
- 3. I'm a fake startup CEO
- 4. Fake startup, real problems
- 5. 3 weeks to minimum viable product
- 10. The user steps into the center of the Cloud Spin area
- 11. The user jumps, and we take a photo from all the phones
- 13. ● Taking a photo at the same EXACT SAME INSTANT ● Coordinating the phone cameras ● File storage ● Scaling ● Image processing ● Controlling cost ● Create multiple versions ● Debugging & Logging Problems to solve
- 14. Taking the photos: how would you do it?
- 15. @bretmcg @googlecloud @googlecloudspin It isn’t that straightforward ● Need to coordinate 19 phones to take photos of the same moment! ● Android - random delay when taking a photo! Hence impossible to take photos of the same moment
- 16. @bretmcg @googlecloud @googlecloudspin What we ended up with? ● Record a video ● 30 fps is like burst mode
- 17. @bretmcg @googlecloud @googlecloudspin More problems ● How do we identify the frames? ● Mark the the frame some how... ● Many Experiments: flash, timestamp
- 18. Use an audio marker!
- 19. @bretmcg @googlecloud @googlecloudspin Mic-In Into the Phone MIC OMTP MIC GND AHJ / CTIA
- 20. @bretmcg @googlecloud @googlecloudspin Audio can be tricky
- 22. @bretmcg @googlecloud @googlecloudspin Finally - Off the shelf components
- 23. @bretmcg @googlecloud @googlecloudspin Record the audio cue in the video
- 24. How do we coordinate 19 phone cameras?
- 25. No, seriously, how do we coordinate 19 phone cameras?
- 26. @bretmcg @googlecloud @googlecloudspin What do we need to coordinate? ● Camera app status & error message ● Exposure ISO & shutter speed ● Internal versus external microphone ● Timestamp to start the recording ● And more...
- 27. @bretmcg @googlecloud @googlecloudspin Let's use Firebase! ● Can power our app's backend ● Stores our data and syncs it in real-time ● JSON NoSQL database with real-time sync ● Updates across devices within milliseconds
- 28. // Set this phone's status on Firebase at /cameras/{cameranumber} Firebase firebaseRef = new Firebase(FIREBASE_BASE_URL); Firebase cameraStatusRef = firebaseRef.child("cameras").child (cameraNumber); CameraStatus cameraStatus = new CameraStatus(); cameraStatus.appVersion = BuildConfig.VERSION_NAME; cameraStatus.externalMic = mExternalMic; ... cameraStatus.status = status; cameraStatusRef.setValue(mCameraStatus); Android: updating camera status with Firebase
- 30. @bretmcg @googlecloud @googlecloudspin Recording video with camera2 API ● camera2 API ● I learned a lot of this API from the android- Camera2Video GitHub project
- 31. Scaling with Google Cloud Platform
- 33. Stitching Q Extraction Q Input Bucket Stitching Q Devices Extraction Q Notification Processor Extractor Stitcher Video Uploads Object Change Notifications Output Bucket Coordinator Input Bucket Notification Processor Extractor Stitcher Output Bucket
- 34. @bretmcg @googlecloud @googlecloudspin Video Uploads and notification processing Input Bucket Stitching Q Devices Extraction Q Notification Processor Extractor Stitcher Video Uploads Notification Output Bucket Coordinator Input Bucket Notification Processor
- 35. Cloud Datastore Cloud SQL Cloud Storage
- 36. 4.5 trillion! Managed MySQL Cat Videos?Cat Videos? Jumping People
- 37. // Credentials for Google Cloud Platform. Credential credential = ... Storage storage = new Storage.Builder(httpTransport, jacksonFactory, credential) .setApplicationName("[email protected]") .build(); StorageObject storageObject = new StorageObject(); storageObject.setBucket(bucket); InputStream fileUploadStream = new FileInputStream(videoFile); try { InputStreamContent content = new InputStreamContent("video/mp4", fileUploadStream); Storage.Objects.Insert insert = storage.objects().insert(bucket, null, content); insert.setName(cloudStoragePath); insert.execute(); } finally { fileUploadStream.close(); }
- 38. @bretmcg @googlecloud @googlecloudspin Object Change Notifications Cloud Storage Bucket [myapp-input-bucket] App Engine Object Change Notification Client Add/update/remove object
- 39. @bretmcg @googlecloud @googlecloudspin Edge Caching
- 40. @bretmcg @googlecloud @googlecloudspin Google App Engine Choice of Runtimes - Java, Python, Go - PHP - Custom, NodeJS! Easy to develop - Build and test locally - Focus on App Code - Versioning - Traffic Splitting Trivial to manage - Fully managed - No patches/updates - 24x7 operation by Google SREs - Autoscale
- 42. @bretmcg @googlecloud @googlecloudspin Building Pipelines Input Bucket Stitching Q Devices Extraction Q Notification Processor Extractor Stitcher Video Uploads Object Change Notifications Output Bucket Coordinator
- 43. Subscriber Subscriber Pub/Sub Pub/Sub Push Subscription Pull Subscription msg ack msg ack RPC Send RPC Return
- 44. @bretmcg @googlecloud @googlecloudspin Cloud Pub/Sub (Beta) Reliable and real-time messaging Designed for Fast Data Provides reliable, real-time, many-to-many, asynchronous messaging between applications Send data into processing pipelines and out to other apps, devices and Google Cloud Services Global by design and highly available. ‘fire-and-forget’ with minimal latency. Redundant forwarding paths Designed for Google scale
- 45. @bretmcg @googlecloud @googlecloudspin Pub/Sub Stitcher Frames Topic Extractor Frame Ready Message Publishers and Subscribers Original Stitcher Subscription
- 46. @bretmcg @googlecloud @googlecloudspin Pub/Sub Original Stitcher Frames Topic Extractor Frame Ready Message Publishers and Subscribers Original Stitcher Subscription Variation A Stitcher Variation A Subscription Variation B Stitcher Variation B Subscription
- 47. @bretmcg @googlecloud @googlecloudspin Variations - How Pub/Sub Can Help
- 48. Meanwhile, in our secret startup lab
- 49. Our first prototype was cheap and scrappy
- 51. We used anything we could find.
- 56. @bretmcg @googlecloud @googlecloudspin Pipeline Processing Input Bucket Stitching Q Devices Extraction Q Notification Processor Extractor Stitcher Video Uploads Object Change Notifications Output Bucket Coordinator Extractor Stitcher
- 59. from moviepy.video.io.VideoFileClip import VideoFileClip import numpy STD_DEV_SCALAR = 4 def calc_peak_time(clip): audio_arr = clip.audio.to_soundarray(buffersize=1000) frames = clip.fps * len(audio_arr) / clip.audio.fps buckets = numpy.array_split(audio_arr, frames) mean_arr = [numpy.sqrt(numpy.mean(bucket**2)) for bucket in buckets] peaks = numpy.where(mean_arr > MEAN_SCALAR * numpy.mean(mean_arr)) if len(peaks) > 0 and len(peaks[0]) > 0: return peaks[0][0] / float(clip.fps) else: return 0.0
- 60. @bretmcg @googlecloud @googlecloudspin Frames extracted
- 61. @bretmcg @googlecloud @googlecloudspin Google Compute Engine Compute Resources - US, Europe and Asia zones - Fast SDN-Based virtual networking Consistently Fast - Fast VM Provisioning - Consistent Performance Cost Effective - Sub-Hour Billing - No IOPS charges for Block Storage - VMs not required for Load balancing
- 62. @bretmcg @googlecloud @googlecloudspin Compute Engine Autoscaler Managed Instance Group create/ destroy VMs Instance Group Manager actuator monitor Autoscaler Cloud Monitoring utilisation VMVMVM Pub/Sub Topic
- 63. @bretmcg @googlecloud @googlecloudspin Next step: Containerization Extractor Workers create/ destroy VMs Instance Group Manager actuator monitor Autoscaler Cloud Monitoring utilisation VMVM Container VM Extract Topic
- 64. @bretmcg @googlecloud @googlecloudspin Stitching the frames together Extract Q Temp Store Extractor Stitch QExtractorExtractors READY Stitcher Firebase All READY? Retrieve Images Visualizer
- 66. ● Taking a photo at the same EXACT SAME INSTANT - video with audio beep ● Coordinating the phone cameras - Firebase ● File storage - Google Cloud Storage ● Scaling - Cloud Pub/Sub, Compute Engine Instance Groups ● Image processing - Compute Engine & Containers ● Controlling cost - Scaling instance group sizes ● Create multiple versions - Cloud Pub/Sub topics ● Debugging & Logging - Google Cloud Logging Problems to solve
- 67. DEMO VIDEO TIME
- 68. @bretmcg @googlecloud @googlecloudspin Productivity! We built a lot in just 3 weeks Focus 100% of time coding
- 69. @bretmcg @googlecloud @googlecloudspin • Talk with us! • Firebase • Containers • Kubernetes • Big data • BigQuery Want to know more?