2018512 AWS上での機械学習システムの構築とSageMaker
5 likes•3,105 views
The document discusses Amazon Web Services (AWS) offerings and pricing, highlighting that AWS does not provide binding price quotes and estimates are based on user input. It outlines various AWS services, including machine learning and real-time data processing, while emphasizing that actual charges may vary. Additionally, it includes references to research sources and tools for managing data and analytics within the AWS ecosystem.
2018512 AWS上での機械学習システムの構築とSageMaker
- 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 1,8 2 / /5 0./ , ,2 A
- 2. / ) • • CW EUI T S • C F G • ( ? m • SH A • ) a F Aeb a • / O CP • ) r A h • o
- 3. • 2018 5 12 AWS (http://aws.amazon.com) • AWS AWS • • AWS does not offer binding price quotes. AWS pricing is publicly available and is subject to change in accordance with the AWS Customer Agreement available at http://aws.amazon.com/agreement/. Any pricing information included in this document is provided only as an estimate of usage charges for AWS services based on certain information that you have provided. Monthly charges will be based on your actual use of AWS services, and may vary from the estimates provided.
- 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4
- 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Sources: Gartner: User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011 IDC: Worldwide Business Analytics Software 2012–2016 Forecast and 2011 Vendor Shares 0 1 2 3 4 5 6 7 8 9 1990 2000 2010 2020 DataVolume Year Generated Data
- 6. . • • & . . • • . . . • • M L
- 8. / x a • l • l • & • • • • • I • • Q o • • x • I • • IA • m n e 8
- 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Kinesis Database Migration Service Oracle, Netezza Amazon S3 Direct Connect Snowball Interoperate with everything and many more… Redshift EMR Athena AI and Machine Learning as-a-service QuickSight and many more…
- 11. A S W L 6 5 W % M 6 S A S 5 8S 0 S 5 5 2 % 1 2 3
- 12. 12 Services • • I • Platforms • M • M P • Frameworks • M L • • A M
- 13. Services Amazon Rekognition Image Platform Frameworks TensorFlow Caffe Torch Theano CNTK Keras CPU IoT MobileInfrastructure GPU MXNet AWS Deep Learning AMI (Amazon Linux / Ubuntu / Windows) Amazon Polly Amazon Lex Amazon Comprehend Amazon Translate Amazon Transcribe Amazon Rekognition Video Amazon Machine Learning Apache Spark Amazon EMR Amazon Kinesis AWS Batch Amazon ECS Amazon SageMaker
- 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 14
- 17. Batch / Real time AWS AWS ETL Amazon Mechanical Turk
- 18. Batch / Real time AWS AWS ETL Amazon Mechanical TurkAWS ML
- 19. Batch / Real time AWS AWS ETL Amazon Mechanical TurkAWS ML AWS
- 20. Batch / Real time EMR Athena QuickSight S3 Glue Redshift SageMaker Kinesis
- 21. Amazon Kinesis AWS Glue Amazon S3 Amazon Glacier Amazon ECS Amazon EC2 Lambda function Amazon DynamoDB Amazon RDS Amazon Redshift Amazon Athena Amazon ES Amazon EMR Amazon QuickSight Amazon Kinesis Analytics Amazon Kinesis Streams Amazon SageMaker Amazon Machine Learning AWS Batch AWS IoT AWS Greengrass Lambda function Ingest Store Compute Database Analyze Intelligence The Edge
- 22. Redshift EMR (Yarn/Spark) Aggregator Model Development &Training / Amazon Athena Realtime Event Processor Agg. Cached Data Aggregator Realtime Search Kinesis Analytics Decision on the Cloud Feedback Aggregator Quicksight Kinesis Streams Kinesis Firehose Amazon S3 AWS Greengrass Lambda function AWS Glue AWS IOT Amazon SageMaker
- 23. Amazon SageMaker
- 26. : d cu • n r o • L : B a o • • B M o • M g k
- 27. • IA e a M S P
- 28. Amazon SageMaker
- 29. S M A 29
- 30. P • • N B • r D A g • B g • I • A k M • a S / J g • I e • / / g g
- 31. . 3 A AX W N M XS M XS • 1 23 2 • T A N M M XS T 31
- 32. . 3 A A a T S W TX • 1 23 2 • A T W TX e N e A M 32
- 33. K A . 3 D 1. 3 33 SageMaker S3
- 34. ) E AM g 34 SageMaker S3 S 2 ) f AD W ai ) . ( ) ai D K e ai S
- 35. ) Dd 35 SageMaker S3 KSD 3 3 D ) W ge M E D ) . ( ) KS iAaA
- 36. 36
- 37. 37 SageMaker SDK Tensorflow Estimator S3
- 38. 38
- 39. 3
- 40. ( ) ( A $$ $ A$ ($ • o m r • iw z nG Sgj • 52 Ib P U I • s e e 59 IaP O A $$ $ A$ ($AB :$ • I iw % % ($. $ • I t 0 3 % % $. • 1/cW p %%%h T % %%) Amazon S3
- 41. • • • • 1 5TB • HTTPS PUT/GET • • • AWS 100 s3:// / / ( ) S3 PUT GET
- 43. • N - 3 • N - 0 32 CEGA I • D 0 1 2 13
- 44. (8 A: ) 1 > 03 ( 12 DCE GI • 1 3: > 1 > DCE GI 1 > 3 3: > DCE GI 0 20 40 60 80 100 120 140 K80 P100 V100 Mixed/FP16 Perf (TFLOPS) 0 2 4 6 8 10 12 14 16 K80 P100 V100 FP32 Perf (TFLOPS) 0 1 2 3 4 5 6 7 8 K80 P100 V100 FP64 Perf (TFLOPS) 0 1000 2000 3000 4000 5000 6000 K80 P100 V100 Resnet-50 8 GPU (Images/sec) 1 0 2 .
- 45. 3 2 2 A U S L IM C E A D a P 89 AW 45 https://aws.amazon.com/jp/amazon-ai/amis/ https://aws.amazon.com/jp/blogs/ai/announcing-new-aws-deep-learning-ami-for-amazon-ec2-p3-instances/#more-2093
- 46. 46 CUDA9, nvidia-docker AMI nvidia-docker NVIDIA GPU Cloud Docker Docker
- 48. - - ID • • • - - - • ID <E DH ID - - >C <E F E
- 49. / / / CA K E E G / / / , / .
- 50. // / / / 50 • IN Xe MSG g AMPT • / / / GM LAg A F • T S hXe c G U f G a • W Trained models and Lambdas Extracted IntelligenceInferences and take actions locally on device AWS Cloud for training
- 51. A GD C AA )A C AA AA A C - AC )A C )A C A BB C A AC BB C A AC BB A A C C A DB CCG 2BA C ( 2 ) D D BAG C A 3 )A (C C DD AC DA
- 52. g S WM L B D W f A eb a d
- 53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 54. MG K F S A Q 54 .1 . . 1 . . 33
- 55. . . A . . A . . 1 M 55
- 56. A G K M F G . 3 S 56 . 3 3 3 3 1
- 57. S R VQ c M C A C M 1 aG CF e C G K 57 3 1 1 .3 1
- 58. B B G RQa FA B C E A D M K S D 58 3 . 3 . 23 .
- 59. D C DA B 2 E 2 DA S 59 . .
- 60. G M R K F B EA S D M F 60 . . 3 . . . 2 .
- 61. R Q R AC F SAC T VA a SA K G K 61 2 2 2 2.3 2
- 62. I DSGTM Pec DW UC A Q Aa 62 Camera Device 3 Greengrass Core . . . 3 3 . . .
- 63. A C U D G 3 P 63 Camera Device .. Greengrass Core
- 64. G I . MA 3 64 .
- 65. D 3 A . . G A 65 Greengrass Core . .
- 66. AG I A S Q 3 66 3 . . 3
- 67. G C M U A G C P S Q 67 Camera Device 4 . 3 . 4 HTML/Javascript/model
- 68. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 68
- 69. • W DS p e k D aA r Dg D • O p g aA r g • M s D aA 69