SlideShare a Scribd company logo

Machine Learning on Your Hand - Introduction to Tensorflow Lite Preview

Download as PPTX, PDF
Modulabs
Modulabs

The document presents an overview of the TensorFlow Lite framework introduced at the TF Dev Summit 2018, focusing on mobile machine learning applications. It highlights the advantages of on-device ML, including improved user experience and data privacy, while contrasting it with cloud ML limitations. Additionally, it discusses the Android Neural Network API (NNAPI) integration with TensorFlow Lite for optimized model execution on mobile devices.

Software
1 of 53
Downloaded 43 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Machine Learning on Your Hand - Introduction to Tensorflow Lite Preview TF Dev Summit X ModuLABS Jaewook Kang Apr. 5th, 2018 1 All Copyright Reserved @ MoT Lab 2018
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. ▪ GIST EEC Ph.D. (2015) ▪ 신호처리 과학자, 삽질러 ▪ MoT Lab Leader ▪ https://www.facebook.com/jwkkang ▪ 좋아하는 것: ▪ 통계적 신호처리 / 무선통신 신호처리 ▪ C++ Native 라이브러리 구현 ▪ Mobile Machine learning ▪ 수영 덕력 6년 2 ▪ 대표논문: Jaewook Kang, et al., "Bayesian Hypothesis Test using Nonparametric Belief Propagation for Noisy Sparse Recovery," IEEE Trans. on Signal process., Feb. 2015 Jaewook Kang et al., "Fast Signal Separation of 2D Sparse Mixture via Approximate Message- Passing," IEEE Signal Processing Letters, Nov. 2015 Jaewook Kang (강재욱) 소 개
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. MoT Contributors 3 Jaewook Kang (Soundlly) Joon ho Lee (Neurophet) Yonggeun Lee () Jay Lee (Vingle) SungJin Lee (DU) Seoyoen Yang (SNU) Yunbum Beak (신호시스템)
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al.4 •Mobilenet_quant_v1_224.tflite •테스트 디바이스: Samsung Galaxy S7 (SM-G930L) + Android 7.0 (Nougat) •빌드 환경: • Mac OSX 10.11.6 • bazel Version : 0.7 • Android Studio 3.0 • Android Build Tools Level: 26.1.1 • Android NDK Version: 16.04442984 1.mobilenet_v1_1.0_224 : 67.9 MB, Top-1 Accuracy=70.7, Top-5 Accuracy=89.5 1.mobilenet_quant_v1_224.tflite: 4.3 MB, Top-1 Accuracy=??, Top-5 Accuracy=??
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al.5 •Mobilenet_quant_v1_224.tflite •테스트 디바이스: Samsung Galaxy S7 (SM-G930L) + Android 7.0 (Nougat) •빌드 환경: • Mac OSX 10.11.6 • bazel Version : 0.7 • Android Studio 3.0 • Android Build Tools Level: 26.1.1 • Android NDK Version: 16.04442984 1.mobilenet_v1_1.0_224 : 67.9 MB, Top-1 Accuracy=70.7, Top-5 Accuracy=89.5 1.mobilenet_quant_v1_224.tflite: 4.3 MB, Top-1 Accuracy=??, Top-5 Accuracy=??
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Machine Learning of Things is Coming! ❖MoT Lab은 모바일 머신러닝에 관심을 갖습니다 –1. 모바일에서 머신러닝을 한다는 것! –2. Tensorflow Lite Preview version • About • Android NN API hardward acceleration • Model converting to tflite 6
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 1. 모바일에서 머신러닝을 한다는것 - Why on-device ML? - 해결해줘야 하는 부분 7
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖Why on-device ML? – Cloud ML의 제약 • UX 측면 8
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖Why on-device ML? – Cloud ML의 제약 • UX 측면 – 서비스 반응 속도= » 입력 데이터 업로드시간 » +클라우드 Inference 시간 » +결과 다운로드 시간 9
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖Why on-device ML? – Cloud ML의 제약 • UX 측면 – 서비스 반응 속도 – 오프라인 상황 10
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖Why on-device ML? – Cloud ML의 제약 • 데이터 소모 측면 – Inference할때마다 » server call 필요 » 입력데이터 업로드 필요 – 큰 데이터 소비 APP → 순삭 ㅠ 11
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖Why on-device ML? – Cloud ML의 제약 • 프라이버시 측면 – 개인화 ← → 프라이버시 » 개인화 서비스는 받고 싶은데 내데이터를 주는 건 싫다 • 데이터 퓨젼의 어려움 – 한 클라우드 서비스에서 다양한 개인정보를 수집하기 어려 움 » 모바일: 위치정보 / 사진 / 영상 /오디오 12
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖해결해줘야 하는 부분 – UX + 데이터 소모 측면 → On-device inference • 반응 속도 (Fast Response) • 배터리 (Efficient Computation) • 모델 사이즈 • 메모리 제한? – 프라이버시 측면→ On-device training • 다른 사람 말고 내 얼굴을 잘 인식해라 이놈아 • Transfer learning? • Personal Data fusion 13
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖해결해줘야 하는 부분 – UX + 비용 측면 → On-device inference • 반응 속도 (Fast Response) • 배터리 (Efficient Computation) • 모델 사이즈 • 메모리 제한? – 프라이버시 측면→ On-device training • 다른 사람 말고 내 얼굴을 잘 인식해라 이놈아 • Transfer learning? • Personal Data fusion 14
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 15 - 하이퍼커넥트 신범준님 발표자료 중 -
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 2. Tensorflow Lite Preview - About Tensorflow Lite - Android Neural Network API - Model conversion to tflite 16
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About ❖A lightweight ML library and tool or mobile devices – https://www.tensorflow.org/mobile/tflite/ – 지원 플랫폼: • Android Mobile • Raspberry Pi 3 (Android Things) • iOS – 지원 ops: Tensorflow >= Tensorflow Lite – 사이즈: Core Interpreter (+supp. Ops) 70kB ( 400kB) – 버전: Developer preview (2017 Nov, w/ TF v1.5) 17
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About 18 이미지 출처: https://www.t ensorflow.org /mobile/tflite/
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About ❖A lightweight ML library and tool or mobile devices 19 Tflite모델을 각 플랫폼의 커널에서 사용할 수 있도록 번역하는 Api 플랫폼 별 tflite 모델을 최적화 용 op set on device HW 계산 자원 할당 최적화 이미지출처: https://www.youtube.com/watch?v=FAMfy7izB6A - Android NNAPI - iOS CoreML- Tensorflow lite framework
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About ❖A lightweight ML library and tool or mobile devices 20 Run on device! Run on device! Run on device! 이미지출처: https://www.youtube.com/watch?v=FAMfy7izB6A - Android NNAPI - iOS CoreML- Tensorflow lite framework
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About ❖A lightweight ML library and tool or mobile devices – iOS develop has another option! – coreML converter 따로 있음: tfcoreml github repo • (tflite+coreml >>10배 속도>> tflite+nnapi) 21 이미지출처: https://www.youtube.com/watch?v=FAMfy7izB6A
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About ❖Why TensorFlow Lite is faster? – FlatBuffer: • A new model file format – Operation kernels optimized for NEON on ARM – Hardware acceleration support • Android NN API (Android Oreo) – Qualcomm Hexagon DSP SDK (Android P) – Direct GPU support • iOS CoreML – Metal2 – Quantization: Integer-arithmetic only support • Quantize both weights and activation as 8-bit integers • Just a few parameters(bias vectors) as 32-bit integers • 가장 범용적인 multiply-add instruction HW에서의 개선 • 용량 줄이기 보다 Inference 속도개선에 집중 22
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About ❖Why TensorFlow Lite is faster? – FlatBuffer: • A new model file format – Operation kernels optimized for NEON on ARM – Hardware acceleration support • Android NN API (Android Oreo) – Qualcomm Hexagon DSP SDK (Android P) – Direct GPU support • iOS CoreML – Metal2 – Quantization: Integer-arithmetic only support • Quantize both weights and activation as 8-bit integers • Just a few parameters(bias vectors) as 32-bit integers • 가장 범용적인 multiply-add instruction HW에서의 개선 • 용량 줄이기 보다 Inference 속도개선에 집중 23
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – On-deivce에서 계산효율적 ML을 위해서 설계된 Android C/C++ API – TensorFlow Lite 모델은 Android NN API의 Kernel Interpreter로 재구 성 + 최적화 되어 계산 하드웨어에 연결됨. – Hardware-specific processing을 통해서 neural net inference 속도 개 선! • Android 에서 잘 돌아가도록 tflite모델을 재구성 + 계산 자원 분배 • 디바이스가 보유하는 계산 유닛(CPU/CPU/DSP)에 효율적으로 계산 workload를 할당 할 예정 • 현재는 CPU만 지원됨 (2018 Mar) – Supporting Android 8.1 (API level 27 + NDK level 14) or higher • - tflite + nnapi : api level >= 27, ndk level > 14 (neon arm processor 에 최적화) • - tflite only : api level >=21 (안빠름) – tflite는 nnapi가 없어도 돌지만 그 경우 전혀 빠르지 않다! 24 This slide is powered by J. Lee
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/java/d emo/app/src/main/java/com/example/android/tflitecamerademo/ImageClassifier.java 25 This slide is powered by J. Lee - .tflite파일은 JAVA의 Interpreter 클래스가 생성될 때 내부적으로 Native C++ API를 호출하고 그 안에서 로드된다.
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/java/d emo/app/src/main/java/com/example/android/tflitecamerademo/ImageClassifier.java 26 This slide is powered by J. Lee - .tflite파일은 JAVA의 Interpreter 클래스가 생성될 때 내부적으로 Native C++ API를 호출하고 그 안에서 로드된다. tflite inference 수행하는 class 내부에서 tflite Interpreter 실행 - tflite interpreter JAVA객체 생 성
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/li te/java/src/main/native/nativeinterpreterwrapper_jni.cc 27 This slide is powered by J. Lee 1) JAVA API tflite.run() 이 실행되면 NativeInterpreterWrapper (JNI)를 경유해서 그 안에서 C++ API Interpreter→Invoke()을 호출
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/li te/java/src/main/native/nativeinterpreterwrapper_jni.cc 28 This slide is powered by J. Lee 1) JAVA API tflite.run() 이 실행되면 NativeInterpreterWrapper (JNI)를 경유해서 그 안에서 C++ API Interpreter→Invoke()을 호출 Java_org_tensorflow_lite_NativeInterpreterWrapper_run()
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 2) C++ API Interpreter→Invoke()안 에서 nnapi_delegate→Invoke() 가 호출됨 Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/li te/interpreter.cc 29 This slide is powered by J. Lee
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 2) C++ API Interpreter→Invoke()안 에서 nnapi_delegate→Invoke() 가 호출됨 Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/li te/interpreter.cc 30 This slide is powered by J. Lee
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 3) nnapi_delegate→invoke()안에서 nnapi_delegate→BuildGraph()호출 Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite/nnapi – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/nnapi_delegate.h – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/nnapi_delegate.cc 31 This slide is powered by J. Lee
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 3) nnapi_delegate→invoke()안에서 nnapi_delegate→BuildGraph()호출 Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite/nnapi – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/nnapi_delegate.h – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/nnapi_delegate.cc 32 This slide is powered by J. Lee
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – 33 This slide is powered by J. Lee 1) .tflite파일를 JAVA/C++ API 을 통해서 로드해서 2) C++ Android Kernal Interpreter를 통해서 NNAPI 클래스로 넘겨주고 3) C++ NNAPI Op set을 이용해 서 내부에서 tflite 모델을 low- level로 내부적으로 빌드한다. 4) low-level tflite 모델을 NNAPI를 통해서 실행한다.
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – https://developer.android.com/ndk/reference/neural_networks_8h.html – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/nnapi_delegate.h – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/nnapi_delegate.cc 34 This slide is powered by J. Lee NNAPI class methods - For Model build and compile - For Model execution
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Hardware acceleration via Android NN API ❖ Android NN API 개요 – https://developer.android.com/ndk/guides/neuralnetworks/index.html 35 This slide is powered by J. Lee Android NN API Op set - nnapi_delegate→BuildGraph() 에서 생성되는 Low-level 모델은 다음과 같은 NNAPI op set을 사용해서 구성된다.
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NNAPI 프로그래밍 flow – https://developer.android.com/ndk/guides/neuralnetworks/index.html 36 This slide is powered by J. Lee 모델생성 Building and Compiling an NNAPI model into lower- level code Inference실행 종료대기
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖더 궁금하시면? – https://developer.android.com/ndk/guides/neuralnetworks/index.html – 또는 MoT로 오세요! 같이 공부해요! – 머신러닝에 관심있는 안드로이드 개발자 모집중! • Android NN API • JAVA Native Interface • Android Things • Tensorflow Lite 37
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. From TF model to Android APP build 38 이미지 출처: https://www.t ensorflow.org /mobile/tflite/
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. From TF model to Android APP build 39 이미지 출처: https://www.t ensorflow.org /mobile/tflite/
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Model converting to tflite ❖전체 Tf model to Tflite 변환 과정 40 Get a Model Exporting the Inference Graph Freezing the exported Graph Conversion to TFLITE • Model Design or Downloading • Training with training graph • Fine Tunning • Evaluate the performance with Inference graph Convert • Graph def (.pb) • Check point (.ckpt) • Frozen graph (.pb) • Tensorflow lite (.tflite) This slide is powered by S. Yang
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Model converting to tflite ❖두가지 방식 – Using frozen graph (.pb) – Using Saved Model 41 This slide is powered by S. Yang
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Model converting to tflite ❖두가지 방식 – Using frozen graph (.pb) – Using Saved Model (더 쉬운방식 ㅠ, 이쪽이 답이 다) 42 This slide is powered by S. Yang
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Model converting to tflite ❖ 1) Tensorflow에서 그래프정보 (.pb+ .ckpt) 추출! • GraphDef (.pb): TF 계산그래프의 구조 정보만을 담고 있는 객체! • 세션안에서 tf.train.write_graph()를 이용해서 저장! • CheckPoint(.cpkt): TF계산 그래프의 훈련된 weight / bias 값이 저장된 lookup table 파일 • 세션안에서 tf.train.Savor()를 이용해서 저장 43 With tf.Session(graph=graph) as sess: tf.train.write_graph(graph_or_graph_def = sess.graph_def, logdir= "models/", name= "graph.pb") Saver = tf.train.Saver() With tf.Session() as sess: saver.save(sess=sess, save_path= "models/model.ckpt”, global_step=epoch) This slide is powered by S. Yang
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. ❖ 2) Frozen Graph (.pb) 생성 • FrozenGraphDef (.pb) : GraphDef를 checkpoint파일정보를 결합해서 variable 노드를 constant노드로 변환하여 저장한 것 • 1) Command line interface를 이용하는 방법 (freeze_graph bazel빌드 필요!) • 2) freeze_graph.py를 이용해서 파이썬 스크립트를 구성해서 변화하는 방법 Model converting to tflite 44 $ freeze_graph --input_graph=/tmp/mobilenet_v1_224.pb --input_checkpoint=/tmp/checkpoints/mobilenet-10202.ckpt --input_binary=true --output_graph=/tmp/frozen_mobilenet_v1_224.pb --output_node_names=MobileNetV1/Predictions/Reshape_1 This slide is powered by S. Yang freeze_graph.freeze_graph( input_graph=“/tmp/mobilenet_v1_224_.pb”, input_saver= "", # this argument is used with SavedModel input_binary=True, input_checkpoint=“/tmp/checkpoints/mobilenet-10202.cpkt”, output_node_names=“MobileNetV1/Predictions/Reshape_1”, restore_op_name="save/restore_all", # unused in freeze_graph() filename_tensor_name="save/Const:0", # unused in freeze_graph() output_graph=“/tmp/frozen_mobilenet_v1_224.pb ”, clear_devices=False, # not clear how to use initializer_nodes="")
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. ❖ 3) tflite model 파일 (.tflite) 생성 • Tensorflow Lite Model (.tflite) : Tflite interpreter가 해석가능하도록 변환한 모델 파일 • 1) Command line interface를 이용하는 방법 (toco bazel빌드 필요!) • Tensorflow source파일 git clone 필요 • Tensorflow source 파일을 local repository directory에서 실행 필요 Model converting to tflite 45 $ bazel clean –expunge $ bazel run --config=opt //tensorflow/contrib/lite/toco:toco -- --input_file= input_frozen_graph_pb_path # 입력 frozen pb path --output_file= output_tflite_path # 출력 tflite path inference_type=FLOAT # floating-poing conversion input_shape=1,28,28,1 # model의 input shape input_array=input # model의 input node name (from Tensorboard) output_array=model_out/Softmax # model의 output node name (from Tensorboard) This slide is powered by S. Yang
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. ❖ 3) tflite model 파일 (.tflite) 생성 • Tensorflow Lite Model (.tflite) : Tflite interpreter가 해석가능하도록 변환한 모델 파일 • 2) tf.contrib.lite.toco_convert를 이용해서 Tensorflow 스크립트에서 바로 생 성가능! (3/30에 push됨 ㅜ ) • https://www.tensorflow.org/versions/master/api_docs/python/tf/contrib/lite/toco_convert Model converting to tflite 46
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Model converting to tflite 47 ❖몇가지.. 정신건강을 위해 – Tensorflow 모델을 구성할때 TF lite에서 지원하는 operator만 사용 해야한다. • 지원op리스트: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/g3doc/tf_ops_compatibility.md – Bazel 은 사전에 최신버전으로 업그레이드 하자 – Tensorboard로 input_shape/input node name/output node name확 인 필요 – 절대 경로를 사용하는게 좋다 – win10에서 동작 미확인ㅠ
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Model converting to tflite 48 ❖Further, 여러분의 정신건강을 위해 준비했습니다. – jwkang's Tensorflow lite Github repo • A GraphDef+Checkpoint Generation from Lenet5 Tensorflow model example • A GraphDef+Checkpoint to Frozen GraphDef conversion example • A Frozen GraphDef to Tflite conversion example
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Future Direction of Tflite ❖ More suppoting ops ❖ On-device training ❖ Improved tools ➢ Easier tflite conversion ➢ Easier platform adaptation 49
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 참고자료 ❖ The Tensorflow GitHub doc (updated): – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lit e/toco/g3doc/cmdline_examples.md#savedmodel ❖ The Tensorflow Lite contrib repo documents – https://github.com/tensorflow/tensorflow/tree/master/tensorflow/co ntrib/lite/g3doc ❖ The Tensorflow.org documents for prepare model – https://www.tensorflow.org/mobile/prepare_models ❖ Tensorflow Lite support prebuilt models – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lit e/g3doc/models.md 50
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Reviewers ❖ 김승일님 (모두연) ❖ 이일구님 (모두연) ❖ 박은수님 (모두연) ❖ 전태균님 (쎄트렉아이) ❖ 신범준님 (하이퍼커넥트) ❖ 신정규님 (레블업) 51
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모두연 MoT랩 소개 ❖딥러닝을 활용하여 세상을 이롭게 할 IoT/Mobile App 개발에 대한 연구를 같이 해봐요!! ❖ https://www.facebook.com/lab4all/posts/761099760749661 ❖ jwkang10@gmail.com 로 메일 ❖MoT 추가 멤버 모집중! – 모바일앱에 포팅하고 싶지만 엄두가 안나는 연구자 – 머신러닝에 관심있는 안드로이드 개발자 – TF코딩은 잘하지만 이론을 더 공부하고 싶은 TF개발자! - 서비스 기획자 52
TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al.53 The End All copyright reserved @ MoT Lab 2018

More Related Content

What's hot (20)

PDF
How to Schedule Machine Learning Workloads Nicely In Kubernetes #CNDT2020 / C...
Preferred Networks
 
PPTX
Pulumi. Modern Infrastructure as Code.
Yurii Bychenok
 
PDF
CDNの仕組み（JANOG36）
J-Stream Inc.
 
PDF
Accelerating Envoy and Istio with Cilium and the Linux Kernel
Thomas Graf
 
PDF
GeoServer on Steroids
GeoSolutions
 
PDF
深層学習向け計算機クラスター MN-3
Preferred Networks
 
PDF
ACRiウェビナー：小野様ご講演資料
直久 住川
 
PDF
OpenStack Neutronの機能概要 - OpenStack最新情報セミナー 2014年12月
VirtualTech Japan Inc.
 
PDF
EnrootとPyxisで快適コンテナ生活
Kuninobu SaSaki
 
PDF
Linux field-update-2015
Chris Simmonds
 
PDF
ZynqMPのブートとパワーマネージメント : (ZynqMP Boot and Power Management)
Mr. Vengineer
 
PDF
TensorFlow Lite for mobile & IoT
Mia Chang
 
PDF
Fun with Network Interfaces
Kernel TLV
 
PPTX
Yocto bspを作ってみた
wata2ki
 
PDF
Profiling deep learning network using NVIDIA nsight systems
Jack (Jaegeun) Han
 
PPTX
root権限無しでKubernetesを動かす
Akihiro Suda
 
PDF
Kubernetes meetup-tokyo-13-customizing-kubernetes-for-ml-cluster
Preferred Networks
 
PDF
マルチコアを用いた画像処理
Norishige Fukushima
 
PDF
10GbE時代のネットワークI/O高速化
Takuya ASADA
 
PPTX
QEMU - Binary Translation
Jiann-Fuh Liaw
 
How to Schedule Machine Learning Workloads Nicely In Kubernetes #CNDT2020 / C...
Preferred Networks
 
Pulumi. Modern Infrastructure as Code.
Yurii Bychenok
 
CDNの仕組み（JANOG36）
J-Stream Inc.
 
Accelerating Envoy and Istio with Cilium and the Linux Kernel
Thomas Graf
 
GeoServer on Steroids
GeoSolutions
 
深層学習向け計算機クラスター MN-3
Preferred Networks
 
ACRiウェビナー：小野様ご講演資料
直久 住川
 
OpenStack Neutronの機能概要 - OpenStack最新情報セミナー 2014年12月
VirtualTech Japan Inc.
 
EnrootとPyxisで快適コンテナ生活
Kuninobu SaSaki
 
Linux field-update-2015
Chris Simmonds
 
ZynqMPのブートとパワーマネージメント : (ZynqMP Boot and Power Management)
Mr. Vengineer
 
TensorFlow Lite for mobile & IoT
Mia Chang
 
Fun with Network Interfaces
Kernel TLV
 
Yocto bspを作ってみた
wata2ki
 
Profiling deep learning network using NVIDIA nsight systems
Jack (Jaegeun) Han
 
root権限無しでKubernetesを動かす
Akihiro Suda
 
Kubernetes meetup-tokyo-13-customizing-kubernetes-for-ml-cluster
Preferred Networks
 
マルチコアを用いた画像処理
Norishige Fukushima
 
10GbE時代のネットワークI/O高速化
Takuya ASADA
 
QEMU - Binary Translation
Jiann-Fuh Liaw
 

Similar to Machine Learning on Your Hand - Introduction to Tensorflow Lite Preview (20)

PDF
Google APAC Machine Learning Expert Day
Evan Lin
 
PDF
Koss Lab 세미나 오픈소스 인공지능(AI) 프레임웍파헤치기
Mario Cho
 
PDF
Bringing TensorFlow to Android - a War Story
Yoni Tsafir
 
PDF
Bringing TensorFlow to Android: a war story - Yoni Tsafir, JoyTunes
DroidConTLV
 
PDF
Python enterprise vento di liberta
Simone Federici
 
PPTX
2019 04-23-tf lite-avid-f
Avid Farhoodfar, PhD, MSSE, IEEE Santa Clara CE
 
PDF
TFLite NNAPI and GPU Delegates
Koan-Sin Tan
 
PPTX
Technology and AI sharing - From 2016 to Y2017 and Beyond
James Huang
 
PPTX
Deep learning on mobile
Anirudh Koul
 
PPTX
Simplifying training deep and serving learning models with big data in python...
Holden Karau
 
PPTX
Powering Tensorflow with big data using Apache Beam, Flink, and Spark - OSCON...
Holden Karau
 
PDF
Monitoring of GPU Usage with Tensorflow Models Using Prometheus
Databricks
 
PDF
Presentation Resources - H2O Gen AI Ecosystem Overview - Level 2
Sri Ambati
 
PPTX
Flutter Beta but Better and Better
Donghyeok Kang
 
PDF
Supercharge Your AI Development with Local LLMs
Francesco Corti
 
PDF
Aprendizaje automático aplicado utilizando TensorFlow
Nicolas Bortolotti
 
PPTX
The Project Compass - GDG on Campus MSIT
dscmsitkol
 
PDF
How to Choose a Deep Learning Framework
Navid Kalaei
 
PPTX
Nautral Langauge Processing - Basics / Non Technical
Dhruv Gohil
 
PDF
Session 1 - The Current Landscape of Big Data Benchmarks
DataBench
 
Google APAC Machine Learning Expert Day
Evan Lin
 
Koss Lab 세미나 오픈소스 인공지능(AI) 프레임웍파헤치기
Mario Cho
 
Bringing TensorFlow to Android - a War Story
Yoni Tsafir
 
Bringing TensorFlow to Android: a war story - Yoni Tsafir, JoyTunes
DroidConTLV
 
Python enterprise vento di liberta
Simone Federici
 
2019 04-23-tf lite-avid-f
Avid Farhoodfar, PhD, MSSE, IEEE Santa Clara CE
 
TFLite NNAPI and GPU Delegates
Koan-Sin Tan
 
Technology and AI sharing - From 2016 to Y2017 and Beyond
James Huang
 
Deep learning on mobile
Anirudh Koul
 
Simplifying training deep and serving learning models with big data in python...
Holden Karau
 
Powering Tensorflow with big data using Apache Beam, Flink, and Spark - OSCON...
Holden Karau
 
Monitoring of GPU Usage with Tensorflow Models Using Prometheus
Databricks
 
Presentation Resources - H2O Gen AI Ecosystem Overview - Level 2
Sri Ambati
 
Flutter Beta but Better and Better
Donghyeok Kang
 
Supercharge Your AI Development with Local LLMs
Francesco Corti
 
Aprendizaje automático aplicado utilizando TensorFlow
Nicolas Bortolotti
 
The Project Compass - GDG on Campus MSIT
dscmsitkol
 
How to Choose a Deep Learning Framework
Navid Kalaei
 
Nautral Langauge Processing - Basics / Non Technical
Dhruv Gohil
 
Session 1 - The Current Landscape of Big Data Benchmarks
DataBench
 
Ad

More from Modulabs (7)

PPTX
Debugging with tensor board
Modulabs
 
PDF
TensorFlow.js & Applied AI at the Coca-Cola Company
Modulabs
 
PDF
Rear-World Robot Learning
Modulabs
 
PDF
TF.data & Eager Execution
Modulabs
 
PDF
Machine learnig with ModuLABS
Modulabs
 
PPTX
쫄지말자딥러닝2 - CNN RNN 포함버전
Modulabs
 
PDF
[모두의연구소] 쫄지말자딥러닝
Modulabs
 
Debugging with tensor board
Modulabs
 
TensorFlow.js & Applied AI at the Coca-Cola Company
Modulabs
 
Rear-World Robot Learning
Modulabs
 
TF.data & Eager Execution
Modulabs
 
Machine learnig with ModuLABS
Modulabs
 
쫄지말자딥러닝2 - CNN RNN 포함버전
Modulabs
 
[모두의연구소] 쫄지말자딥러닝
Modulabs
 
Ad

Recently uploaded (20)

PDF
AOMEI Partition Assistant Crack 10.8.2 + WinPE Free Downlaod New Version 2025
bashirkhan333g
 
PPTX
Foundations of Marketo Engage - Powering Campaigns with Marketo Personalization
bbedford2
 
PPTX
In From the Cold: Open Source as Part of Mainstream Software Asset Management
Shane Coughlan
 
PPTX
ChiSquare Procedure in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PDF
유니티에서 Burst Compiler+ThreadedJobs+SIMD 적용사례
Seongdae Kim
 
PPTX
Coefficient of Variance in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PDF
Odoo CRM vs Zoho CRM: Honest Comparison 2025
Odiware Technologies Private Limited
 
PPTX
Homogeneity of Variance Test Options IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PDF
[Solution] Why Choose the VeryPDF DRM Protector Custom-Built Solution for You...
Lingwen1998
 
PDF
Download Canva Pro 2025 PC Crack Full Latest Version
bashirkhan333g
 
PPTX
OpenChain @ OSS NA - In From the Cold: Open Source as Part of Mainstream Soft...
Shane Coughlan
 
PDF
MiniTool Partition Wizard 12.8 Crack License Key LATEST
hashhshs786
 
PPTX
Milwaukee Marketo User Group - Summer Road Trip: Mapping and Personalizing Yo...
bbedford2
 
PDF
Automate Cybersecurity Tasks with Python
VICTOR MAESTRE RAMIREZ
 
PPTX
Hardware(Central Processing Unit ) CU and ALU
RizwanaKalsoom2
 
PPTX
Finding Your License Details in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PDF
Generic or Specific? Making sensible software design decisions
Bert Jan Schrijver
 
PDF
Add Background Images to Charts in IBM SPSS Statistics Version 31.pdf
Version 1 Analytics
 
PDF
4K Video Downloader Plus Pro Crack for MacOS New Download 2025
bashirkhan333g
 
PPTX
Comprehensive Risk Assessment Module for Smarter Risk Management
EHA Soft Solutions
 
AOMEI Partition Assistant Crack 10.8.2 + WinPE Free Downlaod New Version 2025
bashirkhan333g
 
Foundations of Marketo Engage - Powering Campaigns with Marketo Personalization
bbedford2
 
In From the Cold: Open Source as Part of Mainstream Software Asset Management
Shane Coughlan
 
ChiSquare Procedure in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
유니티에서 Burst Compiler+ThreadedJobs+SIMD 적용사례
Seongdae Kim
 
Coefficient of Variance in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
Odoo CRM vs Zoho CRM: Honest Comparison 2025
Odiware Technologies Private Limited
 
Homogeneity of Variance Test Options IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
[Solution] Why Choose the VeryPDF DRM Protector Custom-Built Solution for You...
Lingwen1998
 
Download Canva Pro 2025 PC Crack Full Latest Version
bashirkhan333g
 
OpenChain @ OSS NA - In From the Cold: Open Source as Part of Mainstream Soft...
Shane Coughlan
 
MiniTool Partition Wizard 12.8 Crack License Key LATEST
hashhshs786
 
Milwaukee Marketo User Group - Summer Road Trip: Mapping and Personalizing Yo...
bbedford2
 
Automate Cybersecurity Tasks with Python
VICTOR MAESTRE RAMIREZ
 
Hardware(Central Processing Unit ) CU and ALU
RizwanaKalsoom2
 
Finding Your License Details in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
Generic or Specific? Making sensible software design decisions
Bert Jan Schrijver
 
Add Background Images to Charts in IBM SPSS Statistics Version 31.pdf
Version 1 Analytics
 
4K Video Downloader Plus Pro Crack for MacOS New Download 2025
bashirkhan333g
 
Comprehensive Risk Assessment Module for Smarter Risk Management
EHA Soft Solutions
 

Machine Learning on Your Hand - Introduction to Tensorflow Lite Preview

  • 1. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Machine Learning on Your Hand - Introduction to Tensorflow Lite Preview TF Dev Summit X ModuLABS Jaewook Kang Apr. 5th, 2018 1 All Copyright Reserved @ MoT Lab 2018
  • 2. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. ▪ GIST EEC Ph.D. (2015) ▪ 신호처리 과학자, 삽질러 ▪ MoT Lab Leader ▪ https://www.facebook.com/jwkkang ▪ 좋아하는 것: ▪ 통계적 신호처리 / 무선통신 신호처리 ▪ C++ Native 라이브러리 구현 ▪ Mobile Machine learning ▪ 수영 덕력 6년 2 ▪ 대표논문: Jaewook Kang, et al., "Bayesian Hypothesis Test using Nonparametric Belief Propagation for Noisy Sparse Recovery," IEEE Trans. on Signal process., Feb. 2015 Jaewook Kang et al., "Fast Signal Separation of 2D Sparse Mixture via Approximate Message- Passing," IEEE Signal Processing Letters, Nov. 2015 Jaewook Kang (강재욱) 소 개
  • 3. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. MoT Contributors 3 Jaewook Kang (Soundlly) Joon ho Lee (Neurophet) Yonggeun Lee () Jay Lee (Vingle) SungJin Lee (DU) Seoyoen Yang (SNU) Yunbum Beak (신호시스템)
  • 4. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al.4 •Mobilenet_quant_v1_224.tflite •테스트 디바이스: Samsung Galaxy S7 (SM-G930L) + Android 7.0 (Nougat) •빌드 환경: • Mac OSX 10.11.6 • bazel Version : 0.7 • Android Studio 3.0 • Android Build Tools Level: 26.1.1 • Android NDK Version: 16.04442984 1.mobilenet_v1_1.0_224 : 67.9 MB, Top-1 Accuracy=70.7, Top-5 Accuracy=89.5 1.mobilenet_quant_v1_224.tflite: 4.3 MB, Top-1 Accuracy=??, Top-5 Accuracy=??
  • 5. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al.5 •Mobilenet_quant_v1_224.tflite •테스트 디바이스: Samsung Galaxy S7 (SM-G930L) + Android 7.0 (Nougat) •빌드 환경: • Mac OSX 10.11.6 • bazel Version : 0.7 • Android Studio 3.0 • Android Build Tools Level: 26.1.1 • Android NDK Version: 16.04442984 1.mobilenet_v1_1.0_224 : 67.9 MB, Top-1 Accuracy=70.7, Top-5 Accuracy=89.5 1.mobilenet_quant_v1_224.tflite: 4.3 MB, Top-1 Accuracy=??, Top-5 Accuracy=??
  • 6. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Machine Learning of Things is Coming! ❖MoT Lab은 모바일 머신러닝에 관심을 갖습니다 –1. 모바일에서 머신러닝을 한다는 것! –2. Tensorflow Lite Preview version • About • Android NN API hardward acceleration • Model converting to tflite 6
  • 7. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 1. 모바일에서 머신러닝을 한다는것 - Why on-device ML? - 해결해줘야 하는 부분 7
  • 8. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖Why on-device ML? – Cloud ML의 제약 • UX 측면 8
  • 9. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖Why on-device ML? – Cloud ML의 제약 • UX 측면 – 서비스 반응 속도= » 입력 데이터 업로드시간 » +클라우드 Inference 시간 » +결과 다운로드 시간 9
  • 10. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖Why on-device ML? – Cloud ML의 제약 • UX 측면 – 서비스 반응 속도 – 오프라인 상황 10
  • 11. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖Why on-device ML? – Cloud ML의 제약 • 데이터 소모 측면 – Inference할때마다 » server call 필요 » 입력데이터 업로드 필요 – 큰 데이터 소비 APP → 순삭 ㅠ 11
  • 12. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖Why on-device ML? – Cloud ML의 제약 • 프라이버시 측면 – 개인화 ← → 프라이버시 » 개인화 서비스는 받고 싶은데 내데이터를 주는 건 싫다 • 데이터 퓨젼의 어려움 – 한 클라우드 서비스에서 다양한 개인정보를 수집하기 어려 움 » 모바일: 위치정보 / 사진 / 영상 /오디오 12
  • 13. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖해결해줘야 하는 부분 – UX + 데이터 소모 측면 → On-device inference • 반응 속도 (Fast Response) • 배터리 (Efficient Computation) • 모델 사이즈 • 메모리 제한? – 프라이버시 측면→ On-device training • 다른 사람 말고 내 얼굴을 잘 인식해라 이놈아 • Transfer learning? • Personal Data fusion 13
  • 14. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 ❖해결해줘야 하는 부분 – UX + 비용 측면 → On-device inference • 반응 속도 (Fast Response) • 배터리 (Efficient Computation) • 모델 사이즈 • 메모리 제한? – 프라이버시 측면→ On-device training • 다른 사람 말고 내 얼굴을 잘 인식해라 이놈아 • Transfer learning? • Personal Data fusion 14
  • 15. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모바일 머신러닝 15 - 하이퍼커넥트 신범준님 발표자료 중 -
  • 16. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 2. Tensorflow Lite Preview - About Tensorflow Lite - Android Neural Network API - Model conversion to tflite 16
  • 17. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About ❖A lightweight ML library and tool or mobile devices – https://www.tensorflow.org/mobile/tflite/ – 지원 플랫폼: • Android Mobile • Raspberry Pi 3 (Android Things) • iOS – 지원 ops: Tensorflow >= Tensorflow Lite – 사이즈: Core Interpreter (+supp. Ops) 70kB ( 400kB) – 버전: Developer preview (2017 Nov, w/ TF v1.5) 17
  • 18. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About 18 이미지 출처: https://www.t ensorflow.org /mobile/tflite/
  • 19. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About ❖A lightweight ML library and tool or mobile devices 19 Tflite모델을 각 플랫폼의 커널에서 사용할 수 있도록 번역하는 Api 플랫폼 별 tflite 모델을 최적화 용 op set on device HW 계산 자원 할당 최적화 이미지출처: https://www.youtube.com/watch?v=FAMfy7izB6A - Android NNAPI - iOS CoreML- Tensorflow lite framework
  • 20. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About ❖A lightweight ML library and tool or mobile devices 20 Run on device! Run on device! Run on device! 이미지출처: https://www.youtube.com/watch?v=FAMfy7izB6A - Android NNAPI - iOS CoreML- Tensorflow lite framework
  • 21. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About ❖A lightweight ML library and tool or mobile devices – iOS develop has another option! – coreML converter 따로 있음: tfcoreml github repo • (tflite+coreml >>10배 속도>> tflite+nnapi) 21 이미지출처: https://www.youtube.com/watch?v=FAMfy7izB6A
  • 22. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About ❖Why TensorFlow Lite is faster? – FlatBuffer: • A new model file format – Operation kernels optimized for NEON on ARM – Hardware acceleration support • Android NN API (Android Oreo) – Qualcomm Hexagon DSP SDK (Android P) – Direct GPU support • iOS CoreML – Metal2 – Quantization: Integer-arithmetic only support • Quantize both weights and activation as 8-bit integers • Just a few parameters(bias vectors) as 32-bit integers • 가장 범용적인 multiply-add instruction HW에서의 개선 • 용량 줄이기 보다 Inference 속도개선에 집중 22
  • 23. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. About ❖Why TensorFlow Lite is faster? – FlatBuffer: • A new model file format – Operation kernels optimized for NEON on ARM – Hardware acceleration support • Android NN API (Android Oreo) – Qualcomm Hexagon DSP SDK (Android P) – Direct GPU support • iOS CoreML – Metal2 – Quantization: Integer-arithmetic only support • Quantize both weights and activation as 8-bit integers • Just a few parameters(bias vectors) as 32-bit integers • 가장 범용적인 multiply-add instruction HW에서의 개선 • 용량 줄이기 보다 Inference 속도개선에 집중 23
  • 24. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – On-deivce에서 계산효율적 ML을 위해서 설계된 Android C/C++ API – TensorFlow Lite 모델은 Android NN API의 Kernel Interpreter로 재구 성 + 최적화 되어 계산 하드웨어에 연결됨. – Hardware-specific processing을 통해서 neural net inference 속도 개 선! • Android 에서 잘 돌아가도록 tflite모델을 재구성 + 계산 자원 분배 • 디바이스가 보유하는 계산 유닛(CPU/CPU/DSP)에 효율적으로 계산 workload를 할당 할 예정 • 현재는 CPU만 지원됨 (2018 Mar) – Supporting Android 8.1 (API level 27 + NDK level 14) or higher • - tflite + nnapi : api level >= 27, ndk level > 14 (neon arm processor 에 최적화) • - tflite only : api level >=21 (안빠름) – tflite는 nnapi가 없어도 돌지만 그 경우 전혀 빠르지 않다! 24 This slide is powered by J. Lee
  • 25. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/java/d emo/app/src/main/java/com/example/android/tflitecamerademo/ImageClassifier.java 25 This slide is powered by J. Lee - .tflite파일은 JAVA의 Interpreter 클래스가 생성될 때 내부적으로 Native C++ API를 호출하고 그 안에서 로드된다.
  • 26. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/java/d emo/app/src/main/java/com/example/android/tflitecamerademo/ImageClassifier.java 26 This slide is powered by J. Lee - .tflite파일은 JAVA의 Interpreter 클래스가 생성될 때 내부적으로 Native C++ API를 호출하고 그 안에서 로드된다. tflite inference 수행하는 class 내부에서 tflite Interpreter 실행 - tflite interpreter JAVA객체 생 성
  • 27. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/li te/java/src/main/native/nativeinterpreterwrapper_jni.cc 27 This slide is powered by J. Lee 1) JAVA API tflite.run() 이 실행되면 NativeInterpreterWrapper (JNI)를 경유해서 그 안에서 C++ API Interpreter→Invoke()을 호출
  • 28. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/li te/java/src/main/native/nativeinterpreterwrapper_jni.cc 28 This slide is powered by J. Lee 1) JAVA API tflite.run() 이 실행되면 NativeInterpreterWrapper (JNI)를 경유해서 그 안에서 C++ API Interpreter→Invoke()을 호출 Java_org_tensorflow_lite_NativeInterpreterWrapper_run()
  • 29. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 2) C++ API Interpreter→Invoke()안 에서 nnapi_delegate→Invoke() 가 호출됨 Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/li te/interpreter.cc 29 This slide is powered by J. Lee
  • 30. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 2) C++ API Interpreter→Invoke()안 에서 nnapi_delegate→Invoke() 가 호출됨 Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/li te/interpreter.cc 30 This slide is powered by J. Lee
  • 31. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 3) nnapi_delegate→invoke()안에서 nnapi_delegate→BuildGraph()호출 Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite/nnapi – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/nnapi_delegate.h – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/nnapi_delegate.cc 31 This slide is powered by J. Lee
  • 32. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 3) nnapi_delegate→invoke()안에서 nnapi_delegate→BuildGraph()호출 Android Neural Network API ❖ Android NN API 개요 – https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite/nnapi – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/nnapi_delegate.h – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/nnapi_delegate.cc 32 This slide is powered by J. Lee
  • 33. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – 33 This slide is powered by J. Lee 1) .tflite파일를 JAVA/C++ API 을 통해서 로드해서 2) C++ Android Kernal Interpreter를 통해서 NNAPI 클래스로 넘겨주고 3) C++ NNAPI Op set을 이용해 서 내부에서 tflite 모델을 low- level로 내부적으로 빌드한다. 4) low-level tflite 모델을 NNAPI를 통해서 실행한다.
  • 34. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NN API 개요 – https://developer.android.com/ndk/reference/neural_networks_8h.html – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/nnapi_delegate.h – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/nnapi_delegate.cc 34 This slide is powered by J. Lee NNAPI class methods - For Model build and compile - For Model execution
  • 35. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Hardware acceleration via Android NN API ❖ Android NN API 개요 – https://developer.android.com/ndk/guides/neuralnetworks/index.html 35 This slide is powered by J. Lee Android NN API Op set - nnapi_delegate→BuildGraph() 에서 생성되는 Low-level 모델은 다음과 같은 NNAPI op set을 사용해서 구성된다.
  • 36. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖ Android NNAPI 프로그래밍 flow – https://developer.android.com/ndk/guides/neuralnetworks/index.html 36 This slide is powered by J. Lee 모델생성 Building and Compiling an NNAPI model into lower- level code Inference실행 종료대기
  • 37. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Android Neural Network API ❖더 궁금하시면? – https://developer.android.com/ndk/guides/neuralnetworks/index.html – 또는 MoT로 오세요! 같이 공부해요! – 머신러닝에 관심있는 안드로이드 개발자 모집중! • Android NN API • JAVA Native Interface • Android Things • Tensorflow Lite 37
  • 38. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. From TF model to Android APP build 38 이미지 출처: https://www.t ensorflow.org /mobile/tflite/
  • 39. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. From TF model to Android APP build 39 이미지 출처: https://www.t ensorflow.org /mobile/tflite/
  • 40. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Model converting to tflite ❖전체 Tf model to Tflite 변환 과정 40 Get a Model Exporting the Inference Graph Freezing the exported Graph Conversion to TFLITE • Model Design or Downloading • Training with training graph • Fine Tunning • Evaluate the performance with Inference graph Convert • Graph def (.pb) • Check point (.ckpt) • Frozen graph (.pb) • Tensorflow lite (.tflite) This slide is powered by S. Yang
  • 41. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Model converting to tflite ❖두가지 방식 – Using frozen graph (.pb) – Using Saved Model 41 This slide is powered by S. Yang
  • 42. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Model converting to tflite ❖두가지 방식 – Using frozen graph (.pb) – Using Saved Model (더 쉬운방식 ㅠ, 이쪽이 답이 다) 42 This slide is powered by S. Yang
  • 43. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Model converting to tflite ❖ 1) Tensorflow에서 그래프정보 (.pb+ .ckpt) 추출! • GraphDef (.pb): TF 계산그래프의 구조 정보만을 담고 있는 객체! • 세션안에서 tf.train.write_graph()를 이용해서 저장! • CheckPoint(.cpkt): TF계산 그래프의 훈련된 weight / bias 값이 저장된 lookup table 파일 • 세션안에서 tf.train.Savor()를 이용해서 저장 43 With tf.Session(graph=graph) as sess: tf.train.write_graph(graph_or_graph_def = sess.graph_def, logdir= "models/", name= "graph.pb") Saver = tf.train.Saver() With tf.Session() as sess: saver.save(sess=sess, save_path= "models/model.ckpt”, global_step=epoch) This slide is powered by S. Yang
  • 44. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. ❖ 2) Frozen Graph (.pb) 생성 • FrozenGraphDef (.pb) : GraphDef를 checkpoint파일정보를 결합해서 variable 노드를 constant노드로 변환하여 저장한 것 • 1) Command line interface를 이용하는 방법 (freeze_graph bazel빌드 필요!) • 2) freeze_graph.py를 이용해서 파이썬 스크립트를 구성해서 변화하는 방법 Model converting to tflite 44 $ freeze_graph --input_graph=/tmp/mobilenet_v1_224.pb --input_checkpoint=/tmp/checkpoints/mobilenet-10202.ckpt --input_binary=true --output_graph=/tmp/frozen_mobilenet_v1_224.pb --output_node_names=MobileNetV1/Predictions/Reshape_1 This slide is powered by S. Yang freeze_graph.freeze_graph( input_graph=“/tmp/mobilenet_v1_224_.pb”, input_saver= "", # this argument is used with SavedModel input_binary=True, input_checkpoint=“/tmp/checkpoints/mobilenet-10202.cpkt”, output_node_names=“MobileNetV1/Predictions/Reshape_1”, restore_op_name="save/restore_all", # unused in freeze_graph() filename_tensor_name="save/Const:0", # unused in freeze_graph() output_graph=“/tmp/frozen_mobilenet_v1_224.pb ”, clear_devices=False, # not clear how to use initializer_nodes="")
  • 45. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. ❖ 3) tflite model 파일 (.tflite) 생성 • Tensorflow Lite Model (.tflite) : Tflite interpreter가 해석가능하도록 변환한 모델 파일 • 1) Command line interface를 이용하는 방법 (toco bazel빌드 필요!) • Tensorflow source파일 git clone 필요 • Tensorflow source 파일을 local repository directory에서 실행 필요 Model converting to tflite 45 $ bazel clean –expunge $ bazel run --config=opt //tensorflow/contrib/lite/toco:toco -- --input_file= input_frozen_graph_pb_path # 입력 frozen pb path --output_file= output_tflite_path # 출력 tflite path inference_type=FLOAT # floating-poing conversion input_shape=1,28,28,1 # model의 input shape input_array=input # model의 input node name (from Tensorboard) output_array=model_out/Softmax # model의 output node name (from Tensorboard) This slide is powered by S. Yang
  • 46. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. ❖ 3) tflite model 파일 (.tflite) 생성 • Tensorflow Lite Model (.tflite) : Tflite interpreter가 해석가능하도록 변환한 모델 파일 • 2) tf.contrib.lite.toco_convert를 이용해서 Tensorflow 스크립트에서 바로 생 성가능! (3/30에 push됨 ㅜ ) • https://www.tensorflow.org/versions/master/api_docs/python/tf/contrib/lite/toco_convert Model converting to tflite 46
  • 47. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Model converting to tflite 47 ❖몇가지.. 정신건강을 위해 – Tensorflow 모델을 구성할때 TF lite에서 지원하는 operator만 사용 해야한다. • 지원op리스트: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/g3doc/tf_ops_compatibility.md – Bazel 은 사전에 최신버전으로 업그레이드 하자 – Tensorboard로 input_shape/input node name/output node name확 인 필요 – 절대 경로를 사용하는게 좋다 – win10에서 동작 미확인ㅠ
  • 48. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Model converting to tflite 48 ❖Further, 여러분의 정신건강을 위해 준비했습니다. – jwkang's Tensorflow lite Github repo • A GraphDef+Checkpoint Generation from Lenet5 Tensorflow model example • A GraphDef+Checkpoint to Frozen GraphDef conversion example • A Frozen GraphDef to Tflite conversion example
  • 49. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Future Direction of Tflite ❖ More suppoting ops ❖ On-device training ❖ Improved tools ➢ Easier tflite conversion ➢ Easier platform adaptation 49
  • 50. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 참고자료 ❖ The Tensorflow GitHub doc (updated): – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lit e/toco/g3doc/cmdline_examples.md#savedmodel ❖ The Tensorflow Lite contrib repo documents – https://github.com/tensorflow/tensorflow/tree/master/tensorflow/co ntrib/lite/g3doc ❖ The Tensorflow.org documents for prepare model – https://www.tensorflow.org/mobile/prepare_models ❖ Tensorflow Lite support prebuilt models – https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lit e/g3doc/models.md 50
  • 51. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. Reviewers ❖ 김승일님 (모두연) ❖ 이일구님 (모두연) ❖ 박은수님 (모두연) ❖ 전태균님 (쎄트렉아이) ❖ 신범준님 (하이퍼커넥트) ❖ 신정규님 (레블업) 51
  • 52. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al. 모두연 MoT랩 소개 ❖딥러닝을 활용하여 세상을 이롭게 할 IoT/Mobile App 개발에 대한 연구를 같이 해봐요!! ❖ https://www.facebook.com/lab4all/posts/761099760749661 ❖ [email protected] 로 메일 ❖MoT 추가 멤버 모집중! – 모바일앱에 포팅하고 싶지만 엄두가 안나는 연구자 – 머신러닝에 관심있는 안드로이드 개발자 – TF코딩은 잘하지만 이론을 더 공부하고 싶은 TF개발자! - 서비스 기획자 52
  • 53. TF Dev Summit 2018 X Modulab: Learn by Run!! J. Kang Ph.D. et al.53 The End All copyright reserved @ MoT Lab 2018