SF Big Analytics talk: NVIDIA FLARE: Federated Learning Application Runtime Environment for Developing Robust AI Models
1 like500 views
The document discusses NVIDIA FLARE, a federated learning application runtime environment designed to build robust AI models while addressing data privacy concerns in healthcare. It outlines its key features, including privacy-preserving algorithms, flexible architecture, and support for various machine learning frameworks. Additionally, it provides information on the system's functionality, workflows, and available resources for researchers in the field of AI and federated learning.
![NVIDIA FLARE at GTC
Please join us Monday 3/21 for FLARE Dev Day at GTC [SE1991]](https://image.slidesharecdn.com/nvflareoverviewdeck-sfanalyticsmeetup-220310223007/85/SF-Big-Analytics-talk-NVIDIA-FLARE-Federated-Learning-Application-Runtime-Environment-for-Developing-Robust-AI-Models-25-320.jpg)
SF Big Analytics talk: NVIDIA FLARE: Federated Learning Application Runtime Environment for Developing Robust AI Models
- 1. NVFLARE - NVIDIA FEDERATED LEARNING APPLICATION RUNTIME ENVIRONMENT Holger Roth ([email protected])| March 2022
- 2. BUILDING ROBUST, GENERALIZABLE AI MODELS IS HARD DATA PRIVACY Patient Privacy | Data Governance DATA PREP Expert Knowledge | Time Consuming DATA DIVERSITY Rare Diseases | Quantity 101-103
- 3. BUILDING AI FOR REAL-WORLD CLINICAL PERFORMANCE Taking Algorithms Beyond Proof-of-Concept REAL-WORLD AI DESIGN Model to Data | Generalize Model External Validation, Multiple Institutions, Prospective Data FEDERATED LEARNING PARADIGM Global Model w Only 6% of published AI studies have external validation Few included multiple institutions Kim DW, Jang HY, Kim KW, Shin Y, Park SH. Design Characteristics of Studies Reporting the Performance of Artificial Intelligence Algorithms for Diagnostic Analysis of Medical Images: Results from Recently Published Papers. Korean J Radiol. 2019 Mar;20(3):405-410. doi: 10.3348/kjr.2019.0025. PMID: 30799571; PMCID: PMC6389801. Transfer Learning “Adapt” Federated Learning “Generalize”
- 4. IMAGE TITLE GOES HERE Optional Subtitle IMAGE TITLE GOES HERE Optional Subtitle IMAGE TITLE GOES HERE Optional Subtitle IMAGE TITLE GOES HERE Optional Subtitle IMAGE TITLE GOES HERE Optional Subtitle IMAGE TITLE GOES HERE Optional Subtitle IMAGE TITLE GOES HERE Optional Subtitle MELLODDY Multi-task Learning Chemical Assays ERASMUS GENNET Genome Wide Association Study EDRN Early Detection of Pancreatic Cancer U MINNESOTA, FAIRVIEW X-RAY Covid-19 Classification EXAM COVID-19 Oxygen Requirement Prediction FEDERATED LEARNING MOMENTUM
- 5. NVIDIA FLARE § Apache License 2.0 to catalyze FL research & development § Enables Distributed, Multi-Party Collaborative Learning § Adapt existing ML/DL workflows to a Federated paradigm § Privacy Preserving Algorithms § Homomorphic Encryption & Differential Privacy § Secure Provisioning, Orchestration & Monitoring § Programmable APIs for Extensibility § Available on GitHub: https://github.com/nvidia/nvFlare Open-Source SDK for Federated Learning GPU CPU MULTI-GPU NVIDIA FLARE Federated Specification Training Flows Evaluation Flows Learning Algorithms Privacy Preserving Algorithms Management Tools Learner Confiiguration Authenticate Train Evaluate Model Updates NVIDIA FLARE Runtime Provisioning Orchestration Monitoring
- 6. NVFLARE Key Design Principles § Research friendly • Ease of experiment (including multi-site) • Flexible for innovation and extension • Support popular ML/DL frameworks • Application domain agnostic § Applicable to real world scenarios • Security and privacy • System failures and unresponsive sites • Imperfect datasets
- 7. NVIDIA FLARE Key Features https://developer.nvidia.com/flare
- 8. LEARNING ALGORITHMS Adaptive Federated Optimization (FedOpt) Federated Averaging (FedAvg) Federated Proxy (FedProx) https://developer.nvidia.com/blog/creating-robust-and-generalizable-ai-models-with-nvidia-flare/ Cyclic Weight Transfer McMahan et al. Li et al. Reddi et al. Chang et al. • Weighted average to update global model • Clients add a loss to stay close the global model. • Avoids models drifting away from global model in heterogenous datasets. • Global model is updated using an optimizer (SGD w. momentum, Adam, Yogi, Adagrad, etc.) • Models are continuously fine-tuned and circulated around institutions More to come... Algorithms can be extended • Differential privacy • Homomorphic Encryption SCAFFOLD Ditto Karimireddy et al. Li et al. • Adds correction terms during training to deal with non-IID • Fairness through personalization
- 9. NVIDIA FLARE ADDRESSES FEDERATED LEARNING PHASES DATA PREPARATION PROVISION & AUTHENTICATE FEDERATED PROGRAM RECIPE EDGE COLLABORATOR CONFIGURATIONS MONITOR & MANAGEMENT SOLUTION Provisioning startup kits SSL authentication authorization policies to control access Federated workflows Scatter-Gather, Cyclic, Eval, etc. Support any framework (TensorFlow, PyTorch, RAPIDS, Nemo etc.) Privacy preserving Federated data preparation/curation workflows Auxiliary APIs Monitoring, visualization
- 10. NVIDIA FLARE v2.0 High-level Architecture Server Client gRPC gRPC g R P C Provision Tool TCP Admin Provision P r o v i s i o n Provision Client Client … …
- 11. NVIDIA FLARE v2.0 High-level Architecture Server Provision Tool Admin Provision P r o v i s i o n Provision Client Client Client … …
- 12. NVIDIA FLARE v2.0 High-level Architecture Server Client gRPC gRPC g R P C TCP Admin Client Client … …
- 13. NVIDIA FLARE v2.0 High-level Architecture Server Client gRPC gRPC g R P C Provision Tool TCP Admin Provision P r o v i s i o n Provision Client Client API API API API … …
- 14. NVIDIA FLARE APIs Componentized Architecture § Open Provision API § Defines overall project configuration and generates mutually-trusted configuration packages for server and clients using Provisioner and Builder modules. § Server Controller API § The Controller is a python object that defines the global Federated Learning control flow via Tasks and Events. § Client Worker API § The Worker API is used to define Executors that perform Tasks orchestrated by the Server Controller API § Admin API § The Admin API provides a means to control the Federated Learning System and allows application developers to manage operation via external interfaces (e.g., Web UI). … …
- 15. Controller and Worker API Federated Workflows FL Client Worker FL Server Controller Assign Task Submit Task Result Filter Task Data Filter Task Data Filter Task Result Filter Task Result Execute Task The Controller and Worker APIs define the overall control flow via Events, Tasks, and Executors. § Inspired by HPC (OpenMPI) § The Controller defines the series of Tasks to be executed by Workers and determines how these Tasks are distributed (broadcast, cyclic, send). § The Worker implements Executors that execute specific named Tasks as defined and distributed by the Controller. § The Controller aggregates the Workers’ Task Result as defined in the Controller workflow. Filters can be used in both the Controller and Executor Task Data and Task Results.
- 16. SCATTER-GATHER CONTROLLER FOR MODEL TRAINING Typical workflow for FedAvg, FedOpt, FedProx, etc. Scatter-Gather Global Model w 1. Server initializes model 2. For number of rounds: 1. Server broadcasts global model to workers 2. Workers validate global model and train on their data 3. Workers keep track on their locally best model (Personalization) 4. Workers send back updated model or updates 5. Server Gathers (Aggregates) updates and updates the global model Source: https://github.com/NVIDIA/NVFlare/blob/main/nvflare/app_common/workflows/scatter_and_gather.py
- 17. CONTROLLERS FOR MODEL EVALUATION Global model evaluation, Cross-site model evaluation Global Model w FedEval (Global Model Validation/Cross-Site Validation) 1. Server sends models (e.g. global model and registered best local models) to each worker for evaluation 2. Server gathers the resulting metrics Metrics Metrics Metrics Best local models Site-1 Site-2 ... Site-N Global (Final) … … … … Global (Best) … … … … Site-1 … … … … Site-2 … … … … ... … … … … Site-N … … … … Models Evaluation sites Metrics Source: https://github.com/NVIDIA/NVFlare/blob/main/nvflare/app_common/workflows/global_model_eval.py https://github.com/NVIDIA/NVFlare/blob/main/nvflare/app_common/workflows/cross_site_model_eval.py
- 18. PYTHON ADMIN CLIENT Interactive control of federated experiments Docs: https://nvidia.github.io/NVFlare/user_guide/admin_commands.html Server: Client-1: Client-2: Admin client console:
- 19. PYTHON ADMIN API Automate Running FL experiments Initialization Initialize the API with actual values for the FL setup: host, port, paths to files and directories. A provisioned admin package should have ca_cert, client_cert, and client_key in the startup folder, and transfer can be created at the same level as startup. Log in with the admin name that corresponds to the provisioned package. After using FLAdminAPI, the logout() function can be called to log out. Both login() and logout() are inherited from AdminAPI. Usage Simplest sequence to upload, deploy, and start training with the “hello-pt” example app: Contents of the returned FLAdminAPIResponse can be accessed: Example: https://github.com/NVIDIA/NVFlare/blob/main/examples/cifar10/run_fl.py
- 20. INTRO EXAMPLES Github repo has multiple examples § Hello-numpy § Hello-numpy-cross-val § Hello-tf2 § Hello-pt § Hello-monai § CIFAR-10 § Prostate § BraTS
- 21. END-TO-END EXAMPLES (CIFAR10, BRATS18, PROSTATE) § Comprehensive example for researchers to compare algorithms 1. Set up a virtual environment 2. Create your FL workspace 3. Run automated experiments 1. Varying data heterogeneity of data splits 2. Centralized training 3. FedAvg on different data splits 4. Advanced FL algorithms (FedProx and FedOpt) 5. Secure aggregation using homomorphic encryption 6. Differential privacy 4. Results
- 22. CROSS-SITE VALIDATION AND GLOBAL MODEL EVALUATION Performance of locally best models (selected by best validation score on local data) using (a) local training data alone and (b) after federated learning. Source: Federated Learning for Breast Density Classification: A Real-World Implementation
- 23. FEDERATED ANALYSIS Gather summary statistics Example § Compute the local intensity histograms of each client’s data § k-anonymity (e.g., at least 10 images) § Can be enhanced with differential privacy § Compute a global histogram § Result is accessible to admin on the server
- 24. SECURITY & PRIVACY Homomorphic Encryption & Differential Privacy Differential Privacy for BraTS18 Segmentation Example: https://github.com/NVIDIA/NVFlare/tree/main/examples/brats18 validation Dice scores of the global model for 600 training epochs: Federated Learning with Homomorphic Encryption Blog: https://developer.nvidia.com/blog/federated-learning-with- homomorphic-encryption/ Example: https://github.com/NVIDIA/NVFlare/tree/main/examples/cifar10
- 25. NVIDIA FLARE at GTC Please join us Monday 3/21 for FLARE Dev Day at GTC [SE1991]