Sharing Blockchain Performance Knowledge for Edge Service Development
1 like161 views
The document discusses a framework called GIAU that aims to share performance knowledge to help with edge service and blockchain application development. GIAU provides a way to capture information about blockchain software, deployment patterns, benchmarks, and experiments. It uses a microservices architecture with databases to store and search this knowledge. The goal is to help developers select appropriate blockchain technologies and deployments for different edge service topologies.
Sharing Blockchain Performance Knowledge for Edge Service Development
- 1. Sharing Blockchain Performance Knowledge for Edge Service Development Filip Rydzi Independent, Slovakia Hong-Linh Truong, Department of Computer Science http://rdsea.github.io
- 2. Content ▪ Motivation ▪ GIAU (knowledge for blockchaIn Applications and Utilities) ▪ Experiments ▪ Conclusion and future works 5th IEEE CIC 2019, Los Angeles, USA 2 1/10/2020
- 3. Motivation Edge service development: complex software topologies, multiple types of components, interactions, and protocols Blockchain application development: ▪ different blockchain systems ▪ complex interactions ▪ depending on the application, knowledge requirement can be very intensive 5th IEEE CIC 2019, Los Angeles, USA 3 blockchain ops 1/10/2020
- 4. Motivation ▪ Very challenging in mastering knowledge of both edge services and blockchain systems ▪ Key concerns in our work for collaborations among developers ▪ impacts of blockchain deployment on performance of edge services ▪ impacts of structures of edge services on performance of blockchain operations ▪ selection of blockchain systems for coupling with edge services 5th IEEE CIC 2019, Los Angeles, USA 4 1/10/2020
- 5. Example, e.g. report obstacles in the road ▪ choosing blockchain technologies and operation performance (e.g., if just use blockchain- as-a-service) ▪ managing backend deployment of blockchain systems 5th IEEE CIC 2019, Los Angeles, USA 5 1/10/2020
- 6. Contributions ▪ Which blockchain features/software artefact should be selected and deployed for which topologies? ▪ Contribution: extensible, sharing knowledge services about benchmarks, performance monitoring, and testing for developer collaboration 5th IEEE CIC 2019, Los Angeles, USA 6 1/10/2020
- 7. Types of information ▪ Blockchain information ▪ operations (e.g., mining, creating transaction, …) ▪ blockchain deployment (blockchain nodes, locations and resources) ▪ Edge services ▪ IoT/mobile components ▪ edge services ▪ cloud services ▪ Metrics ▪ time, resources, availability, failure, etc. ▪ Dependencies must be captured! 5th IEEE CIC 2019, Los Angeles, USA 7 1/10/2020
- 8. GIAU Architecture 5th IEEE CIC 2019, Los Angeles, USA 8 1/10/2020 Infrastructures External software artefact repositoriesEdge deployment Metrics Application
- 9. Linking blockchain knowledge with service & infrastructure knowledge 5th IEEE CIC 2019, Los Angeles, USA 9 1/10/2020
- 10. Capturing deployment patterns ▪ Deployment patterns ▪ important knowledge when a developer also must develop and operate the blockchain systems ▪ optimization of operations given deployment of blockchain systems ▪ Our consideration ▪ deployment of blockchain is represented in graphs (similar to edge/cloud deployment model) ▪ capture it through deployment description 5th IEEE CIC 2019, Los Angeles, USA 10 1/10/2020
- 11. Representing blockchain software artefacts ▪ We support two levels: ▪ blockchain nodes and operations ▪ Blockchain nodes at the blockchain system level ▪ represent as a whole: an executable blockchain software artefact that take a role as a node/component in the blockchain systems ▪ e.g., a Hyperledger Fabric peer node, ETH full/light/archive nodes ▪ Blockchain operations at the application level ▪ mapped to typical operations in blockchain applications ▪ operations creating, signing and submitting a transaction 5th IEEE CIC 2019, Los Angeles, USA 11 1/10/2020
- 12. Examples 5th IEEE CIC 2019, Los Angeles, USA 12 { name: 'hyperledger-fabric peer', implementation: 'hyperledger', feature: 'creator', executionEnvironment: 'docker', imageTag: 'hyperledger/fabric-peer', configuration: { organization { peer_name: "peer1", domain: "org1.example.com" }, environment_variables: { CORE_PEER_ID: "peer1", CORE_PEER_TLS_ENABLED: false, CORE_PEER_GOSSIP_USELEADERELECTION: true, CORE_PEER_GOSSIP_ORGLEADER: false, ... } } } Node level information Operation level information 1/10/2020
- 13. Modeling experiments information ▪ We manage shared information in terms of experiment management ▪ allow capturing different runtime information of the same deployment structure under different benchmarks/tests ▪ there are a lot of contextual information associated with blockchain and performance ▪ An experiment includes ▪ topology of blockchain-based edge deployment ▪ benchmark/monitoring information 5th IEEE CIC 2019, Los Angeles, USA 13 1/10/2020
- 14. Incorporating benchmark and monitoring data ▪ Key issue for the success of any sharing knowledge service ▪ Our approach is generic: ▪ allow different benchmark and monitoring information to be stored ▪ It is not our goal to say “standard” or “best information” but provide useful information ▪ Combine both document database and graph database to store information ▪ documents: software artefact and infrastructure information from the benchmarks ▪ graphs: deployment pattern 5th IEEE CIC 2019, Los Angeles, USA 14 1/10/2020
- 15. Searching knowledge ▪ Apply basic search ▪ input: a deployment pattern (derived from the application’s topology) ▪ output: a recommendation to the developer about a suitable deployment ▪ Support TOSCA ▪ we derive node types from existing TOSCA Cloudify node types 5th IEEE CIC 2019, Los Angeles, USA 15 1/10/2020 From provided deployment pattern we create interaction pairs and use them to search possibilities (not optimal but work)
- 16. Prototype ▪ Microservices architecture ▪ Typescript/NodeJS ▪ MongoDB/Neo4J for knowledge store 5th IEEE CIC 2019, Los Angeles, USA 16 https://github.com/rdsea/blockchainbenmarkservice/tree/master/giau 1/10/2020
- 17. Experiments ▪ Challenging: difficult to find a real performance for blockain based edge services → we have also developed our own benchmark (not in this talk) https://www.researchgate.net/publication/333388734_Benchmarking_Blockchain_Interaction s_in_Mobile_Edge_Cloud_Software_Systems ▪ Use both realistic data and emulated data ▪ real data: based on 324 benchmarks we run with Hyperledger and Ethereum using cloud resources ▪ emulated data: 250 diverse deployment patterns, generated randomly by following a normal distribution 5th IEEE CIC 2019, Los Angeles, USA 17 1/10/2020
- 18. Example of searching knowledge node_templates: edge_dev1: type: giau.nodes.rsu relationships: - target: iot1 type: giau.relationships.nodes_network edge_node: type: giau.nodes.edge relationships: - target: edge_dev2 type: giau.relationships.nodes_network - target: edge_dev1 type: giau.relationships.nodes_network - target: iot3 type: giau.relationships.nodes_network …… 5th IEEE CIC 2019, Los Angeles, USA 18 Input Output visualized by Cloudify 1/10/2020
- 19. Input & output search example https://github.com/rdsea/blockchainbenmarkservice/tree/master/ giau/tests/data/examples 5th IEEE CIC 2019, Los Angeles, USA 19 1/10/2020
- 20. Performance Evaluation 5th IEEE CIC 2019, Los Angeles, USA 20 Configuration: Intel Core i7-6820HQ CPU, 16GB RAM memory, Ubuntu 18.04 during the testing Data: the most complex emulated deployment patterns is composed of 196 nodes (99 thing node types, 43 edge dev, 41 edge node, 12 cloud). 1/10/2020
- 21. REST APIs 5th IEEE CIC 2019, Los Angeles, USA 21 Enable benchmark data ingestion pipelines e.g., Logstash, Apache Nifi, and Python/JavaScript programs 1/10/2020
- 22. Example of ingesting benchmark results https://github.com/rdsea/blockchainbenmarkservice/tree/master/ experiments/results/benchmarks_results https://github.com/rdsea/blockchainbenmarkservice/tree/master/ utilities/results_parser https://github.com/rdsea/blockchainbenmarkservice/blob/master /utilities/results_parser/allResultsToGIAU.sh 5th IEEE CIC 2019, Los Angeles, USA 22 1/10/2020
- 23. Example of using APIs for search https://github.com/rdsea/blockchainbenmarkservice/tree/master/ giau/src/examples 5th IEEE CIC 2019, Los Angeles, USA 23 1/10/2020
- 24. Conclusions and future work ▪ Sharing knowledge about blockchain and edge deployment ▪ crucial for blockchain-based edge development ▪ GIAU is a generic microservice framework ▪ generic knowledge models and experiment management, extensible APIs for ingestion and search ▪ Future work ▪ prototype improvement, search features & integration with DevOps ▪ Prototype is under continuous development: ▪ https://github.com/rdsea/blockchainbenmarkservice/tree/master/giau 5th IEEE CIC 2019, Los Angeles, USA 24 1/10/2020
- 25. Thanks! Hong-Linh Truong Department of Computer Science rdsea.github.io 5th IEEE CIC 2019, Los Angeles, USA 25 1/10/2020 We are hiring PhD students/Postdocs! Pls. contact me if you are interested!