IBM Blockchain 1 IBM Switzerland Blockchain Event – 16 March 2018 © 2018 IBM
Corporation
Hyperledger Fabric
Technical Deep Dive
Arnaud J Le Hors
Senior Technical Staff Member Web & Blockchain Open Technologies
Member of the Hyperledger Project Technical Steering Committee
2
Introducing Blockchain for Business…
Blockchain
for
Business
Shared,
replicated,
permissioned
ledger
3
Problem…
… inefficient, expensive, vulnerable
Insurer
records
Auditor
records
Regulator
records
Participant
A’s records
Bank
records
Participant
B’s records
4
… with consensus, provenance, immutability and finality
Auditor
records
Regulator
records
Bank
records
Participant
B’s records
Blockchain
Insurer
records
Participant
A’s records
A shared, replicated, permissioned ledger …
5
What?
• Track diamonds across supply chain from mine to retail
How?
• Shared ledger for storing digital certification with
supporting material
Benefits
• Protect against the occurrence of fraud, theft, trafficking
and black markets
• Assist in the identification and reduction of synthetic
stones being labelled as authentic
• Increase speed of transparency for cross border
transactions for insurance companies, banks and
claimants
Legitimize Diamonds and Reduce Fraud
6
What?
• Provide a trusted source of information and traceability to
improve transparency and efficiency across the food
network.
How?
• Shared ledger for storing digital compliance
documentation, test results and audit certificates network.
Benefits
• Reduce impact of food recalls through instant access to
end-to-end traceability data to verify history in the food
network and supply chain.
• Help to address the 1 in 10 people sickened and 400,000
fatalities WW which occur every year from food-born
illnesses.
Food Trust
7
What?
• Ninety percent of goods in global trade are carried by the ocean
shipping industry each year. Costs associated with trade
documentation processing and administration are estimated to be
up to one-fifth the actual physical transportation costs.
How?
• A new blockchain solution from IBM and Maersk will help manage
and track the paper trail of tens of millions of shipping containers
across the world by digitizing the supply chain process.
Benefits
1. Enhance transparency and the highly secure sharing of
information among trading partners and customs officials.
2. Reduce fraud and errors, reduce the time products spend in the
transit and shipping process, improve inventory management
and ultimately reduce waste.
3. Potential to save the industry billions of dollars.
Cross Border Supply Chain
8
What is Hyperledger Fabric
• A foundation for developing blockchain applications for the enterprise:
– Permissioned
– Privacy
– Finality
– Performance
– Smart contracts in general purpose languages
– No “mining” or native crypto-currency required for consensus
– Execute-order-validate vs order-execute
– Highly modular
– Pluggable consensus, ledger, membership services, endorsement and validation
9
Blockchain for Business
• v1.0.0 released July 2017
• v1.1.0 released March 2018
• v1.2.0 released June 2018
• V1.3.0 October 2018
• V1.4.0 January 2019
• V1.4.1 April 2019
• V1.4.2 to be released July 2019
• V2.0 to be released 3Q2019
• Over 291 developers
• 41 companies and 87 individuals
• Over 8,000 change sets
http://hyperledger-fabric.readthedocs.io/
Technical Deep Dive
• [ Architectural Overview ]
• Network Consensus
• Channels and Ordering Service
• Components
• Network setup
• Endorsement Policies
• Membership Services
• Roadmap
11
Hyperledger Fabric Architecture
Client
Application
SDK
(HFC)
Membership
Services
Peer
Endorser
Ledger
Committer
A
Chaincode B
!Events
Ordering-Service
ü
Fabric-CA
ü
External-CA
Hyperledger Fabric Network
optionaloptional
Admin
(1) Endorse
(2) O
rder
(3)Validate
O
O O
12
Blockchain
developer
Smart
Contract
submitsdevelops
develops
recorded
accesses
event
emits
emits
D
Ledger
‘get’, ‘put’, ‘delete’
SDK
!
!
World state
block
txn txn txn
Blockchain
Peer
event
How applications interact with the ledger
Client
Application
1. Client Application in
using Hyperledger
Fabric Client (HFC)
SDK
2. Smart Contract
implemented using
chaincode – managing
the World state
13
• The Fabric ledger is maintained by each peer and includes the blockchain and worldstate
• A separate ledger is maintained for each channel the peer joins
• Transaction read/write sets are written to the blockchain
• Channel configurations are also written to the blockchain
• The worldstate can be either LevelDB (default) or CouchDB
– LevelDB is a simple key/value store
– CouchDB is a document store that allows complex queries
• The smart contact Contract decides what is written to the worldstate
Config
Block
0
Config
Block
1
Transaction
Block
2
Transaction
Block
3
Genesis
Fabric Ledger
Worldstate
Blockchain
Technical Deep Dive
• Architectural Overview
• [ Network Consensus ]
• Channels and Ordering Service
• Components
• Network setup
• Endorsement Policies
• Membership Services
• Roadmap
15
Nodes and roles
Committing Peer: Maintains ledger and state. Commits transactions.
May hold smart contract (chaincode).
Endorsing Peer: Specialized peer also endorses transactions by
receiving a transaction proposal and responds by granting or denying
endorsement. Must hold smart contract.
Ordering Node: Approves the inclusion of transaction blocks into the
ledger and communicates with committing and endorsing peer nodes.
Does not hold smart contract. Does not hold ledger.
16
Hyperledger Fabric Consensus
Consensus is achieved using the following transaction flow:
Endorse Order Validate
17
Application proposes transaction
Endorsement policy:
• “E0, E1 and E2 must sign”
• (P3, P4 are not part of the policy)
Client application submits a transaction
proposal for Smart Contract A. It must
target the required peers {E0, E1, E2}
Sample transaction: Step 1/7 – Propose transaction
E0
E1
E2
Client
Application
S
D
K
Endorser Ledger
Committing
Peer
Application
Ordering Node
Smart Contract
(Chaincode)
Endorsement
Policy
Key:
Hyperledger Fabric Network
Ordering-Service
O
O O
OP
P4P3
A
B
A
B
A
B
A
D
18
Sample transaction: Step 2/7 – Execute proposal
Endorsers Execute Proposals
E0, E1 & E2 will each execute the
proposed transaction. None of these
executions will update the ledger
Each execution will capture the set of
Read and Written data, called RW sets,
which will now flow in the fabric.
Transactions can be signed & encrypted
Key:
Hyperledger Fabric Network
Ordering-Service
Endorser Ledger
Committing
Peer
Application
Ordering Node
Smart Contract
(Chaincode)
Endorsement
Policy
O
O O
O
E0
E1
E2
P
P4P3
A
B
A
B
A
B
A
D
Client
Application
S
D
K
19
Sample transaction: Step 3/7 – Proposal Response
Application receives responses
RW sets are asynchronously returned to
application
The RW sets are signed by each
endorser, and also includes each record
version number
(This information will be checked much
later in the consensus process)
Key:
Hyperledger Fabric Network
Ordering-Service
Endorser Ledger
Committing
Peer
Application
Ordering Node
Smart Contract
(Chaincode)
Endorsement
Policy
O
O O
O
E0
E1
E2
P
P4P3
A
B
A
B
A
B
A
D
Client
Application
S
D
K
20
Sample transaction: Step 4/7 – Order Transaction
Responses submitted for ordering
Application submits responses as a
transaction to be ordered.
Ordering happens across the fabric in
parallel with transactions submitted by
other applications
(other applications)
Key:
Hyperledger Fabric Network
Ordering-Service
Endorser Ledger
Committing
Peer
Application
Ordering Node
Smart Contract
(Chaincode)
Endorsement
Policy
O
O O
O
E0
E1
E2
P
P4P3
A
B
A
B
A
B
A
D
Client
Application
S
D
K
21
Hyperledger Fabric Network
Ordering-Service
Sample transaction: Step 5/7 – Deliver Transaction
Orderer delivers to committing peers
Ordering service collects transactions
into proposed blocks for distribution to
committing peers. Peers can deliver to
other peers in a hierarchy (not shown)
Different ordering algorithms available:
• SOLO (Single node, development)
• Kafka (Crash fault tolerance)
O
O O
O
*
Key:
Endorser Ledger
Committing
Peer
Application
Ordering Node
Smart Contract
(Chaincode)
Endorsement
Policy
E0
E1
E2
P
P4P3
A
B
A
B
A
B
A
D
Client
Application
S
D
K
22
Hyperledger Fabric Network
Ordering-Service
Sample transaction: Step 6/7 – Validate Transaction
Committing peers validate transactions
Every committing peer validates against
the endorsement policy. Also check RW
sets are still valid for current world state
Validated transactions are applied to the
world state and retained on the ledger
Invalid transactions are also retained on
the ledger but do not update world state
Endorser Ledger
Committing
Peer
Application
Ordering Node
Smart Contract
(Chaincode)
Endorsement
Policy
Key:
O
O O
O
E0
E1
E2
P
P4P3
A
B
A
B
A
B
A
D
Client
Application
S
D
K
* * *
*
*
23
Client
Application
S
D
K
Hyperledger Fabric Network
Ordering-Service
Sample transaction: Step 7/7 – Notify Transaction
Committing peers notify applications
Applications can register to be notified
when transactions succeed or fail, and
when blocks are added to the ledger
Applications will be notified by each peer
to which they are connected!
!
!
!
! !
Key:
Endorser Ledger
Committing
Peer
Application
Ordering Node
Smart Contract
(Chaincode)
Endorsement
Policy
O
O O
O
E0
E1
E2
P
P4P3
A
B
A
B
A
B
A
D
Technical Deep Dive
• Architectural Overview
• Network Consensus
• [ Channels and Ordering Service ]
• Components
• Network setup
• Endorsement Policies
• Membership Services
• Roadmap
25
Ordering Service
The ordering service packages transactions into blocks to be delivered to
peers. Communication with the service is via channels.
Fabric v1.4.1 has the following options for the ordering service:
– Solo
• Single node for development
– Kafka
• Crash fault tolerant
• Requires Kafka and Zookeeper
– Raft (recommended)
• Crash fault tolerant (minimum of 3 nodes)
• No additional dependencies required
• Can be run as single node for development
• Works more efficiently than Kafka over geographically diverse networks
Ordering-Service
O
O O
26
Channels
Channels provide privacy between different ledgers
– Ledgers exist in the scope of a channel
• Channels can be shared across an entire network of peers
• Channels can be permissioned for a specific set of participants
– Chaincode is installed on peers to access the worldstate
– Chaincode is instantiated on specific channels
– Peers can participate in multiple channels
– Concurrent execution for performance and scalability
E0
E1
Ordering-Service
O
O O
O
27
Single Channel Network
• Similar to v0.6 PBFT model
• All peers connect to the same
system channel (blue).
• All peers have the same chaincode
and maintain the same ledger
• Endorsement by peers E0, E1, E2 and
E3
Key:
E1
E2
Client
Application
S
D
K
Hyperledger Fabric Network
Ordering-Service
P
A
B
A
B
A
B
E3
A
B
E0
Endorser Ledger
Committing
Peer
Application
Ordering Node
Smart Contract
(Chaincode)
Endorsement
Policy
O
O O
O
28
Multi Channel Network
• Peers E0 and E3 connect to the red
channel for chaincodes Y and Z
• E1, E2 and E3 connect to the blue
channel for chaincodes A and B
Key:
E2
Hyperledger Fabric Network
Ordering-Service
P
Y
Z
A
B
A
B
E3
Y
Z
E0
P
E1
Endorser Ledger
Committing
Peer
Application
Ordering Node
Smart Contract
(Chaincode)
Endorsement
Policy
Client
Application
S
D
K
Client
Application
S
D
K
O
O O
O
A
B
Technical Deep Dive
• Architectural Overview
• Network Consensus
• Channels and Ordering Service
• [ Components ]
• Network setup
• Endorsement Policies
• Membership Services
• Roadmap
30
Fabric Peer
A
B
0 1 2 3
Peer
Ledger
Blockchain WorldState
!Events
Chaincode
– Each peer:
– Connects to one or more channels
– Maintains one or more ledgers per channel
– Maintains installed chaincode
– Manages runtime docker containers for
instantiated chaincode
– Chaincode is instantiated on a channel
– Runtime docker container shared by
channels with same chaincode
instantiated (no state stored in container)
– Has a local MSP (Membership Services
Provider) that provides crypto material
– Emits events to the client application
Channels
Local
MSP
31
Client Application
! Events
– Each client application uses Fabric SDK to:
– Connects over channels to one or more peers
– Connects over channels to one or more
orderer nodes
– Receives events from peers
– Local MSP provides client crypto material
– Client can be written in different languages
(Node.js, Go, Java, Python?)
ChannelsClient
Application
SDK
(HFC)
Local
MSP
32
Fabric-CA
• Default (optional) Certificate Authority within
Fabric network for issuing Ecerts (long-term
identity)
• Supports clustering for HA characteristics
• Supports LDAP for user authentication
• Supports HSM for security
• Can be configured as an intermediate CA
ü
Fabric-CA
Root
Certificate Authority
High
Availability
LDAP
Authenticate
Enroll ID,
secret
HSM
Secure
DB
Technical Deep Dive
• Architectural Overview
• Network Consensus
• Channels and Ordering Service
• Components
• [ Network setup ]
• Endorsement Policies
• Membership Services
• Roadmap
34
Bootstrap Network (1/6) - Configure & Start Ordering Service
Hyperledger Fabric Network
Ordering-Service
An Ordering Service is configured and started for the network:
$ docker-compose [-f orderer.yml] ...
O
O O
O
35
Hyperledger Fabric Network
Bootstrap Network (2/6) - Configure and Start Peer Nodes
E0
E1
E2
P3
A peer is configured and started for each Endorser or Committer in the network:
$ peer node start ...
Ordering-Service
O
O O
O
36
Hyperledger Fabric Network
E0
E1
E2
P3
Bootstrap Network (3/6) - Install Chaincode
A
BA
B
A
B
Chaincode is installed onto each Endorsing Peer that needs to execute it:
$ peer chaincode install ...
Ordering-Service
O
O O
O
37
Hyperledger Fabric Network
E0
E1
E2
P3
A
BA
B
Bootstrap Network (4/6) – Create Channels
A
B
Channels are created on the ordering service:
$ peer channel create –o [orderer] ...
Ordering-Service
O
O O
O
38
Ordering-Service
O
O O
O
Hyperledger Fabric Network
E0
E1
E2
P3
A
B
A
B
A
B
Bootstrap Network (5/6) – Join Channels
Peers that are permissioned can then join the channels they want to transact on:
$ peer channel join ...
39
Ordering-Service
O
O O
O
Bootstrap Network (6/6) – Instantiate Chaincode
Hyperledger Fabric Network
E0
E1
E2
P3
A
B
A
B
A
B
Peers finally instantiate the Chaincode on the channels they want to transact on:
$ peer chaincode instantiate ... –P ‘policy’
An Endorsement Policy is specified and once instantiated chaincode can process transactions.
Technical Deep Dive
• Architectural Overview
• Network Consensus
• Channels and Ordering Service
• Components
• Network setup
• [ Endorsement Policies ]
• Membership Services
• Roadmap
41
Endorsement Policies
An endorsement policy describes the conditions by which a transaction can be
endorsed. A transaction can only be considered valid if it has been endorsed
according to its policy.
– Each chaincode is deployed with an Endorsement Policy
– ESCC (Endorsement System ChainCode) signs the proposal response on the endorsing peer
– VSCC (Validation System ChainCode) validates the endorsements
Chaincode ESCC VSCC Ledger
Propose - Execute - Respond
Order - Deliver
Validate - Commit
Sign Policy
Endorsing Peer Committing Peer
P
42
Endorsement Policy Syntax
Policy Syntax: EXPR(E[, E...])
Where EXPR is either AND or OR and E is either a principal or nested EXPR
Principal Syntax: MSP.ROLE
Supported roles are: member and admin
Where MSP is the MSP ID, and ROLE is either “member” or “admin”
$ peer chaincode instantiate
-C mychannel
-n mycc
-v 1.0
-p chaincode_example02
-c '{"Args":["init","a", "100", "b","200"]}'
-P "AND('Org1MSP.member')“
Instantiate the chaincode mycc on
channel mychannel with the policy
AND('Org1MSP.member')
43
Endorsement Policy Examples
Examples of policies:
• Request 1 signature from all three principals
– AND('Org1.member', 'Org2.member', 'Org3.member')
• Request 1 signature from either one of the two principals
– OR('Org1.member', 'Org2.member')
• Request either one signature from a member of the Org1 MSP or (1 signature from a
member of the Org2 MSP and 1 signature from a member of the Org3 MSP)
– OR('Org1.member', AND('Org2.member', 'Org3.member'))
Technical Deep Dive
• Architectural Overview
• Network Consensus
• Channels and Ordering Service
• Components
• Network setup
• Endorsement Policies
• [ Membership Services ]
• Roadmap
45
Organisations
– Each organisation defines:
• Membership Services Provider (MSP) for identities
• Administrator(s)
• Users
• Peers
• Orderers (optional)
– A network can include many organisations
representing a consortium
– Each organisation has an ID
ID = Org1
User
MSP Peer Orderer
Admin
E
E OO
Organisations define boundaries within a Fabric Blockchain Network
46
Consortium Network
An example consortium network of 3 organisations
• Orgs 1 and 3 run peers
• Org 2 provides the ordering service only
OO
O O
Org1 Org3Org2
Hyperledger Fabric Network
E
E
Z
Y
E
A
B
E
User User
Admin
Admin
Admin
Y
Z
A
B
47
Membership Services Provider - Overview
• Provides identity for:
• Peers and Orderers
• Client Applications
• Administrators
• Identities can be issued by:
• Fabric-CA
• An external CA
• Provides: Authentication, Validation, Signing and
Issuance
• Supports different crypto standards with a pluggable
interface
• A network can include multiple MSPs (typically 1 per
org)
• Includes TLS crypto material for encrypted
communications
Client
Application
S
D
K
E
ü
Fabric-CA
Local
MSP
Local
MSP
Local
MSP
O O
O
Local
MSP
O
Ordering-Service
Channel
MSP
Admin
User
A MSP manages a set of identities within a distributed Fabric network
48
Each client application has a local MSP to store user identities
• Each local MSP includes:
– Keystore
• Private key for signing transactions
– Signcert
• Public x.509 certificate
• May also include TLS credentials
• Can be backed by a Hardware Security Module (HSM)
user@org1.example.com
keystore <private key>
signcert user@org1.example.com-cert.pem
User Identities
Client
Application
S
D
K
Local
MSP
49
Each Administrator has a local MSP to store their identity
• Each local MSP includes:
– Keystore
• Private key for signing transactions
– Signcert
• Public x.509 certificate
• May also include TLS credentials
• Can be backed by a Hardware Security Module (HSM)
admin@org1.example.com
keystore <private key>
signcert admin@org1.example.com-cert.pem
Admin Identities
Admin
Local
MSP
50
Each peer and orderer has a local MSP
• Each local MSP includes:
– keystore
• Private key for signing transactions
– signcert
• Public x.509 certificate
• In addition Peer/Orderer MSPs identify authorized administrators:
– admincerts
• List of administrator certificates
– cacerts
• The CA public cert for verification
– crls
• List of revoked certificates
• Peers and Orderers also receive channel MSP info
• Can be backed by a Hardware Security Module (HSM)
peer@org1.example.com
admincerts admin@org1.example.com-
cert.pem
cacerts ca.org1.example.com-cert.pem
keystore <private key>
signcert peer@org1.example.com-cert.pem
crls <list of revoked admin certificates>
P O
Peer and Orderer Identities
Local
MSP
Local
MSP
51
Channel MSP information
Channels include additional organisational MSP information
• Determines which orderers or peers can join the channel
• Determines client applications read or write access to the channel
• Stored in configuration blocks in the ledger
• Each channel MSP includes:
– admincerts
• Any public certificates for administrators
– cacerts
• The CA public certificate for this MSP
– crls
• List of revoked certificates
• Does not include any private keys for identity
ID = MSP1
admincerts admin.org1.example.com-cert.pem
cacerts ca.org1.example.com-cert.pem
crls <list of revoked admin certificates>
Channels
Channel
MSP
Channel
MSP
Channel
MSP
52
New User Registration and Enrollment
U
3. Enroll(Enroll ID, secret)
Registration and Enrollment
• Admin registers new user with
Enroll ID
• User enrolls and receives
credentials
• Additional offline registration and
enrollment options available
Client
Application
SDK
O
fabric-ca-client
1. Register(Enroll ID)
returns( secret)
returns Ecert
2. Send Enroll ID
and secret
ü
Fabric-CA
User
Local
MSP
Admin
53
Transaction Signing
All transactions within a Hyperledger Fabric network are signed by permissioned actors,
and those signatures validated
• Actors sign transactions with their enrolment private key
– Stored in their local MSP
• Components validate transactions and certificates
– Root CA certificates and CRLs stored in local MSP
– Root CA certificates and CRLs stored in Org MSP in channel
1) Sign proposal
2) Validate client
signature
5) Sign
order
3) Sign response
6) Validate client
signature
4) Validate
endorser
signature
7) Sign delivery
8) Validate all
signatures in delivery
Client
Application
S
D
K
P
O
54
Using Hyperledger Fabric
§ Application integration via:
APIs: gRPC, REST (some)
4 SDKs: Node.js, Python, Java, Go
§ Chaincode: Go, Node.js, Java
§ CLI: launch + interact with peers and interact with membersrvc/fabric-ca
Enroll / login
Peer start + stop
Channel create, join
Chaincode install, instantiate, invoke, query, upgrade
§ Getting help: docs, wiki, RocketChat, mailing list
55
Resources
• Hyperledger http://hyperledger-fabric.readthedocs.io/en/release/
– Docs + tutorials
• IBM Code: https://developer.ibm.com/code/technologies/blockchain/
– Code patterns, lectures, howtos, lab, etc
• IBM Blockchain Dev Center: https://developer.ibm.com/blockchain/
– Blockchain 101
• IBM Blockchain Platform:
– Starter Plan: https://www.ibm.com/blockchain/getting-started.html
Thank you
Arnaud J Le Hors
lehors@us.ibm.com
@lehors
www.ibm.com/blockchain
developer.ibm.com/blockchain
www.hyperledger.org
© Copyright IBM Corporation 2018.

Hyperledger Fabric Technical Deep Dive 20190618

  • 1.
    IBM Blockchain 1IBM Switzerland Blockchain Event – 16 March 2018 © 2018 IBM Corporation Hyperledger Fabric Technical Deep Dive Arnaud J Le Hors Senior Technical Staff Member Web & Blockchain Open Technologies Member of the Hyperledger Project Technical Steering Committee
  • 2.
    2 Introducing Blockchain forBusiness… Blockchain for Business Shared, replicated, permissioned ledger
  • 3.
    3 Problem… … inefficient, expensive,vulnerable Insurer records Auditor records Regulator records Participant A’s records Bank records Participant B’s records
  • 4.
    4 … with consensus,provenance, immutability and finality Auditor records Regulator records Bank records Participant B’s records Blockchain Insurer records Participant A’s records A shared, replicated, permissioned ledger …
  • 5.
    5 What? • Track diamondsacross supply chain from mine to retail How? • Shared ledger for storing digital certification with supporting material Benefits • Protect against the occurrence of fraud, theft, trafficking and black markets • Assist in the identification and reduction of synthetic stones being labelled as authentic • Increase speed of transparency for cross border transactions for insurance companies, banks and claimants Legitimize Diamonds and Reduce Fraud
  • 6.
    6 What? • Provide atrusted source of information and traceability to improve transparency and efficiency across the food network. How? • Shared ledger for storing digital compliance documentation, test results and audit certificates network. Benefits • Reduce impact of food recalls through instant access to end-to-end traceability data to verify history in the food network and supply chain. • Help to address the 1 in 10 people sickened and 400,000 fatalities WW which occur every year from food-born illnesses. Food Trust
  • 7.
    7 What? • Ninety percentof goods in global trade are carried by the ocean shipping industry each year. Costs associated with trade documentation processing and administration are estimated to be up to one-fifth the actual physical transportation costs. How? • A new blockchain solution from IBM and Maersk will help manage and track the paper trail of tens of millions of shipping containers across the world by digitizing the supply chain process. Benefits 1. Enhance transparency and the highly secure sharing of information among trading partners and customs officials. 2. Reduce fraud and errors, reduce the time products spend in the transit and shipping process, improve inventory management and ultimately reduce waste. 3. Potential to save the industry billions of dollars. Cross Border Supply Chain
  • 8.
    8 What is HyperledgerFabric • A foundation for developing blockchain applications for the enterprise: – Permissioned – Privacy – Finality – Performance – Smart contracts in general purpose languages – No “mining” or native crypto-currency required for consensus – Execute-order-validate vs order-execute – Highly modular – Pluggable consensus, ledger, membership services, endorsement and validation
  • 9.
    9 Blockchain for Business •v1.0.0 released July 2017 • v1.1.0 released March 2018 • v1.2.0 released June 2018 • V1.3.0 October 2018 • V1.4.0 January 2019 • V1.4.1 April 2019 • V1.4.2 to be released July 2019 • V2.0 to be released 3Q2019 • Over 291 developers • 41 companies and 87 individuals • Over 8,000 change sets http://hyperledger-fabric.readthedocs.io/
  • 10.
    Technical Deep Dive •[ Architectural Overview ] • Network Consensus • Channels and Ordering Service • Components • Network setup • Endorsement Policies • Membership Services • Roadmap
  • 11.
    11 Hyperledger Fabric Architecture Client Application SDK (HFC) Membership Services Peer Endorser Ledger Committer A ChaincodeB !Events Ordering-Service ü Fabric-CA ü External-CA Hyperledger Fabric Network optionaloptional Admin (1) Endorse (2) O rder (3)Validate O O O
  • 12.
    12 Blockchain developer Smart Contract submitsdevelops develops recorded accesses event emits emits D Ledger ‘get’, ‘put’, ‘delete’ SDK ! ! Worldstate block txn txn txn Blockchain Peer event How applications interact with the ledger Client Application 1. Client Application in using Hyperledger Fabric Client (HFC) SDK 2. Smart Contract implemented using chaincode – managing the World state
  • 13.
    13 • The Fabricledger is maintained by each peer and includes the blockchain and worldstate • A separate ledger is maintained for each channel the peer joins • Transaction read/write sets are written to the blockchain • Channel configurations are also written to the blockchain • The worldstate can be either LevelDB (default) or CouchDB – LevelDB is a simple key/value store – CouchDB is a document store that allows complex queries • The smart contact Contract decides what is written to the worldstate Config Block 0 Config Block 1 Transaction Block 2 Transaction Block 3 Genesis Fabric Ledger Worldstate Blockchain
  • 14.
    Technical Deep Dive •Architectural Overview • [ Network Consensus ] • Channels and Ordering Service • Components • Network setup • Endorsement Policies • Membership Services • Roadmap
  • 15.
    15 Nodes and roles CommittingPeer: Maintains ledger and state. Commits transactions. May hold smart contract (chaincode). Endorsing Peer: Specialized peer also endorses transactions by receiving a transaction proposal and responds by granting or denying endorsement. Must hold smart contract. Ordering Node: Approves the inclusion of transaction blocks into the ledger and communicates with committing and endorsing peer nodes. Does not hold smart contract. Does not hold ledger.
  • 16.
    16 Hyperledger Fabric Consensus Consensusis achieved using the following transaction flow: Endorse Order Validate
  • 17.
    17 Application proposes transaction Endorsementpolicy: • “E0, E1 and E2 must sign” • (P3, P4 are not part of the policy) Client application submits a transaction proposal for Smart Contract A. It must target the required peers {E0, E1, E2} Sample transaction: Step 1/7 – Propose transaction E0 E1 E2 Client Application S D K Endorser Ledger Committing Peer Application Ordering Node Smart Contract (Chaincode) Endorsement Policy Key: Hyperledger Fabric Network Ordering-Service O O O OP P4P3 A B A B A B A D
  • 18.
    18 Sample transaction: Step2/7 – Execute proposal Endorsers Execute Proposals E0, E1 & E2 will each execute the proposed transaction. None of these executions will update the ledger Each execution will capture the set of Read and Written data, called RW sets, which will now flow in the fabric. Transactions can be signed & encrypted Key: Hyperledger Fabric Network Ordering-Service Endorser Ledger Committing Peer Application Ordering Node Smart Contract (Chaincode) Endorsement Policy O O O O E0 E1 E2 P P4P3 A B A B A B A D Client Application S D K
  • 19.
    19 Sample transaction: Step3/7 – Proposal Response Application receives responses RW sets are asynchronously returned to application The RW sets are signed by each endorser, and also includes each record version number (This information will be checked much later in the consensus process) Key: Hyperledger Fabric Network Ordering-Service Endorser Ledger Committing Peer Application Ordering Node Smart Contract (Chaincode) Endorsement Policy O O O O E0 E1 E2 P P4P3 A B A B A B A D Client Application S D K
  • 20.
    20 Sample transaction: Step4/7 – Order Transaction Responses submitted for ordering Application submits responses as a transaction to be ordered. Ordering happens across the fabric in parallel with transactions submitted by other applications (other applications) Key: Hyperledger Fabric Network Ordering-Service Endorser Ledger Committing Peer Application Ordering Node Smart Contract (Chaincode) Endorsement Policy O O O O E0 E1 E2 P P4P3 A B A B A B A D Client Application S D K
  • 21.
    21 Hyperledger Fabric Network Ordering-Service Sampletransaction: Step 5/7 – Deliver Transaction Orderer delivers to committing peers Ordering service collects transactions into proposed blocks for distribution to committing peers. Peers can deliver to other peers in a hierarchy (not shown) Different ordering algorithms available: • SOLO (Single node, development) • Kafka (Crash fault tolerance) O O O O * Key: Endorser Ledger Committing Peer Application Ordering Node Smart Contract (Chaincode) Endorsement Policy E0 E1 E2 P P4P3 A B A B A B A D Client Application S D K
  • 22.
    22 Hyperledger Fabric Network Ordering-Service Sampletransaction: Step 6/7 – Validate Transaction Committing peers validate transactions Every committing peer validates against the endorsement policy. Also check RW sets are still valid for current world state Validated transactions are applied to the world state and retained on the ledger Invalid transactions are also retained on the ledger but do not update world state Endorser Ledger Committing Peer Application Ordering Node Smart Contract (Chaincode) Endorsement Policy Key: O O O O E0 E1 E2 P P4P3 A B A B A B A D Client Application S D K * * * * *
  • 23.
    23 Client Application S D K Hyperledger Fabric Network Ordering-Service Sampletransaction: Step 7/7 – Notify Transaction Committing peers notify applications Applications can register to be notified when transactions succeed or fail, and when blocks are added to the ledger Applications will be notified by each peer to which they are connected! ! ! ! ! ! Key: Endorser Ledger Committing Peer Application Ordering Node Smart Contract (Chaincode) Endorsement Policy O O O O E0 E1 E2 P P4P3 A B A B A B A D
  • 24.
    Technical Deep Dive •Architectural Overview • Network Consensus • [ Channels and Ordering Service ] • Components • Network setup • Endorsement Policies • Membership Services • Roadmap
  • 25.
    25 Ordering Service The orderingservice packages transactions into blocks to be delivered to peers. Communication with the service is via channels. Fabric v1.4.1 has the following options for the ordering service: – Solo • Single node for development – Kafka • Crash fault tolerant • Requires Kafka and Zookeeper – Raft (recommended) • Crash fault tolerant (minimum of 3 nodes) • No additional dependencies required • Can be run as single node for development • Works more efficiently than Kafka over geographically diverse networks Ordering-Service O O O
  • 26.
    26 Channels Channels provide privacybetween different ledgers – Ledgers exist in the scope of a channel • Channels can be shared across an entire network of peers • Channels can be permissioned for a specific set of participants – Chaincode is installed on peers to access the worldstate – Chaincode is instantiated on specific channels – Peers can participate in multiple channels – Concurrent execution for performance and scalability E0 E1 Ordering-Service O O O O
  • 27.
    27 Single Channel Network •Similar to v0.6 PBFT model • All peers connect to the same system channel (blue). • All peers have the same chaincode and maintain the same ledger • Endorsement by peers E0, E1, E2 and E3 Key: E1 E2 Client Application S D K Hyperledger Fabric Network Ordering-Service P A B A B A B E3 A B E0 Endorser Ledger Committing Peer Application Ordering Node Smart Contract (Chaincode) Endorsement Policy O O O O
  • 28.
    28 Multi Channel Network •Peers E0 and E3 connect to the red channel for chaincodes Y and Z • E1, E2 and E3 connect to the blue channel for chaincodes A and B Key: E2 Hyperledger Fabric Network Ordering-Service P Y Z A B A B E3 Y Z E0 P E1 Endorser Ledger Committing Peer Application Ordering Node Smart Contract (Chaincode) Endorsement Policy Client Application S D K Client Application S D K O O O O A B
  • 29.
    Technical Deep Dive •Architectural Overview • Network Consensus • Channels and Ordering Service • [ Components ] • Network setup • Endorsement Policies • Membership Services • Roadmap
  • 30.
    30 Fabric Peer A B 0 12 3 Peer Ledger Blockchain WorldState !Events Chaincode – Each peer: – Connects to one or more channels – Maintains one or more ledgers per channel – Maintains installed chaincode – Manages runtime docker containers for instantiated chaincode – Chaincode is instantiated on a channel – Runtime docker container shared by channels with same chaincode instantiated (no state stored in container) – Has a local MSP (Membership Services Provider) that provides crypto material – Emits events to the client application Channels Local MSP
  • 31.
    31 Client Application ! Events –Each client application uses Fabric SDK to: – Connects over channels to one or more peers – Connects over channels to one or more orderer nodes – Receives events from peers – Local MSP provides client crypto material – Client can be written in different languages (Node.js, Go, Java, Python?) ChannelsClient Application SDK (HFC) Local MSP
  • 32.
    32 Fabric-CA • Default (optional)Certificate Authority within Fabric network for issuing Ecerts (long-term identity) • Supports clustering for HA characteristics • Supports LDAP for user authentication • Supports HSM for security • Can be configured as an intermediate CA ü Fabric-CA Root Certificate Authority High Availability LDAP Authenticate Enroll ID, secret HSM Secure DB
  • 33.
    Technical Deep Dive •Architectural Overview • Network Consensus • Channels and Ordering Service • Components • [ Network setup ] • Endorsement Policies • Membership Services • Roadmap
  • 34.
    34 Bootstrap Network (1/6)- Configure & Start Ordering Service Hyperledger Fabric Network Ordering-Service An Ordering Service is configured and started for the network: $ docker-compose [-f orderer.yml] ... O O O O
  • 35.
    35 Hyperledger Fabric Network BootstrapNetwork (2/6) - Configure and Start Peer Nodes E0 E1 E2 P3 A peer is configured and started for each Endorser or Committer in the network: $ peer node start ... Ordering-Service O O O O
  • 36.
    36 Hyperledger Fabric Network E0 E1 E2 P3 BootstrapNetwork (3/6) - Install Chaincode A BA B A B Chaincode is installed onto each Endorsing Peer that needs to execute it: $ peer chaincode install ... Ordering-Service O O O O
  • 37.
    37 Hyperledger Fabric Network E0 E1 E2 P3 A BA B BootstrapNetwork (4/6) – Create Channels A B Channels are created on the ordering service: $ peer channel create –o [orderer] ... Ordering-Service O O O O
  • 38.
    38 Ordering-Service O O O O Hyperledger FabricNetwork E0 E1 E2 P3 A B A B A B Bootstrap Network (5/6) – Join Channels Peers that are permissioned can then join the channels they want to transact on: $ peer channel join ...
  • 39.
    39 Ordering-Service O O O O Bootstrap Network(6/6) – Instantiate Chaincode Hyperledger Fabric Network E0 E1 E2 P3 A B A B A B Peers finally instantiate the Chaincode on the channels they want to transact on: $ peer chaincode instantiate ... –P ‘policy’ An Endorsement Policy is specified and once instantiated chaincode can process transactions.
  • 40.
    Technical Deep Dive •Architectural Overview • Network Consensus • Channels and Ordering Service • Components • Network setup • [ Endorsement Policies ] • Membership Services • Roadmap
  • 41.
    41 Endorsement Policies An endorsementpolicy describes the conditions by which a transaction can be endorsed. A transaction can only be considered valid if it has been endorsed according to its policy. – Each chaincode is deployed with an Endorsement Policy – ESCC (Endorsement System ChainCode) signs the proposal response on the endorsing peer – VSCC (Validation System ChainCode) validates the endorsements Chaincode ESCC VSCC Ledger Propose - Execute - Respond Order - Deliver Validate - Commit Sign Policy Endorsing Peer Committing Peer P
  • 42.
    42 Endorsement Policy Syntax PolicySyntax: EXPR(E[, E...]) Where EXPR is either AND or OR and E is either a principal or nested EXPR Principal Syntax: MSP.ROLE Supported roles are: member and admin Where MSP is the MSP ID, and ROLE is either “member” or “admin” $ peer chaincode instantiate -C mychannel -n mycc -v 1.0 -p chaincode_example02 -c '{"Args":["init","a", "100", "b","200"]}' -P "AND('Org1MSP.member')“ Instantiate the chaincode mycc on channel mychannel with the policy AND('Org1MSP.member')
  • 43.
    43 Endorsement Policy Examples Examplesof policies: • Request 1 signature from all three principals – AND('Org1.member', 'Org2.member', 'Org3.member') • Request 1 signature from either one of the two principals – OR('Org1.member', 'Org2.member') • Request either one signature from a member of the Org1 MSP or (1 signature from a member of the Org2 MSP and 1 signature from a member of the Org3 MSP) – OR('Org1.member', AND('Org2.member', 'Org3.member'))
  • 44.
    Technical Deep Dive •Architectural Overview • Network Consensus • Channels and Ordering Service • Components • Network setup • Endorsement Policies • [ Membership Services ] • Roadmap
  • 45.
    45 Organisations – Each organisationdefines: • Membership Services Provider (MSP) for identities • Administrator(s) • Users • Peers • Orderers (optional) – A network can include many organisations representing a consortium – Each organisation has an ID ID = Org1 User MSP Peer Orderer Admin E E OO Organisations define boundaries within a Fabric Blockchain Network
  • 46.
    46 Consortium Network An exampleconsortium network of 3 organisations • Orgs 1 and 3 run peers • Org 2 provides the ordering service only OO O O Org1 Org3Org2 Hyperledger Fabric Network E E Z Y E A B E User User Admin Admin Admin Y Z A B
  • 47.
    47 Membership Services Provider- Overview • Provides identity for: • Peers and Orderers • Client Applications • Administrators • Identities can be issued by: • Fabric-CA • An external CA • Provides: Authentication, Validation, Signing and Issuance • Supports different crypto standards with a pluggable interface • A network can include multiple MSPs (typically 1 per org) • Includes TLS crypto material for encrypted communications Client Application S D K E ü Fabric-CA Local MSP Local MSP Local MSP O O O Local MSP O Ordering-Service Channel MSP Admin User A MSP manages a set of identities within a distributed Fabric network
  • 48.
    48 Each client applicationhas a local MSP to store user identities • Each local MSP includes: – Keystore • Private key for signing transactions – Signcert • Public x.509 certificate • May also include TLS credentials • Can be backed by a Hardware Security Module (HSM) [email protected] keystore <private key> signcert [email protected] User Identities Client Application S D K Local MSP
  • 49.
    49 Each Administrator hasa local MSP to store their identity • Each local MSP includes: – Keystore • Private key for signing transactions – Signcert • Public x.509 certificate • May also include TLS credentials • Can be backed by a Hardware Security Module (HSM) [email protected] keystore <private key> signcert [email protected] Admin Identities Admin Local MSP
  • 50.
    50 Each peer andorderer has a local MSP • Each local MSP includes: – keystore • Private key for signing transactions – signcert • Public x.509 certificate • In addition Peer/Orderer MSPs identify authorized administrators: – admincerts • List of administrator certificates – cacerts • The CA public cert for verification – crls • List of revoked certificates • Peers and Orderers also receive channel MSP info • Can be backed by a Hardware Security Module (HSM) [email protected] admincerts [email protected] cert.pem cacerts ca.org1.example.com-cert.pem keystore <private key> signcert [email protected] crls <list of revoked admin certificates> P O Peer and Orderer Identities Local MSP Local MSP
  • 51.
    51 Channel MSP information Channelsinclude additional organisational MSP information • Determines which orderers or peers can join the channel • Determines client applications read or write access to the channel • Stored in configuration blocks in the ledger • Each channel MSP includes: – admincerts • Any public certificates for administrators – cacerts • The CA public certificate for this MSP – crls • List of revoked certificates • Does not include any private keys for identity ID = MSP1 admincerts admin.org1.example.com-cert.pem cacerts ca.org1.example.com-cert.pem crls <list of revoked admin certificates> Channels Channel MSP Channel MSP Channel MSP
  • 52.
    52 New User Registrationand Enrollment U 3. Enroll(Enroll ID, secret) Registration and Enrollment • Admin registers new user with Enroll ID • User enrolls and receives credentials • Additional offline registration and enrollment options available Client Application SDK O fabric-ca-client 1. Register(Enroll ID) returns( secret) returns Ecert 2. Send Enroll ID and secret ü Fabric-CA User Local MSP Admin
  • 53.
    53 Transaction Signing All transactionswithin a Hyperledger Fabric network are signed by permissioned actors, and those signatures validated • Actors sign transactions with their enrolment private key – Stored in their local MSP • Components validate transactions and certificates – Root CA certificates and CRLs stored in local MSP – Root CA certificates and CRLs stored in Org MSP in channel 1) Sign proposal 2) Validate client signature 5) Sign order 3) Sign response 6) Validate client signature 4) Validate endorser signature 7) Sign delivery 8) Validate all signatures in delivery Client Application S D K P O
  • 54.
    54 Using Hyperledger Fabric §Application integration via: APIs: gRPC, REST (some) 4 SDKs: Node.js, Python, Java, Go § Chaincode: Go, Node.js, Java § CLI: launch + interact with peers and interact with membersrvc/fabric-ca Enroll / login Peer start + stop Channel create, join Chaincode install, instantiate, invoke, query, upgrade § Getting help: docs, wiki, RocketChat, mailing list
  • 55.
    55 Resources • Hyperledger http://hyperledger-fabric.readthedocs.io/en/release/ –Docs + tutorials • IBM Code: https://developer.ibm.com/code/technologies/blockchain/ – Code patterns, lectures, howtos, lab, etc • IBM Blockchain Dev Center: https://developer.ibm.com/blockchain/ – Blockchain 101 • IBM Blockchain Platform: – Starter Plan: https://www.ibm.com/blockchain/getting-started.html
  • 56.
    Thank you Arnaud JLe Hors [email protected] @lehors www.ibm.com/blockchain developer.ibm.com/blockchain www.hyperledger.org © Copyright IBM Corporation 2018.