SlideShare a Scribd company logo
Distributed Computing … and introduction of Hadoop -  Ankit Minocha
Outline Introduction to Distributed Computing Parallel vs Distributed Computing.
Computer Speedup Moore’s Law: “ The density of transistors on a chip doubles every 18 months, for the same cost”  (1965)
Scope of problems What can you do with 1 computer? What can you do with 100 computers? What can you do with an entire data center?
Rendering multiple frames of high-quality animation
Parallelization Idea Parallelization is “easy” if processing can be cleanly split into n units:
Parallelization Idea (2) In a parallel computation, we would like to have as many threads as we have processors. e.g., a four-processor computer would be able to run four threads at the same time.
Parallelization Idea (3)
Parallelization Idea (4)
Parallelization Pitfalls But this model is too simple!  How do we assign work units to worker threads? What if we have more work units than threads? How do we aggregate the results at the end? How do we know all the workers have finished? What if the work cannot be divided into completely separate tasks? What is the common theme of all of these problems?
Parallelization Pitfalls (2) Each of these problems represents a point at which multiple threads must communicate with one another, or access a shared resource. Golden rule: Any memory that can be used by multiple threads must have an associated  synchronization system !
What is Wrong With This? Thread 1: void foo() { x++; y = x; } Thread 2: void bar() { y++; x+=3; } If the initial state is y = 0, x = 6, what happens after these threads finish running?
Multithreaded = Unpredictability When we run a multithreaded program, we don’t know what order threads run in, nor do we know when they will interrupt one another. Thread 1: void foo() { eax = mem[x]; inc eax; mem[x] = eax; ebx = mem[x]; mem[y] = ebx; } Thread 2: void bar() { eax = mem[y]; inc eax; mem[y] = eax; eax = mem[x]; add eax, 3; mem[x] = eax; } Many things that look like “one step” operations actually take several steps under the hood:
Multithreaded = Unpredictability This applies to more than just integers: Pulling work units from a queue. Reporting work back to master unit Telling another thread that it can begin the “next phase” of processing eg. Torrent leechers, seeders …  All require synchronization!
Synchronization Primitives A  synchronization primitive  is a special shared variable that guarantees that it can only be accessed  atomically .  Hardware support guarantees that operations on synchronization primitives only ever take one step
Semaphores A semaphore is a flag that can be raised or lowered in one step Semaphores were flags that railroad engineers would use when entering a shared track Only one side of the semaphore can ever be red! (Can both be green?)
Semaphores set() and reset() can be thought of as lock() and unlock() Calls to lock() when the semaphore is already locked cause the thread to  block . Pitfalls: Must “bind” semaphores to particular objects; must remember to unlock correctly
The “corrected” example Thread 1: void foo() { sem.lock(); x++; y = x; sem.unlock(); } Thread 2: void bar() { sem.lock(); y++; x+=3; sem.unlock(); } Global var “Semaphore sem = new Semaphore();” guards access to x & y
Condition Variables A condition variable notifies threads that a particular condition has been met  Inform another thread that a queue now contains elements to pull from (or that it’s empty – request more elements!) Pitfall: What if nobody’s listening?
The final example Thread 1: void foo() { sem.lock(); x++; y = x; fooDone = true; sem.unlock(); fooFinishedCV.notify(); } Thread 2: void bar() { sem.lock(); if(!fooDone) fooFinishedCV.wait(sem); y++; x+=3; sem.unlock(); } Global vars: Semaphore sem = new Semaphore(); ConditionVar fooFinishedCV = new ConditionVar(); boolean fooDone = false;
Too Much Synchronization? Deadlock Synchronization becomes even more complicated when multiple locks can be used Can cause entire system to “get stuck” Thread A: semaphore1.lock(); semaphore2.lock(); /* use data guarded by  semaphores */ semaphore1.unlock();  semaphore2.unlock(); Thread B: semaphore2.lock(); semaphore1.lock(); /* use data guarded by  semaphores */ semaphore1.unlock();  semaphore2.unlock(); (Image: RPI CSCI.4210 Operating Systems notes)
The Moral: Be Careful! Synchronization is hard Need to consider all possible shared state Must keep locks organized and use them consistently and correctly Knowing there are bugs may be tricky; fixing them can be even worse! Keeping shared state to a minimum reduces total system complexity
Hadoop to the rescue!
Prelude to MapReduce We saw earlier that explicit parallelism/synchronization is hard Synchronization does not even answer questions specific to distributed computing, like how to move data from one machine to another Fortunately, MapReduce handles this for us
Prelude to MapReduce MapReduce is a paradigm designed by Google for making a subset (albeit a large one) of distributed problems easier to code Automates data distribution & result aggregation  Restricts the ways data can interact to eliminate locks (no shared state = no locks!)
Thank you so much “Clickable” for your sincere efforts.

More Related Content

Similar to Distributed computing presentation (20)

PPT
Google: Cluster computing and MapReduce: Introduction to Distributed System D...
tugrulh
 
PPT
Introduction to Cluster Computing and Map Reduce (from Google)
Sri Prasanna
 
PPT
Lec1 Intro
mobius.cn
 
PPT
Lec1 Intro
guest9b0d21
 
PPTX
84694646456445645645645665656465464sdd.pptx
mhp821023
 
PDF
Multithreading Introduction and Lifecyle of thread
Kartik Dube
 
PPTX
Multi Threading
Ferdin Joe John Joseph PhD
 
PDF
Low Level Exploits
hughpearse
 
PPT
cs2110Concurrency1.ppt
narendra551069
 
PPT
what every web and app developer should know about multithreading
Ilya Haykinson
 
PPTX
chap 7 : Threads (scjp/ocjp)
It Academy
 
PPT
[CCC-28c3] Post Memory Corruption Memory Analysis
Moabi.com
 
PPT
Java concurrency
ducquoc_vn
 
ODP
Multithreading 101
Tim Penhey
 
PDF
Programming Language Memory Models: What do Shared Variables Mean?
greenwop
 
PDF
CS844 U1 Individual Project
ThienSi Le
 
PPTX
Medical Image Processing Strategies for multi-core CPUs
Daniel Blezek
 
PPTX
Multi-threaded Programming in JAVA
Vikram Kalyani
 
PPTX
04 threads-pbl-2-slots
mha4
 
PPTX
04 threads-pbl-2-slots
mha4
 
Google: Cluster computing and MapReduce: Introduction to Distributed System D...
tugrulh
 
Introduction to Cluster Computing and Map Reduce (from Google)
Sri Prasanna
 
Lec1 Intro
mobius.cn
 
Lec1 Intro
guest9b0d21
 
84694646456445645645645665656465464sdd.pptx
mhp821023
 
Multithreading Introduction and Lifecyle of thread
Kartik Dube
 
Low Level Exploits
hughpearse
 
cs2110Concurrency1.ppt
narendra551069
 
what every web and app developer should know about multithreading
Ilya Haykinson
 
chap 7 : Threads (scjp/ocjp)
It Academy
 
[CCC-28c3] Post Memory Corruption Memory Analysis
Moabi.com
 
Java concurrency
ducquoc_vn
 
Multithreading 101
Tim Penhey
 
Programming Language Memory Models: What do Shared Variables Mean?
greenwop
 
CS844 U1 Individual Project
ThienSi Le
 
Medical Image Processing Strategies for multi-core CPUs
Daniel Blezek
 
Multi-threaded Programming in JAVA
Vikram Kalyani
 
04 threads-pbl-2-slots
mha4
 
04 threads-pbl-2-slots
mha4
 

Recently uploaded (20)

PPTX
COMPARISON OF RASTER ANALYSIS TOOLS OF QGIS AND ARCGIS
Sharanya Sarkar
 
DOCX
Python coding for beginners !! Start now!#
Rajni Bhardwaj Grover
 
PPTX
From Sci-Fi to Reality: Exploring AI Evolution
Svetlana Meissner
 
PDF
The 2025 InfraRed Report - Redpoint Ventures
Razin Mustafiz
 
PDF
What’s my job again? Slides from Mark Simos talk at 2025 Tampa BSides
Mark Simos
 
PDF
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
PPTX
Digital Circuits, important subject in CS
contactparinay1
 
PDF
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
Edge AI and Vision Alliance
 
PPTX
Agentforce World Tour Toronto '25 - MCP with MuleSoft
Alexandra N. Martinez
 
PPTX
MuleSoft MCP Support (Model Context Protocol) and Use Case Demo
shyamraj55
 
PDF
UPDF - AI PDF Editor & Converter Key Features
DealFuel
 
PDF
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
 
PDF
Bitcoin for Millennials podcast with Bram, Power Laws of Bitcoin
Stephen Perrenod
 
PDF
NLJUG Speaker academy 2025 - first session
Bert Jan Schrijver
 
PDF
LOOPS in C Programming Language - Technology
RishabhDwivedi43
 
PDF
“Computer Vision at Sea: Automated Fish Tracking for Sustainable Fishing,” a ...
Edge AI and Vision Alliance
 
PDF
Automating Feature Enrichment and Station Creation in Natural Gas Utility Net...
Safe Software
 
PDF
Kit-Works Team Study_20250627_한달만에만든사내서비스키링(양다윗).pdf
Wonjun Hwang
 
PDF
UiPath DevConnect 2025: Agentic Automation Community User Group Meeting
DianaGray10
 
PPTX
Q2 FY26 Tableau User Group Leader Quarterly Call
lward7
 
COMPARISON OF RASTER ANALYSIS TOOLS OF QGIS AND ARCGIS
Sharanya Sarkar
 
Python coding for beginners !! Start now!#
Rajni Bhardwaj Grover
 
From Sci-Fi to Reality: Exploring AI Evolution
Svetlana Meissner
 
The 2025 InfraRed Report - Redpoint Ventures
Razin Mustafiz
 
What’s my job again? Slides from Mark Simos talk at 2025 Tampa BSides
Mark Simos
 
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
Digital Circuits, important subject in CS
contactparinay1
 
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
Edge AI and Vision Alliance
 
Agentforce World Tour Toronto '25 - MCP with MuleSoft
Alexandra N. Martinez
 
MuleSoft MCP Support (Model Context Protocol) and Use Case Demo
shyamraj55
 
UPDF - AI PDF Editor & Converter Key Features
DealFuel
 
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
 
Bitcoin for Millennials podcast with Bram, Power Laws of Bitcoin
Stephen Perrenod
 
NLJUG Speaker academy 2025 - first session
Bert Jan Schrijver
 
LOOPS in C Programming Language - Technology
RishabhDwivedi43
 
“Computer Vision at Sea: Automated Fish Tracking for Sustainable Fishing,” a ...
Edge AI and Vision Alliance
 
Automating Feature Enrichment and Station Creation in Natural Gas Utility Net...
Safe Software
 
Kit-Works Team Study_20250627_한달만에만든사내서비스키링(양다윗).pdf
Wonjun Hwang
 
UiPath DevConnect 2025: Agentic Automation Community User Group Meeting
DianaGray10
 
Q2 FY26 Tableau User Group Leader Quarterly Call
lward7
 
Ad

Distributed computing presentation

  • 1. Distributed Computing … and introduction of Hadoop - Ankit Minocha
  • 2. Outline Introduction to Distributed Computing Parallel vs Distributed Computing.
  • 3. Computer Speedup Moore’s Law: “ The density of transistors on a chip doubles every 18 months, for the same cost” (1965)
  • 4. Scope of problems What can you do with 1 computer? What can you do with 100 computers? What can you do with an entire data center?
  • 5. Rendering multiple frames of high-quality animation
  • 6. Parallelization Idea Parallelization is “easy” if processing can be cleanly split into n units:
  • 7. Parallelization Idea (2) In a parallel computation, we would like to have as many threads as we have processors. e.g., a four-processor computer would be able to run four threads at the same time.
  • 10. Parallelization Pitfalls But this model is too simple! How do we assign work units to worker threads? What if we have more work units than threads? How do we aggregate the results at the end? How do we know all the workers have finished? What if the work cannot be divided into completely separate tasks? What is the common theme of all of these problems?
  • 11. Parallelization Pitfalls (2) Each of these problems represents a point at which multiple threads must communicate with one another, or access a shared resource. Golden rule: Any memory that can be used by multiple threads must have an associated synchronization system !
  • 12. What is Wrong With This? Thread 1: void foo() { x++; y = x; } Thread 2: void bar() { y++; x+=3; } If the initial state is y = 0, x = 6, what happens after these threads finish running?
  • 13. Multithreaded = Unpredictability When we run a multithreaded program, we don’t know what order threads run in, nor do we know when they will interrupt one another. Thread 1: void foo() { eax = mem[x]; inc eax; mem[x] = eax; ebx = mem[x]; mem[y] = ebx; } Thread 2: void bar() { eax = mem[y]; inc eax; mem[y] = eax; eax = mem[x]; add eax, 3; mem[x] = eax; } Many things that look like “one step” operations actually take several steps under the hood:
  • 14. Multithreaded = Unpredictability This applies to more than just integers: Pulling work units from a queue. Reporting work back to master unit Telling another thread that it can begin the “next phase” of processing eg. Torrent leechers, seeders … All require synchronization!
  • 15. Synchronization Primitives A synchronization primitive is a special shared variable that guarantees that it can only be accessed atomically . Hardware support guarantees that operations on synchronization primitives only ever take one step
  • 16. Semaphores A semaphore is a flag that can be raised or lowered in one step Semaphores were flags that railroad engineers would use when entering a shared track Only one side of the semaphore can ever be red! (Can both be green?)
  • 17. Semaphores set() and reset() can be thought of as lock() and unlock() Calls to lock() when the semaphore is already locked cause the thread to block . Pitfalls: Must “bind” semaphores to particular objects; must remember to unlock correctly
  • 18. The “corrected” example Thread 1: void foo() { sem.lock(); x++; y = x; sem.unlock(); } Thread 2: void bar() { sem.lock(); y++; x+=3; sem.unlock(); } Global var “Semaphore sem = new Semaphore();” guards access to x & y
  • 19. Condition Variables A condition variable notifies threads that a particular condition has been met Inform another thread that a queue now contains elements to pull from (or that it’s empty – request more elements!) Pitfall: What if nobody’s listening?
  • 20. The final example Thread 1: void foo() { sem.lock(); x++; y = x; fooDone = true; sem.unlock(); fooFinishedCV.notify(); } Thread 2: void bar() { sem.lock(); if(!fooDone) fooFinishedCV.wait(sem); y++; x+=3; sem.unlock(); } Global vars: Semaphore sem = new Semaphore(); ConditionVar fooFinishedCV = new ConditionVar(); boolean fooDone = false;
  • 21. Too Much Synchronization? Deadlock Synchronization becomes even more complicated when multiple locks can be used Can cause entire system to “get stuck” Thread A: semaphore1.lock(); semaphore2.lock(); /* use data guarded by semaphores */ semaphore1.unlock(); semaphore2.unlock(); Thread B: semaphore2.lock(); semaphore1.lock(); /* use data guarded by semaphores */ semaphore1.unlock(); semaphore2.unlock(); (Image: RPI CSCI.4210 Operating Systems notes)
  • 22. The Moral: Be Careful! Synchronization is hard Need to consider all possible shared state Must keep locks organized and use them consistently and correctly Knowing there are bugs may be tricky; fixing them can be even worse! Keeping shared state to a minimum reduces total system complexity
  • 23. Hadoop to the rescue!
  • 24. Prelude to MapReduce We saw earlier that explicit parallelism/synchronization is hard Synchronization does not even answer questions specific to distributed computing, like how to move data from one machine to another Fortunately, MapReduce handles this for us
  • 25. Prelude to MapReduce MapReduce is a paradigm designed by Google for making a subset (albeit a large one) of distributed problems easier to code Automates data distribution & result aggregation Restricts the ways data can interact to eliminate locks (no shared state = no locks!)
  • 26. Thank you so much “Clickable” for your sincere efforts.

Editor's Notes

  • #13: There are multiple possible final states. Y is definitely a problem, because we don’t know if it will be “1” or “7”… but X can also be 7, 10, or 11!
  • #14: Inform students that the term we want here is “race condition”
  • #19: Ask: are there still any problems? (Yes – we still have two possible outcomes. We want some other system that allows us to serialize access on an event.)
  • #20: Go over wait() / notify() / broadcast() --- must be combined with a semaphore!