Presentation on Big Data Hadoop (Summer Training Demo)
Download as PPTX, PDF•1 like•1,469 views
This document summarizes a practical training presentation on Big Data Hadoop. It was presented by Ashutosh Tiwari and Ashok Rayal from Poornima Institute of Engineering & Technology, Jaipur under the guidance of Dr. E.S. Pilli from MNIT Jaipur. The training took place from May 28th to July 9th 2014 at MNIT Jaipur and consisted of studying Hadoop and related papers, building a Hadoop cluster, and implementing a near duplicate detection project using Hadoop MapReduce. The near duplicate detection project aimed to comparatively analyze documents to find similar ones based on a predefined threshold. Snapshots of the HDFS, MapReduce processing, and output of the project are
Presentation on Big Data Hadoop (Summer Training Demo)
- 1. POORNIMA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR DEPARTMENT OF COMPUTER ENGINEERING A PRACTICAL TRAINING PRESENTATION ON BIG DATA HADOOP SESSION 2014 – 15 Presented By: Guided By: Ashutosh Tiwari Dr. E.S. Pilli CE/11/083 Assistant Professor Ashok Rayal CS, Department CE/11/025 MNIT, Jaipur.
- 2. Topics 1. Organization Details 2. Training Details 3. Technology Specification 4. Project Summary 5. Snapshots 6. Conclusion
- 3. ORGANIZATION PROFILE Name-Malviya National Institute of Techonology, Jaipur MNIT, Jaipur is one of 30 national institutes of technology in India. MNIT, established in 1963 inspired by Pt. Madan Mohan Malviya. The institute's director is I. K. Bhat and the chairman of the board of Governors is Dr. K. K. Aggarwal. Organization’s contacts: Email : [email protected] Website : www.mnit.ac.in
- 4. Training Details Start Date: 28/05/2014 Last Date: 9/07/2014 No. Of Days: 45(30+15). Timing: 9 AM to 5 PM Our training at MNIT were broadly divided into three phases: o Case study of Hadoop and related papers (first 30 days). o Hadoop cluster making (first 30 days). o Implementation of Near Duplicate Detection Using Hadoop MapReduce (last 15 days).
- 5. ABOUT PROJECT Near Duplicate Detection: Comparative analysis of millions documents exist in network jargon to find similar document based on a predefined threshold value. Near duplicate detection is essentially used in web crawls and many others data mining tasks.
- 6. TECHNOLOGY SPECIFICATION OF PROJECT Project: Near Duplicate Detection Technology Used: Hadoop Map Reduce HDFS SSH and Shell Scripting Java
- 10. CONCLUSION Training in big data helped us to know what is the crazy trend in IT industries and how technology is becoming more fruitful to human development. Big Data is the future. Currently A lot of research is going on in this field. As data is increasing at faster rate thus there is a huge need of such tools and technology which can handle it. Hadoop is the most emerging framework used by most of big firms like Facebook, Microsoft, IBM, Yahoo, Amazon and lots of other more. Our experience at MNIT, was absolutely awesome as it has given as the platform and support for our tasks and case study.