Hadoop interview questions
- 1. HADOOP INTERVIEW QUESTIONS Reach Us: Radiantits.com Contact Us: +12105037100
- 2. 1) What is Hadoop Map Reduce ? For processing large data sets in parallel across a hadoop cluster, Hadoop MapReduce framework is used. Data analysis uses a two-step map and reduce process. Reach Us: Radiantits.com Contact Us: +12105037100
- 3. 2) How Hadoop MapReduce works? In MapReduce, during the map phase it counts the words in each document, while in the reduce phase it aggregates the data as per the document spanning the entire collection. During the map phase the input data is divided into splits for analysis by map tasks running in parallel across Hadoop framework. Reach Us: Radiantits.com Contact Us: +12105037100
- 4. 3) Explain what is shuffling in MapReduce ? The process by which the system performs the sort and transfers the map outputs to the reducer as inputs is known as the shuffle. Reach Us: Radiantits.com Contact Us: +12105037100
- 5. 4) Explain what is distributed Cache in MapReduce Framework ? Distributed Cache is an important feature provided by map reduce framework. When you want to share some files across all nodes in Hadoop Cluster, DistributedCache is used. The files could be an executable jar files or simple properties file. Reach Us: Radiantits.com Contact Us: +12105037100
- 6. 5) Explain what is NameNode in Hadoop? NameNode in Hadoop is the node, where Hadoop stores all the file location information in HDFS (Hadoop Distributed File System). In other words, NameNode is the centrepiece of an HDFS file system. It keeps the record of all the files in the file system, and tracks the file data across the cluster or multiple machines. Reach Us: Radiantits.com Contact Us: +12105037100
- 7. 7) Explain what is heartbeat in HDFS? Heartbeat is referred to a signal used between a data node and Name node, and between task tracker and job tracker, if the Name node or job tracker does not respond to the signal, then it is considered there is some issues with data node or task tracker Reach Us: Radiantits.com Contact Us: +12105037100
- 8. 8) Explain what combiners is and when you should use a combiner in a MapReduce Job? To increase the efficiency of MapReduce Program, Combiners are used. The amount of data can be reduced with the help of combiner’s that need to be transferred across to the reducers. If the operation performed is commutative and associative you can use your reducer code as a combiner. The execution of combiner is not guaranteed in Hadoop Reach Us: Radiantits.com Contact Us: +12105037100
- 9. 9) What happens when a data node fails ? When a data node fails Job tracker and name node detect the failure On the failed node all tasks are re-scheduled Name node replicates the users data to another node Reach Us: Radiantits.com Contact Us: +12105037100
- 10. 10) Explain what is the function of Map Reducer partitioner? The function of Map Reducer partitioner is to make sure that all the value of a single key goes to the same reducer, eventually which helps evenly distribution of the map output over the reducers.
- 11. THANK YOU