Hadoop Platform and Application Framework Course Syllabus

Overview: This course provides a comprehensive introduction to the Hadoop ecosystem and its application framework, designed for beginners with no prior experience. You'll gain hands-on experience working with core components like HDFS, MapReduce, Spark, Pig, Hive, and HBase through practical exercises using the Cloudera virtual machine. The curriculum is self-paced, with approximately 25 hours of content, allowing learners to build foundational skills in big data processing and analysis. By the end, you'll be equipped to handle large-scale data workloads using industry-standard tools.

Module 1: Hadoop Basics

Estimated time: 2 hours

• Introduction to big data concepts
• Overview of the Hadoop ecosystem
• Introduction to the Hadoop stack and associated tools
• Hands-on exploration of the Cloudera virtual machine

Module 2: Introduction to the Hadoop Stack

Estimated time: 3 hours

• Detailed examination of HDFS components
• Understanding application execution frameworks
• Introduction to YARN
• Introduction to Tez and Spark
• Exploration of Hadoop-based applications and services

Module 3: Introduction to Hadoop Distributed File System (HDFS)

Estimated time: 3 hours

• Understanding the design goals of HDFS
• Architecture of HDFS
• Read and write processes in HDFS
• Performance tuning considerations
• Accessing HDFS data through various APIs

Module 4: Introduction to MapReduce

Estimated time: 7 hours

• Learning the MapReduce programming model
• Designing MapReduce tasks
• Executing MapReduce jobs
• Trade-offs in MapReduce processing
• Performance considerations and best practices

Module 5: Introduction to Spark

Estimated time: 9 hours

• Understanding the Spark framework
• Integration of Spark with Hadoop
• Exploring Spark's core components and functionalities
• Hands-on experience with Spark for big data processing

Module 6: Final Project

Estimated time: 4 hours

• Design and implement a complete data processing pipeline using Hadoop and Spark
• Incorporate HDFS, MapReduce, and Spark components
• Submit code, documentation, and results for evaluation

Prerequisites

• Basic understanding of computer science concepts
• Familiarity with command-line interface
• System capable of running virtual machines for hands-on exercises

What You'll Be Able to Do After

• Understand the architecture and components of the Hadoop ecosystem
• Use HDFS for distributed data storage and retrieval
• Implement data processing tasks using MapReduce
• Apply Apache Spark for scalable big data analytics
• Utilize tools like Pig, Hive, and HBase for big data analysis