📎 Referral Code: —
🎁 Referral Earnings ₹0 →
📊 Dashboard Sign In
🎁 ₹0
Your Referral Earnings → Tap to view
Buy
🛒
Buy Course
🎬
Reels
📚
Buy Book
Personal Training
🎬
Live Classes
🎯
My 1:1 Dashboard
🗺️
Course Roadmap
🎤
Mock Interview
🧑‍💼
HR Discussion
📅
My Booked Slots
Courses & Booking
🐍
Our Course — AWS Data Engineering
▶️
Animated Roadmap — Learn with Animations
🎓
Course Marketplace — Other Trainers
🗓️
Slot Booking
Passive Income
🎁
Refer & Earn — ₹1,000 per referral
🤝
Browse Job Support
💰
Become a Supporter
🔗
Browse Referrals
📢
Post a Referral
🧑‍🏫
Teach a Course
📊
My Courses & Earnings
AI Tools
🌐
Project Explanation Agent
👤
My Profile
🏢
Recruiter Board
🛟
Support Works

Spark Architecture

Topic 5 of 22 in PySpark

Previous topic: With Spark & Without Spark Next topic: The problems of processing data on many machines
Language తెలుగు (Telugu)Englishहिन्दी (Hindi)தமிழ் (Tamil)
  1. VideoVideo
  2. ContentSpark architecture
  3. AssignmentsPractice
  4. Interview questionsQuiz

Part 1 of 4: Video

This topic is part of PySpark

Open the stage to see all 22 of its topics, with their animations, assignments and interview questions.

Buy this stage for ₹4,999 or every stage for ₹15,000 Already a student? Sign in
Go to the next part

Next: Content, spark architecture

Go back one part

Topics in PySpark

  1. Why Learn Spark?
  2. Data Filters
  3. Realtime ETL Project
  4. With Spark & Without Spark
  5. Spark Architecture
  6. The problems of processing data on many machines
  7. Inside one EC2 machine
  8. Why 128 MB partitions in Spark?
  9. Number of EC2 Machines Calculator
  10. Predicate pushdown, column pruning, partition pruning: read less from S3
  11. Narrow vs Wide transformations in Spark
  12. Transformations vs Actions in Spark
  13. Broadcast join vs Sort-merge join
  14. Handling data skew in Spark
  15. How many output files does Spark write
  16. From code to tasks: DAG, DAG Scheduler, Task Scheduler, jobs, stages, tasks and execution plans
  17. PySpark Installation Steps In Local Machine
  18. Dataframe Creation
  19. Data Selection
  20. Data Mathematical Operations
  21. Data Transformations
  22. Dataframe Joins