Università di Bologna · Prof. Enrico Gallinucci · a.a. 2025/2026

Big Data

10 chapters~6.5 hours of study62 interactive widgets43 technical plates
The course in one figure: data sources feed two lanes — stored data runs the batch lane through storage, MapReduce and Spark into analytics, while live feeds bypass storage and run the streaming lane; foundations frame both. SOURCES IoT · web · social logs · sensors STORAGE HDFS · Parquet Part II · ch. 3 BATCH MapReduce · YARN Spark · SQL · ch. 4–9 STREAM Kafka · Spark Streaming · ch. 10 data stored analytics live feeds bypass storage — the near-real-time lane LATENCY ↓ · THROUGHPUT ↑ · FAULT TOLERANCE the properties every big data platform must deliver foundations (ch. 1–2) frame both lanes: definitions, lifecycle, reference architectures
Plate 00 — The whole course in one figure: sources feed two lanes. Stored data runs the batch lane (storage, then MapReduce and Spark), while live feeds bypass storage and run the streaming lane; foundations (ch. 1–2) frame both.

How to study

I

Foundations

  1. 01Big Data: definitions, drivers and lifecycle25 min · 4 widgets
  2. 02Infrastructure and reference architectures28 min · 5 widgets
II

Storage

  1. 03Distributed storage: HDFS, file formats and Parquet45 min · 6 widgets
III

Batch processing

  1. 04The MapReduce paradigm40 min · 6 widgets
  2. 05The Data Lake, YARN and Hadoop MapReduce35 min · 6 widgets
  3. 06Apache Spark: RDDs, DAG and architecture42 min · 6 widgets
  4. 07SQL-on-Hadoop, Spark SQL and Catalyst40 min · 6 widgets
  5. 08Tuning Spark: partitioning, shuffling and caching40 min · 9 widgets
  6. 09Beyond simple MapReduce38 min · 5 widgets
IV

Stream processing

  1. 10Stream processing and Spark Streaming52 min · 9 widgets