This module is Module 2 of Machine Learning and Data Mining, 24 hours, taught by Matteo Francia (Assistant Professor, DISI, University of Bologna, [email protected]), member of the BIG group (Business Intelligence Group) together with Prof. Golfarelli and Prof. Gallinucci. The module is presented as hands-on machine learning: theory, practice and case studies of the main steps of the CRISP-DM methodology, with coding labs in Python built mainly on pandas and scikit-learn.
| Aspect | Rule |
|---|---|
| Format | Oral, in presence, roughly 30 minutes per person |
| Content | 3 questions on different topics from the whole program (theory and practice) |
| Assignment | If you do the final assignment on Machine Learning, the 1st question is a discussion with slides about your assignment |
| Sessions | Two separate and independent exam sessions (the other with Prof. Golfarelli); the final mark is the average of the two |
| Booking | No scheduled dates: book when you are ready, at least one week in advance |
| Retry | You have to wait 1 month before trying again, in any case |
| Refusing a grade | Cannot be done more than once (University regulation) |
| Participation | Interaction during lectures and labs counts in the final evaluation |
The self-test suggested by the lecturer: open random slides and check the titles. Can you expose them and make a self-contained speech? If yes, you are prepared. This site is organised exactly for that drill — every section title is a slide title you should be able to talk about without notes.
You must do an assignment, either on Machine Learning (ask Prof. Francia) or on Data Mining (ask Prof. Golfarelli). The Machine Learning assignment is about developing — that is, coding — an AI system, in a group of 2 people at maximum. The four steps:
The procedure is: send an email of about 300 words describing the datasets, a link to the public dataset and the main challenges (the dataset must require some pre-processing); after approval, register the group and the project through the course form; then start the project on Google Colab.
Once the project is completed you write a 4-page paper in LaTeX (Overleaf) using the IEEE transactions template, with four mandatory sections: Introduction (~1 page, including group organisation and the work done by each member), Proposed Method (~1.5 pages), Results (~1 page), Conclusions (~0.5 pages). Paper, assignment and presentation go into a GitHub repository shared with the lecturer; the presentation must be 10 minutes long (no more than 10/12 slides) and the assignment must be executable on Google Colab without errors. When both members have studied and all the material is ready, both must book the exam on the same day.
"You are responsible for your code — I do not know how to program is not a valid excuse." If you cannot explain your code and your choices, and/or you copied and pasted it from colleagues or from LLMs, you will retake the exam. The same warning is repeated in the Modeling deck: if you cannot explain your code, the exam is not passed.
Everything needed is on Virtuale: slides plus Python notebooks to be opened on Google Colab. Slides and notes are sufficient to prepare for the final exam; the suggested support book is Hands-on Machine Learning with Scikit-Learn, Keras & TensorFlow by Aurélien Géron (available for free in the library — check the summary and select the topics of the course). The lecturer warns that keeping up the pace with machine learning and data mining is hard: trends and technologies develop rapidly and not all of them survive, books are easily outdated with respect to cutting-edge services, research papers often describe solutions that are not commercial yet, and in real life you will deal with a lot of (bad) documentation and online articles. Rule of thumb: understand the general concepts and fundamentals… and ask questions.
The tentative outline of the theory is the backbone of this site: 1. CRISP-DM, 2. Business Understanding, 3. Data Understanding, 4. Data Preparation, 5. Neural Networks, 6. Modeling. The labs run in parallel: Introduction to Google Colab, Data Understanding, Housing (understanding), Data Preparation, Housing (preparation), Housing (modeling), Titanic, Breast cancer.
Data mining (Chakrabarti et al. 2006) is the process of extracting and discovering patterns in large data sets, involving methods at the intersection of machine learning, statistics, and database systems. It is an interdisciplinary subfield of computer science and statistics.
Machine learning (ML) is a field of study in artificial intelligence: the development and study of statistical algorithms that can learn from data and generalize to unseen data, and thus perform tasks without explicit (coding) instructions.
The operational definition you must be able to recite is Mitchell's (1997): machine learning is the study of algorithms that improve their performance P at some task T with experience E. A well-defined learning task is given by the triple <P, T, E>. The same definition returns in the Modeling deck, together with Arthur Samuel's 1959 formulation: machine learning is the field of study that gives computers the ability to learn without being explicitly programmed.
Decomposing a problem into its T, P and E is the fastest way to tell whether you have a learning problem at all: if you cannot name the experience, you have no dataset; if you cannot name the performance measure, you have no way to say whether the model is any good.
Pick one of the problems used throughout the module and check its three components.
The clearest way to see what machine learning changes is to look at what goes into the computer and what comes out of it.
The lecturer makes the inversion concrete with the simplest possible task: build a calculator that computes the sum of two numbers, Z = X + Y.
[(x=0,y=0), (x=1,y=0), (x=0,y=1), …]; the program is three lines of Python (def sum(x, y): return x + y); the computer returns the outputs.Supervised machine learning is usually divided into two steps:
| Step | What happens | Data used |
|---|---|---|
| Train | The algorithm gains experience and produces the model | Training set (features + labels) |
| Test | The model generalizes to unseen data and its performance is evaluated | Test set (features + labels kept hidden from training) |
Training and test sets are not overlapping — they do not share the same data. This single sentence is the origin of half of the practice in this course: the split, the random seed that makes the split reproducible, cross-validation, and the whole discussion on overfitting. In the test phase of the calculator example, the slide highlights in red the wrong predictions: the model is graded on data it has never seen.
Implementing a machine learning algorithm, in practice, means writing very few lines: the ML algorithm consumes training data plus labels and produces the ML model. The slide shows the whole cycle with scikit-learn (the same skeleton you will meet again in Chapter 8) and mentions weka as the classic alternative tool.
The resulting model for the calculator task is a decision tree: a hierarchy of tests on the input features, ending in leaves that carry the predicted value. The slide reads two leaves out loud:
| Condition | Predicted X + Y |
|---|---|
if x < 3.5 and y < 3.5 | 3 |
if x >= 7.5 and y < 7.5 | 12 |
The tree is the running example of the whole module: it comes back in Chapter 5 (Isolation Forest isolates anomalies with the same kind of binary partitions), in Chapter 8 (DecisionTreeClassifier on Iris, feature importance, decision boundaries as max_depth grows) and implicitly in the random forest. Keep Plate 1.2 in mind: a tree cuts the space with axis-parallel walls.
The section closes with the quotation the lecturer uses as a hinge (Fellows 1991): "Computer science is no more about computers than astronomy is about telescopes." The tools are not the subject.
CRISP-DM stands for CRoss-Industry Standard Process for Data Mining (Shearer 2000). The slide gives the official characterisation, worth knowing almost literally:
CRISP-DM breaks down the life cycle of a data mining project into six phases: business understanding, data understanding, data preparation, modeling, evaluation, deployment.
Click a phase to read what it owns, then follow an outgoing transition. Notice that the cycle never really ends.
Two things about the arrows are examinable. First, they are the most frequent dependencies, not a rigid sequence — CRISP-DM is a road map, not a waterfall. Second, the outer circle: a data mining project ends by generating better questions. Chapter 2 adds the quantitative side of the same picture — 50 to 70% of the effort goes into data preparation.
Disclaimer 1. This course is orthogonal to Information Systems and to Big Data and Cloud Platforms: it does not replace them and it does not depend on them.
Disclaimer 2. We do not do machine learning (only) for the sake of training machine learning models.
Vibe coding is an AI-assisted software development technique popularized in February 2025. It is a chatbot-based approach to creating software where:
Vibe coding is presented as a phenomenon to understand, and simultaneously as the exact behaviour that makes you fail this exam: if you cannot explain your code and your choices, or you pasted it from an LLM, you retake the exam. The reading list attached to the slide — CACM: Automating Data Science, the pieces on Gen Z facing a more competitive job market and on how AI is transforming jobs and the workplace — is there to make the point that the automatable part of the job is exactly the part that will not be yours.
Four definitions from the Encyclopedia of Machine Learning and Data Mining (Sammut and Webb 2017), plus one from the Encyclopedia of Database Systems (Liu and Özsu 2009). They matter because the synonyms circulate freely in papers, in documentation and in company meetings.
Data: facts and statistics collected together for reference or analysis.
Synonyms: data point, observation, data sample, instance, tuple, etc. — in this course you will hear all of them for the same thing: one row of the table.
Dataset (data set): a collection of data used for some specific purpose.
The "specific purpose" is not decoration: the Iris dataset exists because Fisher needed it for a taxonomic problem, and the flight-delays dataset of Chapter 5 exists to predict arrival delays. A dataset without a purpose cannot be judged fit or unfit.
Feature: properties of things, ways that we, as humans, might describe them.
Synonyms: characteristic, attribute, property, trait, etc. In scikit-learn terms (Chapter 8) the features form the matrix X, one column each; the thing you want to predict is the target array y.
(Database) query: a request for information.
Query language (Liu and Özsu 2009): a specialized programming language (e.g., SQL) for searching and changing the contents of a database.
The last third of the deck is a case study on automation, based on a research experiment (Francia, Gallinucci, Golfarelli and Rizzi 2022 on insight-based vocalization of OLAP sessions; the comparison shown was done in 2024).
Given a query result, return insights that describe it. The task requires to extract interesting patterns/insights, rank them by their interest, and select the most interesting insights given a limited word/time budget — everything automatic, without the human in the loop.
The query result used in the slides is a sales table by product department and gender:
| productDepartment | gender | quantity |
|---|---|---|
| Produce | M | 19076 |
| Produce | F | 18711 |
| Alcoholic Beverages | F | 16939 |
| Snack Foods | M | 16859 |
| Snack Foods | F | 16186 |
| Alcoholic Beverages | M | 15399 |
| Household | M | 13757 |
| Frozen Foods | M | 13644 |
| … | … | … |
| Seafood | F | 947 |
| Seafood | M | 817 |
| Carousel | M | 473 |
| Carousel | F | 368 |
The experiment imagines a session of query results. The first prompt:
You are a data scientist describing the highlights of query results. Given the
following query result in CSV format, return the most interesting quantitative
insights describing it. You can use any algorithm to compute the insights
(e.g., the ones from scikit-learn). The highlights must be 100 words at most.
[Query result is added here]And the prompt for a subsequent query obtained by drilling down the previous one, to verify how sales are characterized by gender:
The following is the result of a drill-down of the previous data. Given the
result in CSV format, return the most interesting quantitative insights
describing it also in relationship with the previous result. You can use any
algorithm to compute the insights (e.g., the ones from scikit-learn).
The highlights must be 100 words at most.
[Query result is added here]LLMs are autoregressive machine learning models that act as statistical next-word predictors (Bowman 2023) after being trained on huge datasets (Bender et al. 2021); applications built on them, such as ChatGPT, are usually oriented to general-purpose information retrieval. Several LLM-based applications for data analysis exist — deployed by LLM providers (e.g. Data Analyst) or by third parties (e.g. Data Scientist) — which let the underlying LLM invoke functions from external libraries (pandas, scikit-learn, scipy) and generate and execute external code.
The verdict on the slide is explicit: at the moment (2024), while plain LLMs have many strengths, they are not typically suitable for data-intensive tasks. Algorithmic tasks such as SQL querying and data mining are better handled by DBMSs and query engines, because:
Since business data are sensitive, they are stored in private repositories (such as data warehouses) unknown to LLMs. To feed data to an LLM, the main possibility is to use the prompt, and prompts have limits in the number of tokens composing input and answer. The figures quoted on the slide, from the OpenAI price calculator in Microsoft Azure as of 2024-06:
| Item | Value (GPT-4-32K, 2024-06) |
|---|---|
| Input | ≈ $0.06 per 103 tokens |
| Output | ≈ $0.12 per 103 tokens |
| Token/word ratio | 1000 tokens ≈ 750 words |
| Consequence | Prompting a table with 105 tuples (if feasible) costs several dollars per execution |
LLMs now allow users to attach files to the prompt, and some applications extract succinct summaries from these files and use them in place of the whole content: this can overcome the limits and the cost of tokens, but if the summary process is incorrect, it will add errors and bias to the final answer.
| Issue | What the slides say |
|---|---|
| Interpretability | The process that leads to the computation of the insights and their interest remains hidden and not interpretable to the end user. Has the model added some distortion to the data? |
| Domain-specific data | Insights depend strongly on context, since LLMs also leverage domain knowledge for their "reasoning". Semantic-rich column names are recommended, but domain knowledge can introduce bias and inconsistencies with the data at hand. Example: in precision agriculture, low temperatures could be bad for production rates but good for pest control and water management. |
| Domain-specific modules | If additional modules are needed to produce domain-specific insights, they should be embedded into the LLM through prompting or calls to external third-party libraries. |
| Libraries | LLMs produce good results with well-known libraries such as pandas and scikit-learn; with less-known libraries they can fail to produce the necessary code. |
| Development | Many LLM-based applications are commercial (or handcrafted) and not associated with research papers, with little documentation: non-empirical comparisons become unfeasible, and exactly reproducing empirical tests may be impossible. |
The argument is not "LLMs are useless". It is a division of labour: give the deterministic, data-intensive, reproducible work to engines that were designed for it, and keep for the language model what is genuinely linguistic. The same reasoning appears in Chapter 5 about the AI Act, and in Chapter 8 about explaining why a model behaves in a certain way.
Data mining (Chakrabarti et al. 2006) is the process of extracting and discovering patterns in large data sets, involving methods at the intersection of machine learning, statistics and database systems. It is an interdisciplinary subfield of computer science and statistics.
Machine learning is the study of algorithms that improve their performance P at some task T with experience E. A well-defined learning task is given by the triple <P, T, E>. Samuel's earlier formulation (1959), quoted in the Modeling deck, is: the field of study that gives computers the ability to learn without being explicitly programmed.
The role of program and output. Classical programming: data + program → output. Machine learning: data + output (the labels, the ground truth) → program (the trained model). In the calculator example Z = X + Y, classical programming provides def sum(x, y): return x + y; machine learning provides the pairs plus the correct sums and receives a model — for instance a decision tree that predicts 3 when x < 3.5 and y < 3.5.
Because the performance P must be measured on the ability to generalize to unseen data. If a sample appears in both sets, the score partly measures memorisation instead of generalisation, and the estimate of the model quality becomes optimistic. Training and test sets, by construction, do not share the same data.
1. Business understanding, 2. data understanding, 3. data preparation, 4. modeling, 5. evaluation, 6. deployment. The arrows indicate the most important and frequent dependencies between the phases — they are not a rigid waterfall. The outer circle symbolizes the cyclical nature of data mining itself: lessons learned during the process trigger new, often more focused, business questions.
A non-proprietary, documented, freely available data mining model, developed by industry leaders with input from more than 200 data mining users and service providers, industry-, tool- and application-neutral. It encourages best practices, offers the structure needed for better and faster results, and provides a road map for planning and carrying out a project.
Vibe coding (popularized in February 2025) is a chatbot-based approach where the developer describes the task to an LLM, the LLM generates the code, and the developer does not review or edit it — evaluating it only through tools and execution results, and iterating by asking for improvements. It is a problem because the exam is built on explaining your own code and choices: code you cannot explain, or that you pasted from colleagues or LLMs, means retaking the exam. And more generally: the part of the job that can be automated is exactly the part that is not your added value.
(1) Engines are optimized for these operations — the example given is using R-trees to speed up clustering in Euclidean spaces; (2) their answers are correct, consistent and reproducible; (3) they do not have hallucinations. The conclusion drawn in 2024 is that plain LLMs, despite many strengths, are not typically suitable for data-intensive tasks.
Three reasons. Privacy/locality: business data are sensitive and live in private repositories unknown to the model. Token limits: prompts are capped in the number of tokens for input and answer. Cost: as of 2024-06 GPT-4-32K was priced at about $0.06 per 103 input tokens and $0.12 per 103 output tokens, with 1000 tokens ≈ 750 words, so prompting a table with 105 tuples costs several dollars per execution. File attachments plus summarisation can bypass limits and cost, but an incorrect summary injects errors and bias.
Data: facts and statistics collected together for reference or analysis (synonyms: data point, observation, data sample, instance, tuple). Dataset: a collection of data used for some specific purpose. Feature: properties of things, ways that we as humans might describe them (synonyms: characteristic, attribute, property, trait). A query is a request for information; a query language is a specialized programming language such as SQL for searching and changing the contents of a database.
Because the added value of a data scientist is the understanding of the problem and of the data, not the ability to call a library. Code generation can be easily automated and outperform you; understanding the domain, choosing what to measure, and knowing why a model behaves in a certain way cannot. This is Disclaimer 2, and it is repeated at the start of the Modeling deck.
With an LLM-based application, the process that computes the insights and ranks their interest remains hidden and not interpretable to the end user, so you cannot tell whether the model added distortion to the data. A decision tree, by contrast, is a sequence of explicit tests on named features — you can read it as rules (x < 3.5 and y < 3.5 → 3) and check every branch. Interpretability is also the row where classical machine learning scores "good" and deep learning scores "low" in the comparison table of Chapter 7.