Lecturer

Eric Medvet

• associate Professor of Computer Engineering at Department of Engineering and Architecture, University of Trieste
• online at: medvet.inginf.units.it

Research interests:

• evolutionary computation
• embodied artificial intelligence
• machine learning applications

Labs:

• Evolutionary Robotics and Artificial Life lab
• Machine Learning lab

Computer Engineering (ING-INF/05) group

Structure of the course

1st part (6 CFUs, 48 hours): for all: IN23, IN19, SM38, SM36, SM34, SM23, SM28, SM13, and SM64

• what is machine learning?
  • definitions, recap
• evaluating an ML system
  • supervised ML, classification, binary classification, multiclass and regression, learning technique, examples, recap
• supervised learning techniques
  • tree-based, bagging, Random Forest, SVM, Naive Bayes, kNN, data pre- and post-processing, intro to R
• clustering
  • definition and assessment, hierarchical, k-means
• application to text

2nd part (3 CFUs, 24 hours): just for IN23 and IN19

• what is evolutionary computation?
• significant applications in robotics

Focus on methodology:

• how to design, build, and evaluate an ML (or EC) system?

Materials

Teacher slides:

• available on the course web page
• might be updated during the course

Notebooks for the lab activity:

• available on the course web page
• please, to fully enjoy lab activities, do not look at notebooks in advance

Textbooks:

• 1st part: James, Gareth, et al.; An introduction to statistical learning. Vol. 112. New York: springer, 2013 available in UniTs library
• 2nd part: De Jong, Kenneth A. Evolutionary Computation: A Unified Approach. MIT Press, 2006.

Disclaimer: overlap with course material is very partial!

How to attend lectures

Depending on your learning style and habits, you might want to take notes to augment the slide content.

Visual syntax

public static String sayHello(String name) {
  return "Hello %s".formatted(name);
}

Lab activities and how to attend

Focus on methodology:

• how to design, build, and evaluate an ML (or EC) system?

Practice (in designing, building, evaluating) is fundamental!

You'll practice doing lab activities:

• $\approx$ 15 hours in the 1st part
• in classroom
  • the teacher is there and always available
  • the teacher actively monitors your progresses
  • ... but you can do the activities also at home
• "solution" shown at the end
  • solution = one way of doing design, build, evaluate
• agnostic w.r.t. concrete tools used
  • teacher is more familiar with R
  • tutor is more familiar with Python
• suitable to be done in small group (2–4 students)

Lecture times

Where:

• Room H, building C1, Piazzale Europa Campus
• Room 3B, building H3, Piazzale Europa Campus
• Room 2A, building D, Piazzale Europa Campus

When:

• Monday, 16.00–19.00, H, C1 $\rightarrow$ 16.00–18.30
• Tuesday, 11.00–13.00, 3B, H3 $\rightarrow$ 11.00–12.30
• Wednesday, 10.00–12.00, 2A, D $\rightarrow$ 10.00–11.30

Tutor

Michel El Saliby

• master student enrolled in Ingegneria Elettronica e Informatica master course
• hopefully future PhD student enrolled in the Applied Data Science and Artificial Intelligence PhD program

Role of the tutor:

• assisting students during lab activities, together with the teacher
• first point-of-contact for course-related questions by students
  • the teacher is always available

Exam

The exam may be done in two ways:

1. project and written test
2. written test only

The written test consists of few ($\approx$ 6) questions, some with medium-length answer, some with short answer, to be done in 1h.

The project consists in the design, development, and assessment of an ML system dealing with one "problem" chosen among a few options (examples).

• the student delivers a description, not the software
• the description is evaluated for clarity, technical soundness, (amount of) results
• may be done in group (you are encouraged to form groups!)

The grade is the average of written test and project grades:

• both must be $\ge 18$
• parts can be repeated
• honors (lode) if and only if both parts $\ge 30$ and one $> 30$

You?

What is Machine Learning?

What is Machine Learning?

A few considerations:

• defining a field of science is hard: science evolves, its boundaries change
• ML "comes" from many communities' (statistics, computer science, ...) efforts: this (the use of computers) is just one point of view
• it captures just some "parts" of ML: we'll see

What is Machine Learning?

A few considerations:

• defining a field of science is hard: science evolves, its boundaries change
• ML "comes" from many communities' (statistics, computer science, ...) efforts: this (the use of computers) is just one point of view
• it captures just some "parts" of ML: we'll see

Let's analyze it in details:

• is the science: what's science?
• getting computer: who is doing that?
• to learn: to learn what? this appears to be the key point!
• without being explicitly programmed: who is not doing that?

An example: spam detection

Spam detection: under the hood

What the user sees:

• unwanted emails (spam) are in a separated place (the spam folder)
• sometimes some spam email is not put in the spam folder
• sometimes some not-spam email is put in the spam folder

Spam detection: under the hood

What the user sees:

• unwanted emails (spam) are in a separated place (the spam folder)
• sometimes some spam email is not put in the spam folder
• sometimes some not-spam email is put in the spam folder

What the web-based email system (a computer) does:

• whenever an email arrives, decides if it is spam or not
• if spam, move it to the spam folder; otherwise, leave it in the main place

In brief: a computer is making a decision about an email

Making a decision

Let's be formal:

Making a decision

Let's be formal:

$y=f(x)$ is a formal notation capturing the idea that $y$ is obtained from $x$ by applying $f$ on it.
But it is lacky about the nature of $x$ and $y$.

None of the two notations says how $f$ works internally.

$x$ and $y$ names

• $x$ is an observation
  • something that can be observed, right because a decision has to be made about it
• $y$ is the response (for a given $x$)
  • if you feed the decision system with an $x$, the system responds with a $y$

$x$ and $y$ names

• $x$ is an observation
  • something that can be observed, right because a decision has to be made about it
• $y$ is the response (for a given $x$)
  • if you feed the decision system with an $x$, the system responds with a $y$

Alternative names:

• $x$ is an/the input, $y$ is an/the output
  • $f$ as information processing system
• $x$ is a data point
  • assuming it carries some data about the underlying entity
• $x$ is an instance
  • instance [ˈɪnst(ə)ns]: an example or single occurrence of something

Names are used interchangeably; some communities tend to prefer some names.

to learn what?

• is the science: what's science?
• getting computer: who is doing that?
• to learn: to learn what?
  • how to make a decision $y$ about an observation $x$. That is: $f: X \to Y$
• without being explicitly programmed: who is not doing that?

to learn what?

• is the science: what's science?
• getting computer: who is doing that?
• to learn: to learn what?
  • how to make a decision $y$ about an observation $x$. That is: $f: X \to Y$
• without being explicitly programmed: who is not doing that?

New version:

we want the computer to learn $f$ and use it, not just learn it

Prediction

$f$ is often denoted as $f\subtext{predict}$ since, given an $x$, predicts a $y$

• when used in practice, i.e., in the prediction phase, $f\subtext{predict}$ guesses about an unknown, real $\hat{y}$

$f$ for a computer

Computers execute instructions grouped in programs and expressed according to some language.
$f$ is the mathematical, abstract notation for a computer program that, when executed on an input $x \in X$ , outputs a $y \in Y$.

public double f(double[] xs) {
  return Math.abs((xs[0] - xs[1]) / xs[0]);
}

Further point of view

Abstract definition ($\approx$ the signature):

• just domain and codomain, not how the function works

double f(double[] xs)

Concrete definition ($\approx$ signature and code):

• domain, codomain, and how the function works

double f(double[] xs) {
  return xs[0] + xs[1];
}

Writing $f$

Usually, computer programs are written by humans, but here:

without being explicitly programmed means that $f\subtext{predict}$ is not written by a human!

It appears verbose, let's get rid of it.

Writing $f$

Usually, computer programs are written by humans, but here:

without being explicitly programmed means that $f\subtext{predict}$ is not written by a human!

It appears verbose, let's get rid of it.

New version:

Finding/writing a program

Alice (computer science instructor) to Bob (student):
"Please, write a program that, given a string, returns the number of vowel occurrences in the string"

Alternative version:
"Please, find a program that, given a string, returns the number of vowel occurrences in the string"

"Find" suggests Bob to apprach the task in two steps:

1. consider the universe of all the possible programs
2. choose the one (or ones) that does what expected

Finding/writing a program

Alice (computer science instructor) to Bob (student):
"Please, write a program that, given a string, returns the number of vowel occurrences in the string"

Alternative version:
"Please, find a program that, given a string, returns the number of vowel occurrences in the string"

"Find" suggests Bob to apprach the task in two steps:

1. consider the universe of all the possible programs
2. choose the one (or ones) that does what expected

In $f$ terms:

1. consider $\mathcal{F}_{X \to Y} = \{f, f: X \to Y\}$
2. choose one $f \in \mathcal{F}_{X \to Y}$ that does what expected

Desired behavior of $f$

1. consider $\mathcal{F}_{X \to Y} = \{f, f: X \to Y\}$
2. choose one $f \in \mathcal{F}_{X \to Y}$ that does what expected

Step 2 is fundamental in practice

• "find a program that, given a string, returns a number" wouldn't make sense alone!

... but it is hard to be further formalized in general.

There has to be some supervision facilitating the search for a good $f$.

Supervised learning

When the supervision is in the form of some examples (observation $\rightarrow$ response) and the learned $f\subtext{predict}$ should process them correctly.

• example: "if I give you this observation $x$, you should predict this response $y$"

New version:

Supervised learning

When the supervision is in the form of some examples (observation $\rightarrow$ response) and the learned $f\subtext{predict}$ should process them correctly.

• example: "if I give you this observation $x$, you should predict this response $y$"

New version:

In unsupervised learning there is no supervision, there are no examples:

• nevertheless, there is some implicit expectation about how to process $x$
• we'll discuss unsupervised learning later

Examples

Formally, examples available for learning $f\subtext{predict}$ are pairs $(x,y)$.

A dataset compatible with $X$ and $Y$ is a bag of pairs $(x,y)$: $$D = \left\{\left(x^{(i)},y^{(i)}\right)\right\}_{i=1}^{i=n}$$ with $\forall i: x^{(i)} \in X, y^{(i)} \in Y$ and $|D|=n$.

Or, more briefly $D = \left\{\left(x^{(i)},y^{(i)}\right)\right\}_i$. examples are also denoted by $(x_i,y_i)$, depending on the community

A bag ($D$ should be called databag...):

• can have duplicates (bag $\ne$ set)
• it does not imply any order among its elements (bag $\ne$ sequence)

In most algorithms, and in their program counterparts, dataset are actually processed sequentially, though

Learning set

A learning set is a dataset that is used for learning an $f\subtext{predict}$.

• may be denoted by $D\subtext{learn}$ or $L$, or $T$, for training set

The learning set has to be consistent with the domain and codomain of the function $f\subtext{predict}$ to be learned:

• if $f\subtext{predict} \in \mathcal{F}_{X \to Y}$, then $D\subtext{learn} \in \mathcal{P}^*(X \times Y)$
  • $X \times Y$ is the Cartesian product of $X$ and $Y$, i.e., the set of all possible $(x,y)$ pairs
  • $\mathcal{P}(A)$ is the powerset of $A$, i.e., the set of all the possible subsets of $A$
  • $\mathcal{P}^*(A)$ is a custom notation for the powerset with duplicates, i.e., the set of all the possible multisets of $A$

Learning technique

In brief: given a $D\subtext{learn} \in \mathcal{P}^*(X \times Y)$, learn an $f\subtext{predict} \in \mathcal{F}_{X \to Y}$.

Learning technique

In brief: given a $D\subtext{learn} \in \mathcal{P}^*(X \times Y)$, learn an $f\subtext{predict} \in \mathcal{F}_{X \to Y}$.

A supervised learning technique is a way for learning an $f\subtext{predict} \in \mathcal{F}_{X \to Y}$ given a $D\subtext{learn} \in \mathcal{P}^*(X \times Y)$.

Learning technique

In brief: given a $D\subtext{learn} \in \mathcal{P}^*(X \times Y)$, learn an $f\subtext{predict} \in \mathcal{F}_{X \to Y}$.

A supervised learning technique is a way for learning an $f\subtext{predict} \in \mathcal{F}_{X \to Y}$ given a $D\subtext{learn} \in \mathcal{P}^*(X \times Y)$.

• learning phase: when $f\subtext{learn}$ is applied to obtain $f\subtext{predict}$ from $D$
• prediction phase: when $f\subtext{predict}$ is applied to obtain a $y$ from an $x$

Learning techniques

Why don't we suffice a single learning technique? Why are there many of them?

They differ in:

• applicability with respect to $X$ and/or $Y$
  • e.g., some require $X = \mathbb{R}^p$, some require $Y = \mathbb{R}$
• efficiency with respect to $|D\subtext{learn}|$
  • e.g., some are really fast for producing $f\subtext{predict}$ ($\mathcal{O}\left(|D\subtext{learn}|^{\approx 0}\right)$), some are slow ($\mathcal{O}\left(|D\subtext{learn}|^2\right)$)
• effectiveness in terms of the quality of the learned $f\subtext{predict}$
• attributes of learned $f\subtext{predict}$
  • nature/type of $f\subtext{predict}$ (a formula, a text, a tree...)
  • interpretability of $f\subtext{predict}$

Who?

getting computer: who is doing that?

• the user of a learning technique, who is likely the designer/developer of a ML system

Who?

getting computer: who is doing that?

• the user of a learning technique, who is likely the designer/developer of a ML system

is the science: what's science?

• there's not only the user: someone designs/develops learning techniques

New version:

Learning as optimization

A supervised learning technique $f\subtext{learn}: \mathcal{P}^*(X \times Y) \to \mathcal{F}_{X \to Y}$ can be seen as a form of optimization:

1. consider $\mathcal{F}_{X \to Y} = \{f, f: X \to Y\}$
2. find the one $f\subtext{predict} \in \mathcal{F}_{X \to Y}$ that works best on $D\subtext{learn}$

Could we use a general optimization technique?
In principle, yes, but:

• $X$ (and maybe $Y$) might be infinite (e.g., $X=\mathbb{R}^p$)
• $X \times Y$ is "more" infinite
• $\mathcal{F}_{X \to Y}$ is "hugely more" infinite

Learning as optimization

A supervised learning technique $f\subtext{learn}: \mathcal{P}^*(X \times Y) \to \mathcal{F}_{X \to Y}$ can be seen as a form of optimization:

1. consider $\mathcal{F}_{X \to Y} = \{f, f: X \to Y\}$
2. find the one $f\subtext{predict} \in \mathcal{F}_{X \to Y}$ that works best on $D\subtext{learn}$

Could we use a general optimization technique?
In principle, yes, but:

• $X$ (and maybe $Y$) might be infinite (e.g., $X=\mathbb{R}^p$)
• $X \times Y$ is "more" infinite
• $\mathcal{F}_{X \to Y}$ is "hugely more" infinite

Practical solution: reduce $\mathcal{F}_{X \to Y}$ size by considering only the $f$ of some nature:

• e.g., for $X=Y=\mathbb{R}$, consider $\mathcal{F}'_{\mathbb{R} \to \mathbb{R}} = \{f: f(x)=ax+b \text{ with } a,b \in \mathbb{R}\}$
• e.g., for $x$ a UTF-8 strings and $y$ a Boolean, consider the $f$ as regular expressions

Templating $f$

Often a learning technique works on a reduced $\mathcal{F}'_{X \to Y}$ which is based on an template $f'$:

• most parts of $f'$ are defined, some parts are undefined, variable
• $f'$ can be used for prediction only if the undefined parts are defined

E.g., for $X=Y=\mathbb{R}$, $f'(x)=ax+b$:

• you need concrete values for $a,b$ in order to apply $f$ to an $x$, i.e., to obtain a response $y$ out of an $x$
• this is univariate linear regression: we'll expand
  • univariate because $X$ has one dimension
  • regression because $Y=\mathbb{R}$
  • linear because of the template

Model

We can make explicit the undefined part of the template: $$f\subtext{predict}(x) = f'\subtext{predict}(x, m)$$ where $m \in M$ is the undefined part.

• e.g., $f'\subtext{predict}(x, a, b) = ax+b$ and $M=\mathbb{R}^2$

Note that $f'\subtext{predict}$ is fixed for a given learning technique and defines the reduced $\mathcal{F}'_{X \to Y} \subset \mathcal{F}_{X \to Y}$ where the learning will look for an $f\subtext{predict}$.

Model

We can make explicit the undefined part of the template: $$f\subtext{predict}(x) = f'\subtext{predict}(x, m)$$ where $m \in M$ is the undefined part.

• e.g., $f'\subtext{predict}(x, a, b) = ax+b$ and $M=\mathbb{R}^2$

Note that $f'\subtext{predict}$ is fixed for a given learning technique and defines the reduced $\mathcal{F}'_{X \to Y} \subset \mathcal{F}_{X \to Y}$ where the learning will look for an $f\subtext{predict}$.

Given a template $f'\subtext{predict}$, $m$ defines an $f\subtext{predict}$ that can be used to predict a $y$ from an $x$.
That is, $m$ is a model of how $y$ depends on $x$.

Learning a model

For techniques based on a template, $f\subtext{learn}$ actually looks just $\mathcal{F}'_{X \to Y}$, hence in $M$, for finding an $f\subtext{predict}$.

Examples of templated $f$

Choosing the learning technique means choosing one $\mathcal{F}'_{X \to Y}$!

Alternative views/terminology

The model $m$ is learned on a dataset $D$.

• $m$ learned from the examples in $D$

The model $m$ is trained on a dataset $D$.

• $m$ trained to correctly work on the examples in $D$

The model $m$ is fitted on a dataset $D$.

• $m$ adjusted until it works well on the examples in $D$

Alternative views/terminology

The model $m$ is learned on a dataset $D$.

• $m$ learned from the examples in $D$

The model $m$ is trained on a dataset $D$.

• $m$ trained to correctly work on the examples in $D$

The model $m$ is fitted on a dataset $D$.

• $m$ adjusted until it works well on the examples in $D$

Formally, a model is one specific $m \in M$ that has been found upon learning.
However, often "model" is used to denote a generic (e.g., still untrained/unfitted) artifact.

• "fit the model": a model exists before fitting (e.g., before the learning phase)
• "learn a model": the model is the outcome of the learning phase

Common cases and terminology

Supervised learning techniques may be categorized depending on the kind of $X,Y,M$ they deal with:

With respect to $Y$, most important cases:

• $Y$ is a finite set without intrinsic ordering $\rightarrow$ classification
  • $y$ is said a categorical (or nominal) variable
  • if $|Y|=2$ $\rightarrow$ binary classification
    otherwise $\rightarrow$ multiclass classification
• $Y = \mathbb{R}$ (or $Y \subseteq \mathbb{R}$) $\rightarrow$ regression
  • $y$ is said a numerical variable

With respect to $X$, common cases:

• $X = X_1 \times \dots \times X_p$, with each $X_i$ being $\mathbb{R}$ or a finite unordered set (each $x$ is a $p$-sized tuple)
  • $X$ is multivariate and each $x_i$ is either numerical or categorical
• $X$ is the set of all strings $\rightarrow$ text mining we'll see

Variables terminology

In the common case of a multivariate $X = X_1 \times \dots \times X_p$:

• each $x_i$ is said a independent variable
  • or feature, since it is a feature of an $x \in X$
  • or attribute, since it is an attribute of an $x \in X$
  • or predictor, since it hopefully helps predicting a $y$
• $y$ is said the dependent variable, since it is hoped to depend on $x$
  • or response variable

Variables terminology

In the common case of a multivariate $X = X_1 \times \dots \times X_p$:

• each $x_i$ is said a independent variable
  • or feature, since it is a feature of an $x \in X$
  • or attribute, since it is an attribute of an $x \in X$
  • or predictor, since it hopefully helps predicting a $y$
• $y$ is said the dependent variable, since it is hoped to depend on $x$
  • or response variable

Given a dataset $D$ with $|D|=n$ examples defined over $X,Y$:

Size of the "problem"

The common notation for the size of a multivariate dataset (i.e., a dataset with a multivariate $X=X_1 \times \dots \times X_p$) is:

• $n$ number of observations
• $p$ number of (independent) variables

On the assumption that a dataset $D$ implicitly defines the problem (since it bounds $X$ and $Y$ and hence $\mathcal{F}_{X \to Y}$), $n$ and $p$ also describe the size of the problem.

What (sup. learning techniques) we will see

A family of learning techniques (tree-based) for:

• multivariate $X = X_1 \times \dots \times X_p$, each $x$ being categorical or numerical
• classification (binary or multiclass) and regression

A family of learning techniques (SVM) for:

• $X = \mathbb{R}^p$
• binary classification

A learning technique (kNN) for:

• any $X$ with a similarity metric (including $X = \mathbb{R}^p$)
• classification (binary or multiclass) and regression

A learning technique (naive Bayes) for:

• multivariate $X = X_1 \times \dots \times X_p$, each $x$ being categorical with mention to the hybrid case
• classification (binary or multiclass)

... and...

What if none of the above learning techniques fits the problem ($X,Y$) at hand?

We'll see:

• a method for applying techniques suitable for $X=\mathbb{R}^p$ to problems where a multivariate $X$ includes categorical variables
• a few methods for applying techniques suitable for $X=\mathbb{R}^p$ to problems where $X=$ strings
• two methods for applying techniques suitable for binary classification ($|Y|=2$) to multiclass classification problems ($|Y|\ge 2$)

What about the other kinds of problems?

ML system

An information processing system in which there is:

• a supervised learning technique (i.e., a pair $f'\subtext{learn},f'\subtext{predict}$)
• other components operating on $X$ or $Y$
  • pre-processing, if "before" the learning technique, i.e., $X \to X'$
  • post-processing, if "after" the learning technique, i.e., $Y' \to Y$

ML system example: Twitter profiling

Goal: given a tweet, determine age range and gender of the author

• problem 1: $X=A^{280}, A=$UTF-16, $Y=\{ \text{0--16}, \text{17--29}, \text{30--49}, \text{50--}\}$
• problem 1: $X=A^{280}, A=$UTF-16, $Y=\{ \text{M}, \text{F}\}$ or broader

One possible ML system for this problem:

• $f\subtext{text-to-num}: A^{280} \to [0,1]^{50}$ (chosen among a few options, maybe adjusted)
• $f\subtext{foreach}: X^* \times \mathcal{F}_{X \to Y} \to Y^*$ (given an $f: X \to Y$ and a sequence $\{x_i\}_i$, apply $f$ to each $x_i$)
• $f'_{\text{learn},1},f'_{\text{predict},1}$ and $f'_{\text{learn},2},f'_{\text{predict},2}$ (two learning techniques suitable for classification)

Designing an ML system

• Who chooses the learning technique(s)?
  • And its parameter values?
• Who chooses/designs the pre- and post-processing components?
  • And their parameter values?

Designing an ML system

• Who chooses the learning technique(s)?
  • And its parameter values?
• Who chooses/designs the pre- and post-processing components?
  • And their parameter values?

The designer of the ML system, that is, you¹!

Phases of design of an ML design

Should I use Machine Learning?

Recall: we need an $f\subtext{predict}: X \to Y$ to make a decision $y$ about an $x$

Domain knowledge and data exploration

Reasons for running $f\subtext{learn}$ on a machine:

• humans cannot design a reasonable $f\subtext{predict}$: yes or no?
• human-made $f\subtext{predict}$ is too costly/slow: yes or no?
• human-made $f\subtext{predict}$ is not good: yes or no?

Answering these questions requires the knowledge of the domain

• (necessary, not sufficient)
• better/more with exploration of the data
  • which data?
  • how to explore it? $\rightarrow$ data visualization

How to choose components?

Component:

• learning technique
• pre- or post-processing technique
• dataset
• assessment technique

Factors: beyond applicability, which is a yes/no matter

• effectiveness
  • the component works well (experimental assessment, evaluation metrics and methods)
• efficiency
  • using the component consumes low resources
• interpretability
  • the working of the component and/or its outcomes is understandable
• familiarity
  • the designer does little effort for using the component: e.g., already knows the software tool, good parameter values, ...
• technological constraints

Example of Iris

Once upon a time, there were Alice, a ML expert, and Bob, an amateur botanist...

Example of Iris

Once upon a time, there were Alice, a ML expert, and Bob, an amateur botanist...

Why a story?

• we need a concrete case in order to practice the phases of the ML design (steps 1–3)
• those steps cannot be made with an abstract case

1. Decide: should I use ML?
2. Decide: supervised vs. unsupervised
3. Define the problem (problem statement)
4. Design the ML system
5. Implement the ML system
6. Assess the ML system

Iris species

Once upon a time, there were Alice, a ML expert, and Bob, an amateur botanist.

Bob liked to collect Iris flowers and to sort them properly in his collection boxes at home. He organized collected flowers depending on their species.

Bob's need

Alice: What's up, Bob?
Bob: I'd like to put new flowers into proper boxes.
Well... I'm not an expert of flowers. Can't you do it by yourself?
No, actually I cannot. But I heard you now master the art of machine learning...
Mmmmhhh... I see that you already have flowers in boxes. How did you sort them? Why ML now?
Well, I used to go to a professional botanist, who was able to tell me the species of each Iris flower I collected. I don't want to bother her anymore and her lab is far from here and it takes time to get there and the fuel is getting more and more pricey... 🦖
Ok, I understand. So you think ML can be helpful here. Let's see...

Bob's need

Alice: What's up, Bob?
Bob: I'd like to put new flowers into proper boxes.
Well... I'm not an expert of flowers. Can't you do it by yourself?
No, actually I cannot. But I heard you now master the art of machine learning...
Mmmmhhh... I see that you already have flowers in boxes. How did you sort them? Why ML now?
Well, I used to go to a professional botanist, who was able to tell me the species of each Iris flower I collected. I don't want to bother her anymore and her lab is far from here and it takes time to get there and the fuel is getting more and more pricey... 🦖
Ok, I understand. So you think ML can be helpful here. Let's see...

Some information about the context up to here (Alice's thoughts 💭):

• problem timings: no real hurry to make a decision
• scale of the problem: how many flowers would Bob collect in the unit of time?
• cost of the solution: Bob is basically trying to replace a free service with another free service...
• expected quality of the solution: how picky will be Bob?

No car accidents to be avoided (timing), no billions of emails to be analyzed (scale), no big business process invoved (cost), no loan decision to be made (quality).

Tackling the Iris problem: phase 1 - ML?

Tackling the Iris problem: phase 1 - ML?

Outcome: ok, let's use ML!

Phase 2 - supervised vs. unsupervised

Phase 2 - supervised vs. unsupervised

Outcome: it's supervised learning!

Phase 3 - problem statement

Phases 3 - shaping $X$

Phases 3 - feature engineering