Skip to content

Commit 3dac521

Browse files
smcclatchysmcclatchy
committed
format challenges
1 parent 142783c commit 3dac521

1 file changed

Lines changed: 156 additions & 106 deletions

File tree

‎episodes/conditional-probabilities-expectations.Rmd‎

Lines changed: 156 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -5,32 +5,44 @@ exercises: 30
55
---
66

77
:::::: questions
8-
- "How to get statistical information from specific subsets in our data?"
8+
- How do we get statistical information from specific subsets in our data?
99
::::::
1010

1111
:::::: objectives
12-
- "Calculate statistics from subsets of a dataset that satisfies especific conditions"
13-
- "Predict the expected outcome of an experiment given certain conditions on a dataset"
12+
- Calculate statistics from subsets of a dataset that satisfies specific conditions.
13+
- Predict the expected outcome of an experiment given certain conditions on a dataset.
1414
::::::
1515

1616
```{r, include=FALSE}
1717
```
1818

1919
## Conditional Probabilities and Expectations
2020

21-
Prediction problems can be divided into categorical and continuous outcomes. However, many of the algorithms can be applied to both due to the connection between _conditional probabilities_ and _conditional expectations_.
21+
Prediction problems can be divided into categorical and continuous outcomes.
22+
However, many of the algorithms can be applied to both due to the connection
23+
between _conditional probabilities_ and _conditional expectations_.
2224

23-
For categorical data, for example binary outcomes, if we know the probability of $Y$ being any of the possible outcomes $k$ given a set of predictors $X=(X_1,\dots,X_p)^\top$,
25+
For categorical data, for example binary outcomes, if we know the probability of
26+
$Y$ being any of the possible outcomes $k$ given a set of predictors
27+
$X=(X_1,\dots,X_p)^\top$,
2428

2529
$$
2630
f_k(x) = \mbox{Pr}(Y=k \mid X=x)
2731
$$
2832

29-
we can optimize our predictions. Specifically, for any $x$ we predict the $k$ that has the largest probability $f_k(x)$.
33+
we can optimize our predictions. Specifically, for any $x$ we predict the $k$
34+
that has the largest probability $f_k(x)$.
3035

31-
To simplify the exposition below, we will consider the case of binary data. You can think of the probability $\mbox{Pr}(Y=1 \mid X=x)$ as the proportion of 1s in the stratum of the population for which $X=x$. Given that the expectation is the average of all $Y$ values, in this case the expectation is equivalent to the probability: $f(x) \equiv \mbox{E}(Y \mid X=x)=\mbox{Pr}(Y=1 \mid X=x)$. We therefore use only the expectation in the descriptions below as it is more general.
36+
To simplify the exposition below, we will consider the case of binary data. You
37+
can think of the probability $\mbox{Pr}(Y=1 \mid X=x)$ as the proportion of 1s
38+
in the stratum of the population for which $X=x$. Given that the expectation is
39+
the average of all $Y$ values, in this case the expectation is equivalent to the
40+
probability: $f(x) \equiv \mbox{E}(Y \mid X=x)=\mbox{Pr}(Y=1 \mid X=x)$. We
41+
therefore use only the expectation in the descriptions below as it is more
42+
general.
3243

33-
In general, the expected value has an attractive mathematical property, which is that it minimizes the expected distance between the predictor $\hat{Y}$ and $Y$:
44+
In general, the expected value has an attractive mathematical property, which is
45+
that it minimizes the expected distance between the predictor $\hat{Y}$ and $Y$:
3446

3547
$$
3648
\mbox{E}\{ (\hat{Y} - Y)^2 \mid X=x \}
@@ -41,133 +53,171 @@ $$
4153

4254
<a name="regression"></a>
4355

44-
We use the son and father height example to illustrate how regression can be interpreted as a machine learning technique. In our example, we are trying to predict the son's height $Y$ based on the father's $X$. Here we have only one predictor. Now if we were asked to predict the height of a randomly selected son, we would go with the average height:
45-
56+
We use the son and father height example to illustrate how regression can be
57+
interpreted as a machine learning technique. In our example, we are trying to
58+
predict the son's height $Y$ based on the father's $X$. Here we have only one
59+
predictor. Now if we were asked to predict the height of a randomly selected
60+
son, we would go with the average height:
4661

4762
```{r height_hist,message=FALSE,fig.cap="Histogram of son heights."}
4863
library(rafalib)
49-
mypar(1,1)
50-
data(father.son,package="UsingR")
51-
x=round(father.son$fheight) ##round to nearest inch
64+
mypar(1, 1)
65+
data(father.son, package="UsingR")
66+
x=round(father.son$fheight) ## round to nearest inch
5267
y=round(father.son$sheight)
53-
hist(y,breaks=seq(min(y),max(y)))
54-
abline(v=mean(y),col="red",lwd=2)
68+
hist(y, breaks=seq(min(y), max(y)))
69+
abline(v=mean(y), col="red", lwd=2)
5570
```
5671

57-
![Histogram of son heights.](../fig/03-conditional-probabilities-expectations-height_hist-1.png)
58-
59-
In this case, we can also approximate the distribution of $Y$ as normal, which implies the mean maximizes the probability density.
72+
![Histogram of son heights.](./fig/03-conditional-probabilities-expectations-height_hist-1.png)
6073

61-
Let's imagine that we are given more information. We are told that the father of this randomly selected son has a height of 71 inches (1.25 SDs taller than the average). What is our prediction now?
74+
In this case, we can also approximate the distribution of $Y$ as normal, which
75+
implies the mean maximizes the probability density.
6276

77+
Let's imagine that we are given more information. We are told that the father of
78+
this randomly selected son has a height of 71 inches (1.25 SDs taller than the
79+
average). What is our prediction now?
6380

6481
```{r conditional_distribution, fig.cap="Son versus father height (left) with the red lines denoting the stratum defined by conditioning on fathers being 71 inches tall. Conditional distribution: son height distribution of stratum defined by 71 inch fathers.",fig.width=10.5,fig.height=5.25}
65-
mypar(1,2)
66-
plot(x,y,xlab="Father's height in inches",ylab="Son's height in inches",
67-
main=paste("correlation =",signif(cor(x,y),2)))
68-
abline(v=c(-0.35,0.35)+71,col="red")
69-
hist(y[x==71],xlab="Heights",nc=8,main="",xlim=range(y))
82+
mypar(1, 2)
83+
plot(x, y,
84+
xlab="Father's height in inches",
85+
ylab="Son's height in inches",
86+
main=paste("correlation =",
87+
signif(cor(x, y), 2)))
88+
abline(v=c(-0.35, 0.35) + 71,
89+
col="red")
90+
hist(y[x==71],
91+
xlab="Heights",
92+
nc=8,
93+
main="",
94+
xlim=range(y))
7095
```
7196

72-
![Son versus father height (left) with the red lines denoting the stratum defined by conditioning on fathers being 71 inches tall. Conditional distribution: son height distribution of stratum defined by 71 inch fathers.](../fig/03-conditional-probabilities-expectations-conditional_distribution-1.png)
97+
![Son versus father height (left) with the red lines denoting the stratum defined by conditioning on fathers being 71 inches tall. Conditional distribution: son height distribution of stratum defined by 71 inch fathers.](./fig/03-conditional-probabilities-expectations-conditional_distribution-1.png)
7398

74-
The best guess is still the expectation, but our strata has changed from all the data, to only the $Y$ with $X=71$. So we can stratify and take the average, which is the conditional expectation. Our prediction for any $x$ is therefore:
99+
The best guess is still the expectation, but our strata has changed from all the
100+
data, to only the $Y$ with $X=71$. So we can stratify and take the average,
101+
which is the conditional expectation. Our prediction for any $x$ is therefore:
75102

76103
$$
77104
f(x) = E(Y \mid X=x)
78105
$$
79106

80-
It turns out that because this data is approximated by a bivariate normal distribution, using calculus, we can show that:
107+
It turns out that because this data is approximated by a bivariate normal
108+
distribution, using calculus, we can show that:
81109

82110
$$
83111
f(x) = \mu_Y + \rho \frac{\sigma_Y}{\sigma_X} (X-\mu_X)
84112
$$
85113

86-
and if we estimate these five parameters from the sample, we get the regression line:
114+
and if we estimate these five parameters from the sample, we get the regression
115+
line:
87116

88117
```{r regression, fig.cap="Son versus father height showing predicted heights based on regression line (left). Conditional distribution with vertical line representing regression prediction.",fig.width=10.5,fig.height=5.25}
89-
mypar(1,2)
90-
plot(x,y,xlab="Father's height in inches",ylab="Son's height in inches",
91-
main=paste("correlation =",signif(cor(x,y),2)))
92-
abline(v=c(-0.35,0.35)+71,col="red")
118+
mypar(1, 2)
119+
plot(x, y,
120+
xlab="Father's height in inches",
121+
ylab="Son's height in inches",
122+
main=paste("correlation =",
123+
signif(cor(x,y),2)))
124+
abline(v=c(-0.35, 0.35) + 71,
125+
col="red")
126+
127+
fit <- lm(y ~ x)
128+
abline(fit, col=1)
129+
130+
hist(y[x==71],
131+
xlab="Heights",
132+
nc=8,
133+
main="",
134+
xlim=range(y))
135+
abline(v = fit$coef[1] + fit$coef[2]*71, col=1)
136+
```
93137

94-
fit <- lm(y~x)
95-
abline(fit,col=1)
138+
![Son versus father height showing predicted heights based on regression line (left). Conditional distribution with vertical line representing regression prediction.](./fig/03-conditional-probabilities-expectations-regression-1.png)
96139

97-
hist(y[x==71],xlab="Heights",nc=8,main="",xlim=range(y))
98-
abline(v = fit$coef[1] + fit$coef[2]*71, col=1)
140+
In this particular case, the regression line provides an optimal prediction
141+
function for $Y$. But this is not generally true because, in the typical machine
142+
learning problems, the optimal $f(x)$ is rarely a simple line.
143+
144+
::::::::::::::::::::::::::::::::::::: challenge
145+
146+
## Exercise 1
147+
Throughout these exercises it will be useful to remember that when our data
148+
are 0s and 1s, probabilities and expectations are the same thing. We can do
149+
the math, but here is some R code:
150+
151+
```{r, eval=FALSE}
152+
n = 1000
153+
y = rbinom(n, 1, 0.25) ## proportion of ones Pr(Y)
154+
sum(y==1)/length(y) ## expectation of Y mean(y)
99155
```
100156

101-
![Son versus father height showing predicted heights based on regression line (left). Conditional distribution with vertical line representing regression prediction.](../fig/03-conditional-probabilities-expectations-regression-1.png)
102-
103-
In this particular case, the regression line provides an optimal prediction function for $Y$. But this is not generally true because, in the typical machine learning problems, the optimal $f(x)$ is rarely a simple line.
104-
105-
> ## Exercises
106-
> Throughout these exercises it will be useful to remember that when our data
107-
> are 0s and 1s, probabilities and expectations are the same thing. We can do
108-
> the math, but here is some R code:
109-
> ```{r, eval=FALSE}
110-
> n = 1000
111-
> y = rbinom(n, 1, 0.25) ## proportion of ones Pr(Y)
112-
> sum(y==1)/length(y) ## expectation of Y mean(y)
113-
> ```
114-
> 1. Generate some random data to imitate heights for men (0) and women (1):
115-
> ```{r, eval=FALSE}
116-
> n = 10000
117-
> set.seed(1)
118-
> # Generate a sample of heights for a mixed population of men and women
119-
> men = rnorm(n, 176, 7) # height in centimeters
120-
> women = rnorm(n, 162, 7) # height in centimeters
121-
> # Assign a class label to each height generated above (0: men, 1:women)
122-
> y = c(rep(0, n), rep(1, n))
123-
> x = round(c(men, women))
124-
> ## mix it up
125-
> ind = sample(seq(along=y))
126-
> y = y[ind]
127-
> x = x[ind]
128-
> ```
129-
> 2. Using the data generated above, what is the E(Y | X = 176)?
130-
>
131-
> > ## Solution
132-
> >
133-
> > ```{r, eval=FALSE}
134-
> > # Take all labels `y` that correspond to individuals whose height is 176 cm.
135-
> > # Then compute the mean value as an insight about an individual with
136-
> > # that height could be a woman or a man.
137-
> > mean(y[x==176])
138-
> > ```
139-
> {: .solution}
140-
{: .challenge}
141-
> 3. Now make a plot of E(Y|X=x) for `x=seq(160, 178)` using the data generated
142-
> in exercise 1.
143-
> If you are predicting female or male based on height and want your probability
144-
> of success to be larger than 0.5, what is the largest height where you predict
145-
> female ?
146-
>
147-
> > ## Solution
148-
> >
149-
> > ```{r, eval=FALSE}
150-
> > mypar()
151-
> > plot(x,y)
152-
> > # This time we test a single height at a time from a set of heights,
153-
> > # between 160 cm and 178 cm.
154-
> > x_list <- seq(160,178)
155-
> > res <- vector('double', length(x_list))
156-
> > for (i in seq_along(x_list)) {
157-
> > # Compute the expected label (woman or man) of an individual whose height
158-
> > # is the same as x_list[i]
159-
> > res[i] <- mean(y[x==x_list[i]])
160-
> > }
161-
> > # Verify that the probability to identify an individual as a woman given their
162-
> > # height is higher than 50 %
163-
> > mean(y[x==x_list[ind]])
164-
> >
165-
> > # And if we move one centimeter higher, our probability falls under 50 %
166-
> > mean(y[x==x_list[ind + 1]])
167-
> > ```
168-
> {: .solution}
169-
{: .challenge}
157+
Generate some random data to imitate heights for men (0) and women (1):
158+
159+
```{r, eval=FALSE}
160+
n = 10000
161+
set.seed(1)
162+
# Generate a sample of heights for a mixed population of men and women
163+
men = rnorm(n, 176, 7) # height in centimeters
164+
women = rnorm(n, 162, 7) # height in centimeters
165+
# Assign a class label to each height generated above (0: men, 1:women)
166+
y = c(rep(0, n), rep(1, n))
167+
x = round(c(men, women))
168+
## mix it up
169+
ind = sample(seq(along=y))
170+
y = y[ind]
171+
x = x[ind]
172+
```
173+
174+
## Exercise 2
175+
Using the data generated above, what is the E(Y | X = 176)?
176+
177+
:::::::::::::::: solution
178+
179+
```{r, eval=FALSE}
180+
# Take all labels `y` that correspond to individuals whose height is 176 cm.
181+
# Then compute the mean value as an insight about an individual with
182+
# that height could be a woman or a man.
183+
mean(y[x==176])
184+
```
185+
186+
:::::::::::::::::::::::::
187+
188+
## Exercise 3
189+
Now make a plot of E(Y|X=x) for `x=seq(160, 178)` using the data generated
190+
in exercise 1.
191+
If you are predicting female or male based on height and want your probability
192+
of success to be larger than 0.5, what is the largest height where you predict
193+
female ?
194+
195+
:::::::::::::::: solution
196+
197+
```{r, eval=FALSE}
198+
mypar()
199+
plot(x,y)
200+
# This time we test a single height at a time from a set of heights,
201+
# between 160 cm and 178 cm.
202+
x_list <- seq(160,178)
203+
res <- vector('double', length(x_list))
204+
for (i in seq_along(x_list)) {
205+
# Compute the expected label (woman or man) of an individual whose height
206+
# is the same as x_list[i]
207+
res[i] <- mean(y[x==x_list[i]])
208+
}
209+
# Verify that the probability to identify an individual as a woman given their
210+
# height is higher than 50 %
211+
mean(y[x==x_list[ind]])
212+
213+
# And if we move one centimeter higher, our probability falls under 50 %
214+
mean(y[x==x_list[ind + 1]])
215+
```
216+
217+
:::::::::::::::::::::::::
218+
219+
:::::::::::::::::::::::::::::::::::::::::::::::
170220

171221
:::::: keypoints
172-
- "For categorical/discrete variables we have used estrict conditions (i.e. X=x); however, conditioning can be applied to continuous vriables by using ranges instead (e.g. X>=x, X<=x, or a<X<b)"
222+
- For categorical/discrete variables we have used strict conditions (i.e. X=x); however, conditioning can be applied to continuous variables by using ranges instead (e.g. X=x, X<=x, or a<X<b)
173223
::::::

0 commit comments

Comments
 (0)