Skip to content

Commit 77342dd

Browse files
MichaelHabermasMichaelHabermas
committed
DS Spr 15 Mod 2 Updates
1 parent 0bc7b92 commit 77342dd

1 file changed

Lines changed: 365 additions & 0 deletions

File tree

  • ds-curriculum/ds-unit-4-sprint-15/modules/module2

‎ds-curriculum/ds-unit-4-sprint-15/modules/module2/index.html‎

Lines changed: 365 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,371 @@ <h2>Learning Objectives</h2>
5757
</div>
5858
</section>
5959

60+
<section class="content-box">
61+
<h2>Objective 01 - Describe Convolution and Pooling</h2>
62+
63+
<h3>Overview</h3>
64+
65+
<p>In the previous module, we used a type of RNN and LSTM network, to predict text. Because of the structure
66+
of RNNs, they are very good at learning how to predict sequences, such as text.</p>
67+
<p>Now, we're going to move onto a type of neural network used to learn about images. Convolutional neural
68+
networks or CNNs classify images, cluster images by similarity, and object recognition in images.</p>
69+
<p>So, what is a convolutional neural network? It is a type of network that makes use of a mathematical
70+
operation called convolution. But, first, let's take a quick look at how the brain uses convolution to
71+
process images.</p>
72+
73+
<h3>The Brain and Image Recognition</h3>
74+
<p>You probably already know that the animal brain is good at image recognition. Animals are good at image
75+
recognition because of the organization of the visual cortex, or the part of the brain that processes
76+
images.</p>
77+
<p>Individual neurons in the brain respond to stimuli from only part of the visual field. Neurons in some
78+
areas of the visual cortex respond to features in images at specific orientations. For example, some
79+
neurons might respond to bars or lines in a specific orientation; these neurons would be good at edge
80+
detection. Neurons in other areas process information about color and motion.</p>
81+
82+
<h3>Convolutional Neural Network</h3>
83+
<p>A CNN uses a similar structure: a convolutional layer in the network uses convolution to filter or
84+
convolve the input image and create an output. However, different filters have different results: some
85+
are good at detecting lines and edges, others are good at identifying other features in the image.</p>
86+
<p>The output from all of this filtering or convolution is then fed into other hidden layers and finally the
87+
output to make a prediction.</p>
88+
<p>Right now, we're going to go over an example of the convolution of an image; in the next part of the
89+
module, we'll implement a CNN for image classification.</p>
90+
91+
<h3>Follow Along</h3>
92+
<pre><code>import requests
93+
94+
img_url = "https://raw.githubusercontent.com/bloominstituteoftechnology/data-science-canvas-images/main/unit_4/palm_trees.jpg"
95+
96+
r = requests.get(img_url)
97+
with open('palm_trees.jpg', 'wb') as f:
98+
f.write(r.content)
99+
100+
# Display the first image
101+
from IPython.display import Image
102+
Image(filename='palm_trees.jpg', width=500)</code></pre>
103+
<p><img src="https://raw.githubusercontent.com/bloominstituteoftechnology/data-science-canvas-images/main/unit_4/palm_trees.jpg"
104+
alt="palm_trees" loading="lazy"></p>
105+
<p>We have a pretty standard image of palm trees. Let's perform a convolution on this image. First, we'll do
106+
an edge detection which we implement with a high-pass filter. The edge detection means that we let
107+
through features that change abruptly in value; in this case, the edges of the leaves and trunk.</p>
108+
<p>The kernel is the filter and is usually an array smaller in size than the image. In our example code
109+
below, the kernel is a 3x3 matrix with a larger positive value in the center surrounded by negative
110+
values. When this kernel is passed over or convolved with our input image, we should have a resulting
111+
image that's just the edges.</p>
112+
<h3>Example credit: https://pythonexamples.org/python-opencv-image-filter-convolution-cv2-filter2d/</h3>
113+
<pre><code># Imports
114+
import numpy as np
115+
import cv2
116+
117+
# Read in the image
118+
img_src = cv2.imread('palm_trees.jpg')
119+
120+
# Edge detection (high-pass filter)
121+
kernel = np.array([[0.0, -1.0, 0.0],
122+
[-1.0, 4.0, -1.0],
123+
[0.0, -1.0, 0.0]])
124+
125+
kernel = kernel / (np.sum(kernel) if np.sum(kernel) != 0 else 1)
126+
127+
# Filter the source image
128+
img_rst = cv2.filter2D(img_src,-1,kernel)
129+
130+
#save result image
131+
cv2.imwrite('palm_trees_edge.jpg', img_rst)
132+
133+
Image(filename='palm_trees_edge.jpg', width=500)
134+
</code></pre>
135+
<p><img src="https://github.com/bloominstituteoftechnology/data-science-canvas-images/blob/main/unit_4/sprint_3/mod2_obj1_palm_trees_edge.png?raw=true"
136+
alt="palm&lt;em&gt;trees&lt;/em&gt;edges" loading="lazy"></p>
137+
<h3>Challenge</h3>
138+
<p>Try out the convolution process on an image of your own! Is the edge detection filter actually finding
139+
what you think are the edges in your image? For a stretch goal, try implementing a low-pass filter. The
140+
filter kernel would look like this for a 5x5 kernel:</p>
141+
<pre><code>kernel = np.array([[1, 1, 1, 1, 1],
142+
[1, 1, 1, 1, 1],
143+
[1, 1, 1, 1, 1],
144+
[1, 1, 1, 1, 1],
145+
[1, 1, 1, 1, 1]])
146+
kernel = kernel / sum(kernel)
147+
</code></pre>
148+
<h3>Additional Resources</h3>
149+
<ul>
150+
<li><a href="https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53"
151+
target="_blank" rel="noopener noreferrer">A Comprehensive Guide to Convolutional Neural
152+
Networks</a></li>
153+
<li><a href="https://wiki.pathmind.com/convolutional-network" target="_blank"
154+
rel="noopener noreferrer">A Beginner's Guide to Convolutional Neural Networks</a></li>
155+
</ul>
156+
</section>
157+
158+
<section class="content-box">
159+
<h2>Objective 02 - Apply a Convolutional Neural Network to a Classification Task</h2>
160+
<h3>Overview</h3>
161+
<p>
162+
A convolutional neural network includes a convolutional layer that maps regions of the input image to
163+
the responsible neurons. They also have a "pooling" layer, which we discussed earlier in the module.
164+
</p>
165+
<p>
166+
After the convolution and a few other layers as needed, we have the output layer and our complete model
167+
architecture.
168+
</p>
169+
<p>
170+
Let's implement a CNN and see what sort of results we can get!
171+
</p>
172+
<h3>Follow Along</h3>
173+
<p>
174+
The example below will use a CNN to classify digits in the MNIST dataset. Some of the following code has
175+
been adapted from <a href="#" target="_blank" rel="noopener noreferrer">this website</a>. Remember that
176+
images are
177+
represented as a matrix of pixels, where the pixel's value is the "intensity" of the color in that
178+
location. Therefore, color images would need to be represented by three separate pixel matrices, one for
179+
each RGB color.
180+
</p>
181+
<p>
182+
The MNIST images are just represented by a single matrix in grayscale. The dataset is available through
183+
the Keras datasets.
184+
</p>
185+
<pre><code># Some example code from: Machine Learning Mastery
186+
# Imports
187+
188+
from keras.datasets import mnist
189+
import matplotlib.pyplot as plt
190+
191+
# Load the MNIST dataset
192+
(X_train, y_train), (X_test, y_test) = mnist.load_data()
193+
194+
# Look at the training/testing sizes
195+
print('Train: X=%s, y=%s' % (X_train.shape, y_train.shape))
196+
print('Test: X=%s, y=%s' % (X_test.shape, y_test.shape))
197+
198+
# Plot the first nine images
199+
for i in range(9):
200+
plt.subplot(330 + 1 + i)
201+
plt.imshow(X_train[i], cmap=plt.get_cmap('gray'))
202+
203+
plt.show()</code></pre>
204+
<pre><code>Train: X=(60000, 28, 28), y=(60000,)
205+
Test: X=(10000, 28, 28), y=(10000,)</code></pre>
206+
<p><img src="https://raw.githubusercontent.com/bloominstituteoftechnology/data-science-canvas-images/main/unit_4/sprint_3/mod2_obj2_digits_in.png"
207+
alt="mod2&lt;em&gt;obj2&lt;/em&gt;digits_in.png" loading="lazy"></p>
208+
<p>
209+
We have 60,000 images to use for training and another 10,000 for testing. For input into the neural
210+
network, we need to reshape the data to have 60000 28x28x1 matrices.
211+
</p>
212+
<p>
213+
We also need to encode the target array to represent the digits between 0 and 9.
214+
</p>
215+
<pre><code># Reshape the training images
216+
trainX = X_train.reshape((X_train.shape[0], 28, 28, 1))
217+
testX = X_test.reshape((X_test.shape[0], 28, 28, 1))
218+
print(X_train.shape)
219+
220+
# Encode target values
221+
from keras.utils import to_categorical
222+
223+
y_train = to_categorical(y_train)
224+
y_test = to_categorical(y_test)</code></pre>
225+
<pre><code>(60000, 28, 28, 1)</code></pre>
226+
<p>
227+
The data also needs to be scaled so that each pixel has a value between 0 and 1; currently they can have
228+
a value between 0 and 255 so we'll just divide by 255.
229+
</p>
230+
<pre><code># scale pixels
231+
def prep_pixels(train, test):
232+
# convert from integers to floats
233+
train_norm = train.astype('float32')
234+
test_norm = test.astype('float32')
235+
# normalize to range 0-1
236+
train_norm = train_norm / 255.0
237+
test_norm = test_norm / 255.0
238+
# return normalized images
239+
return train_norm, test_norm
240+
241+
# Convert the images
242+
X_train_norm, X_test_norm = prep_pixels(X_train, X_test)</code></pre>
243+
<p>
244+
Now that we have the images ready, we'll set up the model. It will include convolutional layers, a
245+
pooling layer, and a few other layers, including the output corresponding to the ten digits we're trying
246+
to classify.
247+
</p>
248+
<pre><code># Import keras models, layers
249+
from keras.models import Sequential
250+
from keras.layers import Conv2D, MaxPooling2D, Dense, Flatten
251+
from keras.optimizers import SGD
252+
253+
# Set-up the model
254+
model = Sequential()
255+
256+
# Convolutional layer with a 3x3 kernel
257+
model.add(Conv2D(32, (3, 3), activation='relu',
258+
kernel_initializer='he_uniform',
259+
input_shape=(28, 28, 1)))
260+
261+
# Pooling layer (takes the max value)
262+
model.add(MaxPooling2D((2, 2)))
263+
model.add(Flatten())
264+
265+
# Dense hidden layer
266+
model.add(Dense(100, activation='relu', kernel_initializer='he_uniform'))
267+
268+
# Output layer
269+
model.add(Dense(10, activation='softmax'))
270+
271+
# Compile model
272+
opt = SGD(lr=0.01, momentum=0.9)
273+
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])</code></pre>
274+
<p>
275+
And now we can train our model. As usual, this can take a while.
276+
</p>
277+
<pre><code>model.fit(X_train_norm, y_train, epochs=5,
278+
batch_size=32, validation_data=(X_test_norm, y_test),
279+
verbose=0)</code></pre>
280+
<pre><code>&lt;tensorflow.python.keras.callbacks.History at 0x7fa5527d4860&gt;</code></pre>
281+
<p>
282+
Finally, we'll evaluate on the test set; the second number displayed is the accuracy.
283+
</p>
284+
<pre><code>model.evaluate(X_test, y_test, verbose=0)</code></pre>
285+
<pre><code>[26.49790382385254, 0.9621000289916992]</code></pre>
286+
<h3>Challenge</h3>
287+
<p>
288+
For this challenge, you can try changing the parameters of the convolutional layer, such as the kernel
289+
size.
290+
</p>
291+
<h3>Additional Resources</h3>
292+
<ul>
293+
<li><a href="https://machinelearningmastery.com/how-to-develop-a-convolutional-neural-network-from-scratch-for-mnist-handwritten-digit-classification/"
294+
target="_blank" rel="noopener noreferrer">How to Develop a Convolutional Neural Network</a></li>
295+
<li><a href="https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53"
296+
target="_blank" rel="noopener noreferrer">A Comprehensive Guide to Convolutional Neural
297+
Networks</a></li>
298+
</ul>
299+
</section>
300+
301+
<section class="content-box">
302+
<h2>Objective 03 - Use a Pre-Trained Convolution Neural Network for Image Classification</h2>
303+
<h3>Overview</h3>
304+
<p>
305+
In this last part of the module, we will take advantage of the transfer learning process, where we store
306+
what we learned from one type of problem and apply it to a different problem. Because neural networks
307+
for image classification take a long time to train, we can use pre-trained models. For image
308+
classification, the model has likely been trained on a very large number of images so that you can use
309+
it for general image classification tasks.
310+
</p>
311+
<p>
312+
The one we will demonstrate here and in the Guided Project is the <a
313+
href="https://arxiv.org/abs/1512.03385" target="_blank" rel="noopener noreferrer">ResNet50</a>
314+
pre-trained classifier available as a Keras application.
315+
</p>
316+
<h3>Follow Along</h3>
317+
<pre><code># Imports
318+
import numpy as np
319+
import requests
320+
321+
from keras.applications.resnet50 import ResNet50
322+
from keras.preprocessing import image
323+
from keras.applications.resnet50 import preprocess_input, decode_predictions
324+
325+
# Image processing
326+
327+
# Set location for images to test
328+
329+
image_urls = [
330+
"https://raw.githubusercontent.com/bloominstituteoftechnology/data-science-canvas-images/main/unit_4/two_llamas.jpg",
331+
"https://raw.githubusercontent.com/bloominstituteoftechnology/data-science-canvas-images/main/unit_4/cat_llama.jpg",
332+
"https://raw.githubusercontent.com/bloominstituteoftechnology/data-science-canvas-images/main/unit_4/palm_trees.jpg"
333+
]
334+
335+
# Write images to local file space
336+
for _id, img in enumerate(image_urls):
337+
r = requests.get(img)
338+
with open(f'example{_id}.jpg', 'wb') as f:
339+
f.write(r.content)
340+
341+
# Function to load images from a path
342+
def process_img_path(img_path):
343+
return image.load_img(img_path, target_size=(224, 224))
344+
345+
346+
# Classify image
347+
348+
def classify_image(img):
349+
350+
# Convert the image to an array
351+
x = image.img_to_array(img)
352+
x = np.expand_dims(x, axis=0)
353+
x = preprocess_input(x)
354+
355+
# Instantiate the model
356+
model = ResNet50(weights='imagenet')
357+
358+
# Predict which features the image has
359+
features = model.predict(x)
360+
361+
# Decode the prediction and display the top three features
362+
results = decode_predictions(features, top=3)[0]
363+
return results
364+
</code></pre>
365+
<p>
366+
Now we have a function to process the input image so that it's in the correct format and then output the
367+
top three "features" or results. Let's try it out on three different images: llamas, a cat and llama,
368+
and palm trees (for something very unlike a llama).
369+
</p>
370+
<pre><code># Display the first image
371+
from IPython.display import Image
372+
373+
Image(filename='./example0.jpg', width=300)</code></pre>
374+
<p><img src="https://raw.githubusercontent.com/bloominstituteoftechnology/data-science-canvas-images/main/unit_4/two_llamas.jpg"
375+
alt="two_llamas" loading="lazy"></p>
376+
<pre><code># Return the classification from the model
377+
classify_image(process_img_path('example0.jpg'))</code></pre>
378+
<pre><code>[('n02437616', 'llama', 0.9999949), ('n02437312', 'Arabian_camel', 4.2163906e-06), ('n02412080', 'ram', 7.4659454e-07)]</code></pre>
379+
<p>
380+
The model correctly identified "llama" with a high degree of certainty. Let's try a different image that
381+
includes a toy llama in addition to a cat.
382+
</p>
383+
<pre><code># Test out the second image
384+
Image(filename='./example1.jpg', width=300)</code></pre>
385+
<p><img src="https://raw.githubusercontent.com/bloominstituteoftechnology/data-science-canvas-images/main/unit_4/cat_llama.jpg"
386+
alt="cat_llama" loading="lazy"></p>
387+
388+
<pre><code>classify_image(process_img_path('example1.jpg'))</code></pre>
389+
<pre><code>[('n02124075', 'Egyptian_cat', 0.63906634), ('n02123045', 'tabby', 0.13549377), ('n02123159', 'tiger_cat', 0.07188996)]
390+
</code></pre>
391+
<p>
392+
This one also correctly identifies a cat, though the correct choice "tabby" is second. But still pretty
393+
good!
394+
</p>
395+
<p>
396+
And our last image is of trees - let's see how this one does.
397+
</p>
398+
<pre><code>#Image(filename='./example2.jpg', width=300)</code></pre>
399+
<p><img src="https://raw.githubusercontent.com/bloominstituteoftechnology/data-science-canvas-images/main/unit_4/palm_trees.jpg"
400+
alt="palm_trees" loading="lazy"></p>
401+
<pre><code>classify_image(process_img_path('example2.jpg'))</code></pre>
402+
<pre><code>[('n09428293', 'seashore', 0.55565596), ('n03837869', 'obelisk', 0.099918865), ('n12768682', 'buckeye', 0.052723538)]
403+
</code></pre>
404+
<p>
405+
This classification isn't as good as the other two: we have seashore as the first result (though these
406+
trees are next to the ocean), followed by "obelisk" and then "buckeye," which is a type of tree that is
407+
not next to the ocean. But, the leaves in this picture do resemble the buckeye tree leaves.
408+
</p>
409+
<h3>Challenge</h3>
410+
<p>
411+
Now is a great time to have some fun! Think of a type of image that you would like to classify and load
412+
them following the above example. Change the name of the function to what you are trying to classify.
413+
</p>
414+
<h3>Additional Resources</h3>
415+
<ul>
416+
<li><a href="https://www.tensorflow.org/hub/tutorials" target="_blank"
417+
rel="noopener noreferrer">TensorFlow
418+
Hub: Tutorials</a></li>
419+
<li><a href="https://www.tensorflow.org/api_docs/python/tf/keras/applications/ResNet50V2"
420+
target="_blank" rel="noopener noreferrer">Keras
421+
Applications: ResNet50 V2</a></li>
422+
</ul>
423+
</section>
424+
60425
<section id="guided-project">
61426
<div class="content-box">
62427
<h2>Guided Project</h2>

0 commit comments

Comments
 (0)