Week 6 content of scribble AI
Hello all! I hope the Fall break was a welcome and much needed break for all. Let's pick right where we left from last time. We will now load the MNIST Dataset and train our model on it and test it. After that, we will switch to Pytorch, which will significantly improve our neural network.
The MNIST dataset is a dataset that consists of hand-drawn digits. To use it with Python we will have to install the following module:
pip install python-mnist
You can download the mnist dataset from here:
make sure you put all the files in a folder named "data" in the same folder as your code files.
After that, just below your Neural Network code, have the following code to load the dataset
# Load the MNIST dataset from files
from mnist import MNIST # Have this import statement at the top of your file
mndata = MNIST('data')
X_train, y_train = mndata.load_training()
X_test, y_test = mndata.load_testing()
# Convert lists to NumPy arrays
X_train = np.array(X_train)
y_train = np.array(y_train)
X_test = np.array(X_test)
y_test = np.array(y_test)
# Normalize the data
X_train = X_train / 255.0
X_test = X_test / 255.0
What does it do?
Here, we load the training and testing data into our file. Since the digits are in numeric form that represent the image of a digit, they are all 28 x 28 pixels. We then normal our datasets. If you know about RGB, they range between 0 to 255. And the data (the 28 x 28 pixels) is also of the range 0 - 255 indicating their color. We divide the dataset by 255 to "normalize" it so that it now ranges between 0 - 1.
# # One-hot encode the labels
encoder = OneHotEncoder()
y_train_encoded = encoder.fit_transform(y_train.reshape(-1, 1)).toarray()
y_test_encoded = encoder.transform(y_test.reshape(-1, 1)).toarray()
nn = NeuralNetwork(num_inputs=28*28, num_hidden_layers=3, num_hidden_layer_neurons=11, num_output_layer_neurons=10)
nn.train(X_train, y_train_encoded, epochs=5, learning_rate=0.01)
print(nn.forward(X_test[801]))
print(mndata.display(X_test[801]))
One Hot Encoding is procedure that we always apply to the "labels". Think of it like this, the data is the actual raw data and corresponding to each data is a label that describes what that data is. For example some data might be a 28 x 28 pixel of numbers that represent the label "5", meaning 5 is our digit. What one-hot encoding does is converts the outputs into a list format containing only zeros and one. The list will start from "zero" and end at "nine" in our specific case. Every element will be "zero" and the correct digit will be "1". For example, if some data represents the label "5", it's one hot encoding will be [0, 0, 0, 0, 0, 1, 0, 0, 0, 0]. Here, the 6th index is 1 meaning, the answer is 5 (remember that it starts from zero).
We then create our neural network with 28x28 inputs, 3 hidden layers with 11 neurons each (you can play around with this), and 10 output neurons since our output has to be a digit from 0 to 9. We then train it with 5 epochs and with a learning rate of 0.01. Since this is a small and easy to learn dataset, 5 epochs is enough, but feel free to increase the epoch to something like 100 and see if the predictions get better.
Now, we can test our NN using the nn.forward() function. We will test our model on the X_test list. You can use mndata.display(X_test[some_num]) to display the digit that is in the some_num position in the X_test list. In our case, we did it on 801, but play around with it. You should see a "3" get printed. And for the prediction, the model should output [0, 0, 0, 1, 0, 0, 0, 0, 0, 0] indicating that the digit was 3.
So, our core idea for this project was to get a good understanding of how a Neural Network works, at least fundamentally. I hope the contents each week was helpful in that regard. Now, we have to recognize that our basic Neural Network is only good enough for simple tasks like recognizing the digits, anything more complicated and our model will simply not work as expected, or will work but with additional tweaks (like mini-batch gradient descent which will "upgrade" our model to become able to be trained on actual images). Even with those, the model might not be very efficient and training it will take forever.
Hence, from this point on, we will be upgrading to Pytorch and the content each week will be less hand-holdy and more focused on external resources (mainly YouTube videos).
Pytorch is a machine learning library that has Neural Networks built-in. Since we already know a good deal about what goes in and comes out of a network, it shouldn't be that hard to get started with Pytorch. To get started, we need to first install it:
pip install torch
We will be providing some starter Pytorch code that gets you up and running.
import torch
import torch.nn as nn
import torch.nn.functional as F
class NeuralNetwork(nn.Module):
def init(self, input_size, h1, h2, output_size):
super().__init__()
self.fc1 = nn.Linear(input_size, h1)
self.fc2 = nn.Linear(h1, h2)
self.fc3 = nn.Linear(h2, output_size)
def forward(self, inputs):
x = F.relu(self.fc1(inputs))
x = F.relu(self.fc2(x))
x = self.fc3(x)
return x
Put the above code in a new file. In the code, we create a Model with Pytorch with two hidden layers h1, h2. We also define a forward method. It should seem familiar to you given that we already know about relu and forward feeding.
Going forward with this project for this week, we would like you to explore the Pytorch docs, online resources, YouTube videos, Google Searches to learn about NNs in Pytorch and complete the code. I will be linking a couple videos
https://www.youtube.com/watch?v=JHWqWIoac2I
https://www.youtube.com/watch?v=Xp0LtPBcos0
https://www.youtube.com/watch?v=rgBu8CbH9XY
You can also watch these:
https://www.youtube.com/watch?v=ixathu7U-LQ&list=PLQVvvaa0QuDdeMyHEYc0gxFpYwHY2Qfdh&index=3
https://www.youtube.com/watch?v=9j-_dOze4IM&list=PLQVvvaa0QuDdeMyHEYc0gxFpYwHY2Qfdh&index=4
You shouldn't copy the code directly from the videos since they are using a different dataset, however, you should learn the concepts and apply them here. After the model class above, you should
Load in the MNIST dataset (basically the same way we did earlier in this post)
Create the model with appropriate input and output size, choose your hidden layers are you wish
Create the loss function and optimizer
Train the model in a number of epochs
All this is achievable if you understand the videos and also look up stuff online, especially errors. We will go through any errors and problems in the help sessions.
Farewell for now!
