Week 8 content of Scribble AI
Hello all! For this final week of content, we just need to connect the Pytorch model you all built and trained on some subset of the Quickdraw dataset to the Streamlit frontend so that whenever someone drew something on the drawable canvas, the app should send the image data to the Pytorch model.
Back to our streamlit content all the way back from week 1. Basically, we are just doing a few simple things to the front end.
- When the user draws something on the drawable canvas, we convert that into an image using Pillow
- We resize that image to 28x28
- Then, it is up to you to feed that 28x28 image to your Pytorch model and see a prediction come out
- Use st.write() to display the predicted doodle
That's it!
We are including a python file that contains the Streamlit code. All the code is exactly the same from week 1, we just added the code that converts the canvas image into a Pillow image and resizes it.
# Do something interesting with the image data and paths
# The image from the canvas is available as an ndarray and can be accessed via "canvas_result.image_data"
if canvas_result.image_data is not None:
# Do something here
if st.button('Save Image'):
# Convert numpy array to PIL Image
img = Image.fromarray(canvas_result.image_data.astype('uint8'))
# LOAD YOUR PYTORCH MODEL HERE
# we need to resize the drawn image to 28x28 since we trained our Pytorch model on 28x28 doodles
new_image = img.resize((28, 28)).convert("L")
# When prediction is calculated, show it by using st.write()
And that is it! If y'all used the Google Quickdraw downloading utility we provided in Discord and downloaded some doodles you wanted, the script automatically downloads 28x28 images. So, if you trained your Pytorch model on those images, then your model can only recognize 28x28 images. That is why we are resizing the image here from Streamlit to be 28x28. With that said, farewell! I hope you all can make good progress on this project and hopefully learned something interesting.
With that said, please feel free to ask any questions. As always, please remember it is important to look up resources online, stackoverflow questions, YouTube videos, online tutorials, documentation, and even ChatGPT responses to further your learning on any topic, including this project.
