Week 6 Content of Face Emoji
Welcome to Week 6 of our Face-Emoji project! Last week you worked on gathering your dataset and training your PyTorch model using one of the two paths we outlined. Now it's time to put your trained model to the test in a real-time environment.
Whether you chose Path 1 (training on images) or Path 2 (training on landmarks), this week you'll be integrating your model with the webcam feed to detect emotions in real-time. Let's see how your model performs with live data!
Regardless of which path you chose, the overall structure of what you need to do this week is similar:
Load your trained model
Set up the webcam feed with MediaPipe
Process frames and run inference with your model
Display the detected emotion on screen
However, the specific implementation details will differ depending on your chosen path.
If you trained your model on facial expression images, you'll need to:
Load your trained PyTorch model from the saved file
Use MediaPipe to detect faces in the webcam feed
For each detected face:
Crop the face region from the frame
Resize the cropped image to match your training data dimensions
Preprocess the image (normalization, etc.) to match your training process
Convert the processed image to a PyTorch tensor
Run inference with your model to get the predicted emotion
Display the emotion label near the detected face
Make sure your image preprocessing matches exactly what you did during training
Watch out for different lighting conditions affecting your model's performance
Consider adding a confidence threshold to only display emotions when the model is reasonably confident
If you trained your model on MediaPipe facial landmarks, your workflow will be:
Load your trained PyTorch model from the saved file
Set up the webcam feed with MediaPipe Face Mesh (not just Face Detection)
For each frame:
Extract facial landmarks using MediaPipe
Format the landmarks to match your training data structure
Convert the landmarks to a PyTorch tensor
Run inference with your model to get the predicted emotion
Display the emotion label near the detected face
Ensure you're extracting and formatting landmarks consistently with your training process
Consider handling cases where multiple faces are detected (if your application supports this)
Think about how to handle frames where no face is detected
For next week, we will focus on mapping the detected emotions to emojis. We will come back with some suggestions on libraries or method that might be able to do that. Then, for the last week, (week 8) our focus will on setting up a simple front-end to package your application.
As you test your models this week, keep these evaluation points in mind:
Speed: How many frames per second can your system process? Is it fast enough for a smooth user experience?
Accuracy: How well does your model detect emotions in different lighting conditions and for different people?
Robustness: Does your system handle edge cases well (glasses, different facial orientations, etc.)?
PyTorch - Saving and Loading Models - Official tutorial on saving and loading models
Model Inference in PyTorch - How to put your model in evaluation mode
Face landmark detection with MediaPipe - Detailed guide on extracting face landmarks
Working with coordinate data in PyTorch - Tensor operations tutorial
Remember, your goal this week is a working real-time emotion detection system. Focus on getting the core functionality working before adding additional features. Good luck, and feel free to collaborate and share your experiences with your fellow club members!
