Inspiration

Don’t have a workout buddy (friends)? Don’t know how to work out (scared to go to the gym)? Don’t have a personal trainer (you’re too broke)?

Introducing BeFit. Your personal fitness trainer that gives you real-time feedback, generates fun workouts, and tracks your analytics – all taking seconds to set up.

What it does

BeFit allows you to:

1. Gain instant feedback for your workouts, compared to your favourite fitness YouTuber

2. Beat your own high score by completing exercises in a game format

3. See your analytics for past workouts

4. To start, you can input a link to your favourite workout video from YouTube, which is embedded into the website instantly. You can start working out right away—by clicking “play” on the video. As you follow along with the workout in the video, BeFit uses computer vision and AI to compare your exercise form to the instructor to give you a score and feedback in real-time. When you finish your workout, you’re presented with a highlight reel of your best moments from the workout, where you can see how your form compares to your favourite fitness YouTuber. Using the details from your workout, you’re given personalized and actionable feedback on how you can improve your exercise form.

5. In this game format, a workout is auto-generated for you. The app prompts you with a series of exercises you must do, and evaluates whether you’ve completed the exercise in proper form. The game keeps track of your score, and allows you to compete against your personal best.

6. You can see your progress and analytics over the past months! In an intuitive dashboard, you can see your best scores, time spent, and progress over time in interactive graphs.

How we built it

The user interface of our web app is built with React, TypeScript, and TailwindCSS. We leveraged Tempo Labs to set up the project structure and generate UI components for our app, including the login page and the components for switching between full screen and split screen for the workout with YouTube page. The user can log in to the web app using Auth0, which allows the app to identify the user with their workout statistics.

On the home page, the user can navigate the dashboard by holding up 1, 2, or 3 fingers, which is classified using hand landmarks identified by Mediapipe. The user is then redirected to the corresponding page, allowing the user to:

1. Gain instant feedback for their workouts, compared to their favourite fitness YouTuber

2. Beat their own high score by completing exercises in a game format

3. See their analytics for past workouts

4. To start, the user inputs a YouTube video link, which is embedded into the website instantly. The user can start working out right away—by clicking “play” on the video. The user’s webcam and the video stream from the YouTube video is fed into our algorithm, which generates a similarity score and feedback in real-time. The program first analyzes the pose coordinates of the user and the instructor from the YouTube video using OpenCV and Mediapipe, and calculates the key points and angles. The joint coordinates and angles of the user and the instructor are then compared to determine a score, shown to the user.

When the user finishes the workout, a highlight reel from the workout and customized feedback is shown to the user. The web app captures the best moments from the workout into a short summary, allowing the user to see how their form compares to the instructor. Using workout details and articles from professional trainers, we leverage RAG through Voiceflow to generate personalized and actionable feedback for the user to improve their workout.

2. In the game format, the user is prompted to do one of four workout exercises: jumping jacks, squats, left side lunges, or right side lunges. The user must perform the exercise in proper form, counted as one rep. The web app uses OpenCV and Mediapipe (as described above) to determine whether the user has properly done the exercise, which increases the total score.

3. The user’s statistics and progress, including the time working out and current streak, are summarized into graphs using React, Recharts, and TailwindCSS.