Projects - FlappyArms

FlappyArms



OpenCV Python Pygame


Inspiration

As much as I'd like to deny it, this project was the byproduct of doom scrolling... To illustrate, there were multiple times when I saw influencers playing flappy bird by doing push ups so I decided to remake it as I felt like it encouraged healthy physical activity.

 

How to play

 

How it works

OOP implementation of bird and pipes allows for easier maintenance. Aside, OpenCV uses our webcam to capture video where a video is just a series of 2D arrays and each value is a color combination such as RGB. Since our video size may not be the same as our game window size, we resize the video and ensure it fits on screen.

 

def capture_frame():
    ret, frame = cap.read()
    if not ret:
        print("Error: Could not read frame")
        return None
    
    original_height, original_width = frame.shape[:2]

    width_scale, height_scale = WIDTH / original_width, HEIGHT / original_height
    scale = min(width_scale, height_scale) # Stays within screen

    new_width = int(original_width * scale)
    new_height = int(original_height * scale)

    frame = cv2.resize(frame, (new_width, new_height), interpolation=cv2.INTER_AREA)

    # Convert the frame from BGR (OpenCV) to RGB (Pygame)
    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    return frame

 

With this, we can turn the frame into a pygame surface and then display it on pygame. As for face recognition, it's also done by OpenCV. Since we want the bird position to be mapped onto our face position, we can use the cooridnates provided from face recognition given as (left pos, top pos, width, height) to get the center of a face. Since we are only interested in the y-position, the height is given by top pos + height // 2 because (0, 0) is the top left position in a pygame window. Since the program may not always run smoothly causing the height of the bird to fluctuate erratically, we implement a technique called the exponential moving average (EMA) also known as exponential smoothing.

 

Specifically, the value we want to smooth is the bird position and here's the formula:

s0 = x0

st = αxt + (1 - α)st - 1 , t > 0

Where { xt } is our raw data set of heights, { st } is our smoothed data set and 0 < α < 1 is the smoothing factor. Evidently, the formula is recursive, but even more evident is the intuition of this formula. If α is closer to 1, then it means that less weight is given to the importance of the previously smoothed term st - 1, so the new smoothed term st will be affected more by the current raw value xt. On the contrary, if α is closer to 0, then more weight is given to the previously smoothed term and less weight for the current raw data, so the final smoothed term is closer to the previously smoothed term. This wikipedia page is also a good reference. Below is the implementation.

 

class Bird(pygame.sprite.Sprite):
    def __init__(self):
        super().__init__()
        self.rect = self.rects[self.current_index]
        self.alive = True
        self.smoothing_factor = 0.2
        self.prev_center_y_pos_smoothed = HEIGHT // 2
        ...

    def update(self, face_pos):
        if self.alive:
            ...

            if face_pos is not None and self.alive:
                (x, y, w, h) = face_pos

                raw_center_y_pos = y + h // 2
                smoothed_center_y_pos = self.smoothing_factor * raw_center_y_pos + \
                    (1 - self.smoothing_factor) * self.prev_center_y_pos_smoothed
                self.prev_center_y_pos_smoothed = smoothed_center_y_pos
                self.rect.center = (100, smoothed_center_y_pos)

            ...

 

Features

 

Links