Week 10: Acquiring and Manipulating Videos#

Laboratory 7
Last updated Jan 5, 2023

00. Content #

Mathematics

  • N/A

Programming Skills

  • Using Software Documentation

  • OpenCV (cv2)

Embedded Systems

  • Thonny and MicroPython

0. Required Hardware #

  • Microcontroller: Raspberry Pi Pico

  • Breadboard

  • USB connector

  • Camera (Arducam HM01B0)

Write your name and email below:

Name: me

Email: me @purdue.edu

import numpy as np
import matplotlib.pyplot as plt
import cv2

1. Reading Videos with OpenCV #

In this lab, we will be using a Python library called OpenCV (cv2). From the official documentation, “OpenCV is an open-source library that includes several hundreds of computer vision algorithms.” Today, we will mostly be using it to read and display modified video frames, and we will be doing the “vision” in computer vision manually with our own functions.

Run the following cell to read in the video file. Using vid.read(), returns two values: the Boolean varible success, which is True if the frame was read without any errors, and frame which is as it sounds the image that is captured from the video. The camera hardware you will connect to your Pico can only return grayscale images, so we are converting each frame to grayscale using the function cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY). For speed in later computations, it is helpful to reduce the frame size. Here, we’ve scaled down each frame by a factor of 4.

Running the cell will cause a separate window to pop up, displaying the smaller, grayscale video. After reading in the whole video, the separate window will close automatically; you can override this by pressing the “Q” key on your keyboard.

Download the video file we’ll be using for this lab: test_vid.mov. This is a video from Youtube showing a bouncing ball.

vid = cv2.VideoCapture('test_vid.mov') 
height = vid.get(cv2.CAP_PROP_FRAME_HEIGHT)
width  = vid.get(cv2.CAP_PROP_FRAME_WIDTH) 
scale = 0.25
new_size = (int(width*scale),int(height*scale))

while vid.isOpened():
    success, frame = vid.read()
    if not success:
        print("Unable to read frame. Exiting ...")
        break
    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    frame = cv2.resize(frame,dsize=new_size)
    cv2.imshow('frame', frame)
    if cv2.waitKey(25) == ord('q'): # press Q on keyboard to stop
        break
vid.release()
cv2.destroyAllWindows()

Exercise 1 #

In the cell above, we scale down the orginal video frames by a factor of 4 by defining new_size and using cv2.resize().

Open and watch the above video again, this time scaled down each frame by a factor of 8. How much less CPU time does it take to play the video, compared to when you only scaled it down by a factor 4?

Write Answers for Exercise 1 Below

2. Modifying Videos #

Exercise 2 #

Read the documentation for the waitKey function.

Part 1

Play the video slower than its original playback speed.

Part 2

Play the video faster than its original playback speed.

Write Answers for Exercise 2 Part 1 Below

Write Answers for Exercise 2 Part 2 Below

Exercise 3 #

Play the video backward.

Write Answers for Exercise 3 Below

Exercise 4 #

Part 1

Flip the video’s visuals over the y-axis.

Part 2

Flip the video’s visuals over the x-axis.

Hint: There are many ways to do this, but you can look into NumPy or OpenCV’s flip function.

Write Answers for Exercise 4 Part 1 Below

Write Answers for Exercise 4 Part 2 Below

Being able to extract specific frames from a video is a useful skill. It can be used, for example, to trim the ends of the video or to remove a section in the middle.

Exercise 5 #

Extract the first 200 frames of the video.

Write Answers for Exercise 5 Below

3. Connecting the Camera #

This time, we will be recording our own videos using the Arducam HM01B0, which is a small camera that can be hooked up to the Pico.

Wiring Instructions#

Please make sure your microcontroller is not plugged to the computer while you are wiring things together. Ask the instructor if you are unsure about your wiring. Use your jumper wires to make the following connections.

HM01B0

Pico

VCC

3V3

SCL

GP5

SDA

GP4

VSYNC

GP16

HREF

GP15

PCLK

GP14

DO

GP6

GND

GND

After ensuring that the wiring is correct, hold down the BOOTSEL button on the Pico and plug it in. Download arducam.uf2 and copy it onto the Pico’s drive, which should have popped up in your folders as an option. The Pico should automatically disconnect once the file has transferred, then its LED will start blinking rapidly.

Once the Pico has been successfully connected, run the following cell to make sure we have detected the Pico successfully.

import time
import serial
from serial.tools import list_ports

PICO_HWID = "2E8A"


def get_pico_port():
    pico_ports = list(list_ports.grep(PICO_HWID))
    if len(pico_ports) == 0:
        raise Exception(
            "No Raspberry Pi Pico was detected. Check to make sure it is plugged in, and that no other programs are accessing it"
        )
    return pico_ports[0].device


print("Here are all the serial devices detected:")
for port in list_ports.comports():
    print(port.device, port.hwid)

port = get_pico_port()
print(f"\nselected port {port} as most likely to have a raspberry pi pico")

Capturing a still image#

Now that the Pico and camera have been connected, run the following cell to get a still image.

buffer = bytearray(96 * 96)
img = np.zeros(shape=(96, 96), dtype="uint8")

with serial.Serial(port, timeout=1) as s:
    s.read_until(b"\x55\xAA")
    s.readinto(buffer)
    img.flat[::-1] = buffer

plt.imshow(img, cmap="gray")
plt.show()

Exercise 6 #

Use the camera to acquire an image of yourself with your hand up, and an image of yourself with your hands down. Use these two frames (you are welcome to use more than 2 frames if desired) to create a video of yourself moving your hands up and down that lasts a few seconds.

Write Answers for Exercise 6 Below

Streaming video#

Now that we can capture a still image, the next goal is to stream a video. Both cells will have to be run before the video is streamed to your screen. After running the first one, a still image should pop up but once you run the second cell it should start streaming what your camera is seeing in real-time.

%matplotlib widget

fig, ax = plt.subplots()
render = ax.imshow(img, cmap='gray')
plt.show(block=False)

try:
    with serial.Serial(port, timeout=1) as s:
        while True:
            s.read_until(b"\x55\xAA")
            s.readinto(buffer)
            img.flat[::-1] = buffer
            render.set_data(img)
            fig.canvas.draw()
except KeyboardInterrupt:
    pass

When you are done, you will have to hit the Interrupt Kernel button, which can found at the top of the screen (the stop symbol) or under Kernel.

4. Modifying Your Own Video #

Now that you are able to capture your own video, you will try the same modifications you made on the prerecorded videos on the one you recorded.

Exercise 7 #

Capture and save a list of frames using the camera.

Part 1: Either trim both ends of the video or remove a section in the middle.

Part 2: Save another list of frames from a second video. Splice the two videos together so that one plays after the other.

Part 3: Pick an editing function we haven’t mentioned (e.g., adding text, video filters, etc.) and try it out on either the prerecorded video or the ones you recorded using the camera.

Write Answers for Exercise 7 Below

Reflection #

1. What parts of the lab, if any, do you feel you did well?
2. What are some things you learned today?
3. Are there any topics that could use more clarification?
4. Do you have any suggestions on parts of the lab to improve?

Write Answers for the Reflection Below