One of the cooler features I’ve found in HTML5's new <video> tag is the ability to easily stream its content to a canvas. By passing the video element to the drawImage method of a canvas, along with some size parameters, you’re able to create a duplicate output of the video. This opens up a whole new set of options for what you can do with your video data. We’re going to use it to find the average color of the frame.
Once you have your video piped into a canvas, you can use the getImageData method on the canvas to pull the current frame. This will return an ImageData object which includes a CanvasPixelArray list named data. The length of this list is video_width * video_height * 4, commonly known as RGBA format. For every pixel there is a red, green, blue and alpha value. By using every pixel on the canvas (or every 4 in our case, to increase speed), we are able to calculate the average color of the frame. In the example below, I set the average frame color as the background at 30fps. You can also see the average frame color listed below the video. You may need to be using Google Chrome or Firefox to see this example.
If you take a look at the source code of this page, it should be fairly clear as to how it works. To reiterate, I am playing a video using the new HTML5 <video> tag, which is hidden. I then stream that video data into a canvas, which I read 30 times per second. Every time I read the canvas, I figure out the average color and set that as the background color. In my next blog post, I will work on moving some of these expensive calculations out of the main event loop using web workers.