Video Motion Tracking: Do’s and Don’ts

Increasing numbers of cameras and processors in our environment allows us to use video motion tracking based on computer vision in areas we haven’t thought of before. It allows us to use interactive lighting and media to create spaces that respond to our movements.

Diving into these uncharted waters can be a daunting task. So here’s a brief explanation of various video motion tracking technologies that are available.

What is video motion tracking?

Video motion tracking algorithms use a video stream in order to track movements of people or objects in it. Most algorithms subtract the current frame of the video pixel-by-pixel from the image of a background. The basic approach is to assume the pixels that are different from the background pixels by a significant margin, are the pixels from the object. The main question is how is the algorithm treating objects moving very slowly or objects that stop moving after a while (such as parked vehicles). Also, how is it treating other moving shapes, such as leaves or moving shadows, that we don’t want to track.

Things to be mindful of

The key challenge of motion tracking based on a video feed is determining what exactly is a background. For example, how do we deal with objects moving differently than expected or moving objects that we don’t want to track? These include leaves or plastic bags being blown by the wind or moving shadows or light spots.

Changing visual conditions

If the visual conditions change rapidly it might be difficult to distinguish between objects that need to be tracked and the changing background. Examples include video walls in the camera view, car headlights or strong moving light spots or similar.


Tracking framerate means how many images per second the tracking algorithm analyses for moving objects. The faster the algorithm the more precise and responsive it is, but also the more resources it consumes.


For video motion tracking we usually don’t need such high video resolutions as are needed for security purposes. Also, it is beneficial to decrease the resolution of the video, because it significantly speeds up the performance of the tracking algorithm. However, if you lower it too much, it might reduce the reliability and accuracy of the motion tracking.

Camera Types

The equipment that’s needed for every project based on video motion tracking is a color or grayscale video camera and a processing platform. When it comes to cameras there are several options:

IP cameras

IP cameras are rapidly gaining popularity as they are using standard Ethernet equipment such as Cat5 cables, Ethernet routers and switches for video transmission. They also transmit video in digital packets, which increases the quality of video compared to analog cameras. They are available in a wide variety of models, both outdoor and indoor. In many cases, they can be powered through Ethernet connection (Power-over-Ethernet), which speeds up the installation time.

Analog cameras

Analog cameras are a cheaper alternative to IP cameras, but they do not offer the same ease of installation, high-quality video signal and versatility of IP cameras. However, if size is a big factor, they can be useful as they usually come in a smaller form factor.

USB cameras

USB cameras (also called web cameras) are the most affordable way to get a video feed for motion tracking into the controller. However, they were designed for desktop use, which usually prohibits their use in anything but the simplest temporary installations.

This is an excerpt from our guidebook on Technologies for Interactive Architecture.

Related Posts
Let’s talk!