Developing Kinect-like Motion Detection System using Canny Edge Detector

Veiw figure View Figure

Figure 2. Reduction of the number of data points using contours [6]

Download as

Veiw figure View Figure

2.2. Canny Edge Detector

Canny edge detector is an operation of edge detection that uses multi-stage algorithm relying on a number of adjustable parameters. Such parameters can affect the computational time and effectiveness of the outcome. An evaluation of the edge detection algorithms shows that canny edge detection algorithm is the most popular edge detector because of its simplicity and accuracy ^[1]. The stages of the algorithm can be described as following ^[8]:

1. Noise reduction: This type of edge detection performs some noise reduction by convolving the raw image with a Gaussian filter.

2. Finding gradient of the image: The edges should be marked where the gradients of the image has large magnitudes.

3. Non-maximum suppression: after gradient is estimated, only local maxima should be marked as edges.

4. Edge tracking and Hysteresis thresholding: Canny uses thresholding with hysteresis which requires two thresholds -high and low hysteresis is used as a means of eliminating streaking. Streaking is the breaking up of an edge contour caused by the operator output fluctuating above and below the threshold ^[9].

In general, Canny edge detection algorithm has two flexible parameters: the size of the Gaussian filter and the threshold. In our case we optimized the thresholds as shown below in Figure 3; the left image is the original representation of the object whereas the right one is the processed image using canny edge detection.

The blue contour around the object as shown in Figure 3 preserves the overall shape of the object. Moreover, the green circle represents the virtual circumference and the red circles represent the hit points which are the intersection between the contour and the virtual circumference. These will be discussed further in the next sections but just note that these are the key components that are critical for our subsequent processing.

Figure 3. Example of canny edge detection

Download as

Veiw figure View Figure

2.3. Virtual Circumference

Note that the original picture on the left in Figure 3 does not have any circle around the object, as it is shown in the processed image on the right. The circle in the processed image is called the virtual circumference and it is displayed only for explaining what is happening behind the scene for each frame during the processing.

The virtual circumference as shown on Figure 3 is the green circle around the object. It is a representation of an expected range of object motion, which can lead to their identification. This identification is accomplished by setting a matrix in Figure 6 that maps a set of points from within the virtual circumference to a particular motion. The way it works is while the video is played and the frames are processed the intersection between the contour of the object and the virtual circumference in each frame as described earlier called hit points, will be used along with the matrix in Figure 6 for that virtual circumference to identify the movement.

If this virtual circumference has a geometrical shape that can be defined by a mathematical equation, then we can verify the set of points forming the contour around the object with that equation. Therefore, we can claim that an intersection between the contours and virtual circumference was made and the motion can be identified. In our case, we noticed that the set of expected movements as shown in Table 1 are within a circle surrounding the object. We can use the Euclidian distance to measure the distance between the center of the object and the points on the contour. When it is equal to the radius of our virtual circle circumference, it implies that we have an intersection. That means we have hit one of our expected target points and the motion is identified.

The Euclidean distance is given by the Pythagorean formula. By using this formula as a distance, Euclidean space becomes a metric space ^[3].

Where:

C is Center

P is point

C (p₁, p₂) is the center of gravity of the object we are tracking.

P (q₁, q₂) is a point on the object contour.

In this paper, for the proof of concept we use the circle. However, it could be any shape as long as it is expected and the intersections identify uniquely a movement, this method can be applied.

3. Experimental Results and Discussion

3.1. Tools

For our implementation, we used the OpenCV (Open Source Computer Vision Library) which was initially introduced by Intel and released to the public at the IEEE Conference on Computer Vision and Pattern Recognition in 2000. This library is implemented in C/C++. Thanks to Heiko Kiessling who made a wrapper around this library and made it available through code project it was possible for us to use a more recent programming language C# build recently from the ground up by Microsoft under the .NET platform. Using visual studio 2010, C# 4.0 and .NET framework 4.5 along with the wrapper interfacing OpenCV called cvlib allowed us to implement a nice application that illustrates the reasoning behind this paper, and a walkthrough this methodology.

3.2. Experiment

Table 1 below describes the set of movements that needs to be identified, the methodology described above is applied and the results and discussion will follow thereafter.

Table 1. Expected movements with the details of each movement

Download as

View current table in a new window

Tables index

Veiw figure View Table

Figure 4. Up Movement identified

Download as

Veiw figure View Figure

3.3. Results

All movements were identified successfully using the technique explained earlier. Figure 4 illustrates screen shots from processing the basic movement video for a simple movement.

Figure 5 is another example from playing the advanced movement video. The same principle applies but in this case note that a second virtual circle displayed with a smaller radius, that was required to detect a slightly more complex movements such as a circle detection, where we needed extra timer to insure that the start and end of the circle is done within a certain amount of time other than that advanced movement such as horizontal stretch which is really a combination of a two simple movements Left and Right is successfully identified as shown below.

Figure 5. Horizontal Stretch identified

Download as

Veiw figure View Figure

First of all, we started by playing the video for the basic movement. The format is AVI at a rate of 25 FPS (frames per second). Then, we paused at a particular frame. However, the movement as displayed is an “Up” movement identified in the status bar as shown in Figure 4 while processing the video.

In order to better understand what is going on behind the scene, we added a tab that helps braking up the steps performed in real time to accomplish the motion identification.

Step 1: Determine the radius of the virtual circumference in our case it’s a circle that is good enough to capture all expected movements, now as its name implies it’s not a real circle it’s a theoretical one but we displayed it for the purpose of demonstration.

Step 2: Compute the distance between the center of gravity of the object, in our case the person making the movements, and the points on the contour detected by Canny. This was done using the Euclidian norm as explained earlier. If that distance happen to be equal to the radius of our virtual circle that means there is an intersection which is marked by the small red circles as the edges of the contour crosses the virtual circle. Note that the two bottom red circles are skipped from the interpretation as they denote the idle position. All points within a certain threshold that fall into the idle category are taken care of by our algorithm.

Step 3: The final step to this process is the motion identification, based on the intersection points flashed out in step2 and a known Matrix of the expected movement as shown in Figure 6, the identification is accomplished and is displayed on the status bar furthermore as a bonus we did add sound to the identification to enhance the user experience as well as kept the log of all identifications in the image tracking tab for reference.

Figure 6. Movements Identification Matrix

Download as

Veiw figure View Figure