This piece of code utilizes OpenCV libraries to optimize a series of time lapse photos. In applications where a time lapse intents to capture movements in a scene, a movement triggered camera is used. However, such camera is expensive. A more common setup is to capture photos with a fixed time interval, this is supported by cameras like GoPro and many DSRL natively, or can be otherwise achieved with a cheap intervalometer with other cameras.
The problem with fixed time interval capturing is that, it will generate a large percentage of no-movement photo series. Culling these photos by human comparison is time consuming and unfeasible with large data set. For example a 5 second time lapse in 24 hours will generate 17280 photos. Not to mention the inherit computer storage that consumes.
The solution to this problem is to use computer vision techniques to automatically compare the photos within the series. There are two sub-problem, first we need to decide on a method to visually compare two images which can conclude a numerical “Difference Value”, next we need to decide on a logic that will use to cull a set of images based on such 1 to 1 comparison.
Diff Value
The amount of difference between two images is a very subjective question. Human perception works in a very complicated manner, we recognize not only brightness and colour difference but also gradient, shapes, silhouettes. In my implementation, I have choose to compare the RGB values of the corresponding pixel in two images, then average all the values in the image to get one scalar value as percentage to represent the image. Three channels are compared separately, thus the value ranges from 0.0 to 3.0
Note that I have added a ThresholdBinary filter after the AbsDiff command to enhance the contrast of the comparison. The value of 30 is determined by experiments.
Since I’m writing in VB.net with OpenCv via Emgu (Engu is a .net wrapper for OpenCV). My code contains such lines:
Dim previousImage As New Image(Of Bgr, Byte)(previousImageLocation) Dim thisImage As New Image(Of Bgr, Byte)(thisImageLocation) Dim imgAbs As Image(Of Bgr, Byte) imgAbs = thisImage.AbsDiff(previousImage) imgAbs = imgAbs.ThresholdBinary(New Bgr(30, 30, 30), New Bgr(255, 255, 255)) Dim result As MCvScalar = Emgu.CV.CvInvoke.cvAvg(imgAbs, Nothing) Dim difference As Float = Math.Round(result.v0 + result.v1 + result.v2, 2) previousImage.Dispose() thisImage.Dispose() imgAbs.Dispose()
The culling algorithm
- Start with first image from the “original list”, move that image to the “culled list”,
- Compare the last image (the one you just placed in) in the “culled list” it with the next image in the “original list”.
- If the difference value is below a threshold, move that image from the “original list” to the “culled list”
- If the difference value is above or equal to that threshold,  discard that image from the “original list”
- Jump to (instruction 2) unless the “original list” is empty.
- “culled list” is the final list of  images that contains motion.
Various points to note
- Camera is assumed to be non-moving. Comparing photos when the camera is also moving requires a stabilizing process that matches the moved images. This can be done via featuring detection and matching and transforming one image onto the space of another. I have never written one, but more information can be found about Feature matching here.
- Exposure settings is assumed to be non changing.
- Gradual changes in the images (think about sunset) will cause the code to cull out a sparsed-out list of the original images. This is a caused by the comparator discarding the image below the threshold. This is a desirable effect in certain applications, the effect can be tuned by adjusting the threshold. Once the gradual changes accumulates above the threshold value (assuming the change is gradual and with trend)Â the comparator will pick that image into the “culled list”.
- Colour difference is a topic in colour science that deals with colour distance in a certain colour space. Some colour space are more related to perception of human eyes.
