Search sheldonbrown.com and sheldonbrown.org
You really only want to perform some video processing functions once -- for example, inverting upside-down video; image stabilization; resizing. The freeware Windows video processing application VirtualDub, (at http://virtualdub.com/) is lean and fast. By preprocessing in VirtualDub and saving the resulting file, you avoid having your main video editing suite perform the same processing again and again as it renders video.
There is another article on this site about VirtualDub. Please read it in connection with this article.
VirtualDub performs most functions using filters. One Virtualdub filter, Deshaker, is the best image stabilization software I have encountered. VirtualDub is supplied with a number of filters by default, but Deshaker is an add-on. You get it from the Deshaker Web site (at http://www.guthspot.se/video/deshaker.htm), and install it in the VirtualDub folder on your computer. Like VirtualDub, Deshaker is freeware, but the Web site requests a donation. I paid.
Filters are the top item in VirtualDub's Video menu. When you click on it, the filter dialog box will open. Click on "Add" to see the list of available filters. Then click on a filter in the list to add it. Filter information will appear in the filter dialog box. You may use more than one filter at a time: think of the filter list as a sort of pipeline, working in order from top to bottom of the list. So, for example, you may invert the video image, stabilize it and then adjust color, all in one pass through VirtualDub.
There is another article on this site about image stabilization in general. While excellent documentation is available on the Deshaker site, here are some specifics relating to bicycle video.
Whether you shoot it with a helmet camera or with a camera mounted on the bicycle, bicycle video is going to be shaky. There will be small, rapid shakes and also wide sweeps from a turn of the head, or of the bicycle.
Deshaker runs two passes through the video, to establish settings before actually adjusting the video. In this way, Deshaker can look ahead in time, preventing it from confusing panning with shaking. Single-pass or real-time image stabilization can run to the end of its range, resulting in jerky motion during these wide sweeps.
Deshaker lets you restrict the part of the image which it senses, and that can be useful. More about that later.
You may have Deshaker sense rotation in the first pass. You might as well, just in case you want to correct for it later.
Deshaker's default stabilization values of 1000 are designed to produce floating-on-a cloud-smooth pans. For bicycle video, I recommend lower values for horizontal and vertical pan, around 100. These settings will eliminate the rapid shake that is usual on a bicycle, while decentering the image less and producing only small border areas of the image outside what the camera could see; also, minimal geometric distortion when using a camera with a fisheye lens.
On the first pass, select "Run Video Analysis Pass" in the File menu, and preferably check "Uncompressed RGB/YCbCr" in the Video menu. On this pass, Deshaker will save a log file, and VirtualDub will not save a video file. Still, processing will run much faster without a compressor -- more than twice as fast as with the XVid MPEG-4 compressor. Evidently, Virtualdub sends the video to the compressor for processing even if it isn't saving a file.
I recommend saving a different log file for each video that you stabilize. You can then re-use it with different settings on the second pass. The video must be the same one to work with the same log file.
Processing on the second pass can be slow, but on the other hand, I prefer to have the entire stabilized file to work with, and so I let my computer run Deshaker when I'm working on something else. If the original file is in any format other than AVI, you can use the same filename, just with the different filetype. Keeping the original file, log file and processed file in the same folder makes them easy to find.
On the second pass select "Save as AVI", and a compressor in the file menu. The free XVid MPEG-4 compressor is my favorite. Uncompressed video files are so huge (5 GB per minute!) that you will quickly run out of disk space. These files won't play at normal speed either, even from a fast hard drive.
If you make any geometric changes to the video image (for example, resizing or rotation) before Deshaker, in the same VirtualDub run with DeShaker, you must run this on both passes, or else the adjustments logged in the first pass will not match those on the second pass. The result may be of higher quality because the image is then not being compressed for storage until after both processes, but extra time is required to run the geometric adjustment twice. VirtualDub itself can enlarge or shrink the image, but it will still only be of the size you select in the upper right of the dialog box on pass 2. An enlarged or rotated image will be cropped unless you select a size large enough to hold all of it.
Also, filling in borders using neighboring frames will not work correctly if you run the geometric correction before Deshaker. With a rotated image, for example, the fill will only be at the corners, where it reaches the edges of the frame. It usually makes more sense, then, to run other processes after Deshaker. The exception would be if you are correcting for geometric distortions such as fisheye distortion which affect the operation of Deshaker itself.
As the image shifts position to remove shake, black border areas will appear unless you use one of the several tricks that Deshaker offers.
One is adaptive zoom, which enlarges the image to push the border areas off the screen. If adaptive zoom is turned on in Deshaker, the image will "balloon" in size whenever there is a big shake. If the camera is facing forward or rearward, this can make it look as if you are suddenly speeding up or slowing down -- even riding backwards.
Another option is to use earlier and later frames to fill in the border areas. This doesn't work perfectly with a moving camera, because motion forward or backward only looks like zooming if everything in the picture is at the same distance from the camera -- images of objects closer to the camera change size faster. Still, this works well enough that it's worth doing even if the results aren't perfect. With a camera facing sideways, zooming is no help at all, but you can get a good match if the margins of the image are all at the same distance (for example, a wall).
On the second pass, setting motion smoothness for zooming to zero will eliminate the "ballooning" effect. Setting 100 for horizontal, vertical and rotational motion smoothness (or 200 at 50 or 60 frames per second) will eliminate annoying rapid shake and produce a result which is more like what you see as you are riding -- while minimizing decentering of the image.
You might also disable stabilization against rotation in the second pass. In helmet-camera video, or with a camera mounted steadily on the bicycle, there is little rotational shake. There's quite a bit of slower rotation due to tilting the head or leaning the bicycle, but that is normal and expected. Sensing rotation would result in large, tilted border areas which you will have to crop out of the image to avoid their being distracting.
If you do not have Deshaker correct for rotation, you will also somewhat avoid degrading the image, because horizontal lines in the image remain horizontal. If you use no zoom, you also retain the full resolution of the original image More about this later.
If the camera doesn't change position or zoom, you can use the stabilization options more freely. Stabilized images then look as smooth as if the camera is on a tripod. The only errors likely to be visible occur if moving objects are in different positions in the fill frames and the current frame.
If you have decided from the start to use only a fixed, zoomed-in area of your image, you can have Deshaker crop it for you. A bit of additional zoom -- about 1.1. -- will keep border areas out of the picture almost all the time with low motion-smoothness settings.. If you need to change the panning and zooming during a shot to remove borders, you will probably want to do this later, in your editing suite. It's time-consuming, but you can make better artistic decisions than Deshaker can. This effort is mostly necessary if you want to use as much of the image as you can, even though occasional wide, rapid pans increase the size of border areas. Shoot at the highest resolution your camera and editing suite can support, to avoid much loss of sharpness. I typically shoot at 1080 x 1920, and display at 720 x 1280.
I recommend that you have Deshaker look backward and forward the full 30 frames to fill in the border areas. When these areas fill in seamlessly, you have a larger image area to work with. Not correcting for rotation will result in smaller but more poorly-matched border area.
Many cameras have a "rolling shutter", which scans the image from top to bottom, rather than capturing it all at once. The image will then appear stretched vertically if the camera is panning upwards quickly, and squished if the camera is panning downward. If Deshaker knows the amount of rolling shutter for your camera, it can compensate for this. It cannot, however, compensate for very rapid shake which occurs during the scan of each image. If this is vertical, objects at different heights in the image may become compressed and expanded -- the dreaded "Jello-o" effect. Horizontal panning skews background objects so vertical lines appear slanted, but this is not very noticeable. Very rapid horizontal shake can make vertical lines appear wavy.
The Deshaker Web site lists rolling-shutter values for a number of cameras and describes a special test procedure to measure the rolling shutter amount of a camera, but you can calculate it without having to do a special test, using a video clip with a rapid pan across a building or other object that has horizontal and vertical lines at right angles. This is common enough in helmet-camera videos, when the cyclist turns the head. If the camera has a rolling shutter, the object will appear skewed. Stop the image on the screen, so you can advance it a frame at a time, just as with the test described on the Deshaker Web site. Lay a straightedge -- the edge of a sheet of paper will do -- along a vertical edge which appears skewed in the image. Extend the line upward and downward to measure the horizontal positions at the top and bottom of the screen.
Measured values for rolling shutter for my cameras are:
Checking for rolling shutter without having to do a special test shoot.
The camera was panning and so the buildings are skewed.
The still-frame values in the rolling shutter formula serve only to make a correction if the vertical line in the test shot was not precisely vertical -- so, draw a right angle across a horizontal line in the image you are measuring from, and carry it to the top and bottom of the screen to simulate the locations of the top and bottom of a still frame. A sheet of paper conveniently has square corners and can be used to derive the vertical line from the horizontal. The horizontal and vertical lines should cross near the center of the screen, to avoid geometric errors caused by the lens itself, and especially if it is a fisheye lens. Now, more about lenses...
Deshaker can produce astonishingly good results with a conventional lens -- one which images straight lines as straight lines. Telephoto and "normal" lenses are of this type, as are some wide-angle lenses. These lenses project an accurate geometric representation of the subject onto the flat sensor in the camera. This kind of lens cannot look behind itself, or extremely far out to the sides.
A fisheye image is easy to recognize: straight lines curve like parentheses near edges of the image. A fisheye lens can cover a wider angle, up to 180 degrees and sometimes even more. Because a fisheye lens compresses the image at the edges, shake will be different there -- edges of the image are in very different directions compared with the center.
There is a lot of image area out at the edges, and Deshaker takes an average when calculating how to reposition the image to stabilize it. Including the edges in the sensed area when using a fisheye lens will leave the center of the image shaky -- though the center is usually the main area of interest. With a fisheye lens, I recommend tracking an area only about half as high and wide as the image when running Deshaker's pass 1, the analysis pass. (For example, if you are shooting in 1080 x 1920 HD, exclude everything outside 270 pixels from the top and bottom, and 480 pixels from the right and left.) Some visible shake will still occur in your final result, but it will mostly be near the edges of the image. You may want to use a smaller sensing area if you are going to crop the image significantly.
In a fisheye image with rapid panning, image stabilization will result in visible changes in image geometry as compressed and curved parts come closer to the center. The image can become "wavy" when a pan is reversed. On a bicycle, most panning is horizontal and most shake is vertical.-- also, the image is wider than it is high -- so, one way to minimize geometric distortion is to set horizontal smoothness to zero or to a low value in Deshaker's pass 2. Another way to minimize distortion is to crop the image after stabilizing it, but the best way of all is the old-fashioned way, to avoid shake as much as possible in the first place!
You may also want to exclude the margins in pass 1 even if a camera has a non-fisheye wide-angle lens (that is, if straight lines remain straight at the edges of the image) With these lenses, objects appear larger near the edges of the image, and image geometry changes noticeably as the camera pans, or if it shakes wildly. With a normal or telephoto lens, though, it's better to sense the entire image, giving Deshaker more to work with.
A feature not yet available would compensate for geometric distortions, but there is only so much software can do. Projection of an image covering angles approaching -- even exceeding -- 180 degrees onto the flat plane of a computer screen is simply not possible without geometric distortion. A wraparound image would have to be projected onto a wraparound screen to look normal. (We-re talking IMax here.)
|Articles by Sheldon Brown and others|