An interesting post on CCD vs CMOS from Mr Anderson (syntheyes) himself. =)
—————
This first post is a biggie, the impetus for finally setting up a ~blog. The story starts with my recent purchase of a Canon HV-20 HDV camera, which has a great big sensor, likely derived from Canon’s excellent still cameras, such as the Digital Rebel XT of which I am rather fond.
So the HV-20 generates these incredibly beautiful pictures as you train it on your various typical test scenes. Great color.
BUT! Ooohhh how the story changes when you train it on a moving scene. Little did I realize that the HV-20 has something generally called a “rolling shutter” which means that the top of the image corresponds to an earlier period in time than the bottom of the image.
I have built cameras from scratch (bare chips) in years past, so I know instantly that a rolling shutter is A REALLY BAD IDEA. Sure, it takes great pictures of the kids and dog. But for professional use in moving-camera shots, JUST SAY NO.
Here’s what happens. A vertical line, if you pan left or right, becomes slanted. Standard internet folklore says “don’t do that”—don’t pan fast and it won’t be a problem. Again, that is great for your kids and dog, but for camera tracking and inserting 3-D objects it is completely unacceptable. Even a very slow pan rate will come out to several pixels per frame at HD resolution—- but with SynthEyes and your 3-D rendering software, we are looking to match at a sub-pixel !
If you think that because you don’t pan, this won’t be a problem, think again. The same thing happens on dolly or truck moves. If you tilt up and down, instead of slanting you get very visible vertical stretching or compression. If you shooting forward from a car, your image will be mistimed and probably difficult to see that it has happened.
The most spectacular disaster came from my helicopter charter. With new camera gleefully in hand, I got a quick charter flight together to acquire some new HD example footage, especially for the Bentley Exposition next week.
When you have a rolling shutter, and the vibration of a helicopter, you get one of the most spectacular disasters you have ever seen. In the viewfinder during the flight, it looked fine. But back in the studio, OUCH! It looked like the images had been projected onto a shaking bowl of Jello(tm whoever). This part of the image was going this way, another part a different way, etc. It was all completely and utterly $200 of useless footage straight into the garbage can.
Sony has some CMOS cameras where they apparently try to minimize this by reading out the image several times faster, but it can not eliminate it.
Even if your images do not visibly have distortions due to the rolling shutter, the effect is still there at some level. The tracker positions will be wrong, by up to one frame’s worth of velocity. The errors will be larger, but it’s not like you’ll just have some larger numbers to look at.
The error due to the rolling shutter will be built into the geometry of the scene and camera, resulting in a distorted scene geometry. I know people are going to email me about how SynthEyes is generating the wrong scene geometries and camera paths from their scenes shot with these cameras.
Just to make life interesting, note that a standard film camera can also have a rolling shutter. The design of the shutter will determine the extent of the problem; it is pretty easy to calculate. It may contribute some geometric errors depending on camera motion.
Is there a magic fix for this problem? No. Obviously if you have a CMOS camera you can try to minimize it by avoiding and minimizing camera motion, but that sort of spoils the point of moving pictures, right? There’s a technical thing that SynthEyes could do with a Sizzle script, if the camera motion is very smooth, to compensate for it so that the geometry comes out better. But the images themselves would still be bad, and you wouldn’t be able to render out of max or maya
with matching mistimed images.
Your best approach is to stick to CCD cameras. Leave the CMOS ones for the kids and dogs.
_____________
Russ Andersson
Andersson Technologies LLC
championship.
Diary