"Rolling shutter effect" on today's video devices

[Matthew Crowther]

I do not own a dedicated video camera any more, I have a cheapish dash cam and also have my DSLRs and iPhone 8. The dash cam and phone have issues at night with lightning- and I usually only use the DSLRs for still photography. Do the newer 4K video cameras exhibit the same issues with night lightning? What about the higher end wide angle action cams like GoPro and newer phones like the S10 or iPhone XS? Thanks for any insight.

 

[Dan Robinson]

Unfortunately almost everything is using CMOS chips these days, with rolling shutter a given. The lone exception are the BlackMagic Ursa cameras which at the moment are the only prosumer models offering 4K with a global shutter. They are cinema cameras though that use DSLR lenses, and there have been a lot of complaints about banding and noise with them in low light. There have been rumors of more global shutter 4K cameras being developed, but to my knowledge none have hit the market yet. Rolling shutter effects for lightning is actually worse than HD with most of today's 4K cameras, with 2 to 4 splits in the image common instead of the typical 1 with a HD camera.

 

[Marshall Stoner]

If you can manually adjust shutter speed and ISO in video mode, what's the best way to minimize rolling shutter cutoff effects? I notice rolling shutter effects are always present with quick flickering lightning at night (no setting will fix it), but the bigger brighter bolts often come out okay during the day. Maybe it's because they're positive flashes with a single long-lasting return stroke rather than the typical flickering negative CG that usually gets cut up.

I'm not really sure how CMOS rolling shutter works though. Does the sensor scan a single frame over the same amount of time regardless of shutter speed? In video mode the shutter speed seems to refer to how long an individual point on the sensor is on, not the time it takes to scan the whole frame. Scanning the entire frame at a speed faster than the framerate seems desirable in terms of capturing lighting without splits (though more quick flashes will be missed), but I'm not sure how to force the camera to do this.

 

[Warren Faidley]

Sorry, no way to stop it without using a global shutter that captures each frame individually. CMOS exposes different parts of the sensor in different points in time. Thus, some moving subjects have a rolling effect. Lightning footage is destroyed by the effect. I did some testing for a motion picture camera company two years ago and although a global shutter will completely eliminate the "rolling effect," the problem with current global shutters is low light quality. You could use an old CCD camera, but the quality would be HD level. As soon as someone develops a 8k global shutter camera that works great in low, it's game over and photography / videography will finally merge into one fantastic camera.

 

[Marshall Stoner]

Sorry, no way to stop it without using a global shutter that captures each frame individually. CMOS exposes different parts of the sensor in different points in time. Thus, some moving subjects have a rolling effect. Lightning footage is destroyed by the effect. I did some testing for a motion picture camera company two years ago and although a global shutter will completely eliminate the "rolling effect," the problem with current global shutters is low light quality. You could use an old CCD camera, but the quality would be HD level. As soon as someone develops a 8k global shutter camera that works great in low, it's game over and photography / videography will finally merge into one fantastic camera.

Global shutter is probably way out of my price range. My only DSLR is a Canon EOS 6D MarkII. It makes nice bulb-shot timelapses of night lightning with it as it's good with the low light. This would be good enough for me, but in the past I had fun recording late-night storms out my garage or out the passenger side window with my little powershot S95. Quality wasn't that good, but it had a decent built-in stereo mic and global shutter. I just love the sound of thunder. Not having an accurate representation of the lightning to go along with it is a slight annoyance with the new CMOS video, but I'm not really looking to capture lightning artwork on video (stills and timelapses are for that). I know there's no way to eliminate rolling shutter. Just wondering what settings might increase the likelihood of getting full bolts. I notice footage looks better sometimes compared to others but I have no idea what it is.

 

[Marshall Stoner]

Sorry, no way to stop it without using a global shutter that captures each frame individually. CMOS exposes different parts of the sensor in different points in time. Thus, some moving subjects have a rolling effect. Lightning footage is destroyed by the effect. I did some testing for a motion picture camera company two years ago and although a global shutter will completely eliminate the "rolling effect," the problem with current global shutters is low light quality. You could use an old CCD camera, but the quality would be HD level. As soon as someone develops a 8k global shutter camera that works great in low, it's game over and photography / videography will finally merge into one fantastic camera.

Great. I just looked and all the companies that once made global shutter cameras in 1920x1080 30p immediately stopped making them when they upgraded to 4k. They hardly ever talk about global vs. rolling in their specs either and hardly anyone on the internet seems to give a shit so there's no way to search. Seriously, f*** 4K and 8K. I don't even see the damn point. My eyes aren't good enough to see the difference. It's cool if you have a screen the size of a wall. Guess I have to stick to my PowershotS95 720p with ugly sensor bright spots that can't be fixed for lightning.

 

[Dan Robinson]

From my perspective, 4K is overkill for chase video right now. Definitely not enough to worry about not having. Compelling content always matters more than resolution. Most people consume video content on tablets and phones where even standard definition is still passable and can even do well. I have yet to receive a stock request for any of my 4K weather footage. There are simply too few ways to efficiently get 4K to the end user at the moment. Broadcast TV is still HD and probably will be for many years. Youtube and some streaming services support 4K. My best performing videos on Youtube right now are older 1080p HD content, I haven't seen much if any benefit from shooting in 4K (at least not yet).

 

[Marshall Stoner]

From my perspective, 4K is overkill for chase video right now. Definitely not enough to worry about not having. Compelling content always matters more than resolution. Most people consume video content on tablets and phones where even standard definition is still passable and can even do well. I have yet to receive a stock request for any of my 4K weather footage. There are simply too few ways to efficiently get 4K to the end user at the moment. Broadcast TV is still HD and probably will be for many years. Youtube and some streaming services support 4K. My best performing videos on Youtube right now are older 1080p HD content, I haven't seen much if any benefit from shooting in 4K (at least not yet).

Yea. They can't get it to work with a global shutter, yet camera manufacturers insist on having it now. Almost seems like an advertising gimmick. I hear people in the graphics/gaming industry complaining about it because it consumes CPU resources that could be better spent on things like ray tracing and better ambient occlusion. It's just sad because I wanted to see if there were any options to upgrade from 720p to 1080p on global shutter footage, and apparently there isn't anymore as the ones that were on the market in 2015 abandoned it when they upgraded their sensors to 4K. The only option now is overpriced used cameras on ebay. Don't know if there's any way to repair a 720p CCD sensor with many defective pixels that auto-illuminate in low light making a cloudy sky look like it's full of stars at night.

 

[Marshall Stoner]

Does anyone know how well the GoPro Hero 7 Black performs with lightning? I know it's a rolling shutter, but it does do 1080p at 240fps. The slow motion lightning videos on youtube don't look too bad, but I want to know if it's also decent looking at 30fps? If it can shoot 240fps at 1080p then it's clearly capable of reading the entire sensor in 1/240 second. I'm not sure though because the i-phone 7 has a "slow motion" mode but is still complete garbage with lighting at the normal 30 fps framerate because of the way the rolling shutter is implemented and the fact that you can't manually slow the shutter speed down to 1/30 to match the framerate. I-phone 7 also has a huge problem with wind-vibration jello from it's slow rolling shutter.

Does GoPro Hero 7 Black have these same issues? Slow motion footage is cool, but if there's a short limit to slow motion record time it would be nice to get footage at 60 or 30 fps and still look nice. If you can't slow the shutter speed down during the day so as to match the framerate and as a result get horizontal strips rather than full frame lightning flashes then it's just as useless as the i-phone 7.

Clearly some rolling shutters are much much better than others. I see a lot of CMOS footage on youtube that isn't all that bad. Unfortunately the Canan EOS 6D MarkII I bought just isn't good at all for lighting. It takes beautiful crisp video and timelapse, but with lightning at night almost every damn flash has very obvious horizontal lines and major bright flashes are torn in half most of the time. I'm guessing this is because it has a big sensor and doesn't read it very fast. It's better than the i-phone, but still pretty bad compared to the better footage I see on youtube. People at the camera store are clueless about this stuff and can't help me.

 

[Michael Snyder]

Does anyone know how well the GoPro Hero 7 Black performs with lightning? I know it's a rolling shutter, but it does do 1080p at 240fps. The slow motion lightning videos on youtube don't look too bad, but I want to know if it's also decent looking at 30fps? If it can shoot 240fps at 1080p then it's clearly capable of reading the entire sensor in 1/240 second. I'm not sure though because the i-phone 7 has a "slow motion" mode but is still complete garbage with lighting at the normal 30 fps framerate because of the way the rolling shutter is implemented and the fact that you can't manually slow the shutter speed down to 1/30 to match the framerate. I-phone 7 also has a huge problem with wind-vibration jello from it's slow rolling shutter.

Does GoPro Hero 7 Black have these same issues? Slow motion footage is cool, but if there's a short limit to slow motion record time it would be nice to get footage at 60 or 30 fps and still look nice. If you can't slow the shutter speed down during the day so as to match the framerate and as a result get horizontal strips rather than full frame lightning flashes then it's just as useless as the i-phone 7.

Clearly some rolling shutters are much much better than others. I see a lot of CMOS footage on youtube that isn't all that bad. Unfortunately the Canan EOS 6D MarkII I bought just isn't good at all for lighting. It takes beautiful crisp video and timelapse, but with lightning at night almost every damn flash has very obvious horizontal lines and major bright flashes are torn in half most of the time. I'm guessing this is because it has a big sensor and doesn't read it very fast. It's better than the i-phone, but still pretty bad compared to the better footage I see on youtube. People at the camera store are clueless about this stuff and can't help me.

The Go pro7 Black is pretty good with lightning. Rolling shutter is there at times, but it seems to give nice clear shots of the Lightning also when you are going through your video. 4k 60fps works really well.

 

[Dan Robinson]

Certain types of lightning do better with rolling shutter than others. Slower lightning discharges like upward bolts and horizontal "anvil crawlers" can be slow enough that there is no change in brightness during the chip scan. Fast lightning phenomena, like cloud-to-ground bolts and some return strokes, are too fast for even the fastest-performing rolling shutter sensor to capture in a single frame. Occasionally you can get a CG with a long enough current pulse to fully expose during the sensor scan, but it's very rare. More distant bolts can also avoid splits easier since they take up less of the sensor area.

Rolling shutter is a big enough problem that there is some progress taking place to develop global shutter CMOS chips. I think in 5 years or so we might start seeing some prosumer-level cameras with those improvements.

 

[James K]

I'm truly fine with 1080p (I don't think I even have anything computer/software wize that would handle 4k+ anyway) .. plus only one TV I could watch it on.

You could use an old CCD camera, but the quality would be HD level.

I'd be curious to know what (lower-priced/general consumer-level) camcorders are out there, and in particular how well they work at night. Not interested in anything big & bulky, expensive/high-end.

I currently have a Panasonic HC-V180(which I'm assuming is CMOS, but don't know.) I've videoed a couple storms in the distance & they turned out ok - no lines, but never had the chance to try on any close ones (simply haven't had a storm to try it on).
A couple of the primary reasons I got this particular model were its zoom, and price. Main thing I'm really dissappointed in is the fact it pretty much sucks for anything in the dark (won't focus).

Broadcast TV is still HD and probably will be for many years.

From what I've heard, broadcast itself simply doesn't have the bandwidth for anything more than 1080p (and very few stations broadcast in 1080p, because then they have to give sub-channels)

 

[Bill Hark]

I still use a Sony HVR-Z1U, an old 3CCD HDV camcorder that takes great lightning stills. Only HD, but still pretty good. I bought it used off of Ebay in 2008. It's quite robust and has survived several blow overs including on pavement. Used ones now on ebay are between 480 to 1000.

 

[Warren Faidley]

Those old HD 3CCD cameras are great for lightning. The good thing about nighttime lightning is that it interfaces with 4k to some degree because of the dark background, as opposed to daylight footage where going from HD to 4k is quite obvious. You do need to know the settings to limit noise. Daylight lightning really suffers from HD vs. 4k, but I still sell some of the old HD footage. The other option is to shoot multiple frames (time lapse) on a still camera. There are many ways to do this and you can search them out on the Internet. Unfortunately, it looks like time lapse motion and loses the lightning motion effect. I think in 3-5 years someone will figure out a way to design a global shutter camera that works well in low light, but when you think about the physics and engineering of trying to capture 30 or 60+ frames per second on a single sensor, it's obviously complicated and will require a massive amount of memory and processing speed, in addition to capturing detail in low light.

 

[Drew Terril]

From what I've heard, broadcast itself simply doesn't have the bandwidth for anything more than 1080p (and very few stations broadcast in 1080p, because then they have to give sub-channels)

With the FCC pushing narrowbanding across all commercially used frequencies at or above the VHF band (to include public safety and business communications) , I don't see this changing anytime soon. I think that actually played a huge role in the digital OTA mandate if memory serves. So I don't see broadcast going 4k anytime soon.

 

[Mark Blue]

Don't know if there's any way to repair a 720p CCD sensor with many defective pixels that auto-illuminate in low light making a cloudy sky look like it's full of stars at night.

Have you tried cleaning the sensor following guides found on Google? Maybe it has picked up dust or other particulates over the years.

 

[Marshall Stoner]

Have you tried cleaning the sensor following guides found on Google? Maybe it has picked up dust or other particulates over the years.

They're defective spots on the sensor (hot pixels), not dust. They appear in low light (high ISO + slow shutter) and get brighter as the sensor gets warm from recording video. I think it's a manufacturer defect that gets worse over time. It's probably been the same pixels broken from the beginning, they've just gotten brighter over the years. The problem is you don't notice them until you try shooting video at night. I think it's a common problem with CCD sensors because they get hotter and this somehow allows stray charge from the camera to leak backwards into pixels that aren't adequately protected (probably due to defective micro-connections). It's not a problem that affects most shots as it only occurs with a combination of high ISO, slow shutter, and heat, so there probably weren't ever enough complaints to force the sensor manufacturer to improve quality control.

 

[Marshall Stoner]

Certain types of lightning do better with rolling shutter than others. Slower lightning discharges like upward bolts and horizontal "anvil crawlers" can be slow enough that there is no change in brightness during the chip scan. Fast lightning phenomena, like cloud-to-ground bolts and some return strokes, are too fast for even the fastest-performing rolling shutter sensor to capture in a single frame. Occasionally you can get a CG with a long enough current pulse to fully expose during the sensor scan, but it's very rare. More distant bolts can also avoid splits easier since they take up less of the sensor area.

I've done a little research now. Partial scans are only a problem when the shutter speed is faster than the frame rate, which you can avoid if you have a manual shutter control option. The main problem is most DSLRs in video mode set the sensor read rate to match the frame rate. If your shutter speed setting matches the frame rate, you will capture most of the light from even the briefest flash. Capturing the flash isn't the problem. The problem is, unless you get extremely lucky, individual bright flashes that last for less than a single frame are split into pairs. One part of the flash appears on the bottom of one frame while the rest appears on the top of the next. Sometimes the faster frame-rate footage looks worse because you get darker horizontal black strips between very strong discrete flashes whereas a slower frame-rate would at least blend flash pairs together. Positive CGs and anvil crawlers turn out okay regardless of framerate because the main visible bolts seem to pulse at a frequency much slower than 1/30 second, exposing several frames at a time. You still get flicker lines from the dimmer but faster pulsing negative in-cloud channels that precede the main visible flashes though.

The real solution is to stick with a 30 fps framerate but allow the camera to read the sensor at the speed it would if you were shooting at 120 fps. Then you would at least have 75% chance of not splitting a quick flash, as the sensor read/reset cycle would only last 1/4 of a frame. CMOS sensors in DSLR cameras do seem to operate this way when shooting still photos. You don't get severe rolling shutter artifacts when shooting stills at 1/30 second shutter speed. Only in video mode does the camera not utilize the full speed potential of the sensor. I haven't had anyone explain why this is the case though. Simply reading the data as fast as the sensor is capable of doing wouldn't consume extra CPU power because there is a "rest" period where data is not being read from the sensor at all. Perhaps the sensor read function itself consumes more battery power at faster speeds, regardless of breaks. I know with GoPro the sensor read is matched to the frame-rate for the special image stabilization software algorithms it has.

Rolling shutter is a big enough problem that there is some progress taking place to develop global shutter CMOS chips. I think in 5 years or so we might start seeing some prosumer-level cameras with those improvements.

From what I understand, unlike CCD sensors, CMOS sensors have no way to instantaneously store pixel values. The CCD sensors could transfer all the data simultaneously because they used an analog method to transfer the charge to an unexposed buffer, which the CPU then read digitally behind the scenes. A CPU has to read individual pixels in binary one at a time. At the present moment chips on the sensor just aren't capable of reading an entire 20MP analog array into digital memory in parallel. It's the conversion from analog to digital that takes time, and if analog data isn't read directly from the sensor it must be stored somewhere in analog first. Apparently the extra analog storage step is what degrades image quality in low light (and also generates extra heat).

 

[Jack R. Jester]

You know that old tube based camcorders did not have the same issues with lightning strikes that modern CCD and CMOS sensors have. This is because they have something called charge storage in their tubes, this allows charge to build up on the photo-cathode before it is scanned by the electron beam. The downside to this is that old tubes like this are susceptible to burn-in and ghosting, but still interesting none the less.

I have recorded some lightning strikes with my 1981 RCA CC011 Newvicon Tube Camera, and they have a cool effect. I should post some pictures some time.

 

[Todd Lemery]

I have nothing to add. Just reading these posts makes me realize once again how little I know about cameras. Either that, or you all are geniuses!