Storage requirements for the X5R shooting 4k raw

[B-Scene Films]

My wife shoots about 70% of her work in raw. After the job is done, she may extract some shots for personal use and if she does she will archive that. But, 90 days following content delivery she nukes all the raw footage as well as pretty much everything else associated to the project except a fairly compressed version of the final delivery.

She uses 2 32TB arrays for her work and archives to tape but the volume of data that she will generate for a 30 second spot can exceed 30TB. She has no way to retain that volume of data over time.

 

[jb.]

If you're shooting 30-second spots and you know the footage will never be repurposed, then by all means dump the RAW... but that's a relatively limited market. Most are going to want to be able to repurpose the footage they have if they can... which will also mean returning to the RAW and applying a different LUT to it for the new usage.

Your wife also seems like she has her RAW workflow pretty well nailed down. My only point is you need to know what you're getting into shooting RAW. Even the transcodes can take forever unless you're running a pretty beefy ($5-10k) computer with plenty of fast storage.

The amount that goes onto the card is only a TEENY TINY portion of the impact of shooting RAW on your workflow.

 

[damoncooper]

If you shoot in RAW, you're going to want to archive in RAW. That's the WHOLE POINT of shooting in RAW... except in cases like this where DJI has decided to shove you into a RAW workflow if you want better than GoPro-level bitrate.

Guys with D810s don't shoot NEF just to come back, convert to a JPEG and throw the NEF away. This is EXACTLY the same thing.

If you dispose of the RAW video immediately after transcode, you're handicapping yourself if you ever want to go back and apply a different LUT or re-grade the footage to match another camera, etc. It's not wise to throw out your highest-fidelity, first-generation version of an image.

I understand your workflow but for my purposes saving ProRes works for me.

 

[jb.]

I understand your workflow but for my purposes saving ProRes works for me.

Right, that's fine... but most people shooting RAW video want to archive the RAW video (at least during the project and for a while after). Doing an immediate transcode to a lesser format and then throwing away the RAW circumvents the whole point of RAW. At some point a lot of commercial shooters are going to either completely offline or destroy the RAW originals, but they'll still be storing them for some time. I have projects that have many many hours of uncut video in them... that's one of the reasons I have no interest in a RAW workflow. Even when I've shot with a Blackmagic, we've used the ProRes recording options unless we needed RAW for a beauty shot or two.

Me personally? I'd like to skip the RAW step completely and get 150-200Mb AVC on the card. Easier on the storage, easier on the card space, easier on the computer.

 

[jb.]

Other than real image quality (which I'm sure will be awesome on the X5R) my biggest question around the X5R is how well CineLight is going to work performance-wise. RAW transcoding is pretty expensive (on computing resources)... even for a simple LUT-and-transcode.

 

[damoncooper]

Right, that's fine... but most people shooting RAW video want to archive the RAW video (at least during the project and for a while after). Doing an immediate transcode to a lesser format circumvents the whole point of RAW. At some point a lot of commercial shooters are going to either completely offline or destroy the RAW originals, but they'll still be storing them for some time.

Me personally? I'd like to skip the RAW step completely and get 150-200Mb AVC on the card. Easier on the storage, easier on the card space, easier on the computer.

Couldn't agree with you more on that last statement

 

[jb.]

Couldn't agree with you more on that last statement

The lack of an intermediate recording option (even if it was on the RAW camera) is puzzling to me. Make your X5R record RAW / 200Mb AVC / 60Mb AVC if you're so concerned about the X5 cannibalizing sales of the fancier camera...

I'm guessing there's a hardware limit there that's present in both cameras. Both of them start with the same RAW data dump off of the CMOS and they both have a processing pipeline to convert that RAW data into 60Mb AVC... the X5R just has the ability to write the data stream directly out to CinemaDNG too (which is MORE bandwidth intensive but LESS processor intensive than the 60Mb encoding).

I cringe when I say this, but I'm guessing the CPU that's processing the data stream and encoding the AVC video can't do any higher than 60Mb... which means the hopes of a higher-bitrate codec through firmware are pipe dreams.

 

[purezerg]

having did a 4 min video with D810 and i1, the amount of remaining data already hitting 100Gb.

I hope that my 8x6Tb NAS is going to pull thru with x5r, that said, I still cant decide to get x5 or x5r.

as for raws, yeah, I dump them after 6 months. would probably dump it to a BR.

what really concerns me is the processing power. the x5r is starting to make my 2tb SSD on my laptop cringe.

 

[ianwood]

I cringe when I say this, but I'm guessing the CPU that's processing the data stream and encoding the AVC video can't do any higher than 60Mb... which means the hopes of a higher-bitrate codec through firmware are pipe dreams.

It's all but confirmed they're using the same Ambarella A9 SOC processor as on the X3 and GoPro Hero 4 which is limited to ~60Mbps. They could switch at some point to the new H1 SOC but it's not at all clear if that will provide a significant jump in data processing capabilities.

 

[purezerg]

or maybe NVIDIA x1 or even intel 5Y71. At 4.5w should provide ample rates.

 

[x5yo]

Just to confirm, each dng is around 6mb on average per frame based on UHD (3840x2160) @ 30 FPS...

So.

30 frames per second = 6mb x 30=180mb per SECOND of footage at above specs.

or

900 mb every 5 seconds.
1.8gb every 10 seconds.

Do the math from here on in....

I'm not a fan of 24 FPS, but any increase in the per frame quality/filesize is nullified by the lower frame rate.

I have been through all my X5R footage and 6mb seems to the be the average frame size.