Blog

My Backup Strategy – From Drives to LTO Tape

And so it starts – the LTO 2018 set… from G-Technology Shuttle XLs to LTO tape the never-ending data backup continues. Hits 8K RED Digital Cinemausers and anyone shooting 4K RAW as well I’m sure.

Been lazy going to cold storage for almost a year but with close to 250 Terabytes of drives online in my office w only 20TB of the critical files on the cloud via Dropbox you gotta be safe.

I have my drives organized as:
ONLINE – In Progress (Fast SSDs)
ONLINE – Projects less than 1 year old
OFFLINE – actually online but 1+ year old and can be deleted at will
ARCHIVED – for my most common stock request and also deletable. But not worth going to tape for a commonly requested asset…

Obviously everything is backed up to LTO-6 Tape ASAP in the initial phase (so it’s actually ALL archived – but this naming convention has worked for me to know where to put things in relative levels of priority.)

Critical in progress files are on DropBox

Trimmed R3Ds and Proxies SELECTS are shipped toNimiaa for stock sales and backed up there too. (And backed up to tape here too eventually.)

And ALL project files are on DropBox – I currently have 22TB on the cloud w their business plan and am super happy to have access to them on any computer or iOS device instantly from anywhere (easily downloadable or shareable.).DaVinci ResolveeFinal Cut Pro XAdobe Photoshop Lightroom Adobe Premiere Proo

My DropBox copies files to the cloud, a computer w Studio XL (64TB) in LA and a third copy in NYC on another Studio XL (64TB)

I already have 100+ LTO drives and am starting my 2018 set this week – my version of “spring cleaning.” Used to have a server w 14-tape LTO drive back in the day but now things have gotten so much simpler and more compact (and affordable!)

This is the least fun part of the gig but one of the most critical. And it feels good when it’s done right – one should NEVER delete a file … Just in case ONE request comes in for a sale it usually pays for the drive …

Wonder if anyone knows of a service that will economically make dupes of the LTO tapes …

What’s your workflow? We don’t all have Full Time DITs …

 

So is the 18-Core iMac Pro Worth It?

So is Apple’s 18-Core iMacPro worth it?

In short: YES – if your software can take advantage of it. That’s the KEY.

But just as key: the 10-core iMac is really at the sweet spot of what the vast majority of users will likely ever need. It’s a little beast!
Final Cut Pro X is clearly designed to take FULL advantage of the 18 cores, whereas DaVinci Resolve is several times slower on similar exports (as much as 5 times slower) and unfortunately Adobe Premiere is currently not even on the same racetrack…

The results continue to be seriously impressive especially for those transcoding high resolution and RAW coded footage or using processor intensive VR footage for example.

The best way I can illustrate this is by sharing the following test:

1. I started off w/ 62GB RED Digital Cinema R3D 6 K files totaling 8 min 21 seconds of aerial footage.

2. I loaded the footage into the latest Final Cut Pro X version and asked it to generate: Optimized Video + Proxies.

Here are the results of how much time that took:

1. iMacPro 18-Core: 25 min 17 seconds to generate both Proxy + Optimized Video

2 I then exported the 8 min+ timeline as a 4K ProRes444 file in 8 min 20 seconds … that’s 1:1 on RED footage (nice!!!)

At this point:

– The 10-core iMac Pro was anywhere from 10-80% slower depending on how the software took advantage of the 8 additional cores and the process.

With software such as Adobe Lightroom the difference was negligible- whereas w FCPX transcoding it was as much as 80% faster due to the 8 additional cores.

– The top-of-the line Mac Pro was still transcoding (generating the initial first of three steps: generating proxy and optimized media for the first of the 2 clips .. it was at 96% of the first clip…)

– The top-of-the line MacBook Pro 15″ took:
1 hour and 11 minutes to render the same files and 27 min to output the ProRes444 output.

– And for of those who are still holding onto the old 15″ MacBook Pro w/ Magsafe… well … it took well over 6 hours to do the same 3 steps.

Here’s where it gets REALLY interesting though:

Comparing dif

ferent pieces of software that are specifically designed to take advantage of the CPU / GPU etc:

I exported a recent shoot via DaVinci Resolve my usual go-to NLE (editing) app that I use because I love to grade in it … and compared the results to FinalCutPro …

I exported a nearly 3 hour timeline shot on the RED Helium 8K at 60 fps to a ProRes444 4K file:

Thru Resolve the export took 5 hours 22 minutes …

Thru the 18-core iMacPro it took … 1 hour 9 minutes!

That’s significant.

In fact the round-tripping between Resolve and Final Cut Pro is so effortless that I’m currently going thru my second round of FCP X tutorials…

Speaking of FCPX – the playback of the 8K footage and scrubbing of it is so utterly effortless that it’s very hard to recommend anything else right now if you’re editing RED R3Ds on a Mac.

The iMacPro 18-core plays the footage in DaVinci Resolve at 5.8 fps whereas the 10-core plays the same footage at 3.8 fps. More importantly the footage (while rezed-down) in FCP X looks sharp on the 18 core prior to any proxy generation … is visibly lightly “blurry” looking on the 10 core (as it’s being rendered at a lower quality.)

So what’s the skinny – is the 18 core vs the 10 core worth $3K more? The answer is simple: what’s your time worth? What format are you shooting (and can it take advantage of these cores) and more importantly: Can the software you’re using take full advantage of the 18 core?

If you’re using FCPX and you transcode RAW 4K-8K footage, let alone RAW RED R3D footage in Final Cut Pro – it’s a TOTAL no brainer…

If you’re purely a Davinci Resolve user – then you’ll need to wait a bit for them to take advantage of those additional 8 cores – and I’m sure they will. Currently you get an error message every time you launch Resolve, stating that it can’t take advantage of the 18-cores with RED’s RAW decoder ?

You’ll notice I haven’t mentioned Adobe Premiere Proe Pro results in these tests … well, you’ve got to realize that all of these pieces of software improve upon one-another with leaps and bounds… and given the amount of friends I have at AdobeAdobe I’m simply going to say that as of this moment: their photo and video apps are on the valley side of the equation … not the peak.

If you use Adobe Photoshop or Adobe Photoshop Lightroom user for example: the software is simply not designed to take full advantage of either the 10 or 18 core machines just yet. In fact the software seems to be taking advantage of less than 20% of the CPU’s power.

Disclaimer: as with my last post, for those on the very highest end, a custom built PC is still likely to give you the best bang for your buck in terms of pure performance. That being said, 50% of those who’ve custom-built one swear by them … whereas the other 50% swear AT them and are utterly frustrated w them and Windows … it’s a mixed bag and larger depends on your Love (or opposite) of Windows.

Just 2 years ago Adobe’s Premiere and notably Adobe’s Media Encoder was TEARNG thru R3D and H.264 footage making it a – if not THE – GO TO NLE – or editing software.

At the end of the day: we’re going to have to become proficient w ALL NLEs – Final Cut Pro. X, Adobe Premiere, Davinci Resolve and … well no: Not Sony Vegas …sorry. Is that still around?

Point is: it’s likely going to ebb and flow and all of these pieces of software have CLEAR advantages. For exaMark Toia Toia is teaching his son to become proficient in all of them. He’s editing 20-30 day commercial shoots (shot in 6K-8K RED) on his MacBook Pro on the road as well as a feature film in Final Cut Pro X … and he is proficient in ALL NLEs. Says a lot – no?

For those who use software that routinely use high end footage that pushes hardware to its limits – the 18-core is AWESOME.

For most people: the iMacPro 10-core is a really, really, really SWEET SPOT. That’s also a key takeaway – I’d venture to bet that most people will be quite happy with the base model 10-core iMacPro for their Video and Still photography needs – that is 2X to 3X faster than any other Mac out there today.

 

Apple’s new iMacPro has an impressive 200%-300% speed bump.

Last week I received a pelican case from Apple with a very special Mac inside of it…  It was an iMac Pro configured as a 10 Core 3GHZ Intel Xeon W, 2TB SSD, 128 GB RAM, Vega 64 Radeon.

After unpacking the (to be expected) beautiful Space Grey hardware, keyboard, mouse and trackpad, it was time to get down to the brass tacks, I had ONE question that I needed to answer:  how fast is this thing, and how much time will it save me in my everyday imaging tasks?

I found a very consistent set of results:  a 2X to 3X boost in speed (relative to my current iMac and MacBook Pro 15”) a noticeable leap from most generational jumps that are generally ten times smaller.

Whether you’re editing 8K RED video, H.264 4K Drone footage, 6K 3D VR content or 50 Megapixel RAW stills – you can expect a 200-300% increase in performance in almost every industry leading software with the iMac Pro.   

I’ve seldom seen a jump this dramatic before on any new generation of Macs – 20%-30% speed increases are the norm … NOT 200%-300% increases.  That’s SIGNIFICANT.  (Feel free to jump straight to the test results below if you have a short attention span or are a numbers person..)

Basically, if you’re debating whether or not to purchase the iMac Pro you should ask yourself just one question:  how much is your time worth to you?   Would you rather be waiting for images and video to render or export, or do you want to go back out and shoot as quickly as possible?   For me the answer is easy:  when compared to the previous generation iMac or the current top of the line 15” MacBook Pro, the iMacPRO tears through footage and images, allowing me to spend less time behind a computer, and more time shooting.

After just under a week I can tell you that I won’t be using any other mac anytime soon.   

I’ll detail the result below for every piece of software that I ran these tests through – because at the end of the day it’s all about the numbers when you’re shooting 1-5 Terabytes per day of 8K RED footage,  or shooting 7,000 fifty Megapixel stills on a Canon 5DsR, let alone churning through 8K VR footage or playing back 4K MavicPRO footage.   It’s safe to say we’re shooting and processing more an more data than we could ever have imagined just a few years ago – and one’s time behind a computer can be a TRUE pain point.

In short – this is a KILLER machine for any serious photographer, filmmaker, or VR producer.  Period.   

I can see a lot of people taking advantage of the nearly 3,000 MB/S transfer speeds to the internal SSD, let alone the four Thunderbolt 3 ports to go thru footage much less painfully.    In fact most of the drives cannot come close to taking advantage of the transfer speeds – YET.   However when you copy multiple media at once to a fast raided drive you’ll quickly see the benefit of this machine.   DITs (Techs who copy cards on still or film sets) will love this machine for its portability and speed (and easy to swap base w/ VESA mounts that can transfer back and forth much more easily than the previous generation iMac.)

Several important disclaimers:

  1. I used the latest MacOS High Sierra 10.13.2 on this an a lot of the software I used has not yet been optimized for the new OS let alone this machine.
  2. I did not use a MacPRO because I couldn’t get one in time, and because I generally don’t use one because the Thunderbolt 2 ports aren’t fast enough for what I do.   I also used my current iMac – an iMac 5K, 4GHZ Intel Core i7 w 32GB RAM and an AMD R9 4GB and 500GB fusion drive, as well as a brand new 15” MacBook Pro with 3.1GHZ Intel i7, 16GB RAM, Radeon Pro 560 w/ 4096 MB and 1TB SSD.
  3. I did the best to reproduce all of these tests using the exact same G-Tech and SanDisk 1TB SSD TB3 drives with the exact same data and timing it w/ a timer on the same builds of software.
  4. Needless to say people have been building monster PCs with liquid cooling etc – but I have been resisting that other pain point.   These custom built-machines are an entirely different experience and level of user and no doubt would do better on footage such as the RED 8K footage for example that require extremely intense graphics card and CPU usage.    Of note:   Apple has made several statements that they are committed to the PRO Market … I don’t think this is their last PRO machine…
  5. Until this machine is officially released, I don’t know the exact price point – I will update that as soon as it’s available.

In these tests I used:   Final Cut Pro X,  Adobe Premiere,  Adobe Media Encoder,  DaVinci Resolve, Insta360 Stitcher, BlackMagic Speed Test and Adobe Lightroom Classic.

I could go on an on about my thoughts on the machine but I think it’s time to cut to the chase and give you the empirical NUMERICAL results:

Final Cut Pro X:

I imported 10.96 GB of DJI Mavic PRO 4K, H.264, 23.98 fps footage, a total duration of 24 minutes 16 seconds of footage transcoded to Proxy ProRes422.

All 3 machines created low resolution Proxy footage in under 8 minutes which was impressive.

I then had the software render ProRes422 Optimized media from the H.264 source media:

iMac Pro – 7minutes 56 seconds

iMac – 15 minutes 47 seconds

MacBookPro 15” – 19 minutes 55 seconds

Black Magic Resolve:

In Black Magic’s DaVinci Resolve, I imported 1 hour and 17 minutes of RED 8K Helium Full Frame footage shot at 60 fps and exported it to 4K ProRes 422 footage, here are the resulting export times:

iMac Pro – 2 hours 6 minutes

iMac – 7 hours 15 minutes

MacBookPro 15” – 6 hours 5 min.

Insta360 3D VR:

I processed 6K Stereo (3D) VR Insta360 PRO footage through their Insta360 Stitcher software, a 56 second clip, here were the export / processing times:

iMacPRO – 5 minutes 55 seconds

iMac – 11 minutes 09 seconds

MacBookPro 15” – 32 minutes

Adobe Lightroom Classic

I imported 788 still CR2 (RAW) Canon 5DsR 50 Megapixel images or 50.74GB of images and then…

I had LR build 1:1 Previews

iMac Pro – 25 minutes 26 seconds

iMac – 50 minutes 45 seconds

MacBookPro 15” –   46 minutes 40 seconds

I applied a look (color correction in the Develop module)  and Exported 788 JPEG files at full resolution or 14.79 GB of JPEGs averaging 9 MB each

iMac Pro -31 Minutes

iMac – 1 hour 17 Minutes

MacBookPro 15” –  1 hour 4 minutes

Black Magic Speed Test:

I ran Black Magic’s software to test the speed of all 3 internal drives and found the following average speeds:

iMac Pro – Write 2996MB/s / Read – 2450 MB/s  (2TB SSD)

iMac  – Write 664MB/s / Read – 720 MB/s (500GB Fusion Drive)

MacBookPro 15” – Write 1743 MB/s / Read – 1400 MB/s (1TB SSD)

RED Cine-X:

I used RED’s conversion Cine-X Software to test playback with a RED (RAW) 8K, Full frame, 4 minute 1 second file (59.68GB) at 60 fps:

  

At 1/2 resolution I could play back (before hitting memory / CPU & GPU limits)

iMac Pro – 14 seconds of smooth playback before buffering  (if I let the computer process and render the file, it was able to render the file using the 128GB of memory and 10 cores in minutes and and playback in full screen 75% of the 60 fps clip.)

iMac – 4 seconds of smooth playback before buffering

MacBookPro 15” -1 second of smooth playback before buffering

At 1/4 res I found the perfect balance:

iMac Pro – I was able to play the clip back at 1/4 res for a full 4 minutes before buffering – the first time I’ve been able to do that on a Mac at 8K in RED Cine-X – and I could truly judge the movement, sharpness and overall detail of the clip in full screen mode

iMac –  1 minute 25 seconds w stuttering playback before buffering

MacBookPro 15” -8 seconds of smooth playback before buffering

Adobe Premiere

I had issues w/ High Sierra and couldn’t work reliably with RED footage.  However I was able to play the Mavic 4K footage reliably on all 3 machines full screen mode at 1/2 resolution.   Both the MacBook Pro 15” and the iMac started to stutter (with any movement in frame or when panning / tilting) when played at Full 4K resolution – noticeably so on the MacBook Pro.)   The iMac did well with playback at full resolution, however I couldn’t scrub the footage at all.   The iMac Pro was silky smooth at full resolution and scrubbing was nearly flawless.

Overall this is a wonderful machine that will satisfy all users out there – with the exception of the 0.1% of users who work in the 8K RAW RED world for example and want to transcode at 1:1 or faster with purpose-built machines.    If you’re shooting 4K or pretty much any still camera available today, this Mac is unlikely to let you down.   I know it’s tackled everything I’ve thrown at it and it’s the first machine that can keep up with me in an still edit of thousands of images or hundreds of minutes of video footage.