Final Cut Pro 10.4.1: ProRes RAW and captions

Thursday, 05 April 2018

Apple have announced the next version of Final Cut Pro X will have two features for high-end workflows. The free update will include ProRes RAW for better footage acquisition and flexible closed captioning for media distribution. It will be available from Monday April 9th from the Mac App Store.

Updated with more information on exporting using roles and Compressor 4.4.1

ProRes RAW - for Final Cut Pro 10.4.1 only

ProRes RAW provides the real-time performance and storage convenience of ProRes 442HQ and 4444 with the postproduction flexibility of camera RAW. The new proposition from Apple is effectively: “Add any camera you have into a RED-like RAW workflow with an Atomos recorder and Apple professional video applications.” This can be done now because Macs are now fast enough to work with multiple layers of camera source media in real time - instead of extracting the information from the source when mastering in a grading application.

Whereas the current family of ProRes codecs is designed for all stages of video production, Apple ProRes RAW and Apple ProRes RAW HQ are designed for acquisition. When ProRes RAW is used in Final Cut Pro 10.4.1, the output for distribution is ProRes 422 HQ or ProRes 4444 (although ProRes RAW would be a good codec for archiving ‘original camera negative’).

A camera sensor is a grid of photosites that can each only record a single red, green or blue brightness value. Footage for postproduction is made of a series of images where each pixel in the grid is made up of at brightness values for red, green and blue. At some point in the workflow, the RGB values for each pixel need to be interpolated from the brightness values of adjacent red, green and blue photosites.

In this case the RGB value of the single pixel in the video frame is based on the red brightness at its location plus green and blue values interpolated from the brightness values recorded at adjacent photosites.

ProRes RAW encodes the information captured by individual camera photosites without extrapolating RGB information for every position in the sensor array. At the point of being used in a timeline, Final Cut Pro creates a grid of RGB values by interpolating the brightness values recorded at individual photosites.

The ProRes RAW advantage is that there is more processing power in a Mac running Final Cut Pro than there is in a camera recording images on location. More processing power means the algorithm that is doing the interpolation can be more advanced. It can also be modified if needed. Cameras must bake in their pixel interpolation into the footage they record.

RAW flexibility at ProRes data rates

In practical terms, ProRes RAW gives REDCODE RAW quality at ProRes data rates. For 1 stream or REDCODE RAW 5:1 or 3 streams of Canon Cinema RAW Light, a Mac running Final Cut Pro 10.4.1 will be able to play back 7 streams of Apple ProRes RAW HQ or 8 streams of Apple ProRes RAW. Also Final Cut Pro is able to render and export ProRes RAW HQ 5-6 times faster than REDCODE RAW 3:1.

In practice you would use ProRes RAW where you used to use ProRes 422 HQ and ProRes RAW HQ where you used ProRes 4444. Because of how each RAW frame can vary, the data rates vary much more with ProRes RAW than they do with standard ProRes.

For more information on storage requirements and data rates for ProRes RAW, read the new Apple White Paper.

There will initially there will be two ways to record Apple ProRes RAW: using the Sumo 19 or Shogun Inferno on-camera recorders from Atomos or a 5K full frame Super35 Zenmuse X7 camera mounted on a DJI Inspire 2 drone.

Atomos’ ProRes RAW page.

Interesting that this new ProRes family initially only works with Apple video applications: Final Cut Pro 10.4.1, Motion 5.4.1 and Compressor 4.4.1. Could this be the start of Apple favouring their own post applications over other macOS tools.

Closed Captions - for TV, streaming services and apps

The other big new feature of Final Cut Pro 10.4.1 and Compressor 4.4.1 is the ability to import, create, edit and export closed caption text. Closed captions are the text that optionally appears at playback - be it in the Netflix application running on a set-top box, on broadcast TV, at special subtitled screenings in cinemas or in the YouTube iOS app.

Of course captioning should be done when picture and sound have been locked, but Apple have done a lot to implement this feature so it works well based on continuous changes made towards the end of postproduction.

The flexibility of Final Cut Pro X video roles means that captions in multiple formats and in multiple languages can be edited and exported from the same timeline.

Individual captions can be associated with video or audio clips in the primary storyline. This means that when these clips are edited and re-ordered, the captions move with their associated clip.

The big news is that captions can also be connected audio and video clips. That means an individual caption can be connected to the specific piece of audio that it is transcribing. So although you should start captioning your production once there is a picture and sound lock, you can start the captioning process earlier. Timeline changes made to clips in the primary storyline and connected clips will be reflected in their associated captions.

Final Cut Pro 10.4.1 works with closed captions in one of two formats: CEA-608 and ITT.

CEA-608 is the long-standing closed caption format used in US broadcast TV and on DVDs worldwide. ‘iTunes Timed Text’ captions are used in iTunes video bundles for movies and TV shows that can be bought or streamed from Apple. They are also used by Amazon Prime Video and YouTube

 

Captions in Final Cut Pro 10.4.1

Captions can be imported as files generated by external services or applications (using the File > Import > Captions command). .scc and .itt formats are recognised for now.

Captions can be extracted from video files with encoded captions. Add the clip to the timeline and use the Edit > Captions > Extract Captions command.

Captions in compound clips or in multcam angles can be extracted and added to their parent timeline (Edit > Captions > Extract Captions).

Add caption to the active language subrole at the playhead location using the Add Caption command (Option-C [or Control-Option-C if the caption editor is open - this means you can add a caption while editing another caption]).

An indvidual caption is shown in a language sub-role lane of the captions lanes of the timeline. You choose which captions are visible in the viewer by activating the caption video subrole in the timeline index.

To open a selected caption in the caption editor, double-click it or choose the Edit Caption command (Shift-Control-C).

Captions can be edited in a floating caption window (to use timeline navigation shortcuts such as J, K, L, I and O without entering them into the caption editor, also hold down the Control key - Control-J, Control-K etc.):

Captions are automatically checked, errors are flagged in the timeline index (you can choose to only show errors)…

or in the timeline. In this example, captions overlap, which most caption formats do not allow:

This problem can be fixed with the Edit > Captions > Resolve Overlaps command. 

For more on fixing problems with captions that would mean they would not be valid when played back, there is an Apple support document on Final Cut Pro X Caption Validation.

Once you have timed the captions for one language, to start work on another language, you can duplicate them as a new language. Select the captions you want to work with, then choose the Edit > Captions > Duplicate Captions to New Language command.

Each caption format has various formatting options. If you are happy with the style of a caption, you can use the Caption Inspector to Save Style as Default and move to another caption to Apply Default Style. 

CEA-608 captions can have more than one field on screen at once. You can use the Inspector to add and format up to three extra fields per caption:

If you have a long caption you can split in into individual captions using the Edit > Captions > Split Captions command (Control-Option-Command-C)

Conversely, you can combine consecutive captions into one longer caption using the Edit > Captions > Join Captions command. 

By default, captions are connected to the primary storyline. To connect a caption to a connected clip that overlaps the caption in the timeline:

  1. Select the caption
  2. Option-command click the connected clip you want the caption to be associated with

Captions are not supported when sharing to Facebook. If you have captions in your project, they will not appear when you share the project to Facebook.

If you want to export just the captions from a timeline, use the File > Export Captions command.

New Roles tab when sharing

To make preparing productions for distribution or for collaboration easer, Final Cut Pro 10.4.1 has a new Roles tab in the Share dialog box:

To make preparing to export easier, Final Cut will respect which roles and subroles are on or off in the timeline when sharing. 

In the Roles tab you can

  • Add an audio track to the export file
  • Choose an audio channel format for a track
  • Combine roles in a track
  • Remove roles from a track
  • Add captions to the export
  • Save a preset (Click the Roles as pop-up menu and choose Save As in the Presets section)

When you share a Master File as Separate Files, in the Roles tab you can

  • Add a track or file to the export
  • Combine roles in an output track or file
  • Remove roles from a track or file
  • Choose an audio channel format for a track (Mono, Stereo, or Surround)
  • Remove a track or file from the export
  • Save a preset (Click the Roles as pop-up menu and choose Save As in the Presets section)

It looks like you can't yet add a video and audio file and then choose which video and audio roles you want it to include. These separate files are either video or audio. 

Compressor 4.4.1

The next version of Compressor has gained some features too:

  • Import, edit and embed closed captions and subtitles (but not author captions from scratch)
  • Add optional voice narration (descriptive audio tracks) to iTunes Store packages
  • Add metadata from a XML property list file 
  • Use a movie (with optional audio) as the background to Blu-ray and DVD menus

Captions in Compressor

For those who need to add captions to finished videos, instead of using a full NLE, they can use Apple’s video distribution preparation application. 

Built-in settings and destinations support captions: “Apple Devices (in both the H.264 and HEVC codecs), ProRes, Publish to YouTube, Create DVD, and other settings that use the QuickTime Movie, MPEG-2, and MPEG-4 formats.” Note that captions are not supported when sharing to Facebook.

Standard Compressor jobs can only import a single .scc (CEA-608) or .itt (iTunes Timed Text) file. If an imported video file already contains embedded CEA-608 closed captions, Compressor adds the caption data to the job.

You can edit each caption’s text, appearance, position, animation style and timing. You can also add new captions at the time of your choice.

If you have multiple captions selected in the captions palette, you can adjust their start or end times by frames, seconds or minutes at the same time.

YouTube and Vimeo support CEA-608 captions that Compressor encodes into videos. If you use iTT subtitles, Compressor will generate a separate .itt file and will automatically upload it if you use the YouTube or Vimeo presets.

Metadata

Compressor has been able to add metadata from QuickTime movie files to jobs. Version 4.4.1 can add metadata stored in XML property list files in the following metadata categories:

  • com.apple.quicktime.album
  • com.apple.quicktime.artist
  • com.apple.quicktime.author
  • com.apple.quicktime.comment
  • com.apple.quicktime.copyright
  • com.apple.quicktime.description
  • com.apple.quicktime.displayname
  • com.apple.quicktime.information
  • com.apple.quicktime.keywords
  • com.apple.quicktime.producer
  • com.apple.quicktime.software

Using a standard set of property lists when exporting batches means that other tools that can read this metadata can make decisions based on property values (such as specific keywords).

Final Cut Pro 10.4.1 and Compressor 4.4.1 - Updates for professional workflows

Although Apple don't often let themselves be guided by external trade events, this is a rare update that seems to be prompted by NAB happening in Las Vegas next weekend. I'm not sure how many naysayers will be swayed by the inclusion of closed captions. ProRes RAW however shows that Apple is serious about trying to attract more high-end workflows to the Mac, and Final Cut Pro X specifically: “Don’t worry about good cameras with bad codecs, we have the acquisition format you need for HDR workflows. Available now in Apple pro video applications only.”