Skip to main content

Track Transformation

When transforming media, there are multiple thing that can be done for each audio or video track:

  • Copying the track without re-encoding
  • Re-encoding the track into a different codec
  • Removing the track

@remotion/webcodecs allows you to decide for each track what to do with it.

Using the defaults​

The minimum amount of configuration is to only specify a src and an output container.

Using the default codecs
tsx
import {convertMedia} from '@remotion/webcodecs';
 
await convertMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  container: 'webm',
});
Using the default codecs
tsx
import {convertMedia} from '@remotion/webcodecs';
 
await convertMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  container: 'webm',
});

The default codecs are defined by getDefaultAudioCodec() and getDefaultVideoCodec().

Choosing codecs​

You can use the videoCodec and audioCodec options to transform all tracks to the codecs you specify.

Choosing video and audio codecs
tsx
import {convertMedia} from '@remotion/webcodecs';
 
await convertMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  container: 'webm',
  videoCodec: 'vp8',
  audioCodec: 'opus',
});
Choosing video and audio codecs
tsx
import {convertMedia} from '@remotion/webcodecs';
 
await convertMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  container: 'webm',
  videoCodec: 'vp8',
  audioCodec: 'opus',
});

Handle each track individually​

With the onVideoTrack and onAudioTrack callbacks, you can decide for each track what to do with it.

Using the onVideoTrack() API
tsx
import {convertMedia} from '@remotion/webcodecs';
 
await convertMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  container: 'webm',
  audioCodec: 'opus',
  onVideoTrack: ({track}) => {
    if (track.codecWithoutConfig === 'vp8') {
      return {type: 'copy'};
    }
 
    return {type: 'reencode', videoCodec: 'vp8'};
  },
});
Using the onVideoTrack() API
tsx
import {convertMedia} from '@remotion/webcodecs';
 
await convertMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  container: 'webm',
  audioCodec: 'opus',
  onVideoTrack: ({track}) => {
    if (track.codecWithoutConfig === 'vp8') {
      return {type: 'copy'};
    }
 
    return {type: 'reencode', videoCodec: 'vp8'};
  },
});

onVideoTrack and onAudioTrack have a higher priority than videoCodec and audioCodec.

The options for video codecs are:

  • {"type": "copy"} - Copy the track without re-encoding
  • {"type": "reencode", "videoCodec": ConvertMediaVideoCodec} - Re-encode the track into the specified codec
  • {"type": "drop"} - Remove the track from the output
  • {"type": "fail"} - Fail and stop the conversion process

The options for audio codecs are:

  • {"type": "copy"} - Copy the track without re-encoding
  • {"type": "reencode", "audioCodec": ConvertMediaAudioCodec; bitrate: number} - Re-encode the track into the specified codec. The suggested bitrate to use is 128000.
  • {"type": "drop"} - Remove the track from the output
  • {"type": "fail"} - Fail and stop the conversion process

The enums ConvertMediaVideoCodec and ConvertMediaAudioCodec can be imported from @remotion/webcodecs.

Checking if a track can be copied​

To check if it is possible to return {"type": "copy"}, you can use canCopyTrack property you get from onVideoTrack.

Using the canCopyVideoTrack() API
tsx
import {convertMedia} from '@remotion/webcodecs';
 
await convertMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  container: 'webm',
  audioCodec: 'opus',
  onVideoTrack: ({track, inputContainer, outputContainer, canCopyTrack}) => {
    if (canCopyTrack) {
      return {type: 'copy'};
    }
 
    return {type: 'reencode', videoCodec: 'vp8'};
  },
});
Using the canCopyVideoTrack() API
tsx
import {convertMedia} from '@remotion/webcodecs';
 
await convertMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  container: 'webm',
  audioCodec: 'opus',
  onVideoTrack: ({track, inputContainer, outputContainer, canCopyTrack}) => {
    if (canCopyTrack) {
      return {type: 'copy'};
    }
 
    return {type: 'reencode', videoCodec: 'vp8'};
  },
});

To check outside of a onVideoTrack handler, you can also use the the canCopyVideoTrack() and canCopyAudioTrack() APIs

Checking if a track can be re-encoded​

To check if it is possible to return {"type": "reencode"}, you can use the canReencodeVideoTrack() and canReencodeAudioTrack() APIs.

Using the canReencodeVideoTrack() API
tsx
import {convertMedia, canReencodeVideoTrack} from '@remotion/webcodecs';
 
await convertMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  container: 'webm',
  audioCodec: 'opus',
  onVideoTrack: async ({track}) => {
    const canReencode = await canReencodeVideoTrack({
      videoCodec: 'vp8',
      track,
    });
 
    if (canReencode) {
      return {type: 'reencode', videoCodec: 'vp8'};
    }
 
    return {type: 'drop'};
  },
});
Using the canReencodeVideoTrack() API
tsx
import {convertMedia, canReencodeVideoTrack} from '@remotion/webcodecs';
 
await convertMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  container: 'webm',
  audioCodec: 'opus',
  onVideoTrack: async ({track}) => {
    const canReencode = await canReencodeVideoTrack({
      videoCodec: 'vp8',
      track,
    });
 
    if (canReencode) {
      return {type: 'reencode', videoCodec: 'vp8'};
    }
 
    return {type: 'drop'};
  },
});

Asynchronously determining action​

The onAudioTrack and onVideoTrack callbacks can be asynchronous.
While the operations are unresolved, reading of the input fill is paused.

Decide behavior upfront​

If you want to display a UI letting the user choose codec settings before the conversion starts, you can do so.

Use parseMedia() to get video and audio tracks respectively:

Using parseMedia() to get tracks upfront.
tsx
import {parseMedia} from '@remotion/media-parser';
 
const {videoTracks, audioTracks} = await parseMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  fields: {
    tracks: true,
  },
});
Using parseMedia() to get tracks upfront.
tsx
import {parseMedia} from '@remotion/media-parser';
 
const {videoTracks, audioTracks} = await parseMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  fields: {
    tracks: true,
  },
});

You now have an array of VideoTrack and AudioTrack objects.

You can now use canReencodeAudioTrack(), canReencodeVideoTrack(), canCopyAudioTrack(), and canCopyVideoTrack() to determine which options to show.

Use the onVideoTrack and onAudioTrack callbacks to return the user selection.
You can use the trackId field as the unique key for each track.

Falling back to default​

The default values for onVideoTrack and onAudioTrack are the functions defaultOnVideoTrackHandler and defaultOnAudioTrackHandler respectively.

If you only want to override part of the logic, you can return the default resolver functions at the end of your logic.

Falling back to the default behavior
tsx
import {convertMedia, defaultOnAudioTrackHandler} from '@remotion/webcodecs';
 
await convertMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  container: 'webm',
  onAudioTrack: (params) => {
    // Custom logic for handling video tracks
    // ...
 
    // Fall back to the default behavior
    return defaultOnAudioTrackHandler(params);
  },
});
Falling back to the default behavior
tsx
import {convertMedia, defaultOnAudioTrackHandler} from '@remotion/webcodecs';
 
await convertMedia({
  src: 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
  container: 'webm',
  onAudioTrack: (params) => {
    // Custom logic for handling video tracks
    // ...
 
    // Fall back to the default behavior
    return defaultOnAudioTrackHandler(params);
  },
});

Debugging​

Pass logLevel: "verbose" to convertMedia() to see debug information in the console, including how the defaults have decided which operations to take.

Reference implementation​

Visit the source code for convert.remotion.dev to see a reference implementation for an online video converter that displays a user interface for all possible options.

See also​