# OpenTimelineIO for Rust: the Rust documentation
> A pure-Rust OpenTimelineIO: the data model, the time math, the file-format adapters, and an SDK for every language they are generated into.
The documentation with every code sample in Rust. The core itself. A document owns its objects; everything else is built on this. The binding lives in `crates/otio-core` (https://github.com/alchemist-editor/otio-rust/blob/main/crates/otio-core).
Index of everything: https://otio-rust-docs.vercel.app/llms.txt
## Rust
> Using the core directly, with a Document that owns its objects and one crate per file format.
Rust is not a binding. The crates in this repository are the library: a port
of upstream OpenTimelineIO's C++ core, with each file format in a crate of its
own. Every other language on this site reaches these same crates, through the
C ABI or, for Python, through PyO3. From Rust you skip that layer and hold the
document yourself.
### Install
Add the crates you need. `otio-core` is the data model; each adapter is its
own crate so that a program reading EDLs does not compile an AAF parser.
```toml
[dependencies]
otio-core = { git = "https://github.com/alchemist-editor/otio-rust" }
otio-cmx3600 = { git = "https://github.com/alchemist-editor/otio-rust" }
```
| Crate | What it holds |
| --- | --- |
| `opentime` | `RationalTime`, `TimeRange` and `TimeTransform` |
| `otio-core` | The object model, the `.otio` reader and writer, the algorithms and the edits |
| `otio-adapter` | The `Adapter` trait every format implements, and the error they share |
| `otio-cmx3600` | CMX 3600 EDL (`Cmx3600`) |
| `otio-ale` | Avid Log Exchange (`Ale`) |
| `otio-fcp7` | Final Cut Pro 7 XML (`Fcp7Xml`) |
| `otio-fcpx` | Final Cut Pro X XML (`FcpxXml`) |
| `otio-aaf` | AAF (`Aaf`) |
| `otio-bundle` | The `.otioz` and `.otiod` bundles |
The workspace is edition 2024 with a `rust-version` of 1.85. `otio-core`
depends on nothing but `opentime`, and the workspace has no third-party
dependencies.
To build and test everything from a clone:
```sh
git clone https://github.com/alchemist-editor/otio-rust
cd otio-rust
cargo build --workspace
cargo test --workspace
```
### Your first program
```rust
use otio_adapter::Adapter;
use otio_cmx3600::{Cmx3600, ReadOptions};
fn main() -> Result<(), otio_adapter::Error> {
// An EDL never says what rate its timecode is at, so this has to be
// right: a file read at the wrong rate puts every event in the wrong
// place rather than failing.
let options = ReadOptions {
rate: 24.0,
..Default::default()
};
let document = Cmx3600::read_from_file("cut.edl", &options)?;
let root = document.root().expect("a parsed document has a root");
for id in document.find_clips(root)? {
let clip = document.try_get(id)?;
println!("{}", clip.name());
}
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/read-an-edl/rust.rs
Reading goes through the `Adapter` trait from `otio-adapter`, implemented by a
unit struct per format. Each format's options are a struct of their own, so an
option meant for one format cannot be handed to another, and a misspelled one
is a compile error. The rate is the option that matters for an EDL: see
[CMX 3600 EDL](https://otio-rust-docs.vercel.app/docs/formats/edl/).
What comes back is a `Document`. Its `root()` is an `Option`, and
`try_get` resolves a handle to the `Node` it names.
### Building a timeline
```rust
use opentime::{RationalTime, TimeRange};
use otio_core::schema::{Base, Clip, ItemData, Stack, Timeline, Track};
use otio_core::{Document, Node};
fn main() -> Result<(), Box> {
// A document is an arena. Objects go into it and are named by handles,
// so nothing here is a pointer into anything.
let mut document = Document::new();
let tracks = document.insert(Node::Stack(Stack::default()));
let timeline = document.insert(Node::Timeline(Timeline {
base: Base {
name: "Cut".into(),
..Base::default()
},
tracks: Some(tracks),
global_start_time: None,
}));
document.set_root(Some(timeline));
let track = document.insert(Node::Track(Track {
kind: "Video".into(),
..Track::default()
}));
document.append_child(tracks, track)?;
for (index, name) in ["A", "B", "C"].into_iter().enumerate() {
let clip = document.insert(Node::Clip(Clip {
item: ItemData {
base: Base {
name: name.into(),
..Base::default()
},
source_range: Some(TimeRange::new(
RationalTime::new(index as f64 * 24.0, 24.0),
RationalTime::new(24.0, 24.0),
)),
..ItemData::new()
},
..Clip::default()
}));
document.append_child(track, clip)?;
}
// Three seconds of picture, written as canonical OpenTimelineIO JSON.
println!("{}", document.duration(track)?.to_seconds());
std::fs::write("cut.otio", otio_core::to_string(&document)?)?;
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/build-a-timeline/rust.rs
Objects are plain structs wrapped in the `Node` enum and inserted into the
document, which hands back a `NodeId` for each. Links between objects, such as
the timeline's `tracks` and a track's children, are handles too, so building
is inserting and then linking with `append_child`.
### How objects and documents work here
Upstream's C++ core uses intrusive reference counting plus raw parent
back-pointers, which is a reference cycle. `otio-core` stores every object in
a generational arena inside a `Document` and refers to them by `NodeId`, a
`Copy` handle. Parent and child links are handles, so a cycle is not
representable and nothing leaks. Removing an object bumps its slot's
generation, so a stale handle reports itself as stale rather than silently
reaching whatever took its place. The reasoning is in
[ADR 0001](https://github.com/alchemist-editor/otio-rust/blob/main/docs/adr/0001-ownership-model.md).
The other bindings hide this: an object built on its own gets a document of
its own, and putting it into another moves it. In Rust that move is explicit.
`Document::absorb` empties one document into another and hands back the map
from old handles to new ones, rewriting every link on the way, including the
ones inside metadata.
There are two deep copies, and they differ only for an object held in two
places. `Document::clone_object` is upstream's `clone()`: an object held twice
comes out as two objects, and an object that holds itself is refused with
`ObjectCycle`. `Document::deep_clone` copies an object held twice once, and
copies a cycle.
### Errors and missing values
Failure is a `Result`. `otio-core`'s `Error` displays as upstream's message,
character for character, because code in the wild matches on that text; where
upstream appends the object concerned, `Error::object` names it. Reading
errors follow upstream's reader too, including RapidJSON's message and
position for a JSON syntax error. The adapters return `otio_adapter::Error`.
"There is nothing here" is an `Option`: a document's `root()`, an item's
`source_range`. One case is an error rather than an answer: asking a bare
`Composition` where a child sits returns `Error::NoLayout`, because unlike a
track or a stack it does not say how its children are laid out.
The reader is more lenient than upstream's on purpose. A missing field or an
explicit `null` takes its default, and a schema `otio-core` does not know is
kept whole and written back out unchanged.
### Editing and algorithms
```rust
use opentime::{RationalTime, TimeRange};
use otio_core::schema::{Base, Clip, ItemData, Track};
use otio_core::{Document, Node, NodeId};
/// One second of picture, named.
fn second(document: &mut Document, name: &str) -> NodeId {
document.insert(Node::Clip(Clip {
item: ItemData {
base: Base {
name: name.into(),
..Base::default()
},
source_range: Some(TimeRange::new(
RationalTime::new(0.0, 24.0),
RationalTime::new(24.0, 24.0),
)),
..ItemData::new()
},
..Clip::default()
}))
}
fn show(document: &Document, track: NodeId) -> Result<(), otio_core::Error> {
let names: Vec<&str> = document
.children_of(track)?
.iter()
.map(|id| document.try_get(*id).map(Node::name).unwrap_or(""))
.collect();
println!("{} - {} frames", names.join(" "), document.duration(track)?.value());
Ok(())
}
fn main() -> Result<(), Box> {
let mut document = Document::new();
let track = document.insert(Node::Track(Track {
kind: "Video".into(),
..Track::default()
}));
for name in ["A", "B", "C"] {
let clip = second(&mut document, name);
document.append_child(track, clip)?;
}
show(&document, track)?;
// Insert makes room: everything from the insertion point onwards moves
// later, and the track gets longer.
let inserted = second(&mut document, "D");
otio_core::edit::insert(
&mut document,
inserted,
track,
RationalTime::new(24.0, 24.0),
false,
None,
)?;
show(&document, track)?;
// Overwrite does not: it lays an item over a span and whatever was in
// that span gives way. The track is the same length afterwards.
let laid = second(&mut document, "E");
otio_core::edit::overwrite(
&mut document,
laid,
track,
TimeRange::new(RationalTime::new(48.0, 24.0), RationalTime::new(24.0, 24.0)),
false,
None,
)?;
show(&document, track)?;
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/edit-operations/rust.rs
The ten edit operations are functions in `otio_core::edit`, each taking the
document first: `slice`, `overwrite`, `insert`, `trim`, `ripple`, `roll`,
`slip`, `slide`, `fill` and `remove`. They are upstream's `editAlgorithm`,
working in place on a track; [Editing a timeline](https://otio-rust-docs.vercel.app/docs/guides/editing/) says
what each does to its neighbours.
`otio_core::algorithm` holds upstream's stack and track algorithms:
`flatten_stack` collapses a stack into the track a viewer would see, and
`track_trimmed_to_range` cuts a track down to a span. Both leave the original
untouched and leave nothing behind in the document but the answer.
### Formats
`.otio` itself is `otio-core`'s: `otio_core::from_str` reads a document and
`otio_core::to_string` writes one. OTIO files are not quite JSON, since
upstream writes `NaN` and `Inf` as bare literals, so `otio-core` carries its
own parser and writer. Older files are upgraded as they are read, following
upstream's registered upgrade functions.
Every other format is its adapter crate, and converting is a read with one
and a write with another:
```rust
use otio_adapter::Adapter;
use otio_cmx3600::{Cmx3600, ReadOptions};
use otio_fcpx::FcpxXml;
fn main() -> Result<(), otio_adapter::Error> {
// An EDL never says what rate its timecode is at, so this has to be
// right: a file read at the wrong rate puts every event in the wrong
// place rather than failing.
let options = ReadOptions {
rate: 24.0,
..Default::default()
};
let document = Cmx3600::read_from_file("cut.edl", &options)?;
// Nothing happens in between. The timeline an EDL parses to is the same
// timeline FCP X writes out, so converting is a read and a write: the
// object model is the interchange, and the file formats are two ways of
// spelling it.
FcpxXml::write_to_file(&document, "cut.fcpxml", &Default::default())?;
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/convert-a-format/rust.rs
Bundles are `otio_bundle::write_otioz`, `write_otiod` and `read_otioz`. See
[Reading and writing files](https://otio-rust-docs.vercel.app/docs/guides/reading-and-writing/) for what each
format carries, and [AAF](https://otio-rust-docs.vercel.app/docs/formats/aaf/) and [Bundles](https://otio-rust-docs.vercel.app/docs/formats/bundles/)
for those two.
### Platforms
CI builds and tests the workspace on Linux, macOS and Windows.
### Reference
- [`crates/otio-core/README.md`](https://github.com/alchemist-editor/otio-rust/blob/main/crates/otio-core/README.md):
ownership, copying, editing, old files and error messages in full.
- [`crates/otio-adapter/README.md`](https://github.com/alchemist-editor/otio-rust/blob/main/crates/otio-adapter/README.md):
the `Adapter` and `TextAdapter` traits.
- [The C ABI](https://otio-rust-docs.vercel.app/docs/languages/c/), which is the layer every generated SDK sits
on, and its [reference](https://otio-rust-docs.vercel.app/reference/).
---
Page: https://otio-rust-docs.vercel.app/docs/languages/rust/
## Start here
### Overview
> A pure-Rust OpenTimelineIO, and a binding for every language it is generated into.
[OpenTimelineIO](https://opentimeline.io) is how editorial decisions move
between programs. A cut assembled in one tool, a conform in another, a review
in a third: each holds the same idea — these shots, in this order, trimmed
here, at this rate — and OTIO is the written form of that idea, so the three
can agree about it without agreeing about anything else.
This library is a port of it to Rust. Not a reimplementation with its own
ideas about how timelines should work: the arithmetic, the rounding and the
timecode behaviour follow upstream exactly, measured against upstream's own
test suites, because a file written here has to open unchanged in every tool
that already reads OTIO.
#### What you get
The whole data model, the time math beneath it, and the interchange formats
post actually uses:
| Piece | What it does |
| --- | --- |
| `opentime` | Rational time, time ranges, SMPTE timecode, drop-frame and all |
| `otio-core` | The schemas, the compositions, the algorithms, and `.otio` JSON |
| `otio-ale` | Avid Log Exchange |
| `otio-cmx3600` | CMX 3600 Edit Decision Lists |
| `otio-fcp7` | Final Cut Pro 7 interchange XML |
| `otio-fcpx` | Final Cut Pro X XML |
| `otio-aaf` | AAF, read and written as upstream's adapter writes it |
Everything above sits on Rust and nothing else. There is no C++ library
underneath and no Python interpreter: a binding links one static library.
#### Why the bindings all agree
Each language SDK is generated from the C interface rather than written by
hand, and that interface describes itself — the generator reads the Rust
source of the ABI, not a header and not a schema somebody maintains alongside
it. So a call added to the core is a call every SDK has, and a call whose
meaning changed cannot keep an old signature in one language and a new one in
another. Drift fails the build.
That is also why this site can promise a switcher rather than a paragraph
apologising for the languages it did not get to.
```rust
use otio_adapter::Adapter;
use otio_cmx3600::{Cmx3600, ReadOptions};
fn main() -> Result<(), otio_adapter::Error> {
// An EDL never says what rate its timecode is at, so this has to be
// right: a file read at the wrong rate puts every event in the wrong
// place rather than failing.
let options = ReadOptions {
rate: 24.0,
..Default::default()
};
let document = Cmx3600::read_from_file("cut.edl", &options)?;
let root = document.root().expect("a parsed document has a root");
for id in document.find_clips(root)? {
let clip = document.try_get(id)?;
println!("{}", clip.name());
}
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/read-an-edl/rust.rs
Reading is the same shape everywhere: reading a file hands back the object it
is about, and you ask that object what is under it. Only Rust, C and Zig show
you the document that owns the objects on the way; every other SDK keeps it
out of sight, as upstream's own bindings do. What differs is what each
language calls a failure and what it calls an absence — an `error` in Go, a
`throws` in Swift, an `NSError` in Objective-C, an optional in Zig — and the
SDK for each follows that language rather than the C interface it came from.
#### Where to go next
- [Getting started](https://otio-rust-docs.vercel.app/docs/getting-started/) builds the core and reaches it from
each language, and each language has [its own page](https://otio-rust-docs.vercel.app/languages/).
- [Reading and writing files](https://otio-rust-docs.vercel.app/docs/guides/reading-and-writing/) is the guide to
the adapters, with a page per format: [EDL](https://otio-rust-docs.vercel.app/docs/formats/edl/),
[ALE](https://otio-rust-docs.vercel.app/docs/formats/ale/), [Final Cut Pro XML](https://otio-rust-docs.vercel.app/docs/formats/final-cut-pro/),
[AAF](https://otio-rust-docs.vercel.app/docs/formats/aaf/) and [bundles](https://otio-rust-docs.vercel.app/docs/formats/bundles/).
- [Editing a timeline](https://otio-rust-docs.vercel.app/docs/guides/editing/) covers the ten edit operations.
- [The data model](https://otio-rust-docs.vercel.app/docs/concepts/data-model/) and
[Time ranges](https://otio-rust-docs.vercel.app/docs/concepts/time-ranges/) are the concepts every other page
assumes.
- [The C ABI reference](https://otio-rust-docs.vercel.app/reference/) lists every call the SDKs are generated
from.
For a language model, [`/llms.txt`](https://otio-rust-docs.vercel.app/llms.txt) indexes every page as Markdown,
and `/llms/.txt` is the whole of the docs with every sample in one
language.
---
Page: https://otio-rust-docs.vercel.app/docs/
### Getting started
> Building the core, and reaching it from your language.
Everything starts with the Rust core. The bindings other than Python link a
static library built from it, so that is the first build in every case. The
TypeScript package is the exception: it is
[on npm](#typescript), and needs none of this.
This page is the short version for every language at once. Each language has
a page of its own with its install steps, a first program, and how it spells
failure and absence:
[Rust](https://otio-rust-docs.vercel.app/docs/languages/rust/), [Python](https://otio-rust-docs.vercel.app/docs/languages/python/),
[TypeScript](https://otio-rust-docs.vercel.app/docs/languages/typescript/), [Go](https://otio-rust-docs.vercel.app/docs/languages/go/),
[Swift](https://otio-rust-docs.vercel.app/docs/languages/swift/), [C++](https://otio-rust-docs.vercel.app/docs/languages/cpp/),
[Zig](https://otio-rust-docs.vercel.app/docs/languages/zig/), [C](https://otio-rust-docs.vercel.app/docs/languages/c/),
[C#](https://otio-rust-docs.vercel.app/docs/languages/csharp/) and [Objective-C](https://otio-rust-docs.vercel.app/docs/languages/objectivec/).
```sh
git clone https://github.com/alchemist-editor/otio-rust
cd otio-rust
cargo build --workspace
cargo test --workspace
```
> [!NOTE]
> Build the workspace before running the tests that include the C ABI's own C
> program. It links against `libotio`, so a test run that is also the first
> build can reach for a library that is not there yet.
#### Rust
Add the crates you need. `otio-core` is the data model; each adapter is its
own crate so that a program reading EDLs does not compile an AAF parser.
```toml
[dependencies]
otio-core = { git = "https://github.com/alchemist-editor/otio-rust" }
otio-cmx3600 = { git = "https://github.com/alchemist-editor/otio-rust" }
```
#### Python
The package is called `opentimelineio`, and the name is the point: it aims to
be a drop-in replacement for upstream's own package, measured by running
upstream's test files unmodified.
```sh
cd crates/otio-python
pip install .
```
Files are read and written the way upstream reads and writes them, through
`opentimelineio.adapters`: `read_from_file("cut.edl", rate=24)` picks the EDL
adapter from the suffix, and each adapter takes upstream's keyword arguments.
Every format on the [reading and writing](https://otio-rust-docs.vercel.app/docs/guides/reading-and-writing/)
page is there, AAF included.
The rest of upstream's package is there as well. Its plugin system loads
adapters, media linkers, hooks, schemadefs and version manifests from
`OTIO_PLUGIN_MANIFEST_PATH` and from installed packages' entry points, the
same way upstream's does, and the built-in formats are declared as plugins
themselves. Installing the package also installs upstream's console tools:
`otiocat`, `otioconvert`, `otiostat`, `otiotool` and `otiopluginfo`.
#### The SDKs
Go, Swift, Zig, C++, C# and Objective-C each link `libotio`, which is not
checked in. Build it and put it where the package expects it:
```sh
cargo build -p otio-capi --release
cp target/release/libotio.a sdk/go/lib/ # or sdk/swift, sdk/zig, sdk/cpp, sdk/objc
```
C# is the one that loads the library at run time rather than linking it, so
it wants the shared one instead — `libotio.so`, or `libotio.dylib` on macOS —
copied into `sdk/csharp/lib/`.
Then each is ordinary for its language — `go test ./...`, `swift test`,
`zig build test`, `cmake --build`, `dotnet run --project tests`, `make check`.
The README beside each SDK has the exact invocation, including the linker
flag Swift needs and the GNUstep packages Objective-C needs on Linux.
#### TypeScript
The TypeScript package is different: it has no static library at all. The C
ABI is compiled to WebAssembly and the package ships the module, so it runs
in a browser and in Node with nothing native to install. It is published to
npm as `@alchemist-edit/otio`, so none of the build above is needed to use it:
```sh
npm install @alchemist-edit/otio
```
Call `init()` once before anything else. In Node it reads the module off
disk; in a browser it fetches it from beside the JavaScript, in the form every
bundler understands, so there is nothing to configure.
To build it from this repository instead, which needs Rust and the
`wasm32-unknown-unknown` target:
```sh
cd crates/otio-wasm/ts
npm install
npm run build
```
There is no filesystem behind a WebAssembly module, so the adapters there
take and return bytes and leave getting hold of them to the host — a `fetch`
in a browser, `fs` in Node. That is a line of code, and it is the host's line
rather than the library's.
---
Page: https://otio-rust-docs.vercel.app/docs/getting-started/
## Guides
### Reading and writing files
> The adapters, which formats each language can read and write, and what a round trip preserves.
An adapter turns a file of some interchange format into a document and back.
Upstream expresses this as a plugin with four loosely specified entry points
and a dictionary of keyword arguments; here it is one trait per format, with
each format's options named and typed — so a misspelled option is a compile
error rather than a silent no-op. The Python package puts upstream's own
`opentimelineio.adapters` back on top of those, with each adapter's names,
keyword arguments and exceptions, so code written against upstream runs
unchanged; there an unknown keyword is a `TypeError` rather than ignored.
| Format | Suffix | Reads | Writes | Guide |
| --- | --- | --- | --- | --- |
| OpenTimelineIO JSON | `.otio` | Yes | Yes | [The .otio file format](https://otio-rust-docs.vercel.app/docs/concepts/otio-file-format/) |
| CMX 3600 EDL | `.edl` | Yes | Yes, one video track | [EDL](https://otio-rust-docs.vercel.app/docs/formats/edl/) |
| Avid Log Exchange | `.ale` | Yes | Yes | [ALE](https://otio-rust-docs.vercel.app/docs/formats/ale/) |
| Final Cut Pro 7 XML | `.xml` | Yes | Yes | [Final Cut Pro XML](https://otio-rust-docs.vercel.app/docs/formats/final-cut-pro/) |
| Final Cut Pro X XML | `.fcpxml` | Yes | Yes | [Final Cut Pro XML](https://otio-rust-docs.vercel.app/docs/formats/final-cut-pro/) |
| AAF | `.aaf` | Yes | Yes | [AAF](https://otio-rust-docs.vercel.app/docs/formats/aaf/) |
| OTIO zip bundle | `.otioz` | Yes, except from TypeScript | Yes, except from TypeScript | [Bundles](https://otio-rust-docs.vercel.app/docs/formats/bundles/) |
| OTIO directory bundle | `.otiod` | Yes, except from TypeScript | Yes, except from TypeScript | [Bundles](https://otio-rust-docs.vercel.app/docs/formats/bundles/) |
```rust
use otio_adapter::Adapter;
use otio_cmx3600::{Cmx3600, ReadOptions};
fn main() -> Result<(), otio_adapter::Error> {
// An EDL never says what rate its timecode is at, so this has to be
// right: a file read at the wrong rate puts every event in the wrong
// place rather than failing.
let options = ReadOptions {
rate: 24.0,
..Default::default()
};
let document = Cmx3600::read_from_file("cut.edl", &options)?;
let root = document.root().expect("a parsed document has a root");
for id in document.find_clips(root)? {
let clip = document.try_get(id)?;
println!("{}", clip.name());
}
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/read-an-edl/rust.rs
Reading hands back whatever the file was about, usually a timeline, and the
same call shape works for every format: name the file, and pass the format's
options when it has any. Each format has its own page for what it carries and
what it cannot, linked from the table above. Two of them have a trap worth
knowing before you start: an [EDL](https://otio-rust-docs.vercel.app/docs/formats/edl/) does not know its own
rate, and an [AAF](https://otio-rust-docs.vercel.app/docs/formats/aaf/) clip cannot be written without a MobID.
#### What a round trip keeps
Reading and writing the same file back should produce the same file. Where a
format carries something the data model has no place for, the adapter keeps
it on the object's metadata under the format's own key — `metadata["cmx_3600"]`
for an EDL — so that writing it out reproduces the line it came from rather
than dropping it.
The opposite direction has limits that are the format's rather than this
library's. An EDL describes a single strand of picture, so a timeline with
two video tracks has no EDL form at all; a wipe reads as a wipe and writes
back out as a dissolve, because CMX 3600's vocabulary for wipes is not one
anybody agrees about. These are upstream's limits too, and they are
documented on each adapter rather than discovered at runtime.
#### Converting between formats
Converting is a read and a write with nothing in between. There is no
conversion step, and there is no per-pair code: an EDL and an FCP X XML are
two spellings of the same object model, so once a file is read the format it
came from stops mattering.
```rust
use otio_adapter::Adapter;
use otio_cmx3600::{Cmx3600, ReadOptions};
use otio_fcpx::FcpxXml;
fn main() -> Result<(), otio_adapter::Error> {
// An EDL never says what rate its timecode is at, so this has to be
// right: a file read at the wrong rate puts every event in the wrong
// place rather than failing.
let options = ReadOptions {
rate: 24.0,
..Default::default()
};
let document = Cmx3600::read_from_file("cut.edl", &options)?;
// Nothing happens in between. The timeline an EDL parses to is the same
// timeline FCP X writes out, so converting is a read and a write: the
// object model is the interchange, and the file formats are two ways of
// spelling it.
FcpxXml::write_to_file(&document, "cut.fcpxml", &Default::default())?;
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/convert-a-format/rust.rs
What survives the trip is the intersection of the two formats, and the table
above is how to predict it. Going to an EDL narrows a cut to one video
strand whatever it came from; going the other way, an EDL carries no media
references beyond reel names, so nothing downstream can relink without being
told where the media is.
Upstream's own adapters fail outright on several of these conversions, and
here they work. An EDL whose tracks start anywhere but `00:00:00:00`, or an
audio-only EDL, could not be written as FCP X XML; neither could an FCP 7
timeline with a 15 fps clip. And FCP X XML written as FCP 7 XML produced a
file that could not be read back. Each is a documented departure on the
adapter it concerns. So is one fix to plain reading: an FCP 7 nested
sequence reads with its tracks, where upstream reads every one as an empty
stack.
#### Writing what you built
`.otio` is the format with no limits — it is the data model's own
serialization, so anything you can build, it can hold.
```rust
use opentime::{RationalTime, TimeRange};
use otio_core::schema::{Base, Clip, ItemData, Stack, Timeline, Track};
use otio_core::{Document, Node};
fn main() -> Result<(), Box> {
// A document is an arena. Objects go into it and are named by handles,
// so nothing here is a pointer into anything.
let mut document = Document::new();
let tracks = document.insert(Node::Stack(Stack::default()));
let timeline = document.insert(Node::Timeline(Timeline {
base: Base {
name: "Cut".into(),
..Base::default()
},
tracks: Some(tracks),
global_start_time: None,
}));
document.set_root(Some(timeline));
let track = document.insert(Node::Track(Track {
kind: "Video".into(),
..Track::default()
}));
document.append_child(tracks, track)?;
for (index, name) in ["A", "B", "C"].into_iter().enumerate() {
let clip = document.insert(Node::Clip(Clip {
item: ItemData {
base: Base {
name: name.into(),
..Base::default()
},
source_range: Some(TimeRange::new(
RationalTime::new(index as f64 * 24.0, 24.0),
RationalTime::new(24.0, 24.0),
)),
..ItemData::new()
},
..Clip::default()
}));
document.append_child(track, clip)?;
}
// Three seconds of picture, written as canonical OpenTimelineIO JSON.
println!("{}", document.duration(track)?.to_seconds());
std::fs::write("cut.otio", otio_core::to_string(&document)?)?;
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/build-a-timeline/rust.rs
---
Page: https://otio-rust-docs.vercel.app/docs/guides/reading-and-writing/
### Editing a timeline
> The ten edit operations, what each one does to its neighbours, and which of them change a track's length.
Appending clips to a track builds a cut. Changing one is a different problem,
because every change has a question attached: what happens to everything
after it?
There are ten operations, and they are the ones a non-linear editor has.
They are functions rather than methods, because each is about two objects and
belongs to neither — which is how upstream arranges them too.
```rust
use opentime::{RationalTime, TimeRange};
use otio_core::schema::{Base, Clip, ItemData, Track};
use otio_core::{Document, Node, NodeId};
/// One second of picture, named.
fn second(document: &mut Document, name: &str) -> NodeId {
document.insert(Node::Clip(Clip {
item: ItemData {
base: Base {
name: name.into(),
..Base::default()
},
source_range: Some(TimeRange::new(
RationalTime::new(0.0, 24.0),
RationalTime::new(24.0, 24.0),
)),
..ItemData::new()
},
..Clip::default()
}))
}
fn show(document: &Document, track: NodeId) -> Result<(), otio_core::Error> {
let names: Vec<&str> = document
.children_of(track)?
.iter()
.map(|id| document.try_get(*id).map(Node::name).unwrap_or(""))
.collect();
println!("{} - {} frames", names.join(" "), document.duration(track)?.value());
Ok(())
}
fn main() -> Result<(), Box> {
let mut document = Document::new();
let track = document.insert(Node::Track(Track {
kind: "Video".into(),
..Track::default()
}));
for name in ["A", "B", "C"] {
let clip = second(&mut document, name);
document.append_child(track, clip)?;
}
show(&document, track)?;
// Insert makes room: everything from the insertion point onwards moves
// later, and the track gets longer.
let inserted = second(&mut document, "D");
otio_core::edit::insert(
&mut document,
inserted,
track,
RationalTime::new(24.0, 24.0),
false,
None,
)?;
show(&document, track)?;
// Overwrite does not: it lays an item over a span and whatever was in
// that span gives way. The track is the same length afterwards.
let laid = second(&mut document, "E");
otio_core::edit::overwrite(
&mut document,
laid,
track,
TimeRange::new(RationalTime::new(48.0, 24.0), RationalTime::new(24.0, 24.0)),
false,
None,
)?;
show(&document, track)?;
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/edit-operations/rust.rs
#### The ten
| Operation | What it does |
| --- | --- |
| `insert` | Puts an item at an instant, pushing what follows later. |
| `overwrite` | Lays an item over a span, replacing what was there. |
| `trim` | Moves an item's in and out points without moving its neighbours. |
| `ripple` | Moves an item's in and out points, sliding everything after it. |
| `roll` | Moves the cut between an item and its neighbour. |
| `slip` | Moves the media inside an item without moving the item. |
| `slide` | Moves an item along its track, taking the time from its neighbours. |
| `slice` | Cuts whatever sits at an instant into two. |
| `fill` | Drops an item into a gap, fitted as the reference point says. |
| `remove` | Takes whatever sits at an instant out. |
#### The question each one answers
The useful way to hold these is by what they do to the track's length and to
the neighbours.
**Length changes.** `insert` makes the track longer by the length of what
went in. `remove` without a fill makes it shorter, because what follows moves
up; `remove` with a fill leaves a gap and the length alone. `ripple` changes
it by however much the in or out point moved.
**Length does not change.** `overwrite` replaces rather than displaces.
`trim`, `roll`, `slip` and `slide` all move boundaries around within the
track without altering how much track there is. `slice` puts a cut in and
changes nothing else — two items where there was one, occupying the same
span.
**Neighbours move.** `insert`, `ripple` and `remove` without a fill. Those
are the three that shift material you did not name.
**Neighbours do not move.** `overwrite`, `trim`, `slip`, `slice` and `fill`.
`roll` and `slide` are the interesting pair: both change a neighbour, but
only by moving the boundary they share, so nothing further along the track
notices.
The sample above is `insert` and `overwrite` side by side, because that is
the distinction people reach for first and the one that decides whether a cut
gets longer.
#### Filling what an edit opens
Several of these can open a hole — `overwrite` past the end of a track,
`remove` with a fill, `insert` at a point past the end.
Each takes a **fill template**: the item to put in that hole. Pass nothing
and it is an ordinary `Gap`, which is transparent, so on a multi-track
timeline whatever is underneath shows through. When you want something else —
solid black, a slate, colour bars — pass a clip with a `GeneratorReference`
as the template and that is what fills it instead.
#### Slip and slide
These two are named after the editing desk and the names are worth learning
the way round.
**`slip`** keeps the item where it is on the track and moves the media inside
it: same four seconds of the cut, a different four seconds of the shot. The
track does not change at all, and neither does any neighbour.
**`slide`** keeps the media and moves the item along the track: the same four
seconds of the shot, arriving earlier or later. The time has to come from
somewhere, so the neighbours on either side absorb it — the one before grows
or shrinks, and the one after does the opposite.
#### Where they are, and where they are not
All ten are in the core, in the C ABI, in every SDK generated from it, and in
Python as `opentimelineio.algorithms.overwrite`, `insert` and the rest, with a
`ReferencePoint` enum for `fill`'s four-point edits.
Upstream's own Python package does not bind them; only its C++ has them. The
Python functions here follow that C++ `editAlgorithm.h` in their names,
parameters and defaults, and raise what its error handler raises. An object an
edit takes out of a track stays usable while Python holds it, with no parent,
as any removed child does.
#### What the copied piece holds
`slice`, `insert` and `overwrite` copy the piece of an item a split leaves
over, and `fill` with `ReferencePoint.Sequence` copies the clip it drops in.
The copy is made as upstream makes it, by the equivalent of writing the item
out and reading it back, so it shares nothing with the original: an object
the item holds in two places — one object under two metadata keys, one effect
listed twice, one media reference under two keys — becomes two separate
objects in the copy. The item left in place keeps what it held.
#### An item that holds itself
Metadata can hold whole objects, including the item it belongs to:
`clip.metadata["self"] = clip`. `slice`, `insert` and `overwrite` copy the
piece of an item a split leaves over, and `fill` copies the clip it drops in;
here that copy keeps the cycle, so the leftover piece holds itself just as
the original does, and the edit goes through like any other.
That is a deliberate difference from upstream. Upstream copies by writing the
item out and reading it back, which cannot carry a cycle, so the same edits
fail there with a cycle error — and `slice`, `insert` and `overwrite` fail
only after changing the track, leaving the clip cut short and the rest of it
lost. The cycle still cannot be saved: writing it as `.otio` is refused, here
as upstream.
---
Page: https://otio-rust-docs.vercel.app/docs/guides/editing/
### Media linkers and hooks
> Pointing every clip a read produces at the right media, and running code of your own at fixed points around a read or a write.
A file that comes from somewhere else rarely points at media where it lives
here. An EDL names a reel, an AAF names a MobID, an `.otio` from another
facility names paths on its disks. Upstream OpenTimelineIO answers that with
two plugin points, and every language here has both.
A **media linker** is handed each clip a read produced, and answers with the
media reference that clip should use instead, or with nothing to leave it
alone. A read names the linker it wants.
A **hook script** is handed the whole result at a named point, and answers
with what the read or write goes on with: the same object changed, or another
one. Scripts are attached to a **hook**, in order, and each is handed what the
one before it returned.
```rust
use otio_adapter::plugins::{self, HookScript, LinkerChoice, MediaLinker, PluginArguments};
use otio_core::schema::{ExternalReference, MediaReferenceData};
use otio_core::{Any, AnyDictionary, Node};
// A cut of two clips, as an .otio file would hold it.
const CUT: &str = r#"{
"OTIO_SCHEMA": "Track.1",
"name": "V1",
"kind": "Video",
"children": [
{"OTIO_SCHEMA": "Clip.2", "name": "A"},
{"OTIO_SCHEMA": "Clip.2", "name": "B"}
]
}"#;
/// One string out of an argument map.
fn text(arguments: &AnyDictionary, key: &str) -> Result {
match arguments.get(key) {
Some(Any::String(value)) => Ok(value.clone()),
_ => Err(format!("no {key} in the arguments")),
}
}
fn main() -> Result<(), Box> {
// A media linker is handed each clip as it is read, with the arguments
// the read was given, and answers with media it put in the document for
// the clip to use. `None` leaves the clip as it was.
plugins::registry().register_media_linker(
"proxies",
MediaLinker::new(|document, clip, arguments| {
let name = document
.try_get(clip)
.map_err(|error| error.to_string())?
.base()
.map(|base| base.name.clone())
.unwrap_or_default();
let root = text(arguments, "root")?;
Ok(Some(document.insert(Node::ExternalReference(
ExternalReference {
media: MediaReferenceData::default(),
target_url: format!("{root}/{name}.mov"),
},
))))
}),
);
// A hook script is handed the whole result, and answers with what the
// read goes on with: here the same object, stamped.
plugins::registry().register_hook_script(
"stamp",
HookScript::new(|document, target, arguments| {
let who = text(arguments, "who")?;
if let Some(base) = document
.try_get_mut(target)
.map_err(|error| error.to_string())?
.base_mut()
{
base.metadata.insert("read_by".into(), Any::String(who));
}
Ok(target)
}),
);
plugins::registry().attach_hook_script(plugins::POST_ADAPTER_READ, "stamp");
// In Rust the hooks run where you say: an adapter only parses, and
// `after_read` is what upstream runs once it has, in upstream's order.
let mut document = otio_core::from_str(CUT)?;
let mut arguments = PluginArguments {
media_linker: LinkerChoice::Named("proxies".into()),
..PluginArguments::default()
};
arguments
.media_linker_arguments
.insert("root".into(), Any::String("/proxies".into()));
arguments
.hook_arguments
.insert("who".into(), Any::String("the conform".into()));
plugins::after_read(&mut document, &arguments, AnyDictionary::new())?;
let track = document.root().ok_or("the cut has no root")?;
if let Some(Any::String(who)) = document
.try_get(track)?
.base()
.and_then(|base| base.metadata.get("read_by"))
{
println!("read by {who}");
}
for clip in document.find_clips(track)? {
let Node::Clip(found) = document.try_get(clip)? else {
continue;
};
let media = found.media_references[&found.active_media_reference_key];
if let Node::ExternalReference(external) = document.try_get(media)? {
println!("{} -> {}", found.item.base.name, external.target_url);
}
}
plugins::registry().remove_hook_script("stamp");
plugins::registry().remove_media_linker("proxies");
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/link-media-and-run-hooks/rust.rs
#### Where they run
A read parses the file, then runs, in this order, which is upstream's:
| Step | Handed |
| --- | --- |
| `post_adapter_read` hook | the hook arguments, plus `adapter_arguments` and `media_linker_argument_map` |
| the media linker, once per clip | the media linker arguments |
| `post_media_linker` hook | the media linker arguments, not the hook ones, as upstream hands them |
A write runs `pre_adapter_write` before it writes and `post_adapter_write`
after. Writing to a file adds the path as `_filepath`. The pre-write hook
works on a copy, so the object you asked to write is left exactly as it was,
and what the hook returns is what gets written.
Any other hook name is one of your own. Attaching a script to it declares it,
and `run_hook` on an object runs it there and then.
#### Choosing a linker
A read with no linker named uses the one the `OTIO_DEFAULT_MEDIA_LINKER`
environment variable names, or none, as upstream does. Naming one that was
never registered fails the read rather than skipping the step, because a
conform that silently did not link is worse than one that stopped. Asking for
no linking at all (`do_not_link_media`, upstream's
`MediaLinkingPolicy.DoNotLinkMedia`) leaves every clip alone, and the hooks
still run.
#### Arguments
Each read or write can carry two argument maps: one for the linker and one
for the hooks. Outside Rust and Python they cross as JSON text, an object
whose values may be anything OTIO metadata can hold, timeline objects
included. Your function receives them as metadata it can read with the
metadata calls it already knows. A timeline object among the arguments is
moved into the document the plugin works on; one the plugin puts into the
timeline stays there, and the rest are removed once it has run. Arguments
that are not a JSON object fail the read or write, whether or not any
plugin is registered to take them.
#### Failures
A linker or script that fails stops the read or write, and the error carries
its own message, prefixed with the plugin's name. In every language a
failure is whatever that language fails with, an error, an exception, a
panic, and it is turned into the library's `PLUGIN_ERROR` at the boundary
rather than crossing into code that cannot handle it.
#### Registering
Plugins live in one registry per process, as upstream's do. Registering a
name again replaces what it named; unregistering says whether there was
anything to remove. A script that is unregistered while a hook still lists it
fails that hook when it next runs, as upstream fails one its manifest names
and cannot find, so detach it as well.
Python finds plugins the way upstream does, through plugin manifests that
name `.py` modules, and also sees every linker and script registered from any
other language in the same process. Every other language registers functions
in code, which is how a program in that language would load them anyway.
[ADR 0008](https://github.com/alchemist-editor/otio-rust/blob/main/docs/adr/0008-native-linkers-and-hooks.md)
and [ADR 0009](https://github.com/alchemist-editor/otio-rust/blob/main/docs/adr/0009-linkers-and-hooks-across-the-c-abi.md)
say why.
---
Page: https://otio-rust-docs.vercel.app/docs/guides/media-linkers-and-hooks/
## Formats
### EDL (CMX 3600)
> Reading and writing Edit Decision Lists: the rate you must supply, the dialects, and what writing refuses.
An Edit Decision List is the oldest interchange format in post and the most
widely understood: a title, then a numbered list of events, each one or two
lines of timecode with free-form comments beneath. A file reads as a
`Timeline`.
```rust
use otio_adapter::Adapter;
use otio_cmx3600::{Cmx3600, ReadOptions};
fn main() -> Result<(), otio_adapter::Error> {
// An EDL never says what rate its timecode is at, so this has to be
// right: a file read at the wrong rate puts every event in the wrong
// place rather than failing.
let options = ReadOptions {
rate: 24.0,
..Default::default()
};
let document = Cmx3600::read_from_file("cut.edl", &options)?;
let root = document.root().expect("a parsed document has a root");
for id in document.find_clips(root)? {
let clip = document.try_get(id)?;
println!("{}", clip.name());
}
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/read-an-edl/rust.rs
#### An EDL does not know its own rate
This is the one thing to get right. A CMX 3600 file is timecode and nothing
else: it never states the rate that timecode is at, and nothing can infer it
from the contents. The rate you pass is believed. Pass the wrong one and the
file still reads — every event simply lands somewhere it should not.
There is a usual guess, 24, and it is a guess rather than a safe default.
#### What becomes what
| EDL | Document |
| --- | --- |
| `TITLE:` | the timeline's name |
| each channel the events name (`V`, `A1`, `AA/V`) | a track |
| an event | a clip on every track its channel maps to |
| a hole in the record timecode | a gap |
| `D` and `W###` edits | a `Transition` |
| `* FROM CLIP NAME:` | the clip's name |
| `* FROM CLIP:` / `* FROM FILE:` / `* OTIO REFERENCE FROM:` | the clip's media |
| a path with a `[1001-1020]` range in it | an `ImageSequenceReference` |
| `* LOC:` | a `Marker`, with its colour |
| `*ASC_SOP` and `*ASC_SAT` | the clip's `metadata["cdl"]` |
| `M2` | a `LinearTimeWarp` |
| a freeze-frame comment | a `FreezeFrame` |
| `BL`, `BLACK`, `BARS` reels | a `GeneratorReference` |
| everything else | the clip's `metadata["cmx_3600"]` |
Keeping the unrecognized comments is what lets a file survive a round trip:
reading each of upstream's samples, writing it back and reading it again
gives the same timeline.
#### Dialects
The three systems that read EDLs disagree about the comment that names a
clip's media, so the writer takes a style:
| Style | Names media with |
| --- | --- |
| Avid (the default) | `* FROM CLIP:` |
| Nucoda | `* FROM FILE:` |
| Premiere | nothing, with the path in `* OTIO REFERENCE FROM:` |
Premiere reads any `FROM` comment as meaning the clip has no name and calls it
`UNKNOWN`, so that dialect writes `AX` as every reel and puts the path in a
comment Premiere ignores and this adapter can read back. From Rust the style
is an enum, so an unknown dialect cannot be spelled at all; from Python it is
upstream's `style` keyword argument.
#### Reel names
Most systems will only read a reel name of eight characters, so the writer
pads or truncates to that by default. A truncated name is recorded in an
`* OTIO TRUNCATED REEL NAME FROM:` comment, so reading the file back gets the
original. Asking for no limit writes the name in full, which loses nothing
but which most systems will not read.
#### What writing will not do
Writing is narrower than reading, as it is upstream:
- Only one enabled video track, and at most two audio tracks. An EDL describes
a single strand of picture, so a timeline with two video tracks has no EDL
form and is refused.
- Dissolves, but not wipes. A wipe reads, and writes back out as a dissolve.
- One timing effect per clip, and only a speed change or a freeze frame.
#### Where it differs from upstream
The adapter is a port of upstream's `otio-cmx3600-adapter` and reproduces its
behaviour, with two differences:
- The writer works on a copy of the document, so writing does not change what
you handed it. Upstream rewrites the timeline in place, and a caller who
writes twice gets different text the second time.
- A marker's colour is a real colour here rather than the string `"RED"`, so
the name is canonicalized to `"Red"` and given its components. The name the
file used is still on the marker's `metadata["cmx_3600"]`.
The [crate's README](https://github.com/alchemist-editor/otio-rust/blob/main/crates/otio-cmx3600/README.md)
has the Rust options in full.
---
Page: https://otio-rust-docs.vercel.app/docs/formats/edl/
### ALE (Avid Log Exchange)
> Reading and writing shot logs, and where each column lands on a clip.
An ALE is a tab-separated shot log: a `Heading` of key/value pairs, a
`Column` line naming the fields, and a `Data` section with one row per clip.
It is a list of clips rather than a cut, so a file reads as a
`SerializableCollection` of clips, not a timeline.
#### What becomes what
| ALE | Document |
| --- | --- |
| `Heading` pairs | the collection's `metadata["ALE"]["header"]` |
| `Column` order | the collection's `metadata["ALE"]["columns"]` |
| `Name` | the clip's name (and stays as a column) |
| `Start`, `Duration`, `End` | the clip's `source_range` |
| `Source File` | an `ExternalReference` on the clip |
| `CDL`, `ASC_SOP`, `ASC_SAT` | the clip's `metadata["cdl"]` |
| every other column | the clip's `metadata["ALE"]` |
Keeping the unrecognized columns and the column order is what makes a round
trip lossless: upstream's own `sample.ale` reads and writes back byte for
byte.
#### Rates
Unlike an EDL, an ALE states its rate, in the heading's `FPS`, and that rate
wins over the one you pass, because the file was written for it. Timecode
exists only at SMPTE rates, so a heading saying `23.976` is read at
24000/1001; the heading keeps the spelling it arrived with. A rate that is not
near a SMPTE rate is refused rather than quietly rounded.
#### Quirks kept from upstream
- A value containing a tab is written as-is and becomes two columns when read
back. Upstream means to replace tabs with spaces and throws the result away.
- The `Name` column stays in `metadata["ALE"]` as well as becoming the clip's
name, so it is in the document twice. The writer depends on this: it
discovers columns from that metadata.
#### Where it differs from upstream
- A doubled sign such as `--1.0` in an `ASC_SOP` column simply does not start
a number. Upstream matches it and then fails to convert, failing the whole
row.
- Reading moves `ASC_SOP`, `ASC_SAT` and `CDL` into `metadata["cdl"]`, and the
writer rebuilds those columns from there. Upstream's writer looks only at
the `ALE` metadata, so a graded file it has just read writes those columns
blank. The numbers go back out through a float, so `-0.0870` is written as
`-0.087`.
- The writer snaps the heading's rate to a SMPTE rate the same way the reader
does. Upstream writes at the stated decimal, so a file at `23.976` comes
back with every time slid by a couple of frames.
The [crate's README](https://github.com/alchemist-editor/otio-rust/blob/main/crates/otio-ale/README.md)
has the Rust options.
---
Page: https://otio-rust-docs.vercel.app/docs/formats/ale/
### Final Cut Pro XML
> FCP 7 interchange XML and FCP X XML: what each reads as, and what is kept for the round trip.
Two different formats share the name. FCP 7's `.xml` is the older one, and
still how a great many tools hand an edit to one another; FCP X's `.fcpxml`
is Final Cut's current format, and it does not think in tracks. Both are ports
of upstream's adapters, and both keep what OTIO has no field for in metadata
so a file read and written again keeps it.
```rust
use otio_adapter::Adapter;
use otio_cmx3600::{Cmx3600, ReadOptions};
use otio_fcpx::FcpxXml;
fn main() -> Result<(), otio_adapter::Error> {
// An EDL never says what rate its timecode is at, so this has to be
// right: a file read at the wrong rate puts every event in the wrong
// place rather than failing.
let options = ReadOptions {
rate: 24.0,
..Default::default()
};
let document = Cmx3600::read_from_file("cut.edl", &options)?;
// Nothing happens in between. The timeline an EDL parses to is the same
// timeline FCP X writes out, so converting is a read and a write: the
// object model is the interchange, and the file formats are two ways of
// spelling it.
FcpxXml::write_to_file(&document, "cut.fcpxml", &Default::default())?;
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/convert-a-format/rust.rs
#### Final Cut Pro 7 XML
FCP 7 itself is long gone, but its XML is not: Premiere Pro, Resolve, Hiero
and Media Composer all read or write some dialect of it.
The format carries far more per-element detail than OTIO has fields for.
Everything the adapter does not turn into a real OTIO field is kept under the
`fcp_xml` key in the relevant object's metadata and written back out on the
way past, so a file read and written again keeps its colour settings, its
effect parameters and its host application's bookkeeping.
One fix to plain reading departs from upstream: a nested sequence reads with
its tracks, where upstream reads every one as an empty stack.
#### Final Cut Pro X XML
Final Cut does not think in tracks. A sequence holds one `spine`, the main
storyline, and everything layered over or under it hangs off whichever
storyline item it overlaps, carrying a `lane` number saying how far above or
below it sits. Reading works out where each element really starts and groups
the results by lane, one track per lane. Writing puts lane zero back into the
spine and reattaches the rest.
What comes back from a read depends on what the file holds:
| The file holds | It reads as |
| --- | --- |
| a library or an event | a `SerializableCollection` of timelines |
| a bare project | a `Timeline` |
| loose clips | a collection of clips and compound clips |
What Final Cut knows about a piece of media that OTIO has no field for (its
note, its keywords and its Spotlight metadata) is kept under the `fcpx` key in
the media reference's metadata and written back out on the way past.
#### Conversions upstream cannot do
Several conversions that fail outright with upstream's adapters work here,
each a documented departure on the adapter it concerns:
- An EDL whose tracks start anywhere but `00:00:00:00`, or an audio-only EDL,
written as FCP X XML.
- An FCP 7 timeline with a 15 fps clip, written as FCP X XML.
- FCP X XML written as FCP 7 XML, which upstream wrote as a file that could not
be read back.
---
Page: https://otio-rust-docs.vercel.app/docs/formats/final-cut-pro/
### AAF
> Reading, writing and changing AAF files, the options each takes, and how a written file relinks in Media Composer.
AAF is how a cut reaches Avid Media Composer. Reading and writing are ports
of upstream's AAF adapter, checked byte for byte against what upstream reads
and writes, and AAF is available from every language.
#### Reading an AAF
An AAF reads as upstream's adapter reads it, byte for byte once written as
OTIO JSON, on every sample file in upstream's test data. Upstream's four
reading options are all here, with upstream's defaults:
- `simplify` (on) collapses the nesting AAF has and OTIO does not need.
- `attach_markers` (on) moves each marker from the slot that carries it onto
the item it points at.
- `bake_keyframed_properties` (off) records each keyframed effect
parameter's value at every frame of its effect as `keyframe_baked_values`,
interpolated the way pyaaf2 interpolates it.
- `transcribe_log` (off) prints a line for each thing the reader makes,
word for word what upstream prints. From Rust it takes a `TranscribeLog`,
which hands each line to a function of your choosing. It is not in the C
ABI or the SDKs built on it, since it would need a callback across it.
From C and the SDKs, the first two are spelled as what turning them off
does, `aaf_keep_nesting` and `aaf_markers_on_slots`, so that options left at
zero read an AAF the way upstream does. The third is `aaf_bake_keyframes`.
Baked curves go through the platform's `pow`, `acos` and `cos`, so off Linux
the last digit of a baked value can differ from upstream's. Three log lines
upstream prints for rare failures embed a Python object's memory address or
a whole track, and those are worded differently here.
#### Writing an AAF
AAF is how a cut reaches Avid Media Composer, and writing one is a port of
upstream's AAF adapter rather than a design of its own. Each track becomes a
slot of a composition. Each clip becomes a chain of three objects: a master
clip, a file describing the media, and a tape carrying its timecode, which
is the chain Media Composer expects to relink through. Gaps, dissolves,
nested tracks, stacks and markers all have an AAF form; any other transition
is left out, as upstream leaves it out.
```rust
use otio_aaf::{Aaf, WriteOptions};
use otio_adapter::Adapter;
fn main() -> Result<(), otio_adapter::Error> {
// AAF is written from a timeline. A cut that was read from an AAF keeps
// what the reader found under metadata["AAF"], and that is written back,
// so its clips point at the same master mobs and the same media.
let document = otio_core::from_str(&std::fs::read_to_string("cut.otio")?)?;
// Every clip needs a MobID, from its metadata, its media's metadata or
// the AAF its media names. A cut built from scratch has none, so let
// the writer make them up rather than refuse the clip.
let options = WriteOptions::new().with_use_empty_mob_ids(true);
Aaf::write_to_file(&document, "cut.aaf", &options)?;
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/write-an-aaf/rust.rs
A clip is tied to its media by a MobID. A cut read from an AAF already has
one on every clip, kept under `metadata["AAF"]`, and it is written back, so
the file relinks to the media it came from. A clip whose media is itself an
AAF holding one master clip takes that clip's MobID. Anything else, such as
a cut built from scratch, has none, and writing it needs
`use_empty_mob_ids`, which makes MobIDs up. A made-up MobID links to no
media Media Composer knows, which is why upstream refuses a clip without one
by default, and so does this.
Before writing anything, the writer checks the whole timeline the way
upstream does: every item at the timeline's rate, every clip saying how much
media it has, and every transition carrying what an AAF dissolve is built
from. It reports everything it finds wrong at once rather than stopping at
the first.
The file is the same, byte for byte, as the one upstream's adapter writes
from the same timeline, given the same clock and the same random
identifiers. The tests check that on twelve files, and it holds on all 33
samples in upstream's own test data that upstream can write.
That parity is the evidence the file suits Media Composer: none of the files
has been imported into Media Composer as part of testing.
Embedding the media in the file works as upstream's `embed_essence` does.
Each clip's media URL is taken as a path, relative to the working directory
unless it is absolute. An `.aaf` there has the master mob with the clip's
MobID copied out of it, with its source mob and essence. A `.dnx` on a video
track is imported as a raw DNxHD stream. Anything else is refused as
upstream refuses it, and that includes a WAV file, which upstream sends to
the DNxHD import too.
From Python, upstream's four AAF hooks run where upstream runs them:
`otio_aaf_pre_read_transcribe` and `otio_aaf_post_read_transcribe` before and
after the file is transcribed, and `otio_aaf_pre_write_transcribe` and
`otio_aaf_post_write_transcribe` before and after it is written. A
`pre_write` hook is where to transcode other media into something
`embed_essence` can embed, as upstream suggests: what it returns is what is
written. Each hook is handed upstream's arguments, except that `aaf_handle`,
upstream's open pyaaf2 file, is `None`, because there is no pyaaf2 file here.
From Rust, `ReadOptions::with_post_transcribe` runs a callback at the one
point inside the read; the other three run around the read or write, so a
caller runs them itself.
AAF writing is available from every language. From Python,
`otio.adapters.write_to_file(timeline, "cut.aaf")` takes upstream's keyword
arguments, `embed_essence=True` among them. From C and the SDKs, the same options are `aaf_`-prefixed fields of the write
options, along with three a library caller needs and a Python one does not:
`aaf_user`, whom a new marker is credited to when no login name is set;
`aaf_time`, the time the file records; and `aaf_id_seed`, which seeds the
identifiers it makes up. The same time, seed and timeline write the same
file. WebAssembly has no clock or randomness of its own, so the TypeScript
package passes the time and a fresh seed on every write unless you give
your own. Nor has it a file system, so it cannot embed media: with
`aafEmbedEssence` set, a clip whose media names a file stops the write, as
the file cannot be found.
#### Changing an existing AAF
Below the adapter, the `aaf` crate can open an AAF file that already exists,
change it and save it, as pyaaf2 does when it opens a file with `'r+'`. The
same edits in the same order leave the same bytes pyaaf2 leaves, which
twenty scenarios in its tests check against files pyaaf2 changed: properties
changed, added and removed, mobs and slots added and taken out, new
definitions and classes, and essence moved and dropped. It works on the AAF
object model, not on a timeline, and is available from Rust only. The
[crate's README](https://github.com/alchemist-editor/otio-rust/blob/main/crates/aaf/README.md#changing-a-file)
has an example, and
[ADR 0005](https://github.com/alchemist-editor/otio-rust/blob/main/docs/adr/0005-aaf-modify-path.md)
the design.
---
Page: https://otio-rust-docs.vercel.app/docs/formats/aaf/
### Bundles (.otioz and .otiod)
> Packaging a timeline with its media, and what happens to media that is not a file.
A bundle is how a cut travels with its media: the timeline and every file it
references, in one archive or one directory, so it can be moved to another
machine and opened there.
```text
cut.otioz / cut.otiod
├── version.txt "1.0.0"
├── content.otio the timeline, references rewritten to media/...
└── media/
├── shot_010.mov
└── render.0001.exr ...
```
An `.otioz` is a zip archive, with `content.otio` deflated and the media
stored uncompressed so it can be read in place; an `.otiod` is the same
layout as a directory. The `otio-bundle` crate ports upstream's `bundle.cpp`.
#### Writing a bundle
A bundle is how a cut travels with its media. Writing one copies every file
the timeline's media references name into `media/` and points each reference
at its copy, so the bundle can be moved to another machine and opened there.
Every file lands directly under `media/`, so two media files with the same
name in different directories cannot both go in, and the write fails rather
than dropping one. Nor is a bundle ever written over: a path that already
exists stops the write.
```rust
use std::path::Path;
use otio_bundle::{MediaReferencePolicy, ReadOptions, WriteOptions};
fn main() -> Result<(), Box> {
let document = otio_core::from_str(&std::fs::read_to_string("cut.otio")?)?;
let timeline = document.root().ok_or("cut.otio holds nothing")?;
// Every clip whose media is a file on disk has the file copied into the
// bundle and its reference pointed at the copy. Media that is not a file,
// such as a URL on the web, would stop the write, so it is made missing
// instead. write_otiod writes the same layout as a directory.
let options = WriteOptions {
policy: MediaReferencePolicy::MissingIfNotFile,
..WriteOptions::default()
};
otio_bundle::write_otioz(&document, timeline, Path::new("cut.otioz"), &options)?;
// Unpacked, with each reference made absolute, the media is ready to use.
let unpacked = otio_bundle::read_otioz(
Path::new("cut.otioz"),
&ReadOptions {
extract_path: Some("cut".into()),
absolute_media_reference_paths: true,
},
)?;
println!("{}", otio_core::to_string(&unpacked)?);
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/write-a-bundle/rust.rs
Media that is not a file on disk, such as a URL on the web or a generator,
is what upstream's media reference policy is for. By default it stops the
write; told to, the writer replaces each such reference with a missing
reference instead, or replaces every reference and bundles no media at all.
A relative media path is found from the working directory unless you name
another. Reading an `.otioz` reads only the timeline out of the archive
unless you ask for it to be unpacked into a directory, which must not exist
yet; either bundle can then have its references rewritten as absolute paths
into it.
From Rust these are `otio_bundle::WriteOptions` and `ReadOptions`, and from
Python upstream's `media_policy`, `relative_media_base_dir` and
`extract_to_directory` keyword arguments. From C and the SDKs, the bundle
formats are `OTIO_FORMAT_OTIOZ` and `OTIO_FORMAT_OTIOD`, and the options are
fields of the read and write options: `bundle_media_policy` and
`bundle_media_base_dir` for writing, `bundle_extract_path` and
`bundle_absolute_media_paths` for reading. A bundle is read and written
through a path, never as bytes, and the calls that take bytes refuse one.
The TypeScript package has no file system to keep a bundle on, so it has no
bundle formats at all.
#### Media URLs with unusual characters
Media URLs are percent-decoded exactly as upstream decodes them, so a URL
with a `%` that is not followed by a hex digit, such as `a%zz.mov`, fails the
write (`ValueError("stoi")` from Python) whatever the policy, as it does
upstream.
An escape that spells a byte that is not UTF-8, such as `%E9` in
`caf%E9.mov`, names a file whose name has that raw byte, and on Linux (or any
Unix filesystem that allows such a name) that file is found and bundled.
Inside the bundle it is named `caf%E9.mov`, with the escape spelled out,
because upstream's raw name makes `content.otio` invalid JSON. From Python,
`url_utils.filepath_from_url` on such a URL raises `UnicodeDecodeError`, as
upstream's does.
The [crate's README](https://github.com/alchemist-editor/otio-rust/blob/main/crates/otio-bundle/README.md)
covers the zip implementation and the Rust options.
---
Page: https://otio-rust-docs.vercel.app/docs/formats/bundles/
## Concepts
### The data model
> Timelines, stacks, tracks, clips, gaps, transitions — and the handles that name them.
An OTIO document is a tree of objects that occupy time. There are not many
kinds, and the names are the ones upstream uses.
```text
Timeline
└── Stack ("tracks")
├── Track ("V1", kind "Video")
│ ├── Clip ("shot_01")
│ ├── Gap
│ └── Clip ("shot_02")
└── Track ("A1", kind "Audio")
└── Clip ("dialogue")
```
A **Timeline** is the whole cut. It holds one **Stack**, conventionally
called `tracks`, and a stack lays its children over the same span of time —
which is what a track being "above" another means. A **Track** is a strand:
its children run one after another, and a **Gap** is the hole where nothing
plays. A **Clip** is a piece of media with a range taken out of it, and a
**Transition** is the overlap between two of them.
#### Time, and what a range means
Every duration in OTIO is a `RationalTime`: a value and a rate, kept apart.
24 frames at 24fps is not the number 1 — it is `24/24`, and the rate travels
with it. This is the single most consequential decision in the format,
because it is what stops a 23.976 sequence from drifting into a 24 one by
accident.
```rust
use opentime::RationalTime;
fn main() -> Result<(), opentime::TimeError> {
// A time is a value and a rate, not a number of seconds. Four seconds at
// 24 is 96 units; the rate travels with it so nothing has to guess later.
let start = RationalTime::from_timecode("01:00:00:00", 24.0)?;
let duration = RationalTime::from_frames(96.0, 24.0);
let end = start + duration;
println!("{} for {} seconds", end.to_timecode()?, duration.to_seconds());
// Comparison rescales first, so the same instant at two rates is equal.
assert_eq!(RationalTime::new(24.0, 24.0), RationalTime::new(48.0, 48.0));
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/time-math/rust.rs
An item has up to three ranges, and the difference between them is the thing
people get wrong:
- **`available_range`** — everything the media has. A two-hour rush.
- **`source_range`** — the piece of it this item uses. Four seconds of that
rush. An item without one uses all of its media.
- **`trimmed_range`** — the source range, or the available one when there is
no source range. What the item actually contributes.
A clip's `duration` is its trimmed range's duration; a composition's is the
sum of its children's. That is why a track knows how long it is without
anyone saying so.
There are two more — `visible_range` and `range_in_parent` — and the thing
that actually trips people up is which clock each one answers in.
[Time ranges](https://otio-rust-docs.vercel.app/docs/concepts/time-ranges/) is that, in full.
#### Handles rather than pointers
Objects live in a `Document`, which owns them. You never hold an object
directly: you hold a handle, which is an index and a generation, and the
document resolves it. A handle to something that has gone away fails the
lookup rather than reading freed memory, and a handle from one document
cannot be used against another.
This is worth knowing because it shapes every binding. Every SDK but one hides
the document and lets a clip look like an object; Zig keeps it in the open,
because an arena you hand to the things that use it is how Zig already works.
Either way, what is underneath is the same arena.
```rust
use opentime::{RationalTime, TimeRange};
use otio_core::schema::{Base, Clip, ItemData, Stack, Timeline, Track};
use otio_core::{Document, Node};
fn main() -> Result<(), Box> {
// A document is an arena. Objects go into it and are named by handles,
// so nothing here is a pointer into anything.
let mut document = Document::new();
let tracks = document.insert(Node::Stack(Stack::default()));
let timeline = document.insert(Node::Timeline(Timeline {
base: Base {
name: "Cut".into(),
..Base::default()
},
tracks: Some(tracks),
global_start_time: None,
}));
document.set_root(Some(timeline));
let track = document.insert(Node::Track(Track {
kind: "Video".into(),
..Track::default()
}));
document.append_child(tracks, track)?;
for (index, name) in ["A", "B", "C"].into_iter().enumerate() {
let clip = document.insert(Node::Clip(Clip {
item: ItemData {
base: Base {
name: name.into(),
..Base::default()
},
source_range: Some(TimeRange::new(
RationalTime::new(index as f64 * 24.0, 24.0),
RationalTime::new(24.0, 24.0),
)),
..ItemData::new()
},
..Clip::default()
}));
document.append_child(track, clip)?;
}
// Three seconds of picture, written as canonical OpenTimelineIO JSON.
println!("{}", document.duration(track)?.to_seconds());
std::fs::write("cut.otio", otio_core::to_string(&document)?)?;
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/build-a-timeline/rust.rs
#### Schemas, and what happens to ones we do not know
Every object carries a schema name and version — `Clip.2`, `Marker.3` — and
this port reads the versions upstream 0.19 writes, upgrading older ones as it
reads. An object whose schema is not recognised at all is kept verbatim, so a
third-party plugin's data survives a read and a rewrite untouched.
Writing works in the other direction too. A document can be written for an
older release by naming the schema versions it should carry, and each object
is downgraded on the way out, as upstream's serializer does. The upgrade and
downgrade steps live in one registry in the core, keyed by schema and
version, and from Python `register_upgrade_function` and
`register_downgrade_function` add to it.
A schema defined in Python with `register_type` is held in the core the way
upstream's C++ holds it: a generic object carrying its schema name, version
and fields. Python keeps the class for each schema name and hands that class
back when the object is read, so a type defined in Python round-trips through
any document without the core ever holding a Python object.
A Python subclass of a concrete schema, such as a `Clip` that also records
which take it is, stays a clip in the core: it carries the subclass's schema
name, version and extra fields beside its own, so every composition and
algorithm treats it as the clip it is, and it is written under the subclass's
name. A program that has not registered the subclass reads it as an unknown
schema, kept verbatim. Any built-in object read from a file keeps the fields
it does not know in the same place, and writes them back.
---
Page: https://otio-rust-docs.vercel.app/docs/concepts/data-model/
### Timeline structure
> How stacks, tracks, clips, gaps and transitions nest, and what a flattened timeline looks like.
A timeline holds tracks, and tracks hold clips — and then somebody nests a
stack inside a track and it stops being obvious. This page is the shapes:
what contains what, and what the result means when something plays it.
Upstream's figures are pictures; these are the same structures as text,
which is the same information and survives being copied into a terminal.
#### A simple cut list
The smallest useful timeline is a single track of clips laid end to end.
```text
Timeline
└── Stack "tracks"
└── Track "Track-001"
├── Clip "Clip-001" source_range 0+10 → Media-001 available 0+20
├── Clip "Clip-002" source_range 5+10 → Media-002 available 0+15
├── Clip "Clip-003" source_range 2+8 → Media-003 available 100+30
└── Clip "Clip-004" source_range none → Media-004 available 0+12
```
A timeline always has that top-level stack, even with one track in it,
because a timeline can hold several and something has to lay them over each
other.
At the bottom, each media reference has a `target_url` saying where the media
is and an `available_range` saying how much of it there is. Ranges are written
here as `start+duration` in frames; `Media-003` starting at 100 means its
first frame is timecode 00:00:04:04 at 24fps, not that it is offset in the
cut.
Above them, each clip's `source_range` picks the piece it uses.
`Clip-004` has none, so it uses all of `Media-004` — and asking any of them
for `trimmed_range` answers without the caller having to know which case
it is in.
A `source_range` may point outside the `available_range`. That happens for
real — only the first half of a shot has been rendered — and nothing here
snaps it back or complains. What a player does about it is the player's
business.
The track's own length is the sum of its children's trimmed ranges. Nobody
sets it.
#### Transitions
A transition blends two neighbours on the same track: a dissolve, a wipe. It
can sit between any two composable items, not only clips.
```text
Track "Track-001"
├── Clip "Clip-002" ends at frame 10
├── Transition in_offset 2, out_offset 3
└── Clip "Clip-003" starts at frame 0
```
`out_offset` reaches back into the clip before it, `in_offset` forward into
the clip after. So `Clip-002` is on screen for three frames longer than its
own range says, and `Clip-003` for two frames earlier — which is exactly what
`visible_range` reports for each while `trimmed_range` does not. See
[Time ranges](https://otio-rust-docs.vercel.app/docs/concepts/time-ranges/) for the difference.
**A transition changes no length.** The track is as long with it as without,
so a tool that cannot render one may ignore it and still agree about where
everything is.
The format expects that a transition's offsets stay inside its neighbours,
that a clip with transitions at both ends has them not overlapping, and that
two transitions are never adjacent. Nothing in this library enforces any of
that — upstream does not either — so a file can carry all three and will be
read back as written.
A fade to or from black is usually a transition against a `Gap`, which may
have zero duration. A gap is *transparent* rather than black, so with tracks
underneath it the thing below shows through; for real black, use a clip with
a `GeneratorReference`.
#### Several tracks
Tracks in a stack play at the same time, lower entries first.
```text
Stack "tracks"
├── Track-001 [ Clip-001 ][ Gap 4 ][ Clip-002 ]
├── Track-002 [ Gap ][ Clip-003 ][ Clip-005 ]
└── Track-003 [ Clip-006 ] (trimmed by its own source_range)
flattened [ Clip-001 ][Clip-003][ Clip-002 ]
```
Where `Track-001` has a gap, the track below shows through — those four
frames of `Clip-003` are what the flattened result carries. `Clip-005` is
covered by `Clip-003` the whole time, so it does not appear at all.
The gap at the head of `Track-002` is doing nothing but offsetting what
follows it. That is the usual way to shift a track's contents; setting the
track's own `source_range` is the other, and `Track-003` does it that way.
Tracks in a stack need not be the same length. Making a short one match means
appending a gap, or extending its `source_range` — either way `trimmed_range`
ends up the same, which is the only thing downstream cares about.
Image tracks composite in painter order, bottom to top, over a background
that is zero in colour and full in alpha. Audio tracks sum. Neither is
something OTIO does — it is what OTIO *means*, and the application does it.
What an effect on a clip does is not specified at all.
#### Nesting
A track's children can be any composable item, and that includes other tracks
and stacks.
```text
Track "Track-001"
├── Clip "Clip-001"
├── Stack "Nested Stack" source_range 2+6
│ ├── Track-A [ Clip-003 ]
│ └── Track-B [ Clip-005 ] (trimmed by its own source_range)
├── Gap
└── Clip "Clip-004"
```
A nested composition behaves exactly like a clip that happens to have
complicated contents instead of a media reference. Give it a `source_range`
and only that slice of it is included — frames 2 to 7 here. Give it none and
its whole `available_range` is computed and used.
The consequence worth remembering: **nesting is opaque**. The gap in
`Track-001` cannot see `Clip-003`, because `Clip-003` is not its peer — it is
inside `Nested Stack`, and contents do not spill out into the neighbouring
gaps of the outer track.
---
This page is adapted from
[Timeline Structure](https://github.com/AcademySoftwareFoundation/OpenTimelineIO/blob/main/docs/tutorials/otio-timeline-structure.md)
in upstream OpenTimelineIO's documentation, which is Apache-2.0 like this
project. Its figures have been redrawn as text, and the claims about what is
and is not enforced were checked against this implementation.
---
Page: https://otio-rust-docs.vercel.app/docs/concepts/timeline-structure/
### Time ranges
> The five ranges an item has, which clock each one answers in, and why they differ.
A timeline is a one-dimensional coordinate system, and laying clips
end-to-end in a track is the easy case. Trimming, nesting, dissolves and
speed changes are where it stops being obvious, and the confusion is almost
always the same one: two different ranges were read as though they were
measured against the same clock.
They are not. Every range below belongs either to the **item's own clock** —
the media's frame numbering — or to its **parent's clock**, the track it sits
in. Getting this right is most of getting OTIO right.
```rust
use opentime::{RationalTime, TimeRange};
use otio_core::schema::{
Base, Clip, ExternalReference, Gap, ItemData, MediaReferenceData, Track,
};
use otio_core::{Document, Node};
use std::collections::BTreeMap;
fn frames(range: TimeRange) -> String {
format!(
"{} for {}",
range.start_time().value(),
range.duration().value()
)
}
fn main() -> Result<(), Box> {
let mut document = Document::new();
// Ten seconds of rushes on disk. `available_range` belongs to the media,
// not to the clip: it is what the file offers, whoever uses it.
let media = document.insert(Node::ExternalReference(ExternalReference {
media: MediaReferenceData {
available_range: Some(TimeRange::new(
RationalTime::new(0.0, 24.0),
RationalTime::new(240.0, 24.0),
)),
..MediaReferenceData::default()
},
target_url: "file:///A001.mov".into(),
}));
// Three seconds of it, starting two seconds in. A source range is in the
// media's clock, which is why it starts at 48 rather than at 0.
let clip = document.insert(Node::Clip(Clip {
item: ItemData {
base: Base {
name: "shot".into(),
..Base::default()
},
source_range: Some(TimeRange::new(
RationalTime::new(48.0, 24.0),
RationalTime::new(72.0, 24.0),
)),
..ItemData::new()
},
media_references: BTreeMap::from([("DEFAULT_MEDIA".to_string(), media)]),
active_media_reference_key: "DEFAULT_MEDIA".into(),
}));
// A second of black in front of it, so the clip does not start the track.
let head = document.insert(Node::Gap(Gap {
item: ItemData {
source_range: Some(TimeRange::new(
RationalTime::new(0.0, 24.0),
RationalTime::new(24.0, 24.0),
)),
..ItemData::new()
},
}));
let track = document.insert(Node::Track(Track {
kind: "Video".into(),
..Track::default()
}));
document.append_child(track, head)?;
document.append_child(track, clip)?;
// The same clip, asked four questions. The first three answer in the
// media's clock; the last answers in the track's.
println!("available: {}", frames(document.available_range(clip)?));
println!("trimmed: {}", frames(document.trimmed_range(clip)?));
println!("visible: {}", frames(document.visible_range(clip)?));
println!("in parent: {}", frames(document.range_in_parent(clip)?));
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/ranges-of-a-clip/rust.rs
That sample prints four different answers about one clip, and the rest of
this page is why.
#### The ranges in the item's own clock
##### `available_range`
Everything behind the item, before anything trims it.
For a clip that is the active media reference's own `available_range`: if the
clip points at `wedding.mov`, this is how long that file is and what timecode
it starts at. For a track it is the sum of its children; for a stack, the
longest of them.
It can be unknown. Media goes missing, and a URL can be expensive enough to
query that nobody has. Upstream answers `None` in that case; here asking a
clip whose media says nothing about its length is an error rather than an
answer — `NoAvailableRange` in Rust, which reaches C as
`OTIO_STATUS_CORE_ERROR` and each SDK as whatever that SDK makes of a
failure. Note the difference from `source_range` below, where absence *is*
one of the answers and the status is `OTIO_STATUS_NO_VALUE`.
##### `source_range`
The piece of the media this item uses — the trim, and the only one of these
ranges you set rather than ask for.
It may be absent, which means the item uses all of its media. An item with
neither a `source_range` nor an `available_range` is invalid: something has
to say how long it is.
It is usually shorter than what is available, but it does not have to be.
Asking for a range that runs past the end of the media is a real thing to
do — "this shot needs to be four seconds and only two have been rendered" —
and neither upstream nor this port will stop you or snap it back. What a
player does about it is the player's business.
##### `trimmed_range`
The `source_range` if there is one, and the `available_range` if there is
not. This is how long the item is as far as its parent is concerned, and it
is what `duration` reports.
##### `visible_range`
The `trimmed_range`, widened by any transition that reaches into the item.
A clip trimmed to end at frame 10, followed by a dissolve with an
`out_offset` of 5, is on screen until frame 15 — so its `visible_range` ends
at 15 while its `trimmed_range` ends at 10. With no transition touching it,
the two are equal, which is why the sample above prints the same numbers for
both.
#### The ranges in the parent's clock
##### `range_in_parent`
Where the item sits in the composition holding it.
In a **track** this is what you would expect, and one invariant follows from
it: each child begins exactly where the previous one ended, so for adjacent
clips A and B in a track,
```text
A.range_in_parent().end_time_exclusive() == B.range_in_parent().start_time()
```
In a **stack** it is duller, because a stack lays its children over the same
span: the start is always zero and the duration is the child's own. That is
what "these tracks are on top of each other" means. To offset something, put
it in a track and put a gap in front of it — or trim the track itself.
##### `trimmed_range_in_parent`
The same, clipped to the parent's own `source_range`.
Most parents have no `source_range`, so most of the time this is
`range_in_parent` again. When the parent *is* trimmed, this is the honest
answer about where the child is relative to what survives the trim, and when
the parent's trim excludes the child entirely there is no answer at all:
`None` in Rust, `OTIO_STATUS_NO_VALUE` through the C ABI.
One inherited quirk: a *direct* child trimmed out of its composition raises
`InvalidTimeRange` rather than answering "nothing". That is upstream's
behaviour and it is deliberate here, because the alternative — handing back
a zero-length range — would quietly place the item at the head of the track.
#### Compositions
A track or a stack is an item too, so it has all of the above. Two more are
worth naming:
- **`range_of_child_at_index`** — where the nth child sits, transitions
included.
- **`trimmed_range_of_child_at_index`** — the same, relative to the
composition's own trim.
Asking a track or a stack for its `duration` sums its children, which is why
nobody has to tell a track how long it is.
#### Markers
A marker can hang off any item, and its `marked_range` is in **that item's**
clock — the same clock as the item's `source_range`. A marker on a clip is
positioned in the media's numbering; a marker on a track is positioned along
the track.
Its duration may be zero, which means an instant rather than a span. Both
are ordinary.
#### Transitions
A transition has `in_offset` and `out_offset` rather than a range of its own:
how far it reaches back into the item before it and forward into the item
after. `range_in_parent` and `trimmed_range_in_parent` work on one, and both
defer to the parent.
A transition does not make its track any longer or shorter. A tool that
cannot render one can ignore it and still get the timeline's length right.
---
This page is adapted from
[Time Ranges](https://github.com/AcademySoftwareFoundation/OpenTimelineIO/blob/main/docs/tutorials/time-ranges.md)
in upstream OpenTimelineIO's documentation, which is Apache-2.0 like this
project. The behaviour described has been checked against this
implementation rather than carried over on trust; where the two differ — the
shape of "there is no answer", chiefly — it says so above.
---
Page: https://otio-rust-docs.vercel.app/docs/concepts/time-ranges/
### The .otio file format
> What is in an OTIO JSON file, what this implementation guarantees about it, and where it departs from strict JSON.
`.otio` is the format the rest of OpenTimelineIO exists to read and write. It
is JSON, it is meant to be read by a person as well as a program, and it is
the contract between this implementation and every other one.
That contract matters more here than it does upstream. Upstream is the
reference implementation, so whatever it writes is by definition right; this
is a second implementation, and a file it writes has to open unchanged in
tools built on the first.
#### The shape
An OTIO file is a tree of objects. Each object is a JSON dictionary whose
fields are either plain JSON values or more OTIO objects.
Every object carries an `OTIO_SCHEMA` key naming its type and version:
```json
{
"OTIO_SCHEMA": "Timeline.1",
"name": "my timeline",
"metadata": {},
"tracks": {
"OTIO_SCHEMA": "Stack.1",
"name": "tracks",
"children": [
{
"OTIO_SCHEMA": "Track.1",
"name": "video track",
"kind": "Video",
"children": []
}
]
}
}
```
There is **no file-level version**. Each type is versioned on its own, so a
new `Clip.3` can land without renumbering everything else. That is also why
reading starts by looking at whatever the top-level object says it is rather
than at a header.
Children live under `children`, everywhere except a timeline, whose one child
is `tracks`.
#### The top level can be anything
Most files hold a timeline, and most code assumes one. The format does not
require it: a bare `Clip`, a `Track`, even a `RationalTime` is a valid `.otio`
file, and this library writes one as readily.
```text
otio_core::to_string starts at whatever it is pointed at.
Hand it a clip and you get a clip.
```
Code that reads a file should say so when the top-level object is not what it
wanted, rather than assuming. A `SerializableCollection` is the right
container for several objects, because a bare JSON array cannot carry
metadata of its own.
#### No instancing
There are no references to an object from two places. If the same clip or the
same media appears twice in a timeline, it appears as two identical copies.
This is worth knowing before you go looking for a way to share one: the
format does not have it, and neither do the objects in memory — see
[the data model](https://otio-rust-docs.vercel.app/docs/concepts/data-model/) for how handles work here.
The reader does understand the `OTIO_REF_ID` tags and `SerializableObjectRef`
pointers that some files carry in metadata, and holds them to upstream's rule:
two objects declaring the same `OTIO_REF_ID` make the file unreadable, with
`Duplicated object reference while reading: near line N`, because a reference
could not say which of the two it meant.
#### Metadata
Nearly every object has a `metadata` dictionary, holding anything JSON can
hold, nested as deeply as you like, including other OTIO objects.
The core does nothing with it at all. It carries it, unchanged, so adapters
and tools can keep what the schema has no field for — which several of the
adapters here do.
**Namespace it.** Two workflows writing to the same file will collide unless
each keeps to its own key:
```json
"metadata": {
"my_playback_tool": { "loop": false },
"my_production_tracking_system": { "status": "IP", "owner": "rose" }
}
```
It is also where a new schema usually starts life: put the field in metadata,
see whether it earns its place, then move it.
#### What this implementation guarantees
- **Field order is upstream's**, not alphabetical and not this library's
convenience. Fields come out in the order upstream's own `write_to` methods
emit them, so a diff between a file written here and one written there is
empty rather than a reordering.
- **Indentation is four spaces** by default, as upstream's is. The format
permits minifying and the project recommends against it: gzip the file if
size matters, and keep it readable.
- **Round trips are byte-for-byte** on upstream's own sample documents, which
is how that first point is measured rather than asserted.
#### Where it is not quite JSON
Upstream configures RapidJSON with `kWriteNanAndInfFlag`, so a non-finite
number is written as a bare `NaN`, `Infinity` or `-Infinity` literal. None of
those is valid JSON, and a strict parser will refuse the file — **including
upstream's own**, unless it is the one doing the reading.
This library reproduces it exactly, in both directions: it writes those
literals for non-finite values and accepts them when reading, in the
spellings RapidJSON accepts (`Inf` as well as `Infinity`).
That is deliberate, and it is the project's rule in miniature. Writing a
tidier `null` instead would quietly change a document that somebody else's
tool produced, and agreeing with upstream is the only way to agree with every
other reader. The place to be surprised by this is here, in the
documentation, rather than in a file that will not load.
#### Naming
Files should end in `.otio`. Not `.json` — the suffix is how every adapter in
the ecosystem, this one included, decides what it is holding.
---
This page is adapted from the
[File Format Specification](https://github.com/AcademySoftwareFoundation/OpenTimelineIO/blob/main/docs/tutorials/otio-file-format-specification.md)
in upstream OpenTimelineIO's documentation, which is Apache-2.0 like this
project. The guarantees section describes this implementation and has no
counterpart upstream.
---
Page: https://otio-rust-docs.vercel.app/docs/concepts/otio-file-format/
## Extensions
### Extensions
> Work built on otio-rust's core that is not OpenTimelineIO, kept apart from it and off until a program turns it on.
Everything else on this site is OpenTimelineIO: the object model, the time
library, the file formats, ported from upstream and checked against upstream's
own tests. An **extension** is something more, built in the same repository on
the same core, that upstream OpenTimelineIO does not have.
Extensions are kept apart so nobody mistakes one for OpenTimelineIO:
- **Off until enabled.** An extension does nothing until a program turns it
on. While it is off, any schema it owns reads as an unknown schema and is
written back unchanged, as upstream OpenTimelineIO treats a schema whose
plugin is not loaded. A file written with an extension on still opens in
upstream's tools.
- **A module of its own in every language.** Importing OpenTimelineIO never
pulls an extension in. You import the extension's module separately.
- **Its own C header.** The functions are in the same `libotio`, prefixed
`otio_ext__`, and declared in `otio_ext_.h` beside `otio.h`.
| Extension | What it does |
| --- | --- |
| [Timeline sync](https://otio-rust-docs.vercel.app/docs/extensions/timeline-sync/) | Shares a timeline between programs while it is edited |
| [ORI Annotations](https://otio-rust-docs.vercel.app/docs/extensions/ori-annotations/) | Reads, writes and edits the ORI project's review annotations |
#### Turning one on
Every language has the same two calls on the core, by extension id, and each
extension's module has its own `enable` and `is_enabled`:
| Language | Core | Sync's module |
| --- | --- | --- |
| Rust | `otio_ext::enable("sync")` | `otio_ext_sync::install()`, then `otio_ext::enable(otio_ext_sync::ID)` |
| Python | `opentimelineio.ext.enable("sync")` | `from opentimelineio.ext import sync; sync.enable()` |
| C | `otio_ext_enable("sync", &error)` | `otio_ext_sync_enable(&error)`, from `otio_ext_sync.h` |
Enabling an extension that is already on does nothing. Enabling one turns on
the extensions it depends on first. Enabling is for the whole process and
cannot be undone, as registering a schema cannot be.
---
Page: https://otio-rust-docs.vercel.app/docs/extensions/overview/
### Timeline sync
> Sharing an OpenTimelineIO timeline between programs while it is edited, over Alchemist's Furnace or ORI Sync.
Timeline sync keeps one timeline the same in several programs while people
edit it: an editor in the browser, a reviewer on an iPad, a desktop editor,
RV or xStudio. It is otio-rust's first extension.
Two systems already do this. ORI Sync, from the ORI Annotations project, runs
between OpenRV and xStudio over RabbitMQ. Alchemist's Furnace is a WebSocket
server between Alchemist's clients. Both name every object with a GUID in
`metadata.sync.guid` and send changes as patches addressed by GUID. Timeline
sync is one implementation of that model, written once in Rust and reached
from every language, that speaks both systems' wire formats.
#### Where it stands
Sync is being built in phases, tracked in
[epic #115](https://github.com/alchemist-editor/otio-rust/issues/115):
1. **The extension itself**: sync can be enabled, and every SDK has its
module.
2. **Identity and patches**: GUIDs, patches made by transactions or by
comparing two versions of a timeline, and applying them.
3. **Sessions over Furnace**: a session that takes bytes in and hands bytes
out, with no networking of its own, Furnace's wire format, a WebSocket
client, and compression, in Rust and, through the C interface, in every
SDK.
4. **Furnace relays patches** instead of whole timelines.
5. **ORI's wire format** over RabbitMQ, for RV and xStudio, in Rust.
#### Turning it on
| Language | Import | Enable |
| --- | --- | --- |
| Rust | `otio-ext-sync` | `otio_ext_sync::install(); otio_ext::enable(otio_ext_sync::ID)?;` |
| Python | `from opentimelineio.ext import sync` | `sync.enable()` |
| C | `#include "otio_ext_sync.h"` | `otio_ext_sync_enable(&error);` |
| Go | `github.com/alchemist-editor/otio-rust/sdk/go/ext/sync` | `sync.Enable()` |
| Swift | `import OTIOExtSync` | `try Sync.enable()` |
| TypeScript | `import * as sync from "@alchemist-edit/otio/ext/sync"` | `await init(); sync.enable();` |
| C++ | `opentimelineio/ext/sync.hpp` | `otio::ext::sync::enable();` |
| C# | `using OpenTimelineIO.Ext.Sync;` | `SyncExtension.Enable();` |
| Objective-C | `#import ` | `OTIOExtSyncEnable(&error);` |
| Zig | `otio.ext.sync` | `try otio.ext.sync.enable();` |
#### Identity
Every object sync addresses carries a GUID in `metadata.sync.guid`: the
timeline, its stack, and every track, stack and item under it. Markers,
effects and media references travel inside the item that holds them. A GUID
is ordinary metadata, so a file carrying them opens anywhere OpenTimelineIO
does.
Sync gives an object a GUID when it needs one. Beginning a transaction
stamps every object that lacks one, and an object that shares a GUID with
one met earlier (usually a copy made with its metadata) gets a new one; the
first keeps it. You can also stamp a timeline yourself, read or set one
object's GUID, make a new GUID, and find an object by its GUID.
#### Patches
A change to a timeline travels as a list of patches, in Alchemist's Furnace
format: `insert_child`, `move_child`, `remove_child` and `set_property`,
each naming the objects it is about by GUID. A change too big for patches
(more than 500 of them, or more than 40% of the timeline's objects), a change
between two sequences, or one that goes back a generation, is sent as one
`replace_timeline` patch holding the whole timeline instead.
There are two ways to get patches, and both give the same ones.
**Compare two versions.** Diffing two snapshots, each a timeline with its
sequence id and generation as Furnace sends it, gives the patches between
them. It is a port of Furnace's own diff, checked against Furnace's golden
files in Rust and in every SDK, so both ends agree byte for byte.
```rust
use otio_ext_sync::{Snapshot, diff, patch};
// Two versions of one sequence, as Furnace sends them: in the second, clip B
// has moved in front of clip A. Every object carries its GUID in
// `metadata.sync.guid`, which is how a patch names it.
const BEFORE: &str = r#"{"sequenceId": "main", "generation": 1, "otioJson": {
"OTIO_SCHEMA": "Timeline.1", "name": "cut", "metadata": {"sync": {"guid": "cut"}},
"tracks": {"OTIO_SCHEMA": "Stack.1", "metadata": {"sync": {"guid": "tracks"}}, "children": [
{"OTIO_SCHEMA": "Track.1", "name": "V1", "kind": "Video", "metadata": {"sync": {"guid": "v1"}},
"children": [
{"OTIO_SCHEMA": "Clip.2", "name": "A", "metadata": {"sync": {"guid": "a"}}},
{"OTIO_SCHEMA": "Clip.2", "name": "B", "metadata": {"sync": {"guid": "b"}}}]}]}}}"#;
const AFTER: &str = r#"{"sequenceId": "main", "generation": 2, "otioJson": {
"OTIO_SCHEMA": "Timeline.1", "name": "cut", "metadata": {"sync": {"guid": "cut"}},
"tracks": {"OTIO_SCHEMA": "Stack.1", "metadata": {"sync": {"guid": "tracks"}}, "children": [
{"OTIO_SCHEMA": "Track.1", "name": "V1", "kind": "Video", "metadata": {"sync": {"guid": "v1"}},
"children": [
{"OTIO_SCHEMA": "Clip.2", "name": "B", "metadata": {"sync": {"guid": "b"}}},
{"OTIO_SCHEMA": "Clip.2", "name": "A", "metadata": {"sync": {"guid": "a"}}}]}]}}}"#;
fn main() -> Result<(), Box> {
otio_ext_sync::install();
otio_ext::enable(otio_ext_sync::ID)?;
let before = Snapshot::from_json(BEFORE)?;
let after = Snapshot::from_json(AFTER)?;
let (patches, stats) = diff(&before, &after)?;
// One move, of A to after B: a patch names objects by GUID.
println!("{}", patch::to_json_pretty(&patches));
println!("fell back to the whole timeline: {}", stats.fell_back);
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/diff-two-timelines/rust.rs
**Record a transaction.** Begin a transaction on a timeline, edit it with
any of the usual calls, and commit it for the patches. Rolling back puts the
timeline back as it was. The transaction remembers the timeline as it began
and diffs it against the timeline as it is, so its patches are exactly what
comparing the two versions gives.
Applying patches to a timeline changes it in place. The timeline keeps its
identity, and so does every track and item the patches keep, so anything
your program holds onto still refers to the same object afterwards. An
object the patches removed is gone, and one they added can be found by its
GUID.
```rust
use otio_ext_sync::document::{self, Transaction};
// A cut, as an .otio file would hold it. Nothing in it has a GUID yet.
const CUT: &str = r#"{
"OTIO_SCHEMA": "Timeline.1",
"name": "cut",
"tracks": {"OTIO_SCHEMA": "Stack.1", "children": [
{"OTIO_SCHEMA": "Track.1", "name": "V1", "kind": "Video", "children": [
{"OTIO_SCHEMA": "Clip.2", "name": "A"},
{"OTIO_SCHEMA": "Clip.2", "name": "B"}]}]}
}"#;
fn main() -> Result<(), Box> {
otio_ext_sync::install();
otio_ext::enable(otio_ext_sync::ID)?;
// Beginning a transaction gives every object a GUID, and remembers the
// timeline as it was: that snapshot is what a second program starts
// from.
let mut editor = otio_core::from_str(CUT)?;
let timeline = editor.root().ok_or("the cut has no root")?;
let transaction = Transaction::begin(&mut editor, timeline, "main", 1)?;
let mut viewer = otio_core::from_str(&transaction.before().timeline)?;
let shown = viewer.root().ok_or("the snapshot has no root")?;
// Edit as usual, then commit for the patches that say what changed.
if let Some(base) = editor.try_get_mut(timeline)?.base_mut() {
base.name = "fine cut".into();
}
let patches = transaction.commit(&mut editor)?;
println!("{}", otio_ext_sync::patch::to_json(&patches));
// The other copy applies them in place: what it held still names the
// same objects.
document::apply_to(&mut viewer, shown, &patches)?;
let name = viewer.try_get(shown)?.base().map(|base| base.name.clone());
println!("the viewer now shows {}", name.unwrap_or_default());
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/apply-patches/rust.rs
#### Noticing changes
A program that edits a timeline through code it does not own, such as an
editing operation or a script, can turn on the document's change journal.
The journal records which objects any edit touched. Committing it runs the
`document_changed` hook, with the GUIDs of the touched objects as its
`touched` argument, so a hook script can send the change on. The journal is
off unless you turn it on, and costs nothing while it is off.
The design is in
[ADR 0011](https://github.com/alchemist-editor/otio-rust/blob/main/docs/adr/0011-timeline-sync.md).
#### Sessions
A session joins a shared timeline and keeps a copy of it in step. It does no
networking itself: a program hands it what arrived and the session hands back
what to send and what happened. The `otio-ext-sync-transport` crate puts it on
a wire. Its first wire format is Furnace's protocol v2, and it has a blocking
WebSocket client that runs the whole loop:
```rust
use std::time::Duration;
use otio_ext_sync::session::{self, ApplyOptions, Config, Event, Session};
use otio_ext_sync_transport::websocket::Client;
let config = Config { token: join_token, ..Config::default() };
let mut client = Client::connect(&ws_url, Session::new(config))?;
while let Some(event) = client.next_event(Duration::from_secs(1))? {
// Applies timeline changes to `timeline` in place and runs the hooks.
session::handle(&mut document, timeline, &event, &ApplyOptions::default())?;
}
```
The host publishes with `session.publish(&mut document, timeline, "main")`,
which works out the patches since the last publish and sends them. Viewers
receive each change as patches and apply them in place, so handles to the
timeline, its tracks and its items keep working. A host that reconnects to a
Furnace holding an older generation than its own, because the connection
dropped as an edit went out, sends its timeline again.
##### Patches on the wire
When Furnace's welcome lists `timeline.patch`, the host sends each change as
a `patch` message holding the patches and the generations they go from and
to. Furnace applies them to its copy, so anyone who joins later still gets
the whole timeline in their welcome, and relays them to every client whose
hello listed `timeline.patch.receive` (sessions list it unless told not to).
Everyone else, older clients included, gets the timeline Furnace rebuilt, as
before. The host still sends the whole timeline for its first publish, every
64th change after that as a checkpoint, when the patches would be bigger
than the timeline, when the change replaces the timeline outright, and to a
Furnace that does not list `timeline.patch`.
If Furnace's copy is not the generation a patch starts from, or the patch
does not apply to it, Furnace answers `stale_patch` or `patch_rejected`, and
the host sends its whole timeline in reply. A viewer that receives patches
from a generation it does not have is out of sync: `receive` fails with an
out-of-sync error (a `ConnectionError` in Python, `IoError` over the C
interface), and the program reconnects to get the whole timeline in the
welcome. Furnace's conformance corpus, revision 5, holds a patch and the
timeline it makes, and Go and Rust must both reach the same timeline.
Everything a session sends travels on a channel with a delivery rule:
| Channel | Rule | Over Furnace |
| --- | --- | --- |
| timeline | in order | `patch`, or `snapshot`, from the host only |
| playback | newest wins: a playhead not yet sent is replaced | `presence` |
| presence | newest wins | `welcome` and `roster` |
| review | kept until acknowledged, refused or cancelled, sent again after a reconnect | `review_commit` |
| signal | sent once, never replayed | `signal` |
Only the host may change the timeline, and only a client with the
`review.comment.create` capability may commit a review comment. A session
refuses these itself rather than letting Furnace close the connection. For
the same reason `session.send` carries only Furnace's message types that have
no call of their own (`signal`, and from the host only `bin` and the LAN
messages), and review
comments are refused when an older Furnace chooses protocol v1, which has no
review ledger.
Selections and other extensions' channels need Furnace to relay them, which
it does not do yet.
Four hooks run around a session: `sync_before_send` before the host's
changes leave (a script that fails stops them), `sync_before_apply` before a
remote change is applied (a script that fails rejects it), `sync_after_apply`
after, and `sync_joined` once the join is done. The media linker runs on the
clips a remote change inserts.
##### From other languages
Every SDK has the same session, over the C interface. The program owns the
socket: open a WebSocket to Furnace with whatever your language offers,
connect the session, and then keep three things going. Send every frame the
session gives you (text frames as text, binary ones as binary), hand it every
frame that arrives, and act on each event it reports. Passing a `timeline` or
`joined` event to `handle` with your copy of the timeline applies it in
place and runs the hooks, as `session::handle` does in Rust.
A session is a number from `session_new`, released with `session_close`.
Its settings, its events and its state are JSON objects, in the field names
Furnace uses:
| | Fields |
| --- | --- |
| Settings | `token`, `identity` (`displayName`, `app`, `appVersion`, `credential`, `deviceId`, `clientKind`, `formFactor`, `deviceName`, `capabilities`), `encodings` (`br`, `gzip`); all optional |
| Event | `type`: `joined`, `timeline`, `timeline_removed`, `playhead`, `selection`, `roster`, `leader`, `review_relay`, `review_committed`, `review_rejected`, `error`, `ended` or `message`, and that event's fields |
| State | `joined`, `ended`, `sessionId`, `selfId`, `role`, `leader`, `peers`, `activeSequence`, `sequences`, `pendingReviews`, `version`, `encodings` |
Python wraps this in a `Session` class that takes and returns dictionaries,
and `poll_transmit` there returns `str` for a text frame and `bytes` for a
binary one.
```rust
use otio_ext_sync::session::{self, ApplyOptions, Config, Event, Playhead, Session};
use otio_ext_sync_transport::{Connection, Frame, Furnace, now_millis};
// The frame Furnace sends a viewer joining a session whose host has shared
// one timeline: the welcome, holding the timeline as it is.
const WELCOME: &str = r#"{"v":2,"type":"welcome","sessionId":"s_1","seq":0,"ts":0,"order":1,"payload":{"selfId":"p_2","role":"viewer","roster":[{"id":"p_1","displayName":"Host","role":"host","joinedAt":0,"connectionState":"connected","muted":false}],"session":{"id":"s_1","createdAt":0},"snapshot":{"sequenceId":"main","generation":1,"otioJson":{"OTIO_SCHEMA":"Timeline.1","name":"Coastline","metadata":{"sync":{"guid":"t-1"}},"tracks":{"OTIO_SCHEMA":"Stack.1","metadata":{"sync":{"guid":"s-1"}},"children":[{"OTIO_SCHEMA":"Track.1","name":"V1","kind":"Video","metadata":{"sync":{"guid":"v-1"}},"children":[{"OTIO_SCHEMA":"Clip.2","name":"Waves","metadata":{"sync":{"guid":"c-1"}}}]}]}}}}}"#;
// A real program writes each frame to its WebSocket to Furnace, or lets
// `otio_ext_sync_transport::websocket::Client` do it all.
fn send(connection: &mut Connection) -> Result<(), otio_ext_sync_transport::Error> {
while let Some(frame) = connection.poll_transmit(now_millis())? {
let kind = if frame.binary { "binary" } else { "text" };
println!("send a {kind} frame of {} bytes", frame.data.len());
}
Ok(())
}
fn main() -> Result<(), Box> {
otio_ext_sync::install();
otio_ext::enable(otio_ext_sync::ID)?;
// The session owns no socket. Open one to Furnace, then connect: the
// hello is the first frame to send.
let mut config = Config {
token: "join-code".to_string(),
..Config::default()
};
config.identity.display_name = "Ada".to_string();
let mut connection = Connection::new(Session::new(config), Furnace::new());
connection.connect();
send(&mut connection)?;
// Hand the session every frame that arrives. This is the one Furnace
// sends back.
connection.receive(&Frame::text(WELCOME))?;
// The program's copy of the timeline starts empty. Handling the events
// puts the host's timeline in it, and keeps it current from then on.
let mut document = otio_core::from_str(r#"{"OTIO_SCHEMA": "Timeline.1"}"#)?;
let timeline = document.root().ok_or("the document has no root")?;
while let Some(event) = connection.poll_event() {
let handled = session::handle(&mut document, timeline, &event, &ApplyOptions::default())?;
let kind = match event {
Event::Timeline(_) => "timeline",
Event::Joined { .. } => "joined",
_ => "other",
};
println!("{kind} {handled:?}");
}
println!(
"the viewer now shows {}",
document.try_get(timeline)?.name()
);
// Everyone in the session sees where this viewer's playhead is.
connection.session_mut().set_playhead(Playhead {
sequence_id: "main".to_string(),
frame: 24.0,
playing: true,
rate: 1.0,
})?;
send(&mut connection)?;
connection.session_mut().leave();
send(&mut connection)?;
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/join-a-session/rust.rs
##### Compression
Each side lists the encodings it accepts when it joins: the client in its
hello, the server in its welcome. Messages of 1 KiB or more go as Brotli or
gzip when the other side accepts it, in a binary frame whose first byte names
the encoding (1 for gzip, 2 for Brotli); everything else stays JSON text.
Furnace does not list any encodings yet, so today every frame is text; the
in-memory test room does, and CI publishes a table of what each encoding
saves on the repository's timelines. A 103 KiB timeline goes over the wire
as about 3 KiB.
##### Checked against Furnace
Furnace's conformance corpus, every frame it says a client or server sends,
is vendored into the crate's tests: each client frame is read and written
again and must match it value for value, and each server frame is read as what it
says. A CI job also builds furnaced and has a host and a viewer join a real
session.
#### Sessions over ORI Sync
The ORI profile speaks ORI Sync's wire, so a session can join the same
RabbitMQ session as OpenRV and xStudio running ORI's plugins. It is in Rust
for now. There is no server: every peer binds a queue to the session's
fanout exchange, `sync_session_`, and hears everything. The first peer
finds no master within two seconds and elects itself; later peers ask the
master for its state and join with it, and if the master stops answering
another peer takes over.
```rust
use std::time::Duration;
use otio_ext_sync::session::{self, ApplyOptions, Config, Session};
use otio_ext_sync_transport::{amqp::Client, ori};
let options = ori::Options::new("review-42");
let mut client = Client::connect("amqp://guest:guest@localhost:5672/", Session::new(Config::default()), options)?;
loop {
// `None` is a quiet second, not the end: keep going so the client can
// elect itself, answer joiners and send its heartbeat.
if let Some(event) = client.next_event(Duration::from_secs(1))? {
session::handle(&mut document, timeline, &event, &ApplyOptions::default())?;
}
}
```
Any peer may change the timelines, as in ORI. What ORI's peers send arrives
as patches like Furnace's, and what a session publishes goes out as ORI's
own edits (`SET_PROPERTY`, `INSERT_CHILD`, `MOVE_CHILD`, `REMOVE_CHILD`),
or as the whole timeline when ORI's edits cannot say it, such as inserting a
track with clips in it. A changed annotation clip goes out as ORI's
`REPLACE_ANNOTATION_COMMANDS`; see
[ORI Annotations](https://otio-rust-docs.vercel.app/docs/extensions/ori-annotations/#on-ori-s-wire). Several timelines, which one is active, the
playhead and ORI's display settings cross too. Selections are retired in
ORI, and review comments, leaders and muting have no ORI message, so sending
them fails as unsupported.
Snapshots and messages over 1 MB can be zlib-compressed, but ORI's own peers
cannot read either yet, so both are off unless `Options` turns them on.
Compressed input is always read.
##### Checked against ORI
ORI's own recordings of real sessions are vendored into the crate's tests
and replayed as ORI's recording player replays them; the session must end
with the same timelines, active timeline, playhead and display settings as
ORI's own peer, which CI checks against ORI itself. CI also runs sessions on
a real RabbitMQ, between Rust peers and with ORI's Python peer, with each
side as master.
---
Page: https://otio-rust-docs.vercel.app/docs/extensions/timeline-sync/
### ORI Annotations
> Reading, writing and editing the ORI project's review annotations on an OpenTimelineIO timeline.
ORI Annotations are drawings and notes made on a frame during review: paint
strokes, text and shapes. The schemas belong to the ASWF's ORI project, whose
[ORI Annotations](https://github.com/richardssam/ORIAnnotations) repository
defines them as an OpenTimelineIO plugin in Python and exchanges them live
between OpenRV and xStudio. This extension reads and writes the same objects,
under the same names, so a file written here opens in ORI's tools and the
other way round.
It is built on [timeline sync](https://otio-rust-docs.vercel.app/docs/extensions/timeline-sync/): enabling it
enables sync, and every object it creates carries a sync GUID.
#### Where it stands
The extension is being built in phases, tracked in
[epic #126](https://github.com/alchemist-editor/otio-rust/issues/126):
1. **Schemas, the annotation track and coordinates**, in Rust.
2. **Every language**: the C interface, a module in every SDK, and Python.
3. **Live review over Furnace**: review comments with their strokes, folded
into the host's timeline, and strokes in progress.
4. **ORI's wire**, for live review with RV and xStudio.
All four are here. One piece waits on the server: Furnace does not yet pass
strokes in progress along, so over Furnace only finished comments travel
today; over ORI's wire both do. See [Live review](#live-review).
#### Turning it on
| Language | Import | Enable |
| --- | --- | --- |
| Rust | `otio-ext-ori-annotations` | `otio_ext_ori_annotations::install(); otio_ext::enable(otio_ext_ori_annotations::ID)?;` |
| Python | `from opentimelineio.ext import ori` | `ori.enable()` |
| C | `#include "otio_ext_ori.h"` | `otio_ext_ori_enable(&error);` |
| Go | `github.com/alchemist-editor/otio-rust/sdk/go/ext/ori` | `ori.Enable()` |
| Swift | `import OTIOExtOri` | `try Ori.enable()` |
| TypeScript | `import * as ori from "@alchemist-edit/otio/ext/ori"` | `await init(); ori.enable();` |
| C++ | `opentimelineio/ext/ori.hpp` | `otio::ext::ori::enable();` |
| C# | `using OpenTimelineIO.Ext.Ori;` | `OriExtension.Enable();` |
| Objective-C | `#import ` | `OTIOExtOriEnable(&error);` |
| Zig | `otio.ext.ori` | `try otio.ext.ori.enable();` |
The id is `ori-annotations`. While it is off, `PaintStart.1` and the rest
read as unknown schemas and are written back unchanged, exactly as upstream
OpenTimelineIO does when ORI's plugin is not on its path; the annotation calls
refuse to run. With it on, ORI's files still come out byte for byte as they
went in.
#### Commands
An annotation is a list of commands. A stroke is three or more commands that
share one `uuid`: a `PaintStart.1` with the colour, brush and who drew it, one
or more `PaintPoint.1` batches of points, and a `PaintEnd.1`. Text and shapes
are one command each.
| Schema | Holds |
| --- | --- |
| `PaintStart.1` | Opens a stroke: `uuid`, `rgba`, `brush`, `type`, `participant_hash`, `friendly_name`, and ORI's ghosting and hold fields |
| `PaintPoint.1` | A batch of points: a `PaintVertices.1` with parallel `x`, `y`, `size` and `alpha` arrays |
| `PaintEnd.1` | Closes a stroke |
| `TextAnnotation.1` | Text at a `position`, with `font`, `font_size`, `scale`, `rotation` and `spacing` |
| `EllipseAnnotation.1`, `RectangleAnnotation.1` | A shape in the box from `min` to `max`, with outline and fill colours |
| `ArrowAnnotation.1` | An arrow from `start` to `end` |
The batch of points is written `PaintPoint.1`, singular, as ORI's code and
files write it; ORI's prose sometimes says `PaintPoints.1`. ORI's playback
events (`Play.1`, `SetCurrentFrame.1`, `SyncPlayback.1` and the others) are
registered too, so ORI's recordings read as typed objects.
In Rust, `Command::read` gives a typed copy of a command and keeps every field
it does not name; a new command built with a constructor such as
`PaintStart::new` has the same defaults ORI's Python class gives it.
Every other language passes commands as JSON text: a list of ORI objects,
exactly as an annotation clip's `annotation_commands` holds them in a file.
A field a command leaves out takes the default ORI's class gives it, as
reading it with ORI's plugin would, so `{"OTIO_SCHEMA": "PaintStart.1"}` has
ORI's `"brush": "circle"`. Python's calls also take a list of dicts.
#### Where annotations live
A frame's annotations sit on a one-frame clip in a track named `Annotations`,
beside the media:
```text
Stack
├─ Track "V1" Clip "shot" metadata.sync.guid = "shot"
└─ Track "Annotations" Gap Clip "Annotation_10"
source_range: frame 10, one frame long
metadata.clip_guid = "shot"
metadata.annotation_commands = [PaintStart, PaintPoint, PaintEnd, …]
```
The annotation clip's start time is the annotated frame of the media clip,
and `clip_guid` names that clip by its sync GUID. This is the form ORI's sync
protocol and Alchemist's review design both chose. It follows ORI exactly,
including one surprise: an annotation clip for a frame later than the end of
the track is placed after a gap at its own frame, but one for an earlier frame
is simply added at the end, so annotations are found by `clip_guid` and frame,
never by where they sit on the track.
Reading also understands two older forms ORI has written: its review export,
where the track is named after the review and each clip names the media in
`annotated_clip_name`, and `AnnotationEffect.1`, an effect on the media clip
that holds its commands. Neither is written.
```rust
use opentime::RationalTime;
use otio_ext_ori_annotations::annotations;
use otio_ext_ori_annotations::command::{self, Command, PaintStart, TextAnnotation, Vertices};
use otio_ext_ori_annotations::coords::ImageSize;
// A cut with one shot, which sync knows by the GUID `shot`.
const CUT: &str = r#"{
"OTIO_SCHEMA": "Timeline.1",
"name": "review",
"tracks": {"OTIO_SCHEMA": "Stack.1", "children": [
{"OTIO_SCHEMA": "Track.1", "name": "V1", "kind": "Video", "children": [
{"OTIO_SCHEMA": "Clip.2", "name": "shot",
"metadata": {"sync": {"guid": "shot"}}}]}]}
}"#;
fn main() -> Result<(), Box> {
// ORI Annotations is an extension; enabling it enables sync too.
otio_ext_ori_annotations::install();
otio_ext::enable(otio_ext_ori_annotations::ID)?;
let mut document = otio_core::from_str(CUT)?;
let timeline = document.root().ok_or("the cut has no root")?;
// The Annotations track, and the clip on it for frame 10 of the shot.
let track = annotations::ensure_track(&mut document, timeline)?;
let frame = annotations::ensure_clip(&mut document, track, "shot", RationalTime::new(10.0, 24.0))?;
// ORI's coordinates are height-normalised with the origin at the centre,
// so convert from the pixels of the 1920x1080 picture.
let hd = ImageSize::new(1920.0, 1080.0);
let points: Vec<(f64, f64)> = [(200.0, 200.0), (960.0, 540.0), (1720.0, 880.0)]
.into_iter()
.map(|(x, y)| hd.pixels_to_ori(x, y))
.collect();
// A stroke is a PaintStart, its points and a PaintEnd sharing one uuid.
let stroke = command::stroke(
PaintStart {
rgba: Some([1.0, 0.25, 0.25, 1.0]),
participant_hash: Some("reviewer-1".into()),
timestamp: Some(command::timestamp_now()),
..PaintStart::new(otio_ext_sync::guid::new_guid())
},
Vertices::with_size(&points, 0.01),
);
annotations::add_commands(&mut document, frame, &stroke)?;
let (x, y) = hd.pixels_to_ori(960.0, 620.0);
let label = Command::Text(TextAnnotation {
text: Some("Eyeline is late".into()),
position: Some([x, y]),
rgba: Some([1.0, 1.0, 1.0, 1.0]),
..TextAnnotation::new(otio_ext_sync::guid::new_guid())
});
annotations::add_commands(&mut document, frame, &[label])?;
// Read them back: every annotation on the shot, frame by frame.
for annotation in annotations::for_clip(&document, timeline, "shot")? {
println!("{:?}: {} commands", annotation.frame, annotation.commands.len());
}
// The file is plain OpenTimelineIO with ORI's schemas in it, as ORI's
// own Python writes it.
println!("{}", otio_core::to_string_pretty(&document, 4)?);
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/annotate-a-frame/rust.rs
The calls, named as in Rust's `annotations` module and in every language's
own case:
| Call | Does |
| --- | --- |
| `find_track`, `ensure_track` | Finds the timeline's annotation track, or adds one |
| `find_clip`, `ensure_clip` | Finds the annotation clip for a media clip GUID and frame, or adds one |
| `add_commands` | Adds commands at the end of an annotation clip |
| `replace_commands` | Replaces every command with a uuid, such as one whole stroke, in place |
| `remove_commands` | Removes every command with any of a list of uuids |
| `set_commands` | Replaces all of a clip's commands, as ORI's replace does; an empty list clears it |
| `list`, `list_for_clip`, `list_at_frame` | The object holding every annotation on the timeline, or those for a clip or a frame (`for_clip` and `at_frame` in Rust, which also has `by_participant`) |
| `describe`, `commands` | What an annotation says about itself, as JSON, and its commands |
Edits go by uuid, so one person's edit never disturbs another person's
strokes.
#### Live review
Annotations travel between the people in a [sync session](https://otio-rust-docs.vercel.app/docs/extensions/timeline-sync/)
in two ways.
**A finished comment** is a note with the strokes drawn for it. A reviewer
commits it with `commit`, giving the session, their copy of the timeline,
the note, the commands, and an anchor saying where they drew:
```json
{"sequenceId": "main", "capturedGeneration": 1, "clipGuid": "main/shot",
"sourceTime": {"value": 10, "rate": 24},
"capturedSequenceTime": {"value": 10, "rate": 24},
"capturedAt": 1787742000000}
```
The comment goes on the session's review ledger, so it is sent again after
every reconnect until Furnace stores it. Furnace then relays it to everyone
as a `review_relay` event, and the host folds it into its timeline with
`fold`: the strokes merge into the annotation clip for that source frame,
and the comment is recorded on the clip:
```text
Clip "Annotation_10"
metadata.clip_guid = "main/shot"
metadata.annotation_commands = [PaintStart s1, PaintPoint s1, PaintEnd s1]
metadata.note = "Too dark here." the comments' notes, joined
metadata.status = "open"
metadata.alchemist.comments = [{commentId, authorUserId, bodyMarkdown,
commandUuids: ["s1"], state, revision}]
```
Folding the same comment twice changes nothing, and `fold` says so. A
stroke belongs to the user whose comment added it (`owner_of`): `commit`
and `fold` both refuse a comment that would replace someone else's stroke,
or a stroke no comment owns. The host then publishes its timeline as usual,
so everyone sees the annotation.
**A stroke in progress** is what ORI calls a `PARTIAL`: the commands drawn so
far, shown to the others while the pen is still down and then dropped. It
never reaches a timeline. `send_partial` sends one, and `read_partial` reads
one out of a session event:
```json
{"clip_guid": "main/shot", "frame": 10, "fps": 24,
"events": [{"OTIO_SCHEMA": "PaintStart.1", "uuid": "s2"}]}
```
Over Furnace a stroke in progress has nowhere to go yet: Furnace closes the
connection on a message type it does not know, so the session refuses to
send one (an unsupported failure) rather than lose the connection. Over
ORI's wire it travels as `Annotation.1/PARTIAL`, as RV and xStudio send it.
```rust
use otio_ext_ori_annotations::{annotations, command, review};
use otio_ext_sync::session::{Config, Event, Session};
use otio_ext_sync_transport::{Connection, Frame, Furnace, now_millis};
// The cut both sides have: one shot, which sync knows by the GUID main/shot.
const CUT: &str = r#"{"OTIO_SCHEMA":"Timeline.1","name":"review","metadata":{"sync":{"guid":"main"}},
"tracks":{"OTIO_SCHEMA":"Stack.1","children":[{"OTIO_SCHEMA":"Track.1","name":"V1","kind":"Video",
"children":[{"OTIO_SCHEMA":"Clip.2","name":"shot","metadata":{"sync":{"guid":"main/shot"}},
"source_range":{"OTIO_SCHEMA":"TimeRange.1",
"start_time":{"OTIO_SCHEMA":"RationalTime.1","value":0,"rate":24},
"duration":{"OTIO_SCHEMA":"RationalTime.1","value":48,"rate":24}}}]}]}}"#;
// What Furnace tells a reviewer who joins: who they are, and that they may comment.
const WELCOME: &str = r#"{"v":2,"type":"welcome","sessionId":"s_1","seq":0,"ts":0,"order":1,"payload":{"selfId":"p_2","selfUserId":"u_bea","role":"viewer","roster":[],"session":{"id":"s_1","createdAt":0}}}"#;
// Where the reviewer drew: frame 10 of the shot, as the program shows it.
const ANCHOR: &str = r#"{"sequenceId":"main","capturedGeneration":1,"clipGuid":"main/shot",
"sourceTime":{"value":10,"rate":24},"capturedSequenceTime":{"value":10,"rate":24},
"capturedAt":1787742000000}"#;
// One stroke: a PaintStart, its points and a PaintEnd sharing a uuid.
const STROKE: &str = r#"[{"OTIO_SCHEMA":"PaintStart.1","uuid":"s1","rgba":[1,0.25,0.25,1]},
{"OTIO_SCHEMA":"PaintPoint.1","uuid":"s1","points":{"OTIO_SCHEMA":"PaintVertices.1",
"x":[-0.5,0,0.5],"y":[0.2,0,-0.2],"size":[0.01,0.01,0.01],"alpha":[1,1,1]}},
{"OTIO_SCHEMA":"PaintEnd.1","uuid":"s1"}]"#;
// What Furnace tells the host when it opens the session.
const HOST_WELCOME: &str = r#"{"v":2,"type":"welcome","sessionId":"s_1","seq":0,"ts":0,"order":1,"payload":{"selfId":"p_1","selfUserId":"u_ari","role":"host","roster":[],"session":{"id":"s_1","createdAt":0}}}"#;
// The comment as Furnace relays it once it has stored it, to everyone in
// the session. A real host reads this from its own socket.
fn relay() -> String {
format!(
r#"{{"v":2,"type":"review_commit","sessionId":"s_1","senderId":"p_2","seq":1,"ts":0,"order":2,"payload":{{"clientMutationId":"m1","commentId":"c1","authorUserId":"u_bea","authorPresenceId":"p_2","serverOrder":1,"revision":1,"createdAt":0,"anchor":{ANCHOR},"bodyMarkdown":"Too dark here.","annotationCommands":{STROKE}}}}}"#
)
}
// Stands in for the network: a real program writes every frame the session
// has waiting to its WebSocket to Furnace.
fn send(connection: &mut Connection) -> Result<(), otio_ext_sync_transport::Error> {
while let Some(frame) = connection.poll_transmit(now_millis())? {
println!("send {} bytes", frame.data.len());
}
Ok(())
}
fn joining(token: &str, capability: &[&str]) -> Connection {
let mut config = Config {
token: token.to_string(),
..Config::default()
};
config.identity.capabilities = capability.iter().map(|name| name.to_string()).collect();
Connection::new(Session::new(config), Furnace::new())
}
fn main() -> Result<(), Box> {
// ORI Annotations is an extension; enabling it enables sync too.
otio_ext_ori_annotations::install();
otio_ext::enable(otio_ext_ori_annotations::ID)?;
// The reviewer joins with the right to comment.
let mut reviewer = joining(
"join-code",
&["review.comment.create", "review.annotation.create"],
);
reviewer.connect();
send(&mut reviewer)?;
reviewer.receive(&Frame::text(WELCOME))?;
while reviewer.poll_event().is_some() {}
// Commit the stroke with a note. The session sends it again after every
// reconnect until Furnace stores it, so it is not lost.
let seen = otio_core::from_str(CUT)?;
let id = review::commit(
reviewer.session_mut(),
&seen,
seen.root().ok_or("the cut has no root")?,
&review::Anchor::from_json(ANCHOR)?,
"Too dark here.",
&command::from_json(STROKE)?,
"",
)?;
println!("committed {id}");
send(&mut reviewer)?;
// The host hears the stored comment and folds it into its timeline.
let mut host = joining("host-token", &["review.commit.receive"]);
host.connect();
send(&mut host)?;
host.receive(&Frame::text(HOST_WELCOME))?;
host.receive(&Frame::text(relay()))?;
let mut document = otio_core::from_str(CUT)?;
let timeline = document.root().ok_or("the cut has no root")?;
while let Some(event) = host.poll_event() {
// A stroke still being drawn: draw it over the picture, then drop it.
if let Some(partial) = review::read_partial(&event)? {
println!("drawing {} commands", partial.commands.len());
continue;
}
let Event::ReviewRelay(relay) = event else {
continue;
};
let folded = review::fold(&mut document, timeline, &relay)?;
let about = annotations::describe(&document, folded.clip)?;
println!(
"folded {} already there: {}",
about.to_json(&document),
folded.already
);
let owner = review::owner_of(&document, folded.clip, "s1")?;
println!("the stroke belongs to {owner:?}");
}
Ok(())
}
```
Source: https://github.com/alchemist-editor/otio-rust/blob/main/site/content/samples/share-an-annotation/rust.rs
##### Merging
Several people annotating one frame is the normal case, so edits merge
rather than overwrite. These are the rules Alchemist's review uses, and the
calls are there for any program that keeps annotations itself:
| Call | Does |
| --- | --- |
| `merge_commands` | Merges commands into a clip's by uuid: a uuid it has is replaced where it stands, a new one goes at the end |
| `replace_by_kind` | Replaces a clip's strokes, or its text, with new ones, keeping the kinds the new list lacks; an empty list clears all of them. This is how ORI's `REPLACE_ANNOTATION_COMMANDS` reads |
| `merge_duplicates` | Folds every second annotation clip for one clip and frame into the first |
| `commit` | Commits a comment with its strokes on a session |
| `fold` | Folds a relayed comment into the timeline, saying whether it was there already |
| `owner_of` | The user whose comment added a stroke |
| `send_partial`, `read_partial` | Sends a stroke in progress, and reads one from a session event |
##### On ORI's wire
With the [ORI profile](https://otio-rust-docs.vercel.app/docs/extensions/timeline-sync/#sessions-over-ori-sync),
a session talks to RV and xStudio directly. An annotation edit from them
(`REPLACE_ANNOTATION_COMMANDS`, or an annotation clip in `INSERT_CHILD`)
arrives as a timeline change, merged by the rules above. Publishing a
timeline whose annotation clip changed sends `REPLACE_ANNOTATION_COMMANDS`
for that clip. Strokes in progress go both ways.
ORI's roles still apply. A peer is a driver unless its options say
otherwise, and only a driver may clear a clip's annotations. A peer joined
as a reviewer can send strokes in progress, but cannot yet publish finished
ones, because sync lets only the host or a driver publish timeline changes.
#### Coordinates
ORI places every point in height-normalised coordinates with Y up and the
origin at the centre of the picture: `y` runs from -0.5 at the bottom to 0.5
at the top, and `x` from minus to plus half the aspect ratio, ±8/9 on a 16:9
frame.
```text
x = (px - width / 2) / height
y = -(py - height / 2) / height
```
`coords::ImageSize` converts to and from pixels, a port of ORI's `coords.py`
that gives ORI's own test chart its numbers bit for bit; every language has
it as `pixels_to_ori` and `ori_to_pixels`. Where a clip's media
has `available_image_bounds`, `ori_to_bounds` and `bounds_to_ori` map
ORI's space onto OpenTimelineIO's: the bounds' centre is ORI's origin and
their height ORI's unit. Without bounds ORI assumes 16:9.
Drawing the strokes is a renderer's job, not this extension's: it stores and
moves the commands.
#### Checked against ORI
- ORI's own `annotationreview.otio` and test chart round-trip byte for byte,
with the extension on and off.
- A review built through these calls is byte for byte the file ORI's
`SyncManager` and annotation builder write for the same strokes, generated
with ORI's plugin on upstream OpenTimelineIO 0.18.1.
- Every command in ORI's test chart reads as a typed command and writes back
the same.
- ORI's own test suite, run on otio-rust's Python bindings with ORI's plugin,
gives exactly what it gives on upstream OpenTimelineIO 0.18.1.
- ORI's plugin and this extension work in the same Python program: ORI's
classes still read ORI's commands, this extension finds them, and the file
writes back byte for byte.
- The same review is built and edited by each SDK's own tests.
- All 450 strokes in progress in ORI's 26 recorded sessions read as typed
commands, and every recorded session's end state reads back the same.
- A comment committed by a reviewer over a loopback Furnace room, and the
server's own conformance fixture, fold into the host's timeline, in Rust,
through the C interface, and in every SDK.
See [ADR 0012](https://github.com/alchemist-editor/otio-rust/blob/main/docs/adr/0012-ori-annotations.md)
for the design.
---
Page: https://otio-rust-docs.vercel.app/docs/extensions/ori-annotations/
## Internals
### How the SDKs are made
> One description of the C ABI, read out of its own source, becomes every binding.
Nobody writes these SDKs by hand, and nobody writes an interface description
by hand either.
```text
crates/otio-capi/src the C ABI, in Rust
│
│ otio-sdk-model reads it
▼
sdk/api.json every call, its parameters, its docs, its role
│
│ otio-sdk-gen writes one SDK per language
▼
sdk/go sdk/swift sdk/zig sdk/cpp sdk/csharp sdk/objc … and this site's reference pages
```
The description is committed as [`sdk/api.json`](https://github.com/alchemist-editor/otio-rust/blob/main/sdk/api.json),
so a change to the C ABI shows up as a readable diff rather than as a
surprise inside generated code. `cargo test -p otio-sdk-gen` fails when what
is checked in is not what the generator would write today, which is what CI
runs.
Seven kinds of drift fail that check rather than reaching a user: a C
function that fits no naming convention, one whose body does something the
description cannot express, a schema the generator does not know about, two
functions that would collide in a target language, a call that consumes a
document, a struct whose computed layout disagrees with the size the ABI
asserts, and generated files that differ from the ones committed.
#### What "idiomatic" means here
A generated API that reads like a transliteration of C is a failure even if
every function is present. Each backend owns the shape of its own language:
Go gets methods, embedded structs for the schema ladder, slices, strings, an
`error` and a sentinel for "there is no value" — not two-pass out-parameters
and status codes.
And the shape of the API itself — what things are called, which members
exist, which are properties and which are methods — follows upstream
OpenTimelineIO's own bindings for that language rather than being invented
here. Where a generated SDK departs from upstream's shape, the departure is
written down in
[ADR 0003](https://github.com/alchemist-editor/otio-rust/blob/main/docs/adr/0003-sdk-generation.md)
with its reason.
Some of the C ABI is plumbing no SDK user should see. The options structs
open with a `struct_size` field so that the library can tell which header a
caller was built against; every SDK fills it in as it hands the options over
and leaves it out of its own `ReadOptions` and `WriteOptions`, except Zig,
whose options are the C struct itself and so keep the field with the package
filling it in. The defaults come from `otio_read_options_init` and
`otio_write_options_init`, which fill a struct rather than return one, so an
SDK sets the size on the struct it passes before the call. The generator does
that for any out-parameter of a struct that carries its size, and the SDKs'
own `ReadOptionsDefault` calls keep their names.
Zig is the interesting exception, because it has no upstream binding to copy.
It is also the one target that keeps the document visible, and the reasoning
is worth reading if you want to know what the other bindings are hiding.
#### Every SDK is held to the same behaviour
Each SDK tests its own surface in its own language, which catches a binding
that is broken but not one that works and quietly disagrees with the others.
So a small set of scenarios is written once, as data beside the description
([`sdk/conformance.json`](https://github.com/alchemist-editor/otio-rust/blob/main/sdk/conformance.json)),
and rendered by the generator into every SDK's own test framework: build this
timeline and it writes exactly this JSON; a removed object's handle is stale;
an object from another timeline is refused before the library is asked, and
the other timeline is left whole; an object that already has a parent, or
whose handle has gone stale, is refused before its timeline is brought over,
so the two never merge. Each
language's CI job runs them with the rest of its tests.
A failure in them is named by kind — the library's status, or the binding's
own refusal — never by message text, so a scenario asks the same question in
Go's `error`, Swift's `throws` and Zig's error unions. Zig keeps the document
visible, so the scenarios about what hiding it obliges a binding to do are
marked as not applying there, and the generator checks that the ones every
language runs mean the same thing on both sides of that fork.
That rule means every SDK lets you recognise the refusal of another
timeline's object without reading its message:
| SDK | How to recognise it |
| --- | --- |
| Go | `errors.Is(err, otio.ErrOtherTimeline)` |
| Swift | `error.isOtherTimeline` on `OTIOError` |
| Zig | `error.ForeignObject` |
| TypeScript | `err instanceof OtherTimelineError` |
| C++ | `catch (const otio::OtherTimelineError&)` |
| C# | `catch (OtherTimelineException)` |
| Objective-C | `OTIOIsOtherTimeline(error)` |
In Swift, C++, C# and Objective-C the refusal still carries the
invalid-argument status it always had, so code that checked for that keeps
working.
#### This site is generated from the same file
The [reference section](https://otio-rust-docs.vercel.app/reference/) reads `sdk/api.json` at build time: the
groups, the calls, their parameters and their documentation all come from the
description, and each C declaration is lifted verbatim out of the committed
header rather than reassembled from parts. A page here cannot describe a call
the library does not have.
The prose and the code samples are written by hand, because they are
explanation rather than interface. What keeps the samples honest is that
each one is compiled — or, for Python, run — by its own language's toolchain
against the real SDK, in the same CI job that tests that SDK. A change that
leaves a sample describing an API that no longer exists fails the build that
made it. A build-time check also requires every sample to exist in every
language the switcher offers, or to say in writing why it cannot.
---
Page: https://otio-rust-docs.vercel.app/docs/internals/how-the-sdks-are-made/