A screenshot has no time axis.

The problem

why “AI, copy this animation” doesn’t work

A still frame shows you every shape, every colour and every position. It tells you nothing about time. Duration, easing, the gap between two beats, the period of a loop — none of it survives a screenshot. So a vision model can describe an animation it cannot reproduce, because reproduction needs the one axis a picture doesn’t have.

motiscope does exactly the one thing a model can’t, and refuses to do anything a model does better.

Everything below is in service of that split.

Two artifacts, wildly different costs

this is why the analysis can afford to be dense

Frame count tracks motion complexity, not video length. A 10-second clip typically yields about ten frames. The numeric pass can be as dense as physics allows, because it costs nothing to look.

Pass A — the motion-energy curve

one decode, 32×32 grayscale thumbnails, mean absolute difference

ffmpeg -i <video> -vf "fps=<native>,scale=32:32,format=gray" -f rawvideo -

Energy at frame i is the mean absolute difference against frame i−1. No pixels ever reach Python at full size. That series is the source of truth for everything temporal.

Localized energy — why small elements don’t vanish

A whole-frame average is the wrong statistic. One button sliding across a 1280×720 page moves a rounding error’s worth of pixels; average that over the frame and it reads as nothing happened.

So the frame is split into an 8×8 grid, and the signal is the mean of the most active 10% of cells — not of all of them. On the ambient loop, straight from its motion.json:

A 4× amplification of exactly the thing you care about. This wasn’t theoretical: three cards animating on a 1280×720 canvas were being classified as hold before it existed.

The hold threshold, and why it isn’t a fraction of the peak

motion_ref = max(percentile_75(energy), 0.05)
threshold  = max(0.05, 0.18 × motion_ref)

The 75th percentile, not the maximum. A hard cut or a full-screen fade produces near-maximal energy. Anchor the threshold to the peak and one 0.27-second transition raises the bar above every real element animation in the clip — which then all report as hold.

Pass B — ffmpeg signal filters

a second decode, five filters, parsed from stderr

freezedetect is deliberately demoted. It measures whole-frame stillness, so on a large canvas it fires while a small element is mid-animation. It may confirm a hold the energy curve already found; it may never turn a move into a hold. Letting it do so was a bug.

Easing — the part people assume is impossible

you do not need to see the element to measure how it accelerated

Energy is mean pixel change per frame, which is a proxy for speed. Integrate speed and you get displacement. So:

The running integral of the energy curve, normalised, is the segment’s displacement-vs-time curve — which is exactly what a CSS timing function describes.

Normalise it, then match against a library of seven real curves and keep the closest by RMSE. The output is a genuine cubic-bezier, not a coarse label — usable directly as CSS cubic-bezier(...), a Framer array, or a GSAP CustomEase.

• The measured curve lies on top of the ground truth. tests/test-ease.mp4 is synthesised so that position ∝ t², and the integral recovers t² from nothing but frame differences. The mechanism is sound.
• The fitted bezier is the nearest curve in the library, not a perfect match — rmse 0.047. motiscope recovers the easing class and shape faithfully. It does not recover the author’s exact control points, and it never claims to.

Frame curation

the only step that costs money

Candidate timestamps are the union of t=0, t=end, every segment boundary, the energy-curve extrema (keyposes and peaks), and a sparse uniform backbone. Then each candidate is reduced to a grayscale thumbnail of DEDUP_THUMB = 16 squared, and dropped if it is within a mean-absolute-difference of DEDUP_THRESHOLD = 2.0 of the frame already kept. What survives is evenly sampled down to the preset’s budget.

Auto-decompose

For clips of 8 seconds or more with two or more motion beats, the budget follows the motion instead of the clock. Every move segment is drilled densely; every hold gets one representative frame. Move segments are allocated frames in proportion to duration × peak_energy, so a fast, intense beat earns more frames than a slow one.

Loop detection

autocorrelation, with three guards against wishful thinking

The strongest autocorrelation lag above a minimum period is the candidate loop length. A loop is only reported if the clip is mostly moving (a static shot with one periodic blip is not a loop), the peak correlation is at least 0.7, and the second harmonic also correlates — a one-off burst does not repeat at 2×.

Two caveats stated up front. Energy is speed-based, so back-and-forth motion reports a period that is half the visual cycle; the caller is told, so recreation can choose repeat versus yoyo. And a constant-velocity loop — a uniform rotation — has flat energy and is intentionally not detected. There is no periodicity in the signal to find.

What motiscope deliberately does not do

the restraint is the feature

It does not classify animation types. There is no enum of fade | slide | scale. The model looks at the frames and names what it sees — a mask reveal, a path draw, a morph, a 3D flip, a text split, particles, a clip-path wipe. Any hand-coded taxonomy would be a ceiling on what can be recreated.

Two features were built and then removed for exactly that reason:

• Per-element tracking. A sweeping element and a staggered group are indistinguishable in frame-differencing. Doing it properly needs optical flow. It was reporting seven “elements” for one sliding box. Reverted.
• Motion-direction detection. The model reads direction off two frames instantly and correctly. A grid-energy centroid was a worse oracle. Removed.

Measured vs estimated

the distinction is load-bearing — trust the first column

One known artifact: a very gentle ease-in can read as a short leading hold, because sub-pixel motion is invisible in a 32×32 thumbnail. The dominant easing is still recovered — check the first frames.

Verify it yourself

the repo ships clips whose ground truth is known by construction

bash tests/make_test_clip.sh
python3 -m unittest tests.test_analyze_motion tests.test_integrations