StableMotion: Repurposing Diffusion-Based Image Priors for Motion Estimation

This is the official repo for paper StableMotion: Repurposing Diffusion-Based Image Priors for Motion Estimation

Setup

  1. Clone the code repo.
  2. Create your environment from requirements.txt.
  3. Download DIR-D and RS-Real. Put them into StableMotion_SIR and StableMotion_RSC respectively.

StableMotion for Stitched Image Rectangling (SIR)

Inference

  1. Download the checkpoints of StableMotion_SIR
  2. Run cd StableMotion_SIR && sh sample.sh. You may want to change this file to modify the inference configurations.
  3. Run sh metrics.sh to evaluate the results.

Training

  1. Replicate RecDiffusion to get pseudo labels, i.e., the flow labels generated by RecDiffusion on the training set of DIR-D. Put and rename it into StableMotion_SIR/MDM_Flow.
  2. Run cd StableMotion_SIR && sh train.sh. You may want to change this file to modify the training configurations. The default configuration requires approximately 80 GB of VRAM per card.

StableMotion for Rolling Shutter Correction (RSC)

Inference

  1. Download the checkpoints of StableMotion_RSC
  2. Run cd StableMotion_RSC && sh sample.sh. You may want to change this file to modify the inference configurations.
  3. Run sh metrics.sh to evaluate the results.

Training

Run cd StableMotion_RSC && sh train.sh. You may want to change this file to modify the training configurations. The default configuration requires approximately 40 GB of VRAM per card.

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ziyiwhat/StableMotion

Finetuned
(182)
this model