StableMotion: Repurposing Diffusion-Based Image Priors for Motion Estimation
This is the official repo for paper StableMotion: Repurposing Diffusion-Based Image Priors for Motion Estimation
Setup
- Clone the code repo.
- Create your environment from
requirements.txt
. - Download DIR-D and RS-Real. Put them into
StableMotion_SIR
andStableMotion_RSC
respectively.
StableMotion for Stitched Image Rectangling (SIR)
Inference
- Download the checkpoints of
StableMotion_SIR
- Run
cd StableMotion_SIR && sh sample.sh
. You may want to change this file to modify the inference configurations. - Run
sh metrics.sh
to evaluate the results.
Training
- Replicate RecDiffusion to get pseudo labels, i.e., the flow labels generated by RecDiffusion on the training set of DIR-D. Put and rename it into
StableMotion_SIR/MDM_Flow
. - Run
cd StableMotion_SIR && sh train.sh
. You may want to change this file to modify the training configurations. The default configuration requires approximately 80 GB of VRAM per card.
StableMotion for Rolling Shutter Correction (RSC)
Inference
- Download the checkpoints of
StableMotion_RSC
- Run
cd StableMotion_RSC && sh sample.sh
. You may want to change this file to modify the inference configurations. - Run
sh metrics.sh
to evaluate the results.
Training
Run cd StableMotion_RSC && sh train.sh
. You may want to change this file to modify the training configurations. The default configuration requires approximately 40 GB of VRAM per card.
- Downloads last month
- -
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
๐
Ask for provider support
Model tree for ziyiwhat/StableMotion
Base model
stabilityai/stable-diffusion-2