mmdance: MMD dances on any Mixamo-rigged Blender character

English | 简体中文

mmdance makes a Mixamo-rigged character perform a MikuMikuDance (MMD) dance in Blender. It works with AI-generated (image-to-3D) characters from Tripo or Meshy, with Mixamo and AccuRig rigs, or with your own rig mapped by bone name. You give it the character's .blend and an MMD motion package: the VMD dance motion, the VMD camera, the facial VMD and the song. It builds the animated scene and renders a finished 1080p video, headless, from one config file. Each step is cached under a key of its inputs, so a rebuild recomputes only what changed and reproduces the rest.

Demo

https://github.com/user-attachments/assets/532d0b6d-a07f-4ca3-ac91-04a43aa5a862

projects/claude_chika_v3: a Tripo-generated anime character in a layered ball gown dances akomni's MMD motion チカっとチカ千花っ♡ (94 s) with the package's camera, facial track and song, in a student-council room lit by the sun through its windows. build.py makes the whole scene from the untouched character file: the retargeted dance, the gown and the hair simulated lock by lock, hands with drawn-hand finger joints, a face rebuilt as geometry (a mouth that opens, eyes in layers, shape keys keyed from the facial track) and the red cap thrown into the lens. Shown as an under-10 MB copy of the 1080p render; credits in the project.

The opening shot, on the desk (a cloth collider) with the cap; skirt physics on a kick; the closing heart, with the rebuilt face. The first version of this video, from projects/claude_chika (the pipeline's regression standard), is in the 0.1.0 release.

Two characters, a camera from the reference video: DreamWings

https://github.com/user-attachments/assets/26e7c1ac-1053-4331-812b-8187a1a2b12a

projects/dreamwings: two Tripo-generated characters, Claude and then ChatGPT, dance 鸭梨P's MMD motion 梦的翅膀受了伤 (15 s, the meme's choreography) in the Anthropic lobby. The lobby is appended from its own .blend and scaled around the dance. The camera is solved frame by frame from the meme's reference video, whip pans and all, and the hand-off between the two dancers is one continuous whip. Both builds start from the untouched character files: the retargeted dance, the simulated gowns, hair, sleeves and tail, the faces, and the motion blur. Shown as an under-10 MB copy of the 1080p render; credits in the project.

What it does

If you have an AI-generated or auto-rigged anime character and an MMD/VMD dance you want it to perform, this is the pipeline for that. It retargets MMD motion to a non-MMD rig, adds skirt and hair physics, animates the face, and renders with the original camera and music. It handles:

• Motion retargeting. The VMD is imported with mmd_tools onto a proxy MMD skeleton built from your rig's proportions. It is then mapped onto your bones in world space, with twist bones shared between upper arm and forearm. MMD's foot IK is solved by an analytic two-bone leg IK.

• The MMD camera (camera.vmd), plus optional custom shots for stretches where the original camera doesn't suit your model.

• Secondary motion for skirts, layered gowns and aprons, long hair (as one curtain or lock by lock) and tails. Bone chains are laid through the mesh automatically and simulated with a position-based solver:

  • jointed chains with angular limits, as in MMD rigid-body skirts and DynamicBone;
  • capsule colliders measured from the body;
  • furniture box colliders and inelastic contacts.

  The result is baked to keyframes, so the final scene needs no add-on.

• Facial animation from the VMD morph track. It works on shape keys, or on a painted face texture (common for image-to-3D characters) through expression atlases built from the painting itself.

• Repairs for auto-rigged meshes:

  • rebuilding fused finger blobs as real hands from the finger bones, with the finger joints moved to drawn-hand proportions;
  • ripping hair from clothing, and hair curls that carry the clothing material;
  • rigid boot weights;
  • cross-side leg weight cleanup;
  • vertices left without weights.

• Stage, lighting, render (EEVEE), resumable parallel rendering, and encoding with the song. Picture and sound fade out together.

  • A set from another .blend scaled around the dance, a camera keyed from a track (e.g. solved from a reference video), a key light that follows the camera, motion blur.
  • Project code of your own on the built character (a rebuilt face, a prop, sleeves), shared between projects by path.
  • A packed copy of the scene for a cloud GPU, two characters' renders cut into one video, and copies for sharing.

This is the beginning of the README. Read the rest at the original. Read the full README ↗