How AI singing voice cloning works, from sample to song

How AI singing voice cloning works: stem separation, a voice model trained on 1 minute of your singing, its real limits, and how to try it in Tera Studio.

AI singing voice cloning trains a small model on a short recording of your voice, then uses it to re-sing the lead vocal of an existing song, so the melody, words and timing stay and only the voice changes. Tera Studio is a web app that does this from about 1 minute of your singing, for anyone who wants to hear themselves sing a song they love.

This guide covers what happens inside that process, what your sample needs, where the technique falls short, and the steps in Tera Studio, which is made by Tera Labs. For a shorter, non-technical version, read How to hear any song in your own voice.

How AI singing voice cloning works

A finished song is one mixed audio file. To put your voice on it, the software pulls the singing out of the mix, learns what your voice sounds like, and re-sings the original vocal with it. Tera Studio's progress screen shows this as five steps: prepare your files, prepare your voice, separate vocals and music, sing in your voice, and mix and save your song.

1. Stem separation pulls the vocal out of the song

A source-separation model splits the song into a vocal stem and an instrumental (karaoke) stem. The instrumental waits while the vocal goes to your voice model.

This step matters more than most people expect. The voice model converts every voice in the stem it is handed, so leftover backing harmonies or reverb come back as warbly doubles and metallic tails.

2. A voice model learns your timbre from your sample

Tera Studio's voices are RVC models. RVC (Retrieval-based Voice Conversion) is an open-source voice conversion framework, and Tera Studio trains and runs it with the open-source Applio toolkit.

Training extracts two things from your recording: content features that describe what is being sung (Tera Studio uses an encoder called ContentVec) and a pitch line, the fundamental frequency (F0) of each moment. The model learns to rebuild your audio from those inputs. Because the words and notes are handed to it, its weights have to capture everything else: the color and texture of your voice.

Tera Studio's quick voice trains for 60 epochs (one epoch is one full pass over your recording) and saves a checkpoint every 10. It then runs a held-out test clip through its newest checkpoints and keeps the final one unless an earlier checkpoint clearly does better.

Your first song takes about 20 minutes because your voice is being learned; the dashboard estimates 15 to 25 minutes. The voice is saved to your account, so later songs skip training and come back faster.

3. Conversion re-sings the original vocal in your voice

For each song, a pitch tracker and the content encoder read the separated vocal. Your model sings that performance with your timbre, and the result is mixed back over the instrumental.

Because the notes and timing come from the original singer, the result can be in tune even if you are not. If the original lead sits far above or below your voice, Tera Studio can move the converted vocal by a whole octave, so it lands nearer your range while the song keeps its key.

What makes a good 1-minute sample

The model only knows your voice through your recording, so whatever is in that recording is what it learns. Tera Studio asks for your own singing, in a quiet room, with no music behind you.

The length rules are 1 minute minimum, 2 minutes or more for the best voice, and 5 minutes maximum. Training uses up to 2 minutes of singing; from a longer take it uses the best 2 minutes. The open-source RVC project recommends 10 minutes or more of low-noise audio, which is why every second of a short sample has to be clean.

Tera Studio's in-browser recorder splits those 2 minutes into four prompts, which also work as a recipe if you record elsewhere:

  1. Sing the chorus of your song.
  2. Sing it again, a little louder.
  3. Reach for your highest notes, falsetto too. In the app's words, your cover only reaches as high as you do here.
  4. Finish with a slow, soft line.

You don't need to be a good singer: the model learns how your voice sounds, not how accurately you hit notes. Do avoid low-quality audio. The app flags recordings that are phone-call quality, because a voice built from them won't sound fully like you.

What AI voice cloning can't do

  • It isn't text-to-speech. It converts a real sung performance, so the song needs a vocal.
  • It can't out-sing the source. The phrasing and breaths come from the vocal you feed it.
  • It needs a clear lead vocal. Doubled leads, duets and choirs are hard to split from backing vocals and convert less cleanly.
  • Too little of your voice lets the original singer leak through. The fix is a longer take; Tera Studio says 1:30 or more fixes the most.
  • It doesn't give you rights to the music. Use songs you made or have permission to transform, keep the artist credit, and say the vocal is AI-converted when you share.

Tera Studio's terms let you train a voice model only on your own voice, or on a voice whose owner has given you explicit, verifiable permission. Cloning someone's voice without consent is grounds for immediate account termination, and impersonating a person without consent is prohibited.

When you add a recording, the app asks you to confirm it is your own voice. Your trained voice is locked to your account: other users can't select it, use it or hear it. Read the full terms.

How to clone your singing voice in Tera Studio

  1. Sign up with Google, Apple or email. No card is needed.
  2. Choose Make a cover and add your song: a WAV, MP3 or M4A file up to 20 minutes and 150 MB, or the built-in CC BY 3.0 birthday song. Streaming links aren't supported. Use the whole song (up to 5 minutes per version) or a part, and confirm you have permission to use it.
  3. Add your voice. Sing into the browser recorder or upload 1 to 5 minutes of audio, then tick the consent box.
  4. Save, then start your free listening preview.
  5. Wait about 20 minutes. You can close the tab; your song waits in My songs.
  6. Listen and rate it. Tap I love it or Needs work. A song marked Needs work can be remade once, free, from a new recording.
  7. Download. The free preview plays with a soft chime. The clean, full-quality WAV and a song-card video come with a paid plan, and downloading the same song again never uses more minutes.

What it costs to try

Tera Studio has a free plan: every new account gets 2 full-song previews in its own private voice, each up to 4 minutes, played on the website, with no card. Downloads come with the Starter or Creator plan. See current prices.

Record your first minute and hear a whole song in your own voice at Tera Studio.

Questions people ask

How much audio do you need to clone your singing voice?

Tera Studio needs at least 1 minute of your singing and gives the best voice from 2 minutes or more, up to 5 minutes. Record in a quiet room with no music playing.

Can I clone a famous singer's voice with AI?

Not on Tera Studio. Its terms allow only your own voice or one whose owner has given explicit, verifiable permission.

How long does AI voice cloning take?

In Tera Studio, your first song takes about 20 minutes because the model is learning your voice. Later songs reuse your saved voice and come back faster.

Is AI singing voice cloning free?

Tera Studio's free plan gives every new account 2 full-song previews of up to 4 minutes each, with no card. Downloading the WAV requires a Starter or Creator plan.