Skip to content

Wunjo AI: Synthesize & clone voices in English, Russian & Chinese, real-time speech recognition, deepfake face & lips animation, face swap with one photo, change video by text prompts, segmentation, and retouching. Open-source, local & free.

License

Notifications You must be signed in to change notification settings

franko00007/wunjo.wladradchenko.ru

 
 

Repository files navigation

Price Python 3.10 GitHub package version License: MIT v1.0 Dev.to Articles YouTube Chanel Hugging Face Models

[RU]


Logo

Wunjo AI: Advanced Speech & Deepfake Neural Network Tool

Documentation
Issue · Discussions · Tutorial

About

Unlock the unparalleled capabilities of neural networks with Wunjo AI. Whether you're delving into speech synthesis, crafting deepfake animations, drawing Stable Diffusion video by text prompt or video making, Wunjo AI has got you covered.

Key Features:

  • Speech Synthesis: Effortlessly convert text into human-like speech.
  • Voice Cloning: Clone voices from provided audio files or directly record your voice within the app for real-time cloning.
  • Multilingual Support: Currently supports English, Russian, Chinese for voice cloning (from any language audio) and English, Russian synthesis, you can also offer your language for voice cloning. Multilanguage speech by one text.
  • Real-time Speech Recognition: Dictate text and get instant transcriptions. An efficient tool for hands-free content creation.
  • Multidialogue Creation: Craft multi-dialogues using unlimited characters with distinct voice profiles.
  • Audio processing Speech enhancement and separating vocals and music.
  • Video-to-Video by Text Prompt:
    • Reshape videos with by text prompt with difference models of Stable Diffusion. Let generative neural networks craft a new visual narrative.
    • Change individual objects in a video by text prompt with one click, changing them throughout the video with unique text queries.
    • Preserve specific objects without change by using the «pass» keyword.
    • Change video style by changed img2img frame,
  • Deepfake Animation:
    • Animate faces using just one photo combined with audio.
    • Achieve precise lip syncing with your audio using our deepfake lips feature.
    • Effortlessly swap faces in videos, GIFs, and photos using just a single photograph with our "Face Swap" feature.
    • Experimental feature. Change the emotions of a person in the video, with the help of a text description.
  • AI Retouch Tool: Elevate your videos by removing unwanted objects or refining the quality of your deepfakes. Automatic removal of animated text.
  • Automatic Segmentation Mask: Select any object at any time period and get a storyboard of the selected object with a transparent or colored background.

Applications: From voiceovers in commercials to character voicing in games, from audiobook narrations to fun deepfake projects, Wunjo AI offers endless possibilities and all is free and local on your device.

Why Choose Wunjo AI?:

  • All-in-One: A comprehensive tool catering to both your voice and visual AI needs.
  • User-friendly: Designed for all, from beginners to professionals.
  • Privacy First: Functions locally on your desktop, ensuring your data remains private.
  • Open-source & Free: Benefit from community-driven enhancements and enjoy the app without any cost.

Step into the future of AI-powered creativity with Wunjo AI.

Setup

Requirements Python version 3.10 and ffmpeg.

For detailed instructions about setup Wunjo AI from GitHub, refer to the Launch Project from GitHub section in our wiki.

Install packets

For detailed instructions about install Wunjo AI on Ubuntu / Debian OS from installer

MacOS v1.6 (CPU version)

Due to the fact that the author of the project does not have an Apple license, there is currently no way to create an official installer.

For detailed instructions about install Wunjo AI on Windows from installer

Read in Wunjo AI documentation how use GPU on Windows.

Example

Speech synthesis and voice cloning

Face animation from image src

Original Move face + Enhancer Fix face + Enhancer
original move_enhancer static_enhancer

Mouth animation from video src

Original Mouth animation Mouth animation + Enhancer
original move_enhancer static_enhancer

Face swap by one photo

Original photo Original video Face swap + Background enhancer
original original static_enhancer

Remove object by Retouch AI

Original video Remove object
original original

Retouch AI to improve quality of deepfake

Defective lines on the chins after animation lip Retouch lines on the chins + Face swap
original original

Get segmentation mask by one click

Original Mask
original original

Video-to-Video by Text Prompt (Only for GPU)

The higher the video resolution, the better the quality of the drawn frames.

Video resolution 512x512 default model for deepfake

Original Blonde hair + Brown jacket
original pass

Video resolution 512x512 custom model for anime

Additionally, you can use your custom stable diffusion model to redraw video or objects in video with difference timeline.

Original Object pass + Background change
original pass
Object change + Enhancement move Object change + Enhancement anime
full_change full_change

Limit resolution video by GPU VRAM

32 GB 23 GB 18 GB 14 GB 10 GB 8 GB 7 GB
1280x1280 1080x1080 1024x1024 768x768 640x640 576x576 512x512

Emotion deepfake [Experimental]

This is an experimental feature that is under development, but you can take a look at some of the work right now in Wunjo AI.

Original Happy Angry
original original original
Fear Sad Disgust
original original original

Language

The application comes with built-in support for the following languages: English, Russian, Chinese, Portuguese, and Korean.

If you wish to add a new language:

Navigate to .wunjo/settings/settings.json. Add your desired language in the format: "default_language": {"name": "code"}. To find the appropriate code for your language, please refer to the Google Cloud Translate Language Codes.

Update 1.6.0

  • Improved and automated remove object from image or video
  • Improved edit video element
  • Added auto segmentation mask with save
  • Added Video2Video with ControlNet by text prompt tool
  • Added InpaintVideoMask2Video with ControlNet by text prompt tool
  • Optimized using memory for face swapping for long video
  • Optimized using memory for retouch and remove object for long video

Update 1.6.1

  • Fix bug with enhancer. Improve enhancer for video and face. Added enhancer for drawing video
  • Improved vocoder for voice cloning
  • Added cloning speed speech
  • Added model to get background sound for deepfakes and clear voice without background noise
  • Added feature to get background noise from audio or video
  • Improved encoder for voice cloning
  • Imitate voice emotions and improved voice cloning quality.
  • Reducing the amount of RAM used for mouth animation and improving video quality
  • Added speech enhancement
  • Music generation
  • Added module a video style change by images
  • Multilanguage speech by one text
  • Added auto remove text from video or image with auto create text mask

Update 1.6.2

  • Custom browser for WebGUI
  • Added select browser to run
  • Added check between offline and online mode
  • Added a message for the user about missing models and how to download manually

Video

Review How install on Windows?
video video

Support the Project

You can support the author of the project in the development of his creative ideas, or just treat him to a cup of coffee in USD or a slice of pizza in RUB. There are other ways to support the development of the project, more details on page.

Buy a cup of coffee in USD Buy a slice of pizza in RUB
pizza coffee

Contact

Owner: Wladislav Radchenko

Email: [email protected]

Project: https://github.com/wladradchenko/wunjo.wladradchenko.ru

Web site: wladradchenko.ru/wunjo

Premise

Wunjo comes from the ancient runic alphabet and represents joy and contentment, which could tie into the idea of using the application to create engaging and expressive speech. Vunyo (ᚹ) is the eighth rune of the Elder and Anglo-Saxon Futhark. Prior to the introduction of the letter W into the Latin alphabet, the letter Ƿynn (Ƿƿ) was used instead in English, derived from this rune.

Credits

Wunjo AI is built upon the remarkable work of various open-source projects. Each integrated component reflects a commitment to improving and adapting existing technologies within the collaborative landscape of open-source development. The list below highlights the projects that have been adapted and enhanced for inclusion in Wunjo AI:

I extend my deepest gratitude to the original contributors of these technologies. Their groundbreaking work has been instrumental in advancing the capabilities of Wunjo AI. For the persistent storage and versioning of the models I have personally trained, I utilize the Hugging Face Model Storage. If you're interested in contributing to Wunjo AI, especially in the area of voice cloning for new languages, please feel free to propose your models or reach out for collaboration via GitHub Discussions or the Hugging Face.

[to top]

About

Wunjo AI: Synthesize & clone voices in English, Russian & Chinese, real-time speech recognition, deepfake face & lips animation, face swap with one photo, change video by text prompts, segmentation, and retouching. Open-source, local & free.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 88.7%
  • JavaScript 10.2%
  • Other 1.1%