LLM & Foundation Models

NVIDIA NeMo Speech

nvidia-nemo/speech

A scalable generative AI framework built for researchers and PyTorch developers working on Automatic Speech Recognition (ASR), Text-to-Speech (TTS), and Speech LLMs.

Open official repository ↗ Project homepage ↗
★ 17.9KStars
⑂ 3.5KForks
237Open issues
PythonLanguage
Apache-2.0License
Q@project.QualityScoreEditorial score

Project screenshots

Screenshot of NVIDIA NeMo Speech Screenshot of NVIDIA NeMo Speech Screenshot of NVIDIA NeMo Speech Screenshot of NVIDIA NeMo Speech Screenshot of NVIDIA NeMo Speech Screenshot of NVIDIA NeMo Speech

Overview

NVIDIA NeMo Speech is a Python-based framework focused on audio, speech, and multimodal LLMs. It is designed to help users efficiently create, customize, and deploy new speech AI models by leveraging existing code and pre-trained model checkpoints. The repository is actively developed and has pivoted to focus specifically on audio, speech, and multimodal LLMs, while previous broader NeMo modalities are maintained in prior release tags.

Key features

  • Scalable generative AI framework for Speech AI
  • Supports ASR, TTS, and Speech LLMs
  • Compatible with user-defined Python, PyTorch, and CUDA stacks
  • Optional accelerated backends (Transformer Engine, FlashAttention, Mamba, MoE, DeepEP)
  • Turnkey Docker container builds for H100+ and A100 targets
  • Includes recent model releases like MagpieTTS, Nemotron-Speech-Streaming, and Parakeet

Requirements, installation and quick start

Recommended installation is from source using uv: `git clone https://github.com/NVIDIA-NeMo/NeMo.git`, `cd NeMo`, then `uv sync --extra all --extra cu13` (for CUDA 13.x) or `uv sync --extra all --extra cu12` (for CUDA 12.x). Alternatively, build from Docker: `docker buildx build -f docker/Dockerfile -t nemo-speech .` and run with `docker run --rm -it --gpus all -v "$PWD:/workspace" nemo-speech bash`. For a bring-your-own environment using pip: `uv pip install 'nemo-toolkit[asr,tts]'` or `pip install 'nemo-toolkit[asr,tts]'`.

Usage

After installation, users can utilize pre-trained checkpoints and demos available via the HuggingFace collection. For technical usage and framework guidance, refer to the NeMo Framework User Guide documentation. Specific command-line usage examples for model training or inference are not detailed in the provided repository metadata.

Model compatibility and use cases

Recent releases include MagpieTTS Multilingual (supporting 12 languages), Nemotron-3.5-ASR-Streaming-0.6B (40 languages), Parakeet-unified-en-0.6b, Nemotron 3 VoiceChat, Nemotron-Speech-Streaming, Canary V2, and Canary-Qwen-2.5B.

License and risk notes

Licensed under the Apache License 2.0.

Editorial verification 2026-08-09: repository URL, owner, description, license and repository statistics were reviewed. License metadata: Apache-2.0. README was fetched for the channel draft; re-check repository dependencies, releases and model terms before production use.

Release and maintenance

Recent updates include the release of MagpieTTS v2607 (2026-07), Nemotron-3.5-ASR-Streaming-0.6B (2026-06), Parakeet-unified-en-0.6b (2026-04), Nemotron 3 VoiceChat Early Access (2026-03), Nemotron-Speech-Streaming v2603 (2026-03), MagpieTTS v2602 (2026-03), and Nemotron-Speech-Streaming initial release (2026-01). Parakeet V3 and Canary V2 were released in 2025-08, and Canary-Qwen-2.5B in 2025-06.

ClearML

clearml/clearml

★ 6.8KPython