Machine Learning Engineer: Speech & Language Models
Company: Convin.ai
Location: Bengaluru
Experience: 2-5 years
About the role
We're building conversational AI and voicebot systems that run in production at scale, across multiple Indian languages and code-mixed speech. We're looking for an ML Engineer who is comfortable across both speech and text, can own models from training through to serving, and enjoys exploring new model families and turning them into reliable production systems.
You'll work on small language models (SLMs) tailored to our tasks, the inference infrastructure that serves them, and a new initiative to explore and train our own text-to-speech (TTS) model.
What you'll do
Small language models
- Fine-tune, distill, and maintain custom SLMs for company-specific tasks such as call analysis, structured extraction, and agent reasoning.
- Build and curate training and evaluation datasets, including synthetic data generation and labeling pipelines.
- Run systematic evaluations against task-specific metrics and larger reference models, and iterate on quality, latency, and cost.
- Apply model optimization techniques such as quantization, knowledge distillation, and LoRA/QLoRA fine-tuning.
LLM serving infrastructure
- Own and improve our LLM serving stack built on NVIDIA Triton Inference Server, vLLM, SGLang, and NVIDIA Dynamo, including deployment, scaling, and capacity planning across GPUs.
- Tune inference for low-latency, high-concurrency voice workloads: batching, prefix/KV caching, quantized weights, and speculative decoding where useful.
- Maintain routing, health checks, and fallbacks across model backends, and build observability (metrics, dashboards, alerts) for production models.
- Debug production issues end to end, from GPU memory and throughput problems to tool-calling and output-format reliability.
Speech: ASR and TTS
- Work with ASR systems for multilingual and code-mixed audio, including evaluation, fine-tuning, and transcript quality improvements.
- Survey and benchmark open-source TTS models for naturalness, latency, streaming support, and language coverage.
- Lead the training of an in-house TTS model: data collection and cleaning, speaker and prosody considerations, training, evaluation (MOS, intelligibility, latency), and deployment for real-time voice agents.
What we're looking for
Must have
- Strong Python and PyTorch skills, with hands-on experience training or fine-tuning transformer models.
- Practical experience with LLM fine-tuning and evaluation (Hugging Face ecosystem, PEFT, or similar).
- Experience deploying models to production and reasoning about latency, throughput, and GPU cost.
- Hands-on experience with at least one of Triton Inference Server, vLLM, or SGLang in production.
- Comfort working with messy real-world data and building data pipelines around it.
- Ability to read papers, try new approaches quickly, and report results clearly.
Good to have
- Experience with speech models: ASR (Whisper-style, CTC/RNN-T) or TTS (VITS, FastSpeech, flow-matching or codec-based models).
- Experience with NVIDIA Dynamo or other distributed inference frameworks (disaggregated prefill/decode, KV-cache-aware routing).
- Experience with Indian languages or code-mixed text and speech.
- Knowledge of quantization formats (FP8, AWQ, GPTQ) and distributed training (FSDP, DeepSpeed).
- Experience with observability tooling (Prometheus, Grafana) and containerized deployments (Docker, Kubernetes).
- Open-source contributions or published work in NLP or speech.