Multi Lingual Audio Transalation using LLMS

The paper proposes a multilingual audio translation system that utilizes Large Language Models (LLMs) to convert spoken English into Hindi or Telugu. The system leverages three advanced models—Gemma2, Qwen-7B, and Mistral-7B—fine-tuned using the IITB English-Hindi dataset and the Bharat Telugu dataset, both available on Hugging Face. These datasets are essential for training the models to perform high-quality translations across these languages, ensuring both accuracy and contextual relevance. The system works by first converting spoken English into text using automatic speech recognition (ASR), followed by translation into either Hindi or Telugu, depending on the user's choice. The LLMs used in this project are based on transformer architectures, known for their ability to capture long-range dependencies and contextual relationships within sequences. The project demonstrates how fine-tuning pre-trained models on domain-specific datasets can enhance the translation of complex linguistic structures, allowing the system to provide real-time translations with high fidelity. The proposed system offers significant potential for breaking language barriers in communication, with applications in areas such as education, business, and cross-cultural interactions, particularly in regions where Hindi and Telugu are widely spoken.

Keywords: multilingual translation, audio translation, Gemma2, Qwen-7B, Mistral-7B, IITB English-Hindi dataset, Bharat dataset, Hugging Face, speech-to-text, fine-tuning, language models, real-time translation.