Ai Software Architect Llm Agentic Systems Msys Tech India Pvt Ltd Chennai

About the Role

Lead architecture and delivery of production-grade LLM-powered systems and agentic workflows, designing scalable RAG, observability, evaluation, and orchestration for real-world AI products. Provide technical leadership across engineering teams to build reliable, cost-effective AI services integrated with backend, web, and mobile applications.

Directly building agentic LLM systems, RAG pipelines, and observability — so it's all about architecting AI tooling and production LLM workflows.

Location: Chennai

Type: Full Time, Permanent

Posted: 2 days ago

Category: AI/ML & Data

Vibe Markers: 💻 Open Source, 🔥 Hot

Experience Level: Staff/Principal

Job Description

Role

The AI Software Architect / Senior AI Engineer will design, build, and operate production-grade LLM-powered systems and agentic workflows. The role focuses on architecting scalable Retrieval-Augmented Generation (RAG) solutions, tool-calling and multi-agent orchestration, evaluation frameworks, and AI observability while integrating AI services with backend, web, and mobile applications.

Key Responsibilities

• Own architecture and technical decisions for AI-powered services and their integration with backend, mobile, and web apps.
• Design, build, and maintain production-grade LLM applications using modern AI frameworks and orchestration platforms.
• Architect agentic workflows involving tool use, function calling, planning, memory management, and multi-agent systems.
• Establish evaluation frameworks to measure AI quality, reliability, latency, cost, and business impact.
• Implement AI observability and monitoring with platforms such as Langfuse, LangSmith, OpenTelemetry, and related tooling.
• Design and implement RAG architectures using vector databases, embeddings, and document pipelines.
• Lead technical design reviews and drive architectural decisions across AI services, backend systems, data pipelines, and cloud infrastructure.
• Define AI engineering standards, best practices, and governance; document architectures, workflows, and operational procedures.
• Provide technical leadership and mentorship to engineering teams.
• Optimize prompts, retrieval strategies, model selection, performance, and cost efficiency.
• Leverage cloud platforms (e.g., GCP) and cloud-native tooling to deploy and scale AI services.

Requirements

• Master’s degree in Computer Science, AI, ML, Software Engineering, or related field (preferred).
• 10+ years of software engineering experience with at least 5 years focused on LLM-based solutions and generative AI systems.
• Demonstrated experience designing and deploying complex production-grade AI applications using LLMs.
• Extensive experience with AI orchestration frameworks (examples: LangGraph, LangChain, LlamaIndex, CrewAI).
• Hands-on experience implementing AI observability and evaluation frameworks (examples: Langfuse, LangSmith, OpenTelemetry).
• Proven expertise with RAG architectures and vector database technologies.
• Strong Python development experience and solid software engineering fundamentals.
• Experience deploying AI solutions using Docker, Kubernetes, and cloud platforms (GCP mentioned).
• Familiarity with structured and unstructured data processing pipelines and modern databases (PostgreSQL, vector DBs, document stores).
• Strong communication, leadership, and mentoring skills; ability to collaborate with cross-functional teams.

Preferred

• Experience building agentic systems (tool use, planning, memory, multi-agent orchestration).
• Experience with model evaluation, benchmarking, fine-tuning, synthetic data generation, and model optimization.
• Experience with multiple commercial and open-source models (OpenAI, Anthropic, Gemini, Llama, Mistral).
• Experience supporting AI products in regulated, privacy-sensitive, or high-availability environments.
• Familiarity with DevOps practices, CI/CD, and Agile development.

Not included

• Boilerplate company marketing, hiring process steps, and application instructions have been omitted.

Tech Stack

Python, LangGraph, LangChain, LlamaIndex, CrewAI, Langfuse, LangSmith, OpenTelemetry, Vector databases, Embeddings, Google Cloud Platform (GCP), Docker, Kubernetes, PostgreSQL, OpenAI, Anthropic, Gemini, Llama, Mistral, CI/CD

Skills

System Architecture, Software Engineering, Prompt Engineering, Retrieval-Augmented Generation (RAG), Agent Orchestration, AI Observability, Evaluation & Benchmarking, API Design, Cloud Architecture, Performance Optimization, Cost Optimization, Data Pipeline Design, Leadership, Mentoring, Communication, DevOps / CI/CD, Model Integration