Gen AI

Job Summary

This role involves leveraging advanced Python skills alongside GenAI technologies such as Large Language Models (LLMs), Lang Chain, and Prompt Engineering to develop and support data engineering solutions. The candidate will work with Azure Databricks, Azure Data Services, and various data governance frameworks to ensure high-quality data management and facilitate effective AI/ML pipelines. The project duration is 3 months (extendable to 6 months), and the position offers a competitive rate of 2.25 lacs per month (including GST).

Responsibilities

• Develop, implement, and maintain GenAI solutions using Python, LLMs, Lang Chain, and Prompt Engineering techniques.
• Utilize Azure Databricks and other Azure Data Services (Data Lake, Synapse, Blob Storage) to build robust data pipelines and ensure data quality.
• Design and execute data quality rules, including profiling and implementation, in line with internal controls and compliance standards.
• Collaborate with stakeholders to ensure clear communication around data governance and analytics insights.
• Support the integration and operation of AI/ML pipelines, with a focus on GenAI and LLM-based models.
• Troubleshoot and optimize existing processes to improve system performance and data integrity.

Qualifications

• 2-5 years of relevant work experience.
• Strong proficiency in Python, LLMs, Lang Chain, and Prompt Engineering.
• Hands-on experience with Azure Databricks and Azure Data Services including Data Lake, Synapse, and Blob Storage.
• Solid understanding of data governance frameworks, compliance, and internal controls.
• Proven expertise in developing, profiling, and implementing data quality rules.
• Excellent analytical, problem-solving, and stakeholder communication skills.
• Educational degrees such as B.Tech, B.Sc, M.Tech, or equivalent in a relevant field.

Preferred Skills

• Experience supporting AI/ML pipelines, especially those based on GenAI or LLM models.
• Proficiency in additional technologies such as PySpark, SQL, and Delta Lake architecture.
• Hands-on experience with Azure Data Lake, Azure Data Factory, and Azure Synapse Analytics.
• Expertise in data engineering roles focusing on advanced usage of Databricks.

Experience

• 2-5 years of relevant experience in data engineering, AI/ML pipeline development, or a related field.

Environment

• Based in Bangalore with a hybrid work model combining both onsite and remote work.

Salary

• 2.25 lacs per month including GST