Software Engineer

Python + Gen AI + Databricks Engineer Role Summary We are seeking a highly skilled Python + Gen AI + Databricks Engineer to design, build, and deploy scalable data and AI solutions. The role involves developing data pipelines, advanced analytics, and Generative AI applications on cloud-based platforms using Databricks , Python , and modern LLM frameworks . Key Responsibilities

Develop and maintain data pipelines and analytics workflows using Databricks (Spark, PySpark) Build, deploy, and optimize Python-based data engineering and machine learning solutions Design and implement Generative AI use cases such as chatbots, summarization, content generation, and retrieval-augmented generation (RAG) Integrate LLMs (OpenAI, Azure OpenAI, Hugging Face, etc.) into enterprise applications Perform data preprocessing, feature engineering, and model validation Optimize data workflows for performance, scalability, and cost efficiency Collaborate with business stakeholders, data scientists, and product teams to translate requirements into technical solutions Ensure data quality, security, and compliance standards are met Document solutions and provide production support where required

Required Skills & Qualifications Technical Skills

Strong proficiency in Python (data structures, OOP, APIs, performance tuning) Hands-on experience with Databricks :

Apache Spark / PySpark Delta Lake Databricks workflows and jobs

Experience with Generative AI / LLMs :

Prompt engineering Fine-tuning or adapter-based approaches (LoRA, adapters) RAG architectures

Experience with ML/AI frameworks : LangChain, LlamaIndex, TensorFlow, PyTorch (any) Strong SQL skills and experience with large datasets Experience with cloud platforms (Azure preferred; AWS/GCP acceptable)

Data & MLOps

Model deployment and monitoring CI/CD for data and ML pipelines Experiment tracking and version control Familiarity with MLflow is a plus

Preferred Qualifications

• Experience with Azure Databricks and Azure OpenAI Knowledge of vector databases (FAISS, Pinecone, Weaviate) Exposure to real-time or streaming data (Kafka, Event Hubs) Understanding of data governance and security concepts Prior experience in retail, BFSI, healthcare, or large-scale enterprise analytics

• Education

• Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or related field

• Experience

• 4–8 years of experience in data engineering, analytics, or AI development At least 1–2 years of hands-on experience with Generative AI or LLM-based solutions