Source description
About the role
Roles and Responsibilities : - Design, develop, test, deploy and maintain large-scale data pipelines using Data Bricks. - Collaborate with cross-functional teams to gather requirements and deliver high-quality data solutions. - Develop complex data models using PySpark and implement efficient algorithms for retrieval augmented generation. - Troubleshoot issues related to data quality, performance, scalability, and security in a distributed computing environment. Job Requirements : - 9-14 years of experience in Data Engineering with expertise in Agentic AI's Large Language Model (LLM). - Strong proficiency in Python programming language with hands-on experience on Data Bricks. - Experience working with large datasets using PySpark and implementing productive algorithms for retrieval augmented generation. .
More at Premium