Senior Data Engineer

Job Description We are looking for an experienced Senior Data Engineer to design, build, and manage scalable, reliable, and high-performance data solutions. The ideal candidate will have strong expertise in developing production-grade data pipelines, cloud data platforms, distributed processing systems, and modern data engineering practices. This role involves owning end-to-end data workflows, ensuring data quality and reliability, optimizing cloud infrastructure, and collaborating with engineering, analytics, and AI teams to deliver impactful data products. Key Responsibilities Build and manage production data pipelines with a focus on scalability and reliability. Develop Python-based data engineering solutions including API integrations, automation, testing, and pipeline orchestration. Design and optimize distributed data processing workflows using technologies such as Spark, Apache Beam, or Databricks. Implement data partitioning, parallel processing, memory optimization, and performance tuning strategies. Work with cloud data platforms including AWS, Azure, or Google Cloud. Design and maintain data warehouse solutions using Snowflake, BigQuery, Redshift, or Synapse Analytics. Implement Infrastructure as Code using Terraform, CloudFormation, ARM/Bicep, or equivalent tools. Develop CI/CD workflows using Git, Docker, Jenkins, GitHub Actions, or Azure DevOps. Ensure data correctness through idempotent pipelines, deterministic deduplication, stable key generation, and validation frameworks. Support AI/ML initiatives including NLP, LLM workflows, RAG systems, vector search, and MLOps processes (preferred). Optimize LLM-based workflows through rate limiting, concurrency management, cost optimization, and latency improvements. Requirements 5+ years of hands-on experience in Data Engineering with production data pipelines. Strong expertise in Python for data engineering applications. Experience with cloud platforms such as AWS, Azure, or Google Cloud. Strong knowledge of cloud data warehouses including Snowflake, BigQuery, Redshift, or Synapse Analytics. Hands-on experience with distributed data processing frameworks such as Spark, Apache Beam, Databricks, or similar technologies. Strong understanding of data partitioning, shuffling, windowing operations, parallel processing, and memory optimization. Experience with Docker, Git workflows, automated testing, CI/CD pipelines, and deployment processes. Experience implementing Infrastructure as Code solutions using Terraform, CloudFormation, ARM, or Bicep. Strong understanding of data quality, reliability, and production system design principles. Experience with ML/NLP engineering, LLMs, RAG, vector databases, or MLOps is preferred. Ability to design scalable architectures while ensuring downstream system stability and performance. Maintain records of the conversations with the customer and analyze the data. Handling customer queries on Chat and E-mails. Career First Name Last Name Email Contact No Job Applied For Preferred Location Experience (Years) Notice Period (In Days) Curriculum Vitae/Resume Let s Create Together Work with us SourceMash Technologies is a leading solution provider for internet-based applications and product development since 2008. Be a part of our company that is facilitated by highly skilled professionals dedicated to providing total IT solutions under one roof. We offer remarkable services in the areas of Software Development, Quality Assurance, and Support. Welcome Kit An employee welcome kit, like Custom Notepad, T-Shirt, Water Bottle etc. , is also included in employee welcome packages onboard. Medical Insurance SourceMash Technologies offers the best employee health insurance benefit to their employees family members under the same policy. Paid Leaves Annual leaves are paid at the payment rate in the working period before the leave, and no untaken leaves can be considered part of the mandatory notice periods. Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.