Senior Data Engineer

Job Title: Senior Data Engineer Location: Chennai Employment Type: Full-Time Experience Level: 7-10 years Department: Data Engineering Job Summary: We are looking for a Lead Data Engineer with strong expertise in cloud platforms (Google Cloud Platform preferred), advanced skills in Spark, SQL, PySpark, and data modeling/design. The role involves leading the design and development of scalable data pipelines, driving best practices, mentoring the team, and managing complex real-time/streaming data solutions. Key Responsibilities: Lead the design, development, and optimization of scalable and efficient ETL/ELT data pipelines. Oversee data cleaning, transformation, and validation from diverse structured and unstructured sources. Collaborate closely with Data Scientists, Analysts, and stakeholders to define and deliver robust data solutions aligned with business goals. Ensure data quality, consistency, and integrity across all data platforms and pipelines. Develop and maintain comprehensive documentation for data architecture, processes, and best practices. Drive code reviews, enforce engineering standards, and promote continuous improvement across the team. Lead the implementation of streaming data technologies and manage real-time data processing solutions. Mentor and coach junior and mid-level data engineers, fostering technical growth and ensuring successful project delivery. Provide technical leadership in architectural decisions and contribute to the overall data strategy. Required Skills & Qualifications: 7+ years of experience in data engineering or related roles, with proven leadership in complex projects. Strong expertise in cloud data services on a major cloud platform (e.g., Google Cloud Dataflow, Dataproc, Cloud Storage, Secret Manager, Pub/Sub, or equivalent AWS/Azure services). Advanced proficiency in SQL and programming languages such as Python or Scala. Deep understanding of Apache Spark and Spark Core concepts. Extensive experience with cloud-based big data processing (e.g., Google Cloud Dataproc, Databricks) for large-scale data engineering workloads. Strong knowledge of data warehousing, data modeling, and relational database concepts. Exposure to major cloud platforms (GCP preferred; AWS or Azure is a plus). Hands-on experience with ETL tools and enterprise data integration frameworks. Excellent analytical, debugging, and problem-solving skills. Proven ability to lead teams, mentor engineers, and collaborate across functions. Knowledge of streaming technologies such as Apache Flink and/or programming in Java is an added advantage. Familiarity with version control and CI/CD tools like Git, Google Cloud Build, or similar CI/CD pipelines. Strong communication skills with the ability to drive technical discussions and influence stakeholders