Padmi

Data Engineer Python,aws

MumbaiPosted 3 months ago
Infrastructure And DatabasesStaff+Full Time; Regular
Apply at Go Digital Technology Consulting LLP

Opens the source posting on shine.com

Source description

About the role

View original

Role Overview: You will be responsible for designing and maintaining scalable, high-performance data pipelines for large and complex datasets. Your role as a Python-centric Data Engineer will require strong object-oriented Python programming skills, experience with distributed data systems (like Hadoop or Spark), and the ability to build modular, reusable, and testable data solutions. While familiarity with cloud technologies (AWS preferred) is beneficial, this position emphasizes deep Python-based data engineering over platform administration. Key Responsibilities: - Design, develop, and optimize data ingestion and transformation pipelines using Python. - Apply OOP principles to build modular, maintainable, and reusable data components. - Work with distributed systems (e.g., Hadoop/Spark) for processing large-scale datasets. - Develop and enforce data quality, testing, and validation frameworks. - Collaborate with analysts, data scientists, and product teams to ensure data availability and reliability. - Participate in code reviews, CI/CD workflows, and infrastructure automation to maintain high engineering standards. - Contribute to ongoing evolution of data architecture and help integrate with cloud-based data ecosystems. Qualifications Required: - Bachelor's or master's degree in engineering or technology or related field. - 36 years of hands-on experience in data engineering or backend development. - Proven track record of Python-based pipeline development and distributed data processing. - Strong foundation in data modeling, data quality, and pipeline orchestration concepts. - Excellent problem-solving and communication skills, with an ownership-driven mindset. Role Overview: You will be responsible for designing and maintaining scalable, high-performance data pipelines for large and complex datasets. Your role as a Python-centric Data Engineer will require strong object-oriented Python programming skills, experience with distributed data systems (like Hadoop or Spark), and the ability to build modular, reusable, and testable data solutions. While familiarity with cloud technologies (AWS preferred) is beneficial, this position emphasizes deep Python-based data engineering over platform administration. Key Responsibilities: - Design, develop, and optimize data ingestion and transformation pipelines using Python. - Apply OOP principles to build modular, maintainable, and reusable data components. - Work with distributed systems (e.g., Hadoop/Spark) for processing large-scale datasets. - Develop and enforce data quality, testing, and validation frameworks. - Collaborate with analysts, data scientists, and product teams to ensure data availability and reliability. - Participate in code reviews, CI/CD workflows, and infrastructure automation to maintain high engineering standards. - Contribute to ongoing evolution of data architecture and help integrate with cloud-based data ecosystems. Qualifications Required: - Bachelor's or master's degree in engineering or technology or related field. - 36 years of hands-on experience in data engineering or backend development. - Proven track record of Python-based pipeline development and distributed data processing. - Strong foundation in data modeling, data quality, and pipeline orchestration concepts. - Excellent problem-solving and communication skills, with an ownership-driven mindset.

One address, no account. We’ll tell you when matching roles go live.

More at Go Digital Technology Consulting LLP

Related open roles

View all roles