Source description
About the role
As a Lead Big Data Engineer with 9+ years of experience, you will be responsible for designing scalable ETL/ELT pipelines, optimizing distributed data systems, and building cloud-native data engineering solutions for large-scale enterprise environments. Your role will involve working with Scala, Apache Spark, SQL, and AWS-based data platforms to deliver advanced cloud, data engineering, and AI solutions across industries. Key Responsibilities: - Designing and developing scalable big data and cloud-native data engineering solutions. - Building and optimizing ETL/ELT pipelines using Scala, Spark, SQL, and Python. - Working on distributed computing, Spark Streaming, and performance optimization. - Migrating and modernizing on-premise data warehouses to cloud-based platforms. - Mentoring team members and collaborating with cross-functional stakeholders for solution delivery. Qualifications Required: - 8+ years of experience in Big Data Engineering and distributed systems. - Strong hands-on expertise in Scala, Apache Spark, SQL, and Python. - Experience with AWS or other cloud platforms along with Data Lakes and streaming technologies. - Knowledge of Kafka/Kinesis, Hive, Snowflake, and CI/CD pipelines. - Strong understanding of data modeling, data warehousing, and performance tuning. In this role, you will have the opportunity to work on cutting-edge AI and data engineering projects, gain exposure to enterprise-scale cloud transformation initiatives, enjoy a collaborative and innovation-driven work culture, experience fast-paced career growth with leadership opportunities, and receive competitive compensation in a learning environment. As a Lead Big Data Engineer with 9+ years of experience, you will be responsible for designing scalable ETL/ELT pipelines, optimizing distributed data systems, and building cloud-native data engineering solutions for large-scale enterprise environments. Your role will involve working with Scala, Apache Spark, SQL, and AWS-based data platforms to deliver advanced cloud, data engineering, and AI solutions across industries. Key Responsibilities: - Designing and developing scalable big data and cloud-native data engineering solutions. - Building and optimizing ETL/ELT pipelines using Scala, Spark, SQL, and Python. - Working on distributed computing, Spark Streaming, and performance optimization. - Migrating and modernizing on-premise data warehouses to cloud-based platforms. - Mentoring team members and collaborating with cross-functional stakeholders for solution delivery. Qualifications Required: - 8+ years of experience in Big Data Engineering and distributed systems. - Strong hands-on expertise in Scala, Apache Spark, SQL, and Python. - Experience with AWS or other cloud platforms along with Data Lakes and streaming technologies. - Knowledge of Kafka/Kinesis, Hive, Snowflake, and CI/CD pipelines. - Strong understanding of data modeling, data warehousing, and performance tuning. In this role, you will have the opportunity to work on cutting-edge AI and data engineering projects, gain exposure to enterprise-scale cloud transformation initiatives, enjoy a collaborative and innovation-driven work culture, experience fast-paced career growth with leadership opportunities, and receive competitive compensation in a learning environment.
More at CRESCENDO GLOBAL LEADERSHIP HIRING INDIA PRIVATE L