Data Engineer

In this role, you will be responsible for reviewing and understanding business requirements to ensure timely completion of development tasks with minimal defects. You will collaborate with a software development team to implement best practices and optimize the performance of Data applications. Additionally, you will work closely with various teams within the company and engage with customers to translate their business challenges into innovative solutions. Researching new Big Data technologies and assessing their alignment with business and technology strategy will also be a key part of your responsibilities. You will work in a rapid and agile development process to enable faster time-to-market while maintaining appropriate controls. Key Responsibilities: - Review and understand business requirements to complete development tasks within provided timelines - Collaborate with the software development team to optimize Data application performance - Interact with various teams and customers to design innovative solutions - Research new Big Data technologies and assess their alignment with business strategy - Work in a rapid and agile development process to increase speed to market Qualifications Required: - BE/B.Tech/MCA with a minimum of 4+ years of experience in design and development using Data Engineering technology stack and programming languages - Mandatory experience in Python/Scala/Java, Hadoop, HDFS, MR, Spark SQL, Dataframes, RDDs, SQL, Hive/Snowflake/SQL Server/Bigquery, Elastic Search - Preferred experience in 3 or more of the following areas: Spark Streaming, Spark ML, Kafka/Flume, Apache NiFi, Apache Airflow/Oozie, Cloud-based Data Platforms, NoSQL Databases HBase/Cassandra/Neo4j/MongoDB - Good knowledge of the current technology landscape and ability to visualize industry trends - Working knowledge of Big Data Integration with Third-party or in-house built Metadata Management, Data Quality, and Master Data Management solutions - Active community involvement through articles, blogs, or speaking engagements at conferences In this role, you will be responsible for reviewing and understanding business requirements to ensure timely completion of development tasks with minimal defects. You will collaborate with a software development team to implement best practices and optimize the performance of Data applications. Additionally, you will work closely with various teams within the company and engage with customers to translate their business challenges into innovative solutions. Researching new Big Data technologies and assessing their alignment with business and technology strategy will also be a key part of your responsibilities. You will work in a rapid and agile development process to enable faster time-to-market while maintaining appropriate controls. Key Responsibilities: - Review and understand business requirements to complete development tasks within provided timelines - Collaborate with the software development team to optimize Data application performance - Interact with various teams and customers to design innovative solutions - Research new Big Data technologies and assess their alignment with business strategy - Work in a rapid and agile development process to increase speed to market Qualifications Required: - BE/B.Tech/MCA with a minimum of 4+ years of experience in design and development using Data Engineering technology stack and programming languages - Mandatory experience in Python/Scala/Java, Hadoop, HDFS, MR, Spark SQL, Dataframes, RDDs, SQL, Hive/Snowflake/SQL Server/Bigquery, Elastic Search - Preferred experience in 3 or more of the following areas: Spark Streaming, Spark ML, Kafka/Flume, Apache NiFi, Apache Airflow/Oozie, Cloud-based Data Platforms, NoSQL Databases HBase/Cassandra/Neo4j/MongoDB - Good knowledge of the current technology landscape and ability to visualize industry trends - Working knowledge of Big Data Integration with Third-party or in-house built Metadata Management, Data Quality, and Master Data Management solutions - Active community involvement through articles, blogs, or speaking engagements at conferences