Sr Data Engineer

As a Senior Data Engineer at IDMC, your role will primarily involve the design, development, maintenance, and enhancement of Data Pipelines (ETL/ELT) and processes. You should possess thorough knowledge of star/snowflake schemas and hands-on experience in PowerCenter Developer and IDMC Data Integration. Your responsibilities will include: - Developing complex SQL queries and optimizing SQL for efficient data processing. - Designing and implementing data security measures to ensure data integrity. - Monitoring and optimizing data storage and processing for optimal performance. - Utilizing Object function/object-oriented scripting languages such as Scala, C++, Java, and Python. - Working with cloud data warehouse and data lake platforms like Snowflake, Databricks, Redshift, BigQuery, MS Synapse, MS ADLS, and Apache Hadoop. - Demonstrating knowledge of message queuing, stream processing, and highly scalable 'big data' data stores. - Showcasing robust project management and organizational skills to support cross-functional teams effectively. - Understanding data science concepts and familiarity with AI/ML frameworks and libraries. Additionally, you will be expected to exhibit the following qualifications: - Minimum 8+ years of experience in Design and Development within Business Intelligence/Data Warehousing Architecture. - Experience in DW Production Support and troubleshooting data or pipeline issues. - Full Life Cycle development experience including business requirements gathering, data sourcing, testing/data reconciliation, and deployment. - Working knowledge of advanced SQL, relational databases, ETL tools, NoSQL databases, and big data tools like Kafka, Spark, and Hadoop. - Demonstrated expertise in building and optimizing 'big data' data pipelines, architecture, and data sets. - Ability to perform root cause analysis on internal and external data and processes to address specific business questions. This role at IDMC offers you the opportunity to work on developing data services that are resilient, scalable, and future-proof to meet user needs effectively. If you are looking to leverage your expertise in data engineering and contribute to a dynamic and energetic workplace, this position is tailored for you. As a Senior Data Engineer at IDMC, your role will primarily involve the design, development, maintenance, and enhancement of Data Pipelines (ETL/ELT) and processes. You should possess thorough knowledge of star/snowflake schemas and hands-on experience in PowerCenter Developer and IDMC Data Integration. Your responsibilities will include: - Developing complex SQL queries and optimizing SQL for efficient data processing. - Designing and implementing data security measures to ensure data integrity. - Monitoring and optimizing data storage and processing for optimal performance. - Utilizing Object function/object-oriented scripting languages such as Scala, C++, Java, and Python. - Working with cloud data warehouse and data lake platforms like Snowflake, Databricks, Redshift, BigQuery, MS Synapse, MS ADLS, and Apache Hadoop. - Demonstrating knowledge of message queuing, stream processing, and highly scalable 'big data' data stores. - Showcasing robust project management and organizational skills to support cross-functional teams effectively. - Understanding data science concepts and familiarity with AI/ML frameworks and libraries. Additionally, you will be expected to exhibit the following qualifications: - Minimum 8+ years of experience in Design and Development within Business Intelligence/Data Warehousing Architecture. - Experience in DW Production Support and troubleshooting data or pipeline issues. - Full Life Cycle development experience including business requirements gathering, data sourcing, testing/data reconciliation, and deployment. - Working knowledge of advanced SQL, relational databases, ETL tools, NoSQL databases, and big data tools like Kafka, Spark, and Hadoop. - Demonstrated expertise in building and optimizing 'big data' data pipelines, architecture, and data sets. - Ability to perform root cause analysis on internal and external data and processes to address specific business questions. This role at IDMC offers you the opportunity to work on developing data services that are resilient, scalable, and future-proof to meet user needs effectively. If you are looking to leverage your expertise in data engineering and contribute to a dynamic and energetic workplace, this position is tailored for you.