Data Engineer Architect

As a Data Engineer with 10 years of relevant work experience, you have shown continuous growth in your career. Your hands-on programming experience includes implementation on Kafka, Kinesis, Spark, AWS Glue, and AWS LakeFormation. You have expertise in performance optimization for both Batch and Real-time processing applications. Additionally, you possess strong skills in Data Governance and Data Security Implementation. Key Responsibilities: - Work on Scheduling tools like Airflow - Utilize design and programming skills to build reusable tools and products - Develop in AWS or similar cloud platforms, with a preference for ECS, EKS, S3, EMR, DynamoDB, Aurora, Redshift, QuickSight, or similar - Have a good knowledge of Python, with Java/Scala as a plus - Work on systems with high transaction volumes, microservice design, and data processing pipelines like Spark - Efficient data processing using open table formats Delta, Iceberg - Hands-on experience with serverless technologies such as Lambda, MSK, MWAA, Kinesis Analytics is advantageous - Implement practices like Agile, Peer reviews, Continuous Integration - Capable of planning and executing short-term and long-term goals individually and with the team - Design and Architecture experience including High Level and Low Level Design - Implement data catalog solution, data observability framework, Audit Balance control framework - Certification in Data Engineering, AWS, etc. - Implemented Micro-service API to process thousands of events per second Qualifications Required: - 10+ years of relevant work experience as a Data Engineer - Hands-on programming experience in Kafka, Kinesis, Spark, AWS Glue, AWS LakeFormation - Experience in performance optimization for Batch and Real-time processing applications - Expertise in Data Governance and Data Security Implementation - Proficiency in using Scheduling tools like Airflow - Strong design and programming skills for building reusable tools and products - Development experience in AWS or similar cloud platforms, with knowledge in ECS, EKS, S3, EMR, DynamoDB, Aurora, Redshift, QuickSight, or similar - Good knowledge of Python, with Java/Scala as a bonus - Familiarity with systems handling high transaction volumes, microservice design, and data processing pipelines (Spark) - Efficient data processing using open table format Delta, Iceberg - Hands-on experience with serverless technologies such as Lambda, MSK, MWAA, Kinesis Analytics - Expertise in Agile practices, Peer reviews, Continuous Integration - Ability to plan and execute on short-term and long-term goals individually and as part of a team - Design and Architecture experience encompassing High Level and Low Level Design - Implementation of data catalog solution, data observability framework, Audit Balance control framework - Certification in Data Engineering, AWS, etc. - Implemented Micro-service API to process thousands of events per second As a Data Engineer with 10 years of relevant work experience, you have shown continuous growth in your career. Your hands-on programming experience includes implementation on Kafka, Kinesis, Spark, AWS Glue, and AWS LakeFormation. You have expertise in performance optimization for both Batch and Real-time processing applications. Additionally, you possess strong skills in Data Governance and Data Security Implementation. Key Responsibilities: - Work on Scheduling tools like Airflow - Utilize design and programming skills to build reusable tools and products - Develop in AWS or similar cloud platforms, with a preference for ECS, EKS, S3, EMR, DynamoDB, Aurora, Redshift, QuickSight, or similar - Have a good knowledge of Python, with Java/Scala as a plus - Work on systems with high transaction volumes, microservice design, and data processing pipelines like Spark - Efficient data processing using open table formats Delta, Iceberg - Hands-on experience with serverless technologies such as Lambda, MSK, MWAA, Kinesis Analytics is advantageous - Implement practices like Agile, Peer reviews, Continuous Integration - Capable of planning and executing short-term and long-term goals individually and with the team - Design and Architecture experience including High Level and Low Level Design - Implement data catalog solution, data observability framework, Audit Balance control framework - Certification in Data Engineering, AWS, etc. - Implemented Micro-service API to process thousands of events per second Qualifications Required: - 10+ years of relevant work experience as a Data Engineer - Hands-on programming experience in Kafka, Kinesis, Spark, AWS Glue, AWS LakeFormation - Experience in performance optimization for Batch and Real-time processing applications - Expertise in Data Governance and Data Security Implementation - Proficiency in using Scheduling tools like Airflow - Strong design and programming skills for building reusable tools and products - Development experience in AWS or similar cloud platforms, with knowledge in E