Source description
About the role
Skills Required: Python, ETL, AWS, Airflow/DBT, Pyspark , Spark.
Roles Responsibilities:
Mandatory skills (8+ Years of experience in ETL development with 4+ Years on AWS Pyspark scripting)
Experience deploying and running AWS-based data solutions using services or products such as S3, Lambda, SNS, Cloud Step Functions.
Person should be strong inPyspark
Hands on and working knowledge in Python packages like NumPy, Pandas,Etc
Experience deploying and running AWS-based data solutions using services or products such as S3, Lambda, SNS, Cloud Step Functions. Sound knowledge in AWS services is must.
Person should work as Individual contributor
Good to have familiar with metadata management, data lineage, and principles of data governance.
Good to have:
Experience to process large set of data transformations both semi and structured data
Experienceto builddata lake & configuration on delta tables.
Good experiencewith computing & cost optimization.
Understanding the environment and use case and ready to build holistic Data Integration frame works.
Good experiencein MWAA (airflow orchestration)
Soft skill:
Having good communicationto interact with IT-Stake holders and Business.
Understand the pain point to deliver
More at Relevantz