BI AWS Developer

BI AWS Developer

Contarct Role

Sr. Developer AWS

Responsibilities

• Design and Develop ETL Processes in AWS Glue to migrate data from S3 with ORC/Parquet/Text Files.

• Data Extraction, aggregations and consolidation of data

• Create external and managed tables with partitions in Redshift using Glue Catalog

• Create user defined functions UDF in Redshift and in Python

• Build S3 buckets and managed policies for S3 buckets and used S3 bucket and Glacier for storage and backup on AWS .

• Client IAM service enabled to grant permissions and resources to users. Managed roles and permissions of users with the help of AWS IAM .

• Must Have

• Overall 8+ years of experience on BI/Data Platform

• 3+ yrs working experience on AWS platform using data services

• Big Data Ecosystems (Must have working experience): S3, Redshift, Glue and at least one ingestion service like DMS, Appflow, Data Transfer/Data Sync

• Must have used Step Functions with Lamda

• Scripting Languages: Python

• Understanding of cloud watch, SNS and even bridge

• Excellent analytical and problem-solving skills.

• Experience working in Agile, Scrum

• Good to Have

• Exposure to Big Data Ecosystems ( Lake Formation, Data Migration Service, Appflow, AWS EMR, DynamoDB)

• Scripting Languages: PySpark

• Exposure to Alembic

• Exposure to CI/CD pipeline (Gitlab). Preferable AWS Code Commit, Code Build, Code Deploy

• And this is bare minimum needed

• Overall 8+ years of experience on BI/Data Platform with 3+ yrs working experience on AWS platform using data services

• Big Data Ecosystems (Must have working experience): S3, Redshift, Glue and at least one ingestion service like DMS, Appflow, Data Transfer/Data Sync

• Scripting Languages: Python or pySpark

Responsibilities

• Design, Develop and Automate processes in AWS Glue pipelines from S3 with ORC/Parquet/Text Files.

• Defining standards , guidelines and access mechanism (naming conventions, IAM Users and Roles, SSO profile)

• Define Data Extraction, aggregations using Python, pySpark using relevant libraries

• Managing external and managed tables with partitions in S3 and Redshift

• Create libraries for user defined functions UDF

• Build S3 buckets and managed policies for S3 buckets and used S3 bucket and Glacier for storage and backup on AWS .

• Must Have

• Overall 10+ years of experience on BI/Data Platform

• 5+ yrs working experience on AWS platform using data services

• Working experience in S3, Redshift, Glue and ingestion services like DMS, Appflow, Data Transfer/Data Sync

• Create state machines interacting with lamda, glue, clouldwatch, SNS, even bridge etc.

• Scripting Languages: Python, pySpark

• Understanding of cloud watch, SNS and even bridge

• Excellent analytical and problem-solving skills.

• Experience in CI/CD pipeline (Gitlab). Preferably AWS Code Commit, Code Build, Code Deploy

• Good to Have

• Exposure to Big Data Ecosystems ( Lake Formation, AWS EMR)

• Exposure to Alembic

• Experience working in Agile, Scrum

• And this is bare minimum needed

• Overall 10+ years of experience on BI/Data Platform with 5+ yrs working experience on AWS platform using data services

• Working experience in S3, Redshift, Glue and ingestion services like DMS, Appflow, Data Transfer/Data Sync

• Create state machines interacting with lamda, glue, clouldwatch, SNS, even bridge etc.

• Experience in CI/CD pipeline (Gitlab). Preferably AWS Code Commit, Code Build, Code Deploy Job type: Contract Division: eTeam Workforce Limited Reference: 22-06481