Senior Data Engineer-AWS

Position: Senior Data Engineer Location: Pune/Nagpur Type of Employment: FTE Purpose of the Position: This position requires a sound knowledge on Big Data using Cloud technologies such as such as Databricks, EMR, Athena, PySpark, S3, AWS Lambda etc. Strong foundation on database concepts and SQL is also required. Key Result Areas and Activities: Technology Assessment and DesignStudy existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements. Documentation and Stakeholder CommunicationProject documentation adheres to quality guidelines & schedules. Works hand in hand with Architect & PM for successful delivery of project and provide estimation, scoping, scheduling assistance. Process Improvement and AutomationSuggests automation to improve existing processes. Assist junior Data Engineers by providing expert advice or troubleshooting steps whenever required. Training and Knowledge SharingCreate technology-focused training plans whenever required. Deliver technology-focused training sessions with team members whenever required. Essential Skills: In-depth knowledge of the following AWS services: S3, EC2, EMR, Athena, AWS Glue, LambdaExperience with at least one MPP database: AWS Redshift, Snowflake, SingleStoreProficiency in Big Data technologies: Apache Spark, DatabricksMust have strong programming skills in PythonResponsible for building data pipelines in AWS And DatabricksExperience with Big Data table formats, such as Delta Lake (open source)Must have very strong SQL skillsExperience with orchestration tools like Apache AirflowExpertise in developing ETL workflows with complex transformations such as SCD, deduplications, aggregations, etc.Should be a quick and self-learner, ready to adapt to new AWS services or Big Data technologies as requiredStrong understanding of data warehousing concepts Desirable Skills: Cloud Databases Snowflake, AWS AuroraBig Data Hadoop, HiveCloud Databases AWS AuroraAssociate Level or Professional Level AWS Certification or Databricks Certification Qualifications: Overall, 7+ years of IT Exp5+ years of relevant experience in AWS related projectBachelors degree in computer science, engineering, or related field (master's degree is a plus)Demonstrated continued learning through one or more technical certifications or related methods. Qualities: Hold strong technical knowledge and experienceShould have the capability to deep dive and research in various technical related fieldsSelf-motivated and focused on delivering outcomes for a fast-growing team and firmAble to communicate persuasively through speaking, writing, and client presentationsAble to consult, write, and present persuasivelyAble to work in a self-organized and cross-functional teamAble to iterate based on new information, peer reviews, and feedbackPrior experience of working in a large media company would be added advantage Position: Senior Data Engineer Location: Pune/Nagpur Type of Employment: FTE Purpose of the Position: This position requires a sound knowledge on Big Data using Cloud technologies such as such as Databricks, EMR, Athena, PySpark, S3, AWS Lambda etc. Strong foundation on database concepts and SQL is also required. Key Result Areas and Activities: Technology Assessment and DesignStudy existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements. Documentation and Stakeholder CommunicationProject documentation adheres to quality guidelines & schedules. Works hand in hand with Architect & PM for successful delivery of project and provide estimation, scoping, scheduling assistance. Process Improvement and AutomationSuggests automation to improve existing processes. Assist junior Data Engineers by providing expert advice or troubleshooting steps whenever required. Training and Knowledge SharingCreate technology-focused training plans whenever required. Deliver technology-focused training sessions with team members whenever required. Essential Skills: In-depth knowledge of the following AWS services: S3, EC2, EMR, Athena, AWS Glue, LambdaExperience with at least one MPP database: AWS Redshift, Snowflake, SingleStoreProficiency in Big Data technologies: Apache Spark, DatabricksMust have strong programming skills in PythonResponsible for building data pipelines in AWS And DatabricksExperience with Big Data table formats, such as Delta Lake (open source)Must have very strong SQL skillsExperience with orchestration tools like Apache AirflowExpertise in developing ETL workflows with complex transformations such as SCD, deduplications, aggregations, etc.Should be a quick and self-learner, ready to adapt to new AWS services or Big Data technologies as requiredStrong understanding of data warehousing concepts Desirable Skills: Cloud Databases Snowflake, AWS AuroraBig Data Hadoop, HiveCloud Databases AWS A