Source description
About the role
As a Data Engineer with expertise in AWS and PySpark, your role will involve building and maintaining scalable data pipelines and cloud-based solutions. You will collaborate with cross-functional teams to design, develop, and optimize modern data platforms and analytics solutions. Your responsibilities will include: - Designing, developing, and maintaining ETL/ELT pipelines using PySpark and cloud-native technologies. - Building and optimizing data ingestion, transformation, and processing workflows on AWS. - Developing and managing data models, data marts, and enterprise data warehouse solutions in Snowflake. - Ensuring data quality, integrity, security, and governance across data platforms. - Creating and supporting reporting and visualization solutions using Tableau. - Monitoring and optimizing data pipeline performance for large-scale datasets. - Collaborating with business stakeholders, analysts, and data scientists to understand data requirements. - Troubleshooting and resolving data-related issues in production environments. - Implementing best practices for data engineering, automation, and DevOps processes. - Documenting technical designs, workflows, and operational procedures. Qualifications Required: - 4+ years of experience in Data Engineering and Data Warehousing. - Strong hands-on experience with PySpark, AWS (S3, Glue, EMR, Lambda, Redshift, Athena, or related services), Snowflake, and Tableau. - Strong SQL programming and query optimization skills. - Experience designing and developing ETL/ELT pipelines. - Knowledge of data modeling, schema design, and performance tuning. - Experience working with structured and semi-structured data. - Understanding of data governance, security, and compliance best practices. - Familiarity with version control tools such as Git. Preferred Skills: - Experience with Apache Spark ecosystem and big data technologies. - Knowledge of orchestration tools such as Airflow. - Exposure to CI/CD pipelines and DevOps practices. - Experience with Python-based data processing frameworks. - Understanding of Agile/Scrum methodologies. Educational Qualification: - Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field. As a Data Engineer with expertise in AWS and PySpark, your role will involve building and maintaining scalable data pipelines and cloud-based solutions. You will collaborate with cross-functional teams to design, develop, and optimize modern data platforms and analytics solutions. Your responsibilities will include: - Designing, developing, and maintaining ETL/ELT pipelines using PySpark and cloud-native technologies. - Building and optimizing data ingestion, transformation, and processing workflows on AWS. - Developing and managing data models, data marts, and enterprise data warehouse solutions in Snowflake. - Ensuring data quality, integrity, security, and governance across data platforms. - Creating and supporting reporting and visualization solutions using Tableau. - Monitoring and optimizing data pipeline performance for large-scale datasets. - Collaborating with business stakeholders, analysts, and data scientists to understand data requirements. - Troubleshooting and resolving data-related issues in production environments. - Implementing best practices for data engineering, automation, and DevOps processes. - Documenting technical designs, workflows, and operational procedures. Qualifications Required: - 4+ years of experience in Data Engineering and Data Warehousing. - Strong hands-on experience with PySpark, AWS (S3, Glue, EMR, Lambda, Redshift, Athena, or related services), Snowflake, and Tableau. - Strong SQL programming and query optimization skills. - Experience designing and developing ETL/ELT pipelines. - Knowledge of data modeling, schema design, and performance tuning. - Experience working with structured and semi-structured data. - Understanding of data governance, security, and compliance best practices. - Familiarity with version control tools such as Git. Preferred Skills: - Experience with Apache Spark ecosystem and big data technologies. - Knowledge of orchestration tools such as Airflow. - Exposure to CI/CD pipelines and DevOps practices. - Experience with Python-based data processing frameworks. - Understanding of Agile/Scrum methodologies. Educational Qualification: - Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
More at Funic Tech