Source description
About the role
Role Overview We are looking for two Junior-level Big Data Engineers with strong hands-on experience in AWS, Python, Apache Airflow, and Big Data technologies . The ideal candidates will design, build, and maintain scalable data pipelines and workflows in a cloud-based environment, ensuring data reliability, performance, and security. Key Responsibilities Design, develop, and maintain ETL data pipelines using Apache Airflow. Build scalable data solutions on AWS, leveraging services such as: Amazon S3 EC2 AWS Lambda EMR AWS Glue Amazon Redshift Write clean, efficient, and maintainable Python code for data processing and automation. Work with Big Data frameworks such as Spark and Hadoop for large-scale data processing. Optimize workflows for performance, scalability, and cost efficiency. Collaborate with cross-functional teams to gather data requirements and deliver solutions. Ensure data quality, reliability, and security across pipelines and processes. Troubleshoot pipeline failures and resolve performance bottlenecks. Required Qualifications 3–5 years of hands-on experience in Data Engineering roles. Strong experience with AWS cloud services for data engineering workloads. Proficiency in Python, including libraries such as Pandas and PySpark. Hands-on experience with Apache Airflow, including DAG creation and monitoring. Familiarity with Big Data technologies (Spark, Hadoop, or similar). Solid understanding of ETL processes and data modeling concepts. Strong analytical, problem-solving, and debugging skills. Excellent communication and collaboration abilities. Preferred Qualifications Experience with CI/CD pipelines and DevOps practices. Knowledge of SQL and relational databases. Familiarity with containerization technologies such as Docker and Kubernetes. Exposure to monitoring and logging tools in cloud environments Skills: python,airflow,aws,big data
More at zorba ai