Source description
About the role
Spark Developer Experience: 513 Years | Relevant Location: Hyderabad Job Overview: We are looking for an experienced Spark Developer with 510 years of experience in building scalable data processing solutions using Apache Spark. The ideal candidate should have solid expertise in PySpark/Scala Spark, SQL, ETL development, and Big Data technologies, with experience in designing and optimizing high-performance data pipelines. Key Responsibilities: Design, develop, and maintain scalable data processing applications using Apache Spark. Develop ETL/ELT pipelines for processing large volumes of structured and unstructured data. Write optimized Spark jobs using PySpark or Scala. Work with distributed data processing frameworks and Big Data technologies. Integrate data from multiple sources including databases, APIs, and cloud storage. Optimize Spark jobs for performance, scalability, and reliability. Collaborate with data engineers, analysts, and business teams to understand requirements. Perform code reviews, testing, debugging, and production support. Ensure data quality, security, and governance standards are followed. Prepare technical documentation and support deployment activities. Required Skills: 510 years of experience in Data Engineering or Big Data development. Solid hands-on experience with Apache Spark. Expertise in PySpark or Scala. Strong SQL skills. Experience with ETL/ELT development. Knowledge of Hadoop ecosystem (Hive, HDFS, YARN). Experience with Databricks is preferred. Experience with Kafka or other streaming technologies is an advantage. Familiarity with cloud platforms such as Azure, AWS, or GCP. Experience with Git and CI/CD practices. Robust analytical and problem-solving skills. Preferred Skills: Experience with Delta Lake, Snowflake, or Data Lake architectures. Knowledge of Airflow or other workflow orchestration tools. Experience in Agile/Scrum environments. Good communication and stakeholder management skills. Work Location: In person Spark Developer Experience: 513 Years | Relevant Location: Hyderabad Job Overview: We are looking for an experienced Spark Developer with 510 years of experience in building scalable data processing solutions using Apache Spark. The ideal candidate should have solid expertise in PySpark/Scala Spark, SQL, ETL development, and Big Data technologies, with experience in designing and optimizing high-performance data pipelines. Key Responsibilities: Design, develop, and maintain scalable data processing applications using Apache Spark. Develop ETL/ELT pipelines for processing large volumes of structured and unstructured data. Write optimized Spark jobs using PySpark or Scala. Work with distributed data processing frameworks and Big Data technologies. Integrate data from multiple sources including databases, APIs, and cloud storage. Optimize Spark jobs for performance, scalability, and reliability. Collaborate with data engineers, analysts, and business teams to understand requirements. Perform code reviews, testing, debugging, and production support. Ensure data quality, security, and governance standards are followed. Prepare technical documentation and support deployment activities. Required Skills: 510 years of experience in Data Engineering or Big Data development. Solid hands-on experience with Apache Spark. Expertise in PySpark or Scala. Strong SQL skills. Experience with ETL/ELT development. Knowledge of Hadoop ecosystem (Hive, HDFS, YARN). Experience with Databricks is preferred. Experience with Kafka or other streaming technologies is an advantage. Familiarity with cloud platforms such as Azure, AWS, or GCP. Experience with Git and CI/CD practices. Robust analytical and problem-solving skills. Preferred Skills: Experience with Delta Lake, Snowflake, or Data Lake architectures. Knowledge of Airflow or other workflow orchestration tools. Experience in Agile/Scrum environments. Good communication and stakeholder management skills. Work Location: In person
More at Ibroad Solutions