Python-Spark Scala

Key Responsibilities: 5 9 years of experience in Data Engineering Big Data development Strong hands on experience with Python and Apache Spark Scala PySpark Deep understanding of Spark architecture RDD DataFrames Spark SQL Experience in distributed computing and big data processing Strong knowledge of SQL and data modeling concepts Experience with data pipeline development ETL ELT Familiarity with Linux Unix environments Experience with version control tools Git Technical Requirements: Primary skills Technology Big Data Data Processing Spark Technology Functional Programming Scala Technology Machine Learning Python Additional Responsibilities: Experience with cloud platforms AWS Azure or GCP Hands on with Databricks EMR Spark clusters Knowledge of streaming technologies Kafka Spark Streaming Structured Streaming Experience with workflow orchestration tools Airflow Oozie Familiarity with Delta Lake Lakehouse architecture Exposure to NoSQL databases MongoDB Cassandra Knowledge of CI CD and DevOps practices Preferred Skills: Technology->AI-Data science->PYTHON,Technology->Big Data - Data Processing->Spark->SparkSQL,Technology->Functional Programming->Scala .