Source description
About the role
Senior & Lead Data Engineer (610 Years) We are seeking a Senior & Lead Data Engineer with 610 years of experience in designing and building scalable data platforms. The ideal candidate should have solid expertise in Python, Advanced SQL, PySpark/Spark, AWS, Airflow, Kafka/Kinesis, and Databricks. Experience in developing automated batch and real-time data pipelines, ensuring data quality, governance, and performance optimization is required. Key Skills - Python & Advanced SQL - Apache Spark / PySpark - AWS Cloud Services - Apache Airflow & Workflow Orchestration - Kafka / Kinesis Streaming - Databricks (Delta Lake, Unity Catalog, DLT) - dbt (Data Build Tool) - Data Pipeline Automation - Data Governance & Data Quality - Feature Stores - LLM-ready Data Pipelines / RAG - MLOps Exposure Preferred: - Experience with Snowflake, Redshift, or modern data platforms. - Exposure to GenAI, Vector Databases, and ML/AI data ecosystems. - Strong problem-solving and stakeholder management skills. .
More at Iris Software