Source description
About the role
Senior & Lead Data Engineer (610 Years) We are seeking a Senior & Lead Data Engineer with 610 years of experience in designing and building scalable data platforms. The ideal candidate should have strong expertise in Python, Advanced SQL, PySpark/Spark, AWS, Airflow, Kafka/Kinesis, and Databricks. Experience in developing automated batch and real-time data pipelines, ensuring data quality, governance, and performance optimization is required. Key Skills Python & Advanced SQL Apache Spark / PySpark AWS Cloud Services Apache Airflow & Workflow Orchestration Kafka / Kinesis Streaming Databricks (Delta Lake, Unity Catalog, DLT) dbt (Data Build Tool) Data Pipeline Automation Data Governance & Data Quality Feature Stores LLM-ready Data Pipelines / RAG MLOps Exposure Preferred: Experience with Snowflake, Redshift, or modern data platforms. Exposure to GenAI, Vector Databases, and ML/AI data ecosystems. Strong problem-solving and stakeholder management skills. .
More at Iris Software