Lead Data Engineer

Job Purpose: Design, build, and optimize scalable data engineering solutions that enable reliable, high-performance data platforms. Lead the development of modern data pipelines, cloud-native architectures, and data infrastructure while collaborating with cross-functional teams to deliver robust, client-focused data solutions. Who You Are: 8+ years of experience in Data Engineering, with at least 2+ years in a Technical Lead or supervisory role Prior experience working in software services, IT consulting, or agency environments, managing multiple client engagements Strong proficiency in Python, SQL, and PySpark or Scala for large-scale data processing Hands-on experience with at least one major cloud platform (AWS, Azure, or GCP) and its data ecosystem (e.g., AWS Glue, Redshift, EMR, Azure Data Factory, Synapse, GCP BigQuery) Advanced experience working with modern data platforms such as Snowflake, Databricks, or Delta Lake Strong expertise in designing and building scalable ETL/ELT pipelines and data transformation workflows Experience with workflow orchestration tools such as Apache Airflow, dbt, or Prefect Hands-on experience with real-time data streaming technologies like Apache Kafka or Spark Streaming Good understanding of software engineering best practices, version control, testing, and CI/CD processes Excellent communication and stakeholder management skills with the ability to lead technical discussions and mentor engineering teams What Will Excite Us: Experience designing enterprise-scale data platforms and modern cloud data architectures Strong expertise in Databricks, Snowflake, or Delta Lake implementations Experience building real-time data pipelines using Kafka, Spark Streaming, or similar technologies Hands-on experience with DevOps and Infrastructure-as-Code tools such as Docker, Kubernetes, Terraform, GitHub Actions, Azure DevOps, or Jenkins Experience optimizing data pipeline performance, scalability, reliability, and cost efficiency Strong understanding of data governance, data quality, security, and performance optimization Experience leading technical teams, conducting code reviews, and mentoring data engineers Exposure to modern DataOps practices and automation frameworks Ability to work across multiple client projects while ensuring high-quality delivery What Will Excite You: Opportunity to architect and build large-scale cloud-native data platforms for global clients Work with modern technologies including Databricks, Snowflake, Kafka, Airflow, and leading cloud ecosystems Lead technical initiatives while mentoring and growing high-performing engineering teams Solve complex data engineering challenges involving large-scale data processing and real-time analytics Collaborate with cross-functional teams in a fast-paced, innovation-driven environment Competitive compensation, continuous learning opportunities, and significant career growth Location: Ahmedabad (Work From Office)