Technical Lead- AWS, Databricks

Job Description: Experience Range 10+ years Primary (Must have skills)* - To be Screened by TA Team 10+ years total experience in Data Engineering, Cloud Data Platforms 5+ years hands-on experience with Databricks (PySpark/Scala, Delta Lake, Lakehouse architecture) 4+ years' experience in AWS Glue (Glue ETL jobs, Workflows, Crawlers, Glue Catalog). 3+ years' experience designing data ingestion and transformation pipelines using Spark-based frameworks. Job Description of Role Technical Leadership Drive end-to-end technical solutions, ensuring performance, scalability, and reliability. Establish coding standards, best practices, and development guidelines. Conduct code reviews, provide technical mentorship, and resolve complex technical issues. Data Platform Architecture Architect and implement large-scale data ingestion, transformation, and storage layers. Design Lakehouse architecture using Delta Lake, Unity Catalog, Medallion model. Ensure data quality, lineage, and governance across data assets. Collaborate with cloud architects to integrate platform services across Azure and AWS. Solution Development Develop optimized Spark jobs (PySpark/Scala) with strong performance tuning practices. Implement ETL/ELT pipelines using Databricks notebooks, Workflows, and Glue Jobs. Integrate data from multiple sources into ADLS / S3 with appropriate security and metadata management. Build CI/CD pipelines using Azure DevOps / GitHub Actions. Cloud Infrastructure & Security Define IAM roles, managed identities, ACLs, and storage permissions for secure operations. Work with Terraform/CloudFormation for infrastructure automation (preferred). Monitor cloud resource utilization and optimize cost. Cross-Functional Collaboration Interact with stakeholders, analysts, architects, and business teams to translate requirements into technical designs. Present architectural recommendations and solution proposals to leadership. Coordinate with DevOps and Cloud Engineering for deployment and operational readiness. Job Description: Experience Range 10+ years Primary (Must have skills)* - To be Screened by TA Team 10+ years total experience in Data Engineering, Cloud Data Platforms 5+ years hands-on experience with Databricks (PySpark/Scala, Delta Lake, Lakehouse architecture) 4+ years' experience in AWS Glue (Glue ETL jobs, Workflows, Crawlers, Glue Catalog). 3+ years' experience designing data ingestion and transformation pipelines using Spark-based frameworks. Job Description of Role Technical Leadership Drive end-to-end technical solutions, ensuring performance, scalability, and reliability. Establish coding standards, best practices, and development guidelines. Conduct code reviews, provide technical mentorship, and resolve complex technical issues. Data Platform Architecture Architect and implement large-scale data ingestion, transformation, and storage layers. Design Lakehouse architecture using Delta Lake, Unity Catalog, Medallion model. Ensure data quality, lineage, and governance across data assets. Collaborate with cloud architects to integrate platform services across Azure and AWS. Solution Development Develop optimized Spark jobs (PySpark/Scala) with strong performance tuning practices. Implement ETL/ELT pipelines using Databricks notebooks, Workflows, and Glue Jobs. Integrate data from multiple sources into ADLS / S3 with appropriate security and metadata management. Build CI/CD pipelines using Azure DevOps / GitHub Actions. Cloud Infrastructure & Security Define IAM roles, managed identities, ACLs, and storage permissions for secure operations. Work with Terraform/CloudFormation for infrastructure automation (preferred). Monitor cloud resource utilization and optimize cost. Cross-Functional Collaboration Interact with stakeholders, analysts, architects, and business teams to translate requirements into technical designs. Present architectural recommendations and solution proposals to leadership. Coordinate with DevOps and Cloud Engineering for deployment and operational readiness.