Data Engineer

Role Description We are looking for an experienced Data Engineer to design, build, and optimize robust data pipelines and platforms on AWS. You will work closely with data analysts, data scientists, and product teams to deliver reliable, scalable, and secure data solutions that power analytics and decision-making. Designation: Data Engineer / Senior Data Engineer Location: Noida / Gurgaon Job Description: Key Responsibilities: Design, build, and maintain scalable ETL/ELT pipelines and data workflows on AWS Develop and optimize data models (OLTP/OLAP), schemas, and warehousing solutions (e.g., Redshift, Snowflake, or similar) Build data ingestion frameworks for batch and streaming data from diverse sources (APIs, RDBMS, files, event streams) Implement data quality checks, validation rules, and monitoring for data reliability and SLAs Write efficient, production-grade Python and SQL for data transformations and orchestration Manage and optimize data storage, partitioning, and performance (query tuning, indexing, cost optimization) Implement orchestration and workflow management (e.g., Airflow/Step Functions) Collaborate with stakeholders to translate business requirements into technical specs and data contracts Ensure data security, governance, lineage, and compliance (IAM, encryption, PII handling) Set up observability for pipelines (logging, metrics, alerting) and support on-call rotations Contribute to CI/CD for data workloads, infrastructure-as-code, and best practices documentation Required Skills and Qualifications AWS: S3, Glue, Athena, Lambda, Step Functions, EMR or Glue Spark, Redshift, IAM, CloudWatch, CloudFormation/Terraform Programming: Python (pandas, PySpark), SQL (advanced) Orchestration: Apache Airflow (or MWAA), Step Functions Data Warehousing/Lakes: Redshift, Lake Formation, Parquet/Delta/Iceberg Streaming (nice-to-have): Kinesis, MSK (Kafka) Containers (nice-to-have): Docker, ECS/EKS Preferred: 4 - 5 years of hands-on data engineering experience in production environments Strong expertise in AWS data services and cost/performance trade-offs Advanced SQL skills (window functions, CTEs, query tuning) and Python for data engineering Solid experience with ETL/ELT design patterns, data modeling (star/snowflake), and schema evolution Experience with Spark (Glue/EMR) or similar distributed processing frameworks Proficiency with workflow orchestration (Airflow or equivalent) Practical knowledge of data quality frameworks and automated testing for data pipelines Experience with version control (Git) and CI/CD for data jobs Strong understanding of security best practices (IAM roles/policies, encryption, VPC, Secrets Manager)