Freelance Data Engineer

Flexing It is a freelance consulting marketplace that connects freelancers and independent consultants with organisations seeking independent talent. As a Consultant for our client, a leading global specialist in energy management and automation, your role will involve: - ETL pipeline development and delivery within defined timelines - Performance optimization of Spark/Glue jobs and SQL queries - Data pipeline reliability and reduction in production issues - AWS infrastructure automation using Terraform and CI/CD best practices - Code quality, test coverage, and adherence to development standards - Monitoring, logging, and incident resolution efficiency using CloudWatch - Successful implementation of secure IAM and cross-account access controls - Contribution to scalable data lake and governance initiatives Duration: 6 months Location: Bangalore Hybrid Model: 3 days WFO and 2 days WFH Qualifications Required: - 5+ years of experience in ETL design, development, and business system design - Good understanding of AWS networking concepts including VPCs, subnets, routing, NAT gateways, and security groups - Proficiency in IAM concepts such as roles, policies, assume-role, and cross-account access - Hands-on experience with AWS services including S3, KMS, Lambda, Glue (or Spark), SQS, and EventBridge - Experience in writing clean, modular, testable, and maintainable code using functional and/or OOP principles - Strong programming experience in at least one language such as Python, Scala, or Java - Willingness to learn Python/PySpark if not already experienced - Good understanding of Spark architecture, execution plans, and performance tuning - Strong SQL skills with experience in Redshift, Athena, or other distributed compute engines - Experience in writing and maintaining Terraform scripts - Familiarity with Git, GitHub workflows, and CI/CD tools such as Jenkins - Exposure to data lake architectures and data governance practices - Familiarity with monitoring and logging tools such as CloudWatch Flexing It is a freelance consulting marketplace that connects freelancers and independent consultants with organisations seeking independent talent. As a Consultant for our client, a leading global specialist in energy management and automation, your role will involve: - ETL pipeline development and delivery within defined timelines - Performance optimization of Spark/Glue jobs and SQL queries - Data pipeline reliability and reduction in production issues - AWS infrastructure automation using Terraform and CI/CD best practices - Code quality, test coverage, and adherence to development standards - Monitoring, logging, and incident resolution efficiency using CloudWatch - Successful implementation of secure IAM and cross-account access controls - Contribution to scalable data lake and governance initiatives Duration: 6 months Location: Bangalore Hybrid Model: 3 days WFO and 2 days WFH Qualifications Required: - 5+ years of experience in ETL design, development, and business system design - Good understanding of AWS networking concepts including VPCs, subnets, routing, NAT gateways, and security groups - Proficiency in IAM concepts such as roles, policies, assume-role, and cross-account access - Hands-on experience with AWS services including S3, KMS, Lambda, Glue (or Spark), SQS, and EventBridge - Experience in writing clean, modular, testable, and maintainable code using functional and/or OOP principles - Strong programming experience in at least one language such as Python, Scala, or Java - Willingness to learn Python/PySpark if not already experienced - Good understanding of Spark architecture, execution plans, and performance tuning - Strong SQL skills with experience in Redshift, Athena, or other distributed compute engines - Experience in writing and maintaining Terraform scripts - Familiarity with Git, GitHub workflows, and CI/CD tools such as Jenkins - Exposure to data lake architectures and data governance practices - Familiarity with monitoring and logging tools such as CloudWatch