Source description
About the role
We are seeking experienced ETL Data Test Automation Engineers with Databricks expertise to validate complex cloud data pipelines. Work onsite to ensure data quality, integrity, and performance across ETL/ELT processes using Python automation, PySpark testing, and advanced SQL validations. Key Responsibilities Design/execute ETL/ELT test automation frameworks for cloud data pipelines Validate Databricks pipelines (Notebooks, Jobs, Delta Lake, Unity Catalog) Write Python test automation using PyTest/UnitTest + Pandas/PySpark Perform complex SQL validations (joins, window functions, aggregations) Test data lineage, metadata management, SCD transformations Automate API/CLI testing for data platform components Collaborate with data engineers for test-driven development Mandatory Technical Skills ETL Fundamentals: Data modeling (Star/Snowflake), SCDs, lineage Databricks: Notebooks, Jobs, Delta Lake, Unity Catalog, SQL Warehouse Python: PyTest/UnitTest, Pandas, PySpark for test data manipulation SQL: Complex joins, window functions, performance validation Cloud Testing: Data/ETL testing in AWS/Azure/GCP environments Experience: 6-8 years QA (3+ years data testing) Skills: testing,sql,python,automation,cloud,etl,data
More at getege edtech pvt. ltd.