Databricks Platform Engineer

Job Title: Databricks Platform Engineer

Job Type: Contract

Duration: 8 Months

Location: Remote

Role Summary

We are seeking a Databricks Platform Engineer to implement and operationalize core platform capabilities on Databricks running on AWS. This role focuses on frameworks, automation, observability, and reliable delivery aligned to established platform standards.

Key Responsibilities:

Platform Engineering:

Implement metadata-driven ingestion frameworks and standard ingestion patterns

Build batch, incremental, and CDC ingestion pipelines

Ensure pipelines follow platform standards for configuration, schema evolution, tagging, and error handling

Observability & Operations:

Implement pipeline monitoring, metrics, and alerting

Support operational validation, failure simulation, and recovery testing

Data Quality & Governance:

Embed data quality rules and quarantine patterns

Apply Unity Catalog tags (domain, owner, sensitivity, freshness) at ingestion time

Ensure governance metadata is written to governed schemas

CI/CD & Automation:

Build CI/CD pipelines using GitHub Actions

Support configuration validation, environment promotion, and rollback

Ensure secret management via AWS Secrets Manager

Required Qualifications:

5–8 years data or platform engineering experience

3+ years hands-on Databricks experience

Strong PySpark and SQL

Experience with Delta Lake, Unity Catalog, Git-based development

Preferred Qualifications

• Ingestion framework experience

• Monitoring and alerting implementation

• Production platform support experience

• Success Measures:

• Stable, reusable ingestion frameworks

• Reduced custom pipeline development

• Reliable CI/CD and proactive operational monitoring

• Pay rate: 70-70 USD per Hour Job type: Contract Division: eTeam Inc (US) Category: eTeam United States Reference: 26-35741