Collibra Data Lineage Automation Engineer

Collibra Data Lineage Automation Engineer

6- Months Contract

Location: Role is on-site 5 days/week in McLean, VA.

Max rate = $68/hr. on C2C

We are seeking a highly experienced Data Lineage Automation Engineer to lead the design and implementation of automated end-to-end lineage solutions across a highly heterogeneous enterprise data ecosystem. This role requires deep technical expertise in lineage frameworks (such as Spline and OpenLineage), experience across cloud and legacy environments, and a strong AI foundation to support intelligent metadata extraction and traceability.

Key Responsibilities

• Lead the implementation of automated data lineage across a complex data estate that includes: o Cloud platforms (e.g., Snowflake, AWS) o Legacy relational databases and ETLs o NoSQL data stores o BI/reporting platforms (e.g., Tableau, Power BI)

• Implement or extend frameworks such as Spline, OpenLineage, or similar open frameworks to support active lineage capture

• Build connectors, extractors, or agents where necessary to bridge gaps between systems and lineage frameworks

• Integrate with metadata platforms (e.g., Collibra) to publish lineage in a consumable format

• Apply AI/ML techniques to infer lineage where automation is incomplete (e.g., handling Java based ETLs), using logs, query patterns, or usage metadata

• Develop reusable lineage components for operational reuse across domains

• Guide stakeholders on best practices for lineage standardization, storage, and use

Required Skills & Experience

• Proven experience delivering automated data lineage solutions across hybrid architectures

• Hands-on expertise with Spline, OpenLineage, Marquez, or comparable lineage frameworks

• Deep understanding of metadata capture, ETL process tracing, and query execution mapping

• Strong AI/ML background — particularly in metadata intelligence, natural language processing for code parsing, or pattern detection

• Experience integrating lineage with data governance tools (e.g., Collibra, Alation, etc.)

• Strong programming background in Python, Scala, or Java

• Deep familiarity with SQL and query logs from systems like Snowflake, SQL Server, Oracle, MongoDB, etc.

Big Plus Skills

• Experience with third-party commercial data lineage solutions a plus (evaluations and implementations)

• Prior work in regulated environments (e.g., financial services, healthcare)

• Familiarity with event-based architectures for real-time lineage propagation

• Knowledge of data mesh or domain-driven lineage strategies

Ideal Candidate

• Has successfully implemented automated lineage at enterprise scale

• Operates at the intersection of data engineering, metadata management, and AI

• Can act as a technical thought partner to architecture teams and governance leads

• Brings the mindset of automation-first and reuse-oriented design