Associate Principal Engineer (India Office)

Data professional with experience in building and validating end-to-end data pipelines across Azure Data Factory, Databricks, and ADLS. Skilled in data quality assurance, automation using Python and PySpark, and integrating tests into CI/CD workflows. Adept at large-scale data validation, monitoring data health, and ensuring reliable batch and streaming processes, with additional expertise in developing scalable BI dashboards and reporting solutions.

Data professional with experience in building and validating end-to-end data pipelines across Azure Data Factory, Databricks, and ADLS. Skilled in data quality assurance, automation using Python and PySpark, and integrating tests into CI/CD workflows. Adept at large-scale data validation, monitoring data health, and ensuring reliable batch and streaming processes, with additional expertise in developing scalable BI dashboards and reporting solutions.

Design and implement end-to-end data validation strategies for pipelines built on Azure Data Factory, Databricks (Delta Lake), and ADLS

Perform source-to-target reconciliation across ingestion, transformation, and consumption layers

Implement robust checks for Data Quality Dimensions

Build scalable data testing frameworks using python, pyspark and good knowledge of automation tools like playwright & pyspark based automation testing.

Integrate automated data tests into CI/CD pipelines (Azure DevOps / GitHub Actions)

Implement data quality-as-code practices using tools like Soda, dbt etc.

Validate Spark transformations, Delta Lake tables, and streaming pipelines

Perform large-scale data validation using PySpark and SQL

Optimize validation logic for high-volume datasets

Ensure correctness of: Batch and streaming jobs, Incremental loads (CDC pipelines), Slowly changing dimensions (SCD)

Validate data movement, orchestration workflows, and failure handling in Azure Data factory & Azure Databricks services

Define and implement data quality SLAs and KPIs

Build dashboards to track data health and pipeline reliability

Proactively identify anomalies and data drifts

Implement alerting mechanisms for data failures

Design, develop, and optimize interactive dashboards and reports using BI tools (e.g., Power BI, Tableau, Looker).

Translate business requirements into technical reporting solutions.

Identify bottlenecks and improve report/query performance.

Implement best practices for dashboard design and scalability.

Establish BI standards, frameworks, and best practices.

Required Qualifications

8-10 years experience in enterprise data modeling and data architecture roles.

Strong hands-on experience with Azure Data Factory & Pyspark in large-scale environments is preferred.

Experience with Databricks (Delta Lake, Spark, Unity Catalog).

Advanced SQL and strong understanding of pyspark scripts & using pyspark for Data validation.

Experience integrating major enterprise applications (ERP, CRM, OMS, MDM, AR systems).

Strong understanding of data governance, data quality, metadata, and lineage .

Excellent communication skills across business and technical audiences.

Core Technology Stack

Analytics Platform: Databricks (Delta Lake, Unity Catalog), Microsoft PowerBI

Languages: SQL, Spark SQL, Python / PySpark

Governance/Data Quality: Unity Catalog/Informatica DG/DQ

#LI-KS1

At Wesco, we build, connect, power and protect the world. As a leading provider of business-to-business distribution, logistics services and supply chain solutions, we create a world that you can depend on. ​

Our Company’s greatest asset is our people. Wesco is committed to fostering a workplace where every individual is respected, valued, and empowered to succeed. We promote a culture that is grounded in teamwork and respect. With a workforce of over 20,000 people worldwide, we embrace the unique perspectives each person brings. Through comprehensive benefits and active community engagement, we create an environment where every team member has the opportunity to thrive. ​

Learn more about Working at Wesco here and apply online today!​

Founded in 1922 and headquartered in Pittsburgh, Wesco is a publicly traded (NYSE: WCC) FORTUNE 500® company.​

Wesco International, Inc., including its subsidiaries and affiliates (“Wesco”) provides equal employment opportunities to all employees and applicants for employment. Employment decisions are made without regard to race, religion, color, national or ethnic origin, sex, sexual orientation, gender identity or expression, age, disability, or other characteristics protected by law. US applicants only, we are an Equal Opportunity Employer.​

Los Angeles Unincorporated County Candidates Only: Qualified applicants with arrest or conviction records will be considered for employment in accordance with the Los Angeles County Fair Chance Ordinance and the California Fair Chance Act.

This posting is for a current, active vacancy intended for immediate hire.