Source description
About the role
⃣ Sr. Data Engineers (2) – Databricks Pipeline Developers Role Overview Build and maintain data pipelines in Databricks for ingestion, cleansing, enrichment, and storage. Key Responsibilities Develop PySpark pipelines for API ingestion FTP file ingestion Data cleansing Enrichment transformations Implement incremental & batch loads Develop Bronze → Silver → Gold transformations Translate legacy SQL stored procedures into Spark jobs Optimize Delta Lake tables Implement validation checks Support testing & reconciliation Required Skills Strong PySpark / Spark SQL Strong SQL (read & interpret legacy procedures) Experience with API integration Experience handling flat files (FTP ingestion) Experience with Delta Lake Knowledge of partitioning & optimization
More at Virtusa