Source description
About the role
As a Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines using SSIS and SQL Server for enterprise data processing. You will also develop and optimize Databricks (PySpark / Spark SQL) pipelines for large-scale data transformation and analytics. Your key responsibilities include: - Working on batch and near-real-time data ingestion, transformation, and loading (ETL/ELT) across multiple source systems. - Demonstrating strong hands-on experience with SQL, stored procedures, and performance tuning in SQL Server environments. - Integrating legacy ETL (SSIS) processes with modern cloud-based data platforms (Databricks / Lakehouse architecture). - Understanding insurance lifecycle processes and data flows across core insurance systems. - Supporting data migration and modernization initiatives from on-premise to cloud platforms (Azure/AWS preferred). - Ensuring data quality, validation, reconciliation, and governance across all pipelines. - Monitoring and troubleshooting ETL jobs to ensure high availability, reliability, and performance. - Collaborating with business analysts, actuaries, and downstream BI/reporting teams to deliver data solutions. - Working closely with data architects and engineers to define scalable data models and standards. - Implementing logging, error handling, and auditing frameworks for data pipelines. - Participating in Agile ceremonies, code reviews, and production deployment activities. - Ensuring compliance with insurance regulatory, audit, and data security requirements. You should have experience working with P&C Insurance data domains such as Policy, Claims, Billing, Underwriting, and Loss data. Additionally, you will be expected to have a good understanding of insurance regulatory, audit, and data security requirements to ensure compliance. As a Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines using SSIS and SQL Server for enterprise data processing. You will also develop and optimize Databricks (PySpark / Spark SQL) pipelines for large-scale data transformation and analytics. Your key responsibilities include: - Working on batch and near-real-time data ingestion, transformation, and loading (ETL/ELT) across multiple source systems. - Demonstrating strong hands-on experience with SQL, stored procedures, and performance tuning in SQL Server environments. - Integrating legacy ETL (SSIS) processes with modern cloud-based data platforms (Databricks / Lakehouse architecture). - Understanding insurance lifecycle processes and data flows across core insurance systems. - Supporting data migration and modernization initiatives from on-premise to cloud platforms (Azure/AWS preferred). - Ensuring data quality, validation, reconciliation, and governance across all pipelines. - Monitoring and troubleshooting ETL jobs to ensure high availability, reliability, and performance. - Collaborating with business analysts, actuaries, and downstream BI/reporting teams to deliver data solutions. - Working closely with data architects and engineers to define scalable data models and standards. - Implementing logging, error handling, and auditing frameworks for data pipelines. - Participating in Agile ceremonies, code reviews, and production deployment activities. - Ensuring compliance with insurance regulatory, audit, and data security requirements. You should have experience working with P&C Insurance data domains such as Policy, Claims, Billing, Underwriting, and Loss data. Additionally, you will be expected to have a good understanding of insurance regulatory, audit, and data security requirements to ensure compliance.
More at Virtusa