Source description
About the role
Role & responsibilities Design, develop, and maintain scalable data pipelines using Databricks and PySpark . Work with large-scale structured and unstructured data from various sources. Optimize data workflows for performance and reliability. Collaborate with data scientists, analysts, and business stakeholders to understand data requirements. Implement data quality checks, monitoring, and alerting mechanisms. Ensure data security and compliance with governance policies. Participate in code reviews and contribute to best practices in data engineering Preferred candidate profile 8-12 years of experience in data engineering or related roles. Strong hands-on experience with Databricks and Apache Spark (PySpark) . Proficiency in Python and SQL. Experience with cloud platforms such as Azure , AWS , or GCP (preferably Azure). Solid understanding of ETL/ELT processes , data warehousing , and data modeling . Familiarity with Delta Lake and Parquet formats. Experience with CI/CD pipelines and version control (e.g., Git). Excellent problem-solving and communication skills. Contact: Email: swarna.kanniappan@propelis.com Notice Period: Immediate to 1 Month Location: Remote Opportunity