Data Engineer

Key Responsibilities Design, develop, and maintain scalable ETL/ELT data pipelines using PySpark and Databricks . Develop batch and streaming data processing solutions for structured and unstructured datasets. Build and optimize data workflows on cloud platforms. Implement Delta Lake architecture for efficient data storage and processing. Integrate data from multiple sources including databases, APIs, cloud storage, and third-party systems. Optimize Spark jobs for performance, scalability, and cost efficiency. Develop reusable data engineering frameworks and utilities. Work closely with Data Architects, Data Scientists, Business Analysts, and application teams. Ensure data quality, governance, and security standards are maintained. Troubleshoot production issues and provide timely resolutions. Participate in code reviews and follow CI/CD best practices. Create technical documentation for data pipelines and processes.