Database Engineer Chandigarh (India)

Job Description Key Responsibilities Develop and maintain scalable data pipelines for batch and real-time data processing Build robust data ingestion frameworks using APIs, file-based ingestion, and streaming sources Design and implement Python and PySpark-based data transformation logic for large-scale datasets Develop Python-based microservices and REST APIs for secure and effective data access and sharing Integrate data pipelines with platforms such as Databricks, Delta Lake, and Snowflake Experience with serverless data processing (e.g., AWS Lambda, Azure Functions with Python Implement and manage streaming pipelines using Kafka / Kinesis Work with structured, semi-structured, and unstructured data (Parquet, JSON, CSV, sensor/IoT data Optimize data processing jobs for performance, scalability, and cost efficiency Apply best practices for data partitioning, storage optimization, and query performance tuning Ensure data quality, validation, and monitoring across pipelines Collaborate with data architects, analysts, and downstream consumers to deliver reliable datasets Support data modeling and transformation aligned with Lakehouse principles (e.g., medallion architecture) Contribute to automation and deployment of pipelines using orchestration and CI/CD practices Required Skills & Experience 24 years of hands-on experience in Data Engineering Robust programming expertise in Python and PySpark Solid experience in SQL for data transformation and analysis Hands-on experience building data pipelines (ETL/ELT) in production environments Experience working with APIs and microservices for data ingestion and exposure Solid understanding of data processing frameworks such as Spark, AWS Glue, or Dbt Experience handling large-scale datasets across multiple formats (Parquet, JSON, CSV, etc) Good understanding of Data Lake / Lakehouse concepts and implementation patterns .