Data Engineer

Title: Data Engineer Location: Pune Exp: 3-5 Years KEY RESPONSIBILITIES Pipeline development: Design, build, and maintain end-to-end data pipelines for ingestion, transformation, and loading using Python and Databricks. SQL and data modelling: Write efficient SQL queries and scripts for data extraction, transformation, and validation across relational and cloud data stores. Azure data services: Build and manage pipelines leveraging Azure services including Azure Data Factory, Azure Blob Storage, and CosmosDB where applicable. Ad hoc data engineering: Respond to ad hoc data requests and pipeline changes raised by the Project Lead within agreed hour estimates. Data quality and validation: Implement checks, reconciliation steps, and logging to ensure data integrity and traceability across all pipelines. Orchestration and scheduling: Configure and manage job orchestration and scheduling on Databricks and Azure-native services. Documentation: Maintain clear technical documentation pipeline design notes, data dictionaries, runbooks to support knowledge transfer and continuity. Collaboration: Work closely with the Data Scientist, API Developer, and Cloud Engineer within the pod; coordinate with client stakeholders via the Project Lead only. REQUIRED TECHNICAL SKILL-SET Skill Area Tools / Platforms Programming Python (strong) data wrangling, scripting, automation SQL SQL (strong) query writing, optimisation, data validation Big Data / Lakehouse Databricks (hands-on) notebooks, Delta Lake, jobs Cloud Azure Azure Data Factory, Azure Blob Storage, CosmosDB, Azure Functions Data Formats JSON, CSV, Parquet, XML; REST API consumption Version Control Git, GitHub DevOps Tooling JIRA, Confluence (task and documentation management) PREFERRED SKILLS / NICE TO HAVE Experience with Azure Synapse Analytics or Azure Databricks Unity Catalog. Exposure to streaming data pipelines (Azure Event Hubs, Kafka). Familiarity with JFrog Artifactory for artefact management. Prior experience in a managed-services or POD-based delivery model. Domain exposure to manufacturing, automotive, or industrial data environments. ", Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.