Data Engineer

Python Data Engineer About Datachamps Datachamps is a premium Business Intelligence (BI) company that brings Intelligent Dashboards for CXOs and Digital Transformation solutions for the Finance & Strategy teams of enterprises. We count large, global enterprises like Bridgestone Tyres, Mitsubishi, Sandvik, Eaton Corporation, Nivea, responsAbility Investments amongst our esteemed clientele. At the same time we also serve home-grown brands such as Society Tea, Imagicaa, Chitale Express, Garware Fulflex, SIDBI, Kotak Bank etc. We have offices in Mumbai, Pune, and Nashik. Job Description As a talented and motivated Data Engineer with strong Python development skills for our Digital Transformation & Intelligent Dashboards teams, you will play a crucial role in designing, developing, and maintaining data connections, pipelines and infrastructure to support our clients' data-driven initiatives. The role will be: Design, build, and maintain modular, scalable & reusable data pipelines (inbound data or outbound data) and ETL processes, based on business requirements and from various sources such as databases, files, APIs, streaming platforms, sensors, or IoT devices. Developing and maintaining reusable inbound connectors to ERPs and other accounting software such as Zoho, Quickbooks, Xero shall be a key responsibility. For API (REST / SOAP) based connectors, understanding API documentation, efficiently fetch data from APIs and transform for downstream usability. Leveraging Python Libraries and frameworks such as Pandas, Numpy, Beautiful Soup, HTTP requests, Flask, or FastAPI to interact with APIs efficiently. ¢ ¢ ¢ Work with outbound integrations including those into WhatsApp, Outlook etc. Awareness of data connections via pipelines with data visualization tools e.g., Power BI, Tableau, Looker to build, maintain and debug critical data flows and outputs. Model data flows in and out of the data warehouse using SQLServer / MySQL / PostgreSQL / BigQuery and other data into clean, tested, and reusable datasets used for analytical and other downstream purposes like designing report views. Effectively define ETL activities / data transformations, such as cleaning, normalization, aggregation, building data views while ensuring data accuracy, integrity and version management. Hands-on experience with joins (merge, union), appends, overwrite, change data capture (CDC), imputation, sorting, filtering, grouping, pivoting-unpivoting and other data operations is expected. ¢ ¢ ¢ ¢ ¢ ¢ ¢ Apart from Python, experience of working with JavaScript / Google Script, C# will be advantageous. Experience of working with Frappe framework shall be useful. Work with cloud platforms (e.g., AWS, Azure, Google Cloud) to deploy and manage data infrastructure components such as databases, storage, and compute resources. Implementing robust error handling and logging mechanisms to ensure the reliability and traceability of the pipeline. Ensure all managed data flows & data assets are clearly and accurately documented. Assist in solution architectural planning and high-level diagram preparation. Performance tuning and efficiency of the pipeline, such as batching requests or implementing caching mechanisms. Data unit testing of the pipelines built, including thorough testing of the API connectors to ensure they function correctly and documenting their usage for other team members. Security Considerations: Ensuring secure transmission of data by implementing authentication, authorization mechanisms as per API specifications, in the context of the clients data security policy and best practices Continuous integration & continuous deployment (CI & CD) ensuring up-to-date and reliable data. www.datachamps.ai ¢ ¢ ¢ Ability to leverage tools like Power Automate, Zappier and open-source automation testing frameworks like Selenium and workflow management tools like Celery or Apache Airflow Work collaboratively with all stakeholders namely business analysts and data scientists to align client requirements & data assets. Apart from client projects, the role may require you to work on Datachamps own requirements such as building and enhancing internal apps, building apps for workflows our standard deliverables, client samples. Applicants having relevant experience on any one or more of the above responsibilities will be an advantage and will be duly considered. Qualification & Soft Skills Requirements Minimum work experience of 3 years in development and deployment of data engineering solutions from scratch using the following tools, a. Python (libraries like Pandas, NumPy, HTTP Requests, PyODBC, PyOD, PyRFC etc.) b. SQL (PostgreSQL, MySQL, etc.) c. Cloud tools from Azure, AWS or Google Cloud, Databricks Graduation from a top-tier institute in Computer Science, Engineering, Information Management, Data Science, Mathematics etc. Certifications for Azure and/or AWS and/or GCP. E.g. Microsoft Azure Data Engineer Associate (Course DP- 203T00-A) or equivalent for AWS, GCP are must-have. Solid understanding of data structures, algorithms, and software engineering principles. Enthusiastic about technology, Self-motivated and proactive, problem solving / solution-oriented attitude. Interested in working for a dynamic and high-growth venture where your analysis drives important decisions for the clients. Strong analytical skills with ability to collect, organize, analyze, and disseminate information with attention to detail and accuracy. Designing an efficient solution for a given business problem. Strong language & communication skills with the ability to discuss any issues with a wide variety of individuals and groups. A well-organized team player with the ability to perform various tasks, act individually, and think creatively. Location & Application Process ¢ ¢ Pune (Baner) Please submit your applications to Avani Tanna: avani@datachamps.ai www.datachamps.ai