ETL Engineer - Data Pipeline

Job Responsibilities : - Developing new data pipelines and ETL jobs for processing millions of records and it should be scalable with growth. Pipelines should be optimised to handle both real time data, batch update data and historical data. Establish scalable, efficient, automated processes for complex, large scale data analysis. Write high quality code to gather and manage large data sets (both real time and batch data) from multiple sources, perform ETL and store it in a data warehouse. Manipulate and analyse complex, high-volume, high-dimensional data from varying sources using a variety of tools and data analysis techniques. Participate in data pipelines health monitoring and performance optimisations as well as quality documentation. Interact with end users/clients and translate business language into technical requirements. Acts independently to expose and resolve problems. Job Requirements :- 2+ years experience working in software development & data pipeline development for enterprise analytics. 2+ years of working with Python with exposure to various warehousing tools In-depth working with any of commercial tools like AWS Glue, Ta-lend, Informatica, Data-stage, etc. Experience with various relational databases like MySQL, MSSql, Oracle etc. is a must. Experience with analytics and reporting tools (Tableau, Power BI, SSRS, SSAS). Experience in various DevOps practices helping the client to deploy and scale the systems as per requirement. Strong verbal and written communication skills with other developers and business client. Knowledge of Logistics and/or Transportation Domain is a plus. Hands-on with traditional databases and ERP systems like Sybase and People-soft. Job Responsibilities : - Developing new data pipelines and ETL jobs for processing millions of records and it should be scalable with growth. Pipelines should be optimised to handle both real time data, batch update data and historical data. Establish scalable, efficient, automated processes for complex, large scale data analysis. Write high quality code to gather and manage large data sets (both real time and batch data) from multiple sources, perform ETL and store it in a data warehouse. Manipulate and analyse complex, high-volume, high-dimensional data from varying sources using a variety of tools and data analysis techniques. Participate in data pipelines health monitoring and performance optimisations as well as quality documentation. Interact with end users/clients and translate business language into technical requirements. Acts independently to expose and resolve problems. Job Requirements :- 2+ years experience working in software development & data pipeline development for enterprise analytics. 2+ years of working with Python with exposure to various warehousing tools In-depth working with any of commercial tools like AWS Glue, Ta-lend, Informatica, Data-stage, etc. Experience with various relational databases like MySQL, MSSql, Oracle etc. is a must. Experience with analytics and reporting tools (Tableau, Power BI, SSRS, SSAS). Experience in various DevOps practices helping the client to deploy and scale the systems as per requirement. Strong verbal and written communication skills with other developers and business client. Knowledge of Logistics and/or Transportation Domain is a plus. Hands-on with traditional databases and ERP systems like Sybase and People-soft.