Padmi

Associate III - Data Engineering - Python with AWS

BangalorePosted 2 months ago
Software engineeringMid-levelFull Time; Regular
Apply at UST

Opens the source posting on shine.com

Source description

About the role

View original

Role Description Key Responsibilities OT Integration: Design and implement integrations between OT systems (Lab instruments) and IT/cloud platforms Data Pipeline Development: Build and maintain robust ETL/ELT data pipelines in Python to ingest, transform, validate, and deliver data from OT sources to downstream systems and data stores. File Share & Event-Driven Workflows: Develop and manage file-based integration patterns using Windows file shares (SMB/UNC), monitoring file drop events and triggering automated processing workflows. Cloud Data Engineering (AWS): Architect and maintain data flows on AWS using services such as S3, Lambda, Glue, Step Functions, SQS/SNS, and EventBridge to process and route lab and OT data at scale. JavaScript / Node.js Development: Develop lightweight integration services, APIs, and event handlers using JavaScript/Node.js for connector logic, web service interfaces, and real-time data routing. Orchestration & Workflow Automation: Design workflow orchestrations to automate multi-step data processes, handle retries, error states, and notifications across connected systems. Monitoring & Reliability: Implement monitoring, ing, and logging strategies to ensure integration reliability and rapid incident response across OT/IT boundaries. Windows Infrastructure: Manage integration agents and services deployed on Windows Server, including scheduled tasks, service management, network share configuration, and security permissions. Collaboration & Documentation: Work closely with lab operations, IT infrastructure, and platform teams to gather requirements, document integration designs, and maintain technical runbooks. Required Qualifications 3-5 years of experience in data engineering, systems integration, or OT/IT convergence roles. Proficiency in Python for data pipeline development, scripting, and automation (pandas, requests, boto3, etc.). Solid JavaScript/Node.js skills for building integration services, REST APIs, and event-driven handlers. Hands-on AWS experience: S3, Lambda, Glue, Step Functions, SQS/SNS, IAM, CloudWatch. Experience with Windows Server environments: file shares (SMB/UNC), scheduled tasks, service management, Active Directory basics. Strong understanding of ETL/ELT patterns, data transformation, and structured/semi-structured data formats (JSON, XML, CSV, Parquet). Experience with REST API design and consumption; familiarity with message queuing and event-driven architectures. Ability to work independently, troubleshoot complex integration issues, and communicate findings clearly. Bachelors degree in computer science, Software Engineering, Data Engineering, or a related technical field. Skills JavaScript, Python, AWS Glue, AWS Lambda Role Description Key Responsibilities OT Integration: Design and implement integrations between OT systems (Lab instruments) and IT/cloud platforms Data Pipeline Development: Build and maintain robust ETL/ELT data pipelines in Python to ingest, transform, validate, and deliver data from OT sources to downstream systems and data stores. File Share & Event-Driven Workflows: Develop and manage file-based integration patterns using Windows file shares (SMB/UNC), monitoring file drop events and triggering automated processing workflows. Cloud Data Engineering (AWS): Architect and maintain data flows on AWS using services such as S3, Lambda, Glue, Step Functions, SQS/SNS, and EventBridge to process and route lab and OT data at scale. JavaScript / Node.js Development: Develop lightweight integration services, APIs, and event handlers using JavaScript/Node.js for connector logic, web service interfaces, and real-time data routing. Orchestration & Workflow Automation: Design workflow orchestrations to automate multi-step data processes, handle retries, error states, and notifications across connected systems. Monitoring & Reliability: Implement monitoring, ing, and logging strategies to ensure integration reliability and rapid incident response across OT/IT boundaries. Windows Infrastructure: Manage integration agents and services deployed on Windows Server, including scheduled tasks, service management, network share configuration, and security permissions. Collaboration & Documentation: Work closely with lab operations, IT infrastructure, and platform teams to gather requirements, document integration designs, and maintain technical runbooks. Required Qualifications 3-5 years of experience in data engineering, systems integration, or OT/IT convergence roles. Proficiency in Python for data pipeline development, scripting, and automation (pandas, requests, boto3, etc.). Solid JavaScript/Node.js skills for building integration services, REST APIs, and event-driven handlers. Hands-on AWS experience: S3, Lambda, Glue, Step Functions, SQS/SNS, IAM, CloudWatch. Experience with Windows Server environments: file shares (SMB/UNC), scheduled tasks, service management, Active Directory basics. Strong understanding of ETL/ELT patterns, dat

One address, no account. We’ll tell you when matching roles go live.

More at UST

Related open roles

View all roles