Data Engineer

Data Engineer

Address:301 Lindenwood Dr Ste 330, Malvern, Pennsylvania

Remote: 3 days onsite

Hours: 8-5

Position Type: FULL-TIME

KEY RESPONSIBILITIES

Data Architecture & Pipeline Development

Design, construct, and maintain robust data pipelines using Azure Data Factory, Azure Synapse, and Azure Databricks.

Develop ETL/ELT processes to ingest, transform, and store structured and unstructured data from various sources.

Ensure data pipelines are scalable, secure, and optimized for performance.

Document architecture, database, data flow, and algorithm details.

Database Management

Manage and optimize relational databases using Azure SQL and flat files in cloud native, Azure environment.

Write complex SQL queries for data extraction, transformation, and analysis.

Implement indexing, partitioning, and performance tuning strategies.

Establish and maintain processes aligned to master data management; data quality, security, storage, and database tuning throughout the data lifecycle.

Machine Learning Integration

Collaborate with Data Analyst to operationalize ML models using Azure Machine Learning.

Support feature engineering, model versioning, and deployment pipelines.

Utilize Azure ML tools like AutoML, HyperDrive, and Model Registry for experimentation and tracking.

Cloud Infrastructure & Automation

Leverage Azure Functions, Logic Apps, and Event Hubs for event-driven data processing.

Automate data workflows and monitoring using CI/CD pipelines and scripting (PowerShell, PowerAutomate).

Ensure compliance with security and governance policies across all data assets.

Collaboration & Communication

Work closely with cross-functional teams including product, engineering, testing, and information security.

Translate business requirements into technical specifications and data solutions.

Document data flows, architecture, and operational procedures.

Skills AND Experience

• Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.

• 5+ years of experience in data engineering roles.

• Proficiency in SQL and experience with Azure data services, (Synapse, Azure ML).

• Familiarity with machine learning concepts and model deployment practices.

• Strong problem-solving skills and attention to detail.

• Excellent communication and collaboration abilities.

Preferred Qualifications

• Experience with Azure DevOps, GitHub Actions, or other CI/CD tools.

• Working knowledge of .Net.

• Knowledge of data governance frameworks and SOC 2 compliance.

• Exposure to financial services, credit union, banking data environments.