Data Engineering Manager

I need a Data Engineering Lead or Manager. Still 80% hands-on

POSITION DESCRIPTION :

The Data Engineer ensures that the infrastructure handles diverse sources of data seamlessly, understands data analysis, end-user requirements analysis, and business requirements analysis to develop a clear understanding of the business needs and to incorporate these needs into technical solutions. S/he will design, develop, and maintain data pipelines, integrate with new technologies, and oversee the development of new data managed processes that support key customer facing functions. This role will ensure data pipelines are scalable, repeatable, and secure, and can serve multiple business units within the organization. You will facilitate data collection and movement between a variety of different sources, analyze data from the various sources and produce meaningful reports and/or Presentations. Delivery from this team will be not only on delivering business value but also about reliability, agility, and scalability.

PRIMARY RESPONSIBILITIES :

Implement data integration projects, including writing Python and Powershell scripts for data integration and analysis.

Build and maintain optimized and highly available data pipelines that facilitate deeper analysis and reporting by the Data and Analytics department.

Analyze, transform, and prepare data for large scale migration projects.

Identify and work on master data consolidation and management as part of data migration.

Develop automated workflows to perform data migrations using both on-premises and cloud-based technologies.

Design, implement, test, deploy, and maintain stable, secure, and scalable data engineering solutions and pipelines to support numerous business needs.

Produce scalable, replicable code and solutions that help automate repetitive data management tasks.

Perform one-off data manipulation and analysis

Implement and monitor best in class security measures with an eye towards the evolving threat landscape.

Design robust integration solution error handling, monitoring, and reporting ensuring early detection and resolution of issues.

Perform in-depth data analysis to validate system behavior, identify problems, and support business users.

Collaborate with application and IT system administration teams as required and make recommendations on SQL Server sizing and configuration, database design and features, or T-SQL query tuning/best practices.

REQUIRED SKILLS/EXPERIENCE :

Bachelor's degree in computer science, computer engineering, or similar technical discipline or equivalent work experience.

Candidates without an Intermediate+ proficiency with Python, especially for data manipulation and analysis, and ability to build, maintain and deploy sequences of automated processes will not be considered

Experience and high proficiency in

Microsoft SQL Server

T-SQL query language

Python Programming Language a must

SQL Server Management Studio (SSMS)

JSON and XML

Azure Data Bricks / Azure Data Factory is a plus

PowerShell is a plus

2+ years of experience on MS SQL Server, writing complex stored procedures and functions and working on ETL pipelines.

Excellent communication skills

Strong logical, analytical, and problem-solving skills

Can work in a fast paced, ever-changing environment with a growth mindset

Proven experience working hands-on with big data systems.

Experience in master data management for large scale implementations

Experience in working on data migration projects with large data sets

Release management (Git, Visual Studio, etc.) is a plus

Advanced level knowledge of relational databases, database structures and modeling, systems design, data management, data warehouse a plus.

Experience with Microsoft Azure a plus.