Senior Data Engineer (182891)

Senior Data Engineer

Remote

Contract

Key Responsibilities:

Data Pipeline Development: Design, implement, and maintain ETL processes to extract, transform, and load data from diverse sources into data storage systems.

Data Integration: Integrate data from various structured and unstructured sources, including databases, APIs, logs, and external data feeds.

Data Modeling: Develop and maintain data models that align with business requirements, ensuring data is stored efficiently and accurately.

Data Quality Assurance: Implement data quality checks and validation procedures to ensure data accuracy, consistency, and completeness.

Data Warehousing: Build and maintain data warehouses or data lakes, optimizing them for query performance and scalability.

Documentation: Create and maintain documentation for data pipelines, data models, and infrastructure configurations.

Collaboration: Collaborate with analysts, and other stakeholders to understand data requirements and provide data support for analytics and reporting.

Monitoring and Troubleshooting: Set up monitoring systems to proactively identify issues and troubleshoot data pipeline failures or performance bottlenecks.

Qualifications

• Bachelor's or higher degree in computer science engineering, information technology, or a related field.

• Strong programming skills Python and associated libraries: Pandas, SQL Alchemy, Flask, Fast API etc.

• Proficiency in SQL and experience with relational and NoSQL databases.

• Knowledge of data warehousing technologies like Amazon Redshift, Google BigQuery, Snowflake, or similar is a Plus

• Experience with data pipeline orchestration tools like Apache Airflow, Luigi, or similar.

• Familiarity with cloud platform AWS is a plus.

• Understanding of data integration techniques, data modeling, and data architecture best practices.

• Excellent problem-solving and troubleshooting skills.

• Strong communication and collaboration skills.

• Knowledge of data security and compliance standards is a plus.

• Experience with big data technologies (e.g., Hadoop, Spark) is a bonus