Data Engineer - M1

Job Description The Data Engineer is responsible for building, maintaining, and optimizing data infrastructure and pipelines. This role requires collaboration with data scientists, analysts, and other stakeholders to ensure that data is accessible, reliable, and efficiently processed to support decision-making and achieve business objectives. Roles and Responsibilities Data Pipeline Development: Design, develop, and maintain robust data pipelines to collect, process, and store large volumes of structured and unstructured data from various sources. Data Integration: Integrate data from multiple internal and external sources, ensuring consistency, accuracy, and availability for business applications. Database Management: Design and implement scalable, efficient databases and data warehouses that support complex queries and real-time analytics. Data Quality Assurance: Monitor and ensure the quality, integrity, and security of data throughout its lifecycle by implementing data validation and error-checking processes. Performance Optimization: Optimize data processing workflows and storage solutions to enhance performance and reduce latency. Collaboration: Work closely with data scientists, analysts, and stakeholders to understand data needs and translate requirements into technical solutions. Documentation and Reporting: Maintain comprehensive documentation of data processes, systems, and standards, and provide regular updates and reports to stakeholders. Soft Skills Required Excellent verbal and written communication skills. Ability to convey technical information effectively to non-technical stakeholders. Qualifications Education: Bachelor s degree in Computer Science, Information Technology, Data Science, or a related field. A master s degree is a plus. Proven experience as a Data Engineer or in a similar role. Experience with data modeling, ETL processes, and data integration. Proficiency in SQL and database management systems such as MySQL, PostgreSQL, or Oracle. Experience with big data technologies (e.g., Hadoop, Spark) and cloud platforms (e.g., AWS, Azure, GCP). Familiarity with programming languages including Python, Java, or Scala.