Senior Data Engineer

Job Title: Sr. Data Engineer Location: Any Job Type: Full-time About Us We are a specialized IT and AI software development company dedicated to transforming the educational landscape. We partner with higher education institutions and EdTech organizations in the USA to build cutting-edge, scalable AI and IT solutions. We pride ourselves on our Agile mindset, technical excellence, and commitment to delivering products that enhance educational outcomes and operational efficiency Role Summary We are looking for a highly technical and execution-focuse d Sr. Data Engine er to build, optimize, and maintain the robust data pipelines that fuel our AI solutions. Working closely with the Data Warehouse Architect, Data Scientists, and Prompt Engineers, you will be the hands-on expert responsible for turning architectural blueprints into reality. You will extract and transform complex educational data from siloed systems, ensuring it is clean, secure, and ready to power predictive analytics, large language models (LLMs), and enterprise-scale dashboards for our US higher education clients. Key Responsibilities Data Pipeline Development (ETL/ELT) Build, test, and maintain highly scalable data pipelines using modern ETL/ELT tools and frameworks (e.g., dbt, Apache Airflow, Spark. Write clean, efficient, and well-documented code (primarily Python and SQL) to extract data from various sources, transform it into usable formats, and load it into cloud data warehouses (e.g., Snowflake, BigQuery, AWS Redshift. 2. EdTech System Integration Connect and pull data via APIs from a diverse ecosystem of US higher education platforms, including Student Information Systems (SIS), Learning Management Systems (LMS like Canvas or Blackboard), and third-party EdTech tools. Handle complex, high-volume batch processing as well as near-real-time data streaming to ensure data is always up-to-date. 3. AI & Analytics Enablement Transform raw educational data into structured formats optimized for machine learning algorithms and AI model consumption. Collaborate with AI engineers to ensure data flows seamlessly into applications such as student success predictors, educational chatbots, and operational analytics tools. Optimize slow-running queries and troubleshoot database performance bottlenecks. 4. Data Quality, Security, & Governance Implement robust data validation, error handling, and automated alerting systems to guarantee high data fidelity and accuracy. Ensure all data pipelines strictly comply with US data privacy regulations, specifical ly FE RPA, by implementing data masking, encryption, and secure access controls. 5. Agile Engineering & Mentorship Actively participate in Agile ceremonies (Sprint Planning, Stand-ups, Retrospectives) and accurately estimate the effort required for data engineering tasks. Mentor junior data engineers, conduct code reviews, and advocate for engineering best practices across the development team. Required Qualifications Experie nce: 5+ years of hands-on experience in data engineering, with a proven track record of building production-grade data pipelines. . Programm ing: Expert-level proficiency in Py thon and Advanced SQL . Cloud & Data Platfo rms: Deep experience with at least one major cloud provider (AWS, GCP, or Azure) and modern cloud data warehouses (e.g., Snowflake, Databricks, Redshift) Data To ols: Strong experience with workflow orchestration (e.g., Apache Airflow, Prefect) and data transformation tools (e.g., dbt) Methodol ogy: Solid experience working in an Agile/Scrum environment, utilizing tools like Jira and Git for version control. Preferred Qualifications Industry Knowl edge: Prior experience working with Higher Education datasets, EdTech platforms, or standard educational data models (e.g., Ed- Fi) AI/ML Familia rity: Experience building pipelines specifically for machine learning models or working alongside Data Science teams . Certificat ions: Cloud data engineering certifications (e.g., AWS Certified Data Engineer, GCP Professional Data Engineer.