Data Engineer V (Agile One)

Job : Data Engineer V (Agile One)

Location : Remote

Duration 6 Months

Responsibilities

• • Work closely with Subject Matter Experts (SMEs) to design and develop the ontology (data model) and data pipelines.

• • Design, implement, and maintain robust data pipelines and ETL processes.

• • Develop and optimize cloud-based data storage and processing solutions.

• • Collaborate with data analysts and data scientists to understand data requirements and deliver high-quality data solutions.

• • Ensure data integrity and security across all data workflows and storage solutions.

• • Monitor and troubleshoot data pipelines, addressing any issues promptly to ensure the smooth flow of data.

• • End to End data pipeline development – responsible for using Python and Pyspark to perform ETL and derive new datasets necessary for business applications.

Minimum Qualifications

• • Bachelor's Degree in Computer Science, Engineering, or related field or related experience.

• • 5+ years of experience as a data engineer.

• • Hands-on experience with building data pipelines

• • Strong SQL skills and experience working with large datasets and complex data structures.

• • Proficient in Python, and Pyspark

• • Experience in Typescript (preferred) or Javascript

• • Excellent problem-solving and analytical skills with a strong attention to detail.

Preferred Qualifications

• • Experience with Palantir Foundry application development and data visualization tools

• • Knowledge with commercial visualization tools such as Tableau or Power BI.

• • Experience in Typescript (preferred) or Javascript

• • Databases – familiarity with common relational database models and proprietary instantiations, such as SAP, Salesforce etc.

• • Git – knowledge of version control / collaboration workflows and best practices

• • Agile – familiarity with agile and iterative working methodology and rapid user feedback gathering concepts

• • UX design – knowledge of best practices and applications