Source description
About the role
Key Responsibilities: Design, develop, and maintain scalable ETL/ELT pipelines using Python and GCP services (e.g., BigQuery, Dataflow, Pub/Sub, Cloud Functions). Optimize and manage data storage solutions such as BigQuery and Cloud Storage. Collaborate with data scientists, analysts, and stakeholders to understand data needs and deliver actionable insights. Implement data quality checks, validation routines, and monitoring to ensure data integrity and reliability. Automate data ingestion from various sources including APIs, databases, and cloud-based systems. Write clean, efficient, and reusable Python code for data transformation and processing tasks. Contribute to the architecture and design of modern data platforms and workflows in GCP. Monitor performance and troubleshoot issues across data pipelines and infrastructure. Document systems and processes for transparency and knowledge sharing. Required Skills and Qualifications: Bachelor's or Master's degree in Computer Science, Engineering, Mathematics, or related field. 3+ years of experience as a Data Engineer or similar role. Strong programming skills in Python, with experience in data manipulation libraries such as Pandas, PySpark, etc. Hands-on experience with GCP services including: BigQuery Dataflow Cloud Storage Cloud Composer / Airflow Pub/Sub Cloud Functions Experience with SQL for complex query building and data analysis. Interested candidates can send their resumes to [HIDDEN TEXT]
More at Etelligens Technologies