Source description
About the role
Educational Qualification Bachelor's degree in Computer Science, or a related field, or equivalent experience Key Responsibilities Design, develop, test, and maintain scalable ETL data pipelines using Python Architect enterprise solutions with technologies including Kafka, GCP services, GKE, Load Balancers, APIGEE, DBT, LLMs, data redaction, and DLP solutions Build and manage data workflows on Google Cloud Platform using Dataflow, Cloud Functions, BigQuery, Cloud Composer, Cloud Storage, IAM, and Cloud Run Implement data ingestion, transformation, and cleansing to ensure high-quality data delivery Develop and enforce data quality checks, validation rules, and monitoring processes Collaborate with data scientists, analysts, and engineering teams to deliver efficient data solutions Manage version control using GitHub and participate in CI/CD pipeline deployments Write complex SQL queries for data extraction and validation from relational databases (SQL Server, Oracle, PostgreSQL) Document pipeline designs, data flow diagrams, and operational support procedures Required Skills & Competencies Strong hands-on experience in Python for backend or data engineering projects Expertise in GCP services: Dataflow, BigQuery, Cloud Functions, Cloud Composer, Cloud Storage, Cloud Run, IAM Experience in data pipeline architecture, data integration, and transformations Proficiency in Apache Spark, Kafka, Redis/Bigtable, FastAPI, and Airflow Strong SQL skills with at least one enterprise database (SQL Server, Oracle, PostgreSQL) Experience in CI/CD practices and version control (GitHub) Good to Have / Optional Skills Experience with Snowflake cloud data platform Hands-on knowledge of Databricks for big data processing and analytics Familiarity with Azure Data Factory (ADF) and Azure data engineering tools
More at People Prime Worldwide