Source description
About the role
You are a highly skilled Senior Real-time Data Engineer who will be responsible for designing, developing, and scaling a high-performance analytics platform capable of processing millions of real-time events daily. Your expertise in Apache Pinot and experience in building scalable real-time data pipelines and analytics systems will be key to your success in this role. Key Responsibilities: - Design and maintain a low-latency analytics architecture capable of supporting high-concurrency workloads. - Build and optimize real-time ingestion pipelines from PostgreSQL and streaming platforms like Kafka/Kinesis into Apache Pinot. - Develop and maintain semantic data models using Cube.js to ensure consistency across dashboards, reports, and APIs. - Optimize Apache Pinot tables, indexing strategies, and query performance for fast dashboard response times. - Collaborate with frontend and backend teams to expose scalable REST/GraphQL APIs for analytics and reporting. - Implement secure multi-tenant data access and governance within the semantic layer. - Translate complex business requirements into scalable analytical data models and schemas. Required Skills: - Strong experience with Apache Pinot in production environments. - Hands-on experience with Cube.js (Cube) for semantic modeling and pre-aggregations. - Expertise in PostgreSQL with strong analytical query optimization knowledge. - Experience with real-time streaming technologies such as Kafka, Debezium, Flink, or Kinesis. - Proficiency in Node.js or Python. - Strong command of SQL and data modeling concepts. - Experience designing scalable real-time analytics platforms. Preferred Skills: - Experience with OLAP technologies such as ClickHouse or StarRocks. - Knowledge of Infrastructure as Code tools like Terraform and Kubernetes. - Experience in CRM, Sales Analytics, or SaaS analytics ecosystems. - Exposure to multi-tenant data architecture and security implementation. - Open-source contributions related to Apache Pinot or Cube ecosystem are a plus. You should possess strong problem-solving and performance optimization skills, the ability to work in a fast-paced, high-scale environment, excellent collaboration and communication skills, and a passion for building scalable, real-time data products. Interested candidates who can join within 0-10 days are highly preferred. You are a highly skilled Senior Real-time Data Engineer who will be responsible for designing, developing, and scaling a high-performance analytics platform capable of processing millions of real-time events daily. Your expertise in Apache Pinot and experience in building scalable real-time data pipelines and analytics systems will be key to your success in this role. Key Responsibilities: - Design and maintain a low-latency analytics architecture capable of supporting high-concurrency workloads. - Build and optimize real-time ingestion pipelines from PostgreSQL and streaming platforms like Kafka/Kinesis into Apache Pinot. - Develop and maintain semantic data models using Cube.js to ensure consistency across dashboards, reports, and APIs. - Optimize Apache Pinot tables, indexing strategies, and query performance for fast dashboard response times. - Collaborate with frontend and backend teams to expose scalable REST/GraphQL APIs for analytics and reporting. - Implement secure multi-tenant data access and governance within the semantic layer. - Translate complex business requirements into scalable analytical data models and schemas. Required Skills: - Strong experience with Apache Pinot in production environments. - Hands-on experience with Cube.js (Cube) for semantic modeling and pre-aggregations. - Expertise in PostgreSQL with strong analytical query optimization knowledge. - Experience with real-time streaming technologies such as Kafka, Debezium, Flink, or Kinesis. - Proficiency in Node.js or Python. - Strong command of SQL and data modeling concepts. - Experience designing scalable real-time analytics platforms. Preferred Skills: - Experience with OLAP technologies such as ClickHouse or StarRocks. - Knowledge of Infrastructure as Code tools like Terraform and Kubernetes. - Experience in CRM, Sales Analytics, or SaaS analytics ecosystems. - Exposure to multi-tenant data architecture and security implementation. - Open-source contributions related to Apache Pinot or Cube ecosystem are a plus. You should possess strong problem-solving and performance optimization skills, the ability to work in a fast-paced, high-scale environment, excellent collaboration and communication skills, and a passion for building scalable, real-time data products. Interested candidates who can join within 0-10 days are highly preferred.