ElasticSearch Engineer

As an Elasticsearch Engineer at our company, you will be responsible for designing and scaling a distributed data platform to handle high-volume OpenRTB/ad data ingestion and retention. You will collaborate closely with our System Architect to optimize and maintain Elasticsearch clusters as a crucial component of our data infrastructure. Key Responsibilities: - Design and maintain Elasticsearch indices tailored for high-volume data - Implement and manage ILM policies to automate data retention - Enhance query performance and ensure cluster health - Work with the architecture team to develop scalable data storage solutions - Develop data retention strategies for specific device/entity-level data exceeding standard retention windows - Monitor and troubleshoot cluster performance, shard allocation, and storage Qualifications Required: - Minimum 2 years of hands-on experience with Elasticsearch - Strong grasp of Elasticsearch index design, mappings, and settings - Familiarity with index lifecycle management (ILM) policies - Proficient in creating complex queries including aggregations, filters, and nested queries - Experience with hot-warm-cold data tier architecture - Knowledge of Elasticsearch cluster management (nodes, shards, replicas) - Hands-on experience with Kibana for monitoring and visualization - Familiarity with data ingestion pipelines such as Logstash, Kafka, or custom pipelines If you have experience with OpenRTB/AdTech data, Apache Kafka for event-driven architectures, Python or Java for scripting/integration with Elasticsearch, Bloom filters, Parquet/object storage (S3/GCS) for data archival, or understanding of microservices and distributed system design, it would be considered a plus. As an Elasticsearch Engineer at our company, you will be responsible for designing and scaling a distributed data platform to handle high-volume OpenRTB/ad data ingestion and retention. You will collaborate closely with our System Architect to optimize and maintain Elasticsearch clusters as a crucial component of our data infrastructure. Key Responsibilities: - Design and maintain Elasticsearch indices tailored for high-volume data - Implement and manage ILM policies to automate data retention - Enhance query performance and ensure cluster health - Work with the architecture team to develop scalable data storage solutions - Develop data retention strategies for specific device/entity-level data exceeding standard retention windows - Monitor and troubleshoot cluster performance, shard allocation, and storage Qualifications Required: - Minimum 2 years of hands-on experience with Elasticsearch - Strong grasp of Elasticsearch index design, mappings, and settings - Familiarity with index lifecycle management (ILM) policies - Proficient in creating complex queries including aggregations, filters, and nested queries - Experience with hot-warm-cold data tier architecture - Knowledge of Elasticsearch cluster management (nodes, shards, replicas) - Hands-on experience with Kibana for monitoring and visualization - Familiarity with data ingestion pipelines such as Logstash, Kafka, or custom pipelines If you have experience with OpenRTB/AdTech data, Apache Kafka for event-driven architectures, Python or Java for scripting/integration with Elasticsearch, Bloom filters, Parquet/object storage (S3/GCS) for data archival, or understanding of microservices and distributed system design, it would be considered a plus.