Source description
About the role
Engagement Type Contract / Consulting Assignment Experience 15+ Years Overall Experience | 8+ Years in Elasticsearch/OpenSearch Ecosystems Location Remote / Hybrid (Client Engagement Based) Project Overview We are seeking a highly experienced OpenSearch Migration Architect to lead the migration of a large-scale enterprise observability and analytics platform from Elasticsearch (ELK Stack) to OpenSearch. The environment consists of approximately 200 TB of indexed data, multiple ingestion pipelines, dashboards, security controls, and operational integrations supporting mission-critical business functions. This role requires an architect-level consultant who has successfully delivered migrations in 100 TB+ enterprise environments and can provide strategic leadership across architecture, planning, execution, validation, and operational transition. The selected consultant will own the migration lifecycle from assessment through production cutover while ensuring minimal downtime, zero data loss, and performance parity or improvement. Key Responsibilities 1. Current State Assessment & Discovery Perform a comprehensive assessment of the existing ELK ecosystem, including: * Elasticsearch cluster architecture * Logstash ingestion pipelines * Kibana dashboards and visualizations * Index templates and mappings * Security configurations and RBAC models * Snapshot and backup mechanisms * Data retention and archival policies * Monitoring and operational processes Responsibilities include: * Migration readiness assessment * Technical debt identification * Compatibility analysis between Elasticsearch and OpenSearch * Risk assessment and mitigation planning * Dependency mapping across upstream and downstream systems 2. Migration Strategy & Architecture Define and own the enterprise migration strategy. Key activities include: * Designing target-state OpenSearch architecture * Evaluating migration approaches: * In-place migration * Side-by-side migration * Re-index migration * Snapshot and restore migration * Hybrid migration models * Developing migration waves and execution phases * Designing cutover and rollback procedures * Creating migration runbooks and operational playbooks * Defining downtime minimization strategies 3. OpenSearch Platform Architecture Design a highly scalable and resilient OpenSearch platform. Responsibilities include: Cluster Architecture * Cluster topology design * Capacity planning * Node sizing and allocation * Storage architecture * Hot/Warm/Cold tiering strategies Data Management * Shard sizing strategy * Index lifecycle management (ILM) * Data retention policies * Snapshot lifecycle management Reliability & Resilience * High availability architecture * Disaster recovery design * Cross-region replication strategy * Backup and recovery procedures Security * Authentication and authorization design * Role-based access control (RBAC) * Encryption standards * Audit logging * Security best practices 4. Migration Execution Leadership Lead the migration execution for a large-scale 200+ TB environment. Scope includes: Data Migration * Indices * Templates * Mappings * Aliases * Data streams * Snapshots Platform Migration * Dashboards * Visualizations * Alerting configurations * Watches and notifications * Security policies * User roles and permissions Automation * Migration tooling * Validation automation * Operational automation * Infrastructure automation Provide hands-on leadership while coordinating infrastructure, application, and operations teams. 5. Validation, Performance Tuning & Optimization Establish validation frameworks and performance benchmarks. Activities include: Validation * Data integrity verification * Record count reconciliation * Index validation * Security validation * Dashboard and visualization validation Performance Engineering * Query performance benchmarking * Indexing throughput optimization * Cluster health tuning * JVM optimization * Resource utilization optimization Define acceptance criteria and production readiness metrics. 6. Documentation & Knowledge Transfer Develop comprehensive migration and operational documentation. Deliverables include: * Architecture diagrams * Platform design documentation * Migration runbooks * Operational procedures * Disaster recovery guides * Support handover documentation Conduct structured knowledge transfer workshops for engineering and operations teams. Mandatory Qualifications Experience * 15+ years of overall IT experience * 8+ years of Elasticsearch experience * 3+ years of OpenSearch experience * Proven experience leading enterprise-scale ELK migrations * Demonstrated delivery of at Enga