Source description
About the role
Kafka + Java + Data Engineers: Extensive experience designing and developing streaming pipelines, data platform to handle high volume data processing, distributed system for both batch and realtime Strong Expertise in– Java, Kafka and Hadoop ecosystem (Spark, Hive, YARN, HDFS), Scala, Cassandra, Linux (on prem), CI-CD (Jenkins, Github), Obs (Dynatrace). Design and implement real-time streaming pipelines using Kafka Develop producers and consumers for high-throughput, fault-tolerant systems Responsible for both development and strong expertise to debug and fix any post production issues Implement Kafka-based event-driven architectures Experience with Databricks, Azure Cloud PaaS Services, Azure Data Factory, ADLS, Kubernetes, GitHub, Docker, and CI/CD workflows Experience in scripting, ETL (Pandas, Pola.rs, PySpark, Ibis), SQL, and NoSQL databases (Cassandra, MongoDB) Understanding of system architecture concepts: distributed systems, caching, replication, data modeling Leads : Role is 70% of time in Design, Architect, Coding and 30% of time in team mentoring, guidance, coordination Responsible for both development and strong expertise to debug and fix any post production issues for their developed code Interview ask : To write optimal code (in Java/Kafka/Python/Pyspark) in timeboxed period for scenario given (covering data quality, transformation) without AI tool and code to be accurate. To resolve code issue for a bug introduced in sample code Excellent written and verbal communication skills Experience working in Agile environments; familiarity with cloud platforms Experience with orchestration tools (e.g. NiFi, Griffin, Hamilton, Airflow) Functional experience of Retail Banking, capital markets, securities processing will be advantage
More at Cognizant