Senior Hadoop Developer

Job description Work proactively to understand domain (Market Research) and technical trends that will impact/drive the development and design Work collaboratively/individually to write high quality, testable code that meets the Definition of Done Strive to meet team commitments Strive to eliminate technical debt Adopt a Test-First approach to software development, where possible in which the test is developed before the code Participate in design and code reviews with other engineers Use best software engineering practices to evolve the system design Perform unit/module testing of software Continuously evaluate software performance, identify performance degradation and opportunities to gain performance improvements valued by customers Responsible for assigned integration, testing and deployment tasks Take ownership of problems and continuously strive to become a better problem solver May establish and maintain software configuration control and software development environments including program source control to ensure reliable backups and source code integrity Write and maintain documentation to describe program development, logic, coding, testing, changes, and corrections Complex problem solving skills Strong analytical skills and the ability to understand complexities and how components connect and relate to each other Strong communication skills Strong teamwork skills for cross group cooperation Strong technical knowledge across the industry Leadership skills and ability Coaching and mentoring skills Technical capabilities: Minimum 6 year hands on below skill sets Good hands and understanding in hive, HDFS file system and HDFS commands Good troubleshooting knowledge using Spark/Scala or HDFS file system Very good knowledge Hadoop architecture end to end Very good hands on in writing Spark SQL Very good hands on in writing Scala code for a given requirementVery good hands on writing Python scripts Very good hands on writing SQL/HQL scripts Very good hands on writing SQL/HQL scripts Good hands on Python libraries like pandas is preferred