Source description
About the role
Role: Sr. Data Engineer Location: Remote for now (NY, NJ, FL, IL, CA) Contract: 6 months extendable (they don’t have any tenure so it can be 3-5 years) Interview process: zoom with our account manager than zoom with client, Must have: Big data, spark, scala, AWS, and python (must have all that)Sr. Data Engineer must have a broad and deep data skillset as well as strong analytical capabilities . In addition to being a hands on individual contributor, the ideal candidate is a productive team player and a mentor to Junior Data Engineers. Additionally, we are looking for strong technical experts.Actively participate in team technical discussions in all things dataIdentify and address issues with data sets from multiple vendorsIdentify and address code and data quality issuesActively participate in code reviews and grooming sessionsActively participate in technology architecture discussions for product developmentTranslate business requirements into strategyAdvocate for software best practices within your team as well as across engineeringBe ultra-responsive and capable of making instant decisions, always kicking the ball forwardWork on unique and interesting data challenges around architecting, building and managing pipelines that securely process hundreds of terabytes of dataWork closely with analysts and statisticians to ensure the validity of our processesOur engineers are expected to wear a number of hats and have the opportunity to touch all parts of the stack. Our stack includes Apache Spark, Scala, Redshift and an ever-growing list of many other cool technologies.RequirementsSkillful user of Apache SparkExperience wrangling terabytes of big, complicated, imperfect dataExperience with AWS products (Redshift, EMR, S3, IAM, RDS, etc)You have a deep understanding of scalable systems and you have large-scale engineering experience in an Agile development environmentBachelor's degree in Computer Science or a related field (or 4 additional years of relevant work experience)A strong understanding of data structures, algorithms, and effective software designSignificant development experience with a major modern language (e.g. Java, Scala, Python, Ruby, C/C++, etc.)Significant experience working with structured and unstructured data at scale and comfort with a variety of different stores (key-value, document, columnar, etc.) as well as traditional RDBMSes and data warehousesExperience with or interest in AWS Glue, Redshift Spectrum and any other tools that enable data querying at scaleExperience writing unit, functional and integration testsComfort with version control systems (e.g. Git, SVN)Excellent verbal and written communication skills; must work well in an agile, collaborative team environment Thanks & Regards Syam B, 630-923-6952 syam@thoughtwavesoft.com
More at Thoughtwave Software and Solutions