Sr. Data Engineer

Here is all of the updated information for the requirement. PLEASE make this a priority today as much as you can. This is an EXCLUSIVE project with Apex so this is a guaranteed win for you waiting to happen… Advantages: 100% remote, long term project, chance to work with a growing team from the ground up REQ Details: • Client: Fidelity • Location: Remote (based out of NC) • Duration: 1+ years • Visa Restrictions: None • Interview Process: Technical screening through IKM and one more with the team • Timeline: We have interview times this week and can expect to give an offer within one week. If the timeline works, please let me know if you have anyone who you would need to move through the process faster. Tech (Qualification) Notes: Looking for Hadoop, Python, Scala Official JD: • Senior Data Engineer • Our Client is looking for Senior Data Engineer with expertise in data integration and hands on experience in enterprise Hadoop based environments (specifically in AWS Elastic Map Reduce) to help design and build a new platform to ingest data from a variety of backend sources, transform, analyze, and expose views of the data to reporting and visualization tools such as Tableau. Responsibilities • The main focus will be working with architects and other developers to design develop and deploy the new platform to support reporting / analytic needs. • However, the work will include an element of data architecture and governance, for example finding ways to understand data lineage, suitable methods to catalog each attribute defined in the system, detailing its meaning. • Lastly, the role will involve an element of initial support and operational handover, especially during the first release. Role Requirements • Hands on design and development • Ability to work independently or in small teams to design and implement Hadoop based data extraction, transformation / processing for load / presentation into visualization / reporting tools • The solution will ingest data from traditional databases & data warehouses (predominantly Oracle), flat files, web services, Mongo DB sources process / transform and augment the data structuring it into views suitable for integration with visualization and reporting tools (e.g. Tableau, OBIEE) • Be highly proficient in the following languages, tool and frameworks meaningful to the components and techniques relating to Hadoop implementations, e.g.: Hadoop, HDFS, Sqoop, Spark, Presto, Oozie, Hive • Notebook platforms • Scala, Python or Java (and specifics of integration with the Hadoop libraries / components) • Amazon Web Services / EMR architecture and operation • Additionally, experience and knowledge of key data integration points as well as technologies present in the broader Client environment is helpful: • Oracle, SQL, MongoDB • RHEL Linux, basic shell scripting, operating AWS instances and environments • ETL and data warehouse design • Public / hybrid Cloud platforms and practices (ideally AWS) • An understanding of client reporting tools (Tableau, Oracle OBIEE) • DevOps and deployment models • Good communication and collaboration skills • A self-starter who can lead and take ownership • Experience in agile environments – used to iterative delivery • Flexible and able to work across a mix of modern and legacy systems The Value You Deliver • Understanding business needs & processes, crafting, detailing, developing, and implementing Big Data applications in a dynamic financial services environment • Assesses Industry standards for data models, incorporating new advances into internal architecture standards • Ownership of the technology work and providing leadership needed to complete a project of the highest quality, on time and within budget • Sharing written and spoken information in a way that is understood by those needing the information • Performing complex data warehouse development • Monitoring progress, work quality and risk of the projects; calling out situations needing management attention • Solving and investigating issues in support of third line production support • Chipping in to and writing concise and clear technical systems documents based on the analysis of complex business requirements • Staying in sync with technical standards, policies, and procedures