Source description
About the role
This SRF is for DataOps Engineer (FTE) at Prague location.
DataOps (data operations) Engineer will be responsible for designing, implementing and maintaining a distributed data architecture that will support a wide range of open source tools and frameworks in production.
Key Responsibilities
· Create and maintain optimal data and model dataOps pipeline architecture
· Assemble large, complex data sets that meet functional / non-functional business requirements.
· Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
· Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and cloud-based 'big data' technologies from AWS, Azure and others.
· Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
· Keep data separated and secure across national boundaries through multiple data centers and strategic customers/partners.
· Create tool-chains for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
· Work with data and machine learning experts to strive for greater functionality in our data and model life cycle management systems.
Key Qualifications
· Bachelors/Masters/Ph.D. in Computer Science, Information Systems, Data Science, Artificial Intelligence, Machine Learning, Electrical Engineering or related disciplines from any of the reputed institutes. First Class, preferably with Distinction.
· Overall industry experience of 5+ years, at least 3 years' experience as a Data Engineer.
· 3+ years of experience in the following:
· Software/tools: Hadoop, Spark, Kafka, etc.
· Relational SQL and NoSQL databases, including Postgres and Cassandra.
· Data and Model pipeline and workflow management tools: Azkaban, Luigi, Airflow, Dataiku, etc.
· Stream-processing systems: Storm, Spark-Streaming, etc.
· Object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
· Experience performing root cause analysis on internal and external data and processes to answer specific business questions and seek opportunities for improvement.
· Experience in Data warehouse design and dimensional modeling
· Experience building processes supporting data transformation, data structures, metadata, dependency and workload management.
· Advanced SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of other databases/date-sources.
· Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores.
· Experience with Docker containers, orchestration systems (e.g. Kubernetes), continuous integration and job schedulers.
· Knowledge of server-less architectures (e.g. Lambda, Kinesis, Glue).
· Experience with microservices and REST APIs.
· Experience with data visualization and dashboard creation is a plus Job type: Permanent Division: eTeam Workforce Limited Reference: 19-00916
More at eTeam