Source description
About the role
Role: Data Engineer Duration: 3/31/2021 (Possibility for extension) Location: 100% Remote This is NOT a cloud engineer role. This is a data engineer role. Job Description: • Engineer should be able to design, build, operationalize, secure, and monitor data processing systems with a particular emphasis on security and compliance; scalability and efficiency; reliability and fidelity • Proficient in building and consuming REST services. • Experience building big data pipelines with Scala/Spark • Design and development of data ingestion services that can ingest 10s of TB of data; • Experience with object-oriented scripting languages: Java, Python(required), Scala(required) etc. • Research, design, and implement well tested improvements to the existing codebase • Review pull requests and provide valuable feedback to members of the team • Strong fundamentals in distributed systems design and development • Experience with relational and NoSQL database and related technologies (e.g. Postgres, Redis, Cassandra, ELK) • Strong working knowledge of container based infrastructure on GCP (e.g. Docker, Kubernetes, GKE, etc.) • Experience with configuration management tools and concepts (e.g. Chef, Ansible, Docker, Puppet, etc.) • Experience with CI/CD tools and concepts (e.g. Jenkins, Artifactory, etc.) • Automate CI and deployment processes and best practices for the production data pipelines • Experience and working knowledge with HashiCorp Terraform including experience writing Terraform Modules • Ability to communicate effectively (both verbal and written) • Google Cloud Platform Experience A cloud engineer is an IT professional responsible for any technological duties associated with cloud computing, including design, planning, management, maintenance and support. Job Responsibilities for Cloud Engineer: • Installs and maintains cloud-based applications, systems, and associated infrastructures. • Translates business requirements into code and documentation while managing the life-cycle and communication for assigned projects. • Analyzes and defines business requirements to determine specifications and standards. • Oversees the proper implementation and integration of web and cloud-based applications. • Stays informed of emerging cloud technologies and evaluates the value to the organization’s operations. • Monitors system performance, troubleshooting and resolving errors. • Ability to consult with internal and external resources to develop cloud solutions for a range of products. • Ensures quality of the solutions are robust, scalable and efficient to meet the needs of the business. Knowledge and Skills for Cloud Engineer • Requires experience with one or more cloud platform. • Experience working with large infrastructure hosting environments including knowledge of Application Hosting, distributed messaging architectures, network technologies (Ex: VPNs, TCP/IP, Firewalls, Routing, etc.), security technologies (Ex: public key cryptography and infrastructure, smart cards, authentication, authorization, etc.). • Demonstration of thorough understanding of how discrete infrastructure components work together to provide reliable and secure hosting environment. • Vast understanding of cloud services and solutions in platforms such as Azure or AWS • Comprehensive understanding of IT security best practices and threat protection • Must have a good understanding of connected embedded systems tradeoffs, protocols, etc.
More at Thoughtwave Software and Solutions