Padmi

Data Architect

New YorkPosted 1 month ago
Infrastructure And DatabasesUnspecified
Apply at Thoughtwave Software and Solutions

Opens the source posting on atsapp.swarmhr.com

Source description

About the role

View original

Title: Technical Lead (Data Engineering & Data Architecture Team) Location: Remote – living in EST/CST only Team has daily standups and collaboration meetings with off-shore teams generally from 8:30-11:30 AM EST daily. Need to live in EST or CST. Duration: Contract to Hire (candidates must be able to convert to FTE with no current or future sponsorship requirements) Role Summary This role will be responsible for transforming extensive and complex data into consumable business capabilities. Create system architecture, design, and specification using in-depth engineering skills and knowledge to solve complex development problems and achieve engineering goals. Determine and source appropriate data for a given analysis. Work with data modelers/analysts to understand the business problems they are trying to solve, then create or augment data assets to feed their analysis. Acts as a resource and mentor for colleagues with less experience. Lead a team of data engineers through a modernization project of moving on-premise big data implementation to cloud. Core Responsibilities Lead a team of Data Engineers to develop data products and tools, explore new technologies, and continually improve tools and processes. Hands-on experience working with Data Analysis and Architecture, Spark, PySpark, Python, Sqoop, Hive, No SQL Data Stores, Object Store, Design and Development of APIs, Kafka. Must be an experienced Python developer with experience in developing frameworks and enforcing best practices. Create business value by leading the design, get your hands dirty, write code, and ultimately deploy big data and machine learning capabilities. Possess expert knowledge in performance, large-scale data distributed system scalability, system architecture, and data engineering best practices. Give the highest priority to operational excellence, evaluate system performance, security, design system metrics, and driving quality improvements. Provide leadership, work collaboratively, and be a mentor in a fantastic team. Your expertise is deep and broad; you’re hands-on, producing both detailed technical work and high-level architectural designs. 8+ years of recent hands-on in an object-oriented language (Java, Scala, Python). 8+ years of experience designing and building data pipelines and data-intensive applications. Experience using Big Data frameworks (e.g., Hadoop, Spark), databases for complex data assembly and transformation. Experience designing and implementing complex big data implementation using Databricks, Snowflake, Cloud (Azure or Google Dataproc) Experience working with Healthcare data is a plus. Skills Requirements Bachelor’s Degree in Computer Science or related field of study. Strong understanding/experience with Data Engineer and Data Architecture in cloud based environment (Azure). 10+ years of Big Data/Data Engineering experience. 1-2+ years of hands-on Technical leadership experience required. 10+ years of recent hands-on in an object-oriented language (Java, Scala, Python) experience. 4+ years of Big Data technologies such as Hadoop, Pig, Hive & Scoop required. 3+ years of experience with Azure, ADF & Databricks required. 3+ years of Spark, Python and Pyspark required. 2+ years of Spark or Kafka Streaming. 1-2+ years of experience with Snowflake Cloud Database.

One address, no account. We’ll tell you when matching roles go live.

More at Thoughtwave Software and Solutions

Related open roles

View all roles