Data Engineer

Job title: Data Engineer Location: Remote Minimum Experience Required: · At least 2+ years of RECENT SPARK experience using own code in Scala, Python (PySpark), Java. · At least 2+ years of senior level data structures and algorithms experience building, using own code. · At least 2-5 years of relational database experience using own SQL queries and stored procedures. · At least 3-5 years of database development or ORM object relational mapping using .Net or Java. · At least 3-5 years of integrating batch and streaming data pipelines with modern data warehouses. Top Requirements / Essential Skills: • Degree in Computer Science or Computer Information Systems will be preferred – at least a BS Degree. • Software Development experience in one or more object-oriented programming languages (e.g. Python, Go, Java, Scala, C#, C++) Experience with algorithms, data structures, and building systems architecture. • Ideally has a mix of multiple coding languages – they want someone who is open to different languages and somewhat flexible with the target for writing programming code or rewriting it in another language. • Advanced SQL coding skills: writing complex queries and stored procedures for database development. • Experience with multiple relational databases, plus NoSQL, and modern data warehouses on the cloud. • Uses best practices and standards for data engineering, security, data privacy, reliability, & scalability. • Recent deep hands-on experience (typically 5+ years) with developing reliable and scalable Big Data infrastructure and data products in large-scale distributed systems for complex enterprise environments. Key Technologies / Tools: · Spark applications in Databricks, AWS EMR, Azure HDInsight, Cloudera/Hortonworks, or GCP Dataproc. · Data integration to/from Data lakes like AWS S3, Azure Blob, Databricks Delta Lake, or Google Storage. · Building custom data pipelines using Airflow (Python/PySpark), or Kafka (using Scala, Python, or Java). · Cloud data orchestration tools: Azure Data Factory (ADF), AWS Glue, GCP DataFlow, or Apache Beam. · Modern data warehouses: Snowflake Cloud DW, AWS Redshift, Azure Synapse/DW, or GCP Big Query. · Deploying machine learning models into production using Azure ML, SageMaker, TensorFlow, or Keras. Soft Skills / What Gets the Win: • Thirst for continuous learning and making technological improvements across all data lifecycle stages. • Great communication skills with stakeholders and ability to lead large organizations through influence. • Desire to grow with the team; senior level roles will do some architecture and design their own systems. • Prefer candidates from IT companies or ones using the latest innovations and cutting edge technology. Thanks&Regards, Kiran Tumuluri IT-Recruiter Staffing | Thought wave Software and Solutions Limited 6308125139kiran@thoughtwavesoft.com www.thoughtwavesoft.com 314 N. Lake St, Suite 6, Aurora IL 60506,