Azure Data Engineer with Databricks Expertise

Call us 609-945-2590 410 Wall St. Princeton, NJ 08540 support@alltechconsultinginc.com Login / Register Login to Alltech Consulting Services reset form Reset Password Please Enter Username or Email Create a free Alltech Consulting Services account Already have an account? Login heading Azure Data Engineer with Databricks Expertise Iseline January 22, 2025 Contract Apply Now email apply form here Apply for this job Apply with login Please login as "Candidate" to apply Back to jobs Main content job description Job Description Job Description: We are seeking highly skilled Azure Data Engineer with strong expertise in Databricks to join our data team. The ideal candidate will design, implement and optimize large-scale data pipeline, ensuring scalability, reliability and performance. This role involves working closely with multiple teams and business stakeholders to deliver cutting-edge data solutions. Key Responsibilities: Data Pipeline Development: Build and maintain scalable ETL/ELT pipelines using Databricks. Leverage PySpark/Spark and SQL to transform and process large datasets. Integrate data from multiple sources including Azure Blob Storage, ADLS and other relational/non-relational systems. Collaboration & Analysis: Work Closely with multiple teams to prepare data for dashboard and BI Tools. Collaborate with cross-functional teams to understand business requirements and deliver tailored data solutions. Performance & Optimization: Optimize Databricks workloads for cost efficiency and performance. Monitor and troubleshoot data pipelines to ensure reliability and accuracy. Governance & Security: Implement and manage data security, access controls and governance standards using Unity Catalog. Ensure compliance with organizational and regulatory data policies. Deployment: Leverage Databricks Asset Bundles for seamless deployment of Databricks jobs, notebooks and configurations across environments. Manage version control for Databricks artifacts and collaborate with team to maintain development best practices. Technical Skills: Strong expertise in Databricks (Delta Lake, Unity Catalog, Lakehouse Architecture, Table Triggers, Delta Live Pipelines, Databricks Runtime etc.) Proficiency in Azure Cloud Services. Solid Understanding of Spark and PySpark for big data processing. Experience in relational databases. Knowledge on Databricks Asset Bundles and GitLab. Preferred Experience: Familiarity with Databricks Runtimes and advanced configurations. Knowledge of streaming frameworks like Spark Streaming. Experience in developing real-time data solutions. Certifications: Azure Data Engineer Associate or Databricks certified Data Engineer Associate certification. (Optional) Apply Now email apply form here Apply for this job Apply with login Please login as "Candidate" to apply job social Share this post Facebook Twitter LinkedIn Email job releated Related Jobs Senior Project Manager United States JPC-1702 Contract #post-## Senior Consultant United States JPC-1701 Contract #post-## Technical Project Manager Alpharetta - GA JPC-1699 Contract #post-## Job Overview Job ID JPC-557 Date Posted January 22, 2025 Location Iseline Expiration date -- Experience 4-6 Years Qualification Bachelor Degree Job Location Job Skills Azure Business Analysis Cloud Data Management ETL PySpark SQL storage troubleshooting #post-## .site-main .content-area .site-content Alltech Consulting Services 410 Wall St. Princeton, NJ 08540 609-945-2590 609-430-0133 CLIENT Why Alltech Services Expertise CANDIDATES Why Alltech View Jobs Submit Resume Recruitment Referral Program COMPANY About Us Contact Us Careers Privacy Policy Terms of Service Cookie Policy © Copyright Alltech Consulting Services, 2022. All rights reserved. .site WP Job Board Pro Structured Data