Senior Data Engineer

As a Senior Software Engineer - Data Engineering at Wells Fargo, your role will involve leading moderately complex initiatives within technical domain environments. You will contribute to large-scale planning strategies, design, code, test, debug, and document projects associated with technology domain. Your responsibilities will also include reviewing technical challenges, resolving issues, and collaborating with peers and managers to achieve goals. Additionally, you will lead projects, provide guidance to less experienced staff, and act as an escalation point. Key Responsibilities: - Lead moderately complex initiatives and deliverables within technical domain environments - Contribute to large scale planning of strategies - Design, code, test, debug, and document for projects and programs associated with technology domain, including upgrades and deployments - Review moderately complex technical challenges that require an in-depth evaluation of technologies and procedures - Resolve moderately complex issues and lead a team to meet existing client needs or potential current clients needs while leveraging solid understanding of the function, policies, procedures, or compliance requirements - Collaborate and consult with peers, colleagues, and mid-level managers to resolve technical challenges and achieve goals - Lead projects and act as an escalation point, provide guidance and direction to less experienced staff Qualifications Required: - 4+ years of Software Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education Desired Qualifications: - Design, build, test, deploy, and maintain large-scale structured and unstructured data pipelines using Python, SQL, Apache Spark, and modern data lake/lakehouse technologies - Develop and optimize metadata-driven pipelines, wrappers, ingestion frameworks, and validation layers to support GFCM data workflows - Build and maintain high-quality ELT/ETL pipelines following best practices in reliability, performance, observability, and reusability - Engineer and optimize Spark pipelines for large-scale batch and streaming workloads - Work with open table formats such as Iceberg, Delta, or Hudi for versioned data, time-travel, compaction, and schema evolution - Implement Medallion (Bronze/Silver/Gold) architecture patterns for modern lakehouse systems - Implement automated data quality frameworks and build data health monitoring frameworks - Build RESTful and metadata APIs using Python frameworks to enable secure, governed data access - Design and deploy data pipelines in cloud platforms leveraging managed compute, orchestration, and storage - Build resilient orchestration flows and apply modular design with retries, alerts, SLAs, and workflow dependency management - Work with cross-functional Agile teams and independently deliver complex engineering tasks - Hands-on experience with Python, SQL, and bash scripting for automation - Strong experience building big data pipelines using Apache Spark, Hive, Hadoop - Experience with Autosys/Airflow or similar orchestration tools - Working knowledge of REST APIs, Object Storage, Dremio, and CI/CD pipelines - Solid troubleshooting and problem-solving capabilities - Experience with data modeling, database design, and distributed systems patterns - Experience with streaming frameworks like Spark Structured Streaming or Apache Flink Additional Company Details: - Wells Fargo is an equal opportunity employer that values building strong customer relationships and maintaining a risk mitigating and compliance-driven culture - Employees are responsible for executing all applicable risk programs, following and adhering to Wells Fargo policies and procedures, fulfilling risk and compliance obligations, and making sound risk decisions - The company emphasizes proactive monitoring, governance, risk identification and escalation, and making risk decisions aligned with the business unit's risk appetite and all risk and compliance program requirements As a Senior Software Engineer - Data Engineering at Wells Fargo, your role will involve leading moderately complex initiatives within technical domain environments. You will contribute to large-scale planning strategies, design, code, test, debug, and document projects associated with technology domain. Your responsibilities will also include reviewing technical challenges, resolving issues, and collaborating with peers and managers to achieve goals. Additionally, you will lead projects, provide guidance to less experienced staff, and act as an escalation point. Key Responsibilities: - Lead moderately complex initiatives and deliverables within technical domain environments - Contribute to large scale planning of strategies - Design, code, test, debug, and document for projects and programs associated with technology domain, including upgrades and deployments - Review moderately complex technical challe