Source description
About the role
Project Description:
The Fund Accounting Data Project ingests, rationalizes, and publishes fund accounting data from first‑party and third‑party Fund Accounting systems into a centralized Data Platform .
The platform supports two primary outcomes:
A client‑facing portal with analytics and data visualizations
The Core Accounting Model , acting as the internal canonical representation of accounting data
The initiative involves:
SQL Server‑based source systems
API‑based vendor data feeds
Cloud‑based processing and transformation (Databricks)
High requirements for data accuracy, auditability, and lineage
Role Objective
Provide a Senior Software Engineer who will act as a Data Engineer , responsible for building and maintaining scalable, reliable data ingestion and transformation workflows based on specifications produced by the Technical Analyst.
This role is accountable for the implementation, correctness, performance, and operability of data pipelines that publish trusted datasets to the Data Platform.
Role Context
Embedded within a data platform engineering team.
Works closely with:
Technical Analysts (data specifications and mappings)
Other Data Engineers
Platform / Cloud Engineers
QA and Product partners
Operates in an Agile delivery model with frequent releases.
Focused on backend engineering rather than analytics or visualization.
Core Responsibilities (What They Will Do)
Implement data ingestion pipelines from:
SQL Server source systems
External vendor APIs
Build data transformation workflows in Databricks (Spark) using Python and/or SQL .
Implement source‑to‑target mappings and transformation rules defined by the Technical Analyst.
Design and maintain reusable, modular data processing components.
Ensure data pipelines meet requirements for:
Accuracy and completeness
Performance and scalability
Fault tolerance and recoverability
Implement data quality checks, validations, and reconciliations.
Support lineage and auditability through logging, metadata, and structured outputs.
Troubleshoot ingestion and transformation failures across environments.
Partner with Technical Analysts to identify gaps, edge cases, and improvements in specifications.
Explicit Non‑Responsibilities (Important for Profile Filtering)
Not a business analyst or requirements owner.
Not a BI or reporting developer.
Not a front‑end or visualization engineer.
Not a DevOps engineer, though expected to understand deployment and runtime behavior.
Required Experience (Non‑Negotiable)
7+ years experience as a Software Engineer, with significant experience in Data Engineering roles.
Strong hands‑on experience with Databricks or Spark‑based data processing platforms .
Advanced Python proficiency for data transformation and pipeline development.
Strong SQL skills, including querying and transforming large datasets.
Experience ingesting data from:
Relational databases (e.g., SQL Server)
REST or similar APIs
Proven experience building production‑grade data pipelines in regulated or data‑sensitive environments.
Preferred Background (Strong Signals)
Experience in financial services, asset servicing, or fund accounting environments.
Familiarity with accounting data domains (positions, transactions, balances, NAV).
Experience working with canonical or enterprise data models.
Exposure to:
Data quality frameworks
Data observability and monitoring
Cloud‑native data platforms
Experience collaborating closely with Technical Analysts or Data Architects.
Success Criteria (How We Will Judge Fit)
A successful Senior Software Engineer on this project will:
Deliver stable, efficient data pipelines aligned with documented specifications.
Identify and resolve data anomalies and edge cases early in implementation.
Minimize production defects related to data correctness and transformation logic.
Create pipelines that are understandable, supportable, and extensible by other engineers.
Enable reliable downstream consumption by analytics platforms and core accounting models.
Seniority Guidance
Senior individual contributor.
Expected to own pipeline design and implementation end‑to‑end.
Comfortable making technical decisions and tradeoffs within the data platform.
Acts as a technical reference for data ingestion and transformation patterns.
More at Photon