Padmi
Apollo Research logo
Apollo Research

AI safety · scheming detection

AI Security & Control Engineer

San Francisco Bay Area · LondonPosted 1 month ago
SecurityUnspecifiedFull TimeSponsorship mentioned
Apply at Apollo Research

Opens the source posting on jobs.lever.co

Source description

About the role

View original

Security support for research (50%)

Threat modeling for coding agents

Develop threat models for coding agents under the assumption that they could be misaligned or compromised. This includes near-term threats like AI agents leaking private internal data and long-term threats like agents sabotaging safety research.

Treat coding agents through the lens of insider risk: they have credentials, access to code, network access, and the ability to execute arbitrary actions, just like a malicious insider would (see Control agenda ).

Map out kill chains and attack progressions similar to frameworks like MITRE ATT&CK, adapted for agentic AI. See e.g. the Agentic Loss-of-Control Threat Matrix for an example of a high-quality contribution.

Build and maintain our “coding agent security levels” which define what level of robustness Watcher provides against different categories of failure modes. These levels should be concrete, testable, and usable both internally (to guide product priorities) and externally (to communicate our security posture to customers).

Failure mode library and literature

Maintain our library of coding agent failure modes and keep it comprehensive, accurate, and current.

Stay up to date with the academic and industry literature on coding agent security, AI security, and adjacent fields (prompt injection, deception, accidental security implications, supply chain attacks on AI systems, etc.).

Attack design for monitor development

Design realistic attack trajectories and example exploits for monitor development and backtesting, work with the research team to ensure monitors are trained and evaluated against credible adversarial behavior.

Trajectory adjudication

Serve as security adjudicator for flagged trajectories. When monitors flag ambiguous agent behavior, determine whether it represents a genuine attack, a benign-but-unusual workflow, or needs deeper investigation, and feed those judgments back into monitor improvement.

Red-teaming Watcher

Red-team and improve Watcher's monitors and policies, document findings, and feed them into product improvement.

Work with the technical research staff to build adversarial test suites into the backtesting pipeline.

Security expertise for product (25%)

Failure mode prioritization and CISO perspective

Understand which failure modes are the biggest problems and prioritise systematically what is most useful to solve for security buyers.

Customer engagement and landscape awareness

Join customer/prospect calls to understand security needs firsthand and map them onto the threat models and library of failure modes.

Maintain awareness of how enterprises currently secure coding agents (or don't), what tools they use (SIEM, DLP, CSPM), and how Watcher fits into their existing stack.

Customer-facing security artifacts

Own security questionnaire responses, product security whitepapers, and support for customer pen-tests.

Own technical content of customer-facing communications during incidents.

Improving security posture for product (25%)

Attack surface, infrastructure, and isolation

Own AppSec standards for the product engineering team (code review security checklists, dependency scanning, secrets management in CI/CD, container hardening)

Own vendor security decisions for product: which SaaS tools can we integrate and what restrictions should we have.

Reduce attack surface for all product-related deployments, infra and cloud deployment work, and ensure clean isolation between the Apollo Product team and other teams at Apollo to prevent lateral movement either way.

Data handling and multi-tenancy

Define security requirements for tenant isolation, encryption at rest/in transit, access controls, and data retention policies for customer coding agent transcripts with the product engineering lead.

Product threat model and incident response

Co-own the product-specific threat model with the product team lead (distinct from the org-wide threat model owned by existing security engineers) and keep it up to date.

More at Apollo Research

Related open roles

View all roles