LLM Red Team Intern (Evaluation Systems)

View all jobs

LLM Red Team Intern (Evaluation Systems)

Austin, Texas

Engineering

Remote

Intern

Internship | Remote | LLM Evaluation | Reports to CTO or Safety Lead

About Elloe

Elloe is the immune system for AI.

We don’t train models — we protect their outputs. We trace every hallucination, enforce every policy boundary, and create an audit trail for every critical LLM interaction.

Our modules (TruthChecker™, AutoRAG™, Autopsy™) are embedded in hospitals, banks, and regulatory sandboxes. Our job is to make sure these systems are safe before anything hits production.

This role will help us break, stress-test, and harden the models used by governments and enterprises alike.

About the Role

You’ll red team real-world LLM deployments, design eval harnesses, and help scale Elloe’s output-level safety layer. This isn’t just prompt tuning — it’s forensic risk mapping.

You’ll work directly with product and safety leads to uncover failure patterns and codify guardrails for GenAI systems under real-world scrutiny.

What You’ll Own

• 1. Red Teaming & Risk Testing

• Create prompts to trigger hallucinations, policy violations, or failure scenarios

• Stress test Elloe-protected deployments using open and proprietary models

• Document behavioral exploits across use cases (healthcare, compliance, gov)

• 2. Evaluation Design

• Build truthsets and scoring rubrics tied to factuality, policy, or ethical standards

• Benchmark Elloe’s modules across model types (Claude, GPT-4, Gemini, open models)

• Collaborate with product to refine and expand our eval harnesses

• 3. Safety Intelligence

• Identify blind spots in current detection logic

• Recommend scoring methods or red flag thresholds for deployment

• Support internal model comparison reports or customer safety audits

Who You Are

• ML/AI researcher or engineer (undergrad, grad, or early career)
• Experience working with LLMs, eval sets, and prompt design
• Strong attention to detail, grounded in safety and adversarial thinking
• Bonus: exposure to safety benchmarks like TruthfulQA, MMLU, or red teaming tools

Why This Matters

This is real-world alignment, not research theater.

You’ll be helping define how AI gets deployed responsibly — with traceability, transparency, and real-time protection.

You’ll leave this role with

• Exposure to high-stakes LLM safety deployments
• Published frameworks or scoring methods used by enterprises
• Mentorship from technical founders operating at the bleeding edge of AI safety

Logistics & Application

• Start Date: Rolling
• Duration: 12–16 weeks
• Compensation: Research stipend
• Location: Remote-first; flexible for global candidates
• To Apply: Share a jailbreak or eval idea you’d love to run against GPT-4.

Ready to apply?

Powered by

First name *

Last name *

Email *

LinkedIn URL *

Phone number *

Location *

Resume *

Click to upload or drag and drop here

Cover letter *

Click to upload or drag and drop here

Apply

Req ID: R17