Principal Engineer – Agentic Security Operations

First American  ยท  via Get Backend Jobs
๐Ÿ’ฐ $170,900 - $227,900/yr ๐ŸŒŽ Remote ๐Ÿ“ Remote, California

Job Description:

  • Lead the architecture and hands-on implementation of Agentic Security Operations capabilities across the SOC
  • Define and evolve reference architecture for governed data services, agent capabilities, workload routing, execution environments, and deterministic orchestration
  • Build reusable capabilities that translate operational intent into tested, reviewable, and auditable security outcomes
  • Establish engineering standards for creating, testing, registering, approving, releasing, monitoring, and retiring agents
  • Advance detection-as-code practices, including versioning, simulation, deployment, tuning, migration, and rollback
  • Build governed knowledge and context services with authoritative documentation, policies, operational history, citations, freshness information, and permission-aware retrieval
  • Design workload identity, least-privilege access, short-lived authorization, spend controls, approval gates, audit trails, rollback, and emergency shutdown mechanisms
  • Define autonomy levels for automated, human-approved, and analyst-led activities
  • Establish agent evaluation and observability through replay, synthetic scenarios, comparisons, policy tests, traces, outcome verification, and regression testing
  • Measure quality, security, reliability, latency, cost, analyst acceptance, and operational impact
  • Engineer integrations across search, case management, workflow, communications, and response systems
  • Ensure consequential response actions are deterministic, attributable, reversible, and subject to human oversight
  • Design production reliability, continuity, recovery, and support models
  • Lead proofs of concept and architecture evaluations using measurable criteria
  • Collaborate with enterprise architecture, infrastructure, data, risk, privacy, legal, and compliance teams
  • Mentor engineers, establish technical standards, review designs, and influence Threat Detection Engineering strategy
  • Present technical decisions, risks, tradeoffs, and investment recommendations to engineering and executive stakeholders
  • Partner across Threat Detection and Response groups to translate operational feedback into improvements

Requirements:

  • Extensive experience designing and operating enterprise security, software, data, or distributed systems at scale
  • Principal-level technical leadership with hands-on architecture, prototyping, coding, testing, and production delivery
  • Hands-on experience delivering production AI-assisted or agentic systems using tools, memory, retrieval, planning, and structured workflows
  • Strong understanding of security operations, including telemetry management, detection engineering, threat hunting, triage, investigation, response, and automation
  • Experience with high-volume streaming and batch data pipelines, object-based storage, elastic analytics, distributed query, schema management, and data-quality controls
  • Experience designing modular agents, reusable capabilities, workload routing, local and managed execution, and human-in-the-loop workflows
  • Deep understanding of grounding, retrieval, source attribution, context freshness, permission inheritance, and protection against untrusted or manipulated context
  • Experience evaluating nondeterministic systems using versioned test sets, historical replay, synthetic scenarios, regression testing, and measurable outcome criteria
  • Strong software-engineering practices, including source control, automated testing, continuous delivery, infrastructure automation, peer review, and rollback
  • Experience designing secure APIs, event-driven integrations, workload identities, authorization boundaries, secrets management, and auditable action execution
  • Knowledge of AI security risks such as prompt injection, data poisoning, excessive agency, sensitive-data exposure, insecure tool use, and model or workflow drift
  • Strong production-engineering experience covering observability, service objectives, capacity, failure isolation, continuity, recovery, and operational support
  • Ability to evaluate emerging capabilities critically and validate architectural assumptions through controlled testing
  • Excellent written and verbal communication skills
  • Ability to lead through influence and work effectively across organizational and technical boundaries
  • Hands-on experience with cloud-scale data and analytics platforms supporting security operations, including Databricks, Snowflake, or similar technologies
  • Experience operating and migrating security analytics and detection workloads across modern platforms
  • Strong working knowledge of AWS, Azure, and GCP, including identity, networking, logging, data protection, and workload-security controls
  • A minimum of 10 yearsโ€™ experience in cybersecurity or relevant fields
  • Bachelorโ€™s degree in computer science, Information Technology, Cybersecurity, a related field, or equivalent work experience
  • Preferred certifications such as GASAE, GAIPS, GDSA, GCDA, CEH, OSCP, CISSP, etc.
  • Preferred experience modernizing centralized security analytics environments, knowledge graphs, entity resolution, semantic normalization, retrieval-augmented systems, multi-agent coordination, security-content migration, detection engineering, incident response, threat hunting, adversary emulation, security automation, AI governance, and internal security platforms

Benefits:

  • Medical insurance
  • Dental insurance
  • Vision insurance
  • 401(k)
  • PTO/paid sick leave
  • Employee stock purchase plan
  • Inclusive workplace culture
  • Equal opportunity employment