Manager, Software Engineering (Resilience Engineering)

About This RoleAI processing…

Key Responsibilities

• 1
  Own the design and evolution of platforms that enable safe, controlled production load testing and fault injection.
• 2
  Ensure strong safeguards are in place, including isolation boundaries, approval workflows, and automated rollback mechanisms to protect real users.
• 3
  Build systems that provide end-to-end observability, traceability, and auditability for all resilience experiments.
• 4
  Drive reliability improvements by systematically identifying weaknesses through load testing and chaos experiments.
• 5
  Establish monitoring, alerting, and incident response practices tailored to proactive resilience validation.
• 6
  Work closely with engineering teams to design and execute production load tests and chaos experiments safely.
• 7
  Partner with infrastructure teams to build guardrails around tests and experimentations.
• 8
  Enable teams to adopt resilience practices by providing reusable tooling, frameworks, and standardized workflows.
• 9
  Identify systemic weaknesses and lead cross-functional efforts to improve reliability and fault tolerance.
• 10
  Evangelize a culture of “test failure before failure tests you” across the organization.

Requirements

• Proven experience leading engineering teams in reliability, infrastructure, or distributed systems.
• Hands-on experience with production load testing, chaos engineering, or large-scale system validation.
• Experience with leveraging a chaos engineering vendor such as Gremlin, Harness, or something similar.
• Strong understanding of failure modes in distributed systems, including latency, partial failure, and cascading outages.
• Experience building or operating systems with strong safety guarantees (isolation, rate limiting, guardrails, auditability).
• Familiarity with cloud-native environments (AWS, Kubernetes) and observability tooling.
• Strong programming background (e.g., Python, Kotlin, Java, or similar).
• Excellent problem-solving skills and the ability to balance long-term resilience investments with immediate business needs.
• Strong communication and leadership skills, with a track record of influencing engineering practices across teams.

Perks & Benefits