18 nov
|
XChange Software
|
Guadalajara
18 nov
XChange Software
Guadalajara
Role: BigQuery Consultant
Type: Contract
Location: Guadalajara, Mexico - Hybrid
Position Purpose The BigQuery Infrastructure team is seeking a Tier III Senior Software Developer to take full engineering ownership of non-production infrastructure capacity management, CI pipeline optimization, and bug triage automation.
This is not a manual task-management role. You will manage approximately 300 to 500 BigQuery instances encompassing full stacks across multi-cell and multi-region environments (dev stacks, presubmit, continuous, and nightly builds). Your primary mission is to eliminate significant engineering toil for the core SWE team by building durable, automated engineering solutions rather than relying on manual workarounds.
Key Responsibilities
1. Capacity Management & Mitigation Engineering
- Own the operational monitoring, tracking, and remediation of resource stockouts across BigQuery D3E environments.
- Actively track cell-level capacity headroom, evaluate pool resourcing stockouts, and manage job sprawl.
- Execute structured mitigations using tools like Cellmate or Flex API, shifting Devices Under Test (SUTs) and workloads between regions and cells to maintain non-production environment health.
- Maintain and update infrastructure configurations to support new feature requirements.
- Proactively engineer automation to predict and prevent stockouts,
and manage tier-escalations to SRE or owning teams when standard mitigations are exhausted.
2. Presubmit Friction Reduction & Bug Triage
- Improve the automated test quarantine system to drive down slow, flaky, and quarantined tests.
- Own the triage process for incoming BigQuery infrastructure bugs, bridging communication effectively with bug owners.
- Analyze complex distributed systems to determine root causes and ensure issues are routed to the appropriate teams efficiently.
3. Innovation & Engineering Excellence
- Transition daily operations from reactive toil mitigation to proactive, durable system architecture improvements.
- Incorporate innovative, AI-driven workflows (such as agentic workflows or automated triaging) to enhance latency bug analysis and system efficiency.
Core Requirements:
- Experience: 8 years of professional software engineering experience in large-scale distributed systems.
- Primary Language: Strong proficiency in Python for infrastructure automation, tool development, and configuration.
- Infrastructure Mastery: Deep understanding of distributed systems architecture, multi-cell/multi-region environments, and container/resource management orchestration (e.g., Borg or equivalent large-scale cluster management).
- CI/CD Systems: Strong experience managing and optimizing Continuous Integration pipelines (presubmit, nightly builds, test quarantine frameworks).
📌 BigQuery Consultant (Guadalajara)
🏢 XChange Software
📍 Guadalajara