We're looking for a Senior Distributed Systems Engineer (Search Indexing) to join the Agentic Search team. In this role you'll own the Index layer - the systems that transform raw, web-scale crawled content into the searchable data structures that power retrieval for AI agents, operating at hundreds-of-terabytes-to-petabyte scale.
About the Company:
The company is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data and model training through to production deployment, without the cost and complexity of building large in-house AI/ML infrastructure. Listed on Nasdaq, headquartered in Amsterdam, with R&D hubs across Europe, the UK, North America, and Israel, and a team of 1,500+ including hundreds of engineers with deep expertise across hardware, software, and AI R&D. It's a "built by engineers, for engineers" culture that owns the hard problems across compute, storage, networking, and applied AI.
This role sits within the Agentic Search team, which is building an agent-native search platform designed specifically for AI systems rather than human users. The ambition is for it to become the web access layer for AI agents, letting them retrieve fresh, verified, real-world information through programmatic APIs. Unlike traditional search engines that optimise for humans, the entire stack is designed around agents, retrieval systems, ranking models, and future AI-native workflows.
Why this could be interesting:
True greenfield, the entire search stack is being built from scratch, so you'll make foundational architectural decisions, not maintain someone else's legacy.
A genuinely novel problem space, search built for AI agents, not humans, positioned to become the web access layer for the agentic ecosystem.
Serious scale, indexing pipelines processing hundreds of terabytes to petabytes of continuously evolving data.
An exceptionally senior team (L6+ bar, scaling 6 → ~30) where you'll work alongside Staff/Principal-level engineers with high ownership and direct impact.
What You Will Do:
The Index team transforms raw crawled content into searchable data structures. Once documents are collected, this team processes, enriches, and organises them for efficient retrieval. In this role you'll:
Design and build large-scale offline and nearline data-processing pipelines at the core of the search infrastructure.
Construct search indexes from web-scale crawled content and structured feeds.
Develop feature-generation systems that feed retrieval and ranking models.
Build and optimise large-scale distributed batch processing across datasets at hundreds-of-terabytes-to-petabyte scale.
Work on storage formats, compaction strategies, and update mechanisms that balance performance, freshness, and resource efficiency.
Define observability primitives - structured logs, metrics, and data-quality signals - and ensure reliability under high-throughput conditions.
Collaborate with runtime and ML teams to ensure indexing outputs meet retrieval and ranking requirements.
What We're Looking For:
5+ years building production backend or data systems, at a senior/staff+ level of ownership.
Solid hands-on experience with Go in real-world, high-load services (C++ or Rust is a plus).
Proven experience working with very large datasets, terabytes or more, not small-scale environments.
Experience building or operating large-scale data pipelines, data planes, or databases.
You've operated your own systems in production - handled incidents, made real tradeoffs, and understand what running data infrastructure at scale truly involves.
You think in systems and data flows rather than isolated components, and can independently design systems and make architectural calls in a greenfield, ambiguous environment.
You communicate clearly across infrastructure, ML, and product teams.
What the Company Offers:
Competitive compensation package depending on level, experience, and background.
Relocation assistance available for senior candidates relocating to Amsterdam, London, or Tel Aviv.
Flexible setup: hybrid in Tier-1 hubs, with remote possible for candidates on the US East Coast or within the EU. We are currently unable to consider candidates based in Russia or Belarus, ready to help with relocation!
Career growth and continuous learning opportunities.
A collaborative, innovative, "built by engineers, for engineers" culture.
The chance to shape a frontier AI product from the ground up, with real ownership.
An international environment with talented teams across multiple R&D hubs.