How I implemented search on this website - pt. 1 / requirements and technical design

^{DATE : 21.02.2026}

Introduction

It is nice to be able to search a website. I wanted to implement something locally to that end for my website - no external dependencies required, which can be quite an useful trait in face of search engine enshittification. And it is interesting to build, ofc.

In this post, we’ll look at the requirements leading to, and the technical design itself of the implementation.

Requirements

When thinking about the idea, I eventually came to a certain set of requirements the search implementation must fulfill

• Useful and reliable
  • It must be considered that I only have a rather limited amount time and resources. Hence, the chosen method cannot be exceedingly complex to implement – which effectively requires me to forgo full text search. A constrained but effective search is better than a more free-form but less functional one.
• Practical to be operated client-side
  • Hosting an API takes plenty of effort and maintenance – and costs more
  • It is unreasonable to require users to download large indexes for a simple search. What is required at client-side should be of reasonable size even for a modest connection
• Able to understand relationships and suggest useful pages in a meaningful order
  • Assuming the user formulates their search query as precisely as possible, the search engine should have a good idea of what other content might be of interest to the user in addition to the closest match

Technical design

Chosen strategy

With the given requirements, I ended up choosing a graph-based topic search as my approach, on the following basis:

• Topic search is relatively easy to implement – I can manually define appropriate topics per page, which can then be searched for and displayed to the user. No need for parsing and other natural language processing complexity
• By limiting the amount of searchable material (effectively a small set of topics, plus short titles and descriptions for pages), the index will remain reasonably small for a fair while. It can eventually grow to a size where it is unsustainable without additional splitting work, but it isn’t something I need to be immediately concerned about
• Graphs can efficiently model relationships between topics and pages, and provide a reasonable heuristic on what pages might be of most interest for a given search term

Internal structure

I will gloss over most of the implementation code, as it is relatively mundane. However, the central state structure will merit a modicum of attention

/// A search computer is essentially a data structure, to which search topics and other data (e.g. linkages) can be incrementally
/// entered; once ready, JSON-serializable structure(s) can be generated and stored for the usage of client-side scripts
///
/// Internal data structures are definitively ordered either by insert order or by key order, ensuring consistency
/// for the same sequence of operations applied to the structure
#[derive(Debug, Clone)]
pub struct SearchComputer {
    /// Counter of identifiers issued
    counter: usize,
    /// Topic to identifier mapping. One identifier may appear for multiple topics, in case of alternative terms being defined
    topic_to_ident: BTreeMap<String, usize>,
    /// Linkages between IDs; the direction of linkage is from left to right
    /// This is an index set, as insertion order is significant
    id_linkage: IndexSet<(usize, usize)>,
    /// List of issued external identities, for validation purposes
    issued_external_identifiers: Vec<usize>,
}

Whilst one could perhaps change the BTreeMap to a HashMap with relatively little damage (generated data might be in slightly different order, but not in a particularly problematic way), the IndexSet here is actually rather important. If it were replaced by any other type of set, the linkage order would not be determined by the actual insert order.

This would then promptly backfire by muddling the priority likely intended by the user (most important relationships first, less important ones last). And that definitely isn’t something I want.

Interface to client-side code

The index will be stored in a single file, expressed by the following Rust data structure:

#[derive(Debug, Clone, Serialize, PartialEq)]
pub struct SearchComputerExportedData {
  /// Topic to ID mapping. It is permissible to have multiple distinct topics lead to the same index value.
  pub topic_mappings: BTreeMap<String, usize>,
  /// ID to external data mapping (implementation defined, expressed as a JSON value)
  pub externals: BTreeMap<usize, Value>,
  /// Navigation routes from a given topic or external ID to other IDs
  /// This is precomputed to allow for quickly determining a list of related results
  ///
  /// # Format
  ///
  /// Data is keyed by the source ID. Value is a tuple consisting of a numeric count and a list of valid linkages
  ///
  /// The first N IDs in the list indicated by the count are linkages that lead to sub-topics or external data; rest are paths that navigate "backwards" and hence likely less interesting to the user.
  pub routes: BTreeMap<usize, (usize, Vec<usize>)>,
}

To save space, individual graph nodes are keyed with a sequential numeric value. An individual node can be either an external node (reference to a web page) or another topic.