Building a dumb search in astro

Let me tell you how I built my dumb, yet I think quite useful search feature for my [https://joshuastuebner.com/digital-garden/](Digital Garden).

I’m sure there are a thousand good ways to build a search, but this one is simple to set up and requires not backend whatsoever.

This has been a feature for some time on my website and it has been broken and working on and off throughout the astro builds.

The starting point

I’m using markdown files for content. They’re stored elsewhere and being pulled into the build directory whenever I rebuild my website. I wanted some way to make these searchable, as there might be many small snippets and being able to find one is useful. So I needed a way to somehow “index” the markdown files, ideally at build time and make them searchable.

Generating the search data

In astro you can run JS/TS that will be executed during the build (or even SSR it, but that’s a different scope), that is a little bit like in Next.js.

This is usually used to build pages from content, but you can also use it in other ways.

They also added a new feature called collections, which allows you to type and validate your content, which I use in this, but let this just be an aside.

What I do: I get all the files that I want to be searchable and map their content and meta data into one large map. This map is then saved into a single .json file.

This little search file is then written to disk and later on consumed by a svelte component to be searched. That’s it.

Let me show you some code

const allIdeas = await getCollection('DigitalGarden');  
  
const contentForSave = await Promise.all(  
	allIdeas.map(async idea => {  
		const { headings } = await idea.render();  
		return {  
			title: idea.data.title,  
			description: idea.data.description,  
			link: `${Navigation.digitalGardenDetailPage.url}/${slugify(idea.data.title)}`,  
			headers: headings,  
			content: idea.body.replaceAll('\n', '').replaceAll('#', ''),  
		};  
	}),  
);  
  
const writePath = import.meta.env.DEV  
	? new URL('../../../public/search.json', import.meta.url)  
	// write straight to dist folder during build
	: new URL('../../../dist/search.json', import.meta.url);  
  
await fs.writeFile(writePath, JSON.stringify(contentForSave, null, 2));

As you can see, I map through the content, format it a little bit and then use node:fs to write it to the disk.

Small hint: There were some race conditions when generating the search file, where the content of the static /public directory would be copied before the file was written, so now I just write it straight to the dist directory.

You can view the file here: https://joshuastuebner.com/search.json.

Searching the file

Last step: when the search component is initialised, it parses the .json into memory. Whenever someone now searches something, it will

1. Check if there’s a match in the title
2. Check if there’s a match in the meta data
3. Check if there are matches in the content.

The last one I do using a neat regular expression, that I totally build myself:

const regex = new RegExp(`(?:(\\S+\\s+)){0,3}${input}(?:\\s+(?:\\S+\\s+)){0,3}`, 'gmi');

It checks the words before and after and also displays them. I think the positive lookup might not work in every browser, but that is okay. It’s a form of progressive enhancement.

Small thing I learned here: when you use new RegExp you need to escape all slashes with another one. Good to know.

And that’s it. Hope it was a nice read!