Can I build an index from non-HTML sources?

[documentat-alibaev]

I want to index my content without using data-pagefind-* options. In fact, I would like to index Markdown sources before the HTML site is even built. That way, I would be able to hide or even add some content to the index based on my own business logic.

Is there a way to do so? I cannot find documentation for the index file format or any CLI for generating it from non-HTML sources.

[documentat-alibaev]

Oh, I think I found it: https://pagefind.app/docs/node-api/

I was expecting a CLI, but there is only a JS API, is that correct?

[bglw]

Yes, the Node API is the intended path for pulling in non-HTML sources :)

It uses the Pagefind binary under the hood in a special "service" mode, so there's no particular technical barrier to providing a CLI interface, just a design/implementation body of work. But the Node API will likely always be a superset of the functionality available, so the initial development work went there.

Let me know how it works out for you! The Node API / service mode is one of the newer areas of Pagefind, so I'm still looking out for general feedback on its direction 😄

[documentat-alibaev]

@bglw When using addCustomRecord(), is there a way to let Pagefind know about H2+ headers on the page, so that it would produce results with anchored links?

[documentat-alibaev]

I haven't touched weights, filters and sorting yet, so i don't quite know everything that is shown here. But as for content_parts — yes, this looks good and similar enough to the internal structure that I hope to feed to Pagefind. Except your snippet does not include explicit titles for the parts, which I think would be necessary.

[bglw]

I just copied that snippet from the docs — you can safely ignore the meta/filters/sort keys as they're all optional :)

Except your snippet does not include explicit titles for the parts

Internally, Pagefind is non-hierarchical, so the example above would correspond to the HTML:

<p>Aenean eu leo quam. Pellentesque ornare sem lacinia quam venenatis vestibulum</p>
<h2 id="tristique-adipiscing">Tristique Adipiscing Nullam Vulputate Venenatis</h2>
<p>Maecenas sed diam eget risus varius blandit sit amet non magna</p>

That is, the content for a part with an anchor is inherently the title for that anchor, and a non-titled anchor would be added without content:

{ anchor: "tristique-adipiscing" }

The main thing is that anchors don't wrap content in Pagefind, they exist as markers in the flat content.

[documentat-alibaev]

Ok, got it. Looking forward to content_parts!

[documentat-alibaev]

Until content_parts are implemented, the best way to do what I want would be to wrap text into fake pages in runtime and pass them to addHTMLFile(), right?

[bglw]

Yeah, that would be the least fragile way to pipe things through as-is