f53639af2f
A lot more to go, but this is the core. Need to think about how to test the queue handlers.
39 lines
817 B
Markdown
39 lines
817 B
Markdown
# grosbeak
|
|
|
|
*Nee Jemison*.
|
|
|
|
A single-file search engine, for small (100K-1M page) sites.
|
|
|
|
## why?
|
|
|
|
Two reasons.
|
|
|
|
* I would like to build a repository of sites and papers that is searchable. I want to own the content and the index, and I want it to be easy to deploy/self-host.
|
|
* I want a search backend for LLM-based applications where I control the content.
|
|
|
|
## requirements
|
|
|
|
* An S3-compatible object store
|
|
* SQlite or Postgres
|
|
|
|
The use-case is (essentially) single-user.
|
|
|
|
## API
|
|
|
|
### /fetch/<b64:URL>
|
|
|
|
## prereqs
|
|
|
|
* go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest
|
|
|
|
|
|
## packages used
|
|
|
|
* https://github.com/jpillora/go-tld
|
|
* https://pkg.go.dev/modernc.org/sqlite
|
|
* https://github.com/sqlc-dev/sqlc
|
|
* https://github.com/spf13/afero
|
|
* https://github.com/tidwall/gjson
|
|
|
|
//go:generate stringer -type=UpdateFrequency
|