How to Integrate Quepid Into Your Eval Workflow
Do you already have an Search Evaluation workflow in place, or parts of it that you want to preserve, but also take advantage of Quepid? There are three main approaches, from least itnegrated to most integrated:
- Export/Import Data Files between your existing solution and Quepid
- Leverage Quepid's APIs to integrate Quepid into your existing solution
- Wrap Quepid's data model directly into your solution
Export/Import Data Files between your existing solution and Quepid
Do you already have a fairly robust mature solution for doing Search Evaluation? Something homegrown, or maybe you are invested in OpenSearch's "Search Relevance Workbench"? Or you have been using SearchTweak?
Quepid supports exporting and importing data via CSV and JSON formats for almost all of the major data structures (Books, Cases, Users, Queries, Judgements). Yes, you will have to do some data format conversions, and it's a manual integration, but it's also supported in the UI and is pretty easy to use!
Leverage Quepid's APIs to integrate Quepid
Quepid has a robust set of API's that you can use to make a much deeper integration with Quepid. Yes, you have to learn a LOT more about how Quepid's data model fits together, but this also let's you make Quepid a integral part of your process. You can manage all the data structures using the API.
This is really great when you want to have that integration be an ongoing process. Say you run your evaluation process as part of a CI/CD pipeline, and you want to record the scores? Just make a call to the PUT /api/cases/:case_id/scores end point!
Maybe you do judgement calculations using your own implicit judgements pipeline? Do a PUT /api/books/:book_id/query_doc_pair/:query_doc_pair_id/judgements/:id to update an existing query doc pair with the new judgement value.
A community member has vibe coded (cool!) a comparision tool across three major public science journal search engines, and is leveraging the API's to store the comparison data over time, but surfacing all that in their own UI.
Wrap Quepid's data model directly into your solution
The most extreme form of integration is to integrate your application at the database level. There have been a number of examples of folks who have adopted the underlying data model in the database, but otherwise built their own framework around it. For example, see https://github.com/frutik/quepid-api-unofficial for an example.
You might also want to export data from Quepid's database into your own data lake to drive your own analytics dashboards.