Show HN: Open-source Deep Research across workplace applications

Onyx is as close as to magic you can get in this space. It just. Works.

I can talk for literally hours about how good it is when you connect it to your company's Confluence or Jira or Slack or Google Drive or a ton of other things. At a scale of many tens of thousands of documents too.

Their team is awesome too and completely tuned into exactly what their users need. And that it's open source is the cherry on top. No secret in how your data is being used.

- An incredibly happy user looking forward to more from Onyx

The demo looked sharp but I am curious if you have done any formal evaluation of the quality of the results? For example, MRR and recall@k, even on a toy dataset? Seems like the quality of the generated responses will be highly dependent on the docs which are retrieved.

Very cool! One question: how do you handle permissions?

Different apps have different permissions models, not everyone is allowed to see everything. Do you attempt to model this complexity at all or normalize it to some general permissions model?

Before sharing how it works, I want to highlight some of the challenges of a system like this. Unlike deep research over the internet, LLMs aren’t able to easily leverage the built in searches of these SaaS applications. They each have different ways of searching for things, many do not have strong search capabilities, or they rely on their internal query language. There are also a ton of other signals that web search engines use that aren’t available natively in the tools. Examples include: backlinks, clickthrough rates, etc. Additionally a lot of teams rely on internal terminology that is unique to them and hard for the LLM to search for. There’s also the challenge of unifying the objects across all of the apps into a plaintext representation that works for the LLM.

The best way we’ve found to do this is to build a document index instead of relying on application native searches at query time. The document index is a hybrid index of keyword frequencies and vectors. The keyword component addresses issues like team specific terminology and the vector component allows for natural language queries and non-exact matching. Since all of the documents across the sources are processed prior to query time, inference is fast and all of the documents have already been mapped to an LLM friendly representation.

There are also other signals that we can take into account which are applied across all of the sources. For example, the time that a document was last updated is used to prioritize more recent documents. We also have models that run at indexing time to label documents and models that run at inference time to dynamically change the weights of the search function parameters.

I have recently found myself drawing parallels between deep research and knowledge working org structures.

For example,board exec asks senior exec a question about a particular product. The senior exec then has to fire off emails to say 5 managers, who might go down their tree to ICs, all the info is gathered and synthesised into a response.

Normally this response takes into account some angle the senior exec might have.

A lot of knowledge working tasks follow this pattern that I have somewhat simplified.

Cool product. Few Qs:

- What would you say is the agentic approach's special sauce over a typical RAG pipeline, ie query->multi-query generation->HyDE->vector search->bm25 search->RRF->rerank->evaluate->(retry|refuse|respond) that differentiates the approach?

- If a user has 20 services connected, how does the agent know how to call/search/traverse the information in the right order?

- Do you have any internal evals on how well the different model affect the overall quality of output, esp for a "deep search" type of task? I have model-picker fatigue.

- Do you plan to implement knowledge graphs in the future?