Launch HN: Exa (YC S21) – The web as a database

I searched for 'data providers that start with the letter R that sell job postings data', and it's been 15 minutes and it barely verified the first row.

But if it filtered it first to "start with the letter R", it would only have to look at perhaps 5% of the results it's trying to verify!

So it's doing needless verification of results that will be thrown out by another filter that should've been applied first!

This is super cool! It took a while, but did a great job of evaluating the results, and the airtable-like results UI feels great.

Congrats on your launch. With the natural way this lends itself to comparison shopping this is an amazing tool for people trying to find "the best X for me" whether that's a TV, a school, etc. So much content that you find on Google when trying to answer that type of query, is designed to trick, bamboozle, and to hide the facts that you might use to answer this question (but most of all to get you to click affiliate links).

I was so excited for this, but sadly it doesn't work at all, not even UI feedback for the error:(

The UI showed literally no change. So I checked and the console shows:

``` Try: 14 Not Found 681-7df1b139fa2dc9f0.js:14:3379 Try: 15 Not Found 681-7df1b139fa2dc9f0.js:14:3379 Try: 16 Not Found 681-7df1b139fa2dc9f0.js:14:3379 Try: 17 Not Found 681-7df1b139fa2dc9f0.js:14:3379 Try: 18 Not Found 681-7df1b139fa2dc9f0.js:14:3379 Try: 19 Not Found 681-7df1b139fa2dc9f0.js:14:3379 Try: 20 Not Found 681-7df1b139fa2dc9f0.js:14:3379 Gave up after 10 seconds. 681-7df1b139fa2dc9f0.js:14:3379 filteredSuggestions Array(3) [ {…}, {…}, {…} ] 681-7df1b139fa2dc9f0.js:14:3379 ```

Also your table doesn't fit in the viewport so I can't see the results.

Firefox Ubuntu.

I think it might be a good idea to give some kind of indication that work is being done in the background (or perhaps mine stalled out?).

The initial search/experience is good but then I got dumped here [0] and it's not clear to me if things are still happening or if it broke (it's been at least 5 min with no UI updates.

I can't see the full results yet but this is very interesting and a task I ask OpenAI's Deep Research to attempt periodically. It makes a good show of doing the work but the results are not great IMHO (for asking it generate lists/tables of data like this). I can see this tool being incredibly useful for lead generation (how I am testing it out).

[0] https://cs.joshstrange.com/dySqK1mb

I think you guys nailed the "selling shovels during a gold rush" as the biggest issue with LLMs currently is their reliability/hallucinations, not their capabilities. If I can use websets to back up LLM responses through your API, that's super useful.

Since you were part of YC 21, could you share a bit about your pivots/product iterations you went through over the last 4 years?

Did my favorite search query, and the result were pretty bad, as expected:

"robotics servo motors with two-directional control for under $100"

1. https://mjbots.com/ - their motor are $1369. FAIL.

2. https://www.pololu.com/ - this is huge store, but they do have some motors like that. Pass, but wish it linked to specific page and not top top-level one.

3. dh-robotics.com - no prices, but some products on open market are few K$. Likely fail as well.

4. https://www.robotarticulation.com/ - The product is not for sale (early beta), and it looks likely much more than $1K. FAIL.

5. https://www.lynxmotion.com/ - another huge store, most two-directional motors are expensive but there are some under $100... Pass, but wish it linked to specific page and not top top-level one.

This is super cool. You provide examples of “searches that work” - can you give an idea of the limitations here? What kind of searches won’t work?

Hey! Congrats on the launch. I just signed up for a trial account and I’m pretty impressed with the search API (haven’t used websets yet but looks cool).

Our experimental use case is enabling quick and dirty integration of web-based docs into an employee service agentic chatbot - lots of the questions are around “how do I max out my 401k”, which connects to internal information, but some are more like “how do I link a calendar to calendly”.

The one thing I’d love to have in the search product is a cruft cleaner for the results of web queries. Where you have cached the data presumably this wouldn’t add much overhead. Reduces what you have to feed to the LLM downstream and might improve the embeddings performance.

AI crawlers have lead to a big surge in scraping/crawling activity on the web, and many don't use proper user agents and don't stick to any scraping best practices that the industry has developed over the past two decades (robots.txt, rate limits). This comes with negative side effects for website owners (costs, downtime, etc.), as repeatedly reported on HN (and experienced myself).

Do you have any built-in features that address these issues?

The Exa LinkedIn webset is something very innovative. Many current providers make it difficult if not against "Terms of Service" to build a product using their data. The irony is that they simply scraped LinkedIn.

So the crawlers are feeding to database and also something is classifying the data stream and organizing the data and everything is open as a very large dataset. This is an interesting concept.

Congrats on the launch!

How do you dedupe entities, like companies and people? I've noticed ChatGPT tends to provide "great" results when asking about different entities, but in reality it just groups similar sounding entities together in its answer.

For example, I asked ChatGPT about a well known startup. It gave me a confident answer about how much they raised, their current status, etc. When looking at the 3 sources they cited though, it was actually 3 different companies that all had similar sounding names that it just grouped together to form its answer.

Basically, how do I trust the output of your system?