Show HN: Chroma Cloud – serverless search database for AI

thank you!

High-level, Chroma lets you incorporate your own data into AI. That can be business data, like your help docs, or customer data, such as private documents.

I think there's a misconception among many non-technical people that they should "fine tune" models - to make an LLM sound like their brand or know their internal data. But, the best practice today is "context engineering" - to use a commodity LLM, and add propriety information into the prompt.

The hard part of context engineering is knowing which data to incorporate. For example, all help docs in every context creates context rot [1] that hurts accuracy, so you need a way to find which information to put in which context. The solution is essentially an internal search engine, which is Chroma.

Chroma gives AI multiple ways to search, and you can either hard-code which one works best for you - or let an agent write its own queries to conduct research by itself. Vector search is a generation ahead of old search infrastructure, and is useful for relatedness queries - like "help docs about billing". Full-text search is useful for proper nouns, like "Next.js". Regex search is useful for code and laws. Metadata search is more nuanced, but becomes really important in document search (e.g., PDFs). Chroma lets you run all of these search methods against private data, and you can use it to even include citations in results.

So, the high-level answer is: Chroma enables you to incorporate your business or customer data into AI.

[1] https://research.trychroma.com/context-rot

Not sure why this is voted down. The site doesn't really explain what problems of mine it'll solve (if any). I just came away thinking "oh, it's a thing. might have been a useful thing, who knows"

"Sell the sizzle, not the steak" is a real thing for a reason.

your customers might have heard of RAG (retrieval augmented generation) before. chroma powers the “R” in RAG. It enables language models to dynamically pull in information to help them answer questions and solve tasks.

can you tell me more about your use case?

Philip here from the Chroma engineering team.

Chroma supports multiple search methods - including vector, full-text, and regex search.

Four quick ways Chroma is different than pgvector: Better indexes, sharding, scaling, and object storage.

Chroma uses SPANN (Scalable Approximate Nearest Neighbor) and SPFresh (a freshness-aware ANN index). These are specialized algorithms not present in pgvector. [1].

The core issue with scaling vector database indexes is that they don't handle `WHERE` clauses efficiently like SQL. In SQL you can ask "select * from posts where organization_id=7" and the b-tree gives good performance. But, with vector databases - as the index size grows, not only does it get slower - it gets less accurate. Combining filtering with large indexes results in poor performance and accuracy.

The solution is to have many small indexes, which Chroma calls "Collections". So, instead of having all user data in one table - you shard across collections, which improves performance and accuracy.

The third issue with using SQL for vectors is that the vectors quickly become a scaling constraint for the database. Writes become slow due to consistency, disk becomes a majority vector indexes, and CPU becomes clogged by re-computing indexes constantly. I've been there and ultimately it hurts overall application performance for end-users. The solution for Chroma Cloud is a distributed system - which allows strong consistency, high-throughput of writes, and low-latency reads.

Finally, Chroma is built on object storage - vectors are stored on AWS S3. This allows cold + warm storage tiers, so that you can have minimal storage costs for cold data. This "scale to zero" property is especially important for multi-tenant applications that need to retain data for inactive users.

[1] https://www.youtube.com/watch?v=1QdwYWd3S1g

if you want or need to optimize for speed, cost, scalability or accuracy.

dedicated solutions have more advanced search features enable more accurate results. search indexing is resource intensive and can contend for resources with postgres/redis. the cost and speed benefits are naturally more pronounced as data volume scales.

for example - chroma has built in regex+trigram search and copy-on-write forking of indexes. this feature combo is killer for the code-search use case.