Datadog acquires Quickwit

I love quickwit, unfortunately datadog has a history of murdering open source (e.g. vector.io halting development and never fixing gross bugs)

pg_search dev here -- Thanks for mentioning us.

Re: Tantivy. I'm hopeful the community Paul and the Quickwit team have built on top of Tantivy will continue to flourish. I'm sure Datadog will build product(s) with Quickwit, which is built on Tantivy and will contribute to it. Many other companies like ours (ParadeDB) and other databases also integrate it. I can't speak for others, but we'll contribute whenever possible. We're currently working on supporting nested documents in Tantivy, for example, and hoping to upstream this work.

While it's reasonable to be concerned, I'd say this is a win for Quickwit, Tantivy and, of course, the well-deserving team behind them.

Vector was already OSS when they acquired the company that created it, timber.

https://www.datadoghq.com/blog/datadog-acquires-timber-techn...

Yes, I've been using Vector since very early on, long before Datadog acquired it, and Datadog have continued ongoing maintenance and feature additions at a slow-but-steady pace, which I think is good. Like Quickwit, Vector is very stable and already quite complete. So I'm not too unhappy.

But Vector is something that complements Datadog's offering very well, so I think that makes sense for them to be good stewards of it. Quickwit is something that somewhat actively competes against them, which is a big difference. I suspect that unlike Vector, Quickwit is probably going to stop seeing any development in pretty short order, unless the devs now can consciously go out of their way to dedicate extra hours to it.

To be clear, I think that the relicense is great, and I think it's very possible that Quickwit will be picked up/forked by someone and maintenance will continue, because it's very good, and I'd really love to see someone do metrics for it as well. So, I'm not all gloomy or anything like that.

They bought Vector - it was always opensource

Please forgive and help remedy my ignorance: it's a coherent goal to want OrioleDB to be the go-to storage engine for Postgres, on Supabase?

Not hating on Quickwit, but almost never does an acquisition in the modern era mean continual innovation, most companies are now suborned to a greater purpose, and its almost never going to drive them to build the best thing they already have ended their lifecycle - nobody is going to buy them from DD and their quality/dev process will dominate, and that is of a decent size corporation.

It also looks like most of DD's observability acquisitions are either integrated directly (seemingly with a full rewrite) or look a lot like aquihires for senior folks, so I wouldn't hold my breath here.

They have to though, right? If you look at the nonsense people generate in logs, it would be a war crime to not monetize that. 20% of the PB's of data is probably the same 2,100 UTF-16 characters of Windows event id 4624.

Is that not the standard disruption play?

Depends, if you have a majority on the board you can do whatever you want. Some companies get a seed round to then use that to bootstrap to profitability and growth over time.

It’s going to depend on what the terms you agreed to are. In my experience, the company leadership are free to suggest an alternative to the hyper growth plan, but the board (which you likely won’t have a majority on) will reject it. VC’s don’t care about recouping money, they’d rather you fired everyone, pivoted, and started again and tried again with the money you have. They want successful exits, not break evens. It’s not about money it’s about reputation.

If you suggest something they don’t like, they’ll stall until you run out of money, or force you out.

Thanks for the clarification, and sorry for jumping to an incorrect conclusion based on vague wording. (I would edit my comment accordingly but I can't anymore.)

Is that a rhetorical question, or aimed at me specifically?

Regardless, in my case, it's not my first rodeo; I didn't give up any board seats or majority control. I mostly raised from angels that I know personally and trust me. And we're a high-margin B2B business (apparently similar in those limited respects to Quickwit) so I don't feel particularly capital constrained without raising additional rounds.

So I don't have a board pushing me for "valuation events." But I am very familiar with this toxic dynamic from my previous experience.

The product itself is very good, but the sales process is truly awful. Random calls with non-technical reps unable to answer basic questions like, "now that you've added this to my GCP account for 2 weeks, how much is this going to cost?" They'd say they're not sure but they have a startup deal with $xxxx minimum commit for 12 months gets you two months of extra trial, cancel anytime no questions asked. It's not just bad, it's comically bad.

Different person with similar stance: Those specific examples? Whatever.

We did get absolutely burnt by other manifestations of the DataDog approach: The billing model was (is?) very much not good, transparent or predictable and staying on top of costs was close to a nightmare. The way surprise costs and contract changes (triggered by them) was handled did not feel honest.

The product itself is great but from my perspective it's absolutely not worth having to deal with their business side of things and the risks, costs (money, time, attention) and stress associated.

If I were a Quickwit customer I'd start looking for alternatives.

It made me extremely distrustful of any and all interactions I would have with an employee. Is every email I send to my rep going to turn into an upsell? Are they being straight with me in answering my question?

For me, as much as it pains me to admit this, the sales and account relationship process is just as important of a factor now. I'm at a level where I'm not the end user of most of the infrastructure I purchase for the business, but I'm the one that has to deal with most of the vendor interaction.

Datadog is a pain in the ass. I've got two emails and a voicemail from them just this week. We are not an active customer.

Heroku/Salesforce is also a pain in the ass. It causes enough friction with legal that I'll spend whatever effort it takes to replatform our workload just to not have to have those unending inbound calls.

NS1 was easy-peasy, but post-IBM I now receive a PDF invoice for $50 once per month with no credit card-based billing options and have to remind finance to cut a paper check. I'll be rehoming our DNS as soon as we decide on where to move it to.

tl;dr: the business experience is part of the product

That is their product. You want to cancel? Send the army of clowns.

> there will be a valid open source competitor to Grafana ecosystems

You may want to check out SigNoz [1] - takes a all in one app approach compared to different modules for each signal approach which Grafana takes

Disclaimer : I am one of the maintainers

[1] https://github.com/SigNoz/signoz

Is Grafana and its ecosystem no longer open source? I haven't used it in a while

Loki has the following problems:

- It is non-trivial to setup and operate. It requires properly configuring and orchestrating numerous components - distributor, ingestor, querier, query frontend, compactor, consul, ruler, memcache, index service, etc. - see https://grafana.com/docs/loki/latest/get-started/architectur...

- Its' configs tend to break with every new release, since some configs become obsolete and some configs change names and/or structure.

- It is very slow on "needle in the haystack" queries such as "search for logs containing some rarely seen trace_id", since it needs to read, unpack and scan all the logs at object storage. This may be also very expensive if the object storage service charges for read IO.

- It doesn't support log fields with big number of unique values (aka high-cardinality log fields). It you'll try storing logs with high-cardinality fields into Loki, then it will quickly explode with enormous RAM usage. Recent Loki releases provide half-baked solution for this problem - "structured metadata" ( https://grafana.com/docs/loki/latest/get-started/labels/stru... ), but it is still experimental and requires non-trivial configs (aka it doesn't work).

Just don’t look at the list of open issues and the lack of responses.

Loki is significantly different in several ways, including the lack of full-text indexing.

what another open source you can recommend ?

We switched from Datadog to Grafana (do not recommend unless they got you over a barrel on pricing and you need to escape) and one nice thing Grafana gives you is the ability to self-host for local development so you can even run integration tests against your observability... an edge case need but if you need it you're glad it has it.

>it looks like they're hinting towards offering a 'self-hosted' model

That makes sense. Datadog has been pure SaaS the whole time, which is unusual. Buying a good db engine like Quickwit would be a smart head-start into the on-prem segment which is a natural expansion opportunity.

I've previously made the prediction that Datadog is the new Cisco - can expect lots of acquisitions to be made going forward.

(Disclaimer: also datadog employee)

I joined Datadog after the Vector acquisition and now currently am the manager for the Community Open Source Engineering team that works on Vector open source.

Just confirming strongly what ripley12 said, as a person with direct involvement in OSS at Datadog.

(disclaimer: Datadog employee)

I haven't kept super close tabs on it but last year we were hiring for a role to do tech lead stuff and OSS community building for Vector, and yes several of the original Vector employees still work here.

The architecture of Husky seems fairly similar to quickwit. https://www.datadoghq.com/blog/engineering/introducing-husky... https://quickwit.io/docs/main-branch/overview/architecture

And then I'm sure they'll pass the savings onto customers given the current crazy high prices.

The same thing? The community will maintain it

[dead]