How rqlite is tested

didip

I love how dedicated you are with this project, Philip. Been watching it for many many years.

fegu

There seems to be some copy pasta in the FAQ on if any node can be contacted for writes or reads, the paragraph on reads mentions writes.

otoolep

Thanks for flagging -- fixed.

the-alchemist

can't wait for the https://jepsen.io/ report!

otoolep

It's not done by Aphyr himself, but someone ran Jepsen-like testing a couple of years back.

https://github.com/wildarch/jepsen.rqlite/blob/main/doc/blog...

someguy101010

I also enjoyed this in video format https://youtu.be/JLlIAWjvHxM?feature=shared&t=2049

Always have been envious of that performance testing setup that is shown here

qaq

Anyone using rqlite in prod ?

otoolep

Yes, Replicated for example: https://www.replicated.com/blog/app-manager-with-rqlite

It's still in active use by them. https://www.textgroove.com/ is another company that uses it, though I know less about their use case.

qaq

Cool, rqlite looks like a really nice option for c2 storage layer

kopirgan

Thanks for the post...did not know about this.

Just went through the website and read through some documents. It is quite easy to read and understand.

One part I couldn't follow was security - other than items listed (file system, HTTPS, IP based filter), is it correct to say that if you know & have access to the URL API endpoints, any query can be run directly off the db with curl or such tools? How is this aspect managed in production? Sorry if this question is inappropriate or too dumb.

otoolep

rqlite supports a few levels security, which you can use separately or together.

You can enable BasicAuth on the HTTP endpoints. You can also require that any client presenting BasicAuth credentials has the right permission for the operation the client is trying to perform. Finally you can also enable Mutual TLS, requiring that any client that connects must first present a cert that is signed by an acceptable CA.

For full details on security check out https://rqlite.io/docs/guides/security/

null

[deleted]

denysvitali

In case you missed it, OP is also the author of rqlite

(I for one didn't notice at first)

otoolep

Yes, that is right. So here's my disclaimer: I'm the author of rqlite.

I really admire the testing that the SQLite team does[1], and the way it allows them to stand behind the statements they make about quality. It's inspiring.

IMO there have only been two really big improvements to software development relative to when I started programming professionally 25 years ago: 1) code reviews becoming mainstream, and 2) unit testing. (Perhaps Gen AI will be the third). I believe extensive testing is the only reason that rqlite continues to be developed to this day. It's not just that it helps keep the quality high, it's a key design guide. If a new module cannot be unit tested during development, in a straightforward manner, it's a strong sign one's decomposition of the problem is wrong.

[1] https://www.sqlite.org/testing.html

Ericson2314

Now that there is testing like https://turso.tech/blog/introducing-limbo-a-complete-rewrite... going on, tbh the testing described in the linked piece doesn't seem so good.

dangoodmanUT

Probably a lot better than 99.999% of software? This is made by one person, with code that cannot be deterministically tested (external SQLite) But you seem to be cargo culting, as this testing has existed in DBs for nearly a decade

otoolep

rqlite has been in development for about a decade too!

https://github.com/rqlite/rqlite/blob/master/CHANGELOG.md#10...

dangoodmanUT

kudos do you even just for sticking with it for that long, but I know it's well loved

IshKebab

It's not cargo culting. Deterministic Simulation Testing has been the standard for silicon verification practically forever (though it isn't called that because nobody does anything else).

Applying it to software is a logical step if you want similar levels of quality assurance.

Do you need that level of assurance for a database? Probably not in most cases. But I think it's a reasonable thing to mention on a post that is touting database reliability, especially because there are real databases that do use it.

ncruces

Then maybe point of one of those databases: https://foundationdb.org/ https://tigerbeetle.com/

Or if you really wanna keep the SQLite theme: https://github.com/losfair/mvsqlite

Why promote the vaporware project when comparing with something that's been in development for 10 years?

abrookewood

FYI this is a reference to Deterministic Simulation Testing (DST), which certainly sounds interesting: https://apple.github.io/foundationdb/testing.html

cynicalsecurity

1. Is this a one man project? Why? What if the author dies?

2. Why Go? Go is garbage collected, how is this even a good idea for a database engine in the first place?

dangoodmanUT

This is HN outrage bait if I've ever seen it

aabhay

Making good on the username, for sure!

ChocolateGod

> Why Go? Go is garbage collected, how is this even a good idea for a database engine in the first place?

First, the database itself is Sqlite, which is C and arguably the most widely deployed database engine in the galaxy.

Secondly, Go is popular for network services as it's easy to write (relative to languages like C++, Rust etc), is memory-safe and is "fast enough" for the network/disk to be your bottleneck and its concurrency features make it easy to use hardware efficiently without having to worry about scheduling in your application, among other reasons.

anilgulecha

> the most widely deployed database engine in the galaxy.

The planet perhaps. We don't really know at the galaxy level. At best we're atleast 200k years away from knowing for sure :)

/nitpick

shakna

NASA has used sqlite on a number of systems, so the reach is certainly beyond just the planet.