Show HN: voidDB – A transactional key-value DB written in Go for 64-bit Linux

fasteo

  // voidDB is a cherished toy, a journey into the Unknown, a heroic struggle,
  // and a work of love. It is the “Twee!” of a bird; a tree falling in the
  // forest; yet another programmer pouring their drop into the proverbial [bit]
  // bucket. Above all, it is a shrine unto simple, readable, and functional
  // code; an assertion that the dichotomy between such aesthetics and practical
  // performance is mere illusion.

[1] https://github.com/voidDB/voidDB/blob/ac104867749e2c698b3d61...

kunley

Love the poetry in the comment prose! Endeavour so much needed in the industry. It can only come out of the joy of coding, how else could it be born...

jakewins

It always made me happy to hack nearby the Neo4j page cache "Muninn", because it meant I'd pass by Chris Vest's poem and giant ascii Raven: https://github.com/neo4j/neo4j/blob/release/5.26.0/community...

srameshc

I admire the author who took up the challenge of writing a KV DB and cherishing self's efforts. I hope someday I could do something similar and feel accomplished.

joel-ling

Seemingly faster and more compact than lmdb-go, bbolt, Badger, and goleveldb at <2K LOC.

  goos: linux
  goarch: arm64
  pkg: github.com/voidDB/voidDB/test
  BenchmarkVoidPut-2            131072      14933 ns/op
  BenchmarkVoidGet-2            131072       1060 ns/op
  BenchmarkVoidGetNext-2        131072        245.8 ns/op
  BenchmarkLMDBPut-2            131072      22414 ns/op
  BenchmarkLMDBGet-2            131072       1826 ns/op
  BenchmarkLMDBGetNext-2        131072        602.2 ns/op
  BenchmarkBoltPut-2            131072      66984 ns/op
  BenchmarkBoltGet-2            131072       2552 ns/op
  BenchmarkBoltGetNext-2        131072        254.6 ns/op
  BenchmarkLevelPut-2           131072      44182 ns/op
  BenchmarkLevelGet-2           131072      30949 ns/op
  BenchmarkLevelGetNext-2       131072       3441 ns/op
  BenchmarkBadgerPut-2          131072      15182 ns/op
  BenchmarkBadgerGet-2          131072      33114 ns/op
  BenchmarkBadgerGetNext-2      131072      12895 ns/op
  BenchmarkNothing-2            131072          0.3239 ns/op

Check out the docs at https://pkg.go.dev/github.com/voidDB/voidDB

KAdot

The benchmark doesn't accurately represent the real-world database performance because the dataset is too small (roughly half a gigabyte based on [1]?), meaning it fits into the page cache bypassing disk I/O.

[1]: https://github.com/voidDB/voidDB/blob/master/test/bench_test...

chronode

This is a cool effort but their claims are wildly misleading as their benchmarks aren't fair across the board:

For example:

- BenchmarkVoidPut runs a single O(n) sized transaction

- BenchmarkLMDBPut runs a single O(n) sized transaction

- BenchmarkBadgetPut runs O(n) O(n) sized transactions (!!!)

I made a local change so that all Put benchmarks ran O(n) O(1) sized transactions and the results were quite different: Void was the slowest, followed by LMDB, Bold, LevelDB, then Badger.

I'd also wager the LMDB author would also (lovingly!) tell us we're holding it wrong

sirjaz

Since there are no strict OS dependency I can see, has anyone tried to compile this and run it on Windows?

sushidev

Faster than rocksdb?

dangoodmanUT

probably not if it's LMDB-based (at least for writes)

seems pretty bloated

https://github.com/voidDB/voidDB/blob/master/go.sum

seanw444

A lot of it is indirect dependencies, and most of the direct dependencies in the go.mod are just for benchmarks, as far as I can tell.

maxmcd

Only "x/sys/unix" outside of tests/benchmarks

      ~/go/src/github.com/voidDB/voidDB  git:(master)
      $ go run github.com/tailscale/depaware@latest
      github.com/voidDB/voidDB dependencies: (generated by github.com/tailscale/depaware)

            github.com/voidDB/voidDB/common                              from github.com/voidDB/voidDB+
            github.com/voidDB/voidDB/cursor                              from github.com/voidDB/voidDB
            github.com/voidDB/voidDB/free                                from github.com/voidDB/voidDB
            github.com/voidDB/voidDB/node                                from github.com/voidDB/voidDB+
            github.com/voidDB/voidDB/reader                              from github.com/voidDB/voidDB
            golang.org/x/sys/unix                                        from github.com/voidDB/voidDB+
            bytes                                                        from github.com/voidDB/voidDB+
            cmp                                                          from internal/fmtsort+
            encoding/binary                                              from github.com/voidDB/voidDB/common+
            errors                                                       from bytes+
      D    fmt                                                          from golang.org/x/sys/unix
            hash                                                         from github.com/voidDB/voidDB+
            hash/fnv                                                     from github.com/voidDB/voidDB
            io                                                           from bytes+
            io/fs                                                        from internal/filepathlite+
            iter                                                         from reflect+
            math                                                         from encoding/binary+
            math/bits                                                    from golang.org/x/sys/unix+
            os                                                           from fmt+
            path                                                         from io/fs
            reflect                                                      from encoding/binary+
            slices                                                       from encoding/binary+
      LD    sort                                                         from golang.org/x/sys/unix
            strconv                                                      from fmt+
      LD    strings                                                      from golang.org/x/sys/unix
            sync                                                         from encoding/binary+
            sync/atomic                                                  from internal/bisect+
            syscall                                                      from github.com/voidDB/voidDB/cursor+
            time                                                         from github.com/voidDB/voidDB+
            unicode                                                      from bytes+
      W    unicode/utf16                                                from internal/poll+
            unicode/utf8                                                 from bytes+