Added get multiple api for badger #1990
Conversation
harshil-goel
requested review from
akon-dey,
billprovince and
joshua-goldstein
as code owners
db.go
Outdated
| if vs.Meta == 0 && vs.Value == nil { | ||
| continue | ||
| } | ||
| // Found the required version of the key, return immediately. |
There was a problem hiding this comment.
This comment should be refactored, since we don't return immediately like in db.get()
discard_test.go
Outdated
|
|
||
| func TestReadC(t *testing.T) { | ||
| t.Skip() | ||
| allKeysF, err := os.Open("/home/harshil/all_keys_2") |
There was a problem hiding this comment.
Bit weird to have an explicit user path, make it a temp path or something?
There was a problem hiding this comment.
This s just a temp test for my local validation. Hence the t.Skip() before the test starts. I can't commit this as the directory being used is quite big.
|
|
||
| maxVs := make([]y.ValueStruct, len(keys)) | ||
|
|
||
| y.NumGetsAdd(db.opt.MetricsEnabled, 1) |
There was a problem hiding this comment.
If we are using the old metric, we should increment this by len(keys)
| @@ -756,6 +758,44 @@ func (db *DB) getMemTables() ([]*memTable, func()) { | |||
| // do that. For every get("fooX") call where X is the version, we will search | |||
| // for "fooX" in all the levels of the LSM tree. This is expensive but it | |||
| // removes the overhead of handling move keys completely. | |||
| func (db *DB) getBatch(keys [][]byte, done []bool) ([]y.ValueStruct, error) { | |||
There was a problem hiding this comment.
Nit: can we rename done to keyRead or something similar?
| } | ||
| } | ||
| return results, nil | ||
| } else { |
There was a problem hiding this comment.
@harshil-goel I need to understand this code block (lines 344-373) ... will sync up with you offline.
| // Find the table for which the key is in, and then seek it | ||
| getForKey := func(key []byte) (y.ValueStruct, func() error, []*table.Iterator) { | ||
| tables, decr := s.getTableForKey(key) | ||
| keyNoTs := y.ParseKey(key) |
There was a problem hiding this comment.
we should just parse the key once, we are doing it again and again
| for _, it := range itrs { | ||
| it.Seek(key) | ||
| if !it.Valid() { | ||
| continue |
✅ Deploy Preview for badger-docs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
|
|
This PR has been stale for 60 days and will be closed automatically in 7 days. Comment to keep it open. |
harshil-goel
force-pushed
the
harshil-goel/get-mutliple
branch
3 times, most recently
from
723e58f
to
d8a3d2b
Compare
harshil-goel
force-pushed
the
harshil-goel/get-mutliple
branch
from
d8a3d2b
to
7422181
Compare
Added get multiple api in badger.
Able to get upto 60% performance improvement in ldbc s10 dataset queries.
4.69 seconds -> 2.61 seconds