Skip to content

Commit

Permalink
Extend trace filtering to more operation types (#9335)
Browse files Browse the repository at this point in the history
Summary:
- Extended trace filtering to cover `MultiGet()`, `Seek()`, and `SeekForPrev()`. Now all user ops that can be traced support filtering.
- Enabled the new filter masks in `db_stress` since it only cares to trace writes.

Pull Request resolved: #9335

Test Plan:
- trace-heavy `db_stress` command reduced 30% elapsed time  (79.21 -> 55.47 seconds)

Benchmark command:
```
$ /usr/bin/time ./db_stress -ops_per_thread=100000 -sync_fault_injection=1 --db=/dev/shm/rocksdb_stress_db/ --expected_values_dir=/dev/shm/rocksdb_stress_expected/ --clear_column_family_one_in=0
```

- replay-heavy `db_stress` command reduced 12.4% elapsed time (23.69 -> 20.75 seconds)

Setup command:
```
$  ./db_stress -ops_per_thread=100000000 -sync_fault_injection=1 -db=/dev/shm/rocksdb_stress_db/ -expected_values_dir=/dev/shm/rocksdb_stress_expected --clear_column_family_one_in=0 & sleep 120; pkill -9 db_stress
```

Benchmark command:
```
$ /usr/bin/time ./db_stress -ops_per_thread=1 -reopen=0 -expected_values_dir=/dev/shm/rocksdb_stress_expected/ -db=/dev/shm/rocksdb_stress_db/ --clear_column_family_one_in=0 --destroy_db_initially=0
```

Reviewed By: zhichao-cao

Differential Revision: D33304580

Pulled By: ajkr

fbshipit-source-id: 0df10f87c1fc506e9484b6b42cea2ef96c7ecd65
  • Loading branch information
ajkr authored and facebook-github-bot committed Dec 28, 2021
1 parent 2e5f764 commit 2ee20a6
Show file tree
Hide file tree
Showing 4 changed files with 52 additions and 4 deletions.
4 changes: 4 additions & 0 deletions HISTORY.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
# Rocksdb Change Log
## Unreleased
### Public API change
* Added values to `TraceFilterType`: `kTraceFilterIteratorSeek`, `kTraceFilterIteratorSeekForPrev`, and `kTraceFilterMultiGet`. They can be set in `TraceOptions` to filter out the operation types after which they are named.

## 6.28.0 (2021-12-17)
### New Features
* Introduced 'CommitWithTimestamp' as a new tag. Currently, there is no API for user to trigger a write with this tag to the WAL. This is part of the efforts to support write-commited transactions with user-defined timestamps.
Expand Down
3 changes: 3 additions & 0 deletions db_stress_tool/expected_state.cc
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,9 @@ Status FileExpectedStateManager::SaveAtAndAfter(DB* db) {
if (s.ok()) {
TraceOptions trace_opts;
trace_opts.filter |= kTraceFilterGet;
trace_opts.filter |= kTraceFilterMultiGet;
trace_opts.filter |= kTraceFilterIteratorSeek;
trace_opts.filter |= kTraceFilterIteratorSeekForPrev;
s = db->StartTrace(trace_opts, std::move(trace_writer));
}

Expand Down
8 changes: 7 additions & 1 deletion include/rocksdb/options.h
Original file line number Diff line number Diff line change
Expand Up @@ -1851,7 +1851,13 @@ enum TraceFilterType : uint64_t {
// Do not trace the get operations
kTraceFilterGet = 0x1 << 0,
// Do not trace the write operations
kTraceFilterWrite = 0x1 << 1
kTraceFilterWrite = 0x1 << 1,
// Do not trace the `Iterator::Seek()` operations
kTraceFilterIteratorSeek = 0x1 << 2,
// Do not trace the `Iterator::SeekForPrev()` operations
kTraceFilterIteratorSeekForPrev = 0x1 << 3,
// Do not trace the `MultiGet()` operations
kTraceFilterMultiGet = 0x1 << 4,
};

// TraceOptions is used for StartTrace
Expand Down
41 changes: 38 additions & 3 deletions trace_replay/trace_replay.cc
Original file line number Diff line number Diff line change
Expand Up @@ -532,11 +532,46 @@ bool Tracer::ShouldSkipTrace(const TraceType& trace_type) {
if (IsTraceFileOverMax()) {
return true;
}
if ((trace_options_.filter & kTraceFilterGet && trace_type == kTraceGet) ||
(trace_options_.filter & kTraceFilterWrite &&
trace_type == kTraceWrite)) {

TraceFilterType filter_mask = kTraceFilterNone;
switch (trace_type) {
case kTraceNone:
case kTraceBegin:
case kTraceEnd:
filter_mask = kTraceFilterNone;
break;
case kTraceWrite:
filter_mask = kTraceFilterWrite;
break;
case kTraceGet:
filter_mask = kTraceFilterGet;
break;
case kTraceIteratorSeek:
filter_mask = kTraceFilterIteratorSeek;
break;
case kTraceIteratorSeekForPrev:
filter_mask = kTraceFilterIteratorSeekForPrev;
break;
case kBlockTraceIndexBlock:
case kBlockTraceFilterBlock:
case kBlockTraceDataBlock:
case kBlockTraceUncompressionDictBlock:
case kBlockTraceRangeDeletionBlock:
case kIOTracer:
filter_mask = kTraceFilterNone;
break;
case kTraceMultiGet:
filter_mask = kTraceFilterMultiGet;
break;
case kTraceMax:
assert(false);
filter_mask = kTraceFilterNone;
break;
}
if (filter_mask != kTraceFilterNone && trace_options_.filter & filter_mask) {
return true;
}

++trace_request_count_;
if (trace_request_count_ < trace_options_.sampling_frequency) {
return true;
Expand Down

0 comments on commit 2ee20a6

Please sign in to comment.