Skip to content

Commit b9990f4

Browse files
authored
Remove LRU cache from Alpha (#2837)
The LRU cache in Alpha has been causing a lot of contention. This PR removes the cache altogether, so posting lists are read directly from Badger as needed. Instead of one global LRU cache, each Txn object has a local cache, which is used both for mutations and for queries. Query functions share this local cache. Pros: We have seen an astounding 5-10x query latency drops due to this PR! I've also seen 1.7x jump in mutation throughput while running the live loader. Cons: This PR introduces a query local cache, which can potentially cause issues if a query requests for more data than the RAM available in the server. However, that can be dealt with in future PRs. The `lru_mb` flag has not been removed yet. We intend to use it later when we introduce an LRU cache in Badger. Changes: * Remove LRU cache. * Make query after mutation before commit work. Make tests work. * Return list from txn.Store * Fix the query package test failure. Also Txn should always use a local cache. * No need to commit in memory * Bring GetNoStore back * No need to track keys during rolling up. * Make LocalCache use byte slice instead of string as argument.
1 parent 787799b commit b9990f4

19 files changed

+171
-1323
lines changed

dgraph/docker-compose-single.yml

Lines changed: 28 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ services:
1313
- 6080:6080
1414
environment:
1515
- GOPATH=$GOPATH
16-
command: /gobin/dgraph zero --my=zero1:5080 --replicas 3 --idx 1 --bindall --expose_trace --profile_mode block --block_rate 10 --logtostderr -v=2 --jaeger.collector http://jaeger:14268
16+
command: /gobin/dgraph zero --my=zero1:5080 --replicas 3 --idx 1 --bindall --expose_trace --profile_mode block --block_rate 10 --logtostderr -v=2 # --jaeger.collector http://jaeger:14268
1717
volumes:
1818
- type: bind
1919
source: $GOPATH/bin
@@ -35,7 +35,7 @@ services:
3535
- 9999:9999
3636
security_opt:
3737
- seccomp:unconfined
38-
command: /gobin/dgraph alpha --my=dg1:7180 --lru_mb=1024 --zero=zero1:5080 --expose_trace -o 100 --trace 1.0 --profile_mode block --block_rate 10 --logtostderr -v=3 --jaeger.collector http://jaeger:14268
38+
command: /gobin/dgraph alpha --my=dg1:7180 --lru_mb=1024 --zero=zero1:5080 --expose_trace -o 100 --trace 1.0 --profile_mode block --block_rate 10 --logtostderr -v=3 # --jaeger.collector http://jaeger:14268
3939
#command: /gobin/dlv --listen=:9999 --headless=true --api-version=2 --init $GOPATH/src/github.com/dgraph-io/dgraph/dgraph/dlv.init exec /gobin/dgraph -- alpha --my=dg1:7180 --lru_mb=1024 --zero=zero1:5080 -o 100 --expose_trace --trace 1.0 --profile_mode block --block_rate 10 --logtostderr -v=3 --jaeger.collector http://jaeger:14268
4040

4141
jaeger:
@@ -53,34 +53,31 @@ services:
5353
environment:
5454
- COLLECTOR_ZIPKIN_HTTP_PORT=9411
5555

56-
node-exporter:
57-
image: quay.io/prometheus/node-exporter
58-
container_name: node-exporter
59-
pid: "host"
60-
volumes:
61-
- type: bind
62-
source: /
63-
target: /host
64-
read_only: true
65-
66-
prometheus:
67-
image: prom/prometheus
68-
container_name: prometheus
69-
hostname: prometheus
70-
ports:
71-
- "9090:9090"
72-
volumes:
73-
- type: bind
74-
source: $GOPATH/src/github.com/dgraph-io/dgraph/dgraph/prometheus.yml
75-
target: /etc/prometheus/prometheus.yml
76-
read_only: true
77-
78-
grafana:
79-
image: grafana/grafana
80-
container_name: grafana
81-
hostname: grafana
82-
ports:
83-
- "3000:3000"
56+
# node-exporter:
57+
# image: quay.io/prometheus/node-exporter
58+
# container_name: node-exporter
59+
# pid: "host"
60+
# volumes:
61+
# - type: bind
62+
# source: /
63+
# target: /host
64+
# read_only: true
8465

85-
66+
# prometheus:
67+
# image: prom/prometheus
68+
# container_name: prometheus
69+
# hostname: prometheus
70+
# ports:
71+
# - "9090:9090"
72+
# volumes:
73+
# - type: bind
74+
# source: $GOPATH/src/github.com/dgraph-io/dgraph/dgraph/prometheus.yml
75+
# target: /etc/prometheus/prometheus.yml
76+
# read_only: true
8677

78+
# grafana:
79+
# image: grafana/grafana
80+
# container_name: grafana
81+
# hostname: grafana
82+
# ports:
83+
# - "3000:3000"

0 commit comments

Comments
 (0)