Prometheus integration with Redis Cloud
To collect and display metrics data from your databases, you can connect your Prometheus or Grafana server to your Redis Cloud subscription.
- Prometheus is an open source systems monitoring and alerting toolkit that can scrape metrics from different sources.
- Grafana is an open source metrics dashboard and graph editor that can process Prometheus data.
Redis Cloud has an Prometheus compatible endpoint available in order to pull metrics. Prometheus needs to connect to the internal server on port 8070. This is only available on the internal network so VPC peering is required. VPC peering is only available with Flexible or Annual subscriptions. Because VPC peering is not available on Fixed or Free subscriptions, Prometheus and Grafana cannot connect to databases on Fixed or Free subscriptions.
For more information on how Prometheus communicates with Redis Enterprise clusters, see Prometheus integration with Redis Enterprise Software.
Quick start
You can quickly set up Prometheus and Grafana for testing using the Prometheus and Grafana Docker images.
Prerequisites
-
Create a Flexible or Annual subscription with a database.
-
Set up VPC peering for your subscription.
-
Extract the Prometheus endpoint from the private endpoint to your database. The private endpoint is in the Redis Cloud console under the Configuration tab of your database. The Prometheus endpoint is on port 8070 of the internal server.
For example, if your private endpoint is:
redis-12345.internal.<cluster_address>:12345
The Prometheus endpoint is:
internal.<cluster_address>:8070
-
Create an instance to run Prometheus and Grafana on the same provider as your Redis Cloud subscription (Amazon Web Services or Google Cloud Project). This instance must:
- Exist in the same region as your Redis Cloud subscription.
- Connect to the VPC subnet that is peered with your Redis Cloud subscription.
- Allow outbound connections to port 8070, so that Prometheus can scrape the Redis Cloud server for data.
- Allow inbound connections to port 9090 for Prometheus and 3000 for Grafana.
Set up Prometheus
To get started with custom monitoring with Prometheus on Docker:
-
Create a directory on the Prometheus instance called
prometheus
and create aprometheus.yml
file in that directory. -
Add the following contents to
prometheus.yml
. Replace<instance_ip>
with the IP address of the instance and replace<prometheus_endpoint>
with the Prometheus endpoint.global: scrape_interval: 15s evaluation_interval: 15s # Attach these labels to any time series or alerts when communicating with # external systems (federation, remote storage, Alertmanager). external_labels: monitor: "prometheus-stack-monitor" # Load and evaluate rules in this file every 'evaluation_interval' seconds. #rule_files: # - "first.rules" # - "second.rules" scrape_configs: # scrape Prometheus itself - job_name: prometheus scrape_interval: 10s scrape_timeout: 5s static_configs: - targets: ["<instance_ip>:9090"] # scrape Redis Cloud - job_name: redis-cloud scrape_interval: 30s scrape_timeout: 30s metrics_path: / scheme: https static_configs: - targets: ["<prometheus_endpoint>:8070"]
-
Create a
docker-compose.yml
file with instructions to set up the Prometheus and Grafana Docker images.version: '3' services: prometheus-server: image: prom/prometheus ports: - 9090:9090 volumes: - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml grafana-ui: image: grafana/grafana ports: - 3000:3000 environment: - GF_SECURITY_ADMIN_PASSWORD=secret links: - prometheus-server:prometheus
-
To start the containers, run:
$ docker compose up -d
-
To check that all the containers are up, run:
docker ps
-
In your browser, sign in to Prometheus at
http://<instance_ip>:9090
to make sure the server is running. -
Select Status and then Targets to check that Prometheus is collecting data from the Redis Cloud cluster.
If Prometheus is connected to the cluster, you can type node_up in the Expression field on the Prometheus home page to see the cluster metrics.
See Prometheus Metrics for a list of metrics that Prometheus collects from Redis Enterprise clusters.
Set up Grafana
Once the Prometheus and Grafana Docker containers are running, and Prometheus is connected to your Redis Cloud subscription, you can set up your Grafana dashboards.
-
Sign in to Grafana. If you installed Grafana with Docker, go to
http://<instance_ip>:3000
and sign in with:- Username:
admin
- Password:
secret
- Username:
-
In the Grafana configuration menu, select Data Sources.
-
Select Add data source.
-
Select Prometheus from the list of data source types.
-
Enter the Prometheus information:
- Name:
redis-cloud
- URL:
http://<your prometheus address>:9090
- Access:
Server
Note:- If the network port is not accessible to the Grafana server, select the Browser option from the Access menu.
- In a testing environment, you can select Skip TLS verification.
- Name:
-
Add dashboards for cluster, node, and database metrics. To add preconfigured dashboards:
- In the Grafana dashboards menu, select Manage.
- Select Import.
- Copy and enter one of the following configuration files into the Paste JSON field.
database.json
{ "__inputs": [ { "name": "DS_PROMETHEUS1", "label": "Prometheus1", "description": "", "type": "datasource", "pluginId": "prometheus", "pluginName": "Prometheus" } ], "__requires": [ { "type": "grafana", "id": "grafana", "name": "Grafana", "version": "4.4.0-pre1" }, { "type": "panel", "id": "graph", "name": "Graph", "version": "" }, { "type": "datasource", "id": "prometheus", "name": "Prometheus", "version": "1.0.0" }, { "type": "panel", "id": "singlestat", "name": "Singlestat", "version": "" }, { "type": "panel", "id": "text", "name": "Text", "version": "" } ], "annotations": { "list": [] }, "editable": true, "gnetId": null, "graphTooltip": 1, "hideControls": false, "id": null, "links": [ { "icon": "external link", "includeVars": true, "keepTime": true, "tags": [ "RLEC" ], "targetBlank": true, "type": "dashboards" } ], "refresh": false, "rows": [ { "collapse": false, "height": 121, "panels": [ { "content": "<b style=\"font-size: 20pt\">$status<b>\n<b style=\"font-size: 20pt\">$newStatus<b>", "editable": true, "error": false, "id": 9, "links": [], "mode": "html", "span": 2, "style": { "font-size": "72pt" }, "title": "Status for BDB:$bdb", "type": "text" }, { "cacheTimeout": null, "colorBackground": false, "colorValue": false, "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "datasource": "${DS_PROMETHEUS1}", "decimals": null, "editable": true, "error": false, "format": "bytes", "gauge": { "maxValue": 100, "minValue": 0, "show": false, "thresholdLabels": false, "thresholdMarkers": true }, "id": 4, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "nullPointMode": "connected", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "repeat": null, "span": 2, "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": true }, "tableColumn": "", "targets": [ { "expr": "scalar(bdb_used_memory{bdb=\"$bdb\", cluster=\"$cluster\"})", "format": "time_series", "intervalFactor": 2, "legendFormat": "", "refId": "A", "step": 14400 } ], "thresholds": "", "title": "BDB:$bdb used memory", "type": "singlestat", "valueFontSize": "80%", "valueMaps": [ { "op": "=", "text": "N/A", "value": "null" } ], "valueName": "current" }, { "cacheTimeout": null, "colorBackground": false, "colorValue": false, "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "datasource": "${DS_PROMETHEUS1}", "editable": true, "error": false, "format": "none", "gauge": { "maxValue": 100, "minValue": 0, "show": false, "thresholdLabels": false, "thresholdMarkers": true }, "id": 5, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "nullPointMode": "connected", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "span": 2, "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": true }, "tableColumn": "", "targets": [ { "expr": "scalar(bdb_no_of_keys{bdb=\"$bdb\", cluster=\"$cluster\"})", "format": "time_series", "intervalFactor": 2, "legendFormat": "", "refId": "A", "step": 14400 } ], "thresholds": "", "title": "BDB:$bdb #Keys", "type": "singlestat", "valueFontSize": "80%", "valueMaps": [ { "op": "=", "text": "N/A", "value": "null" } ], "valueName": "current" }, { "cacheTimeout": null, "colorBackground": false, "colorValue": false, "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "datasource": "${DS_PROMETHEUS1}", "editable": true, "error": false, "format": "none", "gauge": { "maxValue": 100, "minValue": 0, "show": false, "thresholdLabels": false, "thresholdMarkers": true }, "id": 6, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "nullPointMode": "connected", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "span": 2, "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": true }, "tableColumn": "", "targets": [ { "expr": "scalar(bdb_conns{bdb=\"$bdb\", cluster=\"$cluster\"})", "format": "time_series", "intervalFactor": 2, "legendFormat": "", "refId": "A", "step": 14400 } ], "thresholds": "", "title": "BDB:$bdb #Connections", "type": "singlestat", "valueFontSize": "80%", "valueMaps": [ { "op": "=", "text": "N/A", "value": "null" } ], "valueName": "current" }, { "cacheTimeout": null, "colorBackground": false, "colorValue": false, "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "datasource": "${DS_PROMETHEUS1}", "editable": true, "error": false, "format": "none", "gauge": { "maxValue": 100, "minValue": 0, "show": false, "thresholdLabels": false, "thresholdMarkers": true }, "id": 7, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "nullPointMode": "connected", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "span": 2, "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": false }, "tableColumn": "", "targets": [ { "expr": "count(listener_total_req{bdb=\"$bdb\", cluster=\"$cluster\"})", "format": "time_series", "intervalFactor": 2, "legendFormat": "", "refId": "A", "step": 14400 } ], "thresholds": "", "title": "BDB:$bdb #Listeners", "type": "singlestat", "valueFontSize": "80%", "valueMaps": [ { "op": "=", "text": "N/A", "value": "null" } ], "valueName": "current" }, { "cacheTimeout": null, "colorBackground": false, "colorValue": false, "colors": [ "rgba(245, 54, 54, 0.9)", "rgba(237, 129, 40, 0.89)", "rgba(50, 172, 45, 0.97)" ], "datasource": "${DS_PROMETHEUS1}", "format": "none", "gauge": { "maxValue": 100, "minValue": 0, "show": false, "thresholdLabels": false, "thresholdMarkers": true }, "id": 12, "interval": null, "links": [], "mappingType": 1, "mappingTypes": [ { "name": "value to text", "value": 1 }, { "name": "range to text", "value": 2 } ], "maxDataPoints": 100, "nullPointMode": "connected", "nullText": null, "postfix": "", "postfixFontSize": "50%", "prefix": "", "prefixFontSize": "50%", "rangeMaps": [ { "from": "null", "text": "N/A", "to": "null" } ], "span": 2, "sparkline": { "fillColor": "rgba(31, 118, 189, 0.18)", "full": false, "lineColor": "rgb(31, 120, 193)", "show": true }, "tableColumn": "", "targets": [ { "expr": "count(redis_no_of_keys{cluster=\"$cluster\", bdb=\"$bdb\"}) or count(redis_used_memory{cluster=\"$cluster\", bdb=\"$bdb\"})", "format": "time_series", "intervalFactor": 2, "legendFormat": "", "refId": "A", "step": 14400 } ], "thresholds": "", "title": "BDB:$bdb #Shards", "type": "singlestat", "valueFontSize": "80%", "valueMaps": [ { "op": "=", "text": "N/A", "value": "null" } ], "valueName": "current" } ], "repeat": null, "repeatIteration": null, "repeatRowId": null, "showTitle": false, "title": "Dashboard Row", "titleSize": "h6" }, { "collapse": false, "height": 258, "panels": [ { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 1, "legend": { "alignAsTable": true, "avg": true, "current": true, "max": true, "min": true, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [ { "alias": "/.*max.*/", "dashes": true, "fill": 0 } ], "spaceLength": 10, "span": 4, "stack": false, "steppedLine": false, "targets": [ { "expr": "bdb_avg_latency{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "hide": false, "intervalFactor": 2, "legendFormat": "bdb_avg_latency", "metric": "", "refId": "A", "step": 2400 }, { "expr": "bdb_avg_write_latency{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "hide": false, "intervalFactor": 2, "legendFormat": "bdb_avg_write_latency", "metric": "", "refId": "B", "step": 2400 }, { "expr": "bdb_avg_read_latency{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "hide": false, "intervalFactor": 2, "legendFormat": "bdb_avg_read_latency", "metric": "", "refId": "C", "step": 2400 }, { "expr": "bdb_avg_other_latency{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "hide": false, "intervalFactor": 2, "legendFormat": "bdb_avg_other_latency", "metric": "", "refId": "D", "step": 2400 }, { "expr": "max_over_time(bdb_avg_latency_max{bdb=\"$bdb\", cluster=\"$cluster\"}[$aggregation])", "format": "time_series", "interval": "", "intervalFactor": 1, "legendFormat": "bdb_avg_latency (max over $aggregation)", "refId": "E", "step": 1200 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Avg latency for BDB:$bdb", "tooltip": { "msResolution": false, "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "s", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 55, "legend": { "alignAsTable": true, "avg": true, "current": true, "max": true, "min": true, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [ { "alias": "/.*max.*/", "dashes": true, "fill": 0 } ], "spaceLength": 10, "span": 4, "stack": false, "steppedLine": false, "targets": [ { "expr": "listener_acc_latency{bdb=~\"$bdb.*\", cluster=\"$cluster\"} / listener_total_started_res{bdb=~\"$bdb.*\", cluster=\"$cluster\"}", "format": "time_series", "hide": false, "interval": "", "intervalFactor": 2, "legendFormat": "listener {{proxy}} node {{node}}", "metric": "", "refId": "A", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Avg listener latency for BDB:$bdb", "tooltip": { "msResolution": false, "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "s", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "description": "", "fill": 1, "id": 44, "legend": { "alignAsTable": true, "avg": false, "current": true, "max": true, "min": false, "show": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [ { "alias": "/.*limit.*/", "dashes": true, "fill": 0 } ], "spaceLength": 10, "span": 4, "stack": false, "steppedLine": false, "targets": [ { "expr": "bdb_used_memory{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "used_memory", "metric": "", "refId": "A", "step": 2400 }, { "expr": "bdb_memory_limit{bdb=\"$bdb\", cluster=\"$cluster\"} < bdb_used_memory{bdb=\"$bdb\", cluster=\"$cluster\"} * 1.5", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "limit (when low, near used)", "refId": "B", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Used Memory for BDB:$bdb", "tooltip": { "msResolution": false, "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "bytes", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] } ], "repeat": null, "repeatIteration": null, "repeatRowId": null, "showTitle": false, "title": "Dashboard Row", "titleSize": "h6" }, { "collapse": false, "height": 306, "panels": [ { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 10, "legend": { "alignAsTable": true, "avg": true, "current": true, "max": true, "min": true, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 4, "stack": false, "steppedLine": false, "targets": [ { "expr": "bdb_conns{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "Total", "metric": "", "refId": "A", "step": 2400 }, { "expr": "listener_conns{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "endpoint {{proxy}} node: {{node}}", "metric": "", "refId": "B", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "BDB:$bdb Connections", "tooltip": { "msResolution": false, "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "none", "label": null, "logBase": 1, "max": null, "min": "0", "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 14, "legend": { "alignAsTable": true, "avg": true, "current": true, "max": true, "min": true, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [ { "alias": "/.*max.*/", "dashes": true, "fill": 0 } ], "spaceLength": 10, "span": 4, "stack": false, "steppedLine": false, "targets": [ { "expr": "bdb_total_req{cluster=\"$cluster\",bdb=\"$bdb\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "total requests", "refId": "A", "step": 2400 }, { "expr": "bdb_other_req{cluster=\"$cluster\",bdb=\"$bdb\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "other requests", "refId": "B", "step": 2400 }, { "expr": "bdb_read_req{cluster=\"$cluster\",bdb=\"$bdb\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "read requests", "refId": "C", "step": 2400 }, { "expr": "bdb_write_req{cluster=\"$cluster\",bdb=\"$bdb\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "write requests", "refId": "D", "step": 2400 }, { "expr": "bdb_total_req_max{cluster=\"$cluster\",bdb=\"$bdb\"}", "format": "time_series", "hide": true, "interval": "", "intervalFactor": 2, "legendFormat": "total requests (max)", "refId": "E", "step": 240 }, { "expr": "max_over_time(bdb_total_req_max{cluster=\"$cluster\",bdb=\"$bdb\"}[$aggregation])\n", "format": "time_series", "hide": false, "interval": "", "intervalFactor": 2, "legendFormat": "total requests (max over $aggregation)", "refId": "F", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Requests[total,read,write,other] for bdb:$bdb", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "ops", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 16, "legend": { "alignAsTable": true, "avg": true, "current": true, "max": true, "min": true, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [ { "alias": "Egress", "yaxis": 2 } ], "spaceLength": 10, "span": 4, "stack": false, "steppedLine": false, "targets": [ { "expr": "bdb_egress_bytes{bdb=\"$bdb\",cluster=\"$cluster\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "Egress", "refId": "A", "step": 2400 }, { "expr": "bdb_ingress_bytes{bdb=\"$bdb\",cluster=\"$cluster\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "Ingress", "refId": "B", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "BDB:$bdb Ingress/Egress", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "Bps", "label": "Ingress", "logBase": 1, "max": null, "min": null, "show": true }, { "format": "Bps", "label": "Egress", "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 43, "legend": { "alignAsTable": false, "avg": false, "current": false, "max": false, "min": false, "show": true, "total": false, "values": false }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 4, "stack": false, "steppedLine": false, "targets": [ { "expr": "bdb_main_thread_cpu_system{bdb=\"$bdb\",cluster=\"$cluster\"} + bdb_main_thread_cpu_user{bdb=\"$bdb\",cluster=\"$cluster\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "main threads (masters)", "metric": "", "refId": "A", "step": 2400 }, { "expr": "bdb_shard_cpu_system{bdb=\"$bdb\",cluster=\"$cluster\"} + bdb_shard_cpu_user{bdb=\"$bdb\",cluster=\"$cluster\"}", "format": "time_series", "hide": false, "interval": "", "intervalFactor": 2, "legendFormat": "all threads (masters)", "refId": "B", "step": 2400 }, { "expr": "bdb_fork_cpu_system{bdb=\"$bdb\",cluster=\"$cluster\"} + bdb_fork_cpu_user{bdb=\"$bdb\",cluster=\"$cluster\"}", "format": "time_series", "hide": false, "interval": "", "intervalFactor": 2, "legendFormat": "all forks (masters)", "refId": "C", "step": 2400 }, { "expr": "bdb_main_thread_cpu_system_max{bdb=\"$bdb\",cluster=\"$cluster\"} + bdb_main_thread_cpu_user_max{bdb=\"$bdb\",cluster=\"$cluster\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "main threads max (masters)", "refId": "D", "step": 2400 }, { "expr": "bdb_shard_cpu_system_max{bdb=\"$bdb\",cluster=\"$cluster\"} + bdb_shard_cpu_user_max{bdb=\"$bdb\",cluster=\"$cluster\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "all threads max (masters)", "refId": "E", "step": 2400 }, { "expr": "sum((delta(redis_process_main_thread_cpu_system_seconds_total{bdb=\"$bdb\",role=\"slave\", cluster=\"$cluster\"}[60s]) + delta(redis_process_main_thread_cpu_user_seconds_total{bdb=\"$bdb\",role=\"slave\", cluster=\"$cluster\"}[60s]))/60)", "format": "time_series", "intervalFactor": 2, "legendFormat": "Slave main threads", "refId": "F", "step": 2400 }, { "expr": "sum((delta(redis_process_cpu_system_seconds_total{bdb=\"$bdb\",role=\"slave\",cluster=\"$cluster\"}[60s]) + delta(redis_process_cpu_user_seconds_total{bdb=\"$bdb\",role=\"slave\",cluster=\"$cluster\"}[60s]))/60)", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "Slave all threads", "refId": "G", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "BDB CPU utilization for bdb:$bdb", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "percentunit", "label": null, "logBase": 1, "max": null, "min": "0", "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 13, "legend": { "alignAsTable": true, "avg": true, "current": true, "max": true, "min": true, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 4, "stack": false, "steppedLine": false, "targets": [ { "expr": "(delta(redis_process_cpu_system_seconds_total{bdb=\"$bdb\",cluster=\"$cluster\"}[60s]) + delta(redis_process_cpu_user_seconds_total{bdb=\"$bdb\",cluster=\"$cluster\"}[60s]))/60*100 or redis_cpu_percent{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "shard: {{redis}} role: {{role}} node:{{node}}, slots:{{slots}}", "refId": "B", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "CPU usage by shards for bdb:$bdb", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "percent", "label": null, "logBase": 1, "max": null, "min": "0", "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 53, "legend": { "alignAsTable": true, "avg": false, "current": true, "max": true, "min": true, "rightSide": false, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [ { "alias": "/.*maxmemory.*/", "dashes": true, "fill": 0 } ], "spaceLength": 10, "span": 4, "stack": false, "steppedLine": false, "targets": [ { "expr": "redis_used_memory{bdb=\"$bdb\",cluster=\"$cluster\"}", "format": "time_series", "hide": false, "interval": "", "intervalFactor": 2, "legendFormat": "redis:{{redis}}, role:{{role}}, node:{{node}}, slots:{{slots}}", "metric": "redis_used_memory", "refId": "A", "step": 2400 }, { "expr": "redis_maxmemory{bdb=\"$bdb\",cluster=\"$cluster\"}", "format": "time_series", "hide": true, "interval": "", "intervalFactor": 2, "legendFormat": "maxmemory: redis:{{redis}}, bdb:{{bdb}}, role:{{role}}, node:{{node}}", "metric": "redis_used_memory", "refId": "B", "step": 240 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Sizes of shards for bdb:$bdb", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "bytes", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ] } ], "repeat": null, "repeatIteration": null, "repeatRowId": null, "showTitle": false, "title": "Dashboard Row", "titleSize": "h6" }, { "collapse": false, "height": 196, "panels": [ { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 17, "legend": { "alignAsTable": true, "avg": false, "current": true, "hideEmpty": false, "hideZero": false, "max": true, "min": true, "rightSide": false, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": false, "steppedLine": false, "targets": [ { "expr": "redis_no_of_keys{bdb=\"$bdb\", cluster=\"$cluster\"} or redis_db0_keys{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "redis:{{redis}}, role: {{role}}, node:{{node}}, slots:{{slots}}", "metric": "", "refId": "A", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "# keys/shard", "tooltip": { "msResolution": false, "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 11, "legend": { "alignAsTable": true, "avg": true, "current": true, "hideEmpty": false, "hideZero": false, "max": true, "min": true, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": false, "steppedLine": false, "targets": [ { "expr": "bdb_no_of_keys{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "num of keys", "metric": "", "refId": "A", "step": 2400 }, { "expr": "sum(redis_db0_expires{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"})", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "num of volatile keys", "refId": "B", "step": 2400 }, { "expr": "sum(max(redis_bigdb0_ram{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"} or redis_bigdb_ram{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"}) by (redis,bdb,cluster))", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "ram keys", "refId": "C", "step": 2400 }, { "expr": "sum(max(redis_bigdb0_disk{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"} or redis_bigdb_disk{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"}) by (redis,bdb,cluster))", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "disk keys", "refId": "D", "step": 2400 }, { "expr": "(sum(max(redis_bigdb0_disk{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"} or redis_bigdb_disk{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"}) by (redis,bdb,cluster)) + sum(max(redis_bigdb0_ram{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"} or redis_bigdb_ram{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"}) by (redis,bdb,cluster))) - sum(redis_db0_keys{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"})", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "overlapping keys", "refId": "E", "step": 2400 }, { "expr": "sum(max(redis_bigdb0_dirty{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"} or redis_bigdb_dirty{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"}) by (redis,bdb,cluster))", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "dirty keys", "refId": "F", "step": 2400 }, { "expr": "sum(max(redis_bigdb0_clean{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"} or redis_bigdb_clean{bdb=\"$bdb\", cluster=\"$cluster\",role=\"master\"}) by (redis,bdb,cluster))", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "clean keys", "refId": "G", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "#Keys for BDB", "tooltip": { "msResolution": false, "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 20, "legend": { "alignAsTable": true, "avg": false, "current": true, "hideEmpty": false, "hideZero": false, "max": false, "min": false, "show": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": false, "steppedLine": false, "targets": [ { "expr": "bdb_evicted_objects{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "BDB:$bdb", "metric": "", "refId": "A", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Evicted objects for BDB:$bdb", "tooltip": { "msResolution": false, "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 21, "legend": { "alignAsTable": true, "avg": false, "current": true, "hideEmpty": false, "hideZero": false, "max": false, "min": false, "show": false, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": false, "steppedLine": false, "targets": [ { "expr": "bdb_expired_objects{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "BDB:$bdb", "metric": "", "refId": "A", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Expired objects for BDB:$bdb", "tooltip": { "msResolution": false, "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ] } ], "repeat": null, "repeatIteration": null, "repeatRowId": null, "showTitle": false, "title": "Dashboard Row", "titleSize": "h6" }, { "collapse": false, "height": 202, "panels": [ { "aliasColors": {}, "bars": true, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 5, "id": 48, "legend": { "alignAsTable": true, "avg": false, "current": false, "max": false, "min": false, "rightSide": true, "show": false, "total": false, "values": false }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": true, "steppedLine": false, "targets": [ { "expr": "delta(redis_aof_rewrites{cluster=\"$cluster\", bdb=\"$bdb\"}[$aggregation]) !=0", "format": "time_series", "intervalFactor": 2, "legendFormat": "Redis {{redis}} node {{node}}", "refId": "A", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Shard AOF Rewrites / $aggregation", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "short", "label": null, "logBase": 1, "max": null, "min": "0", "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 5, "id": 57, "legend": { "alignAsTable": true, "avg": false, "current": false, "max": false, "min": false, "rightSide": true, "show": false, "total": false, "values": false }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": true, "steppedLine": false, "targets": [ { "expr": "redis_rdb_bgsave_in_progress{cluster=\"$cluster\", bdb=\"$bdb\"} !=0", "format": "time_series", "intervalFactor": 2, "legendFormat": "BGSAVE {{redis}} node {{node}}", "refId": "A", "step": 2400 }, { "expr": "redis_aof_rewrite_in_progress{cluster=\"$cluster\", bdb=\"$bdb\"} !=0", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "AOFRW {{redis}} node {{node}}", "refId": "B", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "forks", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "short", "label": null, "logBase": 1, "max": null, "min": "0", "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 49, "legend": { "alignAsTable": true, "avg": true, "current": true, "max": true, "min": true, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": false, "steppedLine": false, "targets": [ { "expr": "bdb_read_misses{cluster=\"$cluster\", bdb=\"$bdb\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "Read Misses", "refId": "A", "step": 2400 }, { "expr": "bdb_write_misses{cluster=\"$cluster\", bdb=\"$bdb\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "Write Misses", "refId": "B", "step": 2400 }, { "expr": "bdb_read_hits{cluster=\"$cluster\", bdb=\"$bdb\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "Read Hits", "refId": "C", "step": 2400 }, { "expr": "bdb_write_hits{cluster=\"$cluster\", bdb=\"$bdb\"}", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "Write Hits", "refId": "D", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Read/Write Misses", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "ops", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 23, "legend": { "alignAsTable": true, "avg": false, "current": true, "max": false, "min": false, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "connected", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": false, "steppedLine": false, "targets": [ { "expr": "redis_fragmentation{bdb=\"$bdb\", cluster=\"$cluster\"} or redis_mem_fragmentation_ratio{bdb=\"$bdb\", cluster=\"$cluster\"} ", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "Redis {{redis}} ({{role}})", "metric": "", "refId": "A", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Shard RSS fragmentation", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "percentunit", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 26, "legend": { "alignAsTable": true, "avg": true, "current": true, "max": true, "min": true, "rightSide": false, "show": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": true, "steppedLine": false, "targets": [ { "expr": "(bdb_read_hits{bdb=\"$bdb\", cluster=\"$cluster\"} + bdb_write_hits{bdb=\"$bdb\", cluster=\"$cluster\"}) / (bdb_read_req{bdb=\"$bdb\", cluster=\"$cluster\"} + bdb_write_req{bdb=\"$bdb\", cluster=\"$cluster\"}) > 0", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "DB hit ratio", "refId": "A", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "DB Hit Ratio (hits / requests)", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "percentunit", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 47, "legend": { "alignAsTable": true, "avg": true, "current": true, "max": true, "min": true, "show": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": false, "steppedLine": false, "targets": [ { "expr": "bdb_replicaof_syncer_local_ingress_lag_time{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "BDB $bdb src_id {{src_id}}", "metric": "", "refId": "A", "step": 2400 }, { "expr": "bdb_crdt_syncer_local_ingress_lag_time{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "BDB $bdb crdt src_id {{src_id}}", "metric": "", "refId": "B", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Syncer lag", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "ms", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 34, "legend": { "alignAsTable": false, "avg": false, "current": false, "max": false, "min": false, "show": true, "total": false, "values": false }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": false, "steppedLine": false, "targets": [ { "expr": "sum(redis_mem_fragmentation_ratio{bdb=\"$bdb\",cluster=\"$cluster\"}) / count(redis_mem_fragmentation_ratio{bdb=\"$bdb\",cluster=\"$cluster\"})", "format": "time_series", "intervalFactor": 2, "legendFormat": "redis_mem_fragmentation_ratio", "metric": "", "refId": "A", "step": 2400 }, { "expr": "sum(redis_allocator_active{bdb=\"$bdb\",cluster=\"$cluster\"}) / sum(redis_allocator_allocated{bdb=\"$bdb\",cluster=\"$cluster\"})", "format": "time_series", "intervalFactor": 2, "legendFormat": "Shard allocator fragmentation", "refId": "B", "step": 2400 }, { "expr": "sum(redis_allocator_resident{bdb=\"$bdb\",cluster=\"$cluster\"}) / sum(redis_allocator_active{bdb=\"$bdb\",cluster=\"$cluster\"})", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "shard allocator rss", "refId": "C", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "BDB:$bdb total fragmentation", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "percentunit", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 50, "legend": { "alignAsTable": true, "avg": false, "current": true, "max": false, "min": false, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": false, "steppedLine": false, "targets": [ { "expr": "redis_allocator_active{bdb=\"$bdb\",cluster=\"$cluster\"} / redis_allocator_allocated{bdb=\"$bdb\",cluster=\"$cluster\"}", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "redis:{{redis}} role:{{role}}", "metric": "redis_allocator_active", "refId": "A", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Shard allocator fragmentation % (defraggable)", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "percentunit", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 30, "legend": { "alignAsTable": false, "avg": false, "current": false, "max": false, "min": false, "show": false, "total": false, "values": false }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": true, "steppedLine": false, "targets": [ { "expr": "redis_active_defrag_running{cluster=\"$cluster\",bdb=\"$bdb\"}", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "redis:{{redis}} role:{{role}}", "refId": "A", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "BDB sum redis active defrag running", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "percent", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 22, "legend": { "alignAsTable": true, "avg": false, "current": true, "max": false, "min": false, "show": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "connected", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": false, "steppedLine": false, "targets": [ { "expr": "delta(redis_aof_delayed_fsync{bdb=\"$bdb\", cluster=\"$cluster\"}[$aggregation])", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "Redis {{redis}} ({{role}})", "metric": "redis_aof_delayed_fsync", "refId": "A", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Delayed fsync - (events / $aggregation)", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": false } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 58, "legend": { "alignAsTable": true, "avg": false, "current": true, "max": false, "min": false, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "null", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "span": 3, "stack": false, "steppedLine": false, "targets": [ { "expr": "redis_allocator_active{bdb=\"$bdb\",cluster=\"$cluster\"} - redis_allocator_allocated{bdb=\"$bdb\",cluster=\"$cluster\"}", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "redis:{{redis}} role:{{role}}", "metric": "redis_allocator_active", "refId": "A", "step": 2400 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "Shard allocator fragmentation bytes (defraggable)", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "decbytes", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] } ], "repeat": null, "repeatIteration": null, "repeatRowId": null, "showTitle": false, "title": "Dashboard Row", "titleSize": "h6" }, { "collapse": true, "height": 285, "panels": [ { "aliasColors": {}, "bars": false, "dashLength": 10, "dashes": false, "datasource": "${DS_PROMETHEUS1}", "fill": 1, "id": 51, "legend": { "alignAsTable": false, "avg": false, "current": false, "max": false, "min": false, "show": false, "total": false, "values": false }, "lines": true, "linewidth": 1, "links": [], "nullPointMode": "connected", "percentage": false, "pointradius": 5, "points": false, "renderer": "flot", "seriesOverrides": [ { "alias": "/.*limit.*/", "dashes": true, "fill": 0 } ], "spaceLength": 10, "span": 4, "stack": false, "steppedLine": false, "targets": [ { "expr": "bdb_used_ram{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "interval": "", "intervalFactor": 2, "legendFormat": "bdb_used_ram", "metric": "", "refId": "A", "step": 1200 }, { "expr": "bdb_ram_limit{bdb=\"$bdb\", cluster=\"$cluster\"}", "format": "time_series", "intervalFactor": 2, "legendFormat": "limit (when low, near used)", "refId": "B", "step": 1200 } ], "thresholds": [], "timeFrom": null, "timeShift": null, "title": "BDB (ROF) used RAM for $bdb", "tooltip": { "msResolution": false, "shared": true, "sort": 0, "value_type": "individual" }, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "format": "bytes", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ] }, { "aliasColors": {}, "bars": false, "dashLength": 10,