Monitoring GrapheneDB instance

GrapheneDB provides an endpoint to use the open-source monitoring tool Prometheus that will allow you to monitor the underlying server of your deployment. The provided metrics (CPU, memory, disk, etc.) will help you gather information on how the load affects the available resources and decide on what plan is needed for your use case.

With Prometheus, you can set up charts to visualize metrics, configure alerts, etc. and connect to other tools like Grafana or services like Datadog or Newrelic.

The following article details a basic Prometheus and Grafana implementation, providing some examples of dashboards that you can download.

Finding the Prometheus endpoint

You can find the Prometheus endpoint on the admin console in the Insights tab for every database. Please note that you will need to whitelist the IP or establish a peering connection if the Private Network is set up to private.

Metrics reference

The available metrics are described in the following tables.

JVM metrics

Description

jvm_classes_loaded

The number of classes that are currently loaded in the JVM

jvm_classes_loaded_total

The total number of classes that have been loaded since the JVM has started execution

jvm_memory_pool_bytes_used

Used bytes of a given JVM memory pool

jvm_memory_bytes_init

Initial bytes of a given JVM memory area

jvm_memory_bytes_used

Used bytes of a given JVM memory area

jvm_memory_bytes_max

Max bytes of a given JVM memory area

jvm_gc_collection_seconds_count

Time spent in a given JVM garbage collector in seconds

Station metrics

Description

station_spec_memory_limit_bytes

Memory limit for the station in bytes

station_cpu_usage_seconds_total

Total amount of CPU time in seconds

station_memory_usage_bytes

Current memory usage in bytes

station_memory_failures_total

Cumulative count of memory allocation failures

station_cache_bytes

Amount of bytes of cache

station_cpu_load_average_delta

CPU load average percentage

station_last_seen

Last time the station was seen by the exporter

station_network_transmit_bytes_total

Cumulative count of bytes transmitted

station_network_receive_bytes_total

Cumulative count of bytes received

station_uptime_seconds

Station uptime in seconds

station_scrape_error

1 if there was an error while getting metrics, 0 otherwise

station_fs_reads_bytes_total

Total amount of bytes read from the filesystem

station_fs_writes_bytes_total

Total amount of bytes written to the filesystem

Storage metrics

Description

database_relationships_size

Relationships size on disk in bytes

database_properties_size

Properties size on disk in bytes

database_device_used_size

Used disk size in bytes

database_graphs_size

Graph databases size on disk in bytes

database_nodes_size

Nodes size on disk in bytes

database_plugins_size

Plugins size on disk

database_indexes_size

Indexes size on disk

On Cluster databases, ONgDB and Neo4j Enterprise metrics will be also available. Read more about Clusters monitoring in this section.

Setting up Prometheus and Grafana with Docker

Prerequisites

Docker​ and ​ docker-compose​ are needed to run the following steps. You can read ​here​ how to download and install them.

Configuration files

You will need to create the following configuration files before running the Prometheus and Grafana containers.

global:
  scrape_interval:     15s  # By default, scrape targets every 15 seconds.
  evaluation_interval: 15s
 
scrape_configs:
  - job_name: 'graphenedb-prometheus'
    scheme: 'https'
    static_configs:
    - targets: ['db-aqocpxyfkjc1loorgcuq.graphenedb.com:24780']

📘

Please, note that we have removed the /metrics part from the given URL. Prometheus expects metrics to be available on targets on a path of /metrics.

In case you want to monitor all the cluster nodes, read more about Clusters monitoring in this section.

apiVersion: 1
 
datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus:9090

Finally, to run Grafana and Prometheus you will need to create the following docker-compose file. The volumes path should be set to the location of your prometheus.yml and datasource.yml
files.

version: "2"
services:
 
  prometheus:
    image: prom/prometheus
    container_name: prometheus
    volumes:
      - /path_to/prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"
 
  grafana:
    image: grafana/grafana:master
    container_name: grafana
    ports:
      - "3000:3000"
    volumes:
      - /path_to/datasource.yml:/etc/grafana/provisioning/datasources/datasource.yml
    links:
      - prometheus

Grafana and Prometheus

Once the configuration files have been created, you just need to run the following docker-compose command:

docker-compose run -d -p 3000:3000 grafana

By running this command, Grafana should be ready at ​http://localhost:3000/login​. The default user/pass is admin/admin. You can also check that both containers are up and running with the docker ps command.

After logging into Grafana, you should see on the Grafana Home Dashboard the first step​ Add your first data source​ as complete. At http://localhost:3000/datasources, Prometheus data source should be listed.

To test that the Prometheus endpoint is working as expected, click on​ the Prometheus data source list item. At the bottom of the page, click on the​ Test​ green button.

Creating a Grafana Dashboard

To create a new Dashboard, visit URL ​http://localhost:3000/dashboard/new​. Then click on the top
right Dashboard settings​ button to configure it.

Apart from the name (e.g., GrapheneDB instance), description, and general settings, we can tune our Dashboard further by adding variables.

From Grafana documentation:​ A variable is a placeholder for a value. You can use variables in
metric queries and in panel titles. So when you change the value, using the dropdown at the top
of the dashboard, your panel’s metric queries will change to reflect the new value.

Read more about Dashboard and panel variables here.

All is already set up to start visualizing Prometheus given metrics. The following section will use to select and aggregate time series a functional query language called PromQL (Prometheus Query Language).

It’s recommended to take a look at PromQL documentation to further understand.

In the following section, it is explained how to add some charts to the recently created dashboard. You will find a dashboard example and how to use it in the next section.

CPU usage

To add the first chart to our Grafana dashboard, click on the Add panel button (top right menu) and click on Add Query.

The CPU load information is given by the metric station_cpu_load_average_delta. To represent this information in the way we could expect, we will need to divide this value by 100:

Network I/O

Add a new panel and click on Add Query. We are going to use station_network_transmit_bytes_total and station_network_receive_bytes_total counters and Graph visualization.

To make these counters more valuable for a database state diagnosis, we are going to use irate function. The following visualization represent the per-second average rate of increase as measured over the last 5 minutes:

On the panel edition view right tab Field, select the proper Unit: bytes/sec.

Disk usage

Add a new panel and click on Add Query. The metric we are looking for this time is database_device_used_size to get the percentage of disk usage:

The database maximum size is given by the plan. In the example we are using a DS1, with 1GB (1073741824 Bytes).

On the right Panel tab, under Visualization section, select Gauge. Then, select on the tab Field, the proper unit: percent (0-100) and set as Max value 100.

The next step is to define the threshold values. Thresholds set the color of either the value text or the background depending on conditions that you define. Set one on 80% and the other on 95%.

Unfortunately, although this visualization gives a good way to check the disk space, no alerts are available for Gauge visualization.

Let’s prepare a Graph visualization to be able to set up an alert.

Setting up alerts with Grafana

First of all, you will need to define how you are going to be notified by the Grafana alerts. You can set up different ways to be notified at the Notification channels Alerting tab.

Please, keep in mind that every notification channel must be configured to work with the Grafana alerting feature. Once you have added the needed information on the Edit notification channel form, click on the Test button to ensure everything is working as expected.

Find extended information about notification channels on Grafana docs.

Create an alert rule

Only Graph visualization includes the alert feature. Let’s add an alert for the disk usage metric.

Given the following visualization of database_size_device_used_size:

Click on the Alert tab to add an alert to send a notification when the database is at 80% of its capacity.

In this example, a DS1 with 1GB of disk, we want to be notified when the disk is above 800MB (838860800 Bytes).

The rule will be evaluated every minute and the alert notification will be sent if this rule has been firing for more than 5 minutes:

At the bottom of the form, you can add the notification channels you want to include in this alert and customize the message that will be sent.

Don’t forget to save your changes at the top right corner of the panel edition page.

Cluster monitoring

On Cluster databases, ONgDB and Neo4j Enterprise metrics will also be available. Please, check Neo4j Operation Manual to get a list of the available metrics.

In the following section, you will find a cluster dashboard example and how to use it.

Prometheus configuration file

Find the Prometheus target URL on your GrapheneDB database Insights tab. Please, note that we have removed the /metrics part from the given URL. Prometheus expects metrics to be available on targets on a path of /metrics.

In case you want to monitor all the cluster nodes, you will need to modify the given endpoint URL as in the following example:

global:
  scrape_interval:     15s  # By default, scrape targets every 15 seconds.
  evaluation_interval: 15s
 
scrape_configs:
  - job_name: 'graphenedb-prometheus'
    scheme: 'https'
    static_configs:
    - targets: ['db-1-aqocpxyfkjc1loorgcuq.graphenedb.com:24780', 'db-2-aqocpxyfkjc1loorgcuq.graphenedb.com:24780', 'db-3-aqocpxyfkjc1loorgcuq.graphenedb.com:24780']

Import/Load a dashboard

We’ve prepared some dashboards that will help you to start quickly with a working version with the most important metrics. You can download them on the following links:

Single database dashboard example

To load a dashboard JSON, click on the + icon on the left menu bar and then on the Import link. You can also just visit http://localhost:3000/dashboard/import

Finally click on Upload JSON file button, select the downloaded file and select your data source. Once you click on the Import button, the example dashboard will be loaded.

In case you want to export your dashboard to a JSON file, you can do it by clicking on the Share link near your dashboard name:

Visit the Export tab and click on the Save to file button.

Updated 4 months ago

Monitoring GrapheneDB instance


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.