EPICS Archiver Appliance Datasource

Visualize EPICS Archiver Appliance on Grafana.

See Archiver Appliance site for more information about Archiver Appliance.

Features

• Select multiple PVs by using Regex (Only supports wildcard pattern like PV.* and alternation pattern like PV(1|2))
• Legend alias with regular expression pattern
• Data retrieval with data processing (See Archiver Appliance User Guide for processing of data)
• Using PV names for Grafana variables
• Transform your data with processing functions
• Live update with stream feature
• Find and notify problems with alerting feature

Installing the plugin

Install with Grafana CLI

1. Install the plugin with Grafana CLI. Execute Grafana CLI as following:

# Install latest version. You can also use this command to update the plugin to the latest version.
grafana cli --pluginUrl https://github.com/sasaki77/archiverappliance-datasource/releases/latest/download/archiverappliance-datasource.zip plugins install sasaki77-archiverappliance-datasource

# Install particular version. This example will install v1.4.2.
grafana cli --pluginUrl https://github.com/sasaki77/archiverappliance-datasource/releases/download/1.4.2/archiverappliance-datasource.zip plugins install sasaki77-archiverappliance-datasource

# Install nightly builds.
grafana cli --pluginUrl https://github.com/sasaki77/archiverappliance-datasource/releases/download/nightly/archiverappliance-datasource.zip plugins install sasaki77-archiverappliance-datasource

2. This plugin is unsigned. It must be specially listed by name in the Grafana grafana.ini file to allow Grafana to use it. Add sasaki77-archiverappliance-datasource to the allow_loading_unsigned_plugins parameter in the [plugins] section. See Configure Grafana | Grafana documentation for more detail on grafana.ini.

To update the plugin, execute Grafana CLI again.

Install using Git

Clone this plugin into grafana plugins directory; the default is /var/lib/grafana/plugins.

Configuring the plugin

Please consult Configuration manual for instructions to configure the plugin following installation.

Documentaion

Documentation is available at https://sasaki77.github.io/archiverappliance-datasource.

Configuration

This section lists the available configuration options for the Archiver Appliance data source.

Query editor

Configuration	Description
PV	Set PV name to be visualized. It is allowed to set multiple PVs by using Regular Expressoins alternation pattern (e.g. (PV:1|PV:2)).
Regex	Enable/disable Regex mode. Refer Select Multiple PVs by Regex.
Operator	Controls processing of data during data retrieval (Default: mean). Refer Archiver Appliance User Guide about processing of data. Special operator raw and last are also available. raw allows to retrieve the data without processing. last allows to retrieve the last data in the specified time range.
Stream	Enable/Disable Stream mode. Stream allows to periodically update the data without refreshing the dashboard. The difference data from the last updated values is only retrieved.
Interval	Streaming interval in milliseconds. You can also use a number with unit. e.g. 1s, 1m, 1h. The default is determined by a width of panel and time range.
Capacity	The stream data is stored in a circular buffer. Capacity determines the buffer size. The default is detemined by a initial data size.
Alias	Set alias for legend.
Alias pattern	Set regular expressoin pattern to use PV name for legend alias. Refer Legend Alias with Regex Pattern
Function	Apply processing function for retrieved data. Refer Apply Processing Functions

Variables

Variables are supported for PV names registered in Archiver Appliance.

You can use regular expression pattern same as Query Edit. Only wildcard pattern and alternation pattern are available.

You can also use another variable as a part of query. For example, you have variable group, which is a list of PV name prefixes according to PV naming rules on your institute.

Below is a query in this case.

${group}:.*

Default maximum number of PV names you can use in variables is 100. Limit parameter is available to change maxmum number. Parameters must be follow ? character.

PV:NAME:.*?limit=1000

Development setup

This data source plugin consists of both frontend and backend components.

Front-end Development

LTS version of Node.js is recommended. If you're new to the Node.js ecosystem, Node Version Manager is a good place to start for managing different Node.js installations and environments.

Grafana's plugin tools is used to develop the plugin. Please refer the documentation for more information.

1. Begin by installing Yarn (https://yarnpkg.com/)

   npm install -g yarn
   

2. Install dependencies

   yarn install
   

3. Build plugin in development mode or run in watch mode

   yarn dev
   

   or

   yarn watch
   

4. Build plugin in production mode

   yarn build
   

Back-end Development

If the Mage build tool is not already installed, you may install it using the installation instructions on the Mage homepage

1. Update Grafana plugin SDK for Go dependency to the latest minor version:

   go get -u github.com/grafana/grafana-plugin-sdk-go
   

2. Build backend plugin binaries for Linux, Windows and Darwin:

   mage -v
   

3. List all available Mage targets for additional commands:

   mage -l
   

Benchmarks

The backend benchmarks cover the three stages of the query pipeline and live next to the code they measure:

File	Covers
pkg/archiverappliance/pbparse_bench_test.go	PB response parsing (scalar, waveform, string, meta fields)
pkg/models/scalars_bench_test.go	Scalars transforms (scale, offset, delta, moving average, ranking)
pkg/models/singledata_bench_test.go	SingleData to data.Frame conversion for each array format
pkg/functions/functions_bench_test.go	The processing-function pipeline, including sort and array-to-scalar functions

mage bench                             # everything
mage abench 'BenchmarkPBparseOneday$'  # a subset; the argument is a regular expression
mage abench 'MovingAverage/Window1000' # a sub-benchmark, selected with a slash

Both targets report memory statistics. Watch allocs/op: the parser and the transform functions run once per archived sample, so a constant allocation cost per sample dominates on raw queries that return hundreds of thousands of points.

Comparing against a baseline

A single run is too noisy to draw conclusions from. Take several runs of each side and compare them with benchstat, installed with go install golang.org/x/perf/cmd/benchstat@latest.

For uncommitted work:

git stash
go test ./pkg/... -run '^$' -bench . -benchmem -count=10 > before.txt
git stash pop
go test ./pkg/... -run '^$' -bench . -benchmem -count=10 > after.txt
benchstat before.txt after.txt

For a change that is already committed, check the baseline out into a throwaway worktree, so the two runs need no stashing and cannot disturb each other:

WORKTREE=$(mktemp -d)/base   # not ../base: the parent of the repo is not always writable
git worktree add --detach "$WORKTREE" "$(git merge-base HEAD master)"

(cd "$WORKTREE" && go test ./pkg/... -run '^$' -bench . -benchmem -count=10) > before.txt
go test ./pkg/... -run '^$' -bench . -benchmem -count=10 > after.txt
benchstat before.txt after.txt

git worktree remove --force "$WORKTREE"   # `git worktree prune` if the directory is gone

Benchstat matches benchmarks by name, which cuts two ways. One that exists on a single side is dropped from the comparison, which is what you want for a benchmark added alongside the change. One whose body changed still lines up by name and is compared anyway — ignore those rows, or move the baseline to a commit from after that benchmark settled.

Build documentation

python -m venv env
source env/bin/activate
pip install -r docs/requirements.txt
sphinx-multiversion docs docs/_build/html

Development Environment

Development Containers is available for development. You can launch a development container directly from Visual Studio Code.

The repository also includes a docker-compose.yaml file for the test environment. The compose file defines the following four containers:

• grafana
• mariadb
• archappl
• ioc

The ioc container automatically runs softIoc.

The Archiver Appliance also starts automatically in the archappl container. However, no PVs are registered by default. You need to register PVs manually for archiving.

The Grafana server runs in the grafana container. To retrieve data from archappl, you need to add a new data source using this plugin. Use the following URL to access data from archappl: http://archappl:8080/retrieval

Build Status

Build:	Status:
Latest release
Master Branch
Local Master Branch
Local Latest Branch