scrutiny

WebUI for smartd S.M.A.R.T monitoring

[!NOTE] Scrutiny is a Work-in-Progress and still has some rough edges.

Introduction

If you run a server with more than a couple of hard drives, you're probably already familiar with S.M.A.R.T and the smartd daemon. If not, it's an incredible open source project described as the following:

smartd is a daemon that monitors the Self-Monitoring, Analysis and Reporting Technology (SMART) system built into many ATA, IDE and SCSI-3 hard drives. The purpose of SMART is to monitor the reliability of the hard drive and predict drive failures, and to carry out different types of drive self-tests.

These S.M.A.R.T hard drive self-tests can help you detect and replace failing hard drives before they cause permanent data loss. However, there's a couple issues with smartd:

• There are more than a hundred S.M.A.R.T attributes, however smartd does not differentiate between critical and informational metrics
• smartd does not record S.M.A.R.T attribute history, so it can be hard to determine if an attribute is degrading slowly over time.
• S.M.A.R.T attribute thresholds are set by the manufacturer. In some cases these thresholds are unset, or are so high that they can only be used to confirm a failed drive, rather than detecting a drive about to fail.
• smartd is a command line only tool. For head-less servers a web UI would be more valuable.

Scrutiny is a Hard Drive Health Dashboard & Monitoring solution, merging manufacturer provided S.M.A.R.T metrics with real-world failure rates.

Features

Scrutiny is a simple but focused application, with a couple of core features:

• Web UI Dashboard - focused on Critical metrics
• smartd integration (no re-inventing the wheel)
• Auto-detection of all connected hard-drives
• S.M.A.R.T metric tracking for historical trends
• Customized thresholds using real world failure rates
• Temperature tracking
• Provided as an all-in-one Docker image (but can be installed manually)
• Configurable Alerting/Notifications via Webhooks
• (Future) Hard Drive performance testing & tracking

Getting Started

RAID/Virtual Drives

Scrutiny uses smartctl --scan to detect devices/drives.

• All RAID controllers supported by smartctl are automatically supported by Scrutiny.
  • While some RAID controllers support passing through the underlying SMART data to smartctl others do not.
  • In some cases --scan does not correctly detect the device type, returning incomplete SMART data. Scrutiny supports overriding detected device type via the config file: see example.collector.yaml
• If you use docker, you must pass through the RAID virtual disk to the container using --device (see below)
  • This device may be in /dev/* or /dev/bus/*.
  • If you're unsure, run smartctl --scan on your host, and pass all listed devices to the container.

See docs/TROUBLESHOOTING_DEVICE_COLLECTOR.md for help

Docker

[!IMPORTANT] Using latest- tags is dangerous as it can update your image without warning. It is a best practice to pin a specific version. scrutiny pushes releases with semver tags, so you can use tags like v0.9.3-omnibus, v0.9-web, v0-collector, etc. For a list of all image tags see scrutiny package versions

If you're using Docker, getting started is as simple as running the following command:

See docker/example.omnibus.docker-compose.yml for a docker-compose file.

docker run -p 8080:8080 -p 8086:8086 --restart unless-stopped \
  -v `pwd`/scrutiny:/opt/scrutiny/config \
  -v `pwd`/influxdb2:/opt/scrutiny/influxdb \
  -v /run/udev:/run/udev:ro \
  --cap-add SYS_RAWIO \
  --device=/dev/sda \
  --device=/dev/sdb \
  --name scrutiny \
  ghcr.io/analogj/scrutiny:latest-omnibus

• /run/udev is necessary to provide the Scrutiny collector with access to your device metadata
• --cap-add SYS_RAWIO is necessary to allow smartctl permission to query your device SMART data
  • NOTE: If you have NVMe drives, you must add --cap-add SYS_ADMIN as well. See issue #26
• --device entries are required to ensure that your hard disk devices are accessible within the container.
• ghcr.io/analogj/scrutiny:latest-omnibus is a omnibus image, containing both the webapp server (frontend & api) as well as the S.M.A.R.T metric collector. (see below)

Hub/Spoke Deployment

In addition to the Omnibus image (available under the latest tag) you can deploy in Hub/Spoke mode, which requires 3 other Docker images:

This is the beginning of the README. Read the rest at the original. Read the full README ↗