End-to-end test for ReadonlyREST security plugins

Running

On your host

Prerequisites:

• Yarn (1.22.x or newer) - for running E2E tests
• Docker (27.x.x or newer) - for running ELK environment
• Kind (0.26.x or newer) - for running ECK environment

E2E with one command

runner.sh bootstraps the environment and runs Cypress E2E tests in one shot.

Docker-based environment:

./runner.sh --elk 8.15.2 --env docker

ECK environment (pass the ECK operator version after eck-):

./runner.sh --elk 7.17.24 --env eck-2.15.0

With specific ROR versions (default: latest):

./runner.sh --elk 9.3.0 --env docker --ror-es 1.69.0 --ror-kbn 1.69.0

With dev images (default: prod):

./runner.sh --elk 9.3.0 --env docker --ror-es 1.69.0-pre1 --ror-kbn 1.69.0-pre1 --mode dev

To only bootstrap the environment without running tests (useful for debugging):

./runner.sh --run bootstrap --elk 8.15.2 --env docker

Tested environment & E2E tests separately

You can bootstrap the test env and run tests separately (to not build the ES+KBN+ROR stack every test run).

To run the env:

(running the latest version of ROR from Docker Hub production repo)

./environments/elk-ror/start.sh --es "8.15.0" --kbn "8.15.2"

(running the given version of ROR from Docker Hub development repo)

./environments/elk-ror/start.sh --es "8.15.0" --kbn "8.15.2" --ror-es 1.62.0-pre5 --ror-kbn 1.62.0-pre5 --mode dev

or

(running the latest version of ROR from Docker Hub production repo)

./environments/eck-ror/start.sh --es "8.15.0" --kbn "8.15.2"

(running the given version of ROR from Docker Hub development repo)

./environments/eck-ror/start.sh --es "8.15.0" --kbn "8.15.2" --ror-es 1.62.0-pre5 --ror-kbn 1.62.0-pre5 --mode dev

To run tests on the env from a docker environment:

./e2e-tests/run-tests.sh  "8.15.2" "elk-ror"

To run tests on the env from eck environment:

./e2e-tests/run-tests.sh  "8.15.2" "eck-ror"

Cypress tests in interactive GUI

cd e2e-tests; yarn cypress open --env kibanaVersion=[KBN_VERSION]

In docker isolated environment

Prerequisites:

• Docker (26.x.x or newer)

E2E with one command

Bootstrapping test environment and running tests inside a docker container (you don't need to have Yarn installed on your host - it's a docker-compose based environment):

$ ./docker-based-ror-dev-env/runE2ETests8xInDocker.sh

$ ./docker-based-ror-dev-env/runE2ETests7xInDocker.sh

Development

Conventions

• docs/dev/branching.md — the two long-lived branches, which one a PR targets, and the merges between them.
• docs/dev/code-style.md — the rules for comments: what a comment says, where it goes, and what deserves one.
• docs/dev/writing-style.md — the language we write in, and the rules it gives us.

Test environment

The test environment is created with the Docker Compose. All code is located in the environments/elk-ror folder. Currently, the latest version of ROR is downloaded for the sake of tests. In the future, we are going to improve it and build plugins from sources too.

Cypress tests

The Cypress-based tests are located in the e2e-tests/cypress/e2e. Screenshots and videos for test runs will be stored in results/videos and results/snapshots folders.

Docker-based ROR development environment

If you prefer, you can use scripts from the docker-based-ror-dev-env folder to run them inside a docker container of the Docker-based ROR development environment image. It's not needed, but it can be helpful in the following cases:

1. you don't want to install Yarn (or any other JS-related tools) on your host
2. you want to test if maybe there is some problem with your tools installed on the host
3. you want to check what has to be installed to run the stack with no issues
4. it can be used in the pipeline if you want to use custom runners without special preparation for them

Docker Hub pull mirror

The CI jobs pull the Docker Hub base images through pull-through caches, and then from Docker Hub. Docker Hub limits pulls per IP address, a runner shares its address with other tenants, and a neighbour over the limit makes our pull fail with a bare 429. A login does not prevent that, because the limit ignores the account. A cache can also fail to serve an image, so each pull retries and then goes to the next source.

.github/scripts/docker-hub-mirror.sh holds the list of caches. A local run pulls from Docker Hub, and ROR_DOCKER_HUB_MIRROR=false switches the mirrors off in CI.

Troubleshooting

Remember that most of the tests assume that ROR KBN is run with the Enterprise license. You have to set it on your host as an environment variable ROR_ACTIVATION_KEY. You can obtain a trial activation key in the Customer Portal or use the developer one.