Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proper logging and exception handling #300

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
175 changes: 83 additions & 92 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,106 +1,97 @@
![Build Status](https://github.com/Linaro/lava-test-plans/actions/workflows/test-plans-pipeline.yml/badge.svg)
![REUSE Compliance Check](https://github.com/Linaro/lava-test-plans/actions/workflows/reuse.yml/badge.svg)

# lava-test-plans

The lava-test-plans project makes it easier to generate LAVA job definition files.
It generates the LAVA job definition file from a set of templates.
# LAVA Test Plans
The `lava-test-plans` project makes it easier to generate [LAVA](https://www.lavasoftware.org/) job definition files from a set of templates.

# Installation
Install `lava-test-plans` from pip:

```shell
pip install lava-test-plans
````
or with podman/docker:
```shell
docker run -i -t lavasoftware/lava-test-plans /bin/bash
```

Install lava-test-plans from pip:

lava-test-plans -h

or

via podman/docker:

docker run -i -t lavasoftware/lava-test-plans /bin/bash

lavasoftware/lava-test-plans:latest points to the latest released version.
lavasoftware/lava-test-plans:master points to the latest development.
## Versions
- `lavasoftware/lava-test-plans:latest` points to the latest released version.
- `lavasoftware/lava-test-plans:master` points to the latest development.

If the above commands succeed, you can run to check that the program starts correctly
```shell
lava-test-plans -h
```

lava-test-plans -h

## Developing

## Development
To install the latest development version:
```shell
git clone https://github.com/Linaro/lava-test-plans.git
cd ./lava-test-plans
pip3 install flit
flit install --symlink
```
or you can use Docker:
```shell
docker run --volume /path/to/lava-test-plans:/xyz -i -t lavasoftware/lava-test-plans /bin/bash
cd /xyz
```
There will be a directory with `/lava-test-plans` from either a "released" version or directly from master.

git clone https://github.com/Linaro/lava-test-plans.git
cd ./lava-test-plans

pip3 install flit
flit install --symlink

or

You need to do if you have docker installed:

docker run --volume $HOME/path/to/lava-test-plans:/xyz -i -t lavasoftware/lava-test-plans /bin/bash
cd /xyz

lavasoftware/lava-test-plans:latest points to the latest released version.
lavasoftware/lava-test-plans:master points to the latest development.

There will be a directory with /lava-test-plans from either a "released"
version or directly from master.

If the above commands succeed, you can run to check that the program starts correctly

python3 -m lava_test_plans -h
If the above commands succeed, you can run to check that the program starts correctly:
```shell
python3 -m lava_test_plans --version
```
output should be similar to:
```
__main__.py, 7c588ece
```

# External variables

External variables are set in the *variables.ini* file. Each line in this file
is in the form
```
key=value
```
Lines starting with *#* are omited. Variables can also be set using
*--overwrite-variables* parameter. List of used variables:

* *PROJECT_NAME*: used as the first part in the test job name. Can be set to
differentiate LAVA test jobs between different teams/projects
* *BUILD_NUMBER*: used as last part in the test job name.
* *KERNEL_BRANCH*: used in test job name
* *OS_INFO*: used in test job name
* *LAVA_JOB_PRIORITY*: priority of the LAVA job, used by LAVA scheduler
* *LAVA_JOB_VISIBILITY*: defaults to *public*. This block can be used to restrict job visibility to user or group.
* *LAVA_JOB_VISIBILITY_GROUPS*: variable should contain groups required by job. Formtatting is important and this variable should be
formatted comma separated list. Example: group1, group2. In case of using just one group, end string with comma. Example:
group1,
* *AUTO_LOGIN_*: default *PROMPT='login:', *USERNAME='root' and *PASSWORD=''.
* *BOOT_LABEL*: default BOOT_LABEL='boot'.
* *TAGS*: variable should contain tags required by job. Formtatting is important and this variable should be
formatted comma separated list. Example: tag1, tag2. In case of using just one tag, end string with comma. Example:
tag1,
* *UBOOT_VERSION_STRING*: string that is matched in the u-boot shell from output of command *version*
* *OVERLAY_MODULES_* *: overlays modules into the rootfs.
* *TEST_DEFINITIONS_REPOSITORY*: points to the test repository to use, default: https://github.com/Linaro/test-definitions.git

Variables can also be stored in YAML file. Usual YAML syntax applies.

## Timeouts

Overall job timeout is a sum of action timeouts. There are 6 components:
* *deploy_timeout*
* *boot_timeout*
* *install_fastboot_timeout*
* *fastboot_deploy_timeout*
* *target_deploy_timeout*
* *TARGET_BOOT_TIMEOUT*
* *test_timeout*

When LXC is not in use all *lxc_* timeouts are set to 0. *test_timeout* is defined for each test template. *target_* timeouts can be set separately for each device.

# CI for docker multiarch builds
lava-test-plans gets mirrored to gitlab
https://gitlab.com/Linaro/lava-test-plans to build multiarch docker containers
and publish them to https://hub.docker.com/r/lavasoftware/lava-test-plans, that
is why there is a .gitlab-ci.yml in this repository.

# Repository
Pull requests are welcome to https://github.com/linaro/lava-test-plans.
is in the `key=value` form.

Lines starting with `#` will be omitted.

Variables can also be set using `--overwrite-variables` CLI option.

List of used variables:

* `PROJECT_NAME`: used as the first part in the LAVA test job name. Can be set to differentiate test jobs between different teams/projects.
* `BUILD_NUMBER`: used as the last part in test job's name.
* `KERNEL_BRANCH`: used in test job name.
* `OS_INFO`: used in test job name.
* `LAVA_JOB_PRIORITY`: priority of a job, used by LAVA scheduler.
* `LAVA_JOB_VISIBILITY`: defaults to *public*. This variable can be used to restrict job visibility to user or group.
* `LAVA_JOB_VISIBILITY_GROUPS`: variable should contain groups required by the job. Formatting is important and this variable - must be formatted as comma separated list. For example: `group1, group2`. In case of using just one group, end string with comma - `group1,`.
* `AUTO_LOGIN_`: default *PROMPT='login:', *USERNAME='root' and *PASSWORD=''.
* `BOOT_LABEL`: default is `boot`.
* `TAGS`: variable should contain tags required by a job. Formatting is important and this variable must be formatted comma separated list. For example: `tag1, tag2`. In case of using just one tag, end string with comma - `tag1,`.
* `UBOOT_VERSION_STRING`: string that is matched in the U-Boot shell after command `version`.
* `OVERLAY_MODULES_`: overlays modules.
* `TEST_DEFINITIONS_REPOSITORY`: points to the test definitions repository, default: https://github.com/Linaro/test-definitions.git

Variables can also be stored in YAML-formatted file - `variables.yaml`.

# Timeouts
Overall job timeout is a sum of action timeouts. There are several components:
* `deploy_timeout`
* `boot_timeout`
* `install_fastboot_timeout`
* `fastboot_deploy_timeout`
* `target_deploy_timeout`
* `TARGET_BOOT_TIMEOUT`
* `test_timeout`, is defined for each test template.

### Notes
1. When LXC is not in use all `lxc_*` timeouts are set to 0.
2. `target_*` timeouts can be set separately for each device.

# CI for Docker multi-arch builds
`lava-test-plans` repo gets mirrored to [Gitlab](https://gitlab.com/Linaro/lava-test-plans)
to build multi-arch Docker containers
and publish them to [Docker Hub](https://hub.docker.com/r/lavasoftware/lava-test-plans).

# Contributing
Pull requests are welcome to https://github.com/linaro/lava-test-plans!
Loading