Skip to content

Latest commit

 

History

History
1300 lines (963 loc) · 46.3 KB

index.rst

File metadata and controls

1300 lines (963 loc) · 46.3 KB
tocdepth:2

1   Docker Autotest

1.1   Introduction

Docker Autotest is a sub-framework for standalone testing of docker. It does not strictly depend on docker itself, though it can make use of the python-docker-py package (currently a reference implementation) if/when available. Functionally, testing occurs within a number of sub-test modules, which in some cases also include a number of sub-sub-tests.

It is designed to support extremely simple linear and iterative testing of arbitrary external commands under control of test-specific and default configuration values. Test content and configuration is fully modular, supporting both bundled and external tests.

As with other Autotest Client Tests the main entry point is the test control file. Docker Autotest's control file utilizes Autotest's steps-engine. This allows individual sub-tests and their control, configuration, and execution state to be stored then retrieved should a host-kernel panic or userspace become unresponsive.

1.2   Prerequisites

  • Docker

    • Clean environment (no images or other content), running containers, or dependant services (besides docker -d) at the start of every Autotest run.
    • Docker installation (with docker -d running at startup)
    • Default settings for Docker on your platform/OS, unless otherwise noted.
    • (Optional) The python-docker-py package for your platform

  • Supported Docker OS platform

    • Fedora 20 recommended
    • Linux kernel 3.12.9 or later

  • Platform Applications/tools

    • Core-utils or equivalent (i.e. cat, mkdir, tee, etc.)
    • Tar and supported compression programs
    • Git (and basic familiarity with it's operation)
    • Python 2.6 or greater (but not 3.0)
    • Optional (for building documentation), make and python-sphinx or the equivalent for your platform (supplying the sphinx-build executable)
    • Autotest 0.15.0 or later, specific version is configured.

  • Any specific requirements for particular subtest modules

1.3   Quickstart

  1. Double-check you meet all the requirements in prerequisites.
  2. Clone autotest into /usr/local/autotest
[root@docker ~]# ``git clone https://github.com/autotest/autotest.git /usr/local/autotest``
  1. Change to the client subdirectory.
  2. Create and change into the tests subdirectory (if it doesn't already exist)
[root@docker autotest]# cd client
[root@docker client]# mkdir tests
[root@docker client]# cd tests
[root@docker tests]#
  1. Clone the autotest-docker repository into the docker subdirectory.
[root@docker tests]# git clone https://github.com/autotest/autotest-docker.git docker
Cloning into 'docker'...
remote: Reusing existing pack: ... done.
remote: Counting objects: ..., done.
remote: Compressing objects: ..., done.
remote: Total .., reused ...
Receiving objects: ..., done.
Resolving deltas: ..., done.
Checking connectivity... done.
  1. Change into newly checked out repository directory.
  2. Check out the most recent release by tag.
  3. Make a copy of default configuration, edit as appropriate. Particularly the options for docker_repo_name, docker_repo_tag, docker_registry_host, and docker_registry_user (see default configuration options).
[root@docker tests]# cd docker
[root@docker docker]# git checkout $(git tag --list | tail -1)
[root@docker docker]# cp -abi config_defaults/defaults.ini config_custom/
[root@docker docker]# vi config_custom/defaults.ini
  1. Change back into the autotest client directory.
[root@docker docker]# cd /usr/local/autotest/client
  1. Run the autotest standalone client (autotest-local run docker). The default behavior is to run all subtests. However, the example below demonstrates using the --args parameter to select only two sub-tests:
[root@docker client]# ./autotest-local run docker --args=example,docker_cli/version
Writing results to /usr/local/autotest/client/results/default
START   ----    ----
    START   docker/subtests/example.test_1-of-2
        RUNNING ----    INFO: initialize()
        RUNNING ----    INFO: run_once()
        RUNNING ----    INFO: postprocess_iteration(), iteration #1
        RUNNING ----    INFO: run_once() iteration 2 of 3
        RUNNING ----    INFO: postprocess_iteration(), iteration #2
        RUNNING ----    INFO: run_once() iteration 3 of 3
        RUNNING ----    INFO: postprocess_iteration(), iteration #3
        RUNNING ----    INFO: postprocess()
        RUNNING ----    INFO: cleanup()
        GOOD    docker/subtests/example.test_1-of-2
    END GOOD    docker/subtests/example.test_1-of-2
    START   docker/subtests/docker_cli/version.test_2-of-2
         RUNNING ----    INFO: initialize()
         RUNNING ----    INFO: run_once()
         RUNNING ----    INFO: postprocess_iteration(), iteration #1
         RUNNING ----    INFO: Found docker versions client: 0.7.6 server 0.7.6
         RUNNING ----    INFO: Docker cli version matches docker client API version
         RUNNING ----    INFO: cleanup()
         GOOD    docker/subtests/docker_cli/version.test_2-of-2
    END GOOD    docker/subtests/docker_cli/version.test_2-of-2
END GOOD    ----    ----
[root@docker ~]#

(timestamps and extra text removed for clarity)

Note:Subtest names are all relative to the subtests sub-directory and must be fully-qualified. e.g. docker_cli/version refers to the subtest module subtests/docker_cli/version/version.py.

1.4   Subtests

All subtest modules reside beneath the subtest directory. A subtest module must have the same name as the directory it is in (minus the .py extension). Other files/directories may exist at that level, but they will not be recognized as subtest modules by the Docker client test. This ensures each subtest's code is kept separate from all others.

The structure/layout of the subtest directory tree is not important for locating/executing subtests. However it is relevant for the finding/loading of each subtests configuration. The configuration section name for any sub-tests is formed by the subtest name relative to the subtest directory. For example, the subtest module subtests/docker_cli/version/version.py matches with the [docker_cli/version] configuration section. The relative location of the configuration file does not matter, only the section name.

Additionally, subtests may source their own static content. If this content is further test components, please see the Subtest Module section regarding the dockertest.subtest.SubSubtest class. If static content needs to be built, or in some way made environment-specific, this must happen by overriding the setup() method. Within this method, content it should be copied from from the path referenced in the bindir attribute, to the path referenced by the srcdir attribute. The setup() method will *only* be called once per version number (including revisions). State may be reset by clearing the autotest client tmp directory.

Note:setup() runs after initialize()

1.5   Images

Multiple areas of documentation and output refer to repository, images, and layers almost interchangeably. There are also multiple interfaces available for image creation, retrieval, comparison, etc. these items. However, images are extremely central to working with containers in general, and with docker specifically. A generalized interface for working with these items is provided by the images module.

This module wraps many concepts up inside constructs to provide a measured amount of abstraction. It allows subtests and other callers to expand it's concepts while providing a high-level consistent, extensible and uniform set of helpers. Most of them are designed to be agnostic toward the actual method of image access or representation.

Though for internal use it is highly recommended to reference images only by their long (64-character) ID string. Otherwise, for specific test-subjects, or use, any of the provided interfaces may be used and/or specialized. Extension of interfaces can be done within subtest modules directly, or some combination of sources.

1.6   Configuration

The default configuration files are all located under the config_defaults sub-directory. These are intended to be bundled with the autotest docker test. To customize any subtest or global default configuration, copies should be made manually into the config_custom sub-directory. Any content within config_custom will override anything found under config_defaults. Nothing except for the example config_custom/example.ini should be checked into version control.

1.6.1   Organization

1.6.1.1   Sections

Configuration files use the familiar ini style format with separate sections (e.g. [<section name>]) preventing option names from colliding. All configuration files are loaded into a single name-space, divided by each section. Section names can be arbitrary, however those which exactly match a subtest or subsubtest's name, will be automatically loaded (see Subtests).

1.6.1.2   Defaults

The Default, global values for all sections are located within the special defaults.ini file's DEFAULTS section. These option names and values are supplied for all sections which do not contain a identical named option. See Default configuration options for more details.

1.6.2   Formatting

1.6.2.1   Long values

Long values may be continued onto the next line by prefixing any run of one or more horizontal-whitespace characters with a newline. For example:

option_name = This option value is super duper long,
but will be un-folded into a single string with <---- all this whitespace replaced by a single space.

In this case, the runs of multiple whitespace following the newline will be folded into a single instance, and substituted for the previous newline.

1.6.2.2   Value substitution

Within each section, optional inline-value substitution is supported using the %(<option>)s token format. Where <option> is the literal name of another option. The referenced option must reside in the same section or in the special DEFAULTS section. This is useful when multiple option values need to be different but contain a shared element.

Note:The relative locations of files under config_defaults and config_custom does not matter. Multiple sections may appear in the same file.
1.6.2.3   Type-conversion

The config parser will attempt to parse each item in the following order: integers, booleans, floats, then strings.

  • Integers are in the form of simple numbers, eg: "123"
  • Booleans are in the form 'yes' or 'true', 'no' or 'false' (case insensitive)
  • Floats are in the form of numbers with decimals eg: "123.456" or "123.0"
  • All other items will be returned as strings.

1.7   Versioning Requirements

In order to support external/private subtests and customized configurations, the Docker Autotest API version has been tightly coupled to test content, configuration, and documentation. Version comparison is only significant for the first two numbers (major and minor). The third (revision) number represents insignificant revisions which do not alter the core test or subtest API.

This allows the API to be extended freely, but any changes which could affect external tests or custom configurations will cause automatic test failures when encountered. The most likely cause for version problems is custom and/or outdated configurations. Double-check any customizations within config_custom match any changes to the current API. The same goes for any private or local tests.

Documentation versioning is similarly tied to changes in the API version. While non-fatal, it will introduce a delay in subtest execution. This signal is intended to alert developers a documentation-update pass is required to reflect changes in the API.

1.8   Subtest Modules

The following sections detail specific sub-tests, their configuration and any prerequisites or setup requirements.

1.8.1   Default configuration options

Global default options that apply to all other sections are set in the special DEFAULTS section of the defaults.ini file. This file is loaded either from config_defaults or config_custom.

  • For subtests which use the global default test image/repository, it's fully-qualified name is formed by the values to the docker_repo_name, docker_repo_tag docker_registry_host, and docker_registry_user options.
  • The config_version option is set to the API version this configuration is intended for. Because a copy of the defaults.ini file will not inherit default version number changes, it will cause most tests to fail after changing dockertest API versions. This is intentional behavior and so this option must not be overriden in any default/bundled subtest configuration. It should be overridden in custom/private test configuration.
  • The autotest_version option specifies the minimum version of the autotest framework that is required. It may be overridden by subtests to indicate they require a specific later version than the default. The autotest version is checked before the dockertest version.
  • The docker_path option specifies the absolute path to the docker executable under test. This permits both the framework and/or individual sub-tests to utilize a separate compiled executable (e.g. possibly with non-default build options).
  • The docker_options option specifies the command-line interface options to use before any sub-commands and their options.
  • Any operations calling the docker CLI will by default, use the value in docker_timeout. This may be an integer or floating-point number specifying the number of seconds to allow any single command to complete.
  • Since all tests run by default (when no --args CSV list is used), it could be difficult to skip just a single or several tests while running all others. The disable option may be specified in DEFAULTS or any subtest and contains a comma-separated-list of subtest or sub-subtest names to skip.

1.8.2   example Sub-test

A boiler-plate example subtest intended as a starting place for new sub-tests. Not all methods are required, those not overridden will simply inherit default behavior.

1.8.2.1   example Prerequisites

The example subtest has no prerequisites.

1.8.2.2   example Configuration

The example subtest configuration is to demonstrate its values are overridden by the example.ini under config_custom.

1.8.3   subexample Sub-test

A boiler-plate example subtest with subsubtests, intended as a starting place for new sub-tests. Not all methods are required, those not overridden will simply inherit default behavior.

1.8.3.1   subexample Prerequisites

The example subtest has no prerequisites.

1.8.3.2   subexample Configuration

Includes the requisite subsubtests CSV option, specifying the subtest names to include. Their actual execution order us not defined.

1.8.4   docker_cli/version Sub-test

Simple test that checks the output of the docker version command.

1.8.4.1   docker_cli/version Prerequisites

Docker daemon is running and accessible by it's unix socket.

1.8.4.2   docker_cli/version Configuration

None

1.8.5   docker_cli/build Sub-test

Tests the docker build command operation with a set of options and pre-defined build-content.

1.8.5.1   docker_cli/build Prerequisites
  • Tarballs bundled with the subtest
  • Statically linked 'busybox' executable available over HTTP
1.8.5.2   docker_cli/build Configuration
  • The docker_build_options option specifies additional arguments to add in addition to the DEFAULTS option docker_options.
  • The build_timeout_seconds option specifies a fixed time (in seconds) the build must complete within)
  • try_remove_after_test is a boolean option, selecting whether or not the built-image should be removed when the test is complete. (Any removal errors will be ignored)
  • Both the image_name_prefix and image_name_postfix behave exactly like the docker_cli/dockerimport sub-test test.
  • The location of the statically linked busybox executable is specified by the busybox_url option.

1.8.6   docker_cli/build_paths Sub-test

Tests the docker build against a list of docker build paths or git locations.

1.8.6.1   docker_cli/build_paths Prerequisites
  • Valid docker build paths or git locations with a Dockerfile
1.8.6.2   docker_cli/build_paths Configuration
  • build_paths is a csv list of docker build paths or git locations.
  • build_args are args passed directly to docker build.
  • image_repo_name lets you name the REPOSITORY of the images built. Only applies if --tag is not used in build_args
  • image_tag_postfix lets you add a postfix to the randomly generated TAG of the images built. Only applies if --tag is not used in build_args.

1.8.7   docker_cli/dockerimport Sub-test

This test is actually composed of a number of sub-sub-tests. It demonstrates and supports executing multiple variations of a test for the same docker-command. Here, the test is named dockerimport because import is a python-keyword.

The sub-sub-tests run somewhat in parallel, with the order fixed by a configuration option. This means all sub-sub-test's run_once() methods will be called first before each sub-sub-test's postprocess() methods. All sub-sub-tests cleanup() methods are guaranteed to run, even if an exception or error occurs.

1.8.7.1   docker_cli/dockerimport Prerequisites
  • Enough disk space to construct and import several base images at the same time.
  • The tar, and cat commands.
1.8.7.2   docker_cli/dockerimport Configuration

Configuration for this subtest consists of a few options which control overall sub-sub-test execution. Further, unique sections for each sub-sub-test are also used.

  • The image_name_prefix and image_name_postfix specify values used to automatically generate a unique image name. The unique part will be sandwiched in-between these options values.
  • try_remove_after_test is exactly like the same option in the docker_cli/build sub-test subtest.
  • The test_subsubtest_postfixes contains a CSV listing of the sub-sub-test modules (and class) names to run (in order).
  • The sub-sub-test section options are self-explanatory. For this class of sub-test they list the tar-command location and options to use before sending the content into the docker import command.

1.8.8   docker_cli/images Sub-test

Ultra-simple test to confirm output table-format of docker CLI 'images' command.

1.8.8.1   docker_cli/images Prerequisites
  • None
1.8.8.2   docker_cli/images Configuration
  • None

1.8.9   docker_cli/run_volumes Sub-test

  • volumes_rw: Attempt to read, then write a file from a host path volume inside a container. Intended to test NFS, SMB, and other 'remote' filesystem mounts.
  • volumes_one_source: Have multiple containers mount a directory and then write to files in that directory simultaneously.
1.8.9.1   docker_cli/run_volumes Prerequisites
  • Remote filesystems are mounted and accessible on host system.
  • Containers have access to read & write files w/in mountpoints
1.8.9.2   docker_cli/run_volumes/volumes_rw Configuration
  • The host_paths and corresponding cntr_paths are most important. They are the host paths and container paths comma-separated values to test. There must be 1:1 correspondence between CSVs of both options. The lists must also be the same length.
  • run_template allows fine-tuning the options to the run command.
  • The cmd_template allows fine-tuning the command to run inside the container. It makes use of shell-like value substitution from the contents of host_paths and cntr_paths.
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.
  • The wait_stop option specifies the time in seconds to wait after all docker run processes exit.
1.8.9.3   docker_cli/run_volumes/volumes_one_source Configuration
  • The num_containers is the number of containers to run concurrently.
  • The cmd_timeout is the timeout for each container's IO command.
  • The cntr_path is where to mount the volume inside the container.
  • The exec_command is the command each container should run. This should be an IO command that writes to a file at ${write_path} which will be inside the mounted volume. This command should also take time to allow for taking place while the other containers are also writing IO.

1.8.10   docker_cli/rm Sub-test

Start up a container, run the rm subcommand on it in various ways

1.8.10.1   docker_cli/rm Prerequisites
  • None
1.8.10.2   docker_cli/rm Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.

1.8.11   docker_cli/dockerhelp Sub-test

Several variations of running the dockerhelp command.

1.8.11.1   docker_cli/dockerhelp Prerequisites
  • None
1.8.11.2   docker_cli/dockerhelp Configuration
  • The success_option_list is a CSV list of docker options where a zero-exit code is expected (though a usage message may appear)
  • The failure_option_list is the opposite.

1.8.12   docker_cli/run_simple Sub-test

Three simple subsubtests that verify exit status and signal pass-through capability

1.8.12.1   docker_cli/run_simple Prerequisites
  • Container image with a /bin/bash shell executable
  • Container image with a /bin/true executable returning zero
  • Container image with a /bin/false executable returning non-zero
  • Container image with a /bin/date executable
  • Accurate (relative to host) timekeeping in container
1.8.12.2   docker_cli/run_simple Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.

1.8.13   docker_cli/rmi Sub-test

Several variations of running the rmi command.

1.8.13.1   docker_cli/rmi Prerequisites
1.8.13.2   docker_cli/rmi Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.
  • The remove_after_test option controls cleanup after all sub-subtests.
  • The docker_rmi_force option causes sub-subtests to force remove images
  • docker_expected_result should be "PASS" or "FAIL" to indicate result handling behavior of sub-subtests.

1.8.14   docker_cli/pull Sub-test

Several variations of running the pull command against a registry server.

1.8.14.1   docker_cli/pull Prerequisites
  • A remote registry server
  • Image on remote registry with 'latest' and some other tag
1.8.14.2   docker_cli/pull Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.

1.8.15   docker_cli/commit Sub-test

Several variations of running the commit command

1.8.15.1   docker_cli/commit Prerequisites
  • A remote registry server
  • Image on remote registry with 'latest' and some other tag
1.8.15.2   docker_cli/commit Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.

1.8.16   docker_cli/events Sub-test

Start up a simple /bin/true container while monitoring output from docker events command. Verify expected events appear after container finishes and is removed.

1.8.16.1   docker_cli/events Prerequisites
  • Historical events exist prior to running test (i.e. docker daemon hasn't been restarted in a while)
  • Host clock is accurate, local timezone setup properly.
  • Host clock does not change drastically during test
1.8.16.2   docker_cli/events Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.
  • run_args is a CSV list of arguments to the run command
  • rm_after_run specifies whether or not to use the docker rm command after the container finishes.
  • The wait_stop option specifies time in seconds to wait after removing the container, to check events.
  • expect_events is a CSV of required events for test to pass
  • name_prefix specifies the container name prefix to use. before random characters are added.
  • The unparseable_allowance setting specifies the number of lines with parse errors to allow.
  • The remove_after_test option controls cleanup after test

1.8.17   docker_cli/psa Sub-test

Verify the table output and formatting of the docker ps -a command.

1.8.17.1   docker_cli/psa Prerequisites
  • None
1.8.17.2   docker_cli/psa Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.
  • The wait_stop and wait_start options specify time in seconds to wait before/after starting the test container.
  • The remove_after_test option controls cleanup after test

1.8.18   docker_cli/tag Sub-test

Several variations of running the docker tag command

1.8.18.1   docker_cli/tag Prerequisites
  • None
1.8.18.2   docker_cli/tag Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.
  • tag_force specifies use of --force option
  • The tag_repo_name_prefix option has random characters appended before it is used for tagging.
  • docker_expected_result option allows changing between positive and negative testing.

1.8.19   docker_cli/stop Sub-test

Several variations of running the stop command

1.8.19.1   docker_cli/stop Prerequisites
  • A remote registry server
1.8.19.2   docker_cli/stop Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.
  • The top_name_prefix is prefix of the tested container followed by random characters to make it unique.
  • The run_options_csv modifies the running container options.
  • The stop_options_csv specifies the stop command options
  • The exec_cmd modifies the container command
  • The stop_duration sets the acceptable stop command duration (+-2s)

1.8.20   docker_cli/restart Sub-test

Several variations of running the restart command

1.8.20.1   docker_cli/restart Prerequisites
  • A remote registry server
1.8.20.2   docker_cli/restart Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.
  • The run_options_csv modifies the running container options.
  • The restart_options_csv modifies the restart command options.
  • The stop_options_csv specifies the stop command options.
  • The exec_cmd modifies the container command
  • The start_check, restart_check and stop_check are 'n' separated lines which should be present in specific test stage.
  • The restart_duration and stop_duration are expected command execution durations (+-3s)

1.8.21   docker_cli/start Sub-test

Several variations of using docker start command

1.8.21.1   docker_cli/start Prerequisites
  • A remote registry server
1.8.21.2   docker_cli/start Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.
  • The run_options_csv modifies the running container options.
  • The container_name_prefix is prefix of the tested container followed by random characters to make it unique.
  • The remove_after_test specifies whether to remove the container after the test

1.8.22   docker_cli/kill Sub-test

Several variations of running the kill command * random_* - series of random signals * sigstop - worst case of stopped container scenario * bad - bad input * stress - lots of signals without waiting * stress_parallel - all signals simultaneously * run_sigproxy* - instead of docker kill uses kill on docker run * attach_sigproxy* - instead of dicker kill uses kill on docker attach

1.8.22.1   docker_cli/kill Prerequisites
  • A remote registry server
1.8.22.2   docker_cli/kill Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.
  • The top_name_prefix is prefix of the tested container followed by random characters to make it unique.
  • The run_options_csv modifies the running container options.
  • The exec_cmd modifies the container command
  • The wait_start is duration of container init
  • The no_iterations is number of signals (in some subsubtests)
  • The kill_map_signals chooses between numerical and named signals (USR1) * true - all signals are mapped * false - all signals are numbers * none - randomize for each signal
  • The signals_sequence allows you to force given sequence of signals. it's generated in case it's missing and printed in log for later use.
  • The kill_signals specifies used signals [range(*args)]
  • The skip_signals specifies which signals should be omitted
  • The kill_sigproxy changes the kill command (false - docker kill, true - os.kill $docker_cmd.pid.

1.8.23   docker_cli/top Sub-test

Several variations of running the restart command

1.8.23.1   docker_cli/top Prerequisites
  • A remote registry server
1.8.23.2   docker_cli/top Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.
  • The container_name_prefix is prefix of the tested container followed by random characters to make it unique.
  • The run_options_csv modifies the running container options.
  • The remove_after_test removes the container during the cleanup

1.8.24   docker_cli/wait Sub-test

Several variations of running the restart command

1.8.24.1   docker_cli/wait Prerequisites
  • A remote registry server
1.8.24.2   docker_cli/wait Configuration
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.
  • The run_options_csv modifies the running container options.
  • The wait_options_csv modifies the wait command options.
  • The exec_cmd modifies the container command. Note that in this tests you can specify per-container-exec_cmd using exec_cmd_$container. This command has to contain exit $NUM, which is used as docker exit status and could contain sleep $NUM which signals the duration after which the container finishes.
  • The wait_for specifies the containers the wait command should wait for. Use index of containers or _$your_string. In the second case the leading character _ will be removed.

1.8.25   docker_cli/info Sub-test

Simple test that checks the output of the docker info command. It verifies the output against values obtained from userspace tools.

1.8.25.1   docker_cli/info Prerequisites
  • Docker daemon is running and accessible by it's unix socket.
  • dmsetup and du commands are available.
1.8.25.2   docker_cli/info Configuration

None

1.8.26   docker_cli/cp Sub-test

Simple test that checks the success of the docker cp command. It copies a file to a temporary directory and verifies that it was copied successfully.

1.8.26.1   docker_cli/cp Prerequisites
  • Docker daemon is running and accessible by it's unix socket.
1.8.26.2   docker_cli/cp Configuration
  • The remove_after_test specifies whether to remove the container created during the test.

1.8.27   docker_cli/insert Sub-test

Simple test that checks the success of the docker insert command. It will insert the file at the url into an image, and then verify that it was inserted successfully.

1.8.27.1   docker_cli/insert Prerequisites
  • Docker daemon is running and accessible by it's unix socket.
1.8.27.2   docker_cli/insert Configuration
  • The remove_after_test specifies whether to remove the container created during the test.
  • The file_url is the url to a file to be inserted during the test.

1.8.28   docker_cli/run_twice Sub-test

Verify that could not run a container which is already running.

1.8.28.1   docker_cli/run_twice Prerequisites
  • Docker daemon is running and accessible by it's unix socket.
1.8.28.2   docker_cli/run_twice Configuration
  • The remove_after_test specifies whether to remove the container created during the test.

1.8.29   docker_cli/diff Sub-test

This set of tests modifies files within an image and then asserts that the changes are picked up correctly by docker diff

1.8.29.1   docker_cli/diff Prerequisites
  • Docker daemon is running and accessible by it's unix socket.
1.8.29.2   docker_cli/diff Configuration
  • The remove_after_test specifies whether to remove the container created during the test.
  • command is a csv arg list to docker run that specifies how a test will modify a file for the test
  • files_changed is a csv list of expected change types and the files/directories that are changed. It is in the form of: <change type 1>,<path 1>,<change type 2>,<path 2> and so on.

1.8.30   docker_cli/invalid Sub-test

Simple test that checks the success of the docker run command. It will run container using the invalid character, and then verify that it was not allowed.

1.8.30.1   docker_cli/invalid Prerequisites
  • Docker daemon is running and accessible by it's unix socket.
1.8.30.2   docker_cli/invalid Configuration
  • The section specifies which section to test.
  • The subsubtests specifies which subtests to run.
  • Customized configuration for invalid_run_params, expected_result and invalid_pars_expected_output, invalid_vals_expected_output and input_docker_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.

1.8.31   docker_cli/workdir Sub-test

Simple test that checks the docker run --workdir command could set workdir successfully if the dir is a valid path, and fails if it's not absolute path or not a path, like a file.

1.8.31.1   docker_cli/workdir Prerequisites
  • Docker daemon is running and accessible by it's unix socket.
1.8.31.2   docker_cli/workdir Configuration
  • The remove_after_test specifies whether to remove the container created during the test.

1.8.32   docker_cli/dockerinspect Sub-test

This is a set of subsubtests that test the inspect command.

1.8.32.1   docker_cli/dockerinspect Prerequisites
  • Docker daemon is running and accessible by it's unix socket.
1.8.32.2   docker_cli/dockerinspect Configuration
  • The remove_after_test specifies whether to remove the containers created during the test.
  • The subsubtests tells which subtests to run in this test group.
1.8.32.3   docker_cli/dockerinspect/inspect_container_simple Configuration
  • check_fields specifies which fields to check the existence of when running "docker inspect" on a container.
1.8.32.4   docker_cli/dockerinspect/inspect_all Configuration
  • ignore_fields specifies which fields to ignore when checking all fields when running "docker inspect" on a container.
1.8.32.5   docker_cli/dockerinspect/inspect_keys Configuration
  • note all of these fields are optional. Leave them blank to skip checking for them.
  • image_keys specifies which fields to check for in an image inspect
  • container_keys specifies which fields to check for in a container inspect
  • key_regex asserts that each key matches this regex

1.8.33   docker_cli/run_memory Sub-test

Simple test that checks the output of the docker run -m command. It verifies that if the container's cgroup resource memory.limit_in_bytes equals the value passed or if the container can handle invalid value properly.

1.8.33.1   docker_cli/run_memory Prerequisites
  • Docker daemon is running and accessible by it's unix socket.
1.8.33.2   docker_cli/run_memory Configuration
  • The option remove_after_test specifies whether to remove the container created during the test.
  • Customized configuration for docker_repo_name, docker_repo_tag, and optionally docker_registry_host and/or docker_registry_user. i.e. Copy config_defaults/defaults.ini to config_custom/defaults.ini and modify the values.
  • The option positive, sets the pass/fail logic for results processing.

1.9   Dockertest API Reference

1.9.1   Dockertest Package

.. automodule:: dockertest
   :no-members:
   :no-undoc-members:

1.9.2   Subtest Module

.. py:module:: dockertest.subtest

Adapt/extend autotest.client.test.test for Docker test sub-framework

This module provides two helper classes intended to make writing subtests easier. They hide some of the autotest test.test complexity, while providing some helper methods for logging output to the controlling terminal (only) and automatically loading the specified configuration section (see configuration module)

.. autoclass:: dockertest.subtest.SubBase
   :members:


.. autoclass:: dockertest.subtest.Subtest
   :members:
   :no-inherited-members:


.. autoclass:: dockertest.subtest.SubSubtest
   :members:


.. autoclass:: dockertest.subtest.SubSubtestCaller
   :members:

1.9.3   Images Module

.. automodule:: dockertest.images
   :members:
   :no-undoc-members:

1.9.4   Containers Module

.. automodule:: dockertest.containers
   :members:
   :no-undoc-members:

1.9.5   Environment Module

.. automodule:: dockertest.environment
   :members:
   :no-undoc-members:

1.9.6   Networking Module

.. automodule:: dockertest.networking
   :members:
   :no-undoc-members:

1.9.7   Docker_Daemon Module

.. automodule:: dockertest.docker_daemon
    :members:
    :no-undoc-members:
    :no-inherited-members:

1.9.8   Dockercmd Module

.. automodule:: dockertest.dockercmd
   :members:
   :no-undoc-members:

1.9.9   Output Module

.. automodule:: dockertest.output
   :members:
   :no-undoc-members:

1.9.10   Xceptions Module

.. automodule:: dockertest.xceptions
   :members:
   :undoc-members:

1.9.11   Sphinx Conf Module

.. automodule:: conf
   :members:
   :undoc-members:

1.9.12   Version Module

.. automodule:: dockertest.version
   :members:
   :no-undoc-members:

1.9.13   Configuration Module

.. automodule:: dockertest.config
   :members:
   :no-undoc-members:

1.10   Further Reading

1.11   Indices and Tables