What is Autotest

Autotest is a framework written (mainly) in Python for testing Linux platform. It is a solutioin providing frontend and backend as a web service. However, you could only use part of the service. For example, running the colectioin of local test jobs,autotest-client-tests to invoke a local/client testing process as a command line tool.

This post will focus on autotest-client-tests.

Brief Introduction and History of Autotest

Autotest is:

• a framework. autotest-client-tests is a collection of jobs executed by the framework.
• mainly written in Python2.
• used for testing Linux kernels and aim at fully automated.
• a service that provides frontend and backend.
• a service able to use as a command line only.
• a legacy/inactive project which has no new release for two years. Its next-generation is Avocado.
• a open source project
• used for an inactive project TKO, test.kernel.org at the time of writing.

Why is test.kernel.org inactive? I don't know. I have not found any announcement yet. If you know the context, please let me know.

At the time of writing, although Autotest is inactive for more than 2 years, Python2 has been EOL as well, and TKO is inactive, there are projects are still using Autotest to test their Linux kernels internally.

Foundamental Elements to Compose a Local Job of Autotest

control file and class method test.test:run_once are two foundamental elements to compose a local job of Autotest, a.k.a. a job of autotest-client-tests. Here is their introduction.

Compose a Control file

A control file is:

• a python script essentially.
• is used by autotest/client/autotest-local, which is the entry point of local client testing of Autotest, as a input argument.

The necessary fields are some expected variables e.g. AUTHOR and DOC (refer to the Autotest document for more details), and the function of framework entry point job.run_test (or job.run_test_detail).

Understanding how to setup the entry point may be the most tricky and critical part to compose a control file. the Autotest document does not explictly explain the specification of job.run_test_detail. Luckily, because Autotest is written in Python, so it is easy to read the source code of Autotest and figure out how to use job.run_test_detail and what input means for each argument.

For example, job.run_test_detail('folder_name', test_name='test-name-of-your-job', tag='use-as-output-folder-suffix', timeout=60*3) means:

• Autotest will search for a python file named after folder_name.py under folder_name, and there is a class folder_name implemented in folder_name.py
• the corresponding output folder of result will be named after folder_name.use-as-output-folder-suffix
• the first argument of folder_name.py:folder_name.run_once is test-name-of-your-job

The control file should be under folder_name.

Compose a Sub-class of test.test

The structure is very similar to the concept of unittest, the Python default unittest module if you are familiar with unittest. For example, the necessary elements are:

• the folder_name class inherits test.test. In unittest the class inherits unittest.TestCase.
• the folder_name.setup method is optional and one-off until folder_name.version changes. In unittest we have setUp.
• the folder_name.run_one method is the entry point of the framework. In unittest we use the prefix test_ of a testing method name.

Invoke Your Client Job

Fetch the source of Autotest under HOME:

cd $HOME
git clone https://github.com/autotest/autotest.git
git clone https://github.com/autotest/autotest-client-tests.git
rm -rf autotest/client/tests
ln -fs autotest-client-tests autotest/client/tests

And then put your folder_name under autotest-client-tests

Place the command:

AUTOTEST_PATH=$HOME/autotest sudo -E autotest/client/autotest-local --verbose autotest/client/tests/folder_name/control

Essentially it means:

• the source tree root of the framework (as a python package) and the file hierachy root of the working directory is $HOME/autotest
• invoke the command autotest/client/autotest-local
• execute the job defined by autotest/client/tests/folder_name/control

That is! Isn't straightforward? ;)