From Fedora Project Wiki

Revision as of 16:34, 13 August 2009 by Wwoods (talk | contribs) (→‎Test Objects: Getting test results: system_output raises an exception if the command fails)

Will's notes (to be integrated into main page)

Control Files

The control file is actually interpreted as a Python script. So you can do any of the normal pythonic things you might want to do.

Before it reads the control file, Autotest imports all the symbols from the autotest_lib.client.bin.util module.[1] This means the control files can use any function defined in common_lib.utils or bin.base_utils[2]. This lets you do things like:

arch = get_arch()
baseurl = '%s/development/%s/os/' % (mirror_baseurl, arch)
job.run_test('some_rawhide_test', arch=arch, baseurl=baseurl)

since get_arch is defined in common_lib.utils.

Test Objects: Getting test results

For simple tests, the result of the test is determined by the exit code of the command. Usually you'll run the test like this:

self.results = utils.system_output(cmd, retain_output=True)

If cmd is successful (i.e. it returns an exit status of 0) then utils.system_output() will return a CmdResult object. Otherwise it will raise error.CmdError. This means that if the test command fails an exception will be raised, and the test will exit immediately unless you handle it..

Further test-level info can be returned by using test.write_test_keyval(dict):

extrainfo = dict()
for line in self.results.stdout:
    if line.startswith("kernel version "):
        extrainfo['kernelver'] = line.split()[3]
    ...
self.write_test_keyval(extrainfo)
  • For per-iteration data (performance numbers, etc) there are three methods:
    • Just attr: test.write_attr_keyval(attr_dict)
    • Just perf: test.write_perf_keyval(perf_dict)
    • Both: test.write_iteration_keyval(attr_dict, perf_dict)

Test Objects: Attributes for directories

test objects have the following attributes available[3]:

outputdir       eg. results/<job>/<testname.tag>
resultsdir      eg. results/<job>/<testname.tag>/results
profdir         eg. results/<job>/<testname.tag>/profiling
debugdir        eg. results/<job>/<testname.tag>/debug
bindir          eg. tests/<test>
src             eg. tests/<test>/src
tmpdir          eg. tmp/<tempname>_<testname.tag>