Atheist is a simple framework for running test cases. You write small files in Python language using a set of predefined functions and classes. In many senses, the concept is quite similar to make or the SCons framework although Atheist isn’t a building system at all.
The Task object is the minimal testing unit. Each Task instance defines an individual execution (usually a shell command) that may be checked for its success termination. The Task class accepts many parameters that may change the test exception behavior in several ways. The only mandatory parameter is ‘cmd’ that is the command to exec.
The Task object is responsible to execute the command and check its termination value:
Task('true')
The Task instances need to be defined into text files. Although these files are written in a subset of Python language, they may be seen as declarative programs. You tell Atheist what you want to test and even the order, but the decision about when to run the corresponding action is taken by Atheist; some of them may be never done.
The file does not define sequential imperative sentences. For example, if you write this in a .test file:
Task('ls /')
print "hello"
the “print” statement will be executed BEFORE the “ls” command.
Any Task constructor accepts the next key-val parameters. All of them are optional. Beside the parameter name appears its type and default value.
check – type: bool, default: True
If ‘check’ is False there is no consequences when the Task fails.
cwd – type: str
Directory for the task execution.
delay – type: int, default: 0
Waits for ‘delay’ seconds to launch the task command.
desc – type: str
One-liner textual task description.
detach – type: bool, default: False
When detach is False the next Task does not starts until the current Task ends. When detach is True the next Task is executed even the current one is not finish.
env – type: str:str map
A dictionary with shell environment variables.
expected – type: int
Expected program return code. It may be negative if the process is killed by a signal.
tid – type: str
It is a unique string Task IDentifier. You can get a reference to the Task later giving this value to the get_task() function.
must_fail – type: bool, default: False
When the you expect that the program terminates with error but the return code is not known (i.e: is different that zero).
template – type: Template list
A list of templates. See Templates.
timeout – type: int, default: 5
The maximum Task execution time (in seconds). When the timeout finish, Atheist sends the programmed signal to the process.
save_stdout – type: bool, default: False
Store the process stdout in an actual file. If the stdout parameter is not set, Atheist creates a unique name for the file.
shell – type: bool, default: False
Execute the command within a shell session. bash is the used shell.
signal – type: int, default: SIGKILL
It is the signal that Atheist sends to the process when the timeout finish.
stdout – type: str
Is the file name to save the process stdout. Setting this parameters implies save_stdout = True.
The template is a set of predefined values for the Task key-values. You may use the same configuration for many tests avoiding repeat them. This is an example:
t1 = Template(timeout=2, expected=-9)
Task('foo', templates=[t1])
Task('bar', templates=[t1])
Both tests will be automatically killed after 2 seconds and the expected return value is -9. That means these process receive the SIGKILL(9) signal.
The conditions are predicates (actually functors) that check for specific checks. Conditions may be specified to be checked before (pre-conditions) or after (post-conditions) the task execution. If any of the condition fails then the task fails. This is an example:
t = Task('foo')
t.pre += FileExists('/path/to/foofile')
t.post += FileContains('path/to/barfile', 'some text')
There are two available Task specializations:
Shortcut for detached tasks. Predefined parameters are:
The testfiles may include some useful substitutable variable. You must write the symbol ‘$’ preceding one of the next words:
Note
See variables.test as an example.
You may write tasks and tests to execute before and after each test file. To do this, you need to put this tasks in files called as setup.test and teardown.test.
When your command create files you may track them for automatic cleaning. Just add filenames to the Task gen attribute. Here’s an example:
t = Task('touch foo')
t.gen += 'foo'
You may specify one o more filenames (a string list).
If you want the generated files don’t be automatically removed for manual inspection of results, you must specify the --dirty option (see below). To clean-up these files later, specify -C option.
[ToDo]
[ToDo]
You may use Atheist to monitor the well working of any application. It may send you notification when some is wrong. The easiest way is to run a testcase with cron specifying one --notify-* command-line argument. At the moment, two notificators are implemented:
The destination is a email account using the SMTP protocol. Atheist does not require a SMTP server or smarthost. You only need to configure an email account that will be used by Atheist to send mail to any destination. That information must be written in the .atheist configuration file. This is an example:
[smtp]
host = smtp.server.org
port = 587
user = atheist.notify@server.org
pasw = somesecret
You must indicate a jabber account that will be used by Atheist to send notification messages to any other jabber account. The destination account must accept this contact previously. The next is an example for the configuration in the .atheist file:
[jabber]
user = atheist.notify@server.org
pasw = anothersecret
To ask for a notification you just need to specify a test file and one or more destinations. For example:
$ atheist --notify-jabber John.Doe@jabber.info test/some_test.test
It is possible to give several –notify-* arguments in the same command-line to send notifications to different destinations.