YADI, part 3: timedtest, a decorator for time performance tests

Grig Gheorghiu has released a pretty cool tool, called pyUnitPerf. It's a
port of Mike Clark's JUnitPerf.

This tool runs your tests cases, and measure the time they take to run. It
can pop a Failure when the tests lasts too long. It also has a Load runner,
but nothing comparable to the amazing funkload tool

Thats quite useful in some cases, when your test fixture does not make the
code run slower, like it happens in ZopeTestCase.

So since pyUnitPerf associates a max time with a TestCase class, and since
test fixture can slow down the tests, it makes it really hard to use it, to
measure a distinct test method.

So I have taken back the idea and put it in a decorator that I can use on
all test methods I want to control, performances wise.   

The timedtest decorator

import unittest
import time

# might be changed to allow tolerance
# if tests are run on other boxes
TOLERANCE = 0.05

class DurationError(AssertionError): pass

def timedtest(max_time, tolerance=TOLERANCE):
    """ timedtest decorator
    decorates the test method with a timer
    when the time spent by the test exceeds
    max_time in seconds, an Assertion error is thrown.
    """
    def _timedtest(function):
        def wrapper(*args, **kw):
            start_time = time.time()
            try:
                function(*args, **kw)
            finally:
                total_time = time.time() - start_time
                if total_time > max_time + tolerance:
                    raise DurationError(('Test was too long (%.2f s)'
                                           % total_time))
        return wrapper

    return _timedtest
    

Usage

It's quite easy to use, just decorate a TestCase method with it, giving the max time allowed for the test, in seconds:    

class MyTestCase(unittest.TestCase):

    def __init__(self, name):
        unittest.TestCase.__init__(self, name)

    @timedtest(2)
    def test_timecritical(self):
        time.sleep(1.5)

    @timedtest(1)
    def test_timecritical2(self):
        time.sleep(1.5)

if __name__ == '__main__':
    suite = unittest.makeSuite(MyTestCase)
    unittest.TextTestRunner().run(suite)

Now this can help much to make sure some critical code does not get too
  slow.
  

[tziade@Tarek Desktop]$ python timedtest.py
.F
======================================================================
FAIL: test_timecritical2 (__main__.MyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "timedtest.py", line 28, in wrapper
    raise DurationError(('Test was too long (%.2f s)'
DurationError: Test was too long (1.50 s)

----------------------------------------------------------------------
Ran 2 tests in 2.999s

 

(Post originally written by Tarek Ziadé on the old Nuxeo blogs.)