c++ - Testing wall clock time based algorithms -

i'm maintaining algorithm uses wall clock time make various decisions (eg. solutions quick calculate taking long , need scrapped).

when trying test algorithm, results can different each time due number of variables such machine load, operating system scheduling, io etc.

what standard approach testing such system? cpu instructions executed 1 idea had, i'm not sure how practical on modern multi-core x86 processor.

the fall plan add increments internal counter , change limits of algorithm try , match performance of existing wall clock version. involve lot of trial , error i'd know if there easier way before start going down path.

the simple crude option "abstract away" wall time retrieve logic.

say, use class walltime gettime method , use throughout application.

there 2 "providers" of time class can use. 1 rt clock in system.

the other returns values pre-recorded list.

you record first pass through algorithm , store values returned gettime. these values form "pre-recorded" list of values second "time provider".

assuming second run make calls gettime in exact same order can return same time values in first run :)

you can edit list if want adjust timings. can have multiple stored list, simulate different hardware.

example:

suppose algorithm works this:

1. gettime -> returns t1 clock
2. call function x
3. gettime -> returns t2 clock
4. decide function call next (e.g., y) based on (t2 - t1)
5. gettime -> return t3 clock
6. call function y
7. gettime -> return t4 clock ...

after save t1, t2, t3 , t4 in list can replay above run (with regard values returned gettime).

this solution fail if function executed @ step 6 (which 1 algorithm selects based on performance) not same in second run if (t2 - t1) time difference same, i.e. depends on variable, non-time related parameter.