Adding some statistical tools to enable testing of a game turn that uses random values.
In the last post, Rewriting and Refactoring Hamurabi.bas in Javascript Part 2, I did some testing, and stopped just before writing the test to play a turn.
The tests for most of the functions are straightforward. The functions don’t carry state, so it’s easy to inject the state, and check the results. You put in some input, and expect an output; the output never changes.
This is not the case for the random() function. Random() returns a value from 1 to 5. Yes, “what the hell?” is what I thought as well, but that’s how HAMURABI.BAS did it. To test random(), I ran the function many times, tested that it doesn’t go out of range, and then took a histogram, and logged that. I could visually see the distribution of values and verify that they weren’t all over the place. (It turned out that they were a little wild, but, I’m not going to worry about it.)
Here’s the code:
[code]
function test_random() {
// the random() function should return an int value between 1 and 5, inclusinve.
res = random();
test_assert( res == parseInt(res), "random is not an int" );
// run random 100 times, and test that all of them are in range
function in_range(a) {
return (a<=5 && a>0);
}
test_assert(in_range(res), "random result " + res + " not in range");
ress = [];
for(i=0; i<100; i++) {
ress.push(random());
}
function and_all(a, b) { return a && b; }
test_assert(ress.map(in_range).reduce(and_all), "random returned a bad result");
// i don’t have a test for randomness, so i just dump a historgram
hist = [0, 0, 0, 0, 0];
ress.map(function (i) { hist[i-1]++; });
console.log("Histogram of output of random()", hist);
}
[/code]
Like random(), play_turn() has randomeness. You call it with one value, and the results are different each time. The input is an object with five properties, and the output is an object with a bunch of properties, but some of them involve a call to random().
Testing play_turn() required running the function many times. Looking at the results is not reasonable, so we need to use statistics to determine if the output looks correct over dozens or hundreds of runs.
So, I wrote a couple statistical tools.
[code]
/**
* Statistical tools.
*/
function test_range(a) {
if (! a.length) throw "test_range expects first arg to be an array";
max = a[0];
min = a[0];
len = a.length;
for(i=0; i<len; i++) {
if (a[i] > max) max = a[i];
if (a[i] < min) min = a[i];
}
return { range: (max-min), min: min, max: max };
}
/**
* Calculates the mean and variance.
* Note that the variance is a sample variance, not a population variance.
* Generate a lot of samples so it’ll give a good value.
* (Semantically, we should be using the population variance, but spreadsheets
* use the sample variance formula, so it’s easier to check against a spreadsheet.)
*/
function test_mean_variance(a) {
if (! a.length) throw "test_range expects first arg to be an array";
sum = 0;
len = a.length;
for(i=0; i<len; i++) {
sum += a[i];
}
mean = sum / a.length;
sumdiff = 0;
for(i=0; i<len; i++) {
diff = a[i] – mean;
sumdiff += diff * diff;
}
variance = sumdiff / (a.length – 1);
return { mean: mean, variance: variance };
}
[/code]
I was thinking of rolling them all into one function, for speed, but that made the code long. So I just grouped the functionality into two functions. I didn’t want to make individual functions for min, max, etc. because each requires looping over the entire array. So, it’s a trade off for speed and readability.
I chose variance rather than standard deviation because we aren’t dealing with a normal distribution.
I haven’t yet written the test, but here’s a starting point.
[code]
function test_play_turn() {
// A "turn" reads the state, and applies a few properties to "play" the turn:
// plant – number of acres to plant.
// feed – number of people to feed.
// The value of stores has already been modified before it’s been played.
st = {};
test_assert_fails(play_turn, {}, "0 play_turn should have failed on empty object");
gamestate = { population: 100, stores: 100, year: 1, acres: 100 };
plan = { plant: 100, feed: 100, stores: 0 };
st = Object.assign({}, gamestate, plan);
// play a turn 100 times and see if any values are weird.
results = new Array();
for(i=0; i<100; i++) {
results.push(play_turn(st));
}
console.log(results); // the statistical tests will go here
}
[/code]
To finish this part, I’ll need to write a “reshape” function to turn an array of game results into an object with arrays of results for each property.
Each property could then be analyzed, and tested to make sure the ranges and variances are reasonable.
I’d also need to know what my reasonable values are. That seems a little harder than writing this code.
The full code is attached. Open it in a text editor to read: