Testing the central function of Hamurabi, the game logic that carries out the plan for next year, and adds some random catastrophes, was coded up. This had me a bit confused.
As promised in the previous story, Rewriting and Refactoring Hamurabi.bas Part 3, I created a test that runs the play_turn() function many times, and gathers some stats.
[code]
function test_play_turn() {
// A "turn" reads the state, and applies a few properties to "play" the turn:
// plant – number of acres to plant.
// feed – number of people to feed.
// The value of stores has already been modified before it’s been played.
st = {};
test_assert_fails(play_turn, {}, "0 play_turn should have failed on empty object");
gamestate = { population: 100, year: 1, acres: 100 };
plan = { plant: 100, feed: 100, stores: 100 };
st = Object.assign({}, gamestate, plan);
// play a turn 100 times and see if any values are weird.
results = new Array();
for(i=0; i<100; i++) {
results.push(play_turn(st));
}
results = test_array_to_columns(results);
dump_results(results);
function dump_results(results) {
for(key in results) {
range = test_range(results[key]);
variance = test_mean_variance(results[key]);
console.log(key);
console.log(range);
console.log(variance);
}
}
}
[/code]
As expected, the numbers didn’t make any sense to me.
I’m not the Hamurabi pro. I’m not even good at playing the game, because I never really dug in and figured out the internal formulas.
So, what exactly am I testing? Am I testing that it runs at all? Should I test to see that values are reasonable? I have no spec from which to develop tests, so, should I go into the code and try to write a spec?
Should I gather results, and then compare them, to look for wild variations?
Enter Random Testing
Rather than tackle the problem of devising good tests, we can fall back on some random testing, where we generate inputs and run them through the software, until we encounter an error.
There’s a good article about it at C2: Random Testing.
What I did was make a new web page with code to generate a bunch of random data. What we want is to generate a bunch of objects that look a little like this:
[code]
{
population: 100,
year: 1,
acres: 100,
plant: 100,
feed: 100,
stores: 100
};
[/code]
To make this a little more readable, we’d want some functions to generate each type of value. So we might have random_population(), random_year(), random_acres(), etc.
These functions are similar, and there’s a nice Javascript technique where you return a function as the result of a function. It’s called a “closure” in functional languages like JS, and “function objects” in object oriented languages.
In ES6 there’s a new feature called Generators that does roughly the same thing, but for returning a stream of values. I didn’t opt to use that feature, but this is similar.
We’ll define a function called generate_random_int_func() which takes some arguments specifying the range of values. It also has a way to specify values that we want to return more often than other values. These are the favorites and bias arguments.
With favorites and a bias, we can tell it to pick values from an array like this: [ -1000, -1, 0, 1, 1000 ]
Those are some typical edge cases and bad values. If we set the bias at 0.25, then 25% of the time, it’ll pick a value from favorites.
Here’s the code and test code:
[code]
function generate_random_int_func(low, hi, favorites, bias) {
var range = hi – low + 0.99;
var max = favorites.length;
var result = function () {
if (Math.random() > bias) {
return parseInt(Math.random() * range) + low;
} else {
return favorites[parseInt(max * Math.random())];
}
}
return result;
}
function test_generate_random_int_func() {
var r = generate_random_int_func(-5,5, [-1,0,1,1000], 0.25);
var hist = {};
for(i=0; i<1000; i++) {
x = " " + r();
if (!hist[x]) { hist[x] = 0; }
hist[x]++;
}
console.log(hist);
}
[/code]
The output of the the test code looks like this:
[code]
" -5": 64
" -4": 69
" -3": 67
" -2": 58
" -1": 157
" 0": 133
" 1": 134
" 2": 47
" 4": 66
" 5": 70
" 1000": 65
[/code]
That’s a histogram of random values returned by the function defined by generate_random_int_func(-5,5, [-1,0,1,1000], 0.25);.
I’d say that looks pretty good. Not great, but, OK. I don’t know what’s up with Math.random().
By the way, there’s a gotcha in the code. You must always declare your values within a function with “var”. If you omit “var”, you’re defining a property on the window object:
[code]
function x() {
var y = 100; // defines a local variable y
z = 100; // sets the value of window.z to 100.
}
[/code]
I raise this because, in my prior code, I forgot to do this. It didn’t show up as a problem, because the code just took over the entire environment.
I could have avoided all this headache if I’d just used “use strict”; at the top of my code.
The attached code has “use strict”; enabled.
Going Crazy with Closures
Here’s the code to generate a random object. I didn’t really pick great numbers, but we’ll see what happens.
[code]
function generate_play_turn_object() {
var pop = generate_random_int_func(10, 2000, [-1, 0, 1], 0.10);
var yr = generate_random_int_func(1, 11, [], 0);
var acr = generate_random_int_func(0, 400, [], 0);
var plnt = generate_random_int_func(0, 400, [], 0);
var fd = generate_random_int_func(0, 400, [], 0);
var str = generate_random_int_func(0, 2000, [], 0);
var result = function() {
return {
population: pop(),
year: yr(),
acres: acr(),
plant: plnt(),
feed: fd(),
stores: str()
};
};
return result;
}
r = generate_play_turn_object();
console.log(r());
[/code]
The top half of the function defines a bunch of random number functions.
result is a function that emits a randomized object that can be passed into the play_turn() function.
Here’s an example output:
[code]
acres: 136
feed: 178
plant: 367
population: 312
stores: 860
year: 7
[/code]
The next step is to generate a zillion test cases. However, before I proceed with that, I’m going to refactor the code into three separate files. I’d like to be able to test the random objects on the fly, before I generate the test cases. More on this in the next post.