SPRT returned null where expecting result to be certain (Pass/Failed)

[ricksu978]

For 400 games with 55% gating, I would expect the SPRT result to be

• True/Pass when win >= 220
• False/Failed when lose >= 181

Here are test cases found to be faulty

Win	Lose	SPRT returned	Expect
214	181	null	FALSE
215	181	null	FALSE
216	181	null	FALSE
217	181	null	FALSE
218	181	null	FALSE
220	172	null	TRUE
220	173	null	TRUE
220	174	null	TRUE
220	175	null	TRUE
220	176	null	TRUE
220	177	null	TRUE
220	178	null	TRUE
220	179	null	TRUE
221	173	null	TRUE
221	174	null	TRUE
221	175	null	TRUE
221	176	null	TRUE
221	177	null	TRUE
221	178	null	TRUE
222	174	null	TRUE
222	175	null	TRUE
222	176	null	TRUE
222	177	null	TRUE
223	175	null	TRUE
223	176	null	TRUE

Script to generate test cases

var util = require('./classes/utilities.js');
var tests = [
    { win: 214, lose: 181 },
    { win: 215, lose: 181 },
    { win: 216, lose: 181 },
    { win: 217, lose: 181 },
    { win: 218, lose: 181 },
    { win: 220, lose: 172 },
    { win: 220, lose: 173 },
    { win: 220, lose: 174 },
    { win: 220, lose: 175 },
    { win: 220, lose: 176 },
    { win: 220, lose: 177 },
    { win: 220, lose: 178 },
    { win: 220, lose: 179 },
    { win: 221, lose: 173 },
    { win: 221, lose: 174 },
    { win: 221, lose: 175 },
    { win: 221, lose: 176 },
    { win: 221, lose: 177 },
    { win: 221, lose: 178 },
    { win: 222, lose: 174 },
    { win: 222, lose: 175 },
    { win: 222, lose: 176 },
    { win: 222, lose: 177 },
    { win: 223, lose: 175 },
    { win: 223, lose: 176 },
];

tests.forEach(t => console.log(`SPRT(${t.win},${t.lose})=${util.SPRT(t.win, t.lose)}`));

@roy7
Have I misinterpret how SPRT should work? or maybe we need to hard code the boundaries to fix this.

[roy7]

Well the 55% / 400 game gating is our own imposed secondary test, not a true SPRT result. We could remove the gating checks and add all of this into the SPRT test perhaps but I think it's safer to keep the SPRT function clean and apply the gating over top of it? null result isn't "wrong". we just ignore it if our other gating decision is met.

What we do want eventually though is a stick pass on matches table to get people to stop complaining about it. :) Perhaps just a simple global Map set when we make the Networks table, setting very network with a selfplay game to true, then when making the matches table if the network1 in the Map is true, set SPRT to PASS for that match table row. (This will only be true for networks with selfplay games, meaning they did PASS at some point for sure.)