Upper Tribunal Administrative Appeals Chamber

[2023] UKUT 157 (AAC)

Fecha: 05-Jul-2023

What was the evidence of ‘tutors’ advantage’?

What was the evidence of ‘tutors’ advantage’?

32.

At [72], the FTT decision recorded Ms Walton’s oral evidence that the “detail” of the requested information “would give Tutors an advantage and their clients would benefit from this unfair advantage”; and that the number of questions had always been kept from the wider public knowledge for good reasons - as it would greatly assist private tutors and give their client pupils an unfair advantage.

33.

[72] also records Mr Hilton’s oral evidence that the number of questions needed to be kept out of public domain, as those concerned could by means of reverse engineering provide commercially sensitive information; which would provide an unfair advantage to private tutors.

34.

[73] states that “the evidence before [the FTT]” was that 10 year olds (i.e. those sitting the tests) would be unlikely to be able to work out number of questions in the test; it also records Mr Coombs as disagreeing, and supporting his position by reference to an earlier FTT decision (Reading School v IC EA/2013/0257) (which, as the FTT decision said, was decided on its own facts and was not binding on it).

35.

Paragraphs 21-23 of Mr Hilton’s 31 March 2021 witness statement explained ‘tutors’ advantage’ thus:

21.

If the number of items per section and completion rates of sections were publicly known, then tutors would be able to create targeted materials to coach students based on this information. This is because tutors could ascertain insight into which point in the test do students generally drop off in terms of completion. If, for example, they see that most students drop off completion at question X, they could advise their students to guess the answers from that point in the test if they are short of time because other students won’t have answered those questions at all. This creates an unfair advantage.

22.

If the mean raw scores were widely known, then tutors would use this information to coach students to aim for a certain raw score mark. This would show tutors how difficult the test is, thereby giving them a framework for coaching students.

23.

The reliability figures are strongly affected by the number of items in any given test, so if this information was in the public domain, this information could also be used by tutors to predict the number of items in a test, which, as described above, would help tutors coach students towards an optimal raw mark.

36.

Paragraph 36 added:

There is a legitimate concern (as highlighted above) that the information requested about the test (and contained in the PowerPoint presentation) could, if disclosed to the public, be used to ascertain which areas of the test to give more focus to when preparing children to take future tests in order to maximise marks. This would be particularly advantageous for 11+ tutoring organisations, and those children whose parents can afford to engage those tutors.

37.

Counsel’s note of Mr Hilton’s evidence in “closed session” said:

Q: Why would it benefit tutors?

Mr Hilton responded that in giving information about completion rates, the information also gives the number of questions. This would show where candidates tailed off and tutors could tell students from where to guess the rest if time is low. It would enable coaching techniques to be used

Further, if one has a good foundation in statistics one could deduce the number of questions from the completion rates

38.

Ms Walton’s 31 March 2021 witness statement said this:

“24.
As part of their work tutors try and work out as much as they can about the [test] so that they can tailor their approach to what may be in the test to benefit the children that are being tutored. So, the types of question being asked are of interest for example. Other information such as the number of questions in the test or in a particular section is also useful as tutors can then get children to practise answering a fixed number of questions in a given time frame. Although tutoring is widely available, many children do not have tutoring and many families cannot afford the costs involved. The test must be fair to all children and children from poorer families should not be disadvantaged. It is therefore really important that details about the content and construction of the test are kept confidential to minimise any advantage of tutoring. Test materials are delivered to test centres securely and are stored securely during and after testing before being returned to GL Assessment where they are pulped. Additionally, all involved in testing agree to not discuss the test. Children are also told not to do so at the end of the test.
…
26.
The number of questions in each test paper and section is not published. Claims by Mr Coombs and others that this information is known are incorrect as evidenced by the inaccurate numbers of questions that they claim are in our test. The way our test papers are numbered also means that children would have to count up the number of questions in each section of each paper manually and then remember the total. This is highly unlikely and also assumes that children have time to spare at the end of the test paper. At the end of the test children are also specifically told: You are not allowed to discuss the test questions with anyone once you have left the room. Test centres are similarly required to agree to keep the test materials confidential at all times.”

39.

I take from the foregoing that following can be said of the evidence before the FTT of ‘tutors’ advantage’

the evidence for ‘tutors’ advantage’ was that of Mr Hilton and Ms Walton;

the evidence was that the advantage was derived from tutors discovering (as result of disclosure of the requested information) the number of questions in the test;

the FTT decision did not itself explain how knowledge of the number of questions in a test gave rise to ‘tutors’ advantage’, but, on looking at the evidence before the FTT, one finds the following explanation of how the advantage was said to arise:

per Mr Hilton, the advantage would arise by tutors coaching candidates that, after reaching question number X (being the question after which candidates started to “tail off” – and could only be ascertained if the number of questions was known), they should (if short of time) start to rapidly answer (i.e. guess) the remaining questions (the tests appear, from evidence in the FTT’s open bundle, to be “multiple choice”);

ii.

per Ms Walton, the advantage would arise by tutors getting children to practice answering a fixed number of questions in a given time frame.