Case No. UKUT-00450(IAC)

of the data following completion of the test is not consistent: in particular, the description provided in respect of the Applicant Mr Mohibullah has not been put forward in either of the other two cases.The integrity of the test taking procedures and systems established by ETS in its manuals depends heavily on the reliability and probity of test centre staff. Further, the ETS security precautions concentration on the elicit conduct of candidates and not test centre employees.With the sole exception of audio files, all of the computer files produced have been in the form of “print-out to PDF”: the effect of this “… has been not to preserve any original date – and – time stamps or internal metadata either or both of which would have assisted analysis using digital forensic analysis and helped produce a chronology of events”.The test centre seating plans which have been produced are incomplete.A study of the spreadsheets attached to the witness statements of the Home Office employee, Mr Sewell reveals a lack of any nexus between the data supplied to him by ETS and the unique ID of individual candidates. As a result, the experts say “We do not know the processes by which the candidate’s name is linked to each test”.The experts acknowledge the documentary evidence of “simple impersonation”, with particular reference to the unannounced ETS audit at Synergy College on 16 January 2013. They express the opinion that the simple impersonation mechanism would be “vulnerable” in any speaking tests.While there is also some evidence of “dictated answers”, “viz” answers to test questions being called out by a person in the examination room, this method would not be viable or the spoken English test.The investigation of a particular test centre in Birmingham established the use of the “remote control software” mechanism by the use of “Team Viewer” software whereby a person using another computer could secure access to the computer being used by the candidate. The possibility of other, covert, remote control mechanisms is acknowledged. There is no evidence of the use of any of these mechanisms in the test centres which relate to these Applicants or the Appellant MA.The experts also advert to the possibility of manipulation of file responses held on the local server, the CBT Manager, at the testing centre. If file responses were stored on this server, this would create an opportunity for alteration by test centre staff. Two of the experts opined that this was unlikely.Yet another mechanism, entailing a simultaneous testing session using proxies in a “hidden room” at the test centre or elsewhere is acknowledged.According to the experts, “particular opportunities for mistakes appear to arise if the actual registration on the ETS system is sometimes carried out by test centre staff and not by the candidates themselves”, creating the risk of the data provided by the test centre to ETS mis-matching the candidates and their tests. There was no security precaution available to counter this risk, with the exception of an unannounced ETS audit.As none of the computers or data media associated with the test centres involved in these cases is available, there is no information relating to the important issues of audit, log and configuration files and related time and date stamps. This is one aspect giving rise to the recurring lament of the experts: “We have been limited by the quantity and quality of material actually available to us.”The “naming conventions” for the digital files of the voice recordings produced do not provide an explicit link between the candidate and the recording: rather, there is only reference to the particular test being taken. Contrary to a suggestion emanating from ETS via their solicitors, the file name does not include the candidate’s “unique registration code”. Thus:“… What this naming system does is to provide linkage between a registered candidate and the responses and recording but assumes that the unique registration code is reliably linked to the real candidate. As we have already pointed out, in the two spreadsheets exhibited by Adam Sewell there are no columns uniquely to identify candidates by reference to the ID they originally tendered (e.g. the passport number).”Next, it is observed “The experts have examined the supplied audio files and find that there is no embedded metadata which might assist their enquiries. Time and date stamps appear to be of the most recent copying of the file and not of the point of origination”.The experts’ consideration of the report generated by an unannounced audit of Synergy College on 15 May 2012 highlights that while the auditor expressed “mild concern”, no specific remedies or sanctions vis-à-vis the college were proposed.(16) In the MA appeal, two of the experts, Mr Stanbury and Professor Sommer, gave evidence to the Tribunal. Their oral evidence was confined to certain discrete issues and themes. The choreography of the judicial review cases and statutory appeal resulted in no objection to the evidence particular to one case being considered in all three cases.(17) Mr Stanbury, in his evidence, highlighted the following matters in particular:(d) The absence of any evidence that the security mechanism of password protection vis-à-vis candidate’s test computers was in operation.(e) The “hidden room” theory could involve the falsification of the completed tests of both genuine and fraudulent candidates.(f) Whereas the speaking and writing TOEIC tests, which were undertaken at a single session, were fully computerised, the listening and reading tests, also undertaken at a single session, were manual.(g) There is no evidence of any audit logs. An “audit log” is a computerised record which would demonstrate the chain of storage, handling, processing and transmission of the data generated by the speaking and writing tests (our formulation). (h) Metadata, if they existed, would be located inside the voice recording files: there are none. As a result, these files do not contain particulars of the time, date and location of the recordings therein stored.(i) Finally, Mr Stanbury’s expectation was that there would be in existence certain contemporaneous manual records, relating particularly to the names of candidates and the desk number allocated to each: there are no such records.(18) Professor Sommer further testified that the evidence fails to disclose whether the important act of uploading the files generated by the speaking and writing tests occurred automatically or involved some human intervention. He agreed that if human intervention was part of this process, this would have created an opportunity for manipulation of the files, particularly if there was a time lag. The latter could occur through, for example, a loss of internet connectivity, whether false or genuine. Finally, Professor Sommer focused on the issue of photographing TOEIC test candidates. His evidence was that he “never got to the bottom” of this. While this issue receives some consideration in the ETS test centre manuals and the witness statements of ETS employees, these sources are incomplete. In response to a question from the panel, Professor Sommar stated that the description of the Appellant MA in evidence of group photographs following completion of the test exercises bore no resemblance to what is specified in the manuals.(19) At this juncture, we would observe that while the joint memorandum of the three expert witnesses and the oral testimony of two of them have, inevitably, focused attention on certain discrete issues and themes, we have considered in their entirety the experts’ reports and all of the documentary evidence bearing thereon.