Capita SIMS Autumn 2017 pupil data update caused data corruption

What’s the story?

Capita SIMS — information management supplier in the UK to 21,000 schools — appears to have a serious national schools’ data and IT issue affecting potentially hundreds of thousands of children’s records in England and Wales.
It concerns the corruption of a software matching routine for new pupils since a Capita SIMS software system upgrade distributed in November 2017.
The problem has only just come into the public domain. Schools are reportedly hearing second hand as Capita has only contacted their contract holders, not all schools. While Capita says there is a tool to help schools *identify* the potentially problematic records, Capita is not offering a way to make the corrections.
Some users report, “we’re finding a lot of the listed issues aren’t wrong so not entirely sure which is and isn’t without reading through every applicant forms.”
Schools/LA/MATS and potentially distributed recipients such as app developers, researchers and crucially after the Spring 2018 census, the Department for Education will have to dedicate manual hours to review and make potential fixes.

The consequences of the software update

Contact information for the incoming pupil for example, address, telephone number and email address, may have become associated with other pupil’s records, or the new pupil could themselves be linked to the wrong, more recently added, contact details. The problem could have impacted pre-admissions, pupils on roll and the records of school leavers.
At local level: Mistakes may have meant incorrect contact email or home address used to send information and communications to the incorrect recipient. The SIMS Autumn 2017 upgrade was released on November 6, 2017 as “an essential upgrade in order to run the Spring 2018 census“. Incorrect data has likely been passed on, in both the Spring census and since Common Transfer Files will all recently have been done for all pupils, for the coming September new school intake and Year 7 transfers.
Hampshire has reportedly told schools that, “until we receive more clarity on a way forward from Capita, we cannot identify or fix any records that have already been corrupted. To avoid potential data breaches it is therefore vital that schools do not send out communications that rely on the contact information in SIMS without appropriate additional checks to ensure all contact details are correct for each individual recipient.”
At regional and national level: If these incorrect data including children’s addresses, have been passed on in the Spring 2018 or Summer censuses then national pupil data at Local Authorities and at the Department for Education, other Departments, or other third-parties could be affected.
The Capita SIMS response to schools has been so far reportedly, that they had no plans to inform the ICO and that this is a schools’ responsibility — despite what appears to be the case, that the provider may have failed to notify schools in six months.
The task is for schools to find the issues by using the Database Diagnostics check 99 (patch 24157), then manually working through the list fixing the issues found.
On its website, Capita’s Education Support Services appears to downplay the issues, saying that “only a small number of cases have been reported to our help desk”. 
[Source: http://www.capitaess.co.uk/home/index.asp]
If they knew this was an issue already, why did they not contact schools to point out the errors needed fixing? If they didn’t know, there is an inadequate testing and audit process in their system design, and they should not be making any comment on the scale of the problem for now.
One school user on Edugeek reported:
Our support provider, Somerset County Council, notified us of this about 2 or so weeks ago, and have been working through that time on a way to find the problematic records – which they now have done. There’s no “fix” for the broken records, that’s a manual task. We have nearly 100 pupils identified with incorrect contacts… My issue is that we are now having to put our staff time into fixing this mess, which was caused by Capita. We should either get some compensation for this, or they should be doing it for us!
Onward data sharing means third-party data audit and potential correction is needed, including MATS, Local Authority corrections, DfE, and other potential third-party distribution since December 2017.
 

Our 
Questions for Capita

If schools don’t know it is a problem, how can they be expected to identify the issues? Capita says that they sent a national customer communication last week, but because many contracts are not at school level, but at MAT or LA level for example, onward communication is required and action which may go unnoticed at the end of a summer term and academic year.
How does Capita suggest onward distributed errors be identified and corrected?
Who pays for the staff time and system costs incurred by potentially 21,000 schools and onward third-parties? Legal liability *should* be set in contract with minimum service level agreements and ensure accountability sits with all MIS providers for such issues and correction processes. If not now, then in future.
 

What’s next?

Capita is reportedly offering some users a webinar, and at our suggestion is creating an FAQ for their website. Errors will need correction across the sector and where it has become contaminated in distribution.
Children and parents are the only people who can reliably identify inaccurate data and provide corrections. A process for fixes and regular audit is needed for correction of inadvertently shared errors and general data accuracy. Pupil-level Data Output Reports must be possible to generate on demand so schools can meet Subject Access Requests and Technical reports made suitable for IT and user audit purposes.
The full scale and effects are not yet known. The ICO is investigating. Further questions remain on what the company knew and the timing of the announcement of a new national testing contract.
More will follow.

---

Link to Edugeek forum IT / data managing school staff comments, concerns and questions.
Link to Capita’s public facing comment since they first posted July 12, 2018.
Press mentions
Schools Week 20 July 2018
Capita admitted SIMS bug on day it was handed £109m primary testing contract
The Register 17 July 2018
Capita strikes again: Bug in UK-wide school info system risks huge data breach
Schools Week 17 July 2018:
Pupil data corrupted after bug hits Capita SIMS

---

In Capita’s own words

What is the issue?
Incorrect matching of applicants and on roll students imported by CTF into your SIMS7 database may have linked new pupil records to an existing contact. Additionally, some data on the existing contact record may have been changed.
Will this issue have impacted my school?
Not all schools will be impacted.
Under what circumstance would my school be impacted?
This would happen if a combination of the full name, title, gender and relationship of a pupil’s contact record in the CTF matched the same elements of the contact of another applicant, on roll pupil or leaver already held in your database.
An example of this would be:
Already on SIMS: Pupil = Joanne Bradshaw, with a Parent = Wendy Bradshaw, Mother, Female (original)
New CTF applicant: Pupil = Jamie Bradshaw, with a Parent = Wendy Bradshaw, Mother, Female (new)
As in this instance both pupils have a parent called Wendy Bradshaw, identified also as being Female and with a relationship of Mother, the new CTF applicant would be linked to the original Wendy Bradshaw. The original Wendy Bradshaw address, telephone number and email would also be updated [replaced] with the new [and incorrect] Wendy Bradshaw’s details.
[defenddigitalme notes that the wording could suggest *all* these factors needed to match. Edugeek users experience indicates that errors are happening where only *any* combination of the elements match, not all.]
What is being done to resolve?
We have supplied a fix by either applying patch 24068 on the Spring release or upgrading to the Summer release. This issue has the highest priority and we are currently producing a guide to support you in easily identify whether this issue has effected your data and to support you with the checking of your data.
What should you do
The first step is to fix the bug by either applying patch 24068 on the Spring release or upgrading to the Summer release. The second step is to apply patch 24157 which will add Check 99 to the Database Diagnostics tool. This tool enables you to identify the potential problems with a school’s contacts and contacts addresses that are likely to have been caused by our contacts/address bug. We [Capita SIMS] are currently producing a guide to support you with the checking of your data. Please note we have applied both patch 24068 and 24157 for HOSTED SIMS customers.
— end of Capita’s communication —