Education data reuse for AI development

Today the government announced that pupil assessment data will be used in a ‘store’ to build AI products.

We have written to both departments at DSIT and the Department for Education with questions regarding the data sourcing, as well as where this fits into their strategy for the education sector.

While resources for SEND provision and teacher retention are priorities for the sector, we hope the government is not fooled into thinking fifty-year old tech offers short cuts for serious thinking on the immediate needs and future long term holistic education strategy — because this is one of the first spending announcements in education and it’s not going to schools.

It is unclear from the announcement whether the total spend is cumulative or if the three and four million mentioned are separate or are just not adding up.

“The project, backed by £4 million of government investment, will pool government documents including curriculum guidance, lesson plans and anonymised pupil assessments which will then be used by AI companies to train their tools so they generate accurate, high-quality content, like tailored, creative lesson plans and workbooks, that can be reliably used in schools.”

“The content store, backed by £3 million, is a first-of-its kind approach to processing government data for AI, as the UK government forges ahead with using technology to transform public services and improve lives people across the country.”

Is it three, four, or seven in total?

We at Defend Digital Me are yet to see any products that claim to use AI that offer independent evidence of improved learning outcomes. In recent years we’ve seen a lot of companies with, shall we say generously, “over-hyped marketing” selling their products to schools — for example a well-known seating plan company, that claimed to be selling an AI product but it was not, as the ICO discovered in its assessment for misuse of pupil data to train an AI product without parental permission.

Furthermore, the announcement use of the terms, “personalised learning” and “cost cutting,” are contested in education and across the public sector and concerning that education is being used as a guinea pig for the “first of its kind” plans.

“Using technology to transform public services and improve lives people across the country,” are meaningless buzzwords — and a whole range of public sector services and people have been harmed by tech projects, whether directly or by omission of effort on other things, done badly by governments including in the UK.

Generative AI models are already losing their appeal to investors as accuracy and reliability issues are becoming ever more apparent, and they cannot legally use children’s data or be used by children. It is unsurprising therefore if companies are seeking to embed their products urgently into new markets and to exploit new data sources, so far protected from their re-use by not being online.

But it is unclear if this is, “government data” as the announcement claims (ie about government and admin processes) or is it our public administrative data (ie about people and our interactions with public services and therefore personal data)? “Assessment data” suggests it is pupil data.

Any personal data processing (including separating data to make parts of it non-personal or anonymous) needs a lawful basis that is necessary and proportionate to the task and cannot just be invented retrospectively to fit how the data users have chosen to do things, especially for product development, that does not enjoy the same exemptions as research would, for example.

Data controllers must inform people at the time when their data are *collected* how it will be used and why, and purpose limitation protects people from retrospective changes to that, which aren’t what people either agreed to or at least were told to reasonably expect at the time of collection.

Will the government inform past and present pupils and their families —whose data may be processed to create this store — of their rights, and about who will have access to what data about them to build it, or use it later?

Families don’t send their children to be turned into data products, and of those 1,004 parents Survation polled in 2018, 69% didn’t know a national pupil database even existed at all, never mind that they are in it or how to exercise their connected legal rights.

We will want to see more information about what is planned here for this “data store” and hope government engages soon and widely with the sector and civil society.

Jen Persson said,  “I’m keen to see technology used well, but so far the shiny promises and hype of AI companies seem to be steering UK government tech policy towards their own interests in new product development more than the reality of what is needed by the education sector — a sector that when it comes to technology needs initial teacher training and CPD, and a rights-based data-management infrastructure put in place first to fix what’s not working now, before any new tech initiatives.”

“We are further disappointed with the current government’s lack of engagement to date with expert civil society in data and technology — surprisingly poorer than their predecessors. But it will be for the schools sector to decide how it responds to protect the rights of the pupils and families who entrust their data to them for the purposes of their children’s education, and not for profit.”

“More information must follow soon to ensure public and professional trust in such new initiatives is not harmed, and it becomes toxic to trust in the new government overall.”