IL-2023-000007 - [2025] EWHC 2863 (Ch)

The Expert Evidence as to the scope for generation of watermarks*

In the Agreed Technical Primer, the Experts record that models such as Stable Diffusion can be prone to what is called memorization – in simple terms, the reproduction of an image used in training:

“The network’s weights are optimised on the training data, but its goal is to perform well on previously unseen data. In the context of Stable Diffusion, unseen data means new random noise patterns and/or new text inputs. To work reasonably on such new data, the network must be able to ‘generalise’: to recognise and understand the general patterns and rules in the training data and be able to apply them in a different context.

If a network has been trained for too long on the same training data or an insufficiently diverse training data, it can be prone to ‘overfitting’. Overfitting occurs when the network uses its weights or part of its weights to memorize the individual training images rather than representing a large set of training images jointly with these weights. Overfitting is characterised by small errors on the training data, but a high error rate on new, unseen data. Overfitting is an undesired feature in machine learning, which engineers try to avoid.

Deep networks can both generalize and memorize at the same time. In such case, the network uses most of its weights to represent general patterns in the data, but uses some part of its weights to memorize individual patterns. The presumed primary cause for memorization is duplication of training data, either by explicit duplication or by training the network for too many epochs (Footnote: ⁷), in conjunction with patterns that cannot be easily represented together with other patterns in the dataset – so-called “outliers””.

In their Joint Statement, the Experts record the extent of their (very considerable) agreement on the subject of watermarks*, which they confirm can be generated by the Model in response to certain prompts owing to the memorization process:

“We agree that Stable Diffusion does generate images with what appears to be a Getty watermark (albeit often distorted) and that this is due to the fact that the model was trained on some number of images containing this visible watermark.

…

We agree that the likelihood of a watermark appearing depends on at least the frequency with which the watermark appears in the training data and the user-specified prompt. We also agree that determining the precise likelihood is difficult because of the complexity of the influence of the prompt and also because of the sheer number of representative prompts and images generated per prompt that would need to be evaluated.

We also agree that certain prompts will generate a watermark with a high frequency, while other prompts are unlikely to generate a watermark.

With respect to how many images a model would need to see before it begins to reproduce a visible watermark, we note that in the Carlini study (Footnote: ⁸), they find memorization with duplication between 200 and 3000 duplicate images. While we cannot say for sure if this frequency would be the same for a watermark, it provides some insight into the frequency with which a model would need to see a watermark in the training data before it begins to reproduce it.

…

We note that in order for a watermark to be produced it is likely that the model needed to be trained on a diverse set of images/captions each containing a watermark. If it was only a single (duplicated) training image with a watermark, the model would memorize the whole image and not just the watermark”.

The Experts reiterated in their Second Joint Statement that “it is most likely that the dominant reason for memorization is data duplication during training either through duplication of the same image in the training dataset, or repeated exposures of one image during training”. In response to the subsequent question: “Are watermarks harder to generate than memorized images?”, the Experts responded:

“Whereas it takes multiple exposures to the exact same image to lead to memorization, memorizing a watermark likely requires multiple exposures to the same watermark regardless of the underlying image. It is not clear to us if one of these is easier or harder than the other. It seems to us that it is “easier” to find a prompt that shows a memorized image, because the image and its caption are reproduced in the training and so a caption with the appropriate keywords is more likely to generate the memorized image (assuming that we know the caption and/or keywords of the highly duplicated training image). On the other hand, it is less clear to us what prompt would generate a watermark since the exposure to the watermark would have been across many different prompts. In this regard, generating a watermark may be “harder””.

There does not appear to be any evidence of steps taken during the initial training and development of v1.1 of the Model to filter out watermarked images. In their first Joint Statement, the Experts agree that v1.2 of the Model “resumed training with a dataset that employed a watermark filter” (i.e. a filter designed to exclude the potential for the production of watermarks* in synthesised images by removing watermarked images from the training dataset). This is clear from the Model Card for v1.2, which records an estimated watermark probability of <0.5. Professor Brox accepted in cross examination that this means that “anything with more than a 50% chance of having a watermark has been excluded [from the dataset]”, although he explained that he was unable to say whether this probability threshold was conservative or not and how many watermarked images would remain after filtering. He accepted that the presence of the filter in the training of this version of the Model had plainly not removed the potential for watermarked* images to be generated and that he would expect outputs sometimes to generate watermarks*.

The Experts also agreed in their Joint Statement that, although not expressly stated in the Model Cards, it was likely that subsequent versions of the Model had employed a watermark filter. However, they also agreed that “even the best watermark filter will not be perfect and leave some images with watermarks in the dataset”. Professor Brox observed in cross examination that “[a]s surprising as it is, that we can build complicated models, it seems we still cannot build filters that are 100% correct”. The Experts expressed no opinion as to the likelihood of watermarks* being generated by a Model trained on a dataset which still retained some images with watermarks even after filtering. When asked about SD XL, Professor Brox said that he would assume that the filtering had got better, but he accepted that he did not know for sure, given that the Model Card contained no information as to any filtering that had been undertaken.