Kill it or fix it? Personalisation as the default of commercial internet

“We will give you everything you like for nothing in return. Just keep watching ads.” This is the deal we have been lured into by online platforms, offering “free, personalised service”. The reality has turned out more crude. Instead of truly personalised service, consumers – now called “users” – saw poorly moderated, cheap content and hyper-personalised ads, exploiting their vulnerabilities. Cory Doctorow found the best term to describe what happened with social media and video platforms: “enshittification”.

But personalisation does not have to work this way. Before it was hijacked by surveillance capitalism, personalisation in search engines and social media allowed us to handle huge volumes of information. Algorithms, including ML algorithms trained under human supervision, can still play that role, if only we get their design and data sources right. But it won’t be a quick fix.

In this blogpost we hope to convince European policy makers that it is both possible and necessary to fix personalisation. It is possible because we can design personalised services in a way that protects privacy and follows user agency. And it is necessary, at least until we find a better way to handle galactical volumes of information, made available online.

What makes personalisation fair?

At Panoptykon we believe that personalisation of an online service can be fair if:

• it is based on data knowingly provided or explicitly authorised by the consumer for this very purpose;
• the consumer has meaningful and operational control over the parameters of personalisation;
• it is designed to serve long-term consumer value, which can include: reducing information overload, developing healthy media consumption habits, matching consumers conscious aspirations (rather than momentary impulses), adapting to evolving consumer preferences, preventing rabbit-holes, encouraging experimentation and broadening horizons, eliminating content that is harmful or triggering for a particular individual.

“Users should have access to meaningful alternative options in how a recommender system functions (including choices among operators and personalised services that meet consumer expectations)”.

We agree with the authors of these words: Laurens Naudts and other authors of “A Right to Constructive Optimization: A Public Interest Approach to Recommender Systems in the Digital Services Act”.

The criteria we use to assess whether personalisation is fair or unfair have been developed in Panoptykon’s position paper, co-authored by Aleksandra Olbryk: “Consumer Empowerment in the Digital Fairness Act”.

Unfair personalisation is a business choice, not a technological necessity

In contrast, unfair personalisation, typically is designed with short-term user engagement as a primary objective, and uses behavioural observations, which (sooner or later) reveal individual vulnerabilities. We find a graphic description of this problem in the Knight-Georgetown Institute “Better Feeds” report:

“A person attending a party might eat a whole bowl of potato chips, and the party host might take this as a sign to refill the chip bowl. But perhaps the party guest is eating impulsively, when in fact they have a long-term goal to be eating healthier food. The guest’s impulsive behavior is misaligned with their underlying preference, but the host interprets the behavior as a sign of what the guest wants.

In a similar manner, impulsively dwelling on, clicking, and liking certain content (say, content that is risky in some way) does not necessarily reflect the user’s forward-looking desires to the platform. A platform that concludes from this engagement that the user must want more and more of this content would be falsely assuming that the user’s impulsive, mindless, or myopic behavior equates with their long-term preferences.”

Knight-Georgetown Institute, “Better Feeds”, p. 17

Therefore at Panoptykon we advocate against personalisation that is optimised for short-term user engagement and driven solely by platforms' own commercial interests. At the same time we are convinced that – if it is controlled by the user and/or designed with user wellbeing as the primary objective – personalisation of digital interfaces and algorithmic systems can serve long-term consumer value.

With the Digital Fairness Act, EU can shape business choices

The upcoming Digital Fairness Act (we expect the proposal to be published on November 11th) creates an opportunity to codify this distinction into the law.

How? With a combination of legal provisions that will serve the following, complementary objectives:

1. protect consumers (regardless of their age and personal traits) from manipulation and addictive design (by forcing service providers to comply with safety by design principle and a list of prohibited practices);
2. encourage consumers to make conscious choices and help them break harmful habits/compulsive behaviour (by obliging service providers to introduce “positive friction”);
3. enable authentic choice of personalised experience, including choice of a different operator of a personalised service or key element of the service (by introducing the obligation to interoperate with competing services and/or prohibition of gate-keeping).

In this blogpost we will focus on the first objective: safety by design for personalised services. If you would like to read more about our recipe for the second or third objective, please check relevant sections of our "Safe by Default" brief.

Safety by design for personalised services

Based on her reading of our position paper (mentioned above), Prof. Monika Namysłowska (University of Łódź) came up with a list of practices that, according to her, should be prohibited or, at least, grey-listed in the Digital Fairness Act. It is our first (and certainly not last) attempt to translate safety by default rule into more precise legal provisions:

• No behavioural personalisation as default

Personalisation based on personal data other than provided data, or requiring, for the deactivation of such personalisation, a greater number of actions by the consumer than for its activation, should be prohibited as the default setting of a service.

Making functionalities of the service available only in conjunction with such personalisation, other than functionalities which cannot be provided without the data concerned, shall be treated as applying that personalisation as a default setting.

• No affective targeting

Personalising the content, presentation or timing of a commercial offer or commercial communication on the basis of the consumer’s inferred emotional or psychological state should be prohibited.

In contrast, declared moods or states consciously chosen by the consumer fall outside of this prohibition.

• No vulnerability targeting

Directing or adjusting commercial offers, commercial communications or recommendations to a consumer, or adjusting their content, presentation or timing, on the basis of one or more vulnerability indicators should be prohibited.

This prohibition does not cover: (a) the use of vulnerability indicators solely to withhold or suppress commercial offers or commercial communications; (b) communications whose sole object is to provide the consumer with assistance or protection required or expressly provided for by Union or national law.

The DFA should not enumerate vulnerability indicators. While it would be beneficial to name examples in the law, the list (and the very concept of “vulnerability”) should remain open.

• No personalisation directed at minors

It should be prohibited to personalise commercial offers or commercial communications on the basis of profiling within the meaning of Article 4(4) of Regulation (EU) 2016/679, or personalise recommendations on the basis of behavioural or inferred data, where the trader is aware with reasonable certainty that the consumer is a minor.

This prohibition should not affect measures whose sole purpose and configured effect is the protection of minors. Also, it should not be used by traders as a reason to process additional personal data in order to assess whether the consumer is a minor.

• No engagement (doomscrolling) traps

Increasing the frequency or changing the timing of notifications directed at a consumer, adjusting the selection or ranking of content or recommendations directed at that consumer, or displaying retention prompts to that consumer, on the basis of a disengagement signal should be prohibited.

This prohibition does not cover measures whose sole configured purpose and effect is to reduce, interrupt or facilitate the consumer’s disengagement from the use of the service (so called “positive friction”).

***

With the DFA proposal expected next month, we would like to encourage public discussion about getting personalisation right. If you like our recipe, please send us your comment or share it with your network. If you don’t like it, please do not hesitate to speak up! We are certain that this recipe can be improved and, therefore, we wait for your constructive critique!