Browsing with HTTP Referer and Data Protection

27 Jun 2018 in Digitalisation on Data-protection 28 Jun 2018

Web browsers inform web servers in many cases about the referring webpage when users browse the web. I consider if this can be a good data protection practice.

Screenshot showing HTTP referer

Screenshot showing HTTP referer.

Consider a web page with URL α on web server A containing a link to a web page with URL β on web server B. If a web page visitor clicks on the link to β, many browsers send along with the request to server B for the web page β the referring webpage α. This information is sent in form of a HTTP request header.

Example Request with Referer Header

To illustrate this behaviour, let us assume you browse e.g. an adult-content web page β called https://newborn.xx/videos/some_video and click on a link to α that is https://blog.riemann.cc/digitalisation/2018/06/27/http-referer-and-data-protection/. Then, the HTTP request generated by your browser to download and display the webpage may contain the following fields:

Host: blog.riemann.cc
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-GB,en;q=0.8,de-DE;q=0.5,de;q=0.3
Referer: https://newborn.xx/videos/some_video
DNT: 1

Host: A web server may serve more than one website. Hence, a host must be given to specify the requested webpage β.
User-Agent: The request contains information on the employed browser. Some servers may respond with a different, simplified webpage for e.g. outdated browsers.
Accept: The request contains information on response formats the browser could understand and display.
Accept-Language: The request contains information on the preferred languages of the response. This setting reflects the language setting of the browser or operating system.
Referer [sic]: It is actually a spelling mistake of the word referrer. The referring web page containing the link is provided.
DNT (do not track): Modern browsers notify the web servers if the visitor wishes to not be tracked.

Impact on Privacy

If the request for β to the web server B contains the Referer field, then B learns which web page the visitor has visited before, i.e. https://newborn.xx/videos/some_video in the example above. Furthermore, B learns that this referring web page contains the link to β.

The impact on the visitor privacy is mentioned in the standard RFC 7231 Section 5.5.2:

The Referer field has the potential to reveal information about the request context or browsing history of the user, which is a privacy concern if the referring resource’s identifier reveals personal information (such as an account name) or a resource that is supposed to be confidential (such as behind a firewall or internal to a secured service). Most general-purpose user agents do not send the Referer header field when the referring resource is a local “file” or “data” URI. A user agent MUST NOT send a Referer header field in an unsecured HTTP request if the referring page was received with a secure protocol.

Also the documentation portal of Firefox has a notice on its web page to the Referer field.

The Referer header has the potential to reveal information about the browsing history of the user, which is a privacy concern.

According to the recommendation of the RFC standard, Firefox does not send a Referer field when:

the referring resource is a local “file” or “data” URI,
an unsecured HTTP request is used and the referring page was received with a secure protocol (HTTPS).

That means, if e.g. https://newborn.xx/videos/some_video contains a link to another HTTPS web page, the Referer field is sent.

Good Data Protection Practice

Not all, if not most, visitors do not understand that their browser sends a Referer field revealing a small part of their browsing history. In consequence, those visitors cannot give an informed consent. I remember how I installed my browser to identify a potential contractual legal basis.

My Linux operating system comes with Firefox pre-installed. During the installation of Linux, I confirm varying free software licenses that are all somewhat compatible to each other in one go (one click).

Firefox is provided under the terms of the Mozilla Public License. It contains an explicit Disclaimer of Warranty.

Covered Software is provided under this License on an “as is” basis, without warranty of any kind, either expressed, implied, or statutory, including, without limitation, warranties that the Covered Software is free of defects, merchantable, fit for a particular purpose or non-infringing. The entire risk as to the quality and performance of the Covered Software is with You. Should any Covered Software prove defective in any respect, You (not any Contributor) assume the cost of any necessary servicing, repair, or correction. […]

It is yet to be answered if a) the referer information constitutes personal data and b) the sharing of it (potentially among different parties on different continents) is covered on the basis of the browser license.

Independently of the answer, I think a much better practice would be to allow visitors to switch this sharing by their browser on and off. From a data minimisation point of view, no sharing should occur by default.

Interestingly, a bug has been filed in 2013 already:
(Regression) Data leak: HTTP-Referer sent when it should not be sent

This bug makes though a different assumption on when a Referer field should be sent. Consequently, I created a new bug report:
Data leak: HTTP-Referer sent without consent

Let us see what happens.

Update (2018-06-28)

While we wait for a comfortable user interface to disable the referer field, one can change the configuration of Firefox manually. I found the solution at a Mozilla Support page.

In a new tab, type or paste about:config in the address bar and press Enter/Return. Click the button promising to be careful.
In the search box above the list, type or paste referer and pause while the list is filtered
Double-click the network.http.referer.XOriginPolicy preference and enter the desired value:

Value	Policy
0	no restrictions (default)
1	base domain must match (send from a.example.com to b.example.com)
2	full host name must match (only b.example.com to b.example.com)

velofisch wrote on June 28, 2018

I see your point, but I still think the referer-metadata is much preferable to any kind of third party tool that collects data on the web. Let me explain:
The referer-metainformation does not provide any information, if you just enter a new URL. Neither is it used, when you click an ad on a page. Whenever you click an ad on a webpage, this request will always first go to the adserver and then to the target website. The adserver, of course, knows where you come from and where you go to and the adserver does not need the referrer-metadata for it. The adserver knows this not just for this one click on this one site but for many links on many webservers. The adserver will generate a complete profile of you - whereas the referer-metainformation will provide only much more isolated information. When the adserver redirects the request to the final webserver, the referer-metatag will not provide much information either. The adservice might provide a lot of information about the user to the final website - without having to rely on the referer metadata.
The referer-metadata will provide information when you use a plain normal link. It has a sister-metadata "origin" which is crucial for CORS and XSS protection (see here for more information: https://chromium.googlesour... Regarding link liability (intruduced by the ECJ) and protection against fraudulent links, the referer-metadata might be useful and justified in some cases. In general, it provides much less information in a more decentralized and hence privacy-friendly way than third party tracking tools. But, of course, it provides additional personal information for which there might not always be a justification.
In my view: As long as we allow the use of any third party tools for tracking, fonts, ads, captacha etc., it would be disproportionate to go after the referer-metadata. The referer-metadata is much more privacy preserving than all those tools - even when you apply these not really privacy-enabling anonymize_ip extensions.
We should try to favor privacy-preserving decentralized protocols even if they provide some information and should rather try to limit the centralized data collection. This centralized data collection is often the main motivation for providing nice and useful third party website tools. Restricting the information the standard protocol can provide will foster third party tools - which will lead to less and not more privacy.

Browsing with HTTP Referer and Data Protection

Example Request with Referer Header

Impact on Privacy

Good Data Protection Practice

Update (2018-06-28)

~rriemann

Error

Example Request with Referer Header

Impact on Privacy

Good Data Protection Practice

Update (2018-06-28)

Templates (for web app):

Error