Introduction

To get a better understanding how users navigate through your website and which other websites link to your page, it is important to get access to the referrer information. When using a Reverse Proxy like Nginx this information usually gets lost. In this post, I will describe how to configure Nginx to get the referrer information and log this data in conjunction with other important information.

This post is an extension of my previous post related to forwarding the client remote IP address to a Spring Boot web application. I would recommend reading this post before you proceed.

Where is the code?

The code for this post is available in this GitHub repository.

1. Nginx configuration

Below you will see the Nginx configuration I used for this example.

nginx.conf
server {
    listen       80 default_server;
    server_name  example.com;

    location / {
        proxy_set_header X-Forwarded-Host $host:$server_port;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Referrer $http_referer;
        proxy_pass http://localhost:8090;
    }
}

As you can see I added the custom header X-Forwarded-Referrer and used the Nginx variable $http_referer to set the value.

2. Implement the RequestLogger

We will use a HandlerInterceptor to extract the required information for all requests.

RequestLogger.java
package com.progressive.code.starter.interceptor;

import java.util.Enumeration;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import org.springframework.web.servlet.HandlerInterceptor;
import org.springframework.web.servlet.ModelAndView;

/* Name "RequestLogger" to be able to use @Autowired in the configuration */
@Component("RequestLogger")
public class RequestLogger implements HandlerInterceptor {

  private static final Logger LOGGER = LoggerFactory.getLogger(
    RequestLogger.class);

  @Override
  public boolean preHandle(HttpServletRequest request, HttpServletResponse response,
    Object handler)
    throws Exception {

    /* Logs the "real" remote ip address, as set by Nginx */
    LOGGER.info("Page {} was requested by IP {}", request.getRequestURI(),
      request.getHeader("X-Forwarded-For"));

    /* Logs the referrer, as set by Nginx */
    LOGGER.info("Referrer is {}", request.getHeader("X-Forwarded-Referrer"));

    /* Logs the query string */
    LOGGER.info("Parameters / Query String is {}", request.getQueryString());

    /* Logs the user agent (e.g. Browser and Version */
    LOGGER.info("User Agent {}", request.getHeader("user-agent"));

    /* Logs the accepted language */
    LOGGER.info("Accepted Language {}", request.getLocale());

    /* Log all headers */
    Enumeration headerNames = request.getHeaderNames();
    while(headerNames.hasMoreElements()) {
      String headerName = headerNames.nextElement();
      LOGGER.info("key {}, value {}", headerName, request.getHeader(headerName));
    }

    return true;
  }

  @Override
  public void postHandle(HttpServletRequest request, HttpServletResponse response,
     Object handler,ModelAndView modelAndView) throws Exception {
  }

  @Override
  public void afterCompletion(HttpServletRequest request, HttpServletResponse response,
    Object handler, Exception ex) throws Exception {
  }
}

Let's quickly walk through the implementation:

  • request.getHeader("X-Forwarded-For") returns the remote IP address of the client
  • request.getHeader("X-Forwarded-Referrer") returns the referrer as specified in the Nginx configuration file
  • request.getQueryString() provides the query string within the URL, basically the parameters specified after the "?".
  • request.getHeader("user-agent") provides the client information, e.g. browser name and version
  • request.getLocale() returns the users preferred language

3. Add a link to test the HTTP referrer

To test the referrer and query string implementation I have added a link to the start page.

home.html
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8" />
<title>Spring Web Demo Start Page</title>
</head>
<body>
  <h1>Start Page</h1>
  <p>
    <a href="/?parameter1=test">Test Referrer and Query String</a>
  </p>
</body>
</html>

4. Test the implementation

To test the implementation the web application has to be installed on the backend server, and the Nginx server has to forward the requests to the internal URL. Simply enter the application URL - in my case this is for example http://192.168.1.73/. Please note that the port of the reverse proxy (80) has to be used, not the port of the Spring Boot application (8090).

On the start page you will see the link I mentioned above "Test Referrer and Query String". When you click the link, we can check the log file produced by the application. In my case, the log looks like this.

server log
Page / was requested by IP 192.168.1.120
Referrer is https://example1.local/
Parameters / Query String is parameter1=test
User Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36
Accepted Language en_GB
key x-forwarded-host, value example1.local:443
key x-forwarded-server, value example1.local
key x-forwarded-for, value 192.168.1.120
key x-forwarded-referrer, value https://example1.local/
key host, value localhost:8090
key connection, value close
key upgrade-insecure-requests, value 1
key user-agent, value Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36
key accept, value text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
key referer, value https://example1.local/
key accept-encoding, value gzip, deflate, br
key accept-language, value en-GB,en;q=0.9,en-US;q=0.8,de;q=0.7,fr;q=0.6

As you can see in my example above all the expected information shows up.

This was a simple example how to forward request information when operating a Spring Boot application behind a reverse proxy.