Transparent HTTPS Proxy on a FreeBSD Server

Transparent HTTPS proxies do work by intercepting the TLS encrypted traffic for the client. This may be used for evil eavesdropping, specially in cases where the transparent proxy is not under our control. This may however serve for totally valid purposes as well, like filtering mal web content in general. Of course, I won’t install a system on my home server for eavesdropping me. One application of filtering at the proxy level is given here (German) and here (English). The interception mechanism is:

  1. validade the public certificate from the server,
  2. decrypt the traffic in order to get its hands on the contents of it,
  3. do whatever filtering and rewriting has been configured,
  4. generate a temporary public certificate+private key pair in the name of the remote server, and sign it with the proxy’s local CA instance, which is usually (and even should better be) a private, self-signed certification authority,
  5. re-encrypt the traffic with said personalized private server key,
  6. and finally pass the manipulated an re-encrypted content to the client.

As a pre-requisite, the FreeBSD Home Server has been set up to be the gateway into the internet for a number of local clients as shown in a previous article here (German) or here (English by the Microsoft Translator).

The Proxy Software installation

I utilize the Squid software package which can be installed from the FreeBSD ports.

# cd /usr/ports/www/squid

# make config

My FreeBSD home server employs ipfw(8) as its firewall, and therefore the software needs to be configured with the option Transparent proxying with IPFW being selected. For intercepting HTTPS traffic, the SSL options need to be enabled as well.

# make install clean

For step 1 it is necessary to additionally install security/ca_root_nss.

The Proxy’s Certification Authority

For step 4 a self signed proxy root CA certificate and chain needs to be created, and openssl can be used for this:

# cd /usr/local/etc/squid

# mkdir proxy-certs; cd proxy-certs

# openssl req -new -newkey rsa:2048 -sha256 -days 8766 -nodes -x509 \
          -extensions v3_ca -keyout proxy-ca.pem -out proxy-ca.pem

Generating a RSA private key
writing new private key to 'proxy-ca.pem'
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [BR]:
State or Province Name (full name) [São Paulo]:
São Bernardo do Campo []:
Organization Name (eg, company) [Obsigna!]:
Organizational Unit Name (eg, section) [Certification Authority]:
Common Name (e.g. server FQDN or YOUR name) []:proxy.local
Email Address []

The point here is, that a fully qualified domain name must be informed which our local web browsers behind the firewall would accept, here I chose proxy.local.

We need to generate our Diffie-Hellman parameter in order to harden the DH ephemeral key exchange which provides for Perfect Forward Secrecy:

# openssl dhparam -out dhparam.pem -2 2048

# chown -R squid:squid ./

# chmod -R u-w,go-rwx ./

The Proxy’s TLS Cert Daemon

As well for step 4, Squid’s SSL cert daemon sslcrtd needs to be set up by the commands below:

# /usr/local/libexec/squid/security_file_certgen -c \
                           -s /usr/local/etc/squid/dyn-certs -M 4MB

# chown -R squid:squid /usr/local/etc/squid/dyn-certs

# chmod -R go-rwx /usr/local/etc/squid/dyn-certs

The Squid configuration file

The file /usr/local/etc/squid/squid.conf contains the following configuration settings:

shutdown_lifetime       0 seconds

acl manager             proto cache_object
acl localnet		src
acl safari5             src
acl port_443		port 443
acl ports_80_443	port 80 443

http_access		allow localhost manager
http_access		deny manager
http_access		deny !ports_80_443
http_access		deny CONNECT !port_443
http_access		deny to_localhost
http_access		allow localnet
http_access		deny all

http_port		localhost:3127
http_port intercept
https_port intercept ssl-bump generate-host-certificates=on cert=/usr/local/etc/squid/proxy-certs/proxy-ca.pem tls-dh=/usr/local/etc/squid/proxy-certs/dhparam.pem

acl step1		at_step SslBump1
ssl_bump		peek step1
ssl_bump		bump port_443

sslcrtd_program	        /usr/local/libexec/squid/security_file_certgen -s /usr/local/etc/squid/dyn-certs -M 4MB
sslcrtd_children	4 startup=1 idle=1

cache_mem		512 MB
cache_dir		aufs /var/squid/cache 10000 16 256
coredump_dir            /var/squid/cache

refresh_pattern		-i (/cgi-bin/|\?|\|.*\|)  0    0%    0
refresh_pattern		.                         0   20% 4320

request_header_access	User-Agent deny safari5
request_header_add      User-Agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.1.2 Safari/605.1.15" safari5

tls_outgoing_options	cipher=HIGH:!aNULL:!AES128:!SSLv2:!SSLv3:!TLSv1
tls_outgoing_options	cafile=/etc/ssl/cert.pem
tls_outgoing_options	options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE

url_rewrite_children	4 startup=1 idle=1 concurrency=0
url_rewrite_program	/usr/local/bin/nukie


  1. For the transparent HTTP/HTTPS proxy the traffic on ports 80 and on 443 need to be forwarded by the firewall to two different ports. In the above configuration file, port 3128 is set as the transparent proxy port for the real port 80 and 3129 as the proxy port for the real port 443.
  2. For a working transparent ssl-proxy, we need to place our own CA certificate chain, consisting of a root CA certificate and its private key, all joined together in one .pem file, to a place where squid can read it. In the above configuration file this is /usr/local/etc/squid/proxy-ca.pem.
  3. For Squid being able to validate TLS traffic on behalf of its clients, we need to indicate a store of publicly trusted CA certificates. In the above configuration file this is /etc/ssl/cert.pem, which is actually a dynamic link to the public root CA store which was created by security/ca_root_nss.
  4. We need to inform the directory where Squid can place the dynamically created and signed intermediate certificates. In the above configuration file this is /usr/local/etc/squid/dyn-certs.
  5. The request_header_access directives are for a special purpose of mine, anyway, I left it in here in order to show, where to put any customizations.

Squid would be started by the way of a RC script, and the following entry needs to be added to the file /etc/rc.conf:

# Squid

Squid can be started now:

# service squid start

The IPFW Firewall configuration

The firewall rules on the home server which would forward HTTP and HTTPS traffic to the transparent proxy ports 3128 and 3129 need to come before any NAT rule.

# Transparent HTTP(S) Proxy - Squid
/sbin/ipfw -q add 80 fwd,3128 tcp from to not me 80
/sbin/ipfw -q add 81 fwd,3129 tcp from to not me 443

The above rules force the HTTP/HTTPS traffic of the local clients which have IP addresses in the range from to to be passed through the transparent proxy. Hence, in case a web site cannot be accessed from behind the proxy, the client’s IP may be temporarily set to another address outside of this range.

Inform the root certificate of the Proxy’s CA to the client’s web browser

This is necessary in order the web browsers accepts the personalized server certificate, which has been signed by the self-signed local root CA of the proxy.

Depending on the client's platform, this part is easy (macOS or iOS Safari) to medium complicated (Chrome or Firefox) to obstaculous (Windows). Anyway, without declaring to your browser that it may trust the proxy’s CA, it will continue to complain about non-secure connections, or even in case of HSTS refuse to connect at all.

Some sites are inaccessible using the proxy

There are two options to circumvent accessibility problems with the transparent proxy.

1. The obvious one, temporarily change the client’s IP to an address outside of the transparent range.

2. Enable Automatic Proxy Discovery in the client’s network settings.

Then put the file wpad.dat to the document root of a web service on the gateway machine, i.e. the FreeBSD home server. The respective web service must listen on port 80 of the gateway’s LAN-IP address. Here the file wpad.dat got the following content:

function FindProxyForURL(url, host)
   if (url.substring(0, 6) == "https:" && (dnsDomainIs(host, "") || dnsDomainIs(host, "")))
      return "PROXY";
   return "DIRECT";

With that in place, Apple’s App Store would work without manually switching the IP’s and Netflix movies would come in without impacting the proxy.

Copyright © Dr. Rolf Jansen - 2019-07-23 18:25:42

Discussion on Twitter: 1154144457421414400