📅 November 11, 2019
“Let’s block ads!”
Pi-hole is a free, network-wide ad blocking solution for your network. You can set up Pi-hole to act as your primary DNS server so any device connected to your network must resolve DNS requests through Pi-hole before resolving DNS requests on the Internet.
The idea behind Pi-hole is to maintain block lists of known ad servers. If a URL is on a block list, then Pi-hole resolves that URL to 0.0.0.0, and a connection cannot be made. The result? No ad is shown.
Pi-hole was originally designed for the Raspberry Pi, but it will also run on practically any hardware running Linux. We will set up Pi-hole on the Intel NUC running Xubuntu 19.10, and then configure the DNS chain so all devices connected to the network — computers, cell phones, tablets, game consoles, smart TVs, mobile apps, whatever — automatically receive the Pi-hole as the primary DNS server through DHCP. Most ads will be blocked automatically without any client configuration.
Just connect and ads are blocked!
The official Pi-hole web site provides excellent support for Pi-hole. The site is easy to follow and provides detailed instructions.
Pi-hole is easy to use and includes an optional web interface for monitoring Pi-hole statistics.
We are going to install Pi-hole on the NUC via SSH. At this point, the NUC should be set up and running on its own with Internet access. There should not be a monitor, keyboard, or mouse connected to the NUC unless you want those things. Even though we can install Pi-hole from a local terminal on the NUC, we are going to use a remote SSH connection to administer the NUC.
From another computer on the network, log in using ssh.
192.168.10.100 is the example IP in this article. Use the actual IP address of your bond0 interface. Replace username with the NUC’s root user. After logging in, you should see the command line for the NUC even though you might be seated in front of another computer.
Enter the Automated Install Command
There is no file to download and install. Enter this command:
curl -sSL https://install.pi-hole.net | bash
More instructions about alternative installations can be found on the Pi-hole web site. You can even clone the git repository if you want.
Select an Upstream DNS Provider
The first important question Pi-hole will ask is, “What upstream DNS provider should I use if the URL is valid?”
Pi-hole will be the first DNS server checked when resolving DNS requests. If a URL is on a blocklist, Pi-hole stops there. No connection is made.
But what happens if a URL is okay? Meaning, a URL is not on any block list. This is assumed to be a good URL, so it still needs to be resolved so the connection can be made. To do this, Pi-hole then needs to pass the URL on to another DNS server and on and on until the URL is resolved to its IP address.
Upstream means, “Where does Pi-hole go to resolve the URL?” This should be the Internet router. In this example, the router, whose IP address is 192.168.1.1 will then resolve the URL from the ISP or Internet.
It is okay if the upstream DNS provider is on a different network than the NUC’s 192.168.10.100 IP address. We have another router in-between that acts as a DHCP server. Here is why:
Router A is the ISP router whose job is to connect to the Internet. It contains a firewall. DHCP is disabled on router A because there is no need for it.
Router B handles DHCP on the LAN. It also has a firewall giving two layers of firewall protection and to isolate the two networks if needed. It does not have a direct connection to the Internet, so router A acts as its gateway router. There is a dedicated Ethernet port on router B that connects directly to router A. The IP address of this port is 192.168.1.2, so it will appear as part of router A’s 192.168.1.0 network segment. This is a static IP address.
How DNS Lookup is Handled
Suppose a client connected to router B has the IP address of 192.168.10.23 and wants to access a web page in a browser. The DNS request goes to 192.168.10.1, the IP address of router B. It does not go to the Pi-hole directly, even though we could configure it that way. All devices use router B as the primary DNS server.
“But wait. Isn’t the Pi-hole the primary DNS server?”
Yes, but we do not set it up that way. That would require configuring each client individually, which we do not want. Router B handles DHCP, so we want router B to pass along the Pi-hole address to each client when it connects.
So, as part of its DHCP configuration, the client at 192.168.10.23 is set to use 192.168.10.1 as the primary DNS server. In router B’s configuration, we set up its primary DNS server to be the Pi-hole at IP address 192.168.10.100 — the IP address of bond0 on the NUC. Now, the Pi-hole will reject ad URLs.
“But if the URL is good and Pi-hole does not block it, what next?”
This is where the upstream DNS provided enters the picture. We entered 192.168.1.1, which is the IP address of router A, in this test setup. Router A’s configuration uses the ISP as the DNS server. From there, it is left to the ISP to find and return a valid IP address for the given URL.
This can be a little tricky to follow, so here is another diagram:
“Does this slow down the DNS request?”
No. there is no noticeable difference with or without it. Internet access is still fast an snappy.
This is why we use router A as the upstream DNS provider and why it is okay to use a different IP address, where it exists. The default gateway for router B is router A at IP address 192.168.1.1, so Pi-hole knows how to find it.
Choose Initial Blocklists
Pi-hole provides access to a few third-party blocklists to get started.
When asked for a static IP address, accept the defaults. The gateway, in this case, is router B, which acts as a DHCP server.
Note that if you choose to install the web admin interface, you will be asked to install the lighttpd web server. Go ahead and install it. Log queries too in order to view detailed statistics, but disable it for privacy. You can even set a privacy level for the Faster Than Light (FTL) interface. More information about this is found in the pi-hole documentation. Show everything is the most informative.
Accessing the Web Interface
Open a web browser on your client system, and enter pi.hole/admin in the address bar (or use the IP address). In this case, it is 192.168.10.100/admin. Be sure to use the /admin as part of the address.
You should see a basic statistics page.
It appears rather basic, and that is because we need to log in to see all features. Click Login located in the left pane.
Removing the Password
By default, Pi-hole is password protected. Since this is located on a limited test network where snooping is not an issue, we are going to remove the password.
In the NUC’s SSH terminal, enter this command:
sudo pihole -a -p
Press Enter for a blank password. Refresh the browser, and the page will now look like this:
Caution: A password should be used. Pi-hole logs all queries. If there is no password set, then anyone with access to your network and open the Pi-hole admin console, view the Query log, and see all sites that have been visited. This information does not appear if not logged in.
Pi-hole is ready to use!
“Do I need to configure clients individually?”
No, not according to this example. If you need to, then open a client’s Internet settings, and set the primary DNS server to be the Pi-hole’s IP address, which is 192.168.10.100 in this case.
This configuration does not require client configuration because router B is the DHCP server, and it is configured to use the Pi-hole address as the sole DNS server. Pi-hole then uses router A as the upstream DNS provider.
Setting up router A and router B is beyond the scope of this article because different network configurations vary by their setup. One design will not work for everybody. Consult your router’s configuration. The point is to set the router that handles DHCP to use the Pi-hole as the primary DNS server. Do not use a secondary DNS server. This way, if Pi-hole fails, it will not fallback to a different DNS server and let ads through. Instead, you will see that pages will refuse to connect.
“Is it working?”
In a browser without ad blocking installed, open a few web pages that you know contain ads. Here is one to get started:
Ads should not appear. Whitespace should be left in their place or nothing at all. It depends upon the ad and page construction.
The missing ads are not being hidden, they are simply not loading at all. No connection is being made to an ad server. This saves bandwidth.
To test this theory, open the Pi-hole web console and go to Settings and open the Blocklists tab. This is a list of installed third-party blocklists.
Open one by clicking on it.
Each line in a blocklist contains 0.0.0.0 followed by a URL. This means any attempt to visit that URL will resolve to 0.0.0.0. Nothing can connect to IP address 0.0.0.0. This is how Pi-hole works. Try entering 0.0.0.0 in a web browser and watch what happens.
Now, pick one of the URLs from the blocklist, and enter that in a web browser to see what happens.
Pi-hole blocked the connection. Firefox was not set up in any special way. In fact, it was a default installation. No proxies set. No DNS modifications made to the system. The client used DHCP to set up its networking, and Pi-hole automatically become the DNS server. Since this address was on the blocklist, access was denied.
The Query Log
If logging was enabled during Pi-hole setup, you can view all connections accepted or rejected. In the Pi-hole web console, click on Query Log.
Under the Type column, A means IPv4 addressing, and AAAA means IPv6 address. This is why we block both IPv4 and IPv6.
Domain shows the URL that attempted a DNS request. If red, the domain was found in a blocklist and denied. If green, the domain was allowed. This is useful information because you will no doubt see an ad site appear in a browser during normal usage. New ad sites are being created continuously, and blocklist are not always up to date. Therefore, a few ads get through.
When you see any domain in your browser’s address bar, open the Query Log and click Blacklist. This adds the domain to a custom blacklist. The domain will be blocked from now on. However, you might need to clear the browser’s cache to prevent the site from being accessed again.
Notice how detailed the information is? This information cannot be viewed unless logged in. This is why you should password-protect the web console if several people or even strangers use your network. You do not want them snooping through your network activity.
Updating Pi-hole and Pi-hole-FTL
Occasionally, you might see some update notifications in the footer of the Pi-hole web console. To update Pi-hole, SSH into the NUC, and in the terminal, enter,
sudo pihole -up
Pi-hole DHCP Server
Pi-hole can function as a DHCP server. However, if you already have a DHCP server on your network, then do not enable it.
Adding More Blocklists
The default blocklists configured are quite good. They block ads from a variety of most sources that you will encounter. But ad-blocking is a cat-and-mouse game that continually updates. Having more blocklists is a good thing.
The Block List Project is one of many sites that provide a number of free blocklists (as of the time of this writing) to increase the number of blocked domains covering a variety of categories.
Let’s ad a blocklist from this site to Pi-hole. View all of the lists, and choose a category. Let’s pick the Ads category.
This list contains 225,902 URLs when this was written. Click More Info, and we will see details.
In the Pi-hole web console, open the Dashboard and look at the current number of domains being blocked.
Go to Settings, and click the Blocklists tab.
To add another blocklist, paste the link we just copied from the Block List Project web site into the text area at the bottom of the page, and click Save and Update.
Now, open Dashboard, and recheck the blocked domain count.
“But wait! If the new blocklist contained 225,902 domains, why did it increase by only 68,519 domains?”
Duplicates are ignored. Duplicates are bound to exist many times in multiple lists. What is important is that new domains were added.
The Project List is by no means the only blocklist site available. Many others exist, and people share their personal lists on various forums and sites.
Another excellent source of additional blocklists is The Firebog and its Big Blocklist Collection. Install a new blocklist the same way as described above. If you do not like a blocklist or find that it is too restrictive, then you can always remove it.
Benefits to Using Pi-hole
Well…almost no ads. Pi-hole only blocks what is on its blocklists, of which you can ad as many as you like. If a new ad server is created and it does not exist in any blocklist used by Pi-hole, then the ad server connection will be made, and the ad will appear. Just add the offending URL to the Pi-hole black list, and it will be blocked in future, uncached connections. (You might need to clear your browser’s cache first.)
By refusing to connect to ad servers, those ads are never sent to your system. This results in less network traffic.
Network-wide Ad Blocking
If you have configured your DHCP server to use Pi-hole as its primary DNS server, then any device that connects to the network will automatically receive the benefits of Pi-hole. There will be no need to configure each device individually to use Pi-hole. This is a necessity with some device that do not allow direct Internet DNS configuration, such as the Roku media device, that only receive network setting via DHCP.
Slick Web Interface
An easy-to-use, user-friendly web interface allows you to administer Pi-hole from within a web browser. Everything is easy to use and configure.
Thwart Anti-Adblocker Sites
Some web sites will try to detect if your browser is running an ad blocker, and if it is, then the site makes the page inaccessible and tells you to disable your ad blocker. With Pi-hole, it is possible to disable the ad blocker as requested, but the site is still viewable without ads. Even if the ad blocker is disabled or nonexistent, the anti-adblocker site is fully functional without displaying any ads.
Ads are not merely hidden. Pi-Hole prevents connections to the ad servers themselves, so the ads are never downloaded to your computer. This conserves bandwidth and reduces page clutter even after your browser’s ad blocking add-on is disabled as requested/required by the web site.
What Pi-Hole Will NOT Do
Does Not Block YouTube Ads
Pi-hole has trouble reliably blocking YouTube ads involving https traffic. According to some, ads that play before videos on YouTube will still play because they are served through https, and Pi-hole would need to break the SSL/TLS encryption to perform a man-in-the-middle attack, which Pi-hole does not do. Pi-hole cannot “peek” inside encrypted network traffic (https) in order to remove the ad and pass the rest along. This applies to any ad-serving site. How true this theory might be remains to be verified.
Other YouTube ads depend upon rotating URLs — one might play an ad one day, and then the same URL will serve video files later. Blocking that URL might block the ad, but it also blocks the video at a later date. Some ads can be blocked with Pi-hole, others cannot. The Block List Project provides a YouTube blocklist, so this might be something worth looking into, but no guarantees.
“Then, how can I block YouTube ads?”
You will need to use client-side ad-blocking software, not Pi-hole, for the most reliable method. There is a semi-working way: you can watch the Pi-hole traffic using pihole -t as YouTube plays, and then add the discovered YouTube ad URLs to the black list when one serves an ad.
However, this is hit and miss. You will end up having to monitor and block several URLs, and, while it might seem to be blocking ads for a few days, eventually the ads will return, and you will need to block a whole new set of URLs. In addition, some of the blocked URLs will also prevent the video from playing, so you will see an endless buffering image. The video will never play until you remove the blocked URLs from the Pi-hole black list.
A client-side ad-blocker, such as uBlock, is better for blocking YouTube ads.
Does Not Encrypt Your Network
Pi-hole only resolves known URLs to 0.0.0.0. It does not automatically encrypt your network traffic to protect it from packet sniffers.
Does Not Monitor Your Network Traffic
Pi-hole only resolves DNS requests. Traffic does not pass through Pi-hole. When you transfer a file from one computer to another, the data does not travel through the Pi-hole software.
The Pi-hole web interface shows statistic, yes, but these are related to DNS requests only, not the actual files that were transferred or network bandwidth. For those statistics, you will need different monitoring software.
Pi-hole Is Not a Proxy Server
You cannot hide servers or computers behind Pi-hole like you can with a proxy. Network traffic does not pass through Pi-hole, so it will not slow your network down. Pi-hole should be the first check for resolving DNS requests. That is all Pi-hole does. It resolves DNS requests so clients cannot connect to known ad servers or any other URL in a block list.
Does Not Block Connections to IP Addresses
Pi-hole blocks by prevent DNS resolution. However, Pi-hole does not block connections by addresses. For example, the site http://www.really-nasty-ad-server-that-shows-no-mercy.com will be blocked by Pi-hole because Pi-hole will return its IP address as 0.0.0.0 to the client. Thus, no connection is made.
But if web site uses the actual IP address instead of the URL, then there is nothing to block. The IP address is already known, and the connection is direct. No DNS resolution takes place, and Pi-hole is ignored.
This might seem like a serious weakness in Pi-hole, but in reality it is nothing to be concerned about. An overwhelming majority of ad servers use URLs instead of direct IP addresses. In programming terms, think of a URL as a variable, and think of an IP address as a literal value. It is much easier to maintain code by using a variable and defining it in one location than it is to hard-code literals throughout the source code. Using URLs makes it easier to keep clients connected to ad servers in case the IP addresses change. Sites referencing by URL can still connect by updating IP addresses behind the scenes. This is one theory as to why ads and malicious sites insist upon using URLs instead of IP addresses. Because of this, Pi-hole works extremely well.
Pi-hole is one of the best uses for a dedicated network device, NAS, or project like this. This article barely covers Pi-hole operation, but, hopefully, there is enough here to help you get started. For more configuration information, please have a look at the Pi-hole documentation.
There is still more we can do with the Intel NUC running Xubuntu 19.10, so we will install an FTP server next time.