Ad blocker Router


Based in this awesome idea (http://jacobsalmela.com/instant-raspberry-pi-ad-blocker/) I configured this on GConnect.


1- Change dnsmasq default configuration
Backup the dnsmasq.conf and create the necessary directories first:

# mkdir /opt/NoADS
# mkdir /opt/NoADS/dnsmasq.d
# mkdir /opt/NoADS/tmp
# cp /etc/dnsmasq.conf /opt/NoADS/dnsmasq.conf

Change this configuration:

# cat /etc/dnsmasq.conf | grep -v "#" |

interface=ath0
dhcp-range=172.16.0.2,172.16.0.254,255.255.255.0,12h
dhcp-leasefile=/tmp/dnsmasq.leases
conf-dir=/etc/dnsmasq.d

To this:

domain-needed
interface=ath0
interface=eth0
min-port=4096
cache-size=1000000
log-queries
bogus-priv
dhcp-range=172.16.0.2,172.16.0.254,255.255.255.0,12h
dhcp-leasefile=/opt/NoADS/dnsmasq.leases
conf-dir=/opt/NoADS/dnsmasq.d
# Uncomment for debug
# log-facility=/opt/NoADS/tmp/dnsmasq.log                                  

2- Create the script to generate the blocks in dnsmasq
I did a couple of changes in the original script. In my opinion this change keep future changes in format of sources my simple to maintain. Basically every source has a independent and temporary file. Once all files are captured, the second part of script create a adList.conf directing all calls to this domains to the local GConnect apache.

Copy this script to gravity-adv.sh and give execution permission with:
chmod +x gravity-adv.sh
cat gravity-adv.sh
                                                                                                                
#!/bin/bash                                                                                                                                   
# The G-hole now blocks over 140,000 ad domains                                                                                               
# Address to send ads to (the RPi)                                                                                                            
GholeIP="192.168.1.200"                                                                                                                       
# Optionally, uncomment to automatically detect the address.  Thanks Gregg                                                                    
# GholeIP=$(/sbin/ifconfig eth0 | awk '/inet addr/{print substr($2,6)}')                                                                      
                                                                                                                                            
# Config file to hold URL rules                                                                                                               
eventHorizion="/opt/NoADS/dnsmasq.d/adList.conf"                                                                                              
                                                                                                                                            
# temporary file                                                                                                                              
TEMPFILE1=/opt/NoADS/tmp/matter1.txt                                                                                                          
TEMPFILE2=/opt/NoADS/tmp/matter2.txt                                                                                                          
TEMPFILE3=/opt/NoADS/tmp/matter3.txt                                                                                                          
TEMPFILE4=/opt/NoADS/tmp/matter4.txt                                                                                                          
TEMPFILE5=/opt/NoADS/tmp/matter5.txt                                                                                                          
TEMPFILE6=/opt/NoADS/tmp/matter6.txt                                                                                                          
TEMPFILE7=/opt/NoADS/tmp/matter7.txt                                                                                                          
TEMPFILE8=/opt/NoADS/tmp/matter8.txt                                                                                                          
                                                                                                                                            
                                                                                                                                            
ENDFILE=/opt/NoADS/tmp/andLight.txt                                                                                                           
                                                                                                                                            
echo "Getting yoyo ad list..." # Approximately 2452 domains at the time of writing                                                            
curl -s -d mimetype=plaintext -d hostformat=unixhosts http://pgl.yoyo.org/adservers/serverlist.php? | sort > $TEMPFILE1                        
echo "Found $(cat $TEMPFILE1 | wc -l) entries"                                                                                                
                                                                                                                                            
echo "Getting winhelp2002 ad list..." # 12985 domains                                                                                         
curl -s http://winhelp2002.mvps.org/hosts.txt | grep -v "#" | grep -v "127.0.0.1" | sed '/^$/d' | sed 's/\ /\\ /g' | awk '{print $2}' | sort |
tr -d '\b\r'  | sed '/^$/d'  > $TEMPFILE2                                                                                                     
echo "Found $(cat $TEMPFILE2 | wc -l) entries"                                                                                                
                                                                                                                                            
echo "Getting adaway ad list..." # 445 domains                                                                                                
curl -s https://adaway.org/hosts.txt | grep -v "#" | grep -v "::1" | sed '/^$/d' | sed 's/\ /\\ /g' | awk '{print $2}' | grep -v '^\\' | grep -
v '\\$' | sort > $TEMPFILE3                                                                                                                   
echo "Found $(cat $TEMPFILE3 | wc -l) entries"                                                                                                
                                                                                                                                            
echo "Getting hosts-file ad list..." # 28050 domains                                                                                          
curl -s http://hosts-file.net/.%5Cad_servers.txt | grep -v "#" | grep -v "::1" | sed '/^$/d' | sed 's/\ /\\ /g' | awk '{print $2}' | grep -v '^
\\' | grep -v '\\$' | sort > $TEMPFILE4                                                                                                       
echo "Found $(cat $TEMPFILE4 | wc -l) entries"                                                                                                
                                                                                                                                            
echo "Getting malwaredomainlist ad list..." # 1352 domains                                                                                    
curl -s http://www.malwaredomainlist.com/hostslist/hosts.txt | grep -v "#" | sed '/^$/d' | sed 's/\ /\\ /g' | awk '{print $3}' | grep -v '^\\'
| grep -v '\\$' | sort | tr -d '\b\r' | sed '/^$/d'   > $TEMPFILE5                                                                            
echo "Found $(cat $TEMPFILE5 | wc -l) entries"                                                                                                
                                                                                                                                            
echo "Getting adblock.gjtech ad list..." # 696 domains                                                                                        
curl -s http://adblock.gjtech.net/?format=unix-hosts | grep -v "#" | sed '/^$/d' | sed 's/\ /\\ /g' | awk '{print $2}' | grep -v '^\\' | grep -
v '\\$' | sort > $TEMPFILE6                                                                                                                   
echo "Found $(cat $TEMPFILE6 | wc -l) entries"                                                                                                
                                                                                                                                            
echo "Getting someone who cares ad list..." # 10600                                                                                           
curl -s http://someonewhocares.org/hosts/hosts | grep -v "#" | sed '/^$/d' | sed 's/\ /\\ /g' | grep -v '^\\' | grep -v '\\$' | awk '{print $2}
' | grep -v '^\\' | grep -v '\\$' | sort > $TEMPFILE7                                                                                         
echo "Found $(cat $TEMPFILE7 | wc -l) entries"                                                                                                
                                                                                                                                            
echo "Getting Mother of All Ad Blocks list..." # 102168 domains!! Thanks Kacy                                                                 
curl -silent -A 'Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0' -e http://forum.xda-developers.com/ http://adblock.mahak
ala.is/ | grep -v "#" | awk '{print $2}' | sort > $TEMPFILE8                                                                                  
echo "Found $(cat $TEMPFILE8 | wc -l) entries"                                                                                                
                                                                                                                                            
echo ""                                                                                                                                       
                                                                                                                                            
# Merge all files into one                                                                                                                    
cat  /opt/NoADS/tmp/matter* >  /opt/NoADS/tmp/NEW.txt                                                                                         
                                                                                                                                            
# clean old adList.conf                                                                                                                       
> /opt/NoADS/tmp/adList.conf                                                                                                                  
                                                                                                                                            
# for each line of NEW.txt - clean the WINDOWS new line character and blank lines and construct the new adList.conf                           
for line in `cat /opt/NoADS/tmp/NEW.txt | tr -d '\b\r' | sed '/^$/d' `; do                                                                    
   echo "address=/"$line"/$GholeIP" >> /opt/NoADS/tmp/adList.conf                                                                           
   #ls -l $line;                                                                                                                            
done                                                                                                                                          
                                                                                                                                            
                                                                                                                                            
# Count how many domains were added so it can be displayed to the user                                                                        
numberOfAdsBlocked=$(cat /opt/NoADS/tmp/adList.conf | wc -l | sed 's/^[ \t]*//')                                                              
echo "$numberOfAdsBlocked ad domains added to the blacklist"                                                                                  
echo ""                                                                                                                                       
                                                                                                                                            
# Compare Files                                                                                                                               
DIFF=$(/usr/bin/diff /opt/NoADS/tmp/adList.conf /opt/NoADS/dnsmasq.d/adList.conf | wc -l)                                                     
echo "Found $DIFF diferences between the /opt/NoADS/tmp/adList.conf /opt/NoADS/dnsmasq.d/adList.conf"                                         
echo ""                                                                                                                                       
echo "To use this new file, backup the old adList.conf out of /opt/NoADS/dnsmasq.d/ and copy the new one to this directory"                   
echo "Then restart the dnsmasq - /etc/init.d/dnsmasq restart"                                                                                 
echo "if something goes wrong with the new file, recover the old one and restart dnsmasq"                                                     
                                                                                                                                            
# Restart DNS                                                                                                                                 
#/etc/init.d/dnsmasq restart

2- Run gravity-adv.sh
./gravity-adv.sh
Getting yoyo ad list...
Found 2445 entries   
Getting winhelp2002 ad list...
Found 13058 entries
Getting adaway ad list...
Found 0 entries     
Getting hosts-file ad list...
Found 28117 entries
Getting malwaredomainlist ad list...
Found 1359 entries
Getting adblock.gjtech ad list...
Found 696 entries
Getting someone who cares ad list...
Found 10602 entries
Getting Mother of All Ad Blocks list...
Found 102181 entries 
      
158457 ad domains added to the blacklist 
                                             
Found 316757 diferences between the /opt/NoADS/tmp/adList.conf /opt/NoADS/dnsmasq.d/adList.conf
                                             
To use this new file, backup the old adList.conf out of /opt/NoADS/dnsmasq.d/ and copy the new one to this directory                           
Then restart the dnsmasq - /etc/init.d/dnsmasq restart                                                                                         
if something goes wrong with the new file, recover the old one and restart dnsmasq

Copy the /opt/NoADS/tmp/adList.conf to /opt/NoADS/dnsmasq.d/

cp /opt/NoADS/tmp/adList.conf /opt/NoADS/dnsmasq.d/adList.conf

and restart the dnsmasq service:

# /etc/init.d/dnsmasq restart

Eventually something can goes wrong with this script, usually due the change of the format of sources of Ad blocks lists. Then this error will appear:

Starting dnsmasq: 
dnsmasq: error at line 90918 of /opt/NoADS/dnsmasq.d/adList.conf             [FAILED]

To correct this error, I just remove this line (90918) of the adList.conf file:

vi /opt/NoADS/dnsmasq.d/adList.conf

inside VI type :set number then :90918 then with the cursor over the line type dd to delete the line. type :wq! to save and exit.

To this until the dnsmasq restart command show a [ OK ]  like this:

/etc/init.d/dnsmasq restart 
dnsdomainname: Unknown host 
dnsdomainname: Unknown host 
dnsdomainname: Unknown host 

Starting dnsmasq:                                          [  OK  ]

Eventually you need re-run this script to re-populate the adList.conf with new hosts.



Open DNS port to LAN - Optional
# Open TCP 53 port to resolve DNS requests internally thru eth0
/sbin/iptables -A INPUT -p tcp --dport 53 -j ACCEPT

# cat /etc/resolv.conf
nameserver 8.8.8.8
nameserver 8.8.4.4

I already set the IP 192.168.1.200 in a alias of eth0. With this configuration I can easily include this DNS server in my LAN router to use internally at my home.

/sbin/ifconfig eth0:1 192.168.1.200                                                                                                          

It's necessary open the TCP 53 port to other machines use this DNS thru ethernet.

/sbin/iptables -A INPUT -p tcp --dport 53 -j ACCEPT                                                                                          

OBS: All Uses connected at GCONNECT Wifi are automatically using this Ad blocker. I did this Optional changes because I would like to use this Ad blocker with my 5 ghz cisco router.


Include all changes in /root/rc.local to get up in next reboot.
/root/rc.local
#!/bin/sh               
#                                 
# This script will be executed *after* all the other init scripts.       
# You can put your own initialization stuff in here if you don't           
# want to do the full Sys V style init stuff.                                                              

# Sleep 20 to wait harddrive mount 
sleep 20 
                                 
/opt/bin/transmission-daemon -g /home/admin/.config/transmission-daemon/ >> /root/log.txt 
                                                             
# Start NoADS 
# Open Port 53 TCP 
               
/sbin/iptables -A INPUT -p tcp --dport 53 -j ACCEPT
# Create eth alias 
/sbin/ifconfig eth0:1 192.168.1.200 

# Set Resolv.conf 
echo "nameserver 8.8.8.8" > /etc/resolv.conf 
echo "nameserver 8.8.4.4" >> /etc/resolv.conf 
                                 
# Start dnsmasd DNS server 
/etc/init.d/dnsmasq start


Testing NoADS
Test if the configuration are working on this page: