Squid, an Open Source Web Proxy Server – trial on amazon web services
The Proxy Server program, running on either the firewall server or a separate server within the Private Network, can be accessed by the computers on the Private Network, and the Proxy Server can access the Wild Internet. So, sites allowed by the Proxy Server will be available, and other sites will not.
The Proxy Server I recommend is Squid, version 3.X. This is a free, open source proxy server, that is very full featured (we will only be using a tiny portion of those features) There is a newer version, 4.X, but a production Docker container is not yet available for 4.X.
To try out the Proxy Server, I recommend using Amazon Web Services. This will allow you to run a computer in the cloud for 2 cents per hour, with a minimum of one hour (no kidding!). And, for the first year, you can run a “micro” server for free. Google Amazon Web Services, and sign up for a free account. Go to the “EC2” dashboard to control your cloud servers.
You will need to set up ssh access from your computer to control the Amazon Web Services computer. For Macs, this is very easy — run the Terminal program, and ssh is already installed! For Windows computers, you have to install an ssh client. I recommend Putty, a free ssh client that will work on Windows computers.
You will also need to create a “key pair” to be able to access your AWS instances. A “key pair” is a combination of public and private keys that work together. As the name states, you can distribute the public key anywhere you want, but you MUST keep the private key private, because anyone who gets the private key can access any of your computers. On the Mac, you can generate a key pair by typing:
openssl genrsa -des3 -out private.pem 2048
openssl rsa -in private.pem -outform PEM -pubout -out public.pem
If you remove the -des3, you will not have to add a passphrase, which will be required every time you use your private key. However, not using a passphrase allows anyone who finds your private key file to use it.
This generates the private key first. The public key can be derived from the private key, which is what the second command does.
Use any text viewer program to check the contents of the files. Make sure you know which one contains only the public key. You can Import the public key from the “Key Pairs” link in the EC2 dashboard.
If you don’t want to keep your private key from ever leaving your local computer, you can use the “Create Key Pair” on the Key Pairs page of the EC2 dashboard. Make sure you copy the private key to your local computer, since it won’t be available after you close the window from generating the key pair.
Put the private key where your ssh client can find it, and it will allow you to access your AWS server. On the Mac, that is a file called id_rsa within the .ssh folder in your home folder. The permissions of that file must prohibit access from group and other, and must be owned by you, or ssh will refuse to use it (chmod 600 id_rsa)
Once you have created your key pair, you can start your Squid instance by following these steps:
1) In the EC2 dashboard, Instances page, click “Launch Instance”
2) Find Ubuntu Server 18.04 LTS image. This is the starting configuration of your server.
3) Make sure “64 bit” is selected, and click on “Select” next to your server AMI
4) Pick the instance size. Choose “Micro”, which is 2 cents per hour, and is more than capable enough to run the Squid Server, especially for testing
5) Click “Next: Configure Instance Details”
6) Review the settings on the Configure Instance Details page, but the defaults should be acceptable. Click “Next: Add Storage”
7) Review the storage options. The defaults should work, but note that this includes “Delete on Termination”, which means that your virtual disk drive will be deleted when you terminate your instance. That is probably better than saving it unless you want to re-use it or do something with it.
8) Click “Next: Tag Instance”
9) Click “Next: Configure Security Group”
The Security Group is like an external firewall that AWS sets up for you. You want the least amount of access, to reduce the risk that your server will be attacked.
10) select “Create a new security group”, give it a name and description if you want, and add two rules. The first is Type “SSH”, which will select Protocol TCP, and Port range 22. For “Source”, select “My IP”. That will prevent access from any computer not on the same router that you are on.
11) Add a second rule (click “Add Rule”), type Custom TCP Rule, Protocol TCP, Port Range 3128, Source My IP. This is the port that the Proxy Server will be listening on for connections from your browser.
12) Click “Review and Launch”. You can review the selected options, and then click “Launch”.
13) You will then be presented with a dialog to select or create a key pair. Select the key pair you imported previously. This is the key pair that will work with your new server.
14) Check the acknowledgement box, and click “Launch Instances”
15) You can review the information on that page, but then click “View Instances”
16) It will take about five minutes for your Instance to spin up and become live. If you click on your new instance in the list at the top of the “Instances” page, you will see the details for that instance at the bottom of the page. You can look at Status Checks, or just wait a couple of minutes. Note that you will need the “Public IP” to connect to that server. Make a note of that Public IP address.
17) Go to your ssh client, and try to access your new server, using the “Public IP” as the address, user name “ubuntu”, and the key pair you set previously. For security reasons, it is not possible to log into the server with a username and password. Only the key pair will work. From the Mac’s Terminal window, if the Public IP is 54.82.54.87, type ssh ubuntu@54.82.54.87
18) You may get a message that connection times out or is refused if the server is still booting. Once it succeeds, you will get a prompt asking if you want to “continue connecting”, saving the new server information to your computer. On subsequent connections to the same computer, you will not get that prompt. If you happen to get the same IP address as you had used previously for a different server, the fingerprint will be different, and you will not be able to connect. You will be told which line in the .ssh/known_hosts line is the problem, so delete that line.
19) Once you agree to continue connecting by typing “yes”, if your keys are set up properly, you will be connected.
20) Next, you must update your server to the most current version of its packages, for security reasons. Type: sudo apt-get update to get info about the new packages, and then sudo apt-get upgrade to actually do the upgrades. Type “y” to confirm. The whole upgrade will only take a couple of minutes, since there is not much software in your new server.
21) Install Squid by typing sudo apt-get install squid . This will download, install, and start Squid.
22) Now, check that your new Squid is working by setting up your Internet Explorer or other browser to use it to connect to the Internet. For Internet Explorer, select “Tools/Internet Options”, go to the Connections tab, click on LAN Settings, and in the bottom of the page, check the box that says “Use a Proxy Server”, enter the IP address of your new server (like 54.82.54.87), and port 3128, and click OK twice to save your new settings.
23) Now, use your browser to go to any http site, like http://www.google.com . If everything is working correctly, you will get a big “ERROR” page, saying that Access Denied. This means that your Proxy Server is working properly, and rejecting your attempt to connect to www.google.com (which actually is a dangerous website, since ANYTHING can be found on Google). If you don’t get Access Denied, recheck your Security Group rules that port 3128 is allowed in the Inbound Rules
24) Now, you need to configure your Squid Server to allow access to the sites you want to allow. You need to set BOTH the “incoming” and “outgoing” sites that are allowed, so that only your own browser can access the Squid server, not any browser anywhere in the world (although your Security Group would prevent that, anyway). This will require editing the Squid Configuration file. My favorite Linux editor is “vi”, but you may find “nano” easier, since it has prompts at the bottom of the page. You will need to run nano with sudo, so that you will have permission to edit the configuration file.
25) Type sudo nano /etc/squid/squid.conf to start nano.
26) Lines beginning with “#” are comments, and have no effect. Find the line that says http_access allow localhost (not commented out). After that line, type acl my_ip src 12.23.34.45 , where you replace 12.23.34.45 with the IP address of your router (it will be shown in the lines for the Security Group). Then add a line acl my_dest dstdomain .google.com . The .google.com will allow any site ending in .google.com to be accessed. You can add additional allowed sites and domains after .google.com , each separated by a space. As noted above, you do NOT want .google.com in your production proxy server, but it is helpful for testing
27) Immediately after that add a line http_access allow my_ip my_dest . That MUST be before the http_access deny all , which prevents any requests not explicitly allowed from completing.
28) If you want to allow regular (http://) and secure (https://) access to the sites you select, find the line that says http_access deny CONNECT !SSL_ports , and comment out (add a ‘#’ sign before the first character)
29) Reload the Squid configuration by typing sudo squid -k reconfigure . If everything is OK, it will execute silently. If you made a mistake, you will get a helpful message about what to do.
30) Now, try accessing http://www.google.com from your browser. It should succeed. Try browsing anywhere else, and you will get the ERROR Access Denied page.
31) https will work, too. Browse to https://www.google.com to see. However, if the page is not allowed, you may get a less helpful error message from Internet Explorer. However, if you click “Fix Connection Problems”, you will see a message that you got error message 403 (Forbidden) to let you know what the problem is.
32) Type sudo nano /var/log/squid/access.log to view the access log. This will show TCP_MISS for ALLOWED sites, and TCP_DENIED for denied sites. If you are trying to connect to a new server and entering its domain into dstdomain doesn’t work (don’t forget sudo squid -k reconfigure), look in the access.log to see if the site is trying to get you to go to a different site, and make a decision about whether to allow that site, as well. The same applies if the website looks awful – some of the files needed to show the website properly may be hosted in a different domain. Checking the log will tell you what other sites you need to open to show the site correctly.
33) When you are done testing, don’t forget to terminate your instances, since you will be charged by the hour until they are terminated.