Squid proxy – automatic restart when crashed

I was recently solving problems with squid proxy which stopped responding every few weeks without any entry in log or some obvious reason. It just stopped randomly responding to requests and users had to wait for me to restart a proxy.

I was thinking about a solution. Of course there might be a possibility to debug a problem with dtrace for example – but if it happens just once in every two weeks it would be very difficult. Furthermore I have this experience on more servers with completely different OS and HW so it must be some bug in code of squid.

So I have decided that I will do every minute health check of proxy status and in case it stops responding I will restart it. Definitely not ideal solution but it actually works pretty well (much better than calling me on the phone and waiting for my resolution).

Here is the python script:

#!/usr/bin/python3
import requests
import time
from subprocess import call

proxies = {
  "http": "proxy.host:port",
  "https": "proxy.host:port",
}
while True:
  try:
    r = requests.get("http://seznam.cz/", proxies=proxies)
    if r.status_code!=200:
      raise "Proxy not responding"
    else:
      print("Proxy seems working")
  except:
    print("Restarting proxy")
    call(["systemctl", "restart", "squid"])
  time.sleep(60)

All you need is just to run continuously this script. This can be done pretty easily with Systemd Unit file

[Unit]
Description=Checks if Seznam.cz returns 200, if not, restart squid

[Service]
ExecStart=/usr/local/sbin/squidrestart

[Install]
WantedBy=multi-user.target

Of course this solution has many bugs (what if seznam.cz will crash or return something else than HTTP 200). Or what will happen if squid proxy will keep restarting every 60 seconds. This is just quick solution I wrote which just works now. If it will begin to fail I will update it and add more functionality to address specific issues.