Among other content on this site, I have links to other websites throughout the site. These are links that I personally endorse and believe in (I never accept unsolicited link requests), and this is a 100% non-commercial site. However, with the way the internet works, urls are sadly not permanent. Sites move, or shut down, and many of the sites I link to here aren't always sites that I regularly visit. So, I added a background maintenance script to do it for me, and unpublish any links that have gone offline.

It runs once a week (Sunday at 4am, US Eastern timezone), performs exactly one HEAD request to each link stored in my database, and it doesn't crawl anything or load any content on the site in question. Nor does it follow any redirects, if it gets a 301 or 302 status code. It simply checks the status code, and if the status is not 200, it unpublishes the link and notifies me, so I can personally check it out.

So, if you see this URL show up in your access logs, that's why. I tried to make it as low-impact as possible, but if you have an issue with this script, let me know, and I'll be happy to work it out with you.

The current user agent for this script is:
awi-5.1/deerconnect.health_check (+http://www.lupinia.net/about/link-health.htm, seneca.lupinia.net/Django-1.11)

The code for this background process is on GitHub: https://github.com/lupinia/awi/blob/master/deerconnect/management/commands/health_check.py

This article has been updated as of April 4, 2022 19:15