Fixing and Preventing Broken Links on Your Website Using New Checkbot

In the report:

1. Copy the problem link address. The simplest way to do this is to right-click on the link in the Checkbot report and click Copy Link from the contextual menu that appears.
2. Ctrl-click the link of the page that contains the broken link to open it in a new tab and then right-click in a blank area of the page and select View Source in the contextual menu that appears.
3. Use CTRL-F to bring up the search built into your browser and paste in the address you copied in step 1. It will immediately highlight that link in the content, wherever it is.
   • Note nearby content so you can find it again in the editor.
   • If it occurs in more than one location, you’ll be able to tab between locations to find them all.
4. Now that you know where the problem link is on the page, you can navigate to the page in WordPress to fix the link, depending on what the problem is.
   • Note: Installing the “Toggle BCIT Websites” bookmarklet makes getting to the WordPress edit page easy and greatly speeds up this process.

This means that the web address that you are linking to has been permanently changed to a new address and the website owners have set up a redirect to automatically send visitors to that new address.

Sometimes this error can be caused by an address with a folder missing a final slash. On the BCIT website for example, https://www.bcit.ca/about (without a slash at the end) will redirect to https://www.bcit.ca/about/ (with a slash). This would show up as a permanent redirect in the Checkbot report.

These aren’t broken links, so what is there to fix?

In some cases if the redirect expires this will become a broken link. By updating these links where possible you are pro-actively managing your 404 (broken link) errors before they happen. In other cases, the page you are being redirected to is not appropriate for the context of your link. By fixing these you are improving the quality of your site content and your Google search ranking.

If the error is caused by an address missing a slash, users who click on these links will experience a slight but noticeable delay in the loading of the target page. This is because the web browser requests the page without the trailing slash, the web server responds to the browser with the 301 error code and the corrected link, and then the web browser has to re-request the link with the corrected address. Fixing these errors improves the user experience for your visitors.

How do I fix it?

• If the content being linked to is still valid, update the link to go to the new page.
• If it’s a matter of a missing slash redirect, update the link to add the slash.
• If the content on the redirected page isn’t quite right:
  • find a new page to link to or remove the link
  • If you cannot find a good page to link to, consider removing the link
• Occasionally the cause for the redirect is not obvious; it is okay to leave these alone (after exhausting the above possibilities) as they will likely not cause a future problem.

Tip: When linking to home pages of websites try and select the simplest reference to the address, even though this may mean having it listed as a permanent redirect. For instance www.home.com when typed in may actually go to ww.home.com/default.htm in the browser. If in the future the website owner changes default.htm to index.html this may result in a broken link. By only using the www.home.com you avoid this problem and reduce the number of truly broken links on your site.

This is very similar to a 301 Moved Permanently error. This error occurs when the address in the link actually redirects to a different web address where the content really resides. The address generating the error is simply an alias has been put in place for an undetermined amount of time.

How do I fix it?

Update the link to go the page where the content resides instead of using the redirect address.

This is very similar to a 301 Moved Permanently error. This error occurs when the address in the link actually redirects to a different web address where the content really resides.

How do I fix it?

Update the link to go to the page where the content resides instead of using the redirect address.

This means the web server doesn’t understand the syntax of the link as it is written on your page. To a visitor this is similar to a 404 error in that no content is displayed.

How do I fix it?

• Check that the link syntax is correct, this can be treated in a similar manner as a 404. Check to see the web address starts with either http://or https:// if it is a secured page.
• If the link looks correct you may want to ask Web Services for help.

This means that you need to provide an account name and password in order to view the content. Sometimes this is intentional and you’ve explained on the linking page what kind of login credentials will be required to access the link. In other cases, the link in question is going to the wrong place and needs to be corrected.

How do I fix it?

• Verify that the visitor does need to use an id and password to access this information.
• If the page is internal to BCIT and should not be password protected:
  • Check that is it linking to the page you intend and correct the link
  • Contact Web Services to look into removing the password protection.

Sometimes this is not an error, as the content should be password protected. In that case, there is nothing to fix (though it will show up on future checkbots as the same error).

This means the web server understands the request but refuses to fulfill it because the request is coming from a source that does not have permission to view the resource. It is usually a security block rather than a temporary glitch.

How do I fix it?

In BCIT reports, this error most often occurs because an external site is specifically blocking bots like Checkbot but not blocking individual users, but the only way to know for sure is to manually check the link.

• Click the link and see if the page loads normally without giving a 403 error.
• If it loads correctly, pay attention to the address and make sure the address for the page that loads is the same as the link you started with. 403 errors can sometimes hide a 301 Redirect, which should still be fixed.
• If it loads correctly and the web addresses match exactly, the link is fine and doesn’t need to be fixed at this time. It will likely continue to be reported as a problem link in future reports and you should manually check it occasionally to make sure that it still works.

This means that the link on your page does not actually go to a webpage. It’s a broken link! These are the most critical Checkbot errors to fix.

There are two main causes of broken links:

1. The owner of the page you are linking to has removed that page or site and did not use a redirect. The result is that your page no longer links to any content.
2. The link has been mistyped or incorrectly formatted. This needs to be corrected.

How do I fix it?

• First check to see if you have an obvious error in your link such as a typo.
• If there is no obvious problem with the syntax of the link, the page or entire site may have simply been removed. In that case either:
  • Find a new appropriate page on that site to link to, or
  • Remove the link.

The requested resource is no longer available at the server and no forwarding address is known. This condition is expected to be considered permanent. The 410 response is primarily intended to assist the task of web maintenance by notifying the recipient that the resource is intentionally unavailable and that the server owners desire that remote links to that resource be removed.

These errors should be treated in the same fashion as a 404 error.

This means the web server is not able to complete the request that is initiated when the visitor clicks the link on your page.

Why does this happen?

This could be due to a number of things including incorrectly formatted URLs, non-existent/expired domains, or because the site you are trying to link to is simply not responding correctly. If a site was unavailable due to maintenance or an unexpected outage during the time Checkbot attempted to connect to it, a 500 error may be generated. You may find when you check the link later, it does work.

In other cases these are similar to 404 or 301 errors, incorrectly formatted or non-existent web addresses, and you can clean them up by finding the correct link.

In some cases, a web site may have blocked access to their site to web crawling applications such as Checkbot. These sites generate a 500 Internal Server Error whenever Checkbot attempts to validate a link to their site. In these cases, these links will appear to work when you investigate, but keep showing up on the Checkbot report week after week. Please let us know about these links so that we can investigate further.

How do I fix it?

• Check if the link works, if it does this may have been caused by a temporary problem with the website, in which case this link is fine and does not need fixing (though it may show up in future Checkbot reports).
• If the link is broken, treat this as a 404 error.

The server, while acting as a gateway or proxy, received an invalid response from the upstream server it accessed in attempting to fulfill the request. If the same link(s) continue to show on your reports, week after week, please let Web Services know so that we can investigate further. But, in most cases, this should be treated as a 404 error.

The server is currently unable to handle the request, due to temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. If the same link(s) continue to show on your reports, week after week, please let Web Services know so that we can investigate further. But, in most cases, this should be treated as a 404 error.

The server, while acting as a gateway or proxy, did not receive a timely response from the upstream server specified by the URI (e.g. HTTP, FTP, LDAP) or some other auxiliary server (e.g. DNS) it needed to access in attempting to complete the request. If the same link(s) continue to show on your reports, week after week, please let Web Services know so that we can investigate further. But, in most cases, this should be treated as a 404 error.

This means that Checkbot wasn’t able to connect to any web server at the link provided. This may be because the link is not properly formatted or the domain name no longer exists. It could be caused by a link that simply isn’t valid. Or, it might be a false report of a problem which may have been caused by a short term internet problem at the time that we checked the link. If you can confirm that the link is valid by clicking on it, and the same link shows up being reported as a problem for several weeks, please let Web Services know and we’ll take a closer look at the problem for you.

Common problems that are detected in this area include the following:

Connection to [domain name] timed out

Even though the domain name used in the link seemed to be valid, when we tried to connect to their webserver to request the page or document being linked to, there was no response at all. This could be caused by a temporary internet outage between our Checkbot application and the remote webserver, or it could be caused by a webserver that simply no longer exists at the address given.

Name lookup timed out for [domain name]

This error means that we couldn’t find an internet address for the domain name given. In order to connect to a webserver, we need to first look up the address of that server in the Domain Name System (DNS). In this particular case, there seemed to be no existing address for the given domain name. This could be caused by a temporary problem with the DNS, or it could be caused by the domain name simply no longer existing. It could also be caused by the domain name given not being a valid address. For example, a link like href=”http://bcit/” does not provide a valid domain name and will be flagged as an error here.

Name lookup timed out for http

This is a common problem where there are two http:// protocols in the link used on the page. For example, when looking a the link, you’ll see something like href=”http://http://example.com/”. This is a quick fix as it just involves editing the link and removing the second http://.

Name lookup timed out for domain name

This is a commons problem where there is no domain name found in the link provided. Problem links like this ususally look like href=”http://” or href=”http:///example.com/”. In the first case, only a protocol was given in the link and it is missing the domain name and possibly the page address. In the second case, a third slash has been given after the protocol portion of the address. While some web browsers compensate and auto-correct links like this, others don’t and therefore the link needs to be corrected even if it seems to work okay for you.

How do I fix it?

• Check if the link works, if it does this may have been caused by a temporary server error, in which case this link is fine and does not need fixing.
• If the link is broken, treat this as a 404 error.

This means that the link given is not a valid URL. The most common cause of this is the use of a file:// protocol being used in a link. Even though the link may appear to work fine for the author of the page, the link will not work for other visitors of the page and needs to be corrected.

This means that there was a problem validating the email address used in a mailto: link on the page. Checkbot validates both the format and deliverability of the email address used.

Each email address reported in this section has a message below it describing the problem detected. Common problems that are detected include:

email address missing an @ in it

In this case, the email address used in the href link does not have an @ sign in it. For example, <a href=”mailto:first_last at bcit.ca”>First Last</a> is not a valid email link and will be flagged as such. Creating links such as this causes usability problems for our visitors and needs to be corrected.

not a valid BCIT email address

In the case of bcit.ca email addresses, we validate whether the email address is deliverable to BCIT from the Internet. In most cases of errors being reported here, BCIT email addresses are not valid simply because an employee no longer works at BCIT and the email link needs to be updated. In some rare instances, special email addresses that are set up to work within Outlook are not deliverable to BCIT from the Internet. In this case, you need to contact the Helpdesk to have the problem resolved. In other cases, a BCIT email address may simply be invalid due to a typo in the address to the left of the @ sign.

improperly-formatted or non-deliverable email address: the domain part of the email address contains an invalid character

In the case of external (non-BCIT) email addresses, we validate the deliverability of the email address by looking at the domain part of the email address (the part after the @ sign) and looking to see if that is a valid domain name to be routing email to. This error means that the email address being used contains an invalid character in it. Domain names can only contain alpha-numeric characters, dashes and dots – any other characters are invalid.

improperly-formatted or non-deliverable email address: the domain part of the email address was not found in the DNS

In the case of external (non-BCIT) email addresses, we validate the deliverability of the email address by looking at the domain part of the email address (the part after the @ sign) and looking to see if that is a valid domain name to be routing email to. This error means that the domain portion of the email address (the part to the right of the @ sign) is not deliverable according to the Domain Name System (DNS) entries that exist for that domain. This could be due to the external entity no longer existing or they may have changed their domain name to a new address (e.g. due to corporate restructuring). In some cases, even though someone may have a website at a particular domain, they may not have configured their systems to receive email at that domain name.

How do I fix it?

In the case of invalid BCIT email addresses:

• Check if the email address is still valid. The most common cause of email errors is that the person is no longer with BCIT and therefore the email address no longer exists.
• If the email address is valid, ensure that the format is correct. For instance: <a href=”mailto:first_last@bcit.ca”>First Last</a>
  • Note: email addresses are not case sensitive.

Tip: Be proactive! When someone leaves your department, update your website contact lists. If you know others who may have references to that person’s email, let them know.