Best Practices for WebBlocker
Follow these best practices to optimize the effectiveness and performance of WebBlocker.
Policy Configuration
Use WebBlocker for outbound HTTP and HTTPS proxy policies. You can also use WebBlocker on a TCP-UDP proxy to categorize sites on ports other than 80 and 443.
WebBlocker Exceptions
For best performance, we recommend you define WebBlocker Exceptions as regular expressions. When you use a pattern match or exact match, the Firebox device must convert this to a regular expression before it evaluates each site. When you use a regular expression, this step is not necessary and the WebBlocker lookup occurs more quickly. For more information and configuration instructions, see the knowledge base article Use regular expressions in proxy definitions.
If you want WebBlocker to always allow or always deny access to a website, regardless of the WebBlocker action used, we recommend that you add a global WebBlocker exception for that site and remove any duplicate exceptions from your WebBlocker actions. Global WebBlocker exceptions help to reduce the size of your configuration file. For more information about global exceptions, go to Configure WebBlocker Global Settings.
WebBlocker does not evaluate the query string (any text that follows a question mark symbol) for an HTTP request. You can use the URL Paths configuration of the HTTP proxy to deny a specific query. For more information on how to deny specific paths with HTTP Proxy, go to HTTP Request: URL Paths.
To create a WebBlocker exception for traffic on a non-standard port, see the Knowledge Base article Add a WebBlocker exception for traffic on non-standard ports.
HTTP Proxy Exceptions or WebBlocker Exceptions
When you configure an HTTP proxy policy with WebBlocker, it is important to understand that HTTP Proxy Exceptions only apply to the content of sites the users get to through the proxy. The WebBlocker Exceptions only impact whether access to a site is denied by WebBlocker.
An HTTP Proxy Exceptions entry for a site does not prevent WebBlocker from denying that site, and a WebBlocker exception does not impact whether the HTTP Proxy action can change or remove the content received by the user.
Allow Access only to Specific Web Sites
If you plan to only allow user access to specific sites with the HTTP Proxy, it is not necessary to use WebBlocker. You can configure the HTTP Proxy to allow only specific paths in the HTTP request. To learn more, go to HTTP Request: URL Paths.
WebBlocker over HTTPS without Content Inspection
WebBlocker examines both the Server Name Indication (SNI) and the Common Name (CN) fields during the certificate exchange to determine the web address. This allows WebBlocker to successfully identify the domain or sub-domain of a website to allow or deny.
Server Selection and DNS
To optimize the performance of WebBlocker Cloud, see the knowledge base article Optimize WebBlocker Performance.
WebBlocker Connections for URL Lookups
To connect to WebBlocker Cloud for URL lookups, the Firebox creates an HTTPS connection to the WebBlocker Cloud server address: rp.cloud.threatseeker.com.
To connect to an on-premises WebBlocker Server for URL lookups, the Firebox creates an HTTPS connection to the local WebBlocker Server configured in the WebBlocker Global Settings. For more information, go to Configure WebBlocker Global Settings.
Troubleshoot WebBlocker
WebBlocker can generate log messages that are useful for troubleshooting. If you do not see log messages for sites that are denied by WebBlocker, make sure that logging is enabled in the WebBlocker action used by your HTTP proxy action.
To enable logging in the WebBlocker action:
- Edit the WebBlocker action used by the HTTP proxy action.
- In the WebBlocker action, select the Categories tab.
- To send a log message when a user tries to go to a site in a category or subcategory and is denied, select the relevant check box in the Log column.
- To send a log message when a user tries to go to an uncategorized site, next to the When a URL is uncategorized drop-down list, select the Log this Action check box.
For more information, go to Configure WebBlocker Categories.
To troubleshoot WebBlocker issues, you must first understand the scope of the issue:
- Does this impact only specific sites or all web traffic?
- Does the issue incorrectly allow or deny access to sites?
Issues that Impact Only Specific Sites
If you find that some sites are incorrectly denied, or not denied, there are a few possible causes:
- This can occur if the site does not match the category you expect it to. To learn how to test which category a site falls under, and how to suggest a change, visit the WatchGuard Security Portal. Remember that you can always create a WebBlocker exception to allow or deny a site if you do not wish to change how you handle the full category.
When a site is allowed or denied based on the category, the traffic logs may contain a message like:
Allow 1-Trusted 0-External tcp 10.0.1.2 50.16.210.117 50790 80 msg="ProxyAllow: HTTP Request categories" proxy_act="HTTP-Client.2" cats="Reference Materials" op="GET" dstname="www.walkscore.com" arg="/" (HTTP-proxy-00)
- This can occur if you do not have WebBlocker enabled for an HTTPS proxy. Web sites accessed over HTTPS will not be denied.
- This can occur if you have WebBlocker enabled for HTTPS, but do not use content inspection on those HTTPS Proxy actions. To learn more, see this known issue: WebBlocker may fail to deny HTTPS sites with wildcard certificates.
Issues that Impact All Sites
If all user traffic is allowed or all traffic is denied, there are several possible causes to consider:
- The policy configuration could be incorrect. Make sure the user web traffic is handled by the correct HTTP, HTTPS, or TCP-UDP proxy policy for the WebBlocker action.
- If you use WebBlocker Cloud, the Firebox might be unable to reach the server, or might not receive a timely response. For more information on how to resolve common connection problems for WebBlocker Cloud, go to Optimize WebBlocker Performance.
- If you use the on-premises WebBlocker Server, make sure that the server is running, and able to reply.
If the WebBlocker Server is inactive, or WebBlocker has an incorrect IP address in the configuration, you may see a log message such as:
Deny 2-Internal-traffic 4-External-traffic tcp 192.168.2.23 23.21.224.150 60921 80 msg="ProxyDeny: HTTP Service unavailable" proxy_act="HTTP-Client.1" service="WebBlocker.1" details="Webblocker server is not available" (HTTP-proxy-00)