The bot protection module in Edge Security Accelerator (ESA) blocks malicious crawlers from scraping your static resources. This module uses behavior analysis and traffic feature detection to identify and block these crawlers. This reduces bandwidth consumption from unusual traffic and improves the stability of your services.
Background information
Static resources, such as audio, videos, images, and CSS/JS files, are usually handled by a caching mechanism and do not send requests directly to the origin server. For this reason, traditional mitigation policies focus on bot attacks against dynamic requests. However, malicious websites can directly reference static resource URLs. This leads to abnormal consumption of bandwidth and server resources. ESA provides dedicated security protection policies for these scenarios. ESA detects malicious bots and blocks their requests to access static resources. This effectively prevents crawlers from abusing cached resources or consuming bandwidth, providing active defense for your static resources.
Enable protection for static resource requests
To prevent malicious crawlers from consuming bandwidth through hotlinking or bypassing the caching mechanism, you can set different actions for Definite Bots, Likely Bots, or Verified Bots. Then, you can enable the Static Resource Protection feature. The system applies bot behavior categorization policies to requests for static resources that hit the cache. It then implements different blocking policies based on the threat level, while ensuring that legitimate bots can scrape normally.
In the ESA console, choose Site Management. In the Website column, click the target site.
In the navigation pane on the left, choose .
On the Smart Mode page, click Configure for Definite Bots, Likely Bots, or Verified Bots to set a mitigation action.
Definite Bots: This category includes many malicious crawlers. Set the action to Block or Slider CAPTCHA.
Likely Bots: These requests have a lower risk than Definite Bots but may contain malicious crawlers and other traffic. Set the action to Monitor, or to Slider CAPTCHA during high-risk periods.
Verified Bots: This category usually includes crawlers from search engines that support your website's search engine optimization (SEO). Set the action to Allow. If you do not want any search engine crawlers to access your site, you can set the action to Block.
Turn on the Static Resource Protection switch.
Create a static resource protection rule
In Advanced Mode, you can create a protection rule set to define more fine-grained mitigation policies for static resources. These policies can include bot behavior or feature detection, crawler blocking, and whitelisting.
In the ESA console, choose Site Management. In the Website column, click the target site.
In the navigation pane on the left, choose .
Click . Follow the on-screen instructions to enter a Rule Set Name, and select a Service Type and an SDK Integration.
In the If requests match... area, set the match field to Serves Static Resources, the match operator to equals, and set the switch to
. In the Then execute... area, select and configure the appropriate mitigation policies based on your needs.
In the Effective Time area, click Edit in the Actions column. Set the effective period and click OK.
When you are finished, click OK.