Proof of concept of semantically correct DoS blocking

Use an additional variable to expire the IP:DOS_COUNTER variable

See https://github.com/coreruleset/coreruleset/issues/1999\#issuecomment-1086596939

README.md

@@ -20,42 +20,45 @@ In such a situation, CRS proposes the use of one or several of the following opt
 
 ## Description of Mechanics
 
-When a request hits a non-static resource (`TX:STATIC_EXTENSIONS`), then a counter for the IP address is being raised (`IP:DOS_COUNTER`). If the counter (`IP:DOS_COUNTER`) hits a limit (`TX:DOS_COUNTER_THRESHOLD`), then a burst is identified (`IP:DOS_BURST_COUNTER`) and the counter (`IP:DOS_COUNTER`) is reset. The burst counter expires within a timeout period (`TX:DOS_BURST_TIME_SLICE`).
+When a request hits a non-static resource (`TX:STATIC_EXTENSIONS`), then a counter for the IP address is increased (`IP:DOS_COUNTER`). If the counter (`IP:DOS_COUNTER`) hits a limit (`TX:DOS_COUNTER_THRESHOLD`) within a certain window (`TX:DOS_COUNTER_TIME_SLICE`), then a burst is identified (`IP:DOS_BURST_COUNTER`) and the counter (`IP:DOS_COUNTER`) is reset. The burst counter expires after a defined amount of time (`TX:DOS_BURST_TIME_SLICE`).
 
-If the burst counter (`IP:DOS_BURST_COUNTER`) is greater equal 2, then the blocking flag is being set (`IP:DOS_BLOCK`). The blocking flag (`IP:DOS_BLOCK`) expires within a timeout period (`TX:DOS_BLOCK_TIMEOUT`). All this counting happens in phase 5.
+If the burst counter (`IP:DOS_BURST_COUNTER`) is greater equal 2, then the blocking variable is set (`IP:DOS_BLOCK`). The blocking variable (`IP:DOS_BLOCK`) expires within a timeout period (`TX:DOS_BLOCK_TIMEOUT`). All this counting happens in phase 5.
 
 There is a stricter sibling to this rule (9514170) in paranoia level 2, where the burst counter check (`IP:DOS_BURST_COUNTER`) hits at greater equal 1.
 
 ### Blocking
 
-The blocking is done in phase 1: When the blocking flag is encountered (`IP:DOS_BLOCK`), then the request is dropped without sending a response. If this happens, then a counter is raised (`IP:DOS_BLOCK_COUNTER`).
+The blocking is done in phase 1: When the blocking variable is set (`IP:DOS_BLOCK`), then the request is dropped without sending a response. If this happens, a counter is incremented (`IP:DOS_BLOCK_COUNTER`).
 
-When an IP address is blocked for the first time, then the blocking is reported in a message and a flag (`IP:DOS_BLOCK_FLAG`) is set. This flag expires in 60 seconds.
+When an IP address is blocked for the first time, the blocking is reported in a message and a flag (`IP:DOS_BLOCK_FLAG`) is set. This flag expires after the number of seconds configured with `TX:DOS_REPORTING_TIMEOUT`.
 
-When an IP address is blocked and the flag (`IP:DOS_BLOCK_FLAG`) is set, then the blocking is not being reported (to prevent a flood of alerts). When the flag (`IP:DOS_BLOCK_FLAG`) has expired and a new request is being blocked, then the counter (`IP:DOS_BLOCK_COUNTER`) is being reset to 0 and the block is being treated as the first block (-> alert).
+When an IP address is blocked and the flag (`IP:DOS_BLOCK_FLAG`) is set, the blocking is not reported (to prevent a flood of alerts). When the flag (`IP:DOS_BLOCK_FLAG`) has expired (after `TX:DOS_REPORTING_TIMEOUT` seconds) and a new request is blocked, the counter (`IP:DOS_BLOCK_COUNTER`) is reset to 0 and the block is treated as the first block (-> alert).
 
 In order to be able to display the counter (`IP:DOS_BLOCK_COUNTER`) and resetting it at the same time, we copy the counter (`IP:DOS_BLOCK_COUNTER`) into a different variable (`TX:DOS_BLOCK_COUNTER`), which is then displayed in turn.
 
 ### Variables
 
-| Variable                   | Description                                                 |
-| -------------------------- | ----------------------------------------------------------- |
-| `IP:DOS_BLOCK`             | Flag if an IP address should be blocked                     |
-| `IP:DOS_BLOCK_COUNTER`     | Counter of blocked requests                                 |
-| `IP:DOS_BLOCK_FLAG`        | Flag keeping track of alert. Flag expires after 60 seconds. |
-| `IP:DOS_BURST_COUNTER`     | Burst counter                                               |
-| `IP:DOS_COUNTER`           | Request counter (static resources are ignored)              |
-| `TX:DOS_BLOCK_COUNTER`     | Copy of `IP:DOS_BLOCK_COUNTER` (needed for display reasons) |
-| `TX:DOS_BLOCK_TIMEOUT`     | Period in seconds a blocked IP will be blocked              |
-| `TX:DOS_COUNTER_THRESHOLD` | Limit of requests, where a burst is identified              |
-| `TX:DOS_BURST_TIME_SLICE`  | Period in seconds when we will forget a burst               |
-| `TX:STATIC_EXTENSIONS`     | Paths which can be ignored with regards to DoS              |
-
-As a precondition for these rules, please set the following three variables in `dos-protection-config.conf`:
+| Variable                   | Description                                                                          |
+| -------------------------- | ------------------------------------------------------------------------------------ |
+| `IP:DOS_BLOCK`             | Flag if an IP address should be blocked                                              |
+| `IP:DOS_BLOCK_COUNTER`     | Counter of blocked requests                                                          |
+| `IP:DOS_BLOCK_FLAG`        | Flag keeping track of alert. Flag expires after 60 seconds.                          |
+| `IP:DOS_BURST_COUNTER`     | Burst counter                                                                        |
+| `IP:DOS_COUNTER`           | Request counter (static resources are ignored)                                       |
+| `TX:DOS_BLOCK_COUNTER`     | Copy of `IP:DOS_BLOCK_COUNTER` (needed for display reasons)                          |
+| `TX:DOS_BLOCK_TIMEOUT`     | Period in seconds a blocked IP will be blocked                                       |
+| `TX:DOS_COUNTER_THRESHOLD` | Limit of requests, where a burst is identified                                       |
+| `TX:DOS_COUNTER_TIME_SLICE`| Period in seconds of window for which to count requests against the current burst    |
+| `TX:DOS_BURST_TIME_SLICE`  | Period in seconds when we will forget a burst                                        |
+| `TX:STATIC_EXTENSIONS`     | Paths which can be ignored with regards to DoS                                       |
+
+As a precondition for these rules, please set the following five variables in `dos-protection-config.conf`:
 
 - `TX:DOS_BLOCK_TIMEOUT`
 - `TX:DOS_COUNTER_THRESHOLD`
+- `TX:DOS_COUNTER_TIME_SLICE`
 - `TX:DOS_BURST_TIME_SLICE`
+- `TX:DOS_REPORTING_TIMEOUT`
 
 And make sure that `TX:STATIC_EXTENSIONS` is set as required, also in `dos-protection-config.conf`.
 

plugins/dos-protection-before.conf

@@ -22,26 +22,27 @@ SecRule TX:dos-protection-plugin_enabled "@eq 0" "id:9514099,phase:1,pass,nolog,
 
 #
 # Description of mechanics:
-# When a request hits a non-static resource (TX:STATIC_EXTENSIONS), then a counter for the IP
-# address is being raised (IP:DOS_COUNTER). If the counter (IP:DOS_COUNTER) hits a limit
-# (TX:DOS_COUNTER_THRESHOLD), then a burst is identified (IP:DOS_BURST_COUNTER) and the
-# counter (IP:DOS_COUNTER) is reset. The burst counter expires within a timeout period
+# When a request hits a non-static resource (TX:STATIC_EXTENSIONS), a counter for the IP
+# address is increased (IP:DOS_COUNTER). If the counter (IP:DOS_COUNTER) hits a limit
+# (TX:DOS_COUNTER_THRESHOLD) withing a certain window (`TX:DOS_COUNTER_TIME_SLICE`), then a burst is identified (IP:DOS_BURST_COUNTER) and the
+# counter (IP:DOS_COUNTER) is reset. The burst counter expires after a defined amount of time
 # (TX:DOS_BURST_TIME_SLICE).
 # If the burst counter (IP:DOS_BURST_COUNTER) is greater equal 2, then the blocking flag
 # is being set (IP:DOS_BLOCK). The blocking flag (IP:DOS_BLOCK) expires within a timeout
 # period (TX:DOS_BLOCK_TIMEOUT). All this counting happens in phase 5.
 # There is a stricter sibling to this rule (9514170) in paranoia level 2, where the
 # burst counter check (IP:DOS_BURST_COUNTER) hits at greater equal 1.
 #
-# The blocking is done in phase 1: When the blocking flag is encountered (IP:DOS_BLOCK),
-# then the request is dropped without sending a response. If this happens, then a
-# counter is # raised (IP:DOS_BLOCK_COUNTER).
-# When an IP address is blocked for the first time, then the blocking is reported in a
-# message and a flag (IP:DOS_BLOCK_FLAG) is set. This flag expires in 60 seconds.
-# When an IP address is blocked and the flag (IP:DOS_BLOCK_FLAG) is set, then the
-# blocking is not being reported (to prevent a flood of alerts). When the flag
-# (IP:DOS_BLOCK_FLAG) has expired and a new request is being blocked, then the
-# counter (IP:DOS_BLOCK_COUNTER) is being reset to 0 and the block is being treated
+# The blocking is done in phase 1: When the blocking variable is set (IP:DOS_BLOCK),
+# then the request is dropped without sending a response. If this happens, a
+# counter is incremented (IP:DOS_BLOCK_COUNTER).
+# When an IP address is blocked for the first time, the blocking is reported in a
+# message and a flag (IP:DOS_BLOCK_FLAG) is set. This flag expires after the number of
+# seconds configured with TX:DOS_REPORTING_TIMEOUT.
+# When an IP address is blocked and the flag (IP:DOS_BLOCK_FLAG) is set, the
+# blocking is not reported (to prevent a flood of alerts). When the flag
+# (IP:DOS_BLOCK_FLAG) has expired (after TX:DOS_REPORTING_TIMEOUT seconds) and a new request is
+# blocked, the  counter (IP:DOS_BLOCK_COUNTER) is reset to 0 and the block is treated
 # as the first block (-> alert).
 # In order to be able to display the counter (IP:DOS_BLOCK_COUNTER) and resetting
 # it at the same time, we copy the counter (IP:DOS_BLOCK_COUNTER) into a different
@@ -56,13 +57,16 @@ SecRule TX:dos-protection-plugin_enabled "@eq 0" "id:9514099,phase:1,pass,nolog,
 # TX:DOS_BLOCK_COUNTER      Copy of IP:DOS_BLOCK_COUNTER (needed for display reasons)
 # TX:DOS_BLOCK_TIMEOUT      Period in seconds a blocked IP will be blocked
 # TX:DOS_COUNTER_THRESHOLD  Limit of requests, where a burst is identified
+# TX:DOS_COUNTER_TIME_SLICE Period in seconds of window for which to count requests against the current burst
 # TX:DOS_BURST_TIME_SLICE   Period in seconds when we will forget a burst
 # TX:STATIC_EXTENSIONS      Paths which can be ignored with regards to DoS
 #
-# As a precondition for these rules, please set the following three variables:
+# As a precondition for these rules, please set the following five variables:
 #  - TX:DOS_BLOCK_TIMEOUT
 #  - TX:DOS_COUNTER_THRESHOLD
+#  - TX:DOS_COUNTER_TIME_SLICE
 #  - TX:DOS_BURST_TIME_SLICE
+#  - TX:DOS_REPORTING_TIMEOUT
 #
 # And make sure that TX:STATIC_EXTENSIONS is also set.
 #
@@ -83,9 +87,11 @@ SecRule &TX:dos_burst_time_slice "@eq 0" \
     ver:'OWASP_CRS/3.4.0-dev',\
     chain,\
     skipAfter:END-DOS-PROTECTION-CHECKS"
-    SecRule &TX:dos_counter_threshold "@eq 0" \
+    SecRule &TX:dos_counter_time_slice "@eq 0" \
         "chain"
-        SecRule &TX:dos_block_timeout "@eq 0"
+        SecRule &TX:dos_counter_threshold "@eq 0" \
+            "chain"
+            SecRule &TX:dos_block_timeout "@eq 0"
 
 SecRule &TX:dos_burst_time_slice "@eq 0" \
     "id:9514110,\
@@ -96,9 +102,11 @@ SecRule &TX:dos_burst_time_slice "@eq 0" \
     ver:'OWASP_CRS/3.4.0-dev',\
     chain,\
     skipAfter:END-DOS-PROTECTION-CHECKS"
-    SecRule &TX:dos_counter_threshold "@eq 0" \
+    SecRule &TX:dos_counter_time_slice "@eq 0" \
         "chain"
-        SecRule &TX:dos_block_timeout "@eq 0"
+        SecRule &TX:dos_counter_threshold "@eq 0" \
+            "chain"
+            SecRule &TX:dos_block_timeout "@eq 0"
 
 
 SecRule TX:EXECUTING_PARANOIA_LEVEL "@lt 1" "id:9514011,phase:1,pass,nolog,skipAfter:END-REQUEST-9514-DOS-PROTECTION"
@@ -133,7 +141,7 @@ SecRule IP:DOS_BLOCK "@eq 1" \
         setvar:'ip.dos_block_flag=1',\
         setvar:'tx.dos_block_counter=%{ip.dos_block_counter}',\
         setvar:'ip.dos_block_counter=0',\
-        expirevar:'ip.dos_block_flag=60'"
+        expirevar:'ip.dos_block_flag=%{dos_reporting_timeout}'"
 
 
 #
@@ -201,6 +209,29 @@ SecRule REQUEST_BASENAME "@rx .*?(\.[a-z0-9]{1,10})?$" \
     SecRule TX:EXTENSION "!@within %{tx.static_extensions}" \
         "setvar:'ip.dos_counter=+1'"
 
+#
+# Make the DOS counter expire after TX:DOS_COUNTER_TIME_SLICE.
+# Only set the expiration once we at least once request was
+# counted and don't update the expiration time. We want to
+# measure the time between first request and the time the
+# burst threshold is hit.
+#
+SecRule &IP:DOS_COUNTER "@eq 1" \
+    "id:9514110,\
+    phase:5,\
+    pass,\
+    capture,\
+    t:none,\
+    nolog,\
+    tag:'application-multi',\
+    tag:'language-multi',\
+    tag:'platform-multi',\
+    tag:'paranoia-level/1',\
+    tag:'attack-dos',\
+    tag:'OWASP_CRS',\
+    tag:'capec/1000/210/227/469',\
+    ver:'OWASP_CRS/3.4.0-dev',\
+    expirevar:'ip.dos_counter=%{tx.dos_counter_time_slice}"
 
 #
 # Check DOS Counter

plugins/dos-protection-config.conf

@@ -47,22 +47,28 @@
 #
 # DoS protection against clients making requests too quickly.
 #
-# When a client is making more than 100 requests (excluding static files) within
-# 60 seconds, this is considered a 'burst'. After two bursts, the client is
-# blocked for 600 seconds.
+# When a client is making more than TX:DOS_COUNTER_THRESHOLD requests (excluding static files) within
+# TX:DOS_COUNTER_TIME_SLICE, this is considered a 'burst'. After two bursts, the client is
+# blocked for TX:DOS_BLOCK_TIMEOUT seconds. Bursts expire after TX:DOS_BURST_TIME_SLICE.
+#
+# TX:DOS_REPORTING_TIMEOUT controls how long to wait before reporting the next blocking action in the log.
+# It is important not to log every blocking action to avoid the log being filled (which could itself be
+# turned into an attack).
 #
 # Requests to static files are not counted towards DoS; they are listed in the
-# 'tx.static_extensions' setting, which you can change in this file.
+# TX.STATIC_EXTENSIONS setting, which you can change in this file.
 #
 SecAction \
     "id:9514900,\
     phase:1,\
     nolog,\
     pass,\
     t:none,\
+    setvar:'tx.dos_counter_time_slice=60',\
     setvar:'tx.dos_burst_time_slice=60',\
     setvar:'tx.dos_counter_threshold=100',\
-    setvar:'tx.dos_block_timeout=600'"
+    setvar:'tx.dos_block_timeout=600',\
+    setvar:'tx.dos_reporting_timeout=60"
 
 
 # File extensions considered static files.