Hello all,
The attachment is a patch to fix DoS effect on a large smtp server, I
think it is useful so I post it here.
Here are some references:
http://www.cerias.purdue.edu/coast/ms_penetration_testing/v32.html
http://www.uwsg.iu.edu/hypermail/linux/kernel/9903.3/0474.html
http://www.tux.org/hypermail/linux-net/1999-Sep/0479.html
It seems that this problem has been discussed a few times in kernel
list and networking list but no cure is made.
The LAST_ACK DoS is something like blocking server with thousands of
sockets left in LAST_ACK state.
I see the DoS effect in the smtp servers of a large ICP. The scenario
is:
[ACE Layer 4 Switch] ------Internet------[PIX firewall]
| |
^ Some Clients
----- -----
| | |
linux smtp servers pool
The ACE switch seems not to affect the results. Here is the DoS looks
like:
1. 3-way handshake and then a socket is established
2. SMTP server end socket S sends welcome message "200 ..." to client C
3. For some unknown reason, PIX(or client) sends FIN to server socket
S
4. S receives FIN, gives it to application, changes to CLOSE_WAIT state
5. Applicatioin closes socket, S sends out FIN, changes to LAST_ACK
state. Still, there are unsent data in skb .
6. No ACK from C, so S keeps in LAST_ACK state for about 15-20mins
during retries to send out data.
7. A new socket is established, and then it again becomes "ghost
connection"
8. step 1-7 again and again, Server dumbed because too many LAST_ACK
socket
Because there are many users(clients) behind PIX firewall, the sockets
in LAST_ACK state will increase and reach a fatal threshold that
server no longer responses.
The problem here is the PIX firewall. PIX firewall should do
something in response to "false" data from SMTP server, i.e, sends
back RST. But it doesn't. With this interactive bug, and provided
many PIX firewalls are there, sockets in LAST_ACK state can roar up
to 4000 in 5 minutes.
The annoying thing is that we know denying the PIX can't be good
point because we can't risk denying normal users. And if it is true,
normal users are just victims of this PIX bug, we can't be that cruel
to them. And we can't deny so many PIX's out there!
How the bad guy attack servers? Established TCP connection is
controllable to applications , so bad guy can't create too many
connections in a short time if the applications are well designed.
But 20min gives him good chances if he can make use of LAST_ACK
socket. Normally, bad guy can't attack servers by drop the
connections without further interaction, because applications can
handle that. But LAST_ACK is a kernel thing and applications have no
control over it, bad guy can sploit it to create as many "ghost
connections" as he will until the server dumbed.
Solaris can be tuned by set tcp_abort...(?) parameter to a small number
with ndd tool, but linux has no such thing. Generally, we can do
# echo 4 > /proc/sys/net/ipv4/tcp_retries1
# echo 5 > /proc/sys/net/ipv4/tcp_retries2
but the side effects to slow links can't be ignored.
Considering about the SMTP service characteristics: smtp server can
safely discard the unsent data in LAST_ACK state without problem,
because a smtp session should be closed by server, not client. If
a client close socket first, it doesn't matter whether server has
unsent data. And so do other services such as HTTP, telnet.
The attachment is a patch which handles LAST_ACK state differently. A
/proc/sys/net/ipv4/tcp_last_ack_retries is added, and the value takes
effect if it is between 1 to tcp_retries2. If <= 0, the feature is
disabled, if > tcp_retries2, tcp_retries2 takes effect. I think my
patch is simple and effective enough, but I may be wrong. The patch is
against 2.2.14 vanilla.
I do suggest using "4" ( 3+6+12+24 = 45s), but "1" seems ok :-)
Feedback is welcome. And I am not in the kenrel list, Please CC: me.
-- lark ------------114B21C582D7B45 Content-Type: application/octet-stream; name="last_ack.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="last_ack.patch"ZGlmZiAtdXIgbGludXgtMi4yLjE0Lm9sZC9pbmNsdWRlL2xpbnV4L3N5c2N0bC5oIGxpbnV4LTIu Mi4xNC9pbmNsdWRlL2xpbnV4L3N5c2N0bC5oCi0tLSBsaW51eC0yLjIuMTQub2xkL2luY2x1ZGUv bGludXgvc3lzY3RsLmgJV2VkIEphbiAgNSAwNDoxNDo0MSAyMDAwCisrKyBsaW51eC0yLjIuMTQv aW5jbHVkZS9saW51eC9zeXNjdGwuaAlGcmkgQXByICA3IDAxOjI1OjM2IDIwMDAKQEAgLTIyNyw3 ICsyMjcsOCBAQAogCU5FVF9JUFY0X0lDTVBfRUNIT1JFUExZX1JBVEU9NjMsCiAJTkVUX0lQVjRf SUNNUF9JR05PUkVfQk9HVVNfRVJST1JfUkVTUE9OU0VTPTY0LAogCU5FVF9JUFY0X0lHTVBfTUFY X01FTUJFUlNISVBTPTY1LAotCU5FVF9JUFY0X0FMV0FZU19ERUZSQUc9NjcKKwlORVRfSVBWNF9B TFdBWVNfREVGUkFHPTY3LAorCU5FVF9JUFY0X1RDUF9MQVNUX0FDS19SRVRSSUVTPTY4CiB9Owog CiBlbnVtIHsKZGlmZiAtdXIgbGludXgtMi4yLjE0Lm9sZC9pbmNsdWRlL25ldC90Y3AuaCBsaW51 eC0yLjIuMTQvaW5jbHVkZS9uZXQvdGNwLmgKLS0tIGxpbnV4LTIuMi4xNC5vbGQvaW5jbHVkZS9u ZXQvdGNwLmgJV2VkIEphbiAgNSAwNToyNTo1OSAyMDAwCisrKyBsaW51eC0yLjIuMTQvaW5jbHVk ZS9uZXQvdGNwLmgJRnJpIEFwciAgNyAwMTowNDo1OSAyMDAwCkBAIC0yNDUsNiArMjQ1LDEyIEBA CiAJCQkJICogOTAgbWludXRlcyB0byB0aW1lIG91dC4KIAkJCQkgKi8KIAorI2RlZmluZSBUQ1Bf TEFTVF9BQ0tfUkVUUklFUyAwCS8qCisJCQkJICogTEFTVF9BQ0sgcmV0cmllcywgYWdhaW5zdCBM QVNUX0FDSyBEb1MKKwkJCQkgKiAwIG1lYW5zIGRpc2FibGUgdGhpcyBmZWF0dXJlCisJCQkJICog ZGVmYXVsdCBpcyAwCisJCQkJICovCisKICNkZWZpbmUgVENQX1RJTUVPVVRfTEVOCSgxNSo2MCpI WikgLyogc2hvdWxkIGJlIGFib3V0IDE1IG1pbnMJCSovCiAjZGVmaW5lIFRDUF9USU1FV0FJVF9M RU4gKDYwKkhaKSAvKiBob3cgbG9uZyB0byB3YWl0IHRvIHN1Y2Nlc3NmdWxseSAKIAkJCQkgICog Y2xvc2UgdGhlIHNvY2tldCwgYWJvdXQgNjAgc2Vjb25kcwkqLwpkaWZmIC11ciBsaW51eC0yLjIu MTQub2xkL25ldC9pcHY0L3N5c2N0bF9uZXRfaXB2NC5jIGxpbnV4LTIuMi4xNC9uZXQvaXB2NC9z eXNjdGxfbmV0X2lwdjQuYwotLS0gbGludXgtMi4yLjE0Lm9sZC9uZXQvaXB2NC9zeXNjdGxfbmV0 X2lwdjQuYwlXZWQgT2N0IDIwIDA4OjE0OjAyIDE5OTkKKysrIGxpbnV4LTIuMi4xNC9uZXQvaXB2 NC9zeXNjdGxfbmV0X2lwdjQuYwlGcmkgQXByICA3IDAxOjAxOjU1IDIwMDAKQEAgLTU2LDYgKzU2 LDcgQEAKIGV4dGVybiBpbnQgc3lzY3RsX3RjcF9tYXhfa2FfcHJvYmVzOwogZXh0ZXJuIGludCBz eXNjdGxfdGNwX3JldHJpZXMxOwogZXh0ZXJuIGludCBzeXNjdGxfdGNwX3JldHJpZXMyOworZXh0 ZXJuIGludCBzeXNjdGxfdGNwX2xhc3RfYWNrX3JldHJpZXM7CiBleHRlcm4gaW50IHN5c2N0bF90 Y3BfZmluX3RpbWVvdXQ7CiBleHRlcm4gaW50IHN5c2N0bF90Y3Bfc3luY29va2llczsKIGV4dGVy biBpbnQgc3lzY3RsX3RjcF9zeW5fcmV0cmllczsKQEAgLTE2Niw2ICsxNjcsOCBAQAogCSAmc3lz Y3RsX2ludHZlYywgTlVMTCwgTlVMTCwgJnRjcF9yZXRyMV9tYXh9LAogCXtORVRfSVBWNF9UQ1Bf UkVUUklFUzIsICJ0Y3BfcmV0cmllczIiLAogCSAmc3lzY3RsX3RjcF9yZXRyaWVzMiwgc2l6ZW9m KGludCksIDA2NDQsIE5VTEwsICZwcm9jX2RvaW50dmVjfSwKKwl7TkVUX0lQVjRfVENQX0xBU1Rf QUNLX1JFVFJJRVMsInRjcF9sYXN0X2Fja19yZXRyaWVzIiwKKwkgJnN5c2N0bF90Y3BfbGFzdF9h Y2tfcmV0cmllcywgc2l6ZW9mKGludCksIDA2NDQsIE5VTEwsICZwcm9jX2RvaW50dmVjfSwKIAl7 TkVUX0lQVjRfVENQX0ZJTl9USU1FT1VULCAidGNwX2Zpbl90aW1lb3V0IiwKIAkgJnN5c2N0bF90 Y3BfZmluX3RpbWVvdXQsIHNpemVvZihpbnQpLCAwNjQ0LCBOVUxMLCAKIAkgJnByb2NfZG9pbnR2 ZWNfamlmZmllcywgJnN5c2N0bF9qaWZmaWVzfSwKZGlmZiAtdXIgbGludXgtMi4yLjE0Lm9sZC9u ZXQvaXB2NC90Y3BfdGltZXIuYyBsaW51eC0yLjIuMTQvbmV0L2lwdjQvdGNwX3RpbWVyLmMKLS0t IGxpbnV4LTIuMi4xNC5vbGQvbmV0L2lwdjQvdGNwX3RpbWVyLmMJV2VkIEphbiAgNSAwNDoxNDo0 MSAyMDAwCisrKyBsaW51eC0yLjIuMTQvbmV0L2lwdjQvdGNwX3RpbWVyLmMJRnJpIEFwciAgNyAw MTowMDo0MCAyMDAwCkBAIC0yNyw2ICsyNyw3IEBACiBpbnQgc3lzY3RsX3RjcF9rZWVwYWxpdmVf cHJvYmVzID0gVENQX0tFRVBBTElWRV9QUk9CRVM7CiBpbnQgc3lzY3RsX3RjcF9yZXRyaWVzMSA9 IFRDUF9SRVRSMTsKIGludCBzeXNjdGxfdGNwX3JldHJpZXMyID0gVENQX1JFVFIyOworaW50IHN5 c2N0bF90Y3BfbGFzdF9hY2tfcmV0cmllcyA9IFRDUF9MQVNUX0FDS19SRVRSSUVTOwogCiBzdGF0 aWMgdm9pZCB0Y3Bfc2x0aW1lcl9oYW5kbGVyKHVuc2lnbmVkIGxvbmcpOwogc3RhdGljIHZvaWQg dGNwX3N5bl9yZWN2X3RpbWVyKHVuc2lnbmVkIGxvbmcpOwpAQCAtMTU3LDYgKzE1OCwxMiBAQAog CiAJLyogSGFzIGl0IGdvbmUganVzdCB0b28gZmFyPyAqLwogCWlmICh0cC0+cmV0cmFuc21pdHMg PiBzeXNjdGxfdGNwX3JldHJpZXMyKSAKKwkJcmV0dXJuIHRjcF93cml0ZV9lcnIoc2ssIDApOwor CisJLyogTEFTVF9BQ0sgdGltZW91dCAqLworCWlmICgoc2stPnN0YXRlID09IFRDUF9MQVNUX0FD SyApICYmIAorCSAgICAoc3lzY3RsX3RjcF9sYXN0X2Fja19yZXRyaWVzID4gMCkgJiYgCisJICAg ICh0cC0+cmV0cmFuc21pdHMgPiBzeXNjdGxfdGNwX2xhc3RfYWNrX3JldHJpZXMpKQogCQlyZXR1 cm4gdGNwX3dyaXRlX2VycihzaywgMCk7CiAKIAlyZXR1cm4gMTsK ------------114B21C582D7B45--
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/