How to Protect a Dedicated Server
from DDoS Attacks

This guide covers DDoS protection for people who rent or colocate dedicated servers. We sell Flowtriq, a per-server DDoS detection agent, so we are transparent about where our product fits into the stack. Most of this guide is free, vendor-neutral advice you can apply right now.

The dedicated server DDoS problem

A dedicated server sits on a public IP. Unlike cloud instances behind a load balancer or CDN, your server is directly reachable. Every service you expose, whether it is SSH, a game server, a database, or an API, is a potential target. When someone decides to send traffic your way, it is just you and your hosting provider.

Some providers offer basic anti-DDoS. OVH includes their Anti-DDoS system, Hetzner provides DDoS Protection, and Leaseweb has DDoS IP Protection. These are better than nothing, but they share common limitations:

• Detection is blunt. Most provider systems trigger on raw traffic volume. A 500 Mbps spike gets flagged whether it is a real attack or a legitimate traffic burst from a product launch.
• Mitigation is binary. The typical response is either scrubbing (with limited configurability) or null routing the IP entirely. Null routing stops the attack but also takes your server offline.
• You get no visibility. No alert, no classification of the attack type, no packet captures. You find out something happened when customers start complaining or when you notice your IP is unreachable.
• No per-service awareness. Provider systems see aggregate traffic to your IP. They cannot tell you whether the attack targeted your web server on port 443, your game server on port 27015, or your DNS resolver on port 53.

The practical solution is defense in depth: free OS-level hardening as a baseline, firewall rules for connection-based attacks, provider protection for volumetric floods, and a detection agent for visibility and alerting. Each layer handles a different class of threat.

Layer 1: Kernel and OS hardening

These are free, always-on defenses that should be configured on every dedicated server regardless of whether you expect to be attacked. They harden your TCP/IP stack against common flood techniques and reduce the impact of connection exhaustion attacks.

Enable SYN cookies

SYN cookies let your server handle SYN flood attacks without allocating memory for half-open connections. When the SYN backlog is full, the kernel encodes connection state into the SYN-ACK sequence number instead of storing it in memory. Legitimate clients complete the handshake normally; flood traffic gets dropped without resource consumption.

echo 1 > /proc/sys/net/ipv4/tcp_syncookies

To make it persistent across reboots, add to /etc/sysctl.conf:

net.ipv4.tcp_syncookies = 1

Increase connection tracking table size

The default connection tracking table (conntrack) on most Linux distributions holds 65,536 entries. Under a connection flood, this fills up quickly and causes legitimate connections to be dropped. On a dedicated server with available RAM, increase it substantially.

# Allow up to 1 million tracked connections
net.netfilter.nf_conntrack_max = 1048576

# Increase the hash table size (set at boot via modprobe)
# Add to /etc/modprobe.d/conntrack.conf:
options nf_conntrack hashsize=262144

Also reduce the timeout for established connections sitting idle, which frees up table entries faster:

# Reduce from default 432000 (5 days) to 1 hour
net.netfilter.nf_conntrack_tcp_timeout_established = 3600

Disable ICMP redirect acceptance

ICMP redirects can be used to manipulate your server's routing table. There is almost never a legitimate reason for a dedicated server to accept ICMP redirects. Disable them:

net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv6.conf.all.accept_redirects = 0

Tune TCP backlog and somaxconn

The TCP backlog controls how many pending connections the kernel queues before dropping new ones. The defaults are conservative for a dedicated server.

# Increase the listen backlog
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535

# Increase network buffer sizes
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.netdev_max_backlog = 65535

Enable reverse path filtering

Reverse path filtering drops packets with spoofed source addresses. Many DDoS attacks rely on IP spoofing, and rp_filter catches the obvious cases where the source IP could not have arrived on the interface it came in on.

net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

Apply all changes with sysctl -p. These settings survive reboots when placed in /etc/sysctl.conf or a file in /etc/sysctl.d/.

Layer 2: iptables / nftables rate limiting

Firewall rules on your server can stop connection-based attacks before they reach your applications. This layer handles SYN floods that exceed what SYN cookies can absorb, connection exhaustion attempts, and port scanning.

Rate limit new connections per source IP

This limits how many new TCP connections a single IP can open per second. Legitimate users rarely open more than 20-30 connections per second to a single server. Attackers open thousands.

# iptables: limit new connections to 25/sec per source IP, burst of 50
iptables -A INPUT -p tcp --syn -m connlimit --connlimit-above 50 --connlimit-mask 32 -j DROP
iptables -A INPUT -p tcp --syn -m hashlimit --hashlimit-above 25/sec --hashlimit-burst 50 \
  --hashlimit-mode srcip --hashlimit-name syn_flood -j DROP

# nftables equivalent
nft add rule inet filter input tcp flags syn / syn,ack \
  meter syn_flood { ip saddr limit rate over 25/second burst 50 packets } drop

Drop invalid packets

Packets that do not match any known connection and are not valid new connections should be dropped. These are often part of scanning or malformed flood traffic.

iptables -A INPUT -m conntrack --ctstate INVALID -j DROP

Limit ICMP

ICMP echo (ping) is useful for diagnostics but can be abused in ICMP flood attacks. Rate limiting it to a few packets per second keeps diagnostics working while preventing abuse.

iptables -A INPUT -p icmp --icmp-type echo-request -m limit --limit 5/sec --limit-burst 10 -j ACCEPT
iptables -A INPUT -p icmp --icmp-type echo-request -j DROP

For automated rule generation tailored to your server's services, use our iptables rule generator or nftables rule generator.

Layer 3: Provider-level protection

Your hosting provider is the first line of defense against volumetric attacks. They can filter traffic before it reaches your server's port, which is the only way to stop attacks that exceed your bandwidth.

What providers typically offer

• OVH Anti-DDoS: Included with all dedicated servers. Uses VAC (Vacuum and Clean) infrastructure to scrub traffic. Activates automatically on detected attacks. Effective against common volumetric attacks but limited configurability.
• Hetzner DDoS Protection: Included free. Scrubs traffic through their mitigation infrastructure. Works well for UDP amplification and SYN floods. Limited to automatic mode with no user-facing controls.
• Leaseweb DDoS IP Protection: Included on most plans. Detects and scrubs attacks up to certain thresholds. Higher tiers available for purchase.
• Colocation providers: Varies widely. Some include basic null routing, others offer nothing. If you colocate, ask your provider what DDoS protection is included before you need it.

How upstream null routing works

When provider-level scrubbing cannot handle an attack (or when the provider does not offer scrubbing), the fallback is BGP null routing. Your provider announces a route to your IP that points to a null interface, effectively telling the internet to stop sending traffic to that IP. This is extremely effective at stopping the attack, but it also stops all legitimate traffic. The attacker wins by causing your provider to take your IP offline.

Null routing is useful as a last resort. It protects other customers on the same network segment from collateral damage. But from your perspective, your server is offline either way.

The visibility gap

The fundamental problem with provider-level protection is that you have no visibility into what happened. You typically cannot see:

• When exactly the attack started and stopped
• What type of attack it was (SYN flood, UDP amplification, DNS reflection, etc.)
• How much traffic was involved
• Whether the attack targeted a specific service or port
• Whether there were multiple attack vectors

Without this information, you cannot make informed decisions about hardening your server, reporting to upstream providers, or communicating with affected customers. This is where per-server detection fills the gap.

Layer 4: Per-server detection and alerting

Kernel hardening is passive. Firewall rules are reactive. Provider protection is opaque. The missing piece is active detection: something that watches your traffic in real time, classifies what is happening, and tells you about it immediately.

This is what Flowtriq does. Install a lightweight agent on your server that:

• Monitors all traffic in real time. The agent watches packet rates, byte rates, protocol distribution, and connection patterns. It builds a dynamic baseline of normal traffic so it can distinguish attacks from legitimate spikes.
• Alerts you the moment an attack starts. Slack, Discord, PagerDuty, SMS, webhook. You choose the channel. Alerts fire within seconds of detection, not minutes.
• Classifies the attack type. Not just "high traffic," but SYN flood, UDP amplification, DNS reflection, NTP amplification, memcached reflection, ICMP flood, or multi-vector. Each classification includes a confidence score.
• Captures a PCAP for forensics. Automatic packet capture during attacks gives you the evidence to report to upstream providers, file abuse complaints, or document incidents for compliance.
• Optionally triggers BGP blackhole announcements. If you have a BGP session with your upstream, the agent can trigger an RTBH announcement automatically when an attack exceeds a threshold you define. This gives you automated upstream mitigation without waiting for your provider to detect the attack.

The agent runs in userspace, uses minimal CPU and memory, and does not interfere with your server's workload. It is not a mitigation appliance. It is a detection and alerting layer that gives you the visibility your provider does not.

$9.99/month per server. 14-day free trial. No bandwidth tiers, no activation fees.

Putting it all together

A properly protected dedicated server uses all four layers. Each handles a different class of attack, and together they provide defense in depth at minimal cost:

1. Kernel hardening (free, always-on). SYN cookies, increased connection tracking, reverse path filtering, tuned TCP backlog. Handles the baseline noise of the internet: port scans, low-rate floods, connection exhaustion attempts.
2. iptables/nftables rules (free, always-on). Rate limiting per source IP, dropping invalid packets, ICMP limits. Stops moderate connection-based attacks and application-layer probes before they reach your services.
3. Provider DDoS protection (usually included). Volumetric scrubbing or null routing from your hosting provider. Handles the large floods that would saturate your uplink. Effective but opaque.
4. Flowtriq agent ($9.99/month). Real-time detection, attack classification, multi-channel alerting, PCAP forensics, optional BGP trigger. This is the layer that tells you what is happening and gives you the data to respond intelligently.

Total cost for a single dedicated server: $9.99/month on top of whatever you are already paying for hosting. Everything else is free and should already be configured.

Quick install

pip install ftagent --break-system-packages
sudo ftagent --setup
sudo ftagent --install-service && sudo systemctl enable --now ftagent

Setup takes about 5 minutes. The agent starts monitoring immediately and will alert you on the channels you configure during setup.

Frequently asked questions