Back to Top

Category: Firewall

Alternative to robots.txt

Many web spiders and especially ‘dodgy’ content bots do not respect the robots.txt file.. below is some code which can be added to your .htaccess file which will help block bots which use user_agents. Banning via ip address although useful is a bit of a losing battle, as originator can just switch to another proxy.

## Bot Protection
RewriteCond %{HTTP_USER_AGENT} (Access|appid) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (Capture|Client|Copy|crawl) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (Data|devSoft|Domain|download) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (Engine|fetch|filter|genieo) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (Jakarta|Java|Library|link|wsr-agent|MJ12bot|SeznamBot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (AhrefsBot|MJ12bot|nutch|Preview|Proxy|Publish|Kraken|Baiduspider) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (scraper|spider) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (Win32|WinHttp) [NC]
RewriteRule .* - [F] 
## End Bot Protection
Posted in Firewall |

htaccess protect wp-login.php file

There is a major bout of wordpress hacking doing the rounds.. which on servers with many hundreds of wordpress sites can equate to a denial of service attack.

Try the following as a means to protect ram and cpu resources by .htaccess protecting wp-login.php file from public access. The following method assumes you are using something like cpanel’s file manager, but could be done locally as well with relevant files.

1) Edit root wordpress level .htaccess file and add the following to the top of the file:

1
2
3
4
5
6
7
8
9
10
# BEGIN Admin Protection
ErrorDocument 401 "Unauthorized Access"
ErrorDocument 403 "Forbidden"
<FilesMatch "wp-login.php">
AuthName "Authorized Only"
AuthType Basic
AuthUserFile /home/{account}/.wppassword
require valid-user
</FilesMatch>
# END Admin Protection

Replace {account} with your’s account account name. Save.

2) Next go to the top level – above public folders and files and create a blank file called .wppassword and using an external service like http://www.htaccesstools.com/htpasswd-generator/ create a username and a password to be used in this .wppassword file. Copy returned content and add to your just created .wppassword file and save.

3) Now go to login to your wordpress site.. a popup box should appear asking for your username and password, which will be the same as you selected for (2) above. If not, please check which .htaccess file you edited.. should be the same level as the wp-login.php file – same directory, or if username/password not accepted, check the path to your .wppassword file eg line AuthUserFile /home/{account}/.wppassword

I hope this helps..we have a server with many hundreds of wordpress sites, and due to hacking server load kept on soaring from 4.0 to well over 100 crashing the web server and also denting search engines ability to index the websites..

portflood protection can also provide temporary cover, but false positives tend to not make it a long term solution.

Posted in Firewall, Wordpress |

A php script to find blacklisted ips connected to server

This is a simple script which I find useful in finding blacklisted ips using server (spammers etc), via a third party api (yasb.intuxication.org), connected to server, so can permanently ban them on the server firewall.

Log into the server as root via terminal / sshd (I use putty under Linux), and run this following standard network command to produce a file of ips connected to the server (snapshot)..

netstat -ntu | grep -v “::” | grep “:” | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -nr > ip-check.txt

This will produce a file ip-check.txt, which contains lines of the format: No of connections ip address

eg

2 127.0.0.1
1 213.199.179.141
1 199.59.148.82
1 174.133.195.84

(your list on a web server will be much longer than this example and likely to have many more connections per ip)

Next step is to create a file called check-spam-ips.php in the same folder as ip-check.txt was just created above.

which uses the following php code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
<?php
 
$data=file('ip-check.txt');
 
foreach($data as $line) {
 
$line = trim($line);
list($count,$ip)=explode(' ',$line);
$url = "http://yasb.intuxication.org/api/check.xml?ip=".$ip;
 
$info=file_get_contents($url);
 
$orgxml = simplexml_load_string($info);
 
if ($orgxml===false) {
    echo "Failed loading XML\n";
    foreach(libxml_get_errors() as $error) {
        echo $error->message."<br>\n";
    }
exit;
}
 
$spam=$orgxml->spam;
 
if($spam=='true') {
echo($ip." ".$spam."\n");
}
 
}
 
?>

Then run the script as follows..

php check-spam-ips.php

This will then produce to the screen a list of ips connected to server found to have been previously blacklisted.

You can then use your servers portal software or iptables to ban these blacklisted ips or investigate further.

I hope this is of use to anyone.

Posted in Firewall |