Linux 4 All

Best Linux tricks source

Posts in the HowTo category

When Nginx is running on a server behind a Load Balancer, by default all traffic will come from Load Balancer IP, thus Nginx will only log the Load Balancer IP. In order to get your visitor real IP address, you can use X-Forwarded-For header.

In order to configure Nginx to log real IP address, add the following to nginx.conf, http section:

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    set_real_ip_from 127.0.0.1; # Varnish IP address
    set_real_ip_from 10.0.0.0/8; # Load Balancer IP range

    real_ip_header X-Forwarded-For;
    real_ip_recursive on;

Here is a little curl magic to get you how long it it will take to connect to a site. It will display time to connect, time to namelookup, time to first byte and so on.

curl -s -w "\ntime connect: %{time_connect}\ntime namelookup: %{time_namelookup}\ntime pretransfer: %{time_pretransfer}\ntime redirect: %{time_redirect}\ntime to first byte: %{time_starttransfer}\ntime total: %{time_total}\nhttp code: %{http_code}\n\n" -o /dev/null http://linux4all.uk

time connect: 0.165
time namelookup: 0.133
time pretransfer: 0.166
time redirect: 0.000
time to first byte: 0.392
time total: 0.432
http code: 200

Some time ago I had to deal with a request to delete all but the newest 50 files from a folder. To make my life a little bit miserable, there were other folders inside that folder which shouldn’t be deleted and obviously some files with space in their names

In the end I did a small script taking the folder as first parameter and number of files to keep as the second one.

#!/bin/bash
if [ $# -eq 0 ]
 then
  echo " This script will delete all but newest files from a specified folder."
  echo ""
  echo " Usage: source $0  "
  exit 1
fi

TOTAL_FILES=$(find $1 -maxdepth 1 -type f -print0| xargs -0 ls -lt | wc -l)
FILES_TO_DELETE=$(echo "$TOTAL_FILES-$2" | bc)

find $1 -maxdepth 1 -type f -print0 | xargs -0 ls -t | tail -$FILES_TO_DELETE | sed -e 's/ /\\ /g' | xargs rm

Assuming you have a huge incoming mail queue and you need to remove it quickly, here is how:

cd /var/spool/postfix
service postfix stop
mv incoming incoming.spam
mkdir incoming
chown postfix.root incoming
chmod 700 incoming
service postfix start

Here is how:

SELECT TABLE_SCHEMA, TABLE_NAME FROM information_schema.statistics WHERE index_type LIKE ‘FULLTEXT%’;

This is important as MySQL prior to 5.6 does not support FULLTEXT InnoDB tables.

Should you need to disable Trace and Track methods on Apache, add the following lines to httpd.conf and restart Apache.


 RewriteEngine on
 RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK)
 RewriteRule .* - [F]

TraceEnable off

I admit that is pretty lame, but until today I had no idea how to kill a specific MySQL query. It is quite easy.

First we need to get the queries:

mysql> show processlist;
+------+------+-----------+------+---------+------+-------+------------------+
| Id   | User | Host      | db   | Command | Time | State | Info             |
+------+------+-----------+------+---------+------+-------+------------------+
| 5585 | root | localhost | NULL | Query   |    0 | NULL  | show processlist |
+------+------+-----------+------+---------+------+-------+------------------+
1 row in set (0.00 sec)

Then kill the id:

mysql> kill 5585;
ERROR 1317 (70100): Query execution was interrupted
mysql>

Here is how to test IMAP, POP and SMTP authentication using telnet or openssl:

POP3 test:

telnet $SERVER_IP 110
+OK Hello there. 
USER user@example.com
+OK Password required.
PASS Password
+OK logged in.

IMAP test:

telnet $SERVER_IP 143
a1 LOGIN user@example.com Password
a1 OK LOGIN Ok.

POP3-SSL test:

openssl s_client -connect $SERVER_IP:995
+OK Hello there. 
USER user@example.com
+OK Password required.
PASS Password
+OK logged in.

IMAP-SSL test:

openssl s_client -connect $SERVER_IP:993
a1 LOGIN user@example.com Password
a1 OK LOGIN Ok.

For the SMTP test, first we need to generate base64 encrypted username/password hash and then we can test it:

printf '\0%s\0%s' 'user@example.com' 'Password' | openssl base64
AHVzZXJAZXhhbXBsZS5jb20AUGFzc3dvcmQ=

telnet $SERVER_IP 25
AUTH PLAIN AHVzZXJAZXhhbXBsZS5jb20AUGFzc3dvcmQ=
235 2.7.0 Authentication successful AHVzZXJAZXhhbXBsZS5jb20AUGFzc3dvcmQ=

e-mail

photo credit: Micky.! via photopin cc

In a previous post I was explaining how to log visitor real IP address in access_log. Now it’s time to do the same for error_log.

For this we are going to compile and install a module called mod_vgremoteip

Steps outlined bellow:

1. Get the module

git clone https://github.com/vgno/mod_vgremoteip.git

2. Install gcc and httpd-devel tools
yum install httpd-devel
yum install gcc

3. Compile the module
apxs -a -i -c mod_vgremoteip.c

4. Configure Apache to use it
LoadModule vgremoteip_module  modules/mod_vgremoteip.so

# Name of header which contains the 'real' client IP.
 VGRemoteIPHeader X-Forwarded-For
# Subnet to mark as trusted subnet (this ip will be allowed to set the X-Forwarded-For header and marked as a proxy ip).
# You should specify this.
VGTrustedProxy 10.0.0.0/8
# You can also specify a single ip addresses.
# Do not specify hostnames.
VGTrustedProxy 127.0.0.1

5. Restart Apache and you are done.
service httpd restart

Get mail queue

When dealing with mail issues, the following one-liner will display all Postfix mail IDs from the queue.

postqueue -p | egrep -v ^[[:space:]] | grep -v ^$ | awk '{print $1}' | grep -v ^\( | grep -v ^\- | sed -e 's/\*//g'