Linux 4 All

Best Linux tricks source

Posts in the bash category

Some time ago I had to deal with a request to delete all but the newest 50 files from a folder. To make my life a little bit miserable, there were other folders inside that folder which shouldn’t be deleted and obviously some files with space in their names

In the end I did a small script taking the folder as first parameter and number of files to keep as the second one.

#!/bin/bash
if [ $# -eq 0 ]
 then
  echo " This script will delete all but newest files from a specified folder."
  echo ""
  echo " Usage: source $0  "
  exit 1
fi

TOTAL_FILES=$(find $1 -maxdepth 1 -type f -print0| xargs -0 ls -lt | wc -l)
FILES_TO_DELETE=$(echo "$TOTAL_FILES-$2" | bc)

find $1 -maxdepth 1 -type f -print0 | xargs -0 ls -t | tail -$FILES_TO_DELETE | sed -e 's/ /\\ /g' | xargs rm

The following one-liner will display number of Apache PIDs and sort them after memory usage:

ps aux | awk '$11 ~ /httpd/ {c++; SUM +=$6; print $6/1024" MB, PID:", $2|"sort -rn| head"} END {print c" Total Apache Processes"} END {print SUM/1024" MB Total Memory"} END {print "Top 10 Memory users:"}'

In order to change the system timezone, there are 6 simple steps you need to follow:

1. Edit /etc/sysconfig/clock. It’s up to you if you’d like UTC to be false or true.

ZONE="Europe/London"
UTC=false

2. Edit /etc/php.ini

date.timezone = Europe/London

3. Update system time. If you are running it on a cloud server, second command is not required.

tzdata-update
hwclock --systohc

4. Restart all effected services, including MySQL if necessary.

service crond restart
service rsyslog restart
service syslog restart
service httpd restart
service mysqld restart

5. Restart NTP

service ntpd stop
ntpdate 0.centos.pool.ntp.org
service ntpd start

6. Run date to confirm date was changed on your server.

A small and simple script to get users from a RHEL/CentOS box. It assumes that first UID is 500 and it won’t go beyond UID 999

1. Get all users

awk -F ":"  '$3>=500 && $3<1000  {print $1}' /etc/passwd

2. Get users with valid shell.

awk -F ":"  '($3>=500 && $3<1000) && ($7 ~ /sh/) {print $1,$7}' /etc/passwd

3. Get ftp users

awk -F ":"  '($3>=500 && $3<1000) && ($7 !~ /sh/) {print $1,$7}' /etc/passwd

Strip html tags

  • In Linux
  • On 10 Nov | '2014

The following sed command will strip out html tags from a file. That’s useful to us with curl to get a nice output on the screen.

sed 's|<[^>]*>||g'

This is a small shell script to get all files greater than 100M starting from “/”. Replace SIZE and FS values to fit your needs.

SIZE=100M; FS='/'; find $FS -size +$SIZE -exec du -hs {} \; 2>/dev/null | sort -nrk 1,2

A little “find and sort” magic to find largest 50 files on a server:

find / -type f -size '+1024k' -printf "%s %h/%f\n" | \
sort -rn -k1 | \
head -n50 | \
awk '{ printf("%5dMB\t%s\n", $1/1048576, substr($0, index($0, " ")+1, length($0))) }' 

People do stupid things. Most common is to recursively change all permission on / instead of just a folder:

chown -R user / var/www/html

Ooops, I accidentally added a space there and now I’m screwed. I will have to reinstall the system. 🙁 Well, not exactly. If you were careful enough to get ACL backup of your file system, you might have a chance:

getfacl -epR / | sed -e 's/\/\//\//g' >$(hostname).acl

Restoring all the permissions from an acl file is easy. All you need is running the following command:

setfacl --restore=$(hostname).acl

If you didn’t destroy everything, but just a folder, first you will need to use a text parser like awk, sed, perl or whatever suits you to get a specific folder ACL from the file you now have. Below example restores /etc

sed -n '/^#\ file:\ \/etc/,/^$/p' $(hostname).acl > etc.acl
setfacl --restore=etc.acl

For this, we are going to use egrep with -o option to print only the matched parts and not the entire line.

-o, –only-matching
Print only the matched (non-empty) parts of a matching line, with each such part on a separate output line.

Below there are 2 examples:

egrep -o "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" $LOGFILE
egrep -o "([0-9]{1,3}\.){3}[0-9]{1,3}" $LOGFILE

When troubleshooting a server that is slow, sometimes you have to see top something CPU consuming processes.

The command to see Top 10 CPU processes is:

ps auxf | sort -nr -k 3 | head -10