Performance Tuning Apache Web Server – Enabled Modules

There are many different aspects to tuning Apache. In this post, I will focus on tuning which modules are compiled into Apache and also dynamically enabled. The more modules you have enabled, the more memory Apache will consume and the more processing it needs to perform; try to enable the minimum amount of modules required on your Web server to improve performance and reduce memory consumption:

1) First, list which Apache modules you have enabled on your server:

apache2 -l — this will list all the compiled in modules

a2dismod — this will list all dynamically loaded modules

2) Decide which modules you can disable and disable them:

For the compiled in modules, the list for the compiled modules in Ubuntu’s Apache web server will look something like this:

Compiled in modules:
  core.c
  mod_log_config.c
  mod_logio.c
  prefork.c
  http_core.c
  mod_so.c

These are usually required modules. You may be able to do without these in certain circumstances:

mod_log_config.c — This module is required for logging. If you want to disable logging, you can do without. This is usually not recommended though.
prefork.c — This module is only needed if you use Apache’s prefork mode rather than using threaded workers.

If you determine that you can live without some of these modules, you can choose to recompile Apache without the unnecessary modules.

For the dynamically loaded modules, the output will look something like:

~$ a2dismod
Your choices are: alias auth_basic authn_file authz_default authz_groupfile authz_host
authz_user autoindex cgi dav dav_svn deflate dir env mime negotiation passenger php5
rewrite setenvif ssl status wsgi
Which module(s) do you want to disable (wildcards ok)?

Which modules you require depend strongly on your application. Try to research what each module does and determine if it is required. Once you have determined that a module is not necessary, you can disable it via

sudo a2dismod [module name]
sudo /etc/init.d/apache2 reload

3) Recompile apache with only the needed modules?

Once you have your system narrowed down to the exact list of modules you need, you can gain a bit more performance by compiling them directly into apache rather than loading them dynamically. However, keep in mind that this makes future updating (security updates etc.) much more complex. Usually your time is better spent on other optimizations.

  • Share/Bookmark

Find the largest image file within a directory structure (e.g. largest jpeg on disc)

For our latest project, Urbo.ro, I needed to find the largest image file (.jpg) in a big directory structure with 50,000+ images but also other files in it. Manually searching it would have been impossible. I turned to our Linux ninja, Alexandru Ionica, who came up with the following that I wanted to share:

find /directory/ -iname "*.extension" -printf "%s " -print | sort -n -r | head -n1

This Linux snippet give you the largest file that ends with .extension inside of whichever directory structure you specify. To search the whole file system, just specify /.

  • Share/Bookmark

How to unmount a disk drive in Ubuntu (device is busy)?

Unmounting disks in Ubuntu is a bit more challenging than it should be. Say you have an external drive mounted on /media/disk. Trying the standard

sudo umount /media/disk

will often result in an error such as

umount: /media/drive: device is busy.

It is advisable to check what files may be open on the drive with

lsof | grep /media/disk

If something important is still open on the disk, make sure to close the files or programs using the disk. If nothing else should be keeping the disk busy, you can usually achieve an unmount with

sudo umount -vl /mnt/external_disk

This performs a verbose, lazy unmount – i.e. Ubuntu unmounts the disk drive when the device is no longer busy, showing any errors in the unmount if there are any.

To confirm that the device has been properly unmounted, use

mount

This will list all the currently mounted devices – your drive should no longer be in the list. If your list is too long or you don’t feel like scanning it mount | grep /media/disk will return nothing if the device has been successfully unmounted.

In case of an unresponsive NFS mount, you can also try

sudo umount -vf /mnt/nameofnfsmount

The -f argument forces an unmount for unresponsive NFS and usually doesn’t help in other cases.

Although umount should perform this automatically, running sync after the unmount can make sure that the file system was properly synced. Happy unmounting!

  • Share/Bookmark

Securely erase hard drive on Ubuntu (unrecoverable shred command)

Did you ever want to securely erase a hard disk on your server so the data is unrecoverable? Maybe you are selling the drive or the computer and want to be sure that no sensitive data can be recovered by a future owner. Ubuntu has a command line tool for just that purpose. If you want to completely erase the drive your system is on, first boot your computer using a Ubuntu LiveCD or the server install disk. Then, from the terminal (located in Applications->Accessories->Terminal if you use the Desktop LiveCD) enter the shred:

sudo shred -vfz -n [times] [your hard drive]

For example, you could type
sudo shred -vfz -n 5 /dev/sda

This would overwrite the data on the drive in /dev/sda with random data in 5 passes – the default is 3. Then it would overwrite the drive one more time with just zeros in a sixth pass. The parameters mean the following (see man shred for more):

v

Verbose output (see everything on screen)

f

Force permission changes (so you can overwrite everything

z

Overwrite with zeros (so it’s extra unrecoverable) in one more pass at the end

Overwriting your data with zeros through shred adds extra security, as does repeating the process a few times. 5 times may be a bit overkill though. Even overwriting it with random data once should be secure enough for most purposes. If you are paranoid about security or have truly sensitive data, you may try a higher number. Keep in mind that this increases the time required dramatically! Even on just a 500 GB hard drive (5400 rpm), a single pass with a final overwrite with zeros can take more a day.

Note: shred can also be used to overwrite individual files. However, this does not work well on modern operating systems for the reasons outlined in man shred.

  • Share/Bookmark

Linux guide to common server admin commands (mainly Ubuntu)

THIS POST IS STILL BEING COMPLETED IN SOME SECTIONS

I spend part of my time as a system admin for our servers. Since I am fairly new to Linux, I assembled a guide of the most common CLI commands as a reference for myself. I decided to share my command reference guide in case anyone else finds it useful. Since we are mostly focused on Ubuntu 8.04 LTS, this guide is aimed at that operating system, but these common commands should work with other Linux distributions as well. Since Ubuntu does not recommend using the system as root user, most commands require the sudo command to elevate to root privileges. User input is highlighted in blue. Optional input is surrounded by curly brackets like this {optional parameter}.

Users & Groups
Devices
File system
General
Network configuration
Server application configuration

Users & Groups

Edit adduser configuration
sudo nano /etc/adduser.conf
Change default directory mode:
DIR_MODE=0750
This make user directories private. The digits work as follows:
Mode digits: 2 = write only, 4 = read only, 5 = read and execute, 6 = read & write, 7 = read, write & execute
(execute means show listing for directories)
In order, the numbers apply to the owning user first, his group second and everyone else third
Add/create user
sudo adduser -m username
-m adds a user directory under home with all required files
Delete user (and his home directory)
sudo userdel -r username
Change own password
passwd
Change user’s password
sudo passwd username
Edit password security settings
sudo nano /etc/pam.d/common-password
See Ubuntu Guide on setting stricter password control
Add User group
sudo addgroup groupname
Add user to group
sudo adduser username groupname
Remove user from group
sudo deluser username groupname
Show all groups and users in them
cat /etc/group | less
Show list of all users with names
cat /etc/passwd | awk -F: '{print $1,",",$5}' | more
Show groups you are in
groups
Edit sudoers file / add users to sudo list
sudo visudo
under root, add:
username ALL=(ALL) ALL

Devices

Get processor info
cat /proc/cpuinfo
Get detailed system information
sudo lshw
Device messages (peripherals)
dmesg
Combine with | grep message to search for particular messages
Find new external disk after connection (shows recent messages)
dmesg | tail
Display volume information
sudo vgdisplay
List of mounted partitions
mount
Mount a disk
mount /mnt/mountdirectory/
Unmount a disk
umount /mnt/mountdirectory/
sync
See current mount points and used/free space
df -H

See physical volumes
pvdisplay

Show RAID partitioning
cat /proc/mdstat

List of disks and their partitions
sudo fdisk -l

File system

Directory listing with useful info
ls -la
Path to current directory
pwd
Switch to previous directory used
cd -
Create symbolic link (file reference)
ln -s targetpath linkname
Show text file contents
less filename
Number of files in a directory and its subdirectories
ls -1R | wc -l
Edit file in text editor nano (simple)
nano filename
Edit file in text editor vim (powerful)
vim filename
Remove empty directory
rmdir directoryname
Remove directory and all contained files and directories
rm -r directoryname
Find files by name
sudo find / -name *filename* -print
Find files by file owner
sudo find / -user username or userid -print
Securely erase files, partitions or hard drives
sudo shred -vfz -n [times] [your hard drive]
— Read on for more information on shred on Ubuntu

General

Display date, time and time zone
date
Change time zone
sudo dpkg-reconfigure tzdata
Logout
logout
Change file owner
chown owner-username{:groupname} filename
Change permissions
chmod mode-such-as-777 filename
Update time via network time server (one time)
sudo ntpdate timeserver {additional-timeserver...}
Setup daily cron job to update time
sudo nano /etc/cron.daily/ntpdate
enter ntpdate timeserver {additional-timeserver...}
(Don’t forget to save the file)
sudo chmod 755 /etc/cron.daily/ntpdate
(Makes the file executable)
Get APT repository list
sudo nano /etc/apt/sources.list
(For example, uncomment Universe for wider selection)
Clear screen
clear
Show all command aliases
alias
Change command alias
alias name='some command w/ options'
(for example alias ls = 'ls -lah'
(put in .bashrc to save permanently)
Find where is the command located
which command
Find out if a process is running
ps -fa | grep process name
(e.g. apache)
List active processes
sudo ps aux
Reboot
sudo reboot
Shutdown (immediately)
sudo poweroff
See also ‘Shutdown’ versus ‘poweroff’ versus ‘halt’ on Ubuntu server

Network configuration

Server application configuration

  • Share/Bookmark