Find the largest image file within a directory structure (e.g. largest jpeg on disc)

For our latest project, Urbo.ro, I needed to find the largest image file (.jpg) in a big directory structure with 50,000+ images but also other files in it. Manually searching it would have been impossible. I turned to our Linux ninja, Alexandru Ionica, who came up with the following that I wanted to share:

find /directory/ -iname "*.extension" -printf "%s " -print | sort -n -r | head -n1

This Linux snippet give you the largest file that ends with .extension inside of whichever directory structure you specify. To search the whole file system, just specify /.

  • Share/Bookmark

How to unmount a disk drive in Ubuntu (device is busy)?

Unmounting disks in Ubuntu is a bit more challenging than it should be. Say you have an external drive mounted on /media/disk. Trying the standard

sudo umount /media/disk

will often result in an error such as

umount: /media/drive: device is busy.

It is advisable to check what files may be open on the drive with

lsof | grep /media/disk

If something important is still open on the disk, make sure to close the files or programs using the disk. If nothing else should be keeping the disk busy, you can usually achieve an unmount with

sudo umount -vl /mnt/external_disk

This performs a verbose, lazy unmount – i.e. Ubuntu unmounts the disk drive when the device is no longer busy, showing any errors in the unmount if there are any.

To confirm that the device has been properly unmounted, use

mount

This will list all the currently mounted devices – your drive should no longer be in the list. If your list is too long or you don’t feel like scanning it mount | grep /media/disk will return nothing if the device has been successfully unmounted.

In case of an unresponsive NFS mount, you can also try

sudo umount -vf /mnt/nameofnfsmount

The -f argument forces an unmount for unresponsive NFS and usually doesn’t help in other cases.

Although umount should perform this automatically, running sync after the unmount can make sure that the file system was properly synced. Happy unmounting!

  • Share/Bookmark

‘Shutdown’ versus ‘poweroff’ versus ‘halt’ on Ubuntu server

There seems to be a lot of confusion around how to properly restart or shut down your server under Ubuntu. The are several commands out there: halt, shutdown, poweroff and reboot. Which are the right ones to use and what are the differences?

Historically, halt, poweroff and reboot were fairly low-level commands that would do exactly what their command name implies. However, they would not gracefully take down your server – it was presumed that the server administrator would do that him- or herself before invoking these commands. Many newer Linux users were not familiar with the right usage scenarios and, as a result, the commands were changed to invoke shutdown, a gentler command that gracefully terminates any open processes by giving them an opportunity to complete tasks before exiting.

Since these commands now invoke shutdown, the following are equivalent:

Restarting your server
reboot is the same as
shutdown -r 0

Turning off your server
halt is the same as
poweroff is the same as
shutdown -P 0

Note that these commands need to be invoked as root or via the sudo command, e.g.
sudo halt.

The original usage of these commands is also available via the -f switch. So to invoke halt directly without going through shutdown, use sudo halt -f. This is not recommended unless you are sure all other processes have finished their work. Similarly, sudo shutdown 0 by itself is not very useful – in Ubuntu it drops the server into the Recovery Menu since all processes are terminated but the server is not asked to power down or restart. It can certainly be manually powered down or restarted from there, though. Also, instead of the 0 after shutdown you can use now or specify some other time (either in minutes from now or absolute time) for the server to shut down.

  • Share/Bookmark

Linux guide to common server admin commands (mainly Ubuntu)

THIS POST IS STILL BEING COMPLETED IN SOME SECTIONS

I spend part of my time as a system admin for our servers. Since I am fairly new to Linux, I assembled a guide of the most common CLI commands as a reference for myself. I decided to share my command reference guide in case anyone else finds it useful. Since we are mostly focused on Ubuntu 8.04 LTS, this guide is aimed at that operating system, but these common commands should work with other Linux distributions as well. Since Ubuntu does not recommend using the system as root user, most commands require the sudo command to elevate to root privileges. User input is highlighted in blue. Optional input is surrounded by curly brackets like this {optional parameter}.

Users & Groups
Devices
File system
General
Network configuration
Server application configuration

Users & Groups

Edit adduser configuration
sudo nano /etc/adduser.conf
Change default directory mode:
DIR_MODE=0750
This make user directories private. The digits work as follows:
Mode digits: 2 = write only, 4 = read only, 5 = read and execute, 6 = read & write, 7 = read, write & execute
(execute means show listing for directories)
In order, the numbers apply to the owning user first, his group second and everyone else third
Add/create user
sudo adduser -m username
-m adds a user directory under home with all required files
Delete user (and his home directory)
sudo userdel -r username
Change own password
passwd
Change user’s password
sudo passwd username
Edit password security settings
sudo nano /etc/pam.d/common-password
See Ubuntu Guide on setting stricter password control
Add User group
sudo addgroup groupname
Add user to group
sudo adduser username groupname
Remove user from group
sudo deluser username groupname
Show all groups and users in them
cat /etc/group | less
Show list of all users with names
cat /etc/passwd | awk -F: '{print $1,",",$5}' | more
Show groups you are in
groups
Edit sudoers file / add users to sudo list
sudo visudo
under root, add:
username ALL=(ALL) ALL

Devices

Get processor info
cat /proc/cpuinfo
Get detailed system information
sudo lshw
Device messages (peripherals)
dmesg
Combine with | grep message to search for particular messages
Find new external disk after connection (shows recent messages)
dmesg | tail
Display volume information
sudo vgdisplay
List of mounted partitions
mount
Mount a disk
mount /mnt/mountdirectory/
Unmount a disk
umount /mnt/mountdirectory/
sync
See current mount points and used/free space
df -H

See physical volumes
pvdisplay

Show RAID partitioning
cat /proc/mdstat

List of disks and their partitions
sudo fdisk -l

File system

Directory listing with useful info
ls -la
Path to current directory
pwd
Switch to previous directory used
cd -
Create symbolic link (file reference)
ln -s targetpath linkname
Show text file contents
less filename
Number of files in a directory and its subdirectories
ls -1R | wc -l
Edit file in text editor nano (simple)
nano filename
Edit file in text editor vim (powerful)
vim filename
Remove empty directory
rmdir directoryname
Remove directory and all contained files and directories
rm -r directoryname
Find files by name
sudo find / -name *filename* -print
Find files by file owner
sudo find / -user username or userid -print
Securely erase files, partitions or hard drives
sudo shred -vfz -n [times] [your hard drive]
— Read on for more information on shred on Ubuntu

General

Display date, time and time zone
date
Change time zone
sudo dpkg-reconfigure tzdata
Logout
logout
Change file owner
chown owner-username{:groupname} filename
Change permissions
chmod mode-such-as-777 filename
Update time via network time server (one time)
sudo ntpdate timeserver {additional-timeserver...}
Setup daily cron job to update time
sudo nano /etc/cron.daily/ntpdate
enter ntpdate timeserver {additional-timeserver...}
(Don’t forget to save the file)
sudo chmod 755 /etc/cron.daily/ntpdate
(Makes the file executable)
Get APT repository list
sudo nano /etc/apt/sources.list
(For example, uncomment Universe for wider selection)
Clear screen
clear
Show all command aliases
alias
Change command alias
alias name='some command w/ options'
(for example alias ls = 'ls -lah'
(put in .bashrc to save permanently)
Find where is the command located
which command
Find out if a process is running
ps -fa | grep process name
(e.g. apache)
List active processes
sudo ps aux
Reboot
sudo reboot
Shutdown (immediately)
sudo poweroff
See also ‘Shutdown’ versus ‘poweroff’ versus ‘halt’ on Ubuntu server

Network configuration

Server application configuration

  • Share/Bookmark