Securely erase hard drive on Ubuntu (unrecoverable shred command)

Did you ever want to securely erase a hard disk on your server so the data is unrecoverable? Maybe you are selling the drive or the computer and want to be sure that no sensitive data can be recovered by a future owner. Ubuntu has a command line tool for just that purpose. If you want to completely erase the drive your system is on, first boot your computer using a Ubuntu LiveCD or the server install disk. Then, from the terminal (located in Applications->Accessories->Terminal if you use the Desktop LiveCD) enter the shred:

sudo shred -vfz -n [times] [your hard drive]

For example, you could type
sudo shred -vfz -n 5 /dev/sda

This would overwrite the data on the drive in /dev/sda with random data in 5 passes – the default is 3. Then it would overwrite the drive one more time with just zeros in a sixth pass. The parameters mean the following (see man shred for more):

v

Verbose output (see everything on screen)

f

Force permission changes (so you can overwrite everything

z

Overwrite with zeros (so it’s extra unrecoverable) in one more pass at the end

Overwriting your data with zeros through shred adds extra security, as does repeating the process a few times. 5 times may be a bit overkill though. Even overwriting it with random data once should be secure enough for most purposes. If you are paranoid about security or have truly sensitive data, you may try a higher number. Keep in mind that this increases the time required dramatically! Even on just a 500 GB hard drive (5400 rpm), a single pass with a final overwrite with zeros can take more a day.

Note: shred can also be used to overwrite individual files. However, this does not work well on modern operating systems for the reasons outlined in man shred.

  • Share/Bookmark

CSS Grid Systems (960 gs and blueprint)

CSS Grid Systems have become the latest Web design trend. While controversial among the HTML purists because grids violate the strict separation between content and presentation through the insertion of CSS classes such as class="grid_4", designers big and small are falling in love with them. Grids certainly streamline, simplify and sometimes even inspire the design process.

While there are many grid systems available today, two have attained particular popularity:

And not much time has passed until they were picked up as base themes for our favorite CMS Drupal:

While we prefer to build our own themes from scratch, these base themes are a good starting point for anyone just starting out with grids and Drupal theming. Note, however, that the Ninesixty theme does not appear to be using the latest version of the 960.gs system files.

We are particularly focused on 960 and a couple of very useful tools have been released. The 960 Gridder bookmarklet allows you to overlay a customizable grid over existing Web sites to either study them or assist you in your own development. If you are just starting to lay out your theme, the 960 Layout System allows you to interactively build a layout by nesting different sized container within either 12 or 16 column grid. The resulting layout can then be save as both HTML and CSS for further development in either Drupal or your system of choice.

  • Share/Bookmark

Google adds ‘Site Performance’ to Webmaster Tools – ready for Caffeine?

Google just added a new performance feature to the Labs section of its Google Webmaster tools. The ‘Site Performance’ feature displays a historical overview of your site performance (see picture below) as well as Page Speed suggestions for your slowest pages underneath.

Webmaster Tools - Site performance

 
The suggestions take the form of some stats on the page as well as suggestions to improve performance:

  • [Link to the page]
  • Details: Save up to [xxx] KB, [xxx] requests, [xxx] DNS lookups
  • Enable gzip compression
  • Combine external JavaScript
  • Minimize DNS lookups
  • Combine external CSS

Much has been blogged about Google’s new Caffeine engine and its effect on SEO and SERPS rankings. Google certainly seem to be pushing for faster page loading. I doubt Google will take this to the extreme though. If a feature doesn’t improve the quality of their search results, it won’t improve the user experience and it won’t make them more money. I wonder how much of it is posturing to get Webmasters to make faster sites. Sure, if your site is very slow, they may penalize you. But I doubt it will make a big difference on your SERPS ranking whether your site loads in 0.5 seconds or 1 second.

What it probably will affect is crawl rate. Google only has a certain amount of time available to crawl for each site. If your pages load slower, less of them will be re-indexed before Google moves on to the next stie. So your results may not be as fresh and that’s where I think the biggest penalty will kick in both currently and maybe more so under Caffeine.

  • Share/Bookmark

Has your mail server has been blacklisted for spam?

We used a shared host for one of our projects and recently noticed a lot of emails not making it to their intended destinations. It turns out that someone else on our shared host was using it to send spam and, as a result, everyone’s email from that IP address was marked as spam as well.

How can you find out if your mail server has been blacklisted for spamming? Use these tools below:

Spamhaus SBL Blacklist
MX Toolbox Spam Blacklist lookups

Also, if you have your own dedicated server (or rather dedicated IP address, which is a good thing), make sure you get a reverse DNS entry for your IP address from your provider – this helps a lot in avoiding overly aggressive spam filtering since it creates a bit more trust in your IP address and mail server.

Not sure if you have a reverse DNS entry? Use a lookup tool such as this:

Reverse DNS lookup

Happy emailing! Just make sure you follow local and international laws on spamming.

  • Share/Bookmark

Preventing Web crawlers from accessing your development site with robots.txt

We recently started a development site for a new project.  Soon, we noticed an Amazon AWS instance accessing URLs deep within our site regularly.  The site was access protected (you need to be logged in to see the pages), so those access attempts failed but were annoying nonetheless.

A bit more digging revealed that this was the Alexa bot trying to crawl our site.  I am not sure how they found it so quickly (we have no incoming links and this was a dev.****.*** subdomain) — they probably analyze DNS entries to find sites more efficiently. I also am not sure how they found the deep URLs (they are not exposed to the public part of the dev site), but Alexa was here nonetheless.

This reminded us that it’s a good idea to prevent access to your development sites using a robots.txt file at the root of your domain with the following content:

User-agent: *
Disallow: /

Of course, this will only keep out legitimate web crawlers spidering your site that actually respect the robots.txt file – but at least you have a few less people to worry about.

  • Share/Bookmark

Linux guide to common server admin commands (mainly Ubuntu)

THIS POST IS STILL BEING COMPLETED IN SOME SECTIONS

I spend part of my time as a system admin for our servers. Since I am fairly new to Linux, I assembled a guide of the most common CLI commands as a reference for myself. I decided to share my command reference guide in case anyone else finds it useful. Since we are mostly focused on Ubuntu 8.04 LTS, this guide is aimed at that operating system, but these common commands should work with other Linux distributions as well. Since Ubuntu does not recommend using the system as root user, most commands require the sudo command to elevate to root privileges. User input is highlighted in blue. Optional input is surrounded by curly brackets like this {optional parameter}.

Users & Groups
Devices
File system
General
Network configuration
Server application configuration

Users & Groups

Edit adduser configuration
sudo nano /etc/adduser.conf
Change default directory mode:
DIR_MODE=0750
This make user directories private. The digits work as follows:
Mode digits: 2 = write only, 4 = read only, 5 = read and execute, 6 = read & write, 7 = read, write & execute
(execute means show listing for directories)
In order, the numbers apply to the owning user first, his group second and everyone else third
Add/create user
sudo adduser -m username
-m adds a user directory under home with all required files
Delete user (and his home directory)
sudo userdel -r username
Change own password
passwd
Change user’s password
sudo passwd username
Edit password security settings
sudo nano /etc/pam.d/common-password
See Ubuntu Guide on setting stricter password control
Add User group
sudo addgroup groupname
Add user to group
sudo adduser username groupname
Remove user from group
sudo deluser username groupname
Show all groups and users in them
cat /etc/group | less
Show list of all users with names
cat /etc/passwd | awk -F: '{print $1,",",$5}' | more
Show groups you are in
groups
Edit sudoers file / add users to sudo list
sudo visudo
under root, add:
username ALL=(ALL) ALL

Devices

Get processor info
cat /proc/cpuinfo
Get detailed system information
sudo lshw
Device messages (peripherals)
dmesg
Combine with | grep message to search for particular messages
Find new external disk after connection (shows recent messages)
dmesg | tail
Display volume information
sudo vgdisplay
List of mounted partitions
mount
Mount a disk
mount /mnt/mountdirectory/
Unmount a disk
umount /mnt/mountdirectory/
sync
See current mount points and used/free space
df -H

See physical volumes
pvdisplay

Show RAID partitioning
cat /proc/mdstat

List of disks and their partitions
sudo fdisk -l

File system

Directory listing with useful info
ls -la
Path to current directory
pwd
Switch to previous directory used
cd -
Create symbolic link (file reference)
ln -s targetpath linkname
Show text file contents
less filename
Number of files in a directory and its subdirectories
ls -1R | wc -l
Edit file in text editor nano (simple)
nano filename
Edit file in text editor vim (powerful)
vim filename
Remove empty directory
rmdir directoryname
Remove directory and all contained files and directories
rm -r directoryname
Find files by name
sudo find / -name *filename* -print
Find files by file owner
sudo find / -user username or userid -print
Securely erase files, partitions or hard drives
sudo shred -vfz -n [times] [your hard drive]
— Read on for more information on shred on Ubuntu

General

Display date, time and time zone
date
Change time zone
sudo dpkg-reconfigure tzdata
Logout
logout
Change file owner
chown owner-username{:groupname} filename
Change permissions
chmod mode-such-as-777 filename
Update time via network time server (one time)
sudo ntpdate timeserver {additional-timeserver...}
Setup daily cron job to update time
sudo nano /etc/cron.daily/ntpdate
enter ntpdate timeserver {additional-timeserver...}
(Don’t forget to save the file)
sudo chmod 755 /etc/cron.daily/ntpdate
(Makes the file executable)
Get APT repository list
sudo nano /etc/apt/sources.list
(For example, uncomment Universe for wider selection)
Clear screen
clear
Show all command aliases
alias
Change command alias
alias name='some command w/ options'
(for example alias ls = 'ls -lah'
(put in .bashrc to save permanently)
Find where is the command located
which command
Find out if a process is running
ps -fa | grep process name
(e.g. apache)
List active processes
sudo ps aux
Reboot
sudo reboot
Shutdown (immediately)
sudo poweroff
See also ‘Shutdown’ versus ‘poweroff’ versus ‘halt’ on Ubuntu server

Network configuration

Server application configuration

  • Share/Bookmark

Useful design tools

Here is a list of some useful design tools:

Typography

In-browser debugging

Besides the obvious reference to Firebug for Firefox, these can be useful:

Color

  • Share/Bookmark

Useful tool: DropBox

A useful tool for programmers, designers and power users alike that I came across the other day (credit goes to Mark Boulton in his booklet Five Simple Steps – A Practical Guide to Designing for the Web) is DropBox.
In their own words, “Put your files into your Dropbox on one computer, and they’ll be instantly available on any of your other computers that you’ve installed Dropbox on (Windows, Mac, and Linux too!) Because a copy of your files are stored on Dropbox’s secure servers, you can also access them from any computer or mobile device using the Dropbox website.”

It’s a simple yet immensely useful tool. Forget about emailing or IMing files around. No more FTP. Setup the dropbox on all the computers you want to share the files and they instantly sync between them, with basic version control (it’s not SVN but should be good for most casual use) and backup (via Amazon S3). It works native in your file browser, just as if it were a local folder. Changes sync in the background automatically — it works pretty quickly. The easiest way to share files that I have found to date.

  • Share/Bookmark