Matt Thiessen » Server Admin

Using grep to scrape web pages

matt — Fri, 18 Sep 2009 15:38:20 +0000

In preperation to scrape a number of web pages, I used grep to make a list of URLs I need to scrape. The list of URLs was in an RSS file.

grep -P “\<\![CDATA\[(.*?)]” hawkeye_stories.xml > hawkeye_stories_links.txt

Bash script to backup a WordPress web site

matt — Wed, 26 Aug 2009 15:31:27 +0000

The script (backup_site.sh) for backing up a site has been modified. It now accepts parameters.

Parameter 1: domain
Parameter 2: database [optional]
Parameter 3: subdomain [optional]

Run as root

/home/root/sites/backup_site.sh gazlab.com matt_wp matt

#!/bin/bash

SITE=$1
SUBDOMAIN=$3
DBNAME=$2

if [ -z "$SUBDOMAIN" ]
then
if [ -z "$DBNAME" ]
then
mysqldump $DBNAME > /var/www/vhosts/$SITE/httpdocs/$DBNAME.sql
fi
tar -czvf $SITE.httpdocs.tar.gz /var/www/vhosts/$SITE/httpdocs
else
if [ -z "$DBNAME" ]
then
mysqldump $DBNAME > /var/www/vhosts/$SITE/subdomains/$SUBDOMAIN/httpdocs/$DBNAME.sql
fi
tar -czvf $SUBDOMAIN.$SITE.httpdocs.tar.gz /var/www/vhosts/$SITE/subdomains/$SUBDOMAIN/httpdocs
fi

exit 0

Don’t upgrade yet!

matt — Thu, 11 Jun 2009 13:23:14 +0000

I know how tempting it is to press that upgrade button, but don’t do it without backing up those files. Backing up your files first is just too quick and easy to skip when doing it from the command line. Here’s how:

Backup your MySQL database
DATE=`date | tr ” ” _`
mysqldump -u root -p DatabaseName > DatabaseName.$DATE.sql

Backup your web directory
tar cvzf new_filename.tar.gz /var/www/thiessen.us/httpdocs

c = create
v = verbose
z = compress
f = filename is being provided

If you need to restore these files

Restore MySQL database
mysql -u root -p DatabaseName < DatabaseName.Thu_Jun_11_08:14:41_CDT_2009.sql

Restore the web directory

cd /var/www/thiessen.us (change directory to the location of the web directory)

mv httpdocs httpdocs_bad (rename the current web directory)

tar -xzvf httpdocs.tar.gz (uncompress the backup)

Linux and Apache commands

matt — Mon, 16 Feb 2009 14:31:45 +0000

Here are some commands that we use at the Gazette on our Linux web server. It is running Red Hat Enterprise Linux 5.2.

Reload Apache Conf file
/etc/init.d/httpd reload

Restart Apache
apachectl -k graceful (didn’t work on Rackspace)

Restart Apache from ssh
/etc/rc.d/init.d/httpd restart

Modify Apache conf file
vi /etc/httpd/conf/httpd.conf

vi commands
i enter edit mode
esc to exit edit mode (command mode)
:wq to write changes and quit (in command mode)
:!q quit without saving

Memory Use

free -t -m