Code Snippets

Here's a random collection of code fragments that may be useful or interesting or spark some thought or start a revolution...

Spinner Progress Indicator

  • It took me a little while to figure out the magic combination of the cursor array and ternary operator. My main goal was to make it as self contained and efficient as possible. I am importing millions of records, and didn't want to waste CPU time on computing modulus on some huge $i value. :)
  • However I wanted some way to know that each of the millions of records was being imported. A "progress indicator" if you will. One that quickly illustrates that "yes, something is still churning away, even though it might be taking minutes, hours or *gasp* days to process".
  • I didn't want to print out a single "." for each one, as that still just takes up pages and pages of scrolling as they whiz by.
  • I didn't want to use the new record ID, as that takes up a wasted SQL call to get LAST_INSERT_ID().
  • So a spinner was a likely candidate. It sits in place and rotates 1/8 turn for each record inserted. As errors occur, I can spit them out where I'm sure to see them (or log them to an error log) and they won't get lost in a maxed out scroll-back buffer.
  • Plus it looks freakin' slick as hell and gives the appearance that I am smarter than I am. *grin*

#!/usr/bin/php -q
printf"%c[2J"ESC ); //clear screen

//note how the "-" is at [1], not [0]
$cursorArray = array("/","-","\\","|","/","-","\\","|"); 

"Processing: ";

printf"%c7"ESC ); //save cursor position

for ($count 1$count <= 30$count++) 
//note that $i is all self contained.
    //restore cursor position and print
printf("%c8(".$cursorArray[ (($i++ > 7) ? ($i 1) : ($i 8)) ].") %02d"ESC$count);



I'm just learning (i.e. struggling) with Ruby. I *really* want to learn it, but it's just so foreign.

# this is a little trick that Rails uses to set this file's path into the search path
# in this case it pushes the current file's directory onto the $LOAD_PATH stack ($:)

$:.unshift File.expand_path(File.dirname(__FILE__) + "/..")

Gentoo, Portage, Emerges

I've been a fan of Gentoo Linux for about 4 years or so now (although I am seriously considering switching to Ubuntu).

# This is really handy for any kind of massive emerge where some files might fail.
# It will just keep churning through the list of files until the last one,
# no more starting a big emerge at night only to find out portage puked on the second
# one after you went to bed! 
# You could use 'world' or 'gnome' or 'kde' or whatever instead of 'system' of course.

emerge -avu system || until emerge --resume --skipfirst; do :; done

# I run a clean ship (and VMWare images tend to bloat up), 
# so I like to get rid of things I don't need nohow...

find /var/log/* -mtime +30 -exec rm -rf {} \;
find /var/log/portage-logs/* -type f -empty -exec rm -rf {} \;
rm -rf /var/tmp/portage/*
rm -rf /usr/portage/distfiles/*

for f in $(portageq envvar PORT_LOGDIR)/*.log;
   gzip -f $f 2>/dev/null || echo "Failed to gzip $f"

Production Deployment Script

This is a script useful for deploying a CVS/SVN checkout on a production server (or others). It will do a checkout (or export) into a new timestamped directory, change a symlink, modify a config file, and archive old dirs. Use it as a template for your own needs.


# Written by Daevid Vincent 
# version 2010-05-11 6:30pm
# Change your CVS_USER and any paths

if [[ $EUID -ne 0 ]]; then
   echo -e "\e[00;31mSorry, `basename $0` must be run as sudo.\e[00m"
   exit 1

if [ $# -eq 0 ]
   echo "You must provide the server type (vm|dev|test|prod) and optionally 'export'" 
   echo "     `basename $0` prod" 
   echo "  or"
   echo "     `basename $0` prod export" 
   exit 2

NOW=`date +%Y-%m-%d-%H-%M-%S`

SERVER=$1  # should be 'vm|dev|test|prod'
RELEASE=$2 # can be 'export' or nothing

if [ "$SERVER" = "vm" ]; then

trap cleanup 1 2 3 6
  echo -e "\e[00;31mCaught Signal ... cleaning up.\e[00m"
  rm -vrf $WEB_PATH
  rm -vrf $CVS_DIFF_FILE
  echo -e "\e[00;31mDone cleanup ... quitting.\e[00m"
  exit 1


echo "Creating directory ${WEB_PATH}"
rm -rf $WEB_PATH && mkdir -p $WEB_PATH

export CVS_RSH=ssh
export CVSROOT=:ext:$

if [ "$RELEASE" = "export" ]; then
	echo "Creating an export version on ${SERVER}."
	# we do this to avoid all the CVS stuff, this is also significantly faster to download
	/usr/bin/cvs -z3 -d$CVSROOT export -d $NEW_DIR -r HEAD dart2
	chown -R www-data:www-data $WEB_PATH
	echo "Creating an updateable version on ${SERVER}."
	# in this case we DO want a local CVS repository that we can later do "cvs -q update -P -d" upon.
	/usr/bin/cvs -z3 -d$CVSROOT checkout -d $NEW_DIR dart2
	if [ "$SERVER" = "vm" ]; then
		echo "Setting the ${WEB_PATH} owner to developer:developer since this is a VM"
		chown -R developer:developer $WEB_PATH
		echo "Setting the ${WEB_PATH} owner to ${CVS_USER}"

cd $WEB_PATH/includes
echo "Setting up symbolic link."
#SERVER="`echo $SERVER|tr '[a-z]' '[A-Z]'`"
#ln -vs $WEB_PATH/includes/config-$ $WEB_PATH/includes/
case "$SERVER" in
		ln -s;
		ln -s;
		ln -s;
		ln -s;
		echo "Unknown Server ${SERVER}, so no symbolic link created."
		exit 3


# TODO: add a list of all the files that have changed between tag numbers, not dates per se
echo "Gathering a CVS DIFF log into ${CVS_DIFF_FILE}"
CVS_THEN=`date --date='5 days ago' +%Y-%m-%d`
CVS_NOW=`date +%Y-%m-%d`
/usr/bin/cvs -q diff -N -c -D $CVS_THEN -D $CVS_NOW > $CVS_DIFF_FILE

if [ -n "$EMAIL" ]; then
	echo -e "\e[00;34mEmailing ${SERVER} change log to ${EMAIL}\e[00m"
	CVS_FILE_CHANGES=`/bin/grep "Index:" $CVS_DIFF_FILE | sed 's/Index:/File:/g'`
	# TODO: see if there's a way to get the CVS tag number in here
	BODY="A new version of DART2 has been pushed to this server @ ${NOW}.\n\nChanged files between ${CVS_THEN} and ${CVS_NOW} are:\n\n${CVS_FILE_CHANGES}\n\nDone."
	# these " marks are significant. they tell bash to respect the \n chars in output
	(echo -e "$BODY"; uuencode $CVS_DIFF_FILE $CVS_DIFF_FILE) | mail -s "DART2: new version on ${SERVER}" $EMAIL

echo "Fixing permissions on all files and directories"
find $WEB_PATH -type f -name \*.php -exec dos2unix {} \;
find $WEB_PATH -type d -exec chmod $DIR_PERMS {} \;
find $WEB_PATH -type f -exec chmod $FILE_PERMS {} \;
find $WEB_PATH/UPDATES -type f -regex ".*\.\(sh\|php\)" -exec chmod $DIR_PERMS {} \;

# Do this last, as it is the final switchover
echo "Redirecting ${WEB_TRUNK}/dart2 symbolic link to $WEB_PATH."
rm -f $WEB_TRUNK/dart2 && ln -s $WEB_PATH $WEB_TRUNK/dart2
chown -h www-data:www-data $WEB_TRUNK/dart2
echo -e "\e[00;32mNew DART2 site deployed and live. Please verify it works.\e[00m"

echo "Archiving any 'dart2' directories over 24hrs old..."
#find em
FILELIST=`find ./ -maxdepth 1 -mtime +1 -type d -name dart2-\* | sort`
# tar.bzip2 em then delete em
for f in $FILELIST; do
	echo -e "\t\tTarballing ${f} ..."
	tar jcpf $f.tar.bz2 $f
	if [ $? -eq 0 ]; then
		echo -e "\t\tDeleting ${f} ..."
		rm -rf $f

Subversion pre-commit script

I searched high and low to find a script that would do this seemingly simple task. Why did the Subversion people have to make things so complicated?! GRRR. Put this in your subversion repository as 'hooks/pre-commit' and chmod +x it.


# written by Daevid Vincent ( on 08.12.08
# based upon code found here: 

# I would have liked to write this in PHP, but for now bash will have to do
# reference --

# This isn't the most elegant way to do this. 
# TODO: it would be nice to store all the PHP errors in an array and spit them all out at once.

#echo "REPOS = $REPOS" 1>&2
#echo "TXN = $TXN" 1>&2


BANNEDFILES=( '.DS_Store' 'Thumbs.db' ) 

# redirect all output to stderr rather than line by line
exec 1>&2

# Make sure that the log message contains some text.
LOGMSG=`$SVNLOOK log -t "$TXN" "$REPOS" | grep "[a-zA-Z0-9]"`
#echo "LOGMSG = '$LOGMSG'"  
if [ -z "$LOGMSG" ] ; then
	echo " "  
    echo "A commit log message is required."  
	exit 1

COMMIT_FILES=`$SVNLOOK changed -t "$TXN" "$REPOS" | awk '{print $2}'`
	BASENAME=`basename $MYFILE`

	if [[ "${FILEEXT}" =~ \.r([0-9]+) ]] ; then 
		echo " "  
		echo "Error: ${MYFILE}"
	    echo "Files of type .r${BASH_REMATCH[1]} indicate a conflict remains and are not allowed to be committed."  
		exit 1
	if [[ 'mine' == ${FILEEXT} ]] ; then 
		echo " "  
		echo "Error: ${MYFILE}"
	    echo "Files of type .mine indicate a conflict remains and are not allowed to be committed."  
		exit 1

	if [[ 'log' == ${FILEEXT} || 'log' == ${BASENAME:(-3)} ]] ; then 
		echo " "  
		echo "Error: ${MYFILE}"
		echo "Log files are not allowed to be committed."  
		exit 1

	for i in ${BANNEDFILES[@]}; do 
		if [[ $i == ${BASENAME} ]] ; then 
			echo " "  
			echo "Error: ${MYFILE}"
		    echo "Files named ${BASENAME} are not allowed to be committed."  
			exit 1

	#if [FILEEXT == "php" ]; then
	PHPFILE=`echo $MYFILE | egrep \\.php$`
	if [ $? == 0 ]; then
		MESSAGE=`$SVNLOOK cat -t "$TXN" "$REPOS" "${PHPFILE}" | $PHP -l`
		if [ $? -ne 0 ] ; then
			echo " "  
			echo "Error: ${MYFILE}"
			echo "---------------------------------------------------------------------------------"  
			echo "During automatic PHP syntax checking we found the following PHP errors: "  
			echo " "  
			echo "$MESSAGE" | sed "s| -| $PHPFILE|g"  
			echo " "  
			echo "Please correct the error and try the commit again."  
			echo " "  
			echo "You can check for syntax error on your computer by running the command:"  
			echo "${PHP} -l ${PHPFILE}"  
			echo "---------------------------------------------------------------------------------"  
			exit 1

# All checks passed, so allow the commit.
exit 0

Subversion Flagged Update

I needed a way to make it painfully obvious that someone must run some manual SQL updates for a particular "svn update" (or not), or their build would be broken as the schema wouldn't match the code.

So this will look for the "update.sql" file and if it sees one, it prints out in big red letters a message.

It also solves two other problems. The first being that you have to actually be the proper subversion user to do the checkout/update and the second is that you can run it from any directory you like as it remembers your location and keeps you there.


# Original idea and modifications: Daevid Vincent []
# The meat and potatoes: Jeremiah Wuenschel []
# version 2008-08-11 5:50pm
# save it as 'svnu' and put it in /usr/local/bin (chmod 755 of course)

if [ $USER != "dev" ]
    echo "You must run this as user 'dev'."
    exit 1


cd /home/prod

svn update | awk '
    BEGIN {need_update=0;}
    END {if(need_update) {
            printf "'$(echo -e "\033[1;31m")'"
            print ""
            print "########################################################"
            print "   DON\047T FORGET TO RUN THE site/data/sql/update.sql  "
            print "                   sudo mysql prod                      "
            print "########################################################"
            printf "'$(echo -e "\033[0m")'"

cd $CURR

Automatic Monitoring of remote servers

I needed a way to setup some remote ssh terminals to monitor logs, but I also wanted security. This script will flip workspaces in Gnome to show logs on one vs. mysql mytop on another. This is my solution to Gnome bug #577394 I submitted.


# 2009-05-22 04:10 PM
# Daevid Vincent
# This will setup several remote ssh sessions to monitor logs and mysql's mytop
# Useful for monitoring logs and mysql on remote servers (dev/test/prod) in an automated display/kiosk way.
# It switches views much like a video camera system might.
# xtrlock is awesome and will lock the screen to prevent snoopy snoops
# wmctrl is also required to hack around the deficiencies of gnome-terminal and does the magic switching
# I'm not sure if wmctrl will work with compiz, so if workspaces aren't switching, this may be why.

# If you haven't done so already (and you'd be remiss not to), setup passwordless ssh logins
# Create a gnome-terminal 'taillog' profile. I use a 8pt monospaced font and white background with gray text.
# You will need a proper .mytop file on the remote mySQL servers
# You will need a couple programs installed:
#	sudo apt-get install xtrlock
#	sudo apt-get install wmctrl
# You'll also need to configure the width and height settings at the top of the script for your display.
#    * System > Preferences > Screen Resolution (for pixels)
#    * Open up a gnome terminal using the 'taillog' profile, and size it to the full size of the display and note the WWWxHH number floating in the middle.

#screen width/height in pixels

#screen width/height in characters (NOTE: this is different for the taillog profile as it is smaller font [8pt])

tiles=3  # this is the number of vertical tiles, so if you had 4 windows, it'd be "2", for 6 then set to "3"

echo "Locking the screen. Type password to unlock."
xtrlock &

# TODO: Because Gnome sucks ass so much, you can't open a terminal in a specific workspace:

# get two terminals full width and stacked on Workspace 1 for the logs:
wmctrl -s 0
gnome-terminal --geometry=${chrwidth}x$(expr $chrheight / 2 - 2)+0+0 --window-with-profile=taillog --command 'ssh pse05' --title='Tail Log pse05 (Production)'
gnome-terminal --geometry=${chrwidth}x$(expr $chrheight / 2 - 2)+0+$(expr $pxheight / 2) --window-with-profile=taillog --command 'ssh pse03' --title='Tail Log pse03 (Test)'
sleep $seconds

# get the six 'quadranted' mytops on Workspace 2:
geomwidth=$(expr $chrwidth / 2)
echo "each tile is ${geomwidth} characters wide"
geomheight=$(expr $chrheight / $tiles)
echo "each tile is ${geomheight} characters high"

pxwh=$(expr $pxwidth / 2)
echo "the pxwh offset is ${pxwh}"

yoff=$(expr $pxheight / $tiles)
echo "the y offset is ${yoff}"

wmctrl -s 1
gnome-terminal --geometry=${geomwidth}x${geomheight}+0+0 --window-with-profile=taillog --command 'ssh -t pse01 mytop' --title='pse01 (Development Master)'
gnome-terminal --geometry=${geomwidth}x${geomheight}+$pxwh+0 --window-with-profile=taillog --command 'ssh -t pse02 mytop' --title='pse02 (Development Slave)'

posy=$(expr $pxheight / $tiles)
echo "pse03 and pse04 posy ${posy}"
gnome-terminal --geometry=${geomwidth}x${geomheight}+0+$posy --window-with-profile=taillog --command 'ssh -t pse03 mytop' --title='pse03 (Test Master)'
gnome-terminal --geometry=${geomwidth}x${geomheight}+$pxwh+$posy --window-with-profile=taillog --command 'ssh -t pse04 mytop' --title='pse04 (Test Slave)'

posy=$(expr $posy \* 2)
echo "pse05 and pse06 posy ${posy}"
gnome-terminal --geometry=${geomwidth}x${geomheight}+0+$posy --window-with-profile=taillog --command 'ssh -t pse05 mytop' --title='pse05 (Production Master)'
gnome-terminal --geometry=${geomwidth}x${geomheight}+$pxwh+$posy --window-with-profile=taillog --command 'ssh -t pse06 mytop' --title='pse06 (Production Slave)'
sleep $seconds

while :
	for i in {1..2} #in case we load up more virtual workspaces
		#echo "Switching to Workspace ${i}"
		wmctrl -s $(expr $i - 1)
 		sleep $seconds

Server Birthday (uptime) Reminder

I was very proud of a 1 year uptime for a server at work so I wrote this little bash script that will figure out the exact 1 year birthday/anniversary of a server and email you a reminder via 'at' on that date and exact minute. Geeky I know, but a great lesson in bash scripting.


# Figure out the server's exact birthday and add it to 'at'
# to email you a reminder.
# 2009-06-19


birthday=$(( 60 * 60 * 24 * 365 ))

while read ups idle
	#echo "$uptime ($idle)"
done < /proc/uptime

bds=$(( $birthday - $uptime ))

birthdate=`date --date "now +${bds} seconds"`

echo "${server} birthday happens in ${bds} seconds on ${birthdate}"

min=$(( $bds / 60 ))

echo "Adding this reminder to 'at'..."
echo 'echo "`/usr/bin/uptime`" | mail -s "Happy 1 Year Uptime on ${server}!!" ${email}' \
      | at now + $minutes minutes 2>&1 | tail -n 1

Ping Test

My D-Link DIR-655 router has been flakey after a firmware upgrade, so I wrote a little script to log the major IP address connection statuses to see if there is a trend. Ideally, it would email/SMS me, however the irony is that the server has no connection and therefore can't. However, you could run this on another (stable) server and point it at the one in question. Apparently I'm not the only one experiencing this nightmare.

Add this statement to your /etc/crontab and put the code in /usr/local/bin/

#Ping various key IP addresses including GW and DNS servers because D-LINK DIR-655 sucks balls...
*/15 *  * * * root /usr/local/bin/ > /dev/null

# Simple SHELL script for Linux and UNIX system monitoring with
# ping command
# -------------------------------------------------------------------------
# Copyright (c) 2006 nixCraft project 
# This script is licensed under GNU GPL version 2.0 or above
# -------------------------------------------------------------------------
# This script is part of nixCraft shell script collection (NSSC)
# Visit for more information.
# -------------------------------------------------------------------------
# Setup email ID below
# See URL for more info:
# -------------------------------------------------------------------------
# add ip / hostname separated by white space, put gateway and DNS as good hosts
# if you use domain names, and you can't get to your DNS, then those will error out the "-eq 0" line below
COUNT=2 # number of ping requests to send/recieve
NOW=$(date +"%b %d %H:%M:%S") #syslog format date

for myHost in $HOSTS
  # count=$(ping -c $COUNT $myHost | grep 'received' | awk -F',' '{ print $2 }' | awk '{ print $1 }')
  count=$(ping -c $COUNT $myHost | grep 'received' | cut -f2 -d',' |cut -f2 -d' ')
  if [ $count -eq 0 ]; then
	body="[ping] $NOW host: $myHost unreachable."
#    echo $body | mail -s "Ping $myHost failed" -u dae51d
#    echo $body | mail -s "Ping $myHost failed" $EMAILID
	#echo 'Restarting networking' >> $PINGFILE
	#/etc/init.d/networking restart
	body="[ping] $NOW host: $myHost success."

  echo $body
  echo $body >> $PINGFILE

echo "------------------------------------------------------" >> $PINGFILE

Category as Unordered HTML List and Multi-dimensional Array

I searched everywhere for a basic routine to take a standard list of categories with a parent_id column and convert that to a multi-dimensional array and then to an unordered list.

Everyone had their own ways of doing it, but nothing really was just simple and seemed to work right.

So, if you have a nested set or any other category list in the form:

		| id      | parent_id | name                 | indent | enabled |
		|       1 |         0 | ALL                  |      0 |       1 | 
		|       3 |         1 | Tube                 |      1 |       1 | 
		|       4 |         1 | LCD                  |      1 |       1 | 
		|       5 |         1 | Plasma               |      1 |       1 | 
		|       6 |         1 | Portable Electronics |      1 |       1 | 
		|       7 |         6 | MP3 Players          |      2 |       1 | 
		|       8 |         7 | Flash                |      3 |       1 | 
		|       9 |         6 | CD Players           |      2 |       0 | 
		|      10 |         6 | 2 Way Radios         |      2 |       1 | 

Then I used this multidimensional array code.

And ultimately I drop it into this awesome drag & drop DHTML magic.

<?php $refs = array();
$categoryArray = array();
$result SQL_QUERY("SELECT as id, node.parent_id, AS name, 
        (COUNT( - 1) as indent, node.enabled
         FROM     categories AS node, categories AS parent
         WHERE     node.lft BETWEEN parent.lft AND parent.rgt
            and < 20
         GROUP BY
         ORDER BY node.lft"
$data SQL_ASSOC_ARRAY($result)) 
$thisref $refs$data['id'] ];
$thisref['parent_id'] = $data['parent_id'];
$thisref['name'] = $data['name'];
    if (
$data['parent_id'] == 0)
$categoryArray$data['id'] ] = $thisref;
$refs$data['parent_id'] ]['children'][ $data['id'] ] = $thisref;

//echo multiArray2CategoryTree($categoryArray);

function multiArray2CategoryTree$menu$indent 
    if (!
is_array($menu)) return false;
$html str_repeat("\t",$indent)."   <ul>\n";
    foreach (
$menu as $id => $value)
        if (
$html .= str_repeat("\t",$indent+1).'<li id="node'.$id.'" noDelete="true" noRename="true"><a href="#">'.$value['name']."</a>\n";
$html .= multiArray2CategoryTree($value['children'], $indent+1);
$thml .= str_repeat("\t",$indent+1)."</li>\n";
$html .= str_repeat("\t",$indent+1).'<li id="node'.$id.'" noDelete="true" noRename="true"><a href="#">'.$value['name']."</a></li>\n";
$html .= str_repeat("\t",$indent)."   </ul>\n";