Skip to content

Using MegaUpload as Cloud Backup

09-Mar-10

I’ve been using Dropbox for our offsite backup solution for a while and its worked well , the only issue was that for total offsite security it relied upon another user managing their limited Dropbox storage space.

So instead I am now uploading them to MegaUpload instead.
RapidShare
would work just as well and could also be used concurrently for double the security.

This method has the advantages of :-

  • Auto-Pruning – the files get deleted 90 days after the last download.
  • Unlimited storage.
  • Publicly accessible to anyone knowing the URL via a browser.

By being publicly accessible you need to make sure any data is encrypted , but we all do that anyway , right ?
You may want to use a password protected RAR file , I am using decent encryption algorithms via GPG.

This script require you have mail , GPG and Plowshare setup.

Here is my backup_databases_to_megaupload script :-

#!/bin/sh
# script to backup to the megaupload 'cloud'

# setup variables
DATABASEDUMPSDIR=/mnt/replica/mysql_database_dumps
ZIPFILENAME=/tmp/mysql_database_dumps_`date +%Y%m%d`
MUCREDS="XXXXXXXX:YYYYYYYY"
OFFSITEEMAILS="mark@miel.weloveit.info,mark@miel.collective-b.org.uk"

# zip mysql database dumps
zip $ZIPFILENAME $DATABASEDUMPSDIR/*

# encrypt with gpg
gpg -e -r mark@weloveit.info $ZIPFILENAME.zip

# upload to megaupload and send email
/usr/local/bin/plowup -a $MUCREDS $ZIPFILENAME.zip.gpg megaupload | mail -s "`date`" $OFFSITEEMAILS

# delete temp files
rm $ZIPFILENAME.zip
rm $ZIPFILENAME.zip.gpg

Make sure you change :-
XXXXXXXX = Your MegaUpload username.
YYYYYYYY = Your MegaUpload password.
OFFSITEEMAILS = email addresses (separated by commas) of your offsite contacts who will download and store the backup files.

Posting MythTV recordings to Wordpress

22-Feb-10

I have posted previously about getting our MythTV to post its recordings to a Laconica StatusNet server , it worked well but was limited to just 140 characters.

I have since ditched StatusNet and use Wordpress-MU for all our blogging sites.

Yesterday I discovered charm and its capabilities encouraged me to look into MythTV automated blogging again , this time without the 140 charatcer limitation the posts can be more informative.

I am using Ubuntu 9.10 (Karmic Koala) so can install charm easily with just :-

aptitude install charm

Next I created the simple

~/.charmrc-mythtv

configuration file containing just :-

metaweb = BLOGUSERNAME BLOGPASSWORD http://mythtv.blogs.weloveit.info/xmlrpc.php

I also had to enable the XML-RPC posting in Dashboard , Settings , Writing as it is disabled by default.

Next I wrote a little bash script called

~/bin/post_mythtv_recording_to_wordpress

to grab details from the MySQL database for the MythTV :-

#!/bin/sh
#
# script to post a recording to a wordpress blog
# using charm
# mark waters - 20100221

if [ -z "$1" ];
    then
    echo "this script needs a parameter , please use the mythtv recording filename , %FILE%"
    exit
fi

FILENAME=$1
CHANID="${FILENAME%%_*}"
DESCRIPTION=`mysql -u root mythconverg -sN -e"SELECT description FROM recorded where basename = '$FILENAME';" `
CHANNEL=`mysql -u root mythconverg -sN -e"SELECT name FROM channel where chanid = '$CHANID';" `
DATETIME=`mysql -u root mythconverg -sN -e"SELECT progstart FROM recorded where basename = '$FILENAME';" `
CATEGORY=`mysql -u root mythconverg -sN -e"SELECT category FROM recorded where basename = '$FILENAME';" `
TITLE=`mysql -u root mythconverg -sN -e"SELECT title FROM recorded where basename = '$FILENAME';" `
BLOGCATEGORY="Recorded"
DATE=`echo $DATETIME | awk {'print $1'}`

echo "$DESCRIPTION\nRecorded from $CHANNEL at $DATETIME" | charm -f /home/mark/.charmrc-mythtv -q -s "$TITLE - $DATE" -t "$BLOGCATEGORY" -k "$TITLE , $CATEGORY"

Then I just added the script to the MythTV backend as UserJob2 :-

/home/mark/bin/post_mythtv_recording_to_wordpress %FILE%

mythtv_userjob2_wordpress_posting_20100222

and enabled it for all recordings with a little MySQL update magic :-

update record SET `autouserjob2` = 1

You can see the result on our MythTV Automated Blog.

Managing sabnzbd+ from the command line

17-Feb-10

I’ve put together a little bash script to manage sabnzbd+ from the command line.

So far it can :-

  • Pause
  • Resume
  • Change Download Rate , no value defaults to rate set in config file

Any improvements or suggestions can be left in the comments or contact me directly.

#!/bin/sh
#
# script to manage sabnzbd+ from the command line

APIKEY=accf3f76184de6c221851684dfbe8a61
SERVER=http://core3.collective-b.dyndns.org:8080/sabnzbd
CONFIGFILE=/home/mark/.sabnzbd/sabnzbd.ini
DEFAULTRATE=`grep bandwidth_limit $CONFIGFILE | awk '{ print $3 }'`

if [ -z $1 ];
     then
     echo "Please enter some parameters , You can :-"
     echo "pause - Unending Pause"
     echo "pause xx - Pause for xx Minutes"
     echo "resume - Continue after Pause"
     echo "rate - Set the Download Rate , defaults to Rate set in configuration file"
     echo "rate xxx - Set Download Rate to xxx"
fi

case "$1" in
'pause')
if [ -z $2 ];
    then
    echo "Unending Pause"
    /usr/bin/wget -q --delete-after "$SERVER/api?mode=pause&apikey=$APIKEY"
    else
    echo "Timed Pause , Resuming after $2 minutes"
    /usr/bin/wget -q --delete-after "$SERVER/api?mode=config&name=set_pause&value=$2&apikey=$APIKEY"
fi
;;
'rate')
RATE=$2
if [ -z $RATE ];
    then
    RATE=$DEFAULTRATE
fi
echo "Setting Download rate to $RATE"
/usr/bin/wget -q --delete-after "$SERVER/api?mode=config&name=speedlimit&value=$RATE&apikey=$APIKEY"
;;
'resume')
/usr/bin/wget -q --delete-after "$SERVER/api?mode=resume&apikey=$APIKEY"
;;
'shutdown')
/usr/bin/wget -q --delete-after "$SERVER/api?mode=shutdown&apikey=$APIKEY"
;;
esac

Make sure you update the variables for your setup.

Fail2Ban rule for Asterisk

13-Feb-10

After last months scan / attack against our Asterisk server I thought I better get some extra security sorted.

I am using fail2ban for SSH and HTTP on another server successfully so set it up here too.

Here is the /etc/fail2ban/jail.local file :-

[DEFAULT]

# "ignoreip" can be an IP address, a CIDR mask or a DNS host
ignoreip        =       127.0.0.1 192.168.0.0/24
findtime        =       2678400
bantime         =       -1
maxretry        =       1
backend         =       polling

# Default action to take: ban only
action = iptables[name=%(__name__)s, port=%(port)s]

[asterisk]
enabled         =       true
port            =       5060
filter          =       asterisk
logpath         =       /var/log/asterisk/messages
maxretry        =       1

and the filter rule for the asterisk server , /etc/fail2ban/filter.d/asterisk.conf :-

[Definition]
failregex       =       NOTICE.*Sending fake auth rejection for user.*@
                        NOTICE.* .*: Registration from '.*' failed for '' - Wrong password
                        NOTICE.* .*: Registration from '.*' failed for '' - Username/auth name mismatch
                        NOTICE.*  failed to authenticate as '.*'$
                        NOTICE.* .*: No registration for peer '.*' \(from \)
                        NOTICE.* .*: Host  failed MD5 authentication for '.*' (.*)
                        NOTICE.* .*: Registration from '.*' failed for '' - No matching peer found

ignoreregex =

irssi + bitlbee + xmpp conference auto join

28-Jan-10

Since moving Instant Messaging platforms from Pidgin to irssi I have been having problems automatically logging into a XMPP chatroom automatically.

Got it sorted today.

in the ~/.irssi/config file , under the chatnet section for the bitlbee server I added :-

autosendcmd = "wait 2000;  /msg &bitlbee join_chat 0 ROOM@CONFERENCE.SERVER.TLD &CHATROOM NICK ROOMPASSWORD"

Windows script to keep Dropbox tidy

06-Jan-10

Thought I’d write a little script for Dad to keep his Dropbox folder tidy due to my uploading nightly backup archive files.

I don’t use Windows so its a tricky for me to test but it should work , anyone want to test and comment ?

FORFILES -pC:\Documents and Settings\mark\My Documents\My Dropbox\CloudBackup\ -d-4 -c"CMD /C MOVE @FILE C:\Documents and Settings\mark\Desktop\"

It uses the forfiles command to move all but the 4 newest files from the Dropbox directory structure to a folder on the Desktop , this way the files can be burnt to a DVD-R and don’t clutter up Dad’s Dropbox capacity.

Was hoping Dad would test the script but he has deleted it twice now so I give up , anyone else tried it ?

Apache Proxy for the Gallery

01-Jan-10

For a while I have been running our Gallery on a seperate webserver to the rest of our sites because of disk capacity constraints.
Today I moved the Gallery to my PC as its the fastest machine we have which will make the image resizing smoother.

On the front-facing webserver I have setup Apache to proxy to the backend server with this VirtualHost configuration :-

NameVirtualHost *:80
ServerAdmin mark@weloveit.info
ServerName gallery.weloveit.info
ProxyRequests Off
ProxyPreserveHost On
Order deny,allow
Allow from all
ProxyPass / http://gallery.core6.collective-b.dyndns.org/
ProxyPassReverse / http://gallery.core6.collective-b.dyndns.org/
ErrorLog /var/log/apache2/proxy.gallery.weloveit.info.error.log
CustomLog /var/log/apache2/proxy.gallery.weloveit.info.access.log combined env=!dontlog

With

ProxyPreserveHost Off

the backend server hostname was showing through.

On the backend server I am using this VirtualHost configuration :-

NameVirtualHost *:80
ServerAdmin mark@weloveit.info
ServerName gallery.weloveit.info
DocumentRoot /var/www/zenphoto-1.2.8_RC1/

Exporting just the Draft Wordpress posts

30-Dec-09

I wanted to export just the draft posts from one of our Wordpress-MU sites.

Fired up the text editor and opened the

wp-admin/includes/export.php

file.

Located :-

$post_ids = $wpdb->get_col("SELECT ID FROM $wpdb->posts $where ORDER BY post_date_gmt ASC");

added in this :-

AND post_type = 'post' AND post_status = 'draft'

to change it to this :-

$post_ids = $wpdb->get_col("SELECT ID FROM $wpdb->posts $where AND post_type = 'post' AND post_status = 'draft' ORDER BY post_date_gmt ASC");

Now an export only exports the the draft posts , easy.

Bulk Traffic Management Script

28-Dec-09

Now we are using the free.fr freebox as our router it lacks any ability to shape our traffic.

This script runs every hour via cron , the Asterisk server also calls the script at the start and end of any external call.
Here is the script :-

#!/bin/sh
# script to manage the bulk traffic server rates

HOUR=`date "+%H"`
APP="/usr/local/bin/nzbget"
PAUSED="$APP --pause"
RUNNING="$APP --unpause"
SLOW="$APP --rate 150"
FAST="$APP --rate 175"
VOIPACTIVELOCK=/tmp/voipactive.lock

if [ -f $VOIPACTIVELOCK ]; then
        $PAUSED
        else
        case $HOUR in
        01|02|03|04|05|06|07)
            $RUNNING
            $FAST
            ;;
        08|09|10|11|12|13|14|15|16|17)
            $RUNNING
            $SLOW
            ;;
        18|19|20|21)
            $PAUSED
            ;;
        22|23|00)
            $RUNNING
            $FAST
            ;;
        esac
fi

I am using an if statement to check for a current VOIP call (the Asterisk server sets the lockfile during an external call).
Also using the case conditional to check for the various times of day and adjust the rate or pause entirely.

Cleaning out old Linux kernels

21-Dec-09

After a while and many Ubuntu updates your PC starts acumulating old kernels , not really a problem but you only need the last couple really and might as well delete the old ones.

Get a list of all the installed kernels with :-

dpkg --get-selections | grep linux-image

I got this :-

linux-image-2.6.24-23-generic                   install
linux-image-2.6.24-24-generic                   install
linux-image-2.6.27-14-generic                   install
linux-image-2.6.28-12-generic                   install
linux-image-2.6.28-13-generic                   install
linux-image-2.6.28-14-generic                   install
linux-image-2.6.28-15-generic                   install
linux-image-2.6.28-16-generic                   install
linux-image-2.6.28-17-generic                   install
linux-image-generic                             install

The oldest ones are at the top , so I deleted them with :-

apt-get purge linux-image-2.6.24-23-generic

Each removed Kernel saved over 100Mb of disk space.
Keep your Ubuntu recovery CD handy just-in-case.

Bad Behavior has blocked 166 access attempts in the last 7 days.