Ramblings of the Engineman

mc2xml on ubuntu

2008-10-06T13:03:00.000-07:00

I recently switched to mc2xml for grabbing TV listings from TitanTV. The service doesn't always respond in a timely manner and so I needed a script to retry a couple of times in case it failed the first time. Here is the bash script that I wrote to retry every 60 seconds for 15 minutes. I hope that it helps someone else.

#!/bin/bash

servicebusy=2
repeats=0
maxrepeats=15

cd /home/user/mc2xml
./mc2xml
errorcode=$?

echo Error Code == $errorcode

while [[ $errorcode -eq $servicebusy ]]; do
if [[ $repeats -ge $maxrepeats ]]; then
exit -1
fi
let "repeats += 1"
echo repeating in 60 seconds \( try $repeats of $maxrepeats \)
sleep 60
./mc2xml
errorcode=$?
done

mythfilldatabase --refresh-all --file 1 xmltv.xml

I run it every night with the following command in my crontab

39 2 * * 1-7 bash /home/user/mc2xml/do_mythfill_fixed.sh

inadyn, opendns, and Ubuntu

2008-05-22T10:17:00.001-07:00

I have been using DNS services from OpenDNS.org for a little while now, and I am very happy with some of the features that they provide such as URL shortcuts and finely grained internet filtering. However, in order to use the features, I have to keep my dynamic IP address up to date with their DNS-O-Matic service. Their list of clients includes inadyn, which is available through Ubuntu but doesn't include DNS-O-Matics ssl extensions and so I compiled my own version and installed it.

Here are the steps I followed:

Download modified inadyn source code from OpenDNS -- wget http://www.opendns.com/support/ddns_files/inadyn.source.v1.99.zip
Unzip the source code -- unzip inadyn.source.v1.99.zip
sudo apt-get build-dep inadyn
sudo apt-get install libcurl4-openssl-dev
cd inadyn.source.v1.99
make
sudo cp -p bin/linux/inadyn /usr/local/sbin/.

Create a script named inadyn in /etc/init.d so that it will launch automatically on startup.

#! /bin/sh

### BEGIN INIT INFO
# Provides:          inadyn
# Required-Start:    $network
# Required-Stop:     $syslog
# Should-Start:      $named $syslog
# Should-Stop:       $syslog $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Dynamic IP Adress updater
# Description:       inadyn is a program that updates services such as
#                    dyndns.org and dnsomatic.com when the host's external
#                    external IP address changes.
### END INIT INFO

set -e

# /etc/init.d/inadyn: start and stop the inadyn daemon

DAEMON=/usr/local/sbin/inadyn
INADYN_ENABLE=true
INADYN_OPTS=''
INADYN_CONFIG_FILE=/etc/inadyn.conf

test -x $DAEMON || exit 0

. /lib/lsb/init-functions
. /etc/default/rcS

export PATH="${PATH:+$PATH:}/usr/sbin:/sbin:/usr/local/sbin"

case "$1" in
start)
     if "$INADYN_ENABLE"; then
         log_daemon_msg "Starting inadyn daemon" "inadyn"
         if [ ! -s "$INADYN_CONFIG_FILE" ]; then
             log_failure_msg "missing or empty config file $INADYN_CONFIG_FILE"
             log_end_msg 1
             exit 1
         fi
         if start-stop-daemon --start --quiet \
             --pidfile /var/run/inadyn.pid --make-pidfile\
             --exec $DAEMON
         then
             rc=0
         else
             rc=1
         fi
         if [ $rc -eq 0 ]; then
             log_end_msg 0
         else
             log_end_msg 1
         fi
     else
         if [ -s "$INADYN_CONFIG_FILE" ]; then
             [ "$VERBOSE" != no ] && log_warning_msg "inadyn daemon not enabled in /etc/init.d/inadyn, not starting..."
         fi
     fi
     ;;
stop)
     log_daemon_msg "Stopping inadyn daemon" "inadyn"
     start-stop-daemon --stop --quiet --oknodo --exec $DAEMON
     log_end_msg $?
     ;;
restart)
     set +e
     if $INADYN_ENABLE; then
         log_daemon_msg "Restarting inadyn daemon" "inadyn"
         start-stop-daemon --stop --quiet --oknodo --exec $DAEMON || true
         sleep 1
         if start-stop-daemon --start --quiet \
             --pidfile /var/run/inadyn.pid --make-pidfile \
             --exec $DAEMON
         then
             rc=0
         else
             rc=1
         fi
         if [ $rc -eq 0 ]; then
             log_end_msg 0
         else
             log_end_msg 1
         fi
     else
         [ "$VERBOSE" != no ] && log_warning_msg "inadyn daemon not enabled in /etc/default/inadyn, not starting..."
     fi
     ;;

*)
     echo "Usage: /etc/init.d/inadyn {start|stop|restart}"
     exit 1
esac

exit 0

sudo update-rc.d -f inadyn defaults

Create an inadyn.conf file in /etc

background
syslog

update_period_sec 600 # Check for a new IP every 600 seconds
secure

# Enter your DynDNS username and password here
username <USERNAME GOES HERE>
password <PASSWORD GOES HERE>

# What kind of host is being updated? Choices are dyndns@dyndns.org, statdns@dyndns.org, custom@dyndns.org
dyndns_server_name updates.dnsomatic.com
dyndns_server_url /nic/update?
alias <DNS-O-Matic SERVICE 1 GOES HERE>
alias <DNS-O-Matic SERVICE 2 GOES HERE>

/etc/init.d/inadyn start

all done inadyn is now installed and will keep DNS-o-Matic apprised of you IP address.

New MythTV Box

2008-05-22T09:56:00.001-07:00

I just put together a new MythTV box with an Intel Core2 Duo E7200 processor and a Gigabyte GA-73PVM-S2H. My build was based on one I found on another blog, and so far I am very pleased with the results. I used MythBuntu 8.04 for the OS, and despite the fairly minimal install MythBuntu provides, I have gotten everything working rather quickly. I intend to document a couple of the tweaks that I have to make in order to get everything working just right.

Here is a list of the additional packages that I have installed so far with apt-get:

unzip
nvidia-glx-new
nvidia-settings
sysvconfig (I like to use the Fedora/RH style 'service' script to stop and restart services)
hdhomerun-config

MythFrontend over VNC

2007-12-21T20:06:00.000-08:00

Sometimes I like to access my MythTV box over VNC. However, MythFrontend's default settings use OpenGL which doesn't play nicely with VNC. I found adding a parameter to the mythfrontend command forces it to use the QT painter instead of OpenGL which makes everything work perfectly (except for video of course). Since my MythTV box doesn't normally have a keyboard attached to it, and I find that using the remote to make changes in the setup can sometimes be tedious, so I use this trick to make things a little bit easier. Here's the command:

mythfrontend -O ThemePainter=qt

Circumventing restrictive firewalls using SSH for fun and profit

2007-11-30T09:12:00.000-08:00

I occasionally need to find a way to route network traffic from my machine through another machine in order to circumvent a draconian firewall or a network resource on a private machine. I order to get started you'll need a shell account on the remote computer. This could be as simple as signing up for a free ssh account from a site like those listed here. You could also setup an SSH server like copSSH on your Windows machine if you don't have access to a Linux machine somewhere and don't want to be restricted by the restrictions placed on most free SSH accounts.

Once you have ssh access to the remote machine then you can fire up Putty. Enter in the hostname in the hostname field and then in the Connection->SSH->Tunnels configuration panel put 8080 in the "Source Port" box, select the Dynamic option, and click "Add". Click "Open" and then login to the remote host like normal.
If you happen to be on a machine that has openSSH installed, then you can just type "ssh -D <remote hostname>" to acheive the same effect.

After the connection has been established you should be able to set any of your internet based programs that can use a SOCKS proxy to use the address "localhost:8080" as a proxy server. All of your data will then be automagically rerouted through an encrypted connection between the two computers and appear to be originating from the remote host instead of your local machine. Some of the programs that work well with this method are Pidgin, Firefox, and bit torrent clients like Azureus.

Pidgin Portable updated to version 2.3

2007-11-29T11:26:00.001-08:00

I have packaged a version of PidginPortable using Pidgin 2.3 and the method that I described earlier for Pidgin 2.2.1. You can download the file from here:

http://www.mediafire.com/?8mmnmtxs9jc

edit:Pidgin did a quick update to version 2.3.1. Please use this new download link to get the latest version.

http://www.mediafire.com/?7g1p29yfyjx

UPDATE: PortableApps.com has released an official version of Pidgin 2.4 so I will no longer be updating my version. Thanks to all of you who used it while it was needed.

Suspend to RAM to save power

2007-10-15T10:18:00.002-07:00

Today is Blog Action Day, and so I thought that I would write a bit about how I save energy while still maintaining an (almost) always on computer.

Every modern computer supports Suspend to RAM, but in my experience, few people use it. Most people either leave their computer on all of the time for convenience or turn it off when they are done with it in order to save power. Suspend to RAM allows you to have the best of both worlds very little power usage when it is suspended and instant availability when you are ready to use it again.

Basically, when your computer enters the Suspend to RAM (or goes to "sleep") everything inside the computer gets turned off except for the memory. This includes all of the fans, hard drives, video cards, etc. For all intents and purposes your computer will appear to be off except for possibly a small light somewhere on the computer that blinks to show that the computer is still on. When you wake the computer back up again, everything should appear exactly as it was before the computer went to sleep in less than 5 seconds.

Here are some simple tips on how to get started using Suspend to RAM mode on your Windows PC:

Open the "Power Management" control panel icon (Start->Run->"powercfg.cpl" works too)
Adjust the System Standby time to match your needs.
I prefer to put my computer to sleep and wake it up with the power button. I want to enable this function, then click the "Advanced" tab and change the "When I press the power button on my Computer" setting to be "Standby".

Now you should be all set to go. Your computer should go to sleep after a short period of time, and it will wake back up when you press a key on the keyboard or press the power button on the computer again.

Unfortunately there are a lot of things that can go wrong with this process. The most common problem that I have encountered is that the computer doesn't shut down completely and the fans and hard drives are still running. In this case doing one of the two following tricks normally fixes the problem:

Disable the "Allow this device to bring the computer out of Standby" setting for all of your USB devices.

Right click on "My computer and select "Manage", then open the "Device Manager" (or Start->Run->"devmgmt.msc")
Double click on the device to bring up the properties window
Make sure that "Allow this device to bring the computer out of Standby" is not checked in the "Power Management" tab

Change the power scheme to "Portable/Laptop"

Open Power Options in the Control Panel (or Start->Run->"powercfg.cpl")
Change the Power Scheme to "Portable/Laptop"
Check to see whether the fans still run in Standby
If the problem is fixed, then you should be able to change the Power Options to custom times without a problem.
This solution may seem a little bit sketchy since it doesn't seem to change anything that you couldn't change already, but there is definitely something that gets reset with this hack. I once had a machine that wouldn't sleep properly and using this hack fixed the problem.

If you need more help, there are a couple of people with more detailed writeups that you can follow: (exoid.com) (jillesvangurp.com) (thegreenbutton.com)

So how much power does this actually save? On my desktop, which I consider to be fairly average, I used a Kill-A-Watt meter to measure the power draw of my computer. While running, the machine uses ~150 Watts. If I suspend the machine, then I only draw ~8Watts. For a 24-Hour period that is a savings of about 3.4 KW-Hrs!

Upgrading PidginPortable to version 2.2.1

2007-10-09T07:01:00.000-07:00

If you don't want to read the whole post, here is a link to the download:
PidginPortable 2.2.1

I use Pidgin for most of my IM needs, and in order to keep my logs all in one place, I use the PortableApps.com version of PidginPortable and keep it on my USB Drive.

However, the version of PidginPortable on PortableApps.com hasn't been updated since mid August and is still stuck at version 2.1.1 while the latest official release is 2.2.1 which includes several important new features (such as MySpaceIM protocol), bug fixes, and even an important security fix. I finally got annoyed at being stuck with version 2.1.1 (especially since every time I started PidginPortable up I got a reminder message saying that there was a newer version available), and I decided to see if there was an easy way to upgrade the PortableApps distribution to 2.2.1.

It actually turned out to be really easy. All you need to do to upgrade is to replace the files in your "PidginPortable\App\Pidgin" folder with the same files from the current release of Pidgin. Unfortunately there isn't a binary only distribution on the Pidgin SourceForge page, so I installed Pidgin on my computer using the packaged installer and copied the files out of the installation directory. I also used upx to compress the pidgin.exe in order to save a little bit of space.

In order to make it easier for others who may be looking for the same thing, I have posted my updated version of PidginPortable on mediafire.com. Here is a link:

~~http://www.mediafire.com/?4q1dovsuto5~~

edit: I have updated to version 2.3 please use this new download link

http://www.mediafire.com/?8mmnmtxs9jc

Fake Credit Card Numbers

2007-10-05T10:51:00.000-07:00

Today I registered for a local engineering conference. Although my registration was free, for some reason their eCommerce site wanted a credit card number. Well, I don't particularly care to give my credit card out to just anyone on the internet, so what do I do, and why does their site want my credit card number if they aren't planning to charge it for anything?

Of course, I did the only reasonable thing a security concious engineer could do, I did a quick search for "fake visa number generators" on the internet and found this gem of a site:

http://www.darkcoding.net/index.php/credit-card-numbers/

Then, I took the first number off of the list, made up an expiration date and CVV number, and tried it out. Two pages later I got a lovely confirmation stating that my registration had been accepted.

To scrape or not to scrape?

2007-10-04T08:28:00.000-07:00

Recently Zap2It, the company which had been providing XML based TV listings for free to open source software such as MythTV, decided to shutdown their free XMLTV service. The "official" replacement for the service is a non-profit startup, SchedulesDirect. They now provide XML TV listings for a very reasonable price of $20/year.

I always have a hard time forking over money for something that I can get for free elsewhere, and in the wake of the labs.zap2it.com closure several enterprising individuals have developed programs which scrape TV listings from commercial websites. One of these is a PERL program called Zap2XML. I can report that the program works great, and there are even a couple of setup guides floating around around that target MythTV users.

However, the use of the program in my opinion constitutes a bit of a moral and ethical dilemma. You see there is the little problem of the fact that when I signed up for a free Zap2it.com account, I agreed to their Terms of Service which say "You may not scrape or otherwise copy our Content without permission" which technically prohibits me from scraping their website. However, their TOS also says, "you may download or print a single copy of any portion of the Content solely for your personal, non-commercial use". I believe that even though they prohibit me from scraping their website for listings with a program like Zap2XML, that as long as I am using it only for "personal, non-commercial use" that I am probably ok.

I believe that their reason for prohibiting scraping is two-fold. One, it prevents other websites from scraping Zap2It's TV listings (which they paid TMS a lot of money for) and then rebranding the content and displaying elsewhere. I am clearly not doing this, but the second reason is an even bigger moral dilemma. You see Zap2It (as well as 90% of the rest of the internet) depends upon advertising revenues to cover at least part of the cost of their doing business, and when I use a program like Zap2XML to scrape listings from their website I am obtaining content from them without viewing their advertising. Some people have argued that this is stealing, and this leads into the even bigger question of the morality behind online advertising and ad blockers such as Adblock Plus which is a whole other ball of wax that I don't really want to get into right now, but suffice it to say that I believe that there is some validity to arguments on both sides of the issue

I believe that this issue is the second reason why Zap2It specifically prohibits scrapers. They are providing content that costs them money to produce. Additionally there are costs involved in bandwidth and servers, and they believe that they have a right to recover part of those costs through advertising revenues. Now, I am not opposed to their collecting ad revenues from my viewing of their website, but I also feel like I have the right to select which content I view and how I view it, and so for now I am using Zap2XML.

Anyways, I just thought that I would throw some of this issue out there and see how my readers feel about this issue. Please feel free to post any and all comments or opinions that you might have on the morality and ethics behind this issue.

Mozy Online Backup: Simple, Automatic, Secure

2007-09-24T10:24:00.000-07:00

I installed the Mozy backup client a while ago and never thought twice about it silently backing up my files. Fast forward to last week when I tried installing the Enigmail extension for Thunderbird. I originally installed Enigmail so that I could exchange encrypted emails with a Client who insisted on using GPG instead of S/MIME.

The Enigmail installation monkeyed around with lots of my Thunderbird settings including adding an extra link to download IMAP messages from the server that I had to click everytime that I wanted to read a message. After messing around with things for a few minutes, I decided to give up on Enigmail and ditch it in favor of FireGPG. However, even after uninstalling Enigmail the problems persisted. I probably could have figured out how to get everything back to normal, but my Thunderbird was almost unusable as it was and I just wanted my email program back to normal.

For a moment I pondered with dread what it would take to reinstall Thunderbird and redo all of the customization I had done to personalize the app over the last three years, when suddenly I realized that this was a perfect time to try out a restore from Mozy. Fortunately I had included my Thunderbird directory in my Mozy backup set, so I did a quick restore of my prefs.js file, and when I restarted Thunderbird, lo and behold, everything was back to normal. Here are the steps that I followed in case someone else needs to do the same thing.

I shutdown Thunderbird so that it wasn't running.
I opened the Mozy Virtual Drive in My Computer.
I navigated to the Mozilla Thunderbird profile directory. (C:\Documents and Settings\\Application Data\Thunderbird\Profiles\)
I right clicked on the profile folder and chose "Change Time..." to select yesterday's backup.
I right clicked on the "prefs.js" file and chose "Restore" to restore the old file over the new file.
I restarted Thunderbird and everything was back the way it should be.

Thanks a lot mozy! You saved my day.

Here is what some other people are saying about Mozy
Walter S. Mossberg, The Wall Street Journal
Larry Armstrong, BusinessWeek

Lots and lots of tabs

2007-09-21T07:22:00.000-07:00

One of the things that I love most about Firefox is tabbed browsing. Most day I have between 10-20 tabs open. I don't know that I'm quite what you would call a tabaholic (maybe I'm still in denial), but I definitely have a problem. Tab scrolling in Firefox 2 is utterly worthless in my opinion. I mean what's the point if I can't see all of my lovely tabs at once. I tried the Vertigo extension, but the vertical tabs just didn't do it for me. Then, today, during another one of my marathon development/working/surfing sessions I discovered by the How to Geek that clued me in to the Tab Mix Plus extension. Now I can have my tabs arranged in multiple rows so that I can have my tabs and see them too.

Docufarm without the extension

2007-05-24T22:07:00.000-07:00

Today I found a groovy new website Docufarm that lets you preview .pdf, .doc, and .ppt files without downloading them. This is a real benefit when I want to quickly view a document without loading up a different program (especially a really slow memory hog like Acrobat).

However, at least for now the website doesn't provide an easy way to preview a document that I want to view without using their Firefox extension. I'm not always using a computer that I can install extensions on, but a blog post on their site gave me a pointer that I can use to DocuFarm any document that I want without using the extension. Simply prepending the url with

http://www.docufarm.com/view.jsp?url=

will do the trick. Nifty!

Update: Docufarm now lets you copy and paste a URL directly into their search box at http://www.docufarm.com. So this hack is no longer necessary, but it might be useful if you wanted to add it to document links on your own site.

DivX to iPod the Open Source way

2006-11-08T11:04:00.000-08:00

So I went looking today for some info on how to convert DivX shows to an iPod compatible H.264 format. There are lots of commercial solutions out there, but I prefer to find free software. I looked for a long while and finally found Mediacoder. Here are instruction on how to convert your divx files for your video ipod in 6 easy steps.

1. Go to http://mediacoder.sourceforge.net/ and download Mediacoder
2. Install Mediacoder and run it.
3. Select Add File and add the divx file
4. Select devices button
5. Select extension->digital media player
6. Press "start" to the left of the ipod.

Shazaaam! divx to ipod. in six easy steps.

Solar Powered battery Charger

2006-11-03T11:21:00.000-08:00

This isn't really a tip so much as a cool gadget. My dad and I have talked for a while about including a solar powered battery charger with our emergency kits. I finally found one today: Sundance Solar. Sundance solar also has a good selection of solar panels that I might use for some DIY electronics projects.

Manually Open the Safely remove hardware window

2006-08-05T14:20:00.000-07:00

Sometimes you need to remove an external hard drive or other piece of hardware from your Windows machine, but for some reason the little "Safely Remove Hardware" icon is missing from the System tray. With this tip you can open the Safely remove hardware window if that icon is missing.

copy and paste this into Start->Run:

RunDll32.exe shell32.dll,Control_RunDLL hotplug.dll

Show only items from Amazon.com

2006-07-03T22:26:00.000-07:00

I occasionally shop on Amazon.com, and sometimes I find it useful to show only items from Amazon.com in order to qualify for free shipping. I found that adding this text to the end of the URL will show only items from Amazon.com.

&field-enc-merchantbin=ATVPDKIKX0DER