How to find the IP/MAC addresses of devices on your network

December 28, 2009 by Shahram Javey

Every once in a while I need to find out the IP & MAC addresses of various devices in my home network. Here is a small shell script that will find this information. It takes a while to run. Without the initial ping, the arp command will not return the necessary data on an IP address that you’ve never visited from the machine that you’re running this script on.

Perhaps there is a better way of finding the IP/MAC addresses of devices on your network. If so, please let me know.

#!/bin/sh

for (( i = 0; i < 256; i++))
do
    ping -c 1 10.0.1.$i
    arp 10.0.1.$i
done

“Adobe Unit Types.osax” & debugging Python on Mac

December 4, 2009 by Shahram Javey

On OSX, if you’ve installed one of the Adobe apps, then when you debug your Python application you’ll start getting this annoying set of errors on the console:

osascript[13514:607] Error loading /Library/ScriptingAdditions/Adobe Unit Types.osax/Contents/MacOS/Adobe Unit Types:  dlopen(/Library/ScriptingAdditions/Adobe Unit Types.osax/Contents/MacOS/Adobe Unit Types, 262): no suitable image found.  Did find:
	/Library/ScriptingAdditions/Adobe Unit Types.osax/Contents/MacOS/Adobe Unit Types: no matching architecture in universal wrapper
osascript: OpenScripting.framework - scripting addition "/Library/ScriptingAdditions/Adobe Unit Types.osax" declares no loadable handlers.

Here is how you get finally get rid of this problem. Delete this file alone wont fix the problem since launching another Adobe app will more than likely recreate this file. I found this solution to work:

cd /Library/ScriptingAdditions
sudo ln -s /dev/null ./Adobe\ Unit\ Types.osax

AuthKit, valid_password and wraptools

December 4, 2009 by Shahram Javey

I’m new to AuthKit, and I wanted to override the method authkit.authenticate.valid_password for the form way of handling authentication. But could not figure out a good way of doing this. It appears that they only way overriding this method is to use the forward handler, but then again I had AuthKit working more or less the way I wanted using the form handler.

So how do you go about overriding one method in a library that you’re using? Well in Python you can use the wraptools module. This allows you to override a method using decorators. For example,

@replaces(authkit.authenticate.valid_password)
def valid_password(environ, username, password):
    try:
        # validate the username/password
        return True
    except YourLoginException:
        return False

Flup, AuthKit, & unicode strings

December 1, 2009 by Shahram Javey

If you’re on a hosting environment (e.g., bluehost.com) where the only option to run Python/Pylons is via flup (fast cgi), then you may have experienced this error:


<'exceptions.AssertionError'>: write() argument must be string

The bottom line is that the WSGI spec only supports plain strings.

AuthKit, is a middleware that intercepts HTTP request/response in order to enforce authentication and authorization. One issue that you will encounter is that any page that gets rendered via AuthKit by default will be in unicode. This causes flup to throw the above exception. The fix is pretty straight forward:

def render_signin():
    result = render('/signin.mako')
    result = result.replace('%', '%%').replace('FORM_ACTION', '%s')
    return unicode(result).encode('utf8')

Simply cast the result to unicode and encode it to utf8.

DKIM e-mail signing

November 15, 2009 by Shahram Javey

According to Yahoo best e-mail sending practices, it helps if your e-mails have a DKIM signature. On a separate issue, I was having problems with checking e-mail addresses on Yahoo domain (from home on comcast network), so I thought I may as well add a DKIM signature header and see if it improves anything.

Well I did that, and the 1st signed e-mail I sent to my Yahoo e-mail immediately landed in the SPAM folder. So much for signing the e-mail. Anyhow, here are the steps that I needed to complete to add this capability to out-going e-mails from my application. Hopefully over time, the DKIM signing will help with e-mail delivery.

Create a private key:

openssl genrsa -out rsa.private 1024

Create a public key:

openssl rsa -in rsa.private -out rsa.public -pubout -outform PEM

Create A DNS record for your DKIM mail signature, e.g. (base64 public key truncated to display well in the blog)

YOUR-SELECTOR._domainkey.YOUR-DOMAIN IN TXT
"v=DKIM1; p=MIGfMA0GCSqGSIb3DQEBAQUAA4G...; s=email"

Use this wizard to create the TXT record.

You must add this DNS TXT record to your domain DNS entry. Contact your hosting service or your DNS provider. I contacted my hosting company bluehost.com and within an hour the DNS records were updated.

Install the dnspython and pydkim Python modules.

Wait for a while and use dig to check if your DNS record has been updated:

$ dig YOUR-SELECTOR._domainkey.YOUR-DOMAIN TXT

; <<>> DiG 9.6.0-APPLE-P2 <<>> YOUR-SELECTOR._domainkey.YOUR-DOMAIN TXT
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50920
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;YOUR-SELECTOR._domainkey.YOUR-DOMAIN. IN	TXT

;; ANSWER SECTION:
YOUR_SELECTOR._domainkey.YOUR-DOMAIN. 11678 IN TXT	"v=DKIM1\; p=MIGfMA0GCSqGSIb3DQEBAQUAA4G...; s=email"

;; Query time: 8 msec
;; SERVER: 10.0.1.1#53(10.0.1.1)
;; WHEN: Sun Nov 15 11:26:50 2009
;; MSG SIZE  rcvd: 298

Add the sign/verify option to your e-mail sending program:

    import dkim
    def dkim_sign_message(self, msg):
        try:
            sign = dkim.sign(msg, "YOUR-SELECTOR", "YOUR-DOMAIN", open(self._dkim_private_key, "r").read())
            message = "%s%s" % (sign, msg)

            if not dkim.verify(message):
                print "DKIM verify failed"
                message = msg
            return message
        except Exception as e:
            print "EXCEPTION ... %s" % e
            return msg

Where self._dkim_private_key is the path to the private key file that was created initially.

After you send an e-mail, you can view the e-mail headers from your e-mail client. You should see the Dkim-Signature header. Something like this:

Dkim-Signature: 	v=1; a=rsa-sha256; c=simple/simple; d=aquacue.org; i=@aquacue.org; q=dns/txt; s=accounts; t=1258312155; h=Content-Type : MIME-Version : Subject : From : To : X-Mailer : X-Organization : X-Abuse : X-Sender : X-Originating-IP; bh=JbJO60n6bvNZzyZilOW/hGrD2w5G6cR6o1YHimiwBbU=; b=iy8yt/YQlbhLUt2BpCWmKJ...

Aquacue Barnacle & Water Conservation

October 6, 2009 by Shahram Javey

Here is a follow-up story on Aquacue’s 1st pilot: “Results released by the water district this week show that those who got the feedback used 20 percent less water compared with the same period the year before…..The devices used in the study are manufactured by Los Gatos-based Aquacue and are attached to customers’ water meters. Known as Barnacles, the devices use wireless technology, the Internet and software to maintain constant water-use displays on users’ computer screens.” North County Times.

Here is the 1st story on this pilot

MySQL 5.1.x, Python, OS X 10.6

September 28, 2009 by Shahram Javey

Today I tried to get MySQL 5.1.39 (this version offers UUID_SHORT) working with Python. First you must uninstall mysql, mysqlb, and then issue the following commands:

$ which python
/opt/local/bin/python
$ python --version
Python 2.6.2
$ sudo port install mysql5-devel
$ sudo mysql_install_db5 --user=mysql
$ sudo /opt/local/bin/mysqld_safe5 --user=mysql&
$ /opt/local/lib/mysql5/bin/mysqladmin -u root password 'new-passwd'
$ ARCHFLAGS="-arch x86_64" python setup.py build
$ sudo python setup.py install
$ python
>>> import sys
>>> sys.path
# sys.path should print an entry with
# MySQL_python-1.2.3c1-py2.6-macosx-10.6-i386.egg'
>>> import MySQLdb
>>> MySQLdb.apilevel
'2.0'
>>> 

MySQL, Python on OS X – Snow Leopard

September 26, 2009 by Shahram Javey

I was surprised at how much time I have spent on getting MySQL and Python to work on OS X Leopard. Here are the steps that worked for me.

  1. Install MySQL using mac port.
  2. Download the latest python mysqldb, build and install it.

To install latest MySQL (5.1.x) see this entry.
Read the rest of this entry »

Aquacue 1st pilot

September 2, 2009 by Shahram Javey

Aquacue 1st technical pilot is a success. The results of how much it helped to conserve will be out shortly.

Aquacue Barnacle saves the day

July 29, 2009 by Shahram Javey

Over the past couple of days I was surprised at how much water the Aquacue Barnacle was reporting that we were using. The Barnacle was reporting over 80 gallons per minutes. Initially I though this was a false alert as we had just changed the software. But after some investigation that included help from San Jose Water Company we found the leak under the house. The Barnacle was right after all. Here is the proof of the leak:

The Barnacle reported water usage matched the water meter to 99.44% accuracy. The water pipe connecting the water meter to the house was old rusty galvanized pipe and it had just given up being a pipe. The cost of repair $3500 (included digging the sidewalk and front yard to get access to the water pipe and fixing the sidewalk, …). Damages to the house: none. Without the Barnacle I wouldn’t have found about the leak until after significant damage to the house was done. If I had acted immediately after the first Barnacle alert, the leak would have been detected in two hours.