DKIM e-mail signing

November 15, 2009 by Shahram Javey

According to Yahoo best e-mail sending practices, it helps if your e-mails have a DKIM signature. On a separate issue, I was having problems with checking e-mail addresses on Yahoo domain (from home on comcast network), so I thought I may as well add a DKIM signature header and see if it improves anything.

Well I did that, and the 1st signed e-mail I sent to my Yahoo e-mail immediately landed in the SPAM folder. So much for signing the e-mail. Anyhow, here are the steps that I needed to complete to add this capability to out-going e-mails from my application. Hopefully over time, the DKIM signing will help with e-mail delivery.

Create a private key:

openssl genrsa -out rsa.private 1024

Create a public key:

openssl rsa -in rsa.private -out rsa.public -pubout -outform PEM

Create A DNS record for your DKIM mail signature, e.g. (base64 public key truncated to display well in the blog)

YOUR-SELECTOR._domainkey.YOUR-DOMAIN IN TXT
"v=DKIM1; p=MIGfMA0GCSqGSIb3DQEBAQUAA4G...; s=email"

Use this wizard to create the TXT record.

You must add this DNS TXT record to your domain DNS entry. Contact your hosting service or your DNS provider. I contacted my hosting company bluehost.com and within an hour the DNS records were updated.

Install the dnspython and pydkim Python modules.

Wait for a while and use dig to check if your DNS record has been updated:

$ dig YOUR-SELECTOR._domainkey.YOUR-DOMAIN TXT

; <<>> DiG 9.6.0-APPLE-P2 <<>> YOUR-SELECTOR._domainkey.YOUR-DOMAIN TXT
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50920
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;YOUR-SELECTOR._domainkey.YOUR-DOMAIN. IN	TXT

;; ANSWER SECTION:
YOUR_SELECTOR._domainkey.YOUR-DOMAIN. 11678 IN TXT	"v=DKIM1\; p=MIGfMA0GCSqGSIb3DQEBAQUAA4G...; s=email"

;; Query time: 8 msec
;; SERVER: 10.0.1.1#53(10.0.1.1)
;; WHEN: Sun Nov 15 11:26:50 2009
;; MSG SIZE  rcvd: 298

Add the sign/verify option to your e-mail sending program:

    import dkim
    def dkim_sign_message(self, msg):
        try:
            sign = dkim.sign(msg, "YOUR-SELECTOR", "YOUR-DOMAIN", open(self._dkim_private_key, "r").read())
            message = "%s%s" % (sign, msg)

            if not dkim.verify(message):
                print "DKIM verify failed"
                message = msg
            return message
        except Exception as e:
            print "EXCEPTION ... %s" % e
            return msg

Where self._dkim_private_key is the path to the private key file that was created initially.

After you send an e-mail, you can view the e-mail headers from your e-mail client. You should see the Dkim-Signature header. Something like this:

Dkim-Signature: 	v=1; a=rsa-sha256; c=simple/simple; d=aquacue.org; i=@aquacue.org; q=dns/txt; s=accounts; t=1258312155; h=Content-Type : MIME-Version : Subject : From : To : X-Mailer : X-Organization : X-Abuse : X-Sender : X-Originating-IP; bh=JbJO60n6bvNZzyZilOW/hGrD2w5G6cR6o1YHimiwBbU=; b=iy8yt/YQlbhLUt2BpCWmKJ...

Aquacue Barnacle & Water Conservation

October 6, 2009 by Shahram Javey

Here is a follow-up story on Aquacue’s 1st pilot: “Results released by the water district this week show that those who got the feedback used 20 percent less water compared with the same period the year before…..The devices used in the study are manufactured by Los Gatos-based Aquacue and are attached to customers’ water meters. Known as Barnacles, the devices use wireless technology, the Internet and software to maintain constant water-use displays on users’ computer screens.” North County Times.

Here is the 1st story on this pilot

MySQL 5.1.x, Python, OS X 10.6

September 28, 2009 by Shahram Javey

Today I tried to get MySQL 5.1.39 (this version offers UUID_SHORT) working with Python. First you must uninstall mysql, mysqlb, and then issue the following commands:

$ which python
/opt/local/bin/python
$ python --version
Python 2.6.2
$ sudo port install mysql5-devel
$ sudo mysql_install_db5 --user=mysql
$ sudo /opt/local/bin/mysqld_safe5 --user=mysql&
$ /opt/local/lib/mysql5/bin/mysqladmin -u root password 'new-passwd'
$ ARCHFLAGS="-arch x86_64" python setup.py build
$ sudo python setup.py install
$ python
>>> import sys
>>> sys.path
# sys.path should print an entry with
# MySQL_python-1.2.3c1-py2.6-macosx-10.6-i386.egg'
>>> import MySQLdb
>>> MySQLdb.apilevel
'2.0'
>>> 

MySQL, Python on OS X – Snow Leopard

September 26, 2009 by Shahram Javey

I was surprised at how much time I have spent on getting MySQL and Python to work on OS X Leopard. Here are the steps that worked for me.

  1. Install MySQL using mac port.
  2. Download the latest python mysqldb, build and install it.

To install latest MySQL (5.1.x) see this entry.
Read the rest of this entry »

Aquacue 1st pilot

September 2, 2009 by Shahram Javey

Aquacue 1st technical pilot is a success. The results of how much it helped to conserve will be out shortly.

Aquacue Barnacle saves the day

July 29, 2009 by Shahram Javey

Over the past couple of days I was surprised at how much water the Aquacue Barnacle was reporting that we were using. The Barnacle was reporting over 80 gallons per minutes. Initially I though this was a false alert as we had just changed the software. But after some investigation that included help from San Jose Water Company we found the leak under the house. The Barnacle was right after all. Here is the proof of the leak:

The Barnacle reported water usage matched the water meter to 99.44% accuracy. The water pipe connecting the water meter to the house was old rusty galvanized pipe and it had just given up being a pipe. The cost of repair $3500 (included digging the sidewalk and front yard to get access to the water pipe and fixing the sidewalk, …). Damages to the house: none. Without the Barnacle I wouldn’t have found about the leak until after significant damage to the house was done. If I had acted immediately after the first Barnacle alert, the leak would have been detected in two hours.

How to add revision id to SVN controlled files

July 19, 2009 by Shahram Javey

First register the file with subversion for keyword substitution. For example, you may need to add the actual revision number to the source code.

$ svn propset svn:keywords "Revision" source_file.c
property 'svn:keywords' set on 'source_file.c'
$

Next, edit your source code and add the keywords that you wish to be replaced, e.g.

#define VERSION "V 1.0 $Rev$"

Finally, from now on each time you commit the file to subversion, e.g., after you commit the file:

$ svn commit source_file.c -m "file with rev. id"
Sending        source_file.c
Transmitting file data .
Committed revision 190.
$

Now if you view your source code, it will have the revision number in the code, e.g.

#define VERSION "V 1.0 $Rev 190 $"

and each time you edit and commit the changes the revision number will be updated.

Nine Hills vista, Alamaden Quicksilver Bike Trail

July 6, 2009 by Shahram Javey

Arduino development with command-line tools

July 2, 2009 by Shahram Javey

You may find that after your Arduino program grows to be over 1000 lines of C code that the Arduino IDE becomes increasing more annoying to use. Well, you can use TextMate, but first you need to set yourself up for command line programming. There are some instructions on the Arduino site for this, but it is a bit out of date. Specifically, the Makefile that they recommend simply does not work. Read the rest of this entry »

How to sync real-time clock with network-time on Telit modules

June 30, 2009 by Shahram Javey

You can configure the Telit GM cellular modem to set its internal real-time clock to the network time and have this value automatically updated each time the module is powered on. Here are the AT commands that you need to issue (I’ve omitted the AT response codes):

AT#SELINT=2
AT#NITZ=7,0 // sync network time to internal clock
AT&P0 // profile to be loaded on startup
AT&W0 // store the complete profile
AT#SHDN // this will turn off the modem
// now power on the modem
AT+CCLK? // get current time
// time response is +CCLK: "09/06/29,15:25:32-28"
// time format is yy/MM/dd,hh:mm:ss±zz

The command AT#NITZ=7,0 on your specific modem may not work, check the range of valid inputs for this command using AT#NITZ=?. If you’ve an old module from sparkfun.com then AT#NITZ=? may return #NITZ:(0-1),(0-7). More recent firmwares return #NITZ:(0-7)(0,1). If you’ve an older firmware, then you should set the AT#NITZ=1,0. The difference is that with newer firmware, you will also get the time-zone zz, not so with the older firmware. If you set AT#NITZ=7,1, then when the module gets the network time it is supposed to be echoed as an unsolicited message on the serial port. This didn’t work in my tests. Just as well. In this case, I rather not deal with unsolicited messages for network time, since I can use the real-time clock to get the time when I want to (getting unsolicited messages for AT+CREG is valuable, however).

What do I mean by older firmware? The GM862 module that I bought from sparkfun.com has this firmware:

AT+CGMR
PS:5.02.203/AL:6.04.204-GM862 QUAD

The module that I purchased from SemiConductorStore has this firmware:

AT+CGMR
07.03.600