Seeded (Salted) SHA Passwords

This week I needed to work with passwords from an OpenLDAP database. I needed to create users and encode their passwords as SSHA. After much googling and reading of authentication code examples, this is what I came up with.

import hashlib
import os
from base64 import urlsafe_b64encode as encode
from base64 import urlsafe_b64decode as decode

def makeSecret(password):
 salt = os.urandom(4)
 h = hashlib.sha1(password)
 h.update(salt)
 return "{SSHA}" + encode(h.digest() + salt)

def checkPassword(challenge_password, password):
 challenge_bytes = decode(challenge_password[6:])
 digest = challenge_bytes[:20]
 salt = challenge_bytes[20:]
 hr = hashlib.sha1(password)
 hr.update(salt)
 return digest == hr.digest()

Wow, python rules. So first I want to make sure I can validate a password against one generated by slappasswd

reedobrien$ slappasswd -s topsecret
{SSHA}Ccpjsip2UZL2CR2VsWTH7aF0vWKHQ7jn

And then see if I can validate it

>>> checkPassword('{SSHA}Ccpjsip2UZL2CR2VsWTH7aF0vWKHQ7jn',
...               'topsecret')
   True

Next I need to make sure I can generate one that OpenLDAP can use. So I used a script I wrote to create a user in ldap. This time I use the same password and get a different hash which is good.

>>> pw = "{SSHA}hq5ROYE8RoLA1Zcz6azgNQP5PkdETocx"

which OpenLDAP represents in LDIFs as base64 encoded

...
# tester, people, reedobrien.com
dn: uid=tester,ou=people,dc=reedobrien,dc=com
objectClass: top
objectClass: inetOrgPerson
uid: tester
cn: test test
sn: test
mail: test@example.com
userPassword:: e1NTSEF9MHhaMEdZc2Fob1JNeXZWR2FVdGszS0VwSFZTQnVLTlc=
...

So I try authenticating:

>>> userPasword = "e1NTSEF9MHhaMEdZc2Fob1JNeXZWR2FVdGszS0VwSFZTQnVLTlc="

>>> decode(userPassword) == pw
    True

>>> checkPassword(decode(userPassword), 'topsecret')
    True

Now I want to make sure it works through the ldap server itself
In python:

>>> import ldap

>>> con = ldap.initialize("ldap://127.0.0.1")
>>> con.simple_bind_s('uid=tester,ou=people,dc=reedbrien,dc=com',
                      'topsecret')
    (97, []) ## indicates success

And not using my stuff.

reedobrien$ ldapsearch -x -D "uid=tester,ou=people,dc=reedobrien,dc=com" -w topsecret
# extended LDIF
#
# LDAPv3
# base <> with scope sub
# filter: (objectclass=*)
# requesting: ALL
#

# search result
search: 2
result: 32 No such object

# numResponses: 1

How do they say, `w00t!`

So it was cool doing it and it made me feel smart for a minute. I am sure that there is a module or ten out there that already does this, but I couldn't find it. At least it was educational.

Unfortunately, after getting it to work; I don't I understand any better what it really does or how SSHA is better than plain old SHA.

Oh, well. Hopefully it helps some weary traveller out there one day.

Serving up a ZODB on demand from a repozo backup

Since I said I would on the mailing list/forum.

Here it is:

import os
import shutil
import tarfile
import tempfile

def application(environ, start_response):
  status = '200 OK'

  tempdir  = tempfile.mkdtemp()
  datafile = '%s/Data.fs' % tempdir
  tarball = '%s.tar.bz2' % os.path.join(tempdir,
                                        os.path.basename(datafile))

  ## If I were smarter or had more time
  ## I would import from repozo or at
  ## least use subprocess
  os.system("/usr/local/zope/2.9.7/bin/repozo.py -v -z -R -r \
         /usr/local/zope/sites/2.9.7/msrd/zeo/var/backup/ -o %s" % datafile)

  out = tarfile.TarFile.open(tarball, 'w:bz2')
  os.chdir(tempdir)
  out.add('Data.fs')
  out.close()


  response_headers = [('Content-type', 'application/octet-stream'),
                      ('Content-Length', str(len(open(tarball).read()))),
                      ('Content-Disposition',
                       'inline; filename="Data.fs.tar.bz2"')]

  start_response(status, response_headers)
  try:
      return [open(tarball, 'rb').read()]
  finally:
      shutil.rmtree(tempdir)

I did it really quick. But it solved my problem very well. It gets a copy of the Data.fs from the last repozo backup. So all developers don't need shell access to the server...

A couple things I could see doing that would make this cooler are:

• Not using os.system
• implement a method to do HEAD requests with the time of the last repozo delta so I don't compress and transfer this beast if there are no changes...
  

If you have mod_wsgi installed and working you just need to add something like:

WSGIScriptAlias /Data.fs.tar.bz2 /usr/local/apps/getDataArchived.py

to the appropriate configuration file.

Build PIL on Mac OSX

This entry attempts to give instructions for how to build the Python Imaging Library (PIL) on Mac OSX.

#download and untar the jpeg library
#see here for location http://www.ijg.org/

curl http://path/to/jpegsrc.v6b.tar.gz | tar zxf -

#change directory to

cd jpeg-6b

#run configure

./configure CFLAGS='-fPIC'

#run make

make

#Run make install

sudo make install

#run ranlib

ranlib libjpeg.a

#copy the lib file to lib

cp libjpeg.a /usr/local/lib

#copy headers to include

cp *.h /usr/local/include

#make this your real python

PYTHON_COMMAND="/path/to/your/python"

#get and untar the PIL source

curl http://path/to/Imaging-1.1.6.tar.gz | tar zxf -

# cd

cd Imaging-1.1.6

# copy setup out of it's way

mv setup.py setup.py.tmp

# set up a replacement command
# you can do edit setup.py by hand if you wish

SED_CMD="s|JPEG_ROOT = None|JPEG_ROOT = libinclude(\"/usr/local/lib\")|; s|ZLIB_ROOT = None|ZLIB_ROOT = libinclude(\"/usr/local/lib\")|;"

#replace the stock jpg locations with the new ones you made

cat setup.py.tmp | sed -e "$SED_CMD" > setup.py

#build it

$PYTHON_COMMAND setup.py build_ext -i

#test it

$PYTHON_COMMAND selftest.py

#install it

sudo $PYTHON_COMMAND setup.py install

ipython in zope3 zopectl debug

The solution in my earlier post doesn't fly in zope3. To get it to work in zope3 I modified:
~/instances/z3/bin/debugzope

if __name__ == '__main__':    db = startup()
     del startup
     from zope.app.debug import Debugger
     debugger = app = Debugger.fromDatabase(db)
     root = app.root()
     del db
     del Debugger
     import IPython
     IPython.Shell.IPShell(user_ns={'root': root,
                                     'app': app,                         
                                    'top' : app.root() 
                                     }).mainloop(sys_exit=1)

I haven't figured out much about how the debugger differs in Zope3. But it gives me top as app.root().

In [1]: print top.items()
<OOBTreeItems object at 0x355c598>

In [2]: app.root()['foo']
Out[2]: <zope.app.folder.folder.Folder object at 0x36dfef0>

In [3]: top
Out[3]: <zope.app.folder.folder.Folder object at 0x36a27f0>

In [4]: top['foo']
Out[4]: <zope.app.folder.folder.Folder object at 0x36df830>

In [5]: print [x for x in top.items()]
[(u'foo', <zope.app.folder.folder.Folder object at 0x36df830>)]

ipython in zopectl debug

A long time ago (6mos? a year) I got tired of manually typing support for history and tab completion into the zope debugger. So I just hooked the debug command up to read my .pythonrc file by importing user in the do_debug method of zopectl.py.

def do_debug(self, arg):
   cmdline = self.get_startup_cmd(self.options.python + ' -i',
   "import Zope2; app=Zope2.app(); import user;")

Today I tried getting ipython working as described in the plone docs. I also tried vanrees notes

and a couple others. I could not get it going though. So I decided to return to the hack above and extend it to load ipython.

def do_debug(self, arg):
    cmdline = self.get_startup_cmd(self.options.python + ' -i',
    "import Zope2; app=Zope2.app(); import user;     
     ns={'__name__':'msrd','app':app}; import IPython;   
     IPython.Shell.IPShell(user_ns=ns).mainloop(sys_exit=1);")

Poof, now debug comes up with ipython. Yay... What does that mean you may ask?

In [2]: ??app.msrd
Type: ImplicitAcquirerWrapper
Base Class:
String Form:
Namespace: Interactive
Length: 1
Docstring [source file open failed]:
Make PloneSite subclass CMFSite and add some methods.
This will be useful for adding more things later on.

I know this is probably the wrong way to do this, but nyah nyah. It is working now. I do welcome feedback...

As a note: I also have import user in my args. This picks up stuff from your file defined in PYTHONSTARTUP. But then if you already use PYTHONSTARTUP you probably already know that.

Plone site broken? Fix ir through the zodb

Today someone changed the ip on a dev box running an instance that had the ip as part of the url in CacheFu. So of course it won't load a page anymore. What to do? reinstall? nah. Let 's think for a second. Plone is a nice interface to Zope/CMF. Zope/CMF is a lovely interface to ZODB. ZODB is a persistence engine for Python objects. CacheFu's settings are persisted as atributes of a cache object in the ZODB. Let's just fix it directly....

/usr/local/{instance}/bin/zopectl debug
Starting debugger (the name "app" is bound to the top-level Zope object)
...{ 8< snip } ...
>>> import readline, rlcompleter, transaction
>>> readline.parse_and_bind("tab: complete")
# now we have tab completion
# bind the plone root to s
>>> s = app.itcd
# bind our (s)ites cache settings to cc
>>> cc = s.portal_cache_settings
>>> cc.getDomains()
('http://1.2.3.4:80',) ## <<== look there is our old value!!
# yup it needs to be this new value
>>> cc.setDomains("http://4.3.2.1:8080")
# and persisted. Otherwise it will be aborted at exit
>>> transaction.commit()

That was easy, no?

Quick and dirty regexen search for spamd & postfix logs

#!/usr/bin/env python
#-------------------------------------------------------------
# Name:        finder.py
# Purpose:     This is a script to search through  Posfix 
#              and spamd logs for the last x days and return 
#              hits. Mostly it is a quick and dirty way to 
#              find entris regarding emails blocked by 
#              spamd/postfix   
# Author:      Reed L. O'Brien reed at reedobrien com
#                                                                                                           
# Created:     2007-01-05
# Modified:    2007-01-05
# Copyright:   (c) Reed L. O'Brien 2007
# License:     DWYWWI (improvements welcome)                         
#--------------------------------------------------------------                      


#Do the imports
import os, re, bz2, sys, time

# make sure there is a regex
try:
    # compile the regex
  regx = re.compile(sys.argv[1], re.IGNORECASE)
except IndexError:
  print """
  usage:
         finder  [days back to search]

         ex: finder foobar 2
         will find all occurences of 'foobar' in the last 2
         days of spamd and maillog files.\n\tThe number of days
         is optional and defaults to 1 if not given."""
  sys.exit(0)

# empty list to store hits in
found = []

# move to the log directory
os.chdir('/var/log')
# Start a counter for the number counted
s = 0

#get and set days
try:
 days = int(sys.argv[2])
except:
 days = 1
# Get a list of qualifying files NOTE: you may need stat(x).[st_ctime|st_mtime] depending on your OS
L = [f for f in os.listdir('.')
   if os.stat(f).st_birthtime > time.time() - (days * 86400)
   and (f.startswith('spamd') or f.startswith('maillog'))]
# get a count of how many files to search
n = len(L)

# start a loop on the list
for f in L:
    # If it is a bz2 open it as a bz2 object
if (f.startswith('spamd') or f.startswith('maillog')) and f.endswith('2'):
 # tell em what is happening
   sys.stdout.write("\rsearching: %2s  remain %s                   " % (f,n))
   sys.stdout.flush()
        # set a line count
   c = 1
        # get a handle on the file
        handle = bz2.BZ2File(f)
        # iterate through the lines
        for line in handle:
        # if the regex is found
       if regx.search(line):
           # append the filename, line count and line content to the found list
           found.append("%-10s : %s\n%s" % (c, f, line))
                # increment the line count
           c += 1
       else:
           # or just increment the count if no regex match 
           c += 1
                # decrement the number of files remaining
           n -= 1  
                # increment the number of files searched
           s += 1

    ## DO the same as above as a regular file object if not a bz2 file SEE NOTES FOR last loop
if (f.startswith('spamd') or f.startswith('maillog')) and not f.endswith('2'):
   sys.stdout.write("\rsearching: %2s  remain %s                     " % (f,n))
   sys.stdout.flush()
   c = 1
   handle = open(f)
   for line in handle:
       if regx.search(line):
           found.append("%-10s : %s\n%s" % (c, f, line))
           c += 1
       else:
           c += 1
           n -= 1
           s += 1
                ##make some space to overwrite the sys.stdout text 
print '\n\n\n\n\n'
print 'Searched:', s # Print how many files were searched

#print the results from the found list.
for x in found:
print x

Mostly functional email regex

for word based delims in text:
em = re.compile(r"\b['A-Z0-9._%-]+@[A-Z0-9.-]+\.[A-Z]{2,4}\b",
   re.IGNORECASE)


for start and endline delims:
em = re.compile(r"^['A-Z0-9._%-]+@[A-Z0-9.-]+\.[A-Z]{2,4}$",
    re.IGNORECASE)