Time for Python Neenjah!

I happened upon an administrative assistant today who was renaming files using Windows Explorer. She had a folder with about 50 or so sub-folders, many of which contained sub-folders of their own. Every file needed to be renamed with the company name prefixing it. So for example, the file "january-charts.pdf" needed to be renamed to "Company - january-charts.pdf".

"You know", I said, "I could help that go a little faster if you're interested." I happened to know that these files needed to be sent as of yesterday, so I figured she wouldn't mind me helping to trim an hour or two off them getting out. That must mean it's time for Python Neenjah! (In case you missed it, that was a somewhat veiled reference to xkcd 'Regular Expressions')

A month or two ago I had put together a re-usable python module to allow easy recursively searching a directory. I know, I know, python already includes similar functionality. But it was weird to me, and I wanted a simpler and more flexible format. The module I built, inspire by some ideas I found around the internet, allows a callback to be specified whenever a file is found. It means you can do virtually anything from that directory search, and never really be concerned about how it does it.

from dirsearch import DirSearch

def search_callback(file):
 print file

dir = DirSearch('C:\Path\Whatever', show_output=True)
dir.search(search_callback)

The source for dirsearch.py is included below.

Well with my module it only took a few lines of code to put together a command to complete the task at hand.

DirSearch.py


rrename.py

Structure and Interpretation, LISP and Python

I'm a "city boy" when it comes to programming languages. 90% of my experience in programming is in ridiculously high level languages like VB.NET, C#, and Python. I haven't "roughed it out" much in the wilderness of some low level language.

I had heard talk about MIT's Open Courseware, but wasn't sure what the hoopla was all about. Based on a reference in someone's blog post, I happened upon a 1986 recording of Structure and Interpretation of Computer Programs (YouTube). I watched the whole thing. I am thoroughly blown away.

I'll admit, the first 20 minutes were dizzyingly abstract, but once some code samples started to show up, it began to make sense. What I was floored the most about was seeing how much an influence LISP had over the language I've been working in the most lately -- Python. All of the "funky things" that I wasn't used too from C#, inner methods, using "def" instead of "function", etc. etc. were all elements of LISP.

So having FizzBuzz on the mind, I decided to give it a shot.... in LISP (and then after recursively in Python). The course professor noted that LISP had no for loops. "A challenge" I thought to myself. (Long Side Tangent: I feel at the moment as if I'm creating a programmer's Fight Club where I mentally abuse the comfortable high-level language life I once knew to get down and dirty fighting with the bare essentials of computational logic. [end sensationalistic, metaphoric movie reference])

In case you're not familiar with FizzBuzz, here's the problem:

Write a program that prints the numbers from 1 to 100. But for multiples of three print "Fizz" instead of the number and for the multiples of five print "Buzz". For numbers which are multiples of both three and five print "FizzBuzz".

I "made up" a function definition for print since the details of printing to the screen weren't really my concern.

The odd part for me was after working through the mental process of how the solution would work recursively in LISP, writing it again in Python felt ridiculously easy...

Look mom, no loops!

Determine Model Change v2

Well I got around to doing a 2nd revision on my model change code (being the weekend I was wondering if it would come to pass). Per a suggestion by "thepointer" (#django IRC on freenode), I switched the code from using Python's generic vars() to Django's interal _meta. Using an internal API is probably not the ultimate best, but _meta has been stable and unchanged for quite a while.

I also added a "human_friendly" mode, which will take the model change and attempt to turn it into an understandable statement (string) about what exactly has changed. It still returns it in a dictionary with the field name as the key.

from django.db import models

def determine_model_change(old_model, new_model, human_friendly=False, ignore_fields={}):
    """
    Compares the two models against each other, returning a dictionary of
    values that have changed (new value only).

    Setting human_friendly=True will cause ignore internal fields like
    SlugField.  It will also attempt to parse a meaningful statement for
    the model change. ie 'Event date is now 5/7/2009'
    """
    not_human_fields = (
        models.fields.SlugField,

        models.fields.FilePathField,
        models.IPAddressField,
        models.FileField,
        models.ImageField,
        models.XMLField,
    )
    changed = {}
    
    if isinstance(old_model, models.Model) and isinstance(new_model, models.Model):
        for f in new_model._meta.fields:
            if not f.name in ignore_fields:
                new_value = getattr(new_model, f.name, '')
                old_value = getattr(old_model, f.name, '')
                if cmp(new_value, old_value) != 0:
                    if human_friendly:
                        if not type(f) in not_human_fields:
                            changed[f.name] = __verbose_field_change(old_model, new_model, f)
                    else:
                        changed[f.name] = new_value
    
    return changed

def __verbose_field_change(old_model, new_model, field):
    """
    Returns the human-friendly text for a field change
    """
    value = getattr(new_model, field.name)
    if isinstance(field, models.fields.DateField) or \
        isinstance(field, models.fields.TimeField) or \
        isinstance(field, models.fields.DateTimeField):
            
        value = value.strftime('%b %d, %Y %I:%M %p')    
    return '%s %s has changed to %s' % (
            old_model,
            field.verbose_name,
            value
        )


# -------------
# Sample usage:
# -------------

>>>
>>> from happenings.models import *
>>> import copy, datetime
>>>
>>> event = Event.objects.get(pk=1)
>>> event.name = "My Birthday"
>>> event.start_time = datetime.datetime(2009, 7, 5, 0, 0)
>>>
>>> newevent = copy.copy(event)
>>> newevent.start_time = datetime.datetime(2009, 7, 15, 0, 0)
>>>
>>> determine_model_change(event, newevent)
{'start_time': datetime.datetime(2009, 7, 15, 0, 0)}
>>>
>>> determine_model_change(event, newevent, human_friendly=True)
{'start_time': 'My Birthday start time has changed to Jul 15, 2009 12:00 AM'}

Easier, Faster Property Enumeration in Python

I just discovered a really neat trick in Python to take a collection of objects, and turn one of their properties into a list. It's not terribly difficult to perform this the old way...

subscribers = []
for subscription in self.subscriptions.all():
 subscribers.append(subscription.user)
return subscribers

This would return something like [(User:bob),(User:jerry),(User:tim)] and so on. However, this can be done in just a single line...

return [s.user for s in self.subscriptions.all()]

+1 for Python coolness.

Ignoring .pyc files in NetBeans

I started using the Netbeans 6.5 Python (Early Access) IDE a couple of weeks ago, and while all seemed to be going well, one thing that bugged me was seeing all the .pyc (python compiled file) in the treeviews. Turns out NetBeans has a simple way to fix this:

• On the menu go to Tools > Options
• Then "Miscellaneous"
• Then "Files" tab
• And find the section "Files Ignored by the IDE"

This value is a regular expression which makes it easy to add the functionality we're looking for.

Change this:
^(CVS|SCCS|vssver.?\.scc|#.*#|%.*%|_svn)$|~$|^\.(?!htaccess$).*$

To this:
^(CVS|SCCS|vssver.?\.scc|#.*#|%.*%|_svn)$|~$|^\.(?!htaccess$)|pyc.*$

You could easily use this method for any other file type you'd like to ignore. Just add "|extension" before the .*$.