Meh @reading (but Yay to reading on the iPod)

I can't be the only programmer who finds reading a chore? And what about those 5-billion page programming tomes? Am I the only one who never finishes those?

It happens to be that I just found something about reading that totally rocks: Reading on the iPhone/iPod Touch is totally different (better different). HA! I bet you didn't know I was going to say that (well if you read the title of this post you probably did.. that was totally a spoiler).

Now it seems that I am finding I actually enjoy reading again (did I just write that?). I ended up landing a very fetching iPod touch for Christmas which has been having it's battery drained near non-stop since the package was opened. As a curiosity of sorts, I threw a lot of the big name apps onto it, including Kindle for Ipod, and purchased a book I had been eying for a while, Javascript: The Good Parts (Douglas Crockford), to try out the experience of reading on the iPod.

It seems so wrong but I found I actually prefer reading on the iPod. I've read a number of paper books, and I've tried reading eBook/PDFs on the computer, but neither of them really jump out at me. But the iPod... the iPod I like.

The reason for this, I suspect, is that reading on the iPod is not intimidating. The screen only shows 2 paragraphs at a time. I don't need to be worried about the rest of the book, I'm only looking at these 2 paragraphs. What I think this does for me is it allows me to read the book without feeling like I have to rush to get to the end.

My wife thinks it's silly, but whatever -- I'm already on my 2nd book.... Real World Haskell (Donald Stewart). Oooooh yes, it's Haskell time! Expect some really weirded out blog posts from me in the future about functional programming.

How can I create a page that regularly updates itself with Django?

It seems this question has been coming up a lot on Stack Overflow, and I wrote a pretty lengthy response which I'll include here. The original question is #1883266.

Q: I'm building a web application that needs to have content on the page updated in real time without a page refresh (like a chat or messaging application). How do I do this with Django? In other words, how do I use AJAX with Django?

A: There's a lot going on in order to make this process work...

• The client regularly polls the server for new chat entries
• The server checks for and only replies with the newest
• The client receives the newest entries and appends them to the DOM

This can be confusing when you're first starting because it's not always clear what the client does and what the server does, but if the large problem is broken down I think you'll find it's a simple process.

If the client is going to regularly poll the server for new chat entries, then the server (django) needs to have some type of API to do so. Your biggest decision will be what data type the server returns. You can choose from: rendered HTML, XML, YAML, or JSON. The lightest weight is JSON, and it's supported by most of the major javascript frameworks (and django includes a JSON serializer since it's that awesome).

# (models.py)
# Your model I'm assuming is something to the effect of...
class ChatLine(models.Model):
  screenname = model.CharField(max_length=40)
  value = models.CharField(max_length=100)
  created = models.DateTimeField(default=datetime.now())

# (urls.py)
# A url pattern to match our API...
url(r'^api/latest-chat/(?P<seconds_old>\d+)/$',get_latest_chat),

# (views.py)
# A view to answer that URL
def get_latest_chat(request, seconds_old):
    # Query comments since the past X seconds
  chat_since = datetime.datetime.now() - datetime.timedelta(seconds=seconds_old)
  chat = Chat.objects.filter(created__gte=comments_since)

   # Return serialized data or whatever you're doing with it
   return HttpResponse(simplejson.dumps(chat),mimetype='application/json')

So whenever we poll our API, we should get back something like this (JSON format)...

[
 {
     'value':'Hello World',
     'created':'2009-12-10 14:56:11',
     'screenname':'tstone'
 },
 {
     'value':'And more cool Django-ness',
     'created':'2009-12-10 14:58:49',
     'screenname':'leethax0r1337'
 },
]

On our actual page, we have a <div> tag which we'll call <div id="chatbox"> which will hold whatever the incoming chat messages are. Our javascript simple needs to poll the server API that we created, check if there is a response, and then if there are items, append them to the chat box.

<!-- I'm assuming you're using jQuery -->
<script type="text/javascript">

  LATEST_CHAT_URL = '{% url get_latest_chat 5 %}';

  // On page start...
    $(function() {
      // Start a timer that will call our API at regular intervals
        // The 2nd value is the time in milliseconds, so 5000 = 5 seconds
        setTimeout(updateChat, 5000)
  });

  function updateChat() {
      $.getJSON(LATEST_CHAT_URL, function(data){
          // Enumerate JSON objects
            $.each(data.items, function(i,item){
              var newChatLine = $('<span class="chat"></span>');
              newChatLine.append('<span class="user">' + item.screenname + '</span>');
              newChatLine.append('<span class="text">' + item.text + '</span>');
              $('#chatbox').append(newChatLine);
          });
      });
  }

</script>

<div id="chatbox">
</div>

IE TextRanges, Selections, and Carriage Returns

I spent several hours this week trying to track down a bug with programmatically selecting text via javascript.

if (textarea.setSelectionRange) {
 // Set selection for Mozilla-ish browsers
    textarea.setSelectionRange(start, end);
 }
 else {
 // Set selection for IE
    var range = textarea.createTextRange();
 range.collapse(true);
 range.moveEnd('character', start);
 range.moveStart('character', end);
 range.select();
}

Assuming that 'textarea' is the element of the textarea, and 'start' and 'end' are Number values that indicate the start and end of the selection, respectively.

I was managing this in MarkEdit by keeping a state object that recorded the "Before Select" (everything before the selection), the selection text, and the "After Select" (everything after the selection).

So it seems easy to implement right?

start = beforeSelect.length;
end = start + select.length;

That doesn't always work!

It works sometimes, but occasionally the selection will be the proper length, but randomly offset by 3-7 characters. It took quite a bit to figure out (that's an understatement). Here's what a lot of debugging and experimentation came up with:

When Internet Explorer creates a Range or a TextRange object, it converts all line returns to Carriage Return/Line Feed format; basically [0A] -> [0D 0A]. Easy to counter? Just determine all the instances of [0D] and [0A], subtract the two, then you have the offset? That's what I thought too, but even with the offset, the discrepancy persisted.

So it was time for a new approach. What if we converted all of the [0A] characters in the textarea to [0D 0A]'s ? Again, the offset of the selection persisted.

Here's what I finally narrowed it down to be: When you create a TextRange, IE doesn't actually re-render the text internally, converting all [0A]'s. However, when you request the text (TextRange.text) it converts it at that time. This means if you were to get the length of the selected text, TextRange.text.length, it would be longer than what the TextRange is seeing internally! When using .moveEnd on the TextRange, the value counts the position from the internal text, not from the text it gives you.

That is reeeeeeediculously wrong. The final solution is to "sanitize" the text TextRange gives you by removing all [0D] characters from it. From there you can accurately calculate the selection position and give the TextRange the proper coordinates.

Whew... that was several hours needlessly wasted by IE (again).

WMD Editor & jQuery

I've been using Dana Robinson's WMD Editor fork for a couple of months now. It was a great place to start. The problem is, it's becoming a real pain to customize. Small things were taking huge amounts of time to accomplish so I had a tough decision to make: Continue to use WMD Editor and have to spend hours for customization that should only take a minute or two -OR- re-write the UI myself on top of the existing Markdown rendering component?

Introducing: MarkEdit -- the Javascript MarkDown editor built on jQuery (to replace WMD Editor). It's currently being hosted on BitBucket.

• Built On jQuery/UI -- Lighter weight code and less cross-browser issues
• Themeroller Support -- Use existing or custom jQuery UI Themeroller themes
• International -- Supports the loading of an alternate language
• Sensible Defaults -- You should be able to get up and running with 1 line of javascript
• Configurable -- Almost all defaults can be overridden very easily upon initialization
• Public API -- Interact with whatever part you're interested in
• Documented

Since everybody loves screenshots, here's the editor running under a jQuery theme and using the Chinese language translation.





And again here's a translated popup asking for the URL to insert an image...




And here's preview mode using a different theme but still in Chinese...




So far I've invested about 15 hours into the project with it being probably around 95% complete for initial coding. There is still a TON of testing to be done. I haven't even tried to fire it up in any of the fussy browsers yet.

I actually wrote a whole lot about it so far -- it's all up on the Wiki. There is still lots more documentation to come.

Using jQuery's Dialog Function to Get User Input

The jQuery UI library provides a really useful method: dialog. It allows any tag to be rendered as a fake "window" in the browser in either a modal or non-modal way. It's easily a natural choice for any time we need a dialog to fetch input from a user.

What makes it somewhat difficult for some programmers to use is that it's an asynchronous function. It opens the dialog then keeps on executing code. Here's why this can be confusing.

Let's say for example we're building a web-based instant messaging client. A feature of this client would be the ability for the user to change their nickname, even after they've already logged in (similar to how MSN works). We've decided to implement this functionality by adding a button on the IM toolbar "Change Nickname". When the user clicks it, the dialog should popup, ask them for their new nickname, and then perform whatever action should be taken to update the nickname.

The Natural-But-Doesn't-Work Method
The natural way (using synchronous code) to achieve this would be something like the following:

(Javascript code is included above. I'm not sure if this renders via RSS)

But as you can see from the code, this won't work. Why? Dialog is an asynchronous method. It's easy to get comfortable with synchronous dialogs, after all, Javascript has a built in one: input(); However in this case we need to develop a new strategy for working with an asynchronous dialog.

Now lest you be tempted to do something like while(!closed) { // do nothing }; let me just ask you now: stop!. That's fighting the system. That will most likely lock up the browser, and it's an absolute waste of resources.

Using Callbacks to Trigger Behavior
Stepping back from the problem, what we really want is that we have some code, and we want it to run only when the user has finished using the dialog boxes. Being asynchronous, the dialog function actually provides a measure for doing so: Callbacks. When we defined the OK and Cancel buttons, we specified an anonymous function() that would be called whenever that button was clicked. This is a call back. We can use this exact same mechanic to signal our other code when it should run.

But how? Here's some updated code...

(Javascript code is included above. I'm not sure if this renders via RSS)

To accomplish this would could create a "gating" function -- a function which be responsible for dispatching either a call to our dialog, or a call to the code which should be run when we have a value. This is implemented in the example code as getAndSetNewNickname. In addition to the new method we've also added a parameter to the getNewNick method: callback. This parameter will allow us to pass a function that will be called whenever the "Ok" button is clicked. In our implementation, this function happens to be the same function that's calling it. The situation we end up with is a recursive callback which will call our function a 2nd time when it has a value.

The end result? Our setNewNickname method only gets called when we actually have a value. You've successfully gotten user input, and didn't have to restore to timeouts or a weird while loop solution.

Happy jQuery'ing!

Javascript: The Good Parts (the Video)

Kudos to Google and MIT for uploading such great quality stuff to YouTube (if you've been scared off by the drones of webcam-yourself-complaining-about-current-events then give it another try).

I stumbled upon Doug Crockford at google speaking on "Javascript: The Good Parts". He has a book by the same title as well. In his presentation he also mentions JSLint, a tool he wrote to parse Javascript and act as a syntax checker. I'm wondering if it could be rolled into my Django workflow or automated testing.

Here's the video: