Unicode in Django, It’ll Getcha

Posted in Uncategorized on February 11th, 2010 by Nick – Be the first to comment

I’ve been having an insane time trying to squash a weird unicode bug in Beertraq. This was one of the first bugs that was found after starting the beta and it’s taken me this long to get it worked out. It all started with a cryptic-seeming error upon viewing the add-a-beer page:

UnicodeEncodeError: ‘ascii’ codec can’t encode characters in position 0-9: ordinal not in range(128)

Real descriptive, right? Looking through the stack trace wasn’t much help either. It was a bunch of stuff pertaining to trying to render the template. One thing I did notice is that a brewer with a unicode name had recently gotten added and this error was only happening on the new beer form (which includes a ChoiceField of brewers), but the brewer was being displayed just fine on its own page.

After asking my friend and fellow Django-user Dan-o what he thought, he asked if I had __str__() and __unicode__() defined in my models. I had __str__() defined (generally a good idea) but not __unicode__(). I popped it in to my Brewer model as so:

class Brewer(models.Model):
  TYPE_CHOICES = (
    ('Macrobrewer', 'Macrobrewer'),
    ('Microbrewer', 'Microbrewer/Craft brewer'),
    ('Homebrewer', 'Homebrewer'),
    ('Unknown', 'Unknown'),
  )

  name = models.CharField("Brewer name", max_length=255)
  brewer_type = models.CharField("Type of brewer", max_length=25, choices=TYPE_CHOICES)
  city = models.CharField("Brewer's city", max_length=255, blank=True)
  approved = models.BooleanField(default=False)

  def __str__(self):
    return self.name

  def __unicode__(self):
    return self.name

  class Meta:
    ordering = ['name']

And guess what? My problems went away. I feel like an idiot for being foiled by something so simple that I overlooked. It makes sense too because __str__() is used quite a bit in forms where the model instance’s name needs to be magically generated. Without __unicode__() it would cause Django to barf on unicode names, just like it did on me.

Let this be a lesson to you. If you might EVER have ANY unicode text in your model instance’s names, for crap’s sake define __unicode__()!!

New whiteboard

Posted in Uncategorized on February 9th, 2010 by Nick – Be the first to comment

If you know me personally, you know that I can be pretty scatterbrained from time to time. I’ve desperately needed a whiteboard to keep my thoughts organized, and I’ve finally gotten one. I know, it’s not the most exciting thing in the world to talk about, but it should be a change in the right direction for me.

New Whiteboard

But, wait. What’s this? “Beertraq Road to Stable”?

Beertraq Road to Beta

(click for full-sized version)

Part of why I want to get my thoughts organized is because I want a good view of what all is left before I fully release Beertraq to the public. With this list looming over my head (literally) it will hopefully get my butt in gear to reach a stable release.

The Internet Connection Experiment

Posted in Uncategorized on January 20th, 2010 by Nick – Be the first to comment

Upon moving back to Rolla, I’ve discovered that the local cable company has upped their top tier of internet service from 5 Mbit to 15 Mbit. This excites my greatly because I often find myself downloading large files such as Linux ISOs and streaming high-quality media. Being the curious person I am, I had to figure out how much I could squeeze out of it.

After doing a bit of math, I figured that I could get approximately 158 GB of transfer per day, which is not taking any overhead into account. In order to reach this as a final number, I figured that I needed to saturate my connection with as fast of a download as possible. What better way to do this than with Usenet? Armed with my trusty usenet downloader and a collection of a bunch of Linux ISO NZBs (gotta keep it legit), I set it off on a downloading spree.

Here are the results:

All in all I ended up downloading 110.31 GB in a 24-hour period. As you can see with the above graph, I probably could have gotten closer to the theoretical maximum if not for the various slowdowns that were experienced. I would chalk this up to either my cable company overselling their lines or something causing my usenet connection to slow down.

Given the fact that some of the larger ISPs like to cut you off at a certain point, making their “unlimited” internet connection not so unlimited, this makes me worry a bit. I’ve seen limits under 100 GB, which I would surely blast through within a day or two if I had this speed on that ISP. This makes me feel sorry for all the people who have to suffer with that kind of service provider.

Holiday ham recipe

Posted in Uncategorized on December 25th, 2009 by Nick – 1 Comment

I don’t cook/bake all that often, but when I do I like to have some fun with it. I came up with a ham recipe that turned out well, so I figured I’d share it with everyone.

First, put your ham in a baking pan and cut the diamond pattern into it. Trust Alton Brown when he says that utility knives work well for this. Then, mix the following in a bowl

  • 1 cup brown sugar
  • 1/2 teaspoon ground mustard
  • 4 ounces of fine bourbon. My preference is Maker’s Mark.

Once you have that all mixed up, pack it gently onto the ham. Uniformity is nice, but not required. Then cook the ham at 325 degrees Fahrenheit until the inside meat temperature reaches 150 degrees (about 3 hours with a 20 pound ham). I guarantee that it’ll be tasty when it’s done.

More fun with graphs

Posted in Uncategorized on December 18th, 2009 by Nick – 1 Comment

I figured I’d post the latest graph magic from my work at Nucor-Yamato Steel. This is a graph of the entire network, switches AND hosts!

(click for full-sized version)

(click for full-sized version, warning: LARGE image)

Here, the yellow boxes are center switches/routers, the green boxes are switches, and the peach-colored nodes are hosts. Also, red lines are switch-to-switch connections and blue lines are switch-to-host connections.

Another somewhat off-topic thing about these graphs is that the manager of IT at Nucor-Yamato is interested in open-sourcing the code that manages all of the data and generates these graphs, AND let me use company time to work on and manage the project. If anyone knows of any open-source project (or software that doesn’t cost an arm and a leg) that already does network discovery, data collection, and automated mapping then please let me know! If I’m not going to be re-inventing the wheel, then I’ll probably be kicking the project off shortly after I start working full-time in June 2010.

Beertraq Beta!

Posted in Uncategorized on December 5th, 2009 by Nick – Be the first to comment

I’m glad to announce that my recent pet project, Beertraq, is now in a (somewhat closed) beta stage! The basic idea is there and functioning, but the extra functionality isn’t done and it’s far from polished. Nonetheless, it’s time to take her for a test drive!

So what is Beertraq, you ask? It’s a way for you to keep track of which beers you’ve tasted, compare those with others, read beer reviews, and most importantly discover new beers to try. I originally got the idea from The Flying Saucer’s UFO Club, where members work toward a goal of drinking 200 different beers. Once they complete the task, they get their name on a plate which gets put on the wall of the bar. The cool part about the UFO Club is that it’s all computerized, using a magstripe card to login at a kiosk in the bar. You can also log in to their website to check your progress and read reviews on there. I figured that if The Flying Saucer can have that system for their bar, I could do the same for the world.

If you’re interested in becoming a BeerTraq beta user, send an email to beertraq (at) beertraq (dot) com with the email address you want to use for your account. All I ask is that you give feedback by filling out issue requests with bugs you find or suggestions you might have.

Fun with graphs

Posted in Uncategorized on November 30th, 2009 by Nick – Be the first to comment

I’ve always been utterly fascinated with graph theory, mostly with its applications to networks. As an added bonus, they can be represented with pretty pictures!

Nucor-Yamato Network

(click on image for full-sized version)

That graph represents the network behind Nucor-Yamato Steel and Nucor Castrip Arkansas, sanitized of sensitive information of course. All of the nodes are Cisco switches, the yellow boxes representing backbone switches (6500 series to be exact). This graph is part of the network information system that I’ve been working on during the majority of my internship at NYS and gets auto-generated every day, along with more centralized graphs on a per-switch basis.

The way the system works is that a periodic Python script goes out to a list of known switches and gathers CDP neighbor information as well as the MAC address tables. Then Nmap scans are ran every 6 hours to scan for hosts, gathering IP addresses, hostnames, and MAC addresses. These MAC addresses are correlated with the MAC tables from the switches to determine which hosts are connected to which ports on what switches. The CDP neighbor information also gives which switches are connected to each other, giving a full scope of how the network’s connected.

The script which generates the graphs grabs all of that information out of the database, uses NetworkX and pydot to create the graph, and then graphviz to render it into a PNG image. The graph is pretty plain, though. The real version shows switch names and IP addresses. Since the time between graph generation is so long, any more useful information that I could throw onto the graph would quickly become outdated. My grand scheme is to make a quickly-updated graph showing live stats like switch load, link load, link types (fiber, twisted pair, wireless), downed switches, etc. That way, I (or the network supervisor, I guess…) could have a big-screen TV displaying the live health of the network.

I’ve been asked what parameters I set to get that graph to look that way. I didn’t set anything special in code, it’s all in the command line:

twopi -q -Ksfdp  -Tpng -Goverlap="prism" -Eoverlap="prism" -Gsplines="true" -Gratio="compress"  -oclean.png clean.dot

Really, I’m just a data visualization nerd looking to get a fix.

Doing it right

Posted in Uncategorized on November 26th, 2009 by Nick – Be the first to comment

Well, I finally bit the bullet and got a Linode account. So far I’m pretty happy with it. I figured that with the costs of power and bandwidth, I was almost spending $20/month to run my old server on my own hardware. Incidentally, the lowest-grade Linode VM costs that much and is enough to suit my needs.

So now that I’ve been setting up a webserver from scratch again, I’m doing it right this time. I’m setting up some monitoring software to notify me when things go down, I’m no longer relying on myself for DNS (no more dynamic IPs!), and I’m also branching out and trying an alternative webserver.

The webserver in question is Cherokee which claims to use less memory and perform better than Apache. It sure does use less memory, but as a down side it doesn’t have a native PHP module, so I’m required to use FastCGI for that purpose. Right now, there’s five php-cgi processes running each using about 25-30 MB. This wouldn’t be a problem except that I’ve only got 360 MB of memory to play with. On the plus side its got a pretty sweet admin interface with wizards to help you set up things like Wordpress, Drupal, Ruby on Rails, Django, etc. and you can setup some pretty complex rules for what and how files should be hosted.

On the monitoring side of things, I’m using Munin to monitor the various stats on the server, Piwik for website visit statistics, and I plan on getting Monit going for service monitoring. It’s a bit more important now that I keep and eye on memory and data transfer now that I’m limited on that. Also, if some process goes wild and starts using crazy amounts of CPU power and memory, I’ll be able to catch it.

Unfortunately when you move servers, you have to move everything that was running on them. I’m still in that process, but it’s been going pretty smoothly.

New music, or at least new to me

Posted in Uncategorized on November 23rd, 2009 by Nick – Be the first to comment

I haven’t really been researching new music in quite a while, which is something I miss from being out on co-op and away from my beloved KMNR. I’ve started to turn my laziness around and I’ve found some really awesome stuff which I’d like to share. Keep in mind that I’m probably behind the times a little bit here, but if you haven’t listened to some of this stuff, I HIGHLY recommend it.

Just as a side note: I totally want to make some stepper motors or a tesla coil play Waters of Nazareth by Justice. The song just has that sound to it.

It’s Stout Time!

Posted in Uncategorized on October 21st, 2009 by Nick – Be the first to comment

I got a present on my doorstep today…

IMG_2418-1024

Ok, it’s not really a present but it will certainly taste like Christmas. It’s Austin Homebrew’s Holiday Chocolate Stout kit! I’m branching out a bit after the success of my first brewed beer (not first brew, mind you). Plus, with the soon-to-come cold weather and my recent love of things stout, this is a natural choice.

It’s a shame that I got it now because I’m going to be out of town this weekend. I’ll have to brew this on Monday.