Scalable architectures and Ruby on Rails

July 25th, 2007

Ruby on Rails has been around long enough to get past most of the hype. It is a robust web app framework with a lot to offer. One big drawback seems to be the general lack of performance that many rails apps have. Well, it turns out that there are several things that can be done for that.

First off, go read Charlie Savage’s article on tips to tune your rails app. Charlie discusses profiling your ruby app, and then tracking down the problem areas that are hurting performance. He was able to get his response times down from 7.8 seconds to 0.92 seconds! Wow.

Ok, so what about the case where you’ve tweaked your app, and wrung every last bit of performance you can out of it? But it still isn’t enough. You still cringe everytime your site makes the digg front page. Well, if you have your own servers, you can ensure that you have a scalable architecture. Theo Schlossnagle has a great book on the subject, Scalable Internet Architectures. This book discusses the various issues that a system designer faces when organizing the hardware that a web app will run on. Things like having optimized “static content servers” to stream image assets and css, along with more advanced techniques like setting up a caching proxy for each of your application servers since their CPU time is precious, and should not be wasted shoving content down a slow bandwidth pipe. He also goes into basic database design and advanced logging techniques.

Almost all of the techniques that Schlossnagle talks about are for larger installations, however he also spends time talking about scalability. Particularly the need to be able to scale down as well as up. For example, let’s say you are a fresh new startup, flush with VC cash. You go buy twenty servers, and fill ’em all up with world changing web 2.0 goodness. Six months later realize that the money will run out four weeks before the next version is ready to launch! wouldn’t it be great to be able to consolidate everything onto one or two servers, sell the others, and keep yourself in business a bit longer, until you can get version two out the door? If you designed your software and hardware with both kinds of scalability in mind, you might just stick around long enough to see version three.

So how does all this fit into Railspace? Well, first off, design your application so different bits can be spun out onto separate application servers. Perhaps even allow for logical breaks in the database. Tune your app to get as much bang for each buck as possible. Look at bottlenecks, both at a software, and hardware level. Know the difference between performance and scalability. Know that tuning your software helps with performance, but that scalability is what handles the digg effect. With this knowledge in place, you can be better equipped to handle all the traffic your heart can desire. Ruby on Rails is perfectly capable of handling anything that comes its way, but it does require planning, and work. Building a scalable architecture for Rails to sit on allows all the benefits to shine.

-Joe

Pragmatic approach to programming

February 26th, 2007

I recently had to move my rather extensive library of tech books. In doing so, I marveled at how clumped my library was. Now, in ten+ years of development experience, I’ve worked on a large number of projects. Lots of java, lots of perl, lots of c++, some php, and currently python and ruby. My library does not reflect what I’ve worked on, nor does it really reflect those topics that I worked on the longest. Why do I have four books on ruby on rails, yet only one on python? I use python in my day job, and rails is for my personal projects. There’s lots of documentation out there explaining how to use rails… why four books then? One answer is because I discovered the Pragmatic Programmers. The books from this publisher have been some of the best and most rewarding technical books I have read. The writing style is light, but not overly so. The techniques are described in a logical order. Time isn’t wasted on explaining things over and over. Non-trivial examples are used to boost the subject matter, and display the power of what is being taught.

All in all, the Pragmatic Programmers live up to their name. They teach pragmatic techniques to software development. They teach you what you need to know to get a solid foundation, and develop good habits. From there, you learn enough to get the job done. By the end of the books you may not be an expert, but you have a solid understanding of what is going on, and how to apply it to your projects. If you haven’t discovered these fantastic books, I strongly suggest you put them on your list.

-Joe

Users don’t want to think, and they shouldn’t have to.

February 14th, 2007

Steve Klug, author of Don’t Make Me Thinkamazon.com, has some sage advice for those of us who create user interfaces. When putting ourselves in the users perspective, realize that no matter how nicely we craft something, the user isn’t going to look at our presentation. He will glance at it for cues directing him to where he wants to go. She’ll scan it for relevant information. But no matter how flashy the page is, no matter how much time and effort went into positioning everything, no matter how long we took writing out the content, the user won’t look at it. Don’t get me wrong… content and presentation are extremely important, and should be given plenty of attention. But when the user hits your webpage – or application for that matter – they have a specific task in mind. They probably have a few keywords in mind that they are scanning your page for. As soon as they find a word that could be interpreted as relevant, they’ll click it, or the closest seemingly relevant link. In order to demonstrate this, go over to a casual computer users house, and watch them browse the web, or use email. They will probably drive you crazy… not use any keyboard accelerators, type instead of copy & pasting, click through the menu instead of clicking the toolbar button, etc.

What does this mean to us, the software developers? Should we drop all of the nicely crafted stuff that we work so hard on? Should we limit our pages to one or two links, and a picture? Should we hang it all and go bag groceries? I’d hazard a guess that the answer lies somewhere between the groceries, and no. Specifically I think that the hand crafting is important on pages that are end points. These are the articles, blog posts, help screens, etc. Navigation pages, or task screens should probably have a lot less content on them than they do now. The content is being wasted, in time, pixels, downloads, and everything else.

Don’t Make Me Thinkamazon.com discusses several tips and tricks for handling the dillema, and is worth a read… but the easy and obvious tips are:

  1. Limit the amount of information the user needs to scan
  2. On navigation pages, keep link descriptions to one or two clear, non-gimmicky words.
  3. On content pages, give the user a hand… break the content up, leave plenty of whitespace, and have bold keywords sprinkled throughout the content to help identify potentially relevant areas of information. I.e. don’t force the user to read the entire page – or really, any of it.
  4. Adapt, adapt, adapt. Go watch someone use your app, or website. Don’t have them test it, but have them use it. You may be surprised at how differently they react to the interface. Remember, they are heading over there for a reason, you probably don’t know that reason, but as you learn it, alter your site to accommodate it.
  5. Realize that the user is going to click on the wrong thing, give them a clear way to get back, using the back button, and otherwise.
  6. Read Don’t Make Me Thinkamazon.com, to learn how to get into the users head.

Ultimately we, as developers, need to create something of value and use. Unless we are the ones using it, we have to accommodate the quirks of human nature. Our users time is precious to them, and they will only give you what they feel they have to spare. In order to maximize the value you provide, that time has to be well spent. An intuitive interface is the best way to ensure that. The only way to make an intuitive interface is to know what intuition drives people to do. Given some small changes in how we design applications and websites, we can make life easier for our users, and that can make life easier for us.

Leveraging Mindshare

November 7th, 2006

For quite some time, a good friend of mine has been pestering me to learn the computer language known as haskell. He goes on and on about the virtues of functional programming, and the various nifty things the type system will allow. He thoroughly enjoys using it and, enticed by his enthusiasm, I figured I’d give it a shot. After three weeks, I was as lost as ever. I consider myself a reasonably intelligent human being. I have been writing software for more than ten years, and thought I knew what I was doing. But trying to learn haskell on my own, with only the web, and a few tips from my friend left me really scratching my head. I sort of understand the concepts that the various tutorials are trying to teach, but the syntax of the language is so different that I just can’t seem to see the relation to anything familiar. Upon discussing this with my friend, he disclosed that it took him the better part of a decade to really master haskell, though he could have done it in far less time had he been using it full time. Well, that may be my problem too. Poking at the language for an hour here, and two hours there just isn’t enough for me to really grasp the relationship between haskell and the various software engineering concepts that I’ve used in the course of my professional life. Until then, all the power and beauty of haskell will remain closed to me.

Another experience I had was when our company was looking to get off of cvs as our version control system. Someone decided that TLA, an implementation of the debian arch VCS system, would make a good replacement. It provided all the theoretical solutions to our problems with cvs. Sure the interface was a bit raw, but we could work around that, right? Our needs are interesting. We have several hundred projects, all of which are written by someone else. Most of these projects need to have patches applied, and we need a system to maintain which patches go with which versions of which projects. In cvs land, it was a branch/merge nightmare. In TLA, it was less so. However there are a few projects we have that are written and maintained by us, one of which is my project. I have a small team of three senior engineers. Trying to get us all using TLA instead of cvs was about twice as painful as a trip to the dentist on the day after he ran our of NOX. The base concepts may have been the same, but the entire interface abandoned anything remotely akin to cvs. In fact, this was a deliberate decision by the developers of TLA. They wanted a full fresh start, and threw away all convention. Once again, until we could redesign our development process, the beauty of TLA was closed to us. We just couldn’t apply our old processes to the new system.

Both of these experiences highlighted something very important for me. No matter how cool or revolutionary something is, if a person can’t leverage their existing experiences and skills in using it, then that item is closed to them. It remains useless until such time as they can understand it. Several cases exist where leveraging has occurred. Email, c#, java, cell phones, and countless others. Now this isn’t to say that a revolutionary thing is useless, but it will fail to gain mass adoption if it requires someone to abandon all or most of their previous concepts and start from scratch. Given that, it is important to put yourself in the users mind when looking at something new that you’ve created. Does it have some familiar concept that will allow the user to grasp what it is that you’re doing, or will it appear completely foreign? Mindshare is a valuable thing. It is how we stand on the shoulders of giants. Starting over is expensive.