If you ever need to test HTTP Authentication in your functional tests, here is how you do it:
1 2 3 4 5 6 |
def test_http_auth @request.env['HTTP_AUTHORIZATION'] = ActionController::HttpAuthentication::Basic.encode_credentials("quentin", "password") get :show, :id => @foobar.id assert_response :success end |
This is much like testing SSL.
Hat tip: Philipp Führer for Functional test for HTTP Basic Authentication in Rails 2.
If you ever need to test HTTP Authentication in your functional tests, here is how you do it:
1 2 3 4 5 6 |
def test_http_auth @request.env['HTTP_AUTHORIZATION'] = ActionController::HttpAuthentication::Basic.encode_credentials("quentin", "password") get :show, :id => @foobar.id assert_response :success end |
This is much like testing SSL.
Hat tip: Philipp Führer for Functional test for HTTP Basic Authentication in Rails 2.
I like to be extremely judicious with use of routes. Fewer routes means less memory consumption and fewer confusing magical methods.
I always delete the default route map.connect ':controller/:action/:id' (you should too, otherwise all your pretty RESTful routing is easily circumvented). Since Rails now has the ability to remove unneeded RESTful routes, I’ve been removing those, too.
However, this judiciousness recently painted me into a corner. I have a controller action that I would like to test and it’s wired up like this:
map.logout '/logout', :controller => 'user_sessions', :action => 'destroy', :method => 'delete'
I don’t have this mapped any other way, because why should I?
1 2 3 4 5 6 7 8 |
def test_logout_should_redirect_to_root_path UserSession.create(User.first) delete :destroy assert_match /logged out/, flash[:notice] assert_redirected_to root_path end |
Unfortunately, the test fails with ActionController::RoutingError: No route matches {:action=>"destroy", :controller=>"user_sessions"}! Huh?
The problem is that the delete (and get, post, etc.) method can’t find the route that I created.
Initially, I worked around this using with_routing to define a whole new set of routes just for that test.
1 2 3 4 5 6 7 8 9 10 11 |
with_routing do |set| set.draw do |map| map.resource :user_sessions, :only => [:destroy] map.root :controller => 'foobars', :action => 'index' end delete :destroy assert_match /logged out/, flash[:notice] assert_redirected_to root_path end |
But that was annoying. And after I had more than one route exhibiting this problem, it got really annoying.
Fortunately, I found Sam Ruby’s post Keeping Up With Rails about the challenge of Rails’ minor, quasi-documented API changes. Sam’s post has a bit about how you can add new routes without clearing the existing routes in Rails 2.3.2, which I knew was possible. Following Sam’s link to the commit (there’s no docs for this) showed how to do it.
Now, I’ve added this to test_helper.rb:
1 2 3 4 |
class ActionController::TestCase # add a catch-all route for the tests only. ActionController::Routing::Routes.draw { |map| map.connect ':controller/:action/:id' } end |
The downside to this is that real problems with broken routes may get swept under the rug. You could be more restrictive with the routes you are adding just for tests to overcome that problem.
Update: Thanks to Adam Cigánek in the comments for pointing out my error in why the route didn’t get picked up in the tests. I had the condition hash wrong!
Instead of:
map.logout '/logout', :controller => 'user_sessions', :action => 'destroy', :method => 'delete'
It should be:
map.logout '/logout', :controller => 'user_sessions', :action => 'destroy', :conditions => {:method => :delete}
The first way I had worked correctly when testing manually, but only because without :method, the route responds to all HTTP methods (still no clue why my test didn’t pick it up, though).
Interestingly enough, there’s another gotcha here. Notice that I specified :method => 'delete'. Even when put into the :conditions hash, that doesn’t work. You MUST pass a symbol (:delete) for the HTTP method.
This fixed my problem, but if I ever do need to add routes for tests, now I know how…
I just ran into a tricky gotcha in JavaScript.
I was trying to store some objects in an associative array. Based on my experience with Java, Ruby, and other languages, I expected that given code like this:
1 2 3 4 5 6 7 |
var dictionary = {}; var obj1 = {}; var obj2 = {}; dictionary[obj1] = 'foo' dictionary[obj2] = 'bar' |
The result of dictionary[obj1] would be ‘foo’ and dictionary[obj2] would be ‘bar’.
This is not the case!
The problem is that JavaScript objects are not really hash tables. They’re associative arrays, and the key can only be a String. When you insert an object into a associative array, toString() is called and that is used as the key. Unfortunately, the default toString implementation for JavaScript objects returns “[object Object]”...which is not only very unhelpful when debugging, but doesn’t provide you with a unique key for your associative array.
You can work around this problem by overriding toString. Or you can figure out another way to associate your object with a value. D’oh!
I’ve recently been trying to find a good server automation tool that meets my needs. I looked at Chef and Puppet.
They are both awesome for what they do, but what I don’t like is all the infrastructure I have to maintain to run Chef or Puppet. You need a server to host your server configuration on. But I only have one server![1] Chef does have a solo version which can download configuration from a web server and run it. That’s cool, but I don’t want to have a web server just for putting server configuration on.
When the time commitment to set up one of these tools up greatly exceeds how long it is for me to bring up a new slice and run through the standard Apache/DB/Passenger stack, I lose interest. In the end, these are great tools for managing a cluster of machines and bringing up a new app in the cluster quickly—and keeping it up to date automatically. If you have big infrastructure needs, they make sense. If you just want to set up a single slice…ugh.
After reading a bit about how Puppet and Chef work, what I really wanted was the ability to push server provisioning recipes. I want to maintain the server config in my repository and then provision a new server with a command I run on my machine. Sort of like deprec, but understandable.
Fortunately, I found Sprinkle and passenger-stack.
Sprinkle lets me quickly define which packages I want installed and push it out to a server to run (via Capistrano, Vlad, or Net::SSH). Sprinkle makes it easy to install software using apt, gem, or source. And unlike a simple shell script, Sprinkle tests whether or not the software is installed before running, and has a concept of dependencies.
Passenger-stack removes the pain of writing my own rules for what to install. It comes with the standard stuff you’d need, and you can customize it from there.
Here’s how you install all the software you need for a fresh server, after downloading passenger-stack:
sprinkle -c -s config/install.rb
The best part is that you can run that command again, and it won’t do anything. So you can add new software to your stack, then run it against your server, and only the new software will get installed.
This gives you a great way to manage natively compiled gems and ensure that if you ever need to spin up a staging server or a demo server, everything you need gets installed.
Check out this screencast by Ben Schwartz, author of passenger-stack.
<object height="360" width="640"><param /><param /><param /><embed src="http://vimeo.com/moogaloop.swf?clip_id=2888665&server=vimeo.com&show_title=1&show_byline=1&show_portrait=0&color=00ADEF&fullscreen=1" height="360" width="640"></embed></object>Passenger-stack demo from Ben Schwarz on Vimeo.
It’s not a smart as Chef and Puppet. It’s not transactional and servers don’t check for new software to install automatically. But it sure is easy. That’s why I call Sprinkle “the provisioning tool for people who don’t have huge server clusters.”
1 Basically. I have many servers with many different applications on them. And I have a few servers that have multiple environments, but the same software. That’s my big driver for wanting a provisioning tool.
Estimating software is hard, but most of us have to do it – whether we’re estimating an entire project for a client, or a new feature for a boss, or a change to one of our own projects.
I’ve found the following rule helpful when estimating software. This comes from about four years of estimating Rails projects to consulting clients, and moving from bad – dramatically underestimating fixed-bid projects – to pretty good – usually overestimating time & materials projects slightly. (And more importantly, knowing when I can’t estimate, because the scope is too vague or too large.)
Software difficulty is primarily determined by volume, logic, and integration.
1. Volume is easy to understand. If you’re building software that does more, it will require more work. So if you’re estimating a project that stores recipes, and you’re estimating another project that stores recipes AND shopping lists, you can expect that the second one will take more work (if everything else is equal).
2. Logic refers to the rules or business logic behind a feature. The more rules there are, the more work there is. Imagine that our recipe system requires that recipes from some users are manually approved by an administrator, and checks to see that each ingredient in the recipe is present in the step-by-step instructions, and only allows a user to post 3 recipes per hour, and lets users propose alternative versions of a recipe, and lets an alternative version replace the regular version if it achieves a certain rating, etc. That’s more work than a recipe system that just lets users create and rate recipes, even though the volume of features may not be any larger.
Interestingly, a technology can make some logic trivial and some logic hard. Nested forms are a great example of this. Before Rails 2.3, Rails made it trivial to do CRUD on a single table at a time, but difficult handle multiple tables. Now it is (almost) trivial to do CRUD on multiple tables at a time.
3. Integration points are usually deserving of special consideration in an estimate. This includes talking to a web services API, another local software system, a data feed, a complex library, etc. Not only do integration points often take time to get right, but they can become sinkholes of time when the documentation is inadequate or incorrect, the other system doesn’t play nice, or you can’t easily test the integration. And your estimate depends on something out of your control: the other system.
These rules only apply to the difficulty of the software. Several external factors are important as well. These include, most notably, the client and the team. The client can make a project easy, or they can make a project difficult. Similarly, the right team might be able to blaze through a project quickly, while the wrong team may never finish at all.
Here’s the thing about these rules: they’re relative, not absolute. There is no rule that says “Features take 5 days, and integration points take 10”. So estimating requires comparisons. This means that if you’ve never built a Rails app before, you’ll have trouble estimating a Rails project. But once you’ve built a few, you can compare the volume, logic, and integration points of a new project to volume, logic, and integration points of the previous ones.
So estimating requires intuition and experience as well as analysis (e.g. Jon’s Law of Estimates). The key to estimating is to combine analysis and intuition, and to let each side refine the other.
I’m pleased to announce VeloTweets, the pulse of the peloton, a curated collection of professional cycling Twitter activity. The idea and driving force came from Jamie Thingelstad. I did most of the development, and Norm Orstad designed the site. Chris Hatch helped a lot on the back end, providing a list of cyclists on Twitter, filling out profiles and affiliations, and doing research.
We wanted to make VeloTweets different than the other subject matter aggregators out there. We wanted a hook that would combine the immediacy of Twitter with pro cycling in a compelling way.
Here’s what we came up with.
First, we focused on who to include. Instead of everyone who’s talking about cycling, this contains only pro cyclists (and a few others associated with the sport, like managers or team mechanics).
Second, we extended the data that is given to us by Twitter. We can enter every cyclist’s real name, nationality, and team, as well as expanded biographical data (here’s Lance Armstrong’s profile for instance).
Third, we collected cycling events in a calendar that’s displayed on the site, and added a Message of the Day that’s tuned to what’s happening in the racing world each day.
Forth, we brought in photos from the tweets (only TwitPic is supported right now). We store references to the photos in our DB so we can show the latest photos, along with photos that individuals have posted, and all of them. This turns out to be really cool because where else are you going to see photos like this one as they happen?
After all this we still weren’t totally satisfied with what we’d come up with, because it still looked too much like Twitter (long list of messages in reverse chronological order). Then Jamie came up with the idea of only displaying each cyclist’s most recent tweet in a grid. We really like how this works because people who tweet a lot (like Lance) don’t dominate the page. It gives you an overview of what the whole peloton is talking about without letting a few people dominate it.
This application uses Rails 2.3, the Suspenders base app, make_resourceful, semantic_form_builder and the excellent HTTPClient library for interacting with Twitter (give up on net/http – it is full of fail).
Twitter API access is done directly with JSON. We pull the friends_timeline and insert those tweets into the database.
I’ve been doing a number of Twitter-related projects lately. The first was Twistr, which combines Twitter and Flickr LOLcat style for occasionally amusing results. Then Barry Hess and I built Follow Cost, which tells you how much someone tweets before you follow them. I created a prototype for FanChatter’s next product based on Twitter conversation aggregation. Now comes VeloTweets and another project that’s not public yet.
I really enjoy working with the Twitter APIs. It’s fun to develop applications that utilize the platform that the Twitter folks have built.
On that front, I recently received a copy of Twitter API: Up and Running (Follow Cost is mentioned on page 70!) which I will give a full review to soon. You don’t need a book on the Twitter API to develop applications for it, but it does provide some ideas and a useful reference, as well as details on some interesting aspects of Twitter (for example, I did not know that direct messages disappear if they are deleted by either party.).
Apologies for the 3 week gap in "weekly" posts. I was taking a vacation in Hawaii (pics!) and took a bit of time to enjoy life offline :)
Geocities was once called Beverly Hills Internet. The company was founded in 1994 but it wasn’t until mid-1995 that they publically offered what people now think of as a Geocities trademark: free webpages, or “homesteads”. [An article about the Archive Team trying to save Geocities content before Yahoo takes it down.]
How the OAuth Security Battle Was Won, Open Web Style - ReadWriteWeb
At some point in conversation Hammer-Lahav realized that the problem went far beyond the Twitter implementation. The OAuth protocol had an inherent vulnerability; big companies like Google, Netflix and Yahoo had implemented OAuth and scores of tiny startups had too... OAuth has support, but it doesn't have a centralized authority ready to deal with problems like this. Over the next week a story unfolded as the community moved to deal with the security issue. It's a dramatic story.
Tell me your best worst joke, Reddit.
[Includes such classics as: What's brown and sticky? A stick. --- Why does Snoop carry around an umbrella? Fo Drizzle. --- and, my personal favorite: Two snares and a cymbal fall off a cliff.]
We're sick and tired of hack developers ripping off naive clients. And while I'm completely disgusted by some of the horror-stories I've heard lately, clients keep asking the wrong questions. As real developers, it's our responsibility to make the tough decision to speak the truth. This is an example of what we call the anti-pitch. [Excellent. I'm using this technique next time I'm dealing with potential clients.]
What Twitter Looks Like For Twitter Employees
...hackers sent them screenshots from the site Twitter employees use to manage the microblogging service, admin.twitter.com... [It's amazing to see all of the back-end stuff necessary to run something so "simple" as Twitter.]
Honeypot filter as a Rack middleware
Our site’s suggestion box got hammered by a spambot recently, so I created this simple Rack middleware to protect our app from any requests that include a honeypot field.
Rails Edge: Implement FooController.action(:name)
Rails actions are now Rack endpoints, and can be retrieved via FooController.action(name) and called with an env.
Make your site faster and cheaper to operate in one easy step
Is your web server using using gzip encoding? Surprisingly, many are not. I just wrote a little script to fetch the 30 external links off news.yc and check if they are using gzip encoding. Only 18 were, which means that the other 12 sites are needlessly slow, and also wasting money on bandwidth.
Passenger: Command line done right
What’s really great about Passenger is that the attention to detail doesn’t end at the installer. The Linux process list is a list of programs that are currently running. Usually, programs are shown in this list by their command line name, often an indecipherable mix of letters and numbers. Passenger processes are easy to spot and easy to understand. Human readable names in a machine-centred interface.
Muxtape’s stock parts are highly regimented, allowing bands to express themselves with freedom, though not completely freely. Every component is 300 pixels square, and there is virtually zero layout flexibility; you can have whatever arrangement you like, so long as it comes in rows of three. What’s more, for now there are no ‘social’ components to draw upon; no commenting, no friending, no favoriting, etc. The new Muxtape platform is nothing if not regimented.
Tweetie is a desktop version of an application of the same name for the iPhone which, in my limited experience, is the first time an application has migrated from the phone to the desktop. As a friend mentioned, “Platform merge in progress!” and he’s right... When I use Tweetie, I’m reminded that a maniacal attention to detail not only makes you want to reach out and touch the digitally untouchable, it describes the familiar as the new, and, most importantly, it speaks of an aspirational future.
Gem Command to easily open a ruby gem with the editor of your choice. [Awesome. See the Issues tab for detail, but you need to set GEM_OPEN_EDITOR to 'mate' in your bash profile despite what the instructions might say.]
You can download the free version, which is ad-supported, and try it out for as long as you want. [The only Twitter client I've been able to use, aside from Tweetie on the iPhone.]
The first step to faster tests is knowing what is slow. Fortunately, this is dead simple with the test_benchmark plugin by Tim Connor, and originally built by Geoffrey Groschenbach. Install the plugin, and when you run your tests via Rake, you’ll see handy output showing you the slowest tests, and the slowest test classes.
Twitter Clients Are a UI Design Playground
But perhaps the most important factor that has made Twitter such a rich category for client software is that there is so little friction to switch between apps. There’s nothing to import or export, and zero commitment.
Venture Capital Down 50%. It’s Not Just the Recession, Folks.
There’s a huge difference between what venture capitalists say and what they do. [VC] fell off a cliff in 2001 and 2002 and it’s falling off a cliff now.
I can’t reveal details without breaking confidences, but suffice it to say that a significant number of Rails core contributors - with leadership (if that’s the right word) from DHH - apparently feel that being unwelcoming and “edgy” is not just acceptable, but laudable. The difference between their opinions and mine is so severe that I cannot in good conscience remain a public spokesman for Rails. So, effective immediately, I’m resigning my position with the Rails Activists. [I haven't gotten up to speed with the controversy around this issue, but I can say for certain that Mike Gunderloy stepping back from his participation in the Rails community is a real serious bummer.]
We have over 25,000 apps running on the platform today, and many of our users have been asking for pricing and paid services for some time now. So today we’re pleased to announce that we are officially out of beta and available for commercial use.
It’s been 3 months in the making and 3 months of waiting for Apple approval, but wait no more… ShakeItPhoto is ready for download at the iTunes App store for the low price of 99 cents. Take a photo and shake it like a polaroid to make it develop!
It gives us great pleasure to announce our integrated issue tracking system! On repository pages you’ll now see an “Issues” tab in the top menu.
Phusion Passenger 2.2.0 w/ Nginx support
After spending weeks on further development and intensive testing, we’ve now come to the point wherein we have the distinct honor to announce Phusion Passenger for Nginx as an addition to the Phusion Passenger server line-up.. Our thanks goes out to Engine Yard for financially sponsoring this first release of Phusion Passenger for Nginx, as well as all the people who have in some way donated in the past for making this release possible in the first place.
Is Open Source Experience Overrated?
Just as commercial software can't possibly exist without customers, perhaps open source experience is only valid if you work on a project that attains some moderate level of critical mass and user base. Remember, shipping isn't enough. Open source or not, if you aren't building software that someone finds useful, if you aren't convincing at least a small audience of programmers that your project is worthwhile enough to join... then what are you really doing?
With the latest stable release of rails out the door for about a month, we’ve had a chance to upgrade the bulk of the applications we maintain to 2.3.2.1. Here are some “gotchas”, aka issues, aka roadblocks to Strategic Enterprise Adoption that we discovered while upgrading some of them.
Draft: The problem with Project Management tools
While I agree that it’s important to release code, I think pivotal and other similar tools lead to a mindset where releasing code is in itself the unit progress. But, as any successful team will tell you, completed tickets and releases released are horrible units of progress, since unless your customers love every single thing you do (they don’t), your unit of measurement becomes the amount of features and changes deployed.
Clone TinyURL in 40 lines of Ruby code
I wrote Snip with Sinatra then deployed it up to Heroku so this is also a good excuse also to describe Heroku, a truly amazing service for the Ruby programming community. The total number of lines in Snip is actually 43, in a single file named snip.rb. including the view template and layout. [It's amazing what you can accomplish with Sinatra and Heroku.]
In my experience, a typical production Rails app on Ruby 1.8 can recover 20% to 40% of user CPU by applying Stefan Kaes's Railsbench GC patch to the Ruby binary, and using the following environment variables...
Customer driven iteration vs Whiteboard driven iteration
Customer driven iteration takes customer validation rather than released features as its core unit of progress. It assumes that you have not accomplished anything and therefore cannot feel good until your metrics tell you that your market will use and pay for your stuff.
Can the Statusphere Save Journalism?
...the discussion shifted to deep conversation about the future of journalism in the era of socialized media with one simple question, “are newspapers worth saving?” Walt thought for no more than two seconds and assertively replied, “It’s the wrong question to ask. The real question we should ask is if whether or not we can save good journalism.”
Are Blogs Losing Their Authority To The Statusphere?
Attention is engaged at the point of introduction, and for many of us, we’re presented with worthwhile content outside of our RSS readers or favorite bookmarks. Relevant and noteworthy updates are now curated by our peers and trusted or respected contacts in disparate communities that change based on our daily click paths... Retweets (RT) and favorites in Twitter, Likes and comments in FriendFeed and Facebook, posting shortened links that connect friends and followers back to the source post, have changed our behavior and empowered our role in defining the evolution of the connectivity and dissemination of information.
ActiveRecord plugin for allowing (careful) mass assignment of protected attributes, separate from values provided via users of your application.
Sébastien Wains » Howto : setting up dns2tcp
For the "I can't browse from work" crowd or the "stuck behind the Great Firewall of China" set, there are any number of high-visibility, high-availability solutions: tor, your buddy's apache proxy, etc. For those who want to try an obscurity/security/proxy solution that's a little closer to the metal, there's dns2tcp via ssh which, predictably, sends your encrypted traffic from your computer out of your network as a dns request and returns it the same way: you're secure going out and you're not sending up big, "hey everybody: look at my port 80 requests!" red flags to the secret police or the sysadmin or whomever. Cool stuff.
This is a pretty good read: it's got a little too much depth to be considered a crash course, but it's too abstract to be a tutorial or how-to. A nice, mid-level view of best security practices.
Twitter + Stimulus = Conservative Stupidity
Normally I wouldn't bookmark DailyKos--that would be kind of like bookmarking HuffPo or Reddit--but this is a neat little article about social engineering / industrial espionage that involves exploiting confirmation bias among partisans. Short read. Good read.
Lifehacker - Should Comic Sans Be "Banned"? - Fonts
This made me laugh out loud. It may make you laugh out loud as well.
Supposedly this is the best online format converter. Handy in a pinch (or if you're tired of your CLI converters screwing the pooch on higher ascii and spitting out comic book character swears in place of kanji).
Testing mail servers with swaks
At first glance, this looks like a "for Dummies" tutorial for a piece of software that is, essentially, "telnet for Dummes". But swak lets you do something that you can't (easily) do with plain, old-fashioned telnet. You can, for instance, set a timeout time, specify authentication types, etc. with a commandline flag or two. Handy if you're troubleshooting that new mail server install or doing some eyeball/ball park benchmarking.
Introduction to Quality Assurance and Metrics
If you're looking for a no-bullshit crash course in QA/QC that has decent depth, look no further.
Palm vein biometric authentication? Seriously? I mean, I guess super-futuristic biometric auth devices that scan _inside_ the body for unique identifiers are kind of cool in an aesthetic sense, but they're certainly not very cool from a security sense: I thought we had agreed as a global society that physical objects, no matter how apparently unique they are, are unsuitable for secure auth because they are, at the end of the day, still just objects. And all objects can be replicated.
Skimmers: Reader Finds Card Skimmer On Bank ATM
First reaction: "wow that's totally awesome--I can't believe someone came up with this." Two seconds later's reaction: "wow, my opinion of the human race just got ratcheted down a peg or two: I can't believe it took us this long to invent the ATM card data skimmer."
The peasant mentality lives on in America
You know, three weeks ago, I had no idea who Matt Taibbi was. Then, courtesy of reddit, I got put on to his write-up of the Meltdown and I've been hooked. This guy hits hard, doesn't pull punches and walks the stylistic tightrope between the unnaturally polite tenor of expose journalism and the warbling catachresis of incendiary blogging.
What happens if I don't pay my taxes?
This is a good article because a.) it's timely and b.) is written from a hacker perspective/mentality. It starts with the question, "what is the nature of the system?" and then wonders about different methods of potentially short-circuiting it or circumventing aspects of it. Kind of makes taxes fun. Almost.
What to do when the root partition is full?
This is a good list of comments to scroll through as it discusses Linux mounting tricks, how to use LVM and, basically, lists reasons why not to panic. And, I don' t know about you, but the fewer reasons I have to panic, the better.
Thanko's Latest 4GB Necktie Camera
Yeah, it's basically just a flat camera and a necktie that's been cut open in the back, but the idea is still totally effinf awesome.
A Short Introduction To Cron Jobs
There are two reasons that introductory level, "how to" type documents for the basics of Linux administration are so ubiquitous: those reasons are that they're useful for experienced users to a.) write and b.) comment upon and they're useful for inexperienced users looking things up. This one is about cron and using crontab. And it's a great example of that.
I’ll be speaking at RailsConf 2009 this year on music and software development (Five musical patterns for programmers). The basic premise is that software development and music actually have quite a bit in common. This may be surprising to some people, who see programming as a cold, rational left-brain sort of thing, like science. But we programmers know that this is not really the case at all.
So as a prelude to my talk, I decided to interview two programmer-musicians on the subject: Chad Fowler and Dave Thomas. Both compose and perform music, and both are noted programmers. Here is the interview.
Rail Spikes: Tell us a little about your background with both programming and music.
Chad Fowler: I started my professional life as a saxophonist in Memphis. I played the Beale street clubs and all the typical Memphis professional musician stuff. Among others, I played for a while with Ann Peebles and her husband Don Bryant with the rhythm section from all the old Hi Records recordings. I did mostly R&B and jazz professionally but I was probably most well known in the Memphis community for making “strange” music. Before playing music professionally, I played guitar in punk bands in high school. I was a fan of punk, heavy metal, hip hop, pop, (new) classical and pretty much everything else. As I immersed myself in the world of jazz, it became quickly clear that the jazz community doesn’t like punk and other less “serious” types of music and has an almost religious negative reaction to jazz musicians who do.
It was almost as if any deviation from the “normal” world of jazz made you a traitor. So I did the natural thing: started a group called The Jazz Traitors, which played music that 1) we loved and 2) offended the jazz community (not necessarily in that order).
I was also very interested in composing “classical” music. I studied with a composer named Kamran Ince, who is still my favorite such composer.
As for programming, I’ve been interested in programming since I was a young child using my commodore 64. I wasn’t really that good at it as a kid but I played around a lot. I didn’t get serious until I picked up programming again as a hobby while I was a professional musician. After a late night gig at a bar, it was relaxing to go home and unwind to some C programming tutorials. I didn’t have a need to program, nor did I have a project in mind (except that I have always loved video games and wanted to learn how they worked). But I got so into it, that I ended up getting a job in computer support because a friend filled out an application for me.
Being the gamer I am, as soon as I started in computer support, I naturally wanted to “level up”. That meant becoming a network administrator. Then a system administrator. Then a programmer, then a designer, then an architect, then a CTO, etc. Now here I am. It’s been fun.
Dave Thomas: There was always a lot of music in our house. My father liked to play the piano and the organ (I learned to solder as he built a Heathkit organ from a kit in the late 60s). My mother liked Broadway musicals. So we’d often experience alternating hours of Chopin and South Pacific. My brother was also musical. I wasn’t particularly, but I enjoyed noodling on the piano, and spent hours just playing with chords and progressions.
I’ve been programming since I was 15 or so.
Rail Spikes: Some developers – yourself included – have suggested a similarity between programming and music composition or performance. How exactly are music and programming similar?
Dave Thomas: I’m not sure, but I think it might be something to do with the discovery of patterns. Both music and code consist of nested sets of variations and repetitions. There’s a rythm to executing code, in the same way there’s a rythm to music. It is never exact, but it’s there. After a while, I found I could imagine the rythm and structure of my programs as they run, in the same way you can pick apart the structure of a piece of music as you listen to it. And, jsut as with music, it takes experience to be able to feel the deeper structures and notice the more extreme variations. But being able to spot them in programs makes coding simpler and more interesting. The basic coding structures—loops, method calls, and so on—provide the framework for composing in the same way that staff and bar lines do for music. Algorithms are like the progressions, and data becomes the notes. And in the same way that good music takes all these things and then surprises you, good code does the same thing. It isn’t mechanical and repetitive: instead it uses the constraints to build something bigger and more interesting.
Chad Fowler: It’s hard for me to put my finger on. There’s something similar in the way I think when I do each.
I think it all boils down to language, though. In all of these cases (including learning actual language), you take a bunch of tokens (notes, sounds, grunts, functions, classes) and combine them into a grammar which you use to express ideas. The way you do that is totally up to you as long as the intended ideas are communicated. With computer programs, they have to do what they’re meant to do. With music, they express or evoke emotions, paint pictures, cause anxiety or whatever.
Some computer programs evoke emotions and cause anxiety as well.
Rail Spikes: Is Ruby development more like improvised jazz or composed classical music?
Chad Fowler: I think it’s both. And I don’t think Ruby is any different in this than other languages. Much of the discussion about the relationship between programming and music focuses on the more obvious idea of programming as composition. It makes sense, since programmers tend to sit and type their ideas into an editor and then eventually execute it. The programs can be checked, tested, refactored, etc. before the actual performance. This is how classical composition works as well.
But the less obvious angle is that in many situations, programming is like performance. In fact, even in music, improvisation is really just real time composition. You don’t get a chance to refactor because your “code” is executed as you write it.
I’ve had this same feeling while debugging production problems, hacking new features on a tight deadline, or sometimes during the initial creation of an application. The same synapses are firing as when I was trying to play Cherokee at 200 beats per minute. Mistakes can’t be erased, so they have to be nuanced into (worst case) insignificant events or (best case) important drivers behind the work.
From a purely development-oriented perspective, TDD is more like improvisation than composition. I think that’s what I like about it. It’s motivating and creative in an exciting, time-sensitive way. You take small steps and see where they lead you. Sure, you can always revert your changes if you paint yourself into a corner but part of the fun and challenge is to not paint yourself into a corner.
One thing jazz musicians like to say is that every wrong note is just a half step away from a right note. TDD is like that. You might take a slightly wrong turn. It’s fun to see if you can course-correct without starting over.
Rail Spikes: Do developers need to be musically inclined? Does it help?
Chad Fowler: Obviously not. Some of the best programmers I know are not musicians. I can’t tell if it helps, but I would guess that developers who are also musicians are different than developers who aren’t. I don’t think that’s because being a musician changes people, though. I think it’s because the people who are both are the kind of people who need to do both.
This usually means they’re “right brain” people. This leads to a way of thinking that changes how they approach programming problems.
I think learning music (or another right brain discipline) is a good way to exercise your mind. So I wouldn’t be surprised if leaning music helps people exercise their thought processes in ways that will benefit their work as programmers (or authors, or lawyers, or doctors or whatever).
I also think, though, that if we were all musicians at heart, we wouldn’t get much done. I rely heavily on my less artsy colleagues to ground me and be sometimes more pragmatic than I am. So I don’t think we all need to be a “right brain” programmer. It would be disasterous if we were.
Dave Thomas: Do they need to be? No. But many of the good ones I know are. I’d guess that density of musicians in software development is many times the population norm. But that means you could also ask the question “Do musicians have to know software development?”
I think the more interesting question is to ask “how can people best express what they enjoy doing?” because both music and software development are outlets for this.
Rail Spikes: What sort of music do you listen to? Any recommendations for Ruby developers looking to expand their musical horizons?
Chad Fowler: As I mentioned earlier, I like all kinds of music (with a few exceptions). Lately I’ve been listening to a lot of instrumental hip hop, such as DJ Qbert and Mixmaster Mike. I’ve also been getting into a genre of electronic music called “electro”, which sounds like the bleeps and bloops that are the soundtrack of my dreams (if a computer is going to generate music I always like it to sound like a computer generated it).
As for recommendations, here are a few ideas for things that most developers probably haven’t listened to:
Dave Thomas: I listen to just about anything that’s interesting. My playlist here is very varied, and I try to add new stuff to it farily regularly. I know people who are trained as musicians, and I tend to ask them what they’re listening to. Sometimes that leads to challenges: my ear isn’t as developed as their ears. But often it leads to whole new areas of cool stuff. So I’d recommend everyone should find a friend who knows more than you do about music and ask them to surprise and challenge you. (That advice probably applies to just about everything, thinking about it.) It’s easy to find music that stimulates your lizard brain. Get into the habit of looking for the stuff that engages at a higher level too. And, like everything, have fun with it.
When troubleshooting a nasty bug, it’s often useful to take a look actual production or staging data, or even pull it down into your development database. But this is a huge potential privacy and security concern. Your local environment likely isn’t as secure as your production environment, and you might not want to access this sensitive data (or give it to another team member).
Similarly, you might want to replicate your production data on a staging or QA environment to see how new code will interact with real data. Also a privacy concern.
Simple solution: anonymize the data!
In my current project, I put together an anonymize.rake task to deal with this. The most sensitive data in our app is name and phone number. Without that, private information can’t really be linked back to someone. So I pulled the 200 most common first names and 1000 most common last names (in the United States) and put them into an Anonymizer class. Call Anonymizer.random_name for a random, but realistic, name. The class also includes a simple phone number and email anonymizer.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
class Anonymizer def self.random_name "#{random_first_name} #{random_last_name}" end def self.random_first_name FIRSTNAMES[rand(FIRSTNAMES.size)] end def self.random_last_name LASTNAMES[rand(LASTNAMES.size)] end def self.random_phone "612-555-#{rand(8000) + 1000}" end FIRSTNAMES = %w(James John Robert Michael # etc. |
The rake task is simple:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
namespace :db do namespace :data do desc "Anonymize sensitive information" task :anonymize => :environment do if RAILS_ENV == 'production' puts "Refusing to anonymize production data. You don't really want to do that." else puts "Anonymizing all name and email records in the #{RAILS_ENV} database." # User.find(:all).each do |user| # user.name = Anonymizer.random_name # user.email = Anonymizer.random_email(user.name) # puts "Saving #{user.name} (#{user.email})" # user.save! # end end end end end |
You’ll need to do the actual implementation yourself (see the sample User.all.each {} block). It would be easy enough to extend this to work with social security numbers, addresses, etc. Run with:
rake db:data:anonymize
Code: anonymize.rake