Wednesday, December 26, 2007

Go to mysql command shell using rake

Tired of typing mysql -u foo -pbar -h baz_development yet? Add to that when you have database environments galore like test, QA, selenium, staging, etc.

Here's a rake task for you. Just point it to a Rails environment and execute and you are in the corresponding mysql command prompt.


$ rake mysql RAILS_ENV=qa

Loading mysql in RAILS_ENV=qa...
Executing: mysql -u foo -pbar -h localhost baz_qa_test

Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.0.51 Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.


desc "Loads mysql command prompt"
task :mysql do
puts "\n\nLoading mysql in RAILS_ENV=#{RAILS_ENV}...\n"

database_yml_path = "#{RAILS_ROOT}/config/database.yml"
database_yml = YAML.load(

raise "Could not find environment #{RAILS_ENV} in database.yml" unless database_yml[RAILS_ENV]

config = database_yml[RAILS_ENV]
username = config['username']
password = config['password']
database_name = config['database']
host = config['host'] || 'localhost'
port = config['port'] || 3306
socket = config['socket'] || '/tmp/mysql.sock'

raise "Failed. Setup requires a user and database for environment #{environment} in '#{database_yml_path}'.\n\n#{database_yml.to_yaml}" unless username and database_name

args = "-u #{username}"
args << " -p#{password}" unless password.to_s.strip.empty?
args << " -h #{host}" unless host.to_s.strip.empty?
args << " #{database_name}"

command = []
command << "mysql #{args}"

puts <<-EOS
Executing: #{command.join(' ')}\n


system command.join(' ')

Monday, December 17, 2007

Rid of those TextMate "ctags timed out!" error

If you use TextMate for your Ruby on Rails development, you must use the ctags bundle CTags.tmbundle. It gives you a keyboard shortcut CTRL-] to jump to files that defines the class/method your cursor is on. Similar to "go to file and line" in some IDE. Super handy.

Trouble is when your project gets large, it gets longer and longer to update the tmtags index file to a point where it exceeds the default 15 seconds timout and gives a "ctags timed out!" error.

What you can do is edit the file ~/Library/Application\ Support/TextMate/Pristine\ Copy/Bundles/CTags.tmbundle/Support/bin/tmctags and increase the timeout $CtagsTimeout value to something more reasonable. Problem solved.

p.s. CTags suffers from another shortcoming, and that is it cannot index ruby methods named ending with a ? or !. Anyone knows of a fix/workaround? Please comment if you do!

Friday, December 14, 2007

Why create database migrations, when you don't need to?

Rails out of the box supports database migration. It allows Rails programmers to be more Agile, do less database BDUF and instead change the database schema as the requirements change and the business wants it. Writing a migration is also as easy as running a command line script/generate migration and then executing it with rake db:migrate.

Database migration is useful for two reasons. Number one is to allow developers to change the database schema. This can be as simple as a single ALTER TABLE statement to add a new column. Number two, more importantly, is to migrate the all-important data exist in the database. But if your application is still under development for initial release, then migration may not be buying you too much good, because chances are no one cares how your table structure got to where it is now, and you may not have a whole lot of data.

We all use migrations feverishly probably because most of the Rails books/references/tutorials begins with: "Let's start by creating a database migration, in it we insert some data into the newly created tables, and learn how to do XYZ." Thus, your db/migrate folder is stuffed with migrations that does everything: create new tables; alter existing tables; create data for those tables; update data created from previous migrations; and perhaps all of the above, while without justifiable reasons on what those migrations are trying to migrate. In practice, it is quite time-consuming for a developer to run 200+ migrations every time he blows away a database, which is not uncommon. Not only that, sometimes you have multiple migrations that basically cancel out each other's changes as our customers change their minds back and forth. As a result, you could be creating migrations left and right when there may not be any real beneficiaries: a database loaded with data.

Sometimes your QA team may have their own data set that tests your app, and thus their loaded database is a beneficiary. While that is true, I prefer their data to be scripted and be freshly generated by ActiveRecord models (using create!) every time instead of keep migrating them, because as my application domain model expands, I want not just QA data but all datasets to be cleansed and validated by my model validations. There is no guarentee that after running a drop_column migration the QA dataset does not violate any business logic. Keeping all these data valid while database migrations are being rapidly created is very hard.

To take advantage of the fact that your development environment has nothing to lose until you have an initial release, while maximizing the benefits of keeping all your data valid (for all enviroments) the whole time while your app is under development, it's a simple steps 1-2-3:

Step 1:

Have one migration file, 001_release_one_schema.rb, that captures all database object creations. For example, all your create_table, create_index, views, triggers (*yikes*), etc. After this migration, your database should contain all database objects for your Rails app but in a "blank", data-less state.

$ cat db/migrate/001_release_one_schema.rb

class ReleaseOneSchema < ActiveRecord::Migration
def self.up
create_table "foos" do |t|

(... and many others ...)

def self.down
(... ... ...)

Step 2:

Create a rake task to populate all reference data that your application requires to run with. Reference data meaning all data that your application cannot change through its screens, but are essential for your app to run. For example, all currencies that are used to populate a drop-down on your app. A lot of drop-down lists data are reference data.
$ cat lib/tasks/data.rake

namespace :data do
desc "Loads a default dataset of both reference and user data into database."
task :load => [ :environment,
:user_data ]


task :configuration do
ENV['DATASET'] ||= 'slim'

task :reference_data do
require "#{RAILS_ROOT}/db/data/reference_data/#{ENV['DATASET']}"

$ cat db/data/reference_data/slim.rb

@us_dollar = Currency.create! :name => "US Dollar"
@yen = Currency.create! :name => "Yen"
@euro = Currency.create! :name => "EURO"
... ... ...

Step 3:

Create a rake task to populate all user data that your app requires to run with. User data are data that a user can create/update within your application. They are also required for the Rails app to function properly the first day when it launches. For example, for your flashy Paypal application, the fees structure on how it charges its users. An application administrator is allowed to raise or drop fees in your app.

namespace :data do


task :user_data do
require "#{RAILS_ROOT}/db/data/user_data/#{ENV['DATASET']}"

$ cat db/data/user_data/slim.rb

Fee.create! :amount => 1_000, :currency => @us_dollar
Fee.create! :amount => 1_500, :currency => @yen
Fee.create! :amount => 2_500, :currency => @euro
... ... ...

There are several benefits of managing your database schema and data this way:
  • It is easier and faster to re-populate your entire database to the latest schema from scratch with data, since there are no extraneous migrations.

  • Faster to locate and update data needed for application. They are always in your dataset generation scripts.

  • No need to worry about outdated/removed ActiveRecord classes and declare them inside the migration file itself. They is no "legacy" ActiveRecord models.

  • All data are valid all the time because they are created through Model.create! sanitized by your model validations.

  • Easy to specify datasets to load by preference for DEV (slim, loaded), BA (story sign-off), QA (scenario-based), or demo (full) environments. e.g. rake data:load DATASET=slim RAILS_ENV=qa

  • No worries about broken/incomplete migrations. Fewer code, fewer trouble.

Now, after your Rails app goes to a 1.0 production release, you should switch this back to the normal Rails database migration style. I suspect your application users won't be too happy if you blow away their data every time you roll out a minor update or a major release... (or not?)

Tuesday, December 04, 2007

The last "D" in TDD means more than just "Development"

When asked "Do you write tests?", a lot of developers these days will say "of course" as their answers. However, not everyone can admit to doing TDD (Test Driven Development) correctly. Test Driven Development says, a developer will write a test that fails first, then write code to make the test pass, and refactor when possible, and repeat. This is what most people's TDD rhythm is. For the most part this is fairly easy to do. But to reach the next level, one has to understand TDD as a tool: TDD means more than just test your own code. Here is a couple tips on what the last "D" means:


It takes a great deal of discipline to even write a failing test before writing the actual code. Sometimes, we write a little seudo-code here, or move a method definition there, or changing code else where trying to see if more new code needs to be written after it, and sooner than you think you are writing the actual implementation of the methods you wanted to test (Test Afterwards Development anyone?). Other times you write the test, but you are too anxious to even run it and see it fails. And other times you want to jump into the actual code immediately when you see your new test fails, but failing for the unexpected reasons.

Don't fall into these traps. If anything is true, testing is hard, but it is at the same time rewarding and fun. What's also true is, it will pay off. Write the failing test, draw up your list of tests you will need to write, and satisfy them one by one. Having discipline is the cornerstone of becoming a better programmer.


It takes too long to write a test? Tests are running too slowly? Are your tests difficult to read? Are they too brittle and fail all the time? Hang in there! You ever had the feeling you saw code in the codebase that irks the living hell out of your mind written by someone else on your team? Well, it is time for you to get some of these feedback about your own code. Yay, your code sucks! Your tests are telling you that! Let's address each of these one by one.

Slow running tests? You shouldn't be hitting a database or web service in your unit tests, because you can mock/stub them out. Difficult to mock/stub it out? There probably is a better way to design your classes your tests are hitting. Ever heard of Inversion of Control (or Dependency Injection)? Master them. True TDD masters use them extensively.

Unreadable tests? Is it because of too many mocks/stubs? Or is it the code is 500 lines long and doing high octane 720-double-backflip logic? Either way, you have to learn to like small objects. Check this blog post of mine out.

Hard to test something? Tests too brittle? Perhaps you have encapsulation issues in your class design. If your classes are referencing 15 other neighbors, of course they are hard to mock/stub. Chances are, you have to spend time to debug your tests to find out what's wrong! Heard of Law of Demeter? Even if you have, take a look at this highly entertaining yet informative post. It might change your perspective a little.

The bottom line is, TDD is a way to guide you to writing good code, but only if you know how to use it as a tool. Now that you know, hopefully you will have a new perspective next time you write a test.

Sunday, December 02, 2007

Passing multiple parameters in observe_field

Sometimes you just want to pass multiple parameters on your ajax observe_field web request for processing. If you Google around, you will find out there are multiple ways of doing it.

And here is yet another way:

    observe_field :target_dom_id,
:url => some_named_route_url,
:method => :get,
:with => "Form.serializeElements($('text_field_one', 'text_field_two', ...))"

where 'text_field_one' and 'text_field_two' are html form element ids.

I like this because if my text fields are rendered through form_for or fields_for (e.g. form_for :product), and my action does not want to explicitly specify what the posted parameters are upon processing (e.g. Product.create params[:product]), then by using Form.serializeElements I can hide all the params inside the key params[:product] still.