Hi, I'm Ben Oakes and this is my geek blog.
Currently, I'm a Software Developer at Hedgeye.
Previously, I was a Research Assistant in the Early Social Cognition Lab at Yale University and a student at the University of Iowa.
I also organize NewHaven.rb.
I do development with Ruby, JavaScript, SQL, HTML, and CSS. I have an amazing fiancée named Danielle Smith.
We recently had a need to use Heroku with an external MySQL database. Thankfully, there was already a gem that solved the problem. However, it didn’t have that much by the way of documentation, so I wrote this up and contributed it to the author. Also, I added X.509 support, which we required.
NOTE: After my pull requests are merged, you should look to the main repository’s README which has a copy of the below.
Example: MySQL with a CA Certificate
In this example, we are setting up a connection to an external MySQL server using the default initializer.
What we have:
A CA certificate called ca-cert.pem
A database is available at mysql://username:password@server/dbname (where username, password, server, and dbname are the appropriate values for our server)
Other security, e.g. firewalls. Please make sure to open the appropriate ports and grant the necessary access for your database to be available externally. Further discussion is outside the scope of this document, but for accessing EC2, you might start at devcenter.heroku.com/articles/external-services.
First, we configure Heroku with the appropriate environment variables:
# You may have to specify the app name or remote name here via --app or --remote, respectively
heroku config:add EXTERNAL_DATABASE_CA='ca-cert.pem'
heroku config:add EXTERNAL_DATABASE_URL='mysql://username:password@server/dbname'
By default, heroku_external_db looks for the CA cert in config/ca, so we need to commit it:
mkdir -p config/ca
cp path/to/ca-cert.pem config/ca
git add config/ca
git commit -v # Using -v since we want to make sure the contents are what we expect (e.g. not a private key)
Additionally, we need the mysql gem in our Gemfile since we are setting up a MySQL server:
echo "gem 'mysql', '~> 2.8.1'" >> Gemfile
bundle install # Need Gemfile.lock too
Keep in mind that Heroku installs its own database.yml for Rails apps and we have to install pg as well. Unfortunately, shared databases are mandatory (but are free).
$ heroku addons:remove shared-database:5mb
-----> Removing shared-database:5mb from our-app... failed
! Shared databases cannot be removed
PostgreSQL may still be useful to you if, for example, you want to have feature toggles in a local database, but the main data kept externally. However in our case, it also means all developers will need MySQL and PostgreSQL running locally, which is unfortunate.
One workaround is only installing pg in production:
# File: Gemfile
# *Only* needed on production.
group :production do
gem 'pg', '~> 0.11.0' # Regardless of whether you plan to use the database or not, Heroku requires you have 'pg' installed.
end
With our dependencies out of the way, we can move on to testing the connection.
If you are making a new application, you may wish to have a simple MVC for testing that the connection works. E.g., for a blog style application with posts do:
rails generate scaffold post
# NOTE you probably want to change the default "Post.all" to "Post.limit(5)" or something similar
git add .
git commit # ...
# Don't forget to set a default route, etc.
With all these changes committed, we can deploy to our Heroku app:
git push heroku master # Your remote may be different
Now, since we are connecting to an existing database, we don’t need to run any migrations. (Keep in mind that when sharing a database, it is best to have one authoritative source for migrations to live.) If in your situation you’re creating a new database, you may need to do that, run migrations, seed the database, etc at this point.
Open our-app.heroku.com and we should see our data. If you happen to run into a problem, please check the logs first:
heroku logs --tail # Again, you may need to specify an app
If you are having a problem, a good starting point is double checking your passwords, usernames, security settings, etc.
Example: MySQL with X.509
The process is very much the same as the above example, except two extra environment variables and files are required. Below are the extra steps.
What we have:
A CA certificate called ca-cert.pem
A client certificate called client-cert.pem
A client key called client-key.pem
First, we configure Heroku with the appropriate environment variables:
We have a simple application that doesn’t have an ActiveRecord dependency. It’s deployed to Heroku, and it’s been working fine on Rails 3.0.x since April 2011. We knew we weren’t using ActiveRecord for database connectivity, but we let it be, since it wasn’t causing any issues.
When upgrading to Rails 3.1, we found that every single page would give ActiveRecord::ConnectionNotEstablished on our staging environment on Heroku. The same error didn’t happen in development. Although we might have been able to get gem 'pg' set up and working, we really didn’t need an ActiveRecord dependency at all.
That’s what Rails 3.1.0 generates when running rails new myproject --skip-active-record. (Note that require "active_record/railtie" is commented out.) This solved our ActiveRecord::ConnectionNotEstablished problem, but gave us a few others, namely:
Fixtures
ActiveRecord::RecordNotFound (used for 404s)
Fixtures
There’s some normal stuff to get rid of in terms of spec_helper.rb and/or test_helper.rb. Here’s an example:
# File spec/spec_helper.rb
# # If you're not using ActiveRecord, or you'd prefer not to run each of your
# # examples within a transaction, remove the following line or assign false
# # instead of true.
# config.use_transactional_fixtures = true
You may have others. Tarantula had to be adjusted for us, for example.
ActiveRecord::RecordNotFound
ActiveRecord::RecordNotFound, however, was an interesting problem. Everything worked fine without ActiveRecord except the places where we were using ActiveRecord::RecordNotFound to give a HTTP 404 to the user agent. That seems strange in a lot of ways, because a 404 shouldn’t have anything to do with your chosen ORM. My first intuition was to do require 'active_record/errors' (see also the Rails docs), but that caused problems with assumptions in 'rspec/rails'.
Right now, the below is what we ended up with:
# File: config/application.rb
# Pick the frameworks you want:
# require "active_record/railtie"
require "action_controller/railtie"
require "action_mailer/railtie"
require "active_resource/railtie"
require "sprockets/railtie"
require "rails/test_unit/railtie"
# For errors like ActiveRecord::RecordNotFound
require "active_record"
Our automated tests (Rspec, integration tests, Tarantula, Selenium, etc) all pass with it and we no longer get ActiveRecord::ConnectionNotEstablished, but we still have an ActiveRecord dependency I don’t like. (There must be another error we can raise — I don’t entirely like the render '/404.html', status: 404 solution for several reasons.)
According to the Application Helpers guides, you probably want distance_of_time_in_words or time_ago_in_words.
Example:
%p
= time_ago_in_words(my_date)
ago
For reasons I don’t quite understand, this documentation is also incorrect:
time_ago_in_words(2.days.ago) => "2 days ago"
The actual behavior is:
time_ago_in_words(2.days.ago) => "2 days"
I’m planning to move some of this into the padrino-framework project directly. Incorrect documentation is worse than no documentation; maybe it will be more accurate if kept in the code. :)
I ran into a Rails 3.0.1 timezone issue today that I didn’t see discussed many other places.
Basically, I just want to have a page in my app that shows the time in different time zones. That seems simple and something tailor suited for timezone support.
I started with this:
>> Time.now.in_time_zone('EST')
=> Tue, 23 Nov 2010 11:09:42 EST -05:00
Okay, so far so good. Next:
>> Time.now.in_time_zone('PST')
NoMethodError: undefined method `period_for_utc' for nil:NilClass
[...]
>> Time.now.in_time_zone('CST')
NoMethodError: undefined method `period_for_utc' for nil:NilClass
[...]
Wait, that’s odd… why doesn’t that work? After searching, I found you could use some city names like so:
>> Time.now.in_time_zone('Tokyo')
=> Wed, 24 Nov 2010 01:04:54 JST +09:00
But of course JST won’t work:
>> Time.now.in_time_zone('JST')
NoMethodError: undefined method `period_for_utc' for nil:NilClass
[...]
And neither will major American cities:
>> Time.now.in_time_zone('New York')
NoMethodError: undefined method `period_for_utc' for nil:NilClass
[...]
>> Time.now.in_time_zone('Chicago')
NoMethodError: undefined method `period_for_utc' for nil:NilClass
[...]
Nothing too relevant came up when I googled the above errors and phrases (part of why I’m posting this), but then I came across the rake time:zones:us and rake time:zones:all Rake tasks. They list valid timezones for you.
The thing that gets me is that 'EST' and 'Tokyo' work as expected, but 'PST' and 'New York' don’t. These are what I ended up with:
>> Time.now.in_time_zone('Eastern Time (US & Canada)')
=> Tue, 23 Nov 2010 11:08:12 EST -05:00
>> Time.now.in_time_zone('Central Time (US & Canada)')
=> Tue, 23 Nov 2010 10:06:08 CST -06:00
>> Time.now.in_time_zone('Pacific Time (US & Canada)')
=> Tue, 23 Nov 2010 08:06:23 PST -08:00
Ironically, it lists EST, CST, and PST in the results. It’s still confusing to me why the longhand version is the preferred notation here (sometimes), but at least you’re given the tools to look it up.
Like always, let me know if this post helps you through an error. We’re all in this together.
I spent some time over the last few weeks pulling together lots of command line tools that I’ve written over the last few years. I’ve shared them on GitHub.
Some fill in gaps that I wish *nix systems would have by default (such as prune vs uniq or reverse vs rev). Others just script things that I do commonly or are just tedious to do (such as backups2git, github-init, timestamp, latest-migration-path, and std-timestamps). Some are just there for fun (such as is-computer-on). Most of them are written in Ruby, but some are plain old Bash scripts. Lots of the Ruby scripts make heavy use of ARGF, which is awesome for writting shell scripts if you’ve never used it.
I also spent some time documenting (and remembering) how they worked. (Most of it was just shuffling comments around.) Almost every command has a --help option that prints usage information and a short synopsis now. I hope you find them useful!
Require your Ruby code to be more polite: require ‘politeness’
Inspired by INTERCAL, this library makes Ruby programmers say “please” more often. If Ruby doesn’t like how brash or cocky you’re being, it will stop in the middle of your code and tell you so:
"Not polite enough!"
However, you can say please too much. In that case, Ruby will throw its hands up and tell you that you’re:
"Too polite!"
Most libraries are very rude, so be sure to require politeness after including them (unless you intend to make them a little more well-mannered).
I ran into a problem using the HTMLDiff code found in i2 for doing nice diffs in Ruby. Of course, you’ll probably want to use in Rails, but if you are running Rails 2.1 (or maybe even other versions), you’ll want to use an updated version instead.
Update: It looks like this HTMLDiff code is no longer available at the provided links. I think I still have a copy somewhere, let me know if you need it by leaving a comment.
