I recently helped an intern at Hedgeye work through a problem with a database query. Because I’m working in a separate timezone, I ended up making suggestions through a GitHub pull request. We discussed and decided that what I wrote was self-contained enough that I should re-post so it can help others.
Summary: Switching to Heroku Postgres? If you use a Crane or Kappa database instance, you might run into problems with using a non-standard port when connecting to PostgreSQL in production. (The problem may not happen in your staging environment if you use the dev plan.) Our app is a Padrino-based web service, but the problem described could happen to any framework/code that only expects standard port numbers. Check to make sure the non-standard port is handled correctly.
I hope this helps someone else in a similar predicament! Please let me know in the comments.
Note: Much of the below is based off of a written discussion we had with Heroku support. Thanks for your help, guys! (I’ve changed usernames, passwords, etc., for obvious reasons.)
The steps went as expected for our staging application, but NOT for the production one. This app runs on the Bamboo stack. We could access the PostgreSQL database via heroku pg:psql HEROKU_POSTGRESQL_OUR_COLOR as expected, but when we did heroku console, we got the following error:
PGError: could not connect to server: Connection refused
Is the server running on host "ec2-12-34-567-890.compute-1.amazonaws.com" and accepting
TCP/IP connections on port 5432?
That confused us, as we would expect bothpsql and console to fail the same way.
We investigated and noticed that HEROKU_POSTGRESQL_OUR_COLOR had a non-standard port (5602):
However, that did NOT make a difference. We also checked config/database.yml, and it seemed to have correct values.
We were left with a production application that could not connect to its database. As a workaround, we promoted the shared database back to being in use:
heroku pg:promote SHARED_DATABASE
We opened a support ticket with Heroku, and later heard back that there were issues with the Heroku-generated config/database.yml. It did not pick up the non-default port number, which is required for Crane. They updated some code on their end to generate the config/database.yml correctly.
We confirmed that the database.yml was created correctly, but unfortunately that wasn’t the entire problem. Our config/database.rb file (used by the Padrino framework) did not handle the non-standard Postgres port either. After realizing that, it was a simple fix to get the database connection working again; we just had to include the port value in the call to ActiveRecord::Base.establish_connection.
(For those keeping score, there was a total of 2 “non-standard port” bugs happening at the same time.)
Out of curiosity, we asked why non-standard port numbers were used. We were told that they’re only used in the Crane and Kappa plans, so one of those would need to be used in staging to have the same setup. The current implementation of Crane and Kappa use a multi-tenant system, so they listen on multiple ports instead of a single port. (Crane and Kappa are the least expensive production database plans.) We were also told the port numbers could change in the future.
Moral of the story: Make sure your app works without depending on the defaults. That’s good advice in general; it would probably be best to test that you don’t depend on tacit default values for anything (databases or otherwise). One way of doing this would be testing with non-standard values when developing.
Right now, we have some older, mostly-unused machines running a bunch of browsers attached to JSTD. While doing some maintenance today, I had an interesting idea: we need to support the browsers that are popular according to our analytics. That typically means the current and previous version, except for IE, which we support 7.0+ (9.0 is current).
So, we now have Chrome 16, 17 (current), and 18, along with Firefox 9, 10 (current), and 11.
Here’s some sample JSTD output from a Jasmine test suite we have:
Chrome 16.0.912.77 Windows: Run 50 tests (Passed: 50; Fails: 0; Errors 0) (267.00 ms)
Chrome 17.0.963.56 Mac OS: Run 100 tests (Passed: 100; Fails: 0; Errors 0) (318.00 ms)
Chrome 17.0.963.56 Windows: Run 50 tests (Passed: 50; Fails: 0; Errors 0) (196.00 ms)
Chrome 18.0.1025.39 Mac OS: Run 50 tests (Passed: 50; Fails: 0; Errors 0) (209.00 ms)
Firefox 9.0.1 Mac OS: Run 50 tests (Passed: 50; Fails: 0; Errors 0) (681.00 ms)
Firefox 9.0.1 Windows: Run 50 tests (Passed: 49; Fails: 1; Errors 0) (1598.00 ms)
Firefox 10.0 Windows: Run 50 tests (Passed: 50; Fails: 0; Errors 0) (293.00 ms)
Firefox 10.0.2 Mac OS: Run 50 tests (Passed: 50; Fails: 0; Errors 0) (260.00 ms)
Firefox 11.0 Mac OS: Run 50 tests (Passed: 50; Fails: 0; Errors 0) (368.00 ms)
Microsoft Internet Explorer 7.0 Windows: Run 50 tests (Passed: 50; Fails: 0; Errors 0) (2749.00 ms)
Microsoft Internet Explorer 8.0 Windows: Run 50 tests (Passed: 50; Fails: 0; Errors 0) (2078.00 ms)
Microsoft Internet Explorer 9.0 Windows: Run 100 tests (Passed: 100; Fails: 0; Errors 0) (698.00 ms)
Safari 534.52.7 Mac OS: Run 50 tests (Passed: 50; Fails: 0; Errors 0) (1553.00 ms)
Safari 6533.18.5 iPhone OS: Run 50 tests (Passed: 50; Fails: 0; Errors 0) (184.00 ms)