Bundle install failing on native extensions.. but gem install works?

I had a terrible time trying installing gems with bundler into a new staging environment yesterday (a few EC2 micro instances). My issue was that certain gems with native extensions like nokigiri, or bcrypt-ruby would fail. I was getting fairly ambiguous errors like this;

Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.

    /home/ubuntu/.rbenv/versions/2.0.0-p247/bin/ruby extconf.rb

Gem files will remain installed in /home/ubuntu/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/nokogiri-1.5.11 for inspection.
Results logged to /home/ubuntu/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/nokogiri-1.5.11/ext/nokogiri/gem_make.out

An error occurred while installing nokogiri (1.5.11), and Bundler cannot
Make sure that `gem install nokogiri -v '1.5.11'` succeeds before bundling.

It instructs us to go ahead and try to gem install the gems directly. Interestingly, that would work. I could install nokigiri without a problem via gem install... It only failed during bundle install.

Another thing to note is that two of these servers were twins. By that I mean they were spawned from the same AMI and have all required packages installed (libxml, libsqlite, etc). One of them was working, and the other was failing on bundle install.

This was kind of baffling me, but then I realized there was one difference between them. The working one is an app server running nginx. The broken one is a worker instance without nginx, but does provide an elasticsearch server.

The elasticsearch instance by default, I noticed, does take up a lot of the EC2 micro instances memory.

13466 elastics  20   0 1915m 157m  12m S  0.3 26.7   0:06.32 java

I tried shutting down the search server and ran bundle install again. All the gems were installed successfully.

Without looking into it further, I've reasoned that the bundler ruby process, plus the native gem extensions being compiled is just too much for this little server whilst running an in-memory store. However, regular old gem install is OK.

The takeaway: if you have bundler failing for no apparent reason... Make sure you have enough resources on the machine. Specifically shut down memory intensive processes like in memory stores.

Speed up RSpec. Ditch bcrypt.

At Clambake, we have an RSpec test suite of about 650 cases. The runtime for the whole suite has grown to about 4 minutes. The sheer amount of time spent waiting for them to finish was discouraging me from writing new tests.

I did some googling around, and discovered a fairly obvious reason why the tests are so slow: bcrypt

For every user factory generated in the tests (via factory_girl) the password is salted and hashed using bcrypt. The bcrypt algorithm is purposely slow and difficult for CPUs in order to make brute force attacks incredibly hard. But a test suite is kind of like a brute force attack on your software. As fast as it can, it creates hundreds of users (along with other models) to break your system.

It’s clear that bcrypt is a problem when you’re more concerned with speed then security... for instance, while testing.

Our core user model uses sorcery for password salting and hashing. By default, it uses bcrypt but ships with a handful of other crypto-providers. I changed the sorcery default to md5 (really fast hashing, but susceptible to brute force).

Rails.application.config.sorcery.configure do |config|
  config.user_config do |user|
    user.encryption_algorithm = :md5 if Rails.env.test?

Just that one change took the RSpec runtime down from around 3 minutes to about 45 seconds on my 8GB i7 MacBook Air.

With bcrypt:

$ bundle exec rspec -p

Finished in 3 minutes 40.8 seconds
659 examples, 0 failures, 16 pending

With md5:

$ bundle exec rspec -p

Finished in 45.83 seconds
659 examples, 0 failures, 16 pending

I found a couple other tips, including calming down ruby’s garbage collector, which earned me a few more seconds. But using a faster password hashing algorithm in the test environment is a super easy way to see noticeable performance improvements.

Loving RestKit v0.20

Clambake for iPhone uses RestKit to map data from our API to Objective-C models. I chose it way back in 2012 when I started building the app. Then at v0.10, it was a bit tricky to comprehend and implement, but it definitely got the job done. v0.20 came out in March 2013, but I’ve stuck with the older version because transitioning looked hard and v0.10 worked fine as it was.

Internally, we’ve forked our web-services API to make room for a lot of new stuff in the works. I decided to go ahead and switch to RestKit v0.20 while updating the iPhone app to consume v2 of our API.

After a day of migrating over to the new RestKit, I’m really happy with how the RK community (chiefly Blake Watters) has alleviated a lot of my previous pain points with the library.

Better Response Mapping

The older RestKit had some quirks with mapping response objects that were different from posted objects. For instance, our API returns an updated move when the user submits a vote. I previously had to set an empty move as the RKObjectLoader’s targetObject. Otherwise, the response mapping would fail, since it expected a vote. Something like this:

// v0.10

[[RKObjectManager sharedManager] postObject:vote
                                 usingBlock:^(RKObjectLoader* loader) {

  // Tell the loader to expect something other than a Vote
  loader.objectMapping = [[RKObjectManager sharedManager].mappingProvider objectMappingForClass:[Move class]];
  loader.delegate = self;
  loader.targetObject = [Move new]; // Setup a new Move object as the target

In v0.20, the response mapping is working perfectly for me regardless of disparities between what we send and receive from the web services.

// v0.20

[[RKObjectManager sharedManager] postObject:vote
                                    success:^(RKObjectRequestOperation *o, RKMappingResult *result) {
                                      self.move = result.firstObject; // RestKit maps this correctly as a move!

Also, I’ve noticed that debugging mapping issues has become a lot easier. v0.2 has much better error logging and does good job at explaining what’s broken.

Super Clear Request API

Restkit v0.10 had seemingly 200 ways to request data, or to send a payload. I would have to carefully choose which methods to use based on how the data from the API was structured, and on which protocol methods made the most sense. Setting additional parameters to a request (like auth creds) was also very confusing since it had to happen in the loader block.

Now it feels really clean and clear (and under control). I can do everything I need right from the object manager through the getObject, putObject, postObject, etc. methods. Add an object (conveniently optional!), set the params as a dictionary, add a path. Done.

[[RKObjectManager sharedManager] postObject:nil // Optional object to be serialized to NSDictionary

                                       // Relative path from RKObjectManager baseURL

                                 // Extra information to send along
                                 parameters:@{ @"email" : self.email, @"password" : self.password}

                                    // Success block.
                                    success:^(RKObjectRequestOperation *o, RKMappingResult *result) {
                                      o.HTTPRequestOperation.response.statusCode; // Status code
                                      result.array // All mapped object
                                      result.firstObject; // First mapped object in the array

                                    // Error block
                                    failure:^(RKObjectRequestOperation *o, NSError *error) {
                                      o.HTTPRequestOperation.response.statusCode; // Response code
                                      o.HTTPRequestOperation.responseData; // Raw response data

The success and failure callback blocks are also very intuitive. Definitely an upgrade from the many delegate method and block choices from before.

Mapping Setup & Configuration

Setting up your object mapping is definitely different, but not necessarily easier. I found it fairly time consuming. One aspect that took me a few minutes to get my head around was mapping serializers to NSDictionary instead of the the object your trying to serialize. It totally makes sense, since your model is being mapped to a vanilla dictionary before being sent to the API. But it confused me nonetheless.

RKObjectMapping *gameSerializationMapping = [RKObjectMapping mappingForClass:[NSMutableDictionary class]];

[gameSerializationMapping addAttributeMappingsFromDictionary:@{
  @"name": @"name",
  @"duration": @"duration",
  @"privacy": @"private",
  @"description": @"description",

[objectManager addRequestDescriptor:
    [RKRequestDescriptor requestDescriptorWithMapping:gameSerializationMapping
                                          objectClass:[Game class]

[objectManager addRequestDescriptor:
    [RKRequestDescriptor requestDescriptorWithMapping:gameSerializationMapping
                                          objectClass:[Game class]


RestKit is still awesome. I was thinking about ditching it for AFIncrementalStore; I’m glad I stuck with it.

Use Eye, Not Bluepill

Bluepill isn’t actively maintained and generally doesn’t work (ask discourse). Eye is another ruby-based process manager with a similar DSL... and it does work!

I spent most of yesterday wrangling with my AWS infrastructure and deployment configuration. My current setup for Clambake is entirely managed by Capistrano. I’m splitting off the instance management and configuration to be handled by SaltStack (which is awesome, by the way).

I figured I might as well also switch to a more robust process management tool instead of a simple Foreman export to Upstart. I had read a bit about Bluepill, and decided that’s what I would go with.

However, Bluepill straight-up doesn’t work. I had some really simple requirements: daemonize a couple rake tasks, start/stop unicorn, start/stop delayed_job. I’m pretty sure I was using it in a boilerplate fashion, and I understand unix processes and signals (for the most part). Bluepill would start processes, but couldn’t stop them, and will sometimes respawn itself and I’d have 3-4 Bluepill daemons at once.

After wrestling with it for no fewer than 6 hours, I found a thread on the Discourse forum. They are having identical issues with Bluepill. I learned from the thread that Bluepill is no longer maintained and is generally busted. Discourse is looking to drop it.

In the thread, someone randomly recommended Eye. It’s another ruby-based process manager, and has a similar DSL to Bluepill. I gave it a whirl.


Make Ember Always Hit the API

I'm building the Clambake web client using Ember. It's been fairly awesome since it plugs into the existing APIs that our iOS app consumes. However, it has thrown me a few learning curves.

One of the challenging aspects conceptually for me is the role that routing plays in the Ember MVC paradigm. I'm used to Rails where routing just decides which controller method to fire. In Ember routing plays a bigger role; the setup the 'context' for the controller. In other words, they load the data and hand it off to controllers. That's what the model method is doing.

The point here is to facilitate fast transitions to resource routes from other parts of the app. For instance, you click on a user, and Ember sets up the user controller's context to be the object from the click.

CB.Router.map ()->

  @route 'user', path: 'users/:user_id'

  CB.UserRoute = Ember.Route.extend
     model: (params) ->
        # The `model` method isn't called if the context is passed in
  {{#each user in users}}
      {{#linkTo 'user' user}}

So that's cool, and it feels insanely fast.


Just Read the Slides

My brother and I have been pitching Clambake... a lot. Recently, the team of advisors at GCH Partners took us on to help accelerate business strategy and customer development.

One of the first things they did with us was a 'pitch bootcamp'. They essentially went through every aspect of our standard presentation and fine-tuned it. They put an emphasis on brevity, clarity and some sizzle.

During one practice pitch, I was in the middle of explaining our core technology, and Greg stopped me suddenly. He said something along the lines of:

"Look, I'm not even listening to you right now. You're talking and I'm reading what's on the slide. Let's read each slide out loud before you talk about it."

I studied speech communication in college, and I'm pretty sure the first thing they told us was, "don't read your PowerPoint slides." So my initial reaction was negative.

However, I took a moment to think back on some of the issues we've had with earlier pitches. People don't pay attention; I assumed that was human nature, but perhaps they were just reading our slides instead of listening.

Greg's point was this:

  • Your audience is going to read what's on the slide.
  • They will tune you out while they're reading it.
  • Therefore, take 10 seconds to read exactly what's on the slide.... and then talk about it.

iOS 7: Interactive Pop Gesture With Custom Back Button

Clambake for iPhone has a custom back button on all its navigation bars; it's a simple chevron with no text. The easiest way to implement a custom button like this is to simply set a leftBarButtonItem on the controller's navigationItem.

- (void)viewDidLoad
  self.navigationItem.leftBarButtonItem = [self backButton];

- (UIBarButtonItem *)backButton
  UIImage *image = [UIImage imageNamed:@"back_button"];
  CGRect buttonFrame = CGRectMake(0, 0, image.size.width, image.size.height);

  UIButton *button = [[UIButton alloc] initWithFrame:buttonFrame];
  [button addTarget:self action:@selector(backButtonPressed) forControlEvents:UIControlEventTouchUpInside];
  [button setImage:[UIImage imageNamed:normalImage] forState:UIControlStateNormal];

  UIBarButtonItem *item; = [[UIBarButtonItem alloc] initWithCustomView:button];

  return item;

However, doing this nixes the sweet interactive pop gesture iOS 7 provides. I found a quick fix for that here.

Through my beta testers, I started receiving crash reports centering around this gesture pop behavior. I found that by panning left really quickly after pushing a new controller to the stack would cause the crash. In other words, if the user immediately tries to swipe back while the push is still in progress, the navigation controller soils itself.

I found this in the debugger log:

nested pop animation can result in corrupted navigation bar

After a few hours of wrestling with various solves, I found that I can mitigate the errors like this:

Set the gesture delegate to the navigation controller

Like Stuart Hall talks about in his post, assiging a delegate to the gesture resurrects it's behavior when a custom back button is applied. However, with really fast user pop interactions, the controller is sent a message from the gesture after it's been deallocated.

My solution is to simply make the navigation controller itself be the gesture delegate. This technique works best with a UINavigationController subclass.

@interface CBNavigationController : UINavigationController <UIGestureRecognizerDelegate>

@implementation CBNavigationController

- (void)viewDidLoad
  __weak CBNavigationController *weakSelf = self;

  if ([self respondsToSelector:@selector(interactivePopGestureRecognizer)])
    self.interactivePopGestureRecognizer.delegate = weakSelf;


Disable interactivePopGestureRecognizer during transitions

When the user starts swiping backwards in the middle of a transition, the pop events stack up and "corrupt" the navigation stack. My workaround is to temporarily disable the gesture recognizer during push transitions, and enable it again when the new view controller loads. Again, this is easier with a UINavigationController subclass.

@interface CBNavigationController : UINavigationController <UINavigationControllerDelegate, UIGestureRecognizerDelegate>

@implementation CBNavigationController

- (void)viewDidLoad
  __weak CBNavigationController *weakSelf = self;

  if ([self respondsToSelector:@selector(interactivePopGestureRecognizer)])
    self.interactivePopGestureRecognizer.delegate = weakSelf;
    self.delegate = weakSelf;

// Hijack the push method to disable the gesture

- (void)pushViewController:(UIViewController *)viewController animated:(BOOL)animated
  if ([self respondsToSelector:@selector(interactivePopGestureRecognizer)])
    self.interactivePopGestureRecognizer.enabled = NO;

  [super pushViewController:viewController animated:animated];

#pragma mark UINavigationControllerDelegate

- (void)navigationController:(UINavigationController *)navigationController
       didShowViewController:(UIViewController *)viewController
  // Enable the gesture again once the new controller is shown

  if ([self respondsToSelector:@selector(interactivePopGestureRecognizer)])
    self.interactivePopGestureRecognizer.enabled = YES;