Mocking HTTP Responses in Golang

How to mock HTTP responses in a golang test suite using httptest.Server and http.Transport


I’ve been trying to hone my golang skills lately; this weekend I wanted to especially sharpen my testing chops. I decided to write a simple package for sending emails via the Mandrill API (which is the best transactional email platform out there, IMO). I wanted the package to send both regular messages and template-messages, and to have 90%+ test coverage.

Writing the mandrill interface was fairly trivial: json.Marshal some payload data, post it via an http.Client, and json.Unmarshal the response into some structs. However, testing it was much trickier.

I’m used to testing Ruby where mocking and stubbing is the name of the game. It’s really easy to adjust responses of object methods to cover your test cases. For instance, when writing an API wrapper like this one, I’d mock HTTP responses so the test suite doesn’t actually hit the API.

Golang isn’t as flexible at runtime, and that’s generally a bonus, but it makes mocking/stubbing non-trivial. In the test suite, I needed http.Client.Post to just return some static example JSON instead of making a true connection.

I Googled around for a while, and couldn’t find a clear example, but stumbled upon a few clues regarding http.Transport. It’s the interface that actually makes the connection between the client and the target web service.

It's the right spot to mock a response. The strategy involves:

  • Dependency injecting an http.Client into the package methods
  • Glomming all outgoing requests up in a custom http.Transport
  • Rerouting the request to an ultra-basic httptest.Server that responds with the data and headers for the test

The Package

package mandrill

type Client struct {
  BaseURL string
  HTTPClient *http.Client
}

func (m *Client) MessagesSend(message *Message) (responses []*Response, apiError Error, err error) {

  payload, err := json.Marshal(message)
  if (err != nil) { return responses, apiError, err }

  resp, err := m.HTTPClient.Post(m.BaseURL+"messages.json", "application/json", bytes.NewReader(payload))
  if (err != nil) { return responses, apiError, err }

  defer resp.Body.Close()
  body, err := ioutil.ReadAll(resp.Body)
  if (err != nil) { return responses, apiError, err }

  if (resp.StatusCode >= 400) {
    apiError = &Error{}
    err = json.Unmarshal(body, apiError)
    return responses, apiError, errors.New(fmt.Sprintf("Status code: %i", resp.StatusCode))
  }

  responses = make([]*Response, 0)
  err = json.Unmarshal(body, &responses)
  return responses, apiError, err
}

The Test

package mandrill

func expect(t *testing.T, a interface{}, b interface{}) {
  if a != b {
    t.Errorf("Expected %v (type %v) - Got %v (type %v)", b, reflect.TypeOf(b), a, reflect.TypeOf(a))
  }
}

func Test_MessageSend_Success(t *testing.T) {

  // Test server that always responds with 200 code, and specific payload
  server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(200)
    w.Header().Set("Content-Type", "application/json")
    fmt.Fprintln(w, `[{"email":"bob@example.com","status":"sent","reject_reason":"hard-bounce","_id":"1"}]`)
  }))
  defer server.Close()

  // Make a transport that reroutes all traffic to the example server
  transport := &http.Transport{
    Proxy: func(req *http.Request) (*url.URL, error) {
      return url.Parse(server.URL)
    },
  }

  // Make a http.Client with the transport
  httpClient := &http.Client{Transport: transport}

  // Make an API client and inject
  client := &Client{server.URL, httpClient}

  // Test the method!
  responses, apiError, _ := m.MessagesSend(&Message{})

  correctResponse := &Response{
    Email: "bob@example.com",
    Status: "sent",
    RejectionReason: "hard-bounce",
    Id: "1",
  }

  expect(t, len(responses), 1)
  expect(t, reflect.DeepEqual(correctResponse, responses[0]), true)
}

DRY It Out

In the test suite, I DRY’d out the whole thing to a fairly simple factory that returns a mock client, and server setup to respond with a specific status code and body.

func testTools(code int, body string) (*httptest.Server, *Client)  {
  server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(code)
    w.Header().Set("Content-Type", "application/json")
    fmt.Fprintln(w, body)
  }))

  transport := &http.Transport{
    Proxy: func(req *http.Request) (*url.URL, error) {
      return url.Parse(server.URL)
    },
  }

  httpClient := &http.Client{Transport: transport}
  client := &Client{server.URL, httpClient}

  return server, client
}

func Test_MessageSend_Success(t *testing.T) {
  server, client := testTools(200, `[{"email":"bob@example.com","status":"sent","reject_reason":"hard-bounce","_id":"1"}]`)
  defer server.Close()

  //...
}

func Test_MessageSend_Fail(t *testing.T) {
  server, client := testTools(400, `{"status":"error","code":12,"name":"Unknown_Subaccount","message":"No subaccount exists with the id 'customer-123'"}`)
  defer server.Close()

  //...
}

Checkout my mandrill package on github.

Layout A CAShapeLayer

I’ve recently discovered the magic of using CAShapeLayer for all glyph-iconography throughout my apps. Part of the magic, for me at least, has been my recent investment in PaintCode. The app makes it ridiculously easy to generate shape paths from SVGs, Sketch files, etc.

After a few days of deep CAShapeLayer integration, I have a some key leanings to share regarding positioning and autolayout.

Can I Use Autolayout With a Shape Layer?

You can’t. Autolayout is only applied to UIView and subclasses thereof. CALayer must be 'manually' sized and positioned using rects.

However, you can certainly insert a CAShapeLayer in a UIView, override its intrinsicContentSize and position it using autolayout.

I’ve been using a simple subclass of UIView that manages a single CAShapeLayer property. Whenever the shape property is set, the view will calculate it’s bounds, and invalidate the intrinsic content size of the view.

intrinsicContentSize is what autolayout will look to for understanding the size of the view.

CAShapeLayer *shapeLayer = [CAShapeLayer layer];
UIBezierPath *path = [UIBezierPath bezierPathWithOvalInRect:CGRectMake(0, 0, 100, 100)];
shapeLayer.path = path.CGPath;
shapeLayer.frame = path.bounds;
shapeLayer.fillColor = [UIColor blueColor].CGColor;
self.shapeView = [KTShapeView initWithShapeLayer:shapeLayer];
self.shapeView.translatesAutoresizingMaskIntoConstraints = NO;
[self.view addSubview:self.shapeView];

// Center the view on the X axis
[self.view addConstraint:[NSLayoutConstraint constraintWithItem:self.shapeView
                                                      attribute:NSLayoutAttributeCenterX
                                                      relatedBy:NSLayoutRelationEqual
                                                         toItem:self.view
                                                      attribute:NSLayoutAttributeCenterX
                                                     multiplier:1
                                                       constant:0];

// Center the view on the Y axis
[self.view addConstraint:[NSLayoutConstraint constraintWithItem:self.shapeView
                                                      attribute:NSLayoutAttributeCenterY
                                                      relatedBy:NSLayoutRelationEqual
                                                         toItem:self.view
                                                      attribute:NSLayoutAttributeCenterY
                                                     multiplier:1
                                                       constant:0];
@interface KTShapeView : UIView
@property (nonatomic, strong) CAShapeLayer *shapeLayer;
+ (id)initWithShapeLayer:(CAShapeLayer *)shapeLayer;
@end

@implementation KTShapeView

+ (id)initWithShapeLayer:(CAShapeLayer *)shapeLayer
{
  KTShapeView *view = [KTShapeView new];
  view.shapeLayer = shapeLayer;
  return view;
}

- (void)setShapeLayer:(CAShapeLayer *)shapeLayer
{
  [self.shapeLayer removeFromSuperlayer];
  _shapeLayer = shapeLayer;
  _shapeLayer.anchorPoint = CGPointMake(0.5, 0.5);
  [self.layer addSublayer:self.shapeLayer];

  [self invalidateIntrinsicContentSize];
  [self setNeedsLayout];
}

- (CGSize)intrinsicContentSize
{
  return self.shapeLayer.bounds.size;
}

- (void)layoutSubviews
{
  [super layoutSubviews];
  CGRect frame = self.bounds;
  self.shapeLayer.position = CGPointMake(CGRectGetMidX(frame), CGRectGetMidY(frame));
}
@end

I've added the KTShapeView to my KTCategories project.

Don't Be Secret

During my first year building Clambake, I worked on it through the nights and over the weekends. But I was afraid to tell anyone about it… especially my coworkers at my day job. My fear was a mix of "I don't want to look dumb with a half baked product," and "will my coworkers be upset because I'm doing my own thing on the side?"

It turns out that the partners WERE upset when I decided to leave and pursue my own adventure. However, they were only upset that I had kept my project a secret in the first place.

More importantly, I left months of feedback, advice and testing on the table. The way I see it now, my product was nothing until I released a beta; it was non-existent even though I worked tirelessly on it. After releasing to the App Store, I had to restructure the app almost entirely within two months; I essentially tossed a years work into the recycling bin. All of the sudden I had tons of great feedback and new ideas from real users.

My whole issue was isolation. Tinkering away in the dark was a giant waste of time… I was guaranteed to be missing the mark on what people actually want.

My personal takeaways to be applied to any of my future ventures are:

  • Tell everyone you know about your app/business/idea as soon as you dream it up.
  • Prototype an interactive experience and test it with 5 users regularly. Don't actually build anything difficult or time consuming until you know what you should build. I've discovered Test Tube in NYC the events are the best way to get some user hands-on with your product.

A System Ruby!

I was catching up on a few Changelog episodes last week, and heard some great advice on managing ruby versions in deployment environments.

@postmodern, creator and maintainer of chruby, said essentially, "Don’t use a ruby switcher in production. You don’t need it. Install a system ruby.""

It’s kind of funny coming from a guy who writes a ruby switcher, but he’s totally right. I took a look at my Capistrano recipes and saltstack states, and there was a surprising amount of overhead to using a ruby switcher on production (I’ve been using rbenv on the 20 or so deployment machines that I manage):

  • Every user who needs to run ruby has rbenv installed, configured, and initialized in their dotfiles. Systemwide rbenv configurations are hard.
  • Installing gems via your automated tools? Don’t forget to rbenv rehash!
  • Any ruby command issued without a shell (e.g via Capistrano) needs to have an explicit PATH appended with the rbenv paths/shims
  • How often am I updating the ruby version on your servers? Never?

Ruby switchers are really awesome on your development machine, but they clearly get in the way for deployments.

So I took a few minutes to update a cluster of machines to use a system ruby instead of a shimmed one via rbenv. I use SaltStack to manage infrastructure so it was as easy as something like this to compile and install ruby from source:

wget -c http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.2.tar.gz
tar xzf ruby-2.1.2.tar.gz
cd ruby-2.1.2
./configure
make
make install

This could be even easier, most likely, if I used a containerization tool like docker. But regardless, with way less code and overhead I can resolve all the issues listed above.

Thanks to @postmodern for some good advice!

Decelerate a UIScrollView to a Specific Spot (à la UltraVisual)

For a recent prototyping project, I was reverse engineering the awesome scrolling interaction of UltraVisual. There are many aspects that make up the responsiveness to their interaction: the cell sizes interpolate their heights as they approach being the ‘featured’ item, the tiled images parallax slightly on scroll, etcetera.

A key aspect, though, is how the scrollview decelerates naturally into another cell position. It does not simply animate to a position after the scroll view stops decelerating (for an example of that behavior, check out this other clone I found). I think it’s this nuance that ties everything together.

The technique I landed upon was to implement the scrollViewWillEndDragging:withVelocity:targetContentOffset delegate method, and change the targetContentOffset to be the position of the appropriate next cell.

#pragma mark UIScrollViewDelegate

- (void)scrollViewWillEndDragging:(UIScrollView *)scrollView
                     withVelocity:(CGPoint)velocity
              targetContentOffset:(inout CGPoint *)targetContentOffset
{
  /**
   * Here we target a specific cell index to move towards
   */

  NSInteger nextIndex;
  CGFloat currentY = scrollView.contentOffset.y;
  CGFloat yDiff = abs(targetContentOffset->y - currentY);

  if (velocity.y == 0.f)
  {
    // A 0 velocity means the user dragged and stopped (no flick)
    // In this case, tell the scroll view to animate to the closest index
    nextIndex = roundf(targetContentOffset->y / kUVCellDragInterval);
    *targetContentOffset = CGPointMake(0, nextIndex * kUVCellDragInterval);
  }
  else if (velocity.y > 0.f)
  {
    // User scrolled downwards
    // Evaluate to the nearest index
    nextIndex = ceil((targetContentOffset->y - yDiff / kUVCellDragInterval);
  }
  else
  {
    // User scrolled upwards
    // Evaluate to the nearest index
    nextIndex = floor((targetContentOffset->y + yDiff / kUVCellDragInterval);
  }

  // Return our adjusted target point
  *targetContentOffset = CGPointMake(0, MAX(nextIndex*kUVCellDragInterval, self.collectionView.contentInset.top));
}

This provides a nice smooth deceleration directly to where the view should ‘land’.

Speed up the deceleration

Part of the UltraVisual interaction’s great feel is the very controlled flicks you can give the UI to advance or retreat through the list. Out of the box, the UIScrollView decelerates too slowly, making the experience is too slippery and fast.

First you can set the scroll view’s decelerationRate to UIScrollViewDecelerationRateFast. But that still didn’t feel fast enough. To speed it up more I applied a 'dampener' to the targetContentOffset before calculating the next cell index.

#pragma mark UIScrollViewDelegate

#define kDragVelocityDampener .85

- (void)scrollViewWillEndDragging:(UIScrollView *)scrollView
                     withVelocity:(CGPoint)velocity
              targetContentOffset:(inout CGPoint *)targetContentOffset
{
  /**
   * Here we target a specific cell index to move towards
   */

  NSInteger nextIndex;
  CGFloat currentY = scrollView.contentOffset.y;
  CGFloat yDiff = abs(targetContentOffset->y - currentY);

  if (velocity.y == 0.f)
  {
    // A 0 velocity means the user dragged and stopped (no flick)
    // In this case, tell the scroll view to animate to the closest index
    nextIndex = roundf(targetContentOffset->y / kUVCellDragInterval);
    *targetContentOffset = CGPointMake(0, nextIndex * kUVCellDragInterval);
  }
  else if (velocity.y > 0.f)
  {
    // User scrolled downwards
    // Evaluate to the nearest index
    // Err towards closer a index by forcing a slightly closer target offset
    nextIndex = ceil((targetContentOffset->y - (yDiff*kDragVelocityDampener)) / kUVCellDragInterval);
  }
  else
  {
    // User scrolled upwards
    // Evaluate to the nearest index
    // Err towards closer a index by forcing a slightly closer target offset
    nextIndex = floor((targetContentOffset->y + (yDiff*kDragVelocityDampener)) / kUVCellDragInterval);
  }

  // Return our adjusted target point
  *targetContentOffset = CGPointMake(0, MAX(nextIndex*kUVCellDragInterval, self.collectionView.contentInset.top));
}

Here’s an example interaction on github. It’s not a perfect interactive clone, but it worked for my prototyping purposes.

Handling Errors From Goroutines

I'm writing a web service in Go for processing images. The basic premise is this:

  • A client posts an image file to the service
  • It creates the versions needed (resizes, thumbnails, blurs, etc)
  • It uploads them to an S3 bucket
  • It returns a JSON payload with paths, identifiers, and URLs for all the versions.

The experience is fairly synchronous for the client posting the image. Behind the scenes, though, I want to use Go's awesome concurrency to parrelize the processing of all the image versions. The goal is for the whole shebang to not take too much longer than processing one image (even if there are 3-4 to do).

Because it's a synchronous experience for the end user, if there is a problem anywhere, I want to halt the whole circus and retun an error. I've found that in concurrency land, this is not as easy as it sounds. Here's an outline of how I achieve that using goroutines and channels to catch an error and stop all the goroutines. The rest of the program is blocked while it waits for the goroutines to finish/fail.

First, it's important to know how many goroutines will be spawned to how many are remaining in the select statement. Each go routine is passed both a success and error channel; they'll communicate their outcomes via these channels. An infinite for loop listens for channel messages via a select block. If a message comes through the success channel, it decrements the remaining count. Once the count is zero, the loop is broken and the rest of the program executes.

If an signal is received on the error channel, the function returns and cancels the other goroutines in progress.

From what I learned playing around with sync.WaitGroup, it behaves more or less the same as this strategy, but with no error catching.

The more I worked at this, the more I realized it's not a great fundamental concept anyway (i.e. be concurrent until something screws up). I'm trying to use concurrency for speed/arallelization here... but clearly these goroutines are fighting back against the complexity. They seem best used when you'll 'expect' some to fail, and want the others to continue happily.

But, this is my first foray into Go-land, and I'm getting good performance. So I'm happy!

Hide the Navbar For a Specific Controller in a UINavigationController Stack

I have a UINavigationController stack where the standard navbar is visible in all but one of the view controllers. Specifically the 2nd controller out of 5.

It’s easy to simply [self.navigationController setNavigationBarHidden:YES animated:YES]; in the controller’s viewDidLoad, but the UX of pushing and popping the others on and off the stack is pretty wonky. Especially with the iOS 7 interactive pop gesture. You’ll get weird holes in the UI as the navbars come in and out.

After some trial an error, here’s the strategy that did the trick for me.

// The navigation controller stack

UINavigationController
    UIViewController // regular
    CBViewController // no navbar
    UIViewController // regluar
    UIViewController // regular
@interface CBViewController : UIViewController
@property bool lockNavBarAsHidden;
@end

@implementation CBViewController

- (void)viewWillAppear:(BOOL)animated
{
  [super viewWillAppear:animated];
  self.lockNavBarAsHidden = NO;
  [self.navigationController setNavigationBarHidden:YES
                                           animated:animated];
}

- (void)viewWillDisappear:(BOOL)animated
{
  [super viewWillDisappear:animated];
  [self.navigationController setNavigationBarHidden:self.lockNavBarAsHidden
                                           animated:animated];
}

- (void)presentModal
{
  self.lockNavBarAsHidden = YES;
  UINavigationController *navController;
  navController = [[UINavigationController alloc] initWithRootViewController:[UIViewController new]];
  [self presentViewController:navController
                     animated:YES
                   completion:nil];
}

@end

In viewWillAppear, tell the UINavigationController to hide the navbar with the same animated argument the controller itself receives.

In viewWillDisappear, instruct the navbar to show itself again. Again, use the animated argument from the view controller to determine whether to animate the nav bar back in.

I found one issue with this system, though. If another controller is presented modally from a controller without a navbar, the navbar with show its face briefly. It shows up because viewWillDisappear is called when a modal is presented over a controller… and that’s where we instruct the navbar to come back.

My solution is to put a virtual lock (self.lockNavBarAsHidden) on the navbar when I present a modal. It’s unlocked whenever viewWillAppear is called again.

I hope this helps someone out there get some better navbar UX in their apps.