FlowFX

Jason Swett: How I write characterization tests

Let’s say I come across a piece of code that I want to refactor but unfortunately

a) I don’t understand what it does and
b) it’s not covered by tests.

Yupp. That's a problem. How I write characterization tests shows a technique for how to deal with this situation. I, too, use tests to figure out how a piece of code actually works. Works better for me than actually reading the code. 🤨

Speeding up RSpec with bundler standalone and springified binstubs

Testing Rails with RSpec is slow, or at least it feels slow to me in all the projects I am working on. Any speed gain helps my productivity when it reduces the time I'm waiting between writing code and running tests.

There are quite a few posts out there that tackle this problem. But most are pretty old, and none really worked for me, so I'm not going to link to them.

The book I'm reading inspired me to look into the bundler --standalone command. Not that I really understand what's happening there, but at least I got a little bit of a speed bump out of it.

Here's what I did, and how I'm running my tests now.

Spring

First I made sure my Rails app is installed with Spring enabled. Luckily, this is the default. In order to later run RSpec from spring, I added the spring-commands-rspec gem to my Gemfile.

gem 'spring-commands-rspec'

Bundler

Next, I used bundler's standalone command,

$ bundle install --standalone --path .bundle

and then "springified" the installed binstubs.

$ bundle exec spring binstub --all

Problems

I encountered a problem with SQLite 1.4.0. I didn't investigate it further, but pinned the gem to version 1.3 instead.

group :development, :test do
  gem "sqlite3", "~> 1.3.6"
end

Afterwards I repeated the install command.

$ bundle install --standalone --path .bundle

Anytime you want to use bundle install, you now have to use bundle install --standalone instead. I created the bash alias bis for that.

vim-test

I recently started using the vim-test plugin. That plugin has a neat option that makes it use the springified binstubs.

" .vimrc

let test#ruby#use_spring_binstub = 1

Now, when I'm editing app/models/transformer_spec.rb and I hit the return key in normal mode, vim-test executes

$ ./bin/spring rspec spec/models/transformer_spec.rb

Because of spring, everything's faster and I have actually seen tests being executed in less than a second. Still not super fast but better than before.

Teach vim-rails about request specs

Slowly but surely I'm getting to know the really interesting Rails-related Vim plugins. First and foremost: vim-rails. A new feature that I started using recently is the :A alternate file command. Basically, this makes it super quick to jump from a model file to its model spec file. Or from a controller to its spec.

Since Rails 5.?, controller specs have fallen out of DHH's favor. We now use request specs. Unfortunately, vim-rails doesn't know about request specs. Fortunately, we can tell it about them.

vim-rails is very well documented, and :help rails-projections provides the solution to this problem. This is what I put in my .vimrc, and now :A jumps between my controller files and the related request specs.

let g:rails_projections = {
      \ "app/controllers/*_controller.rb": {
      \   "test": [
      \     "spec/controllers/{}_controller_spec.rb",
      \     "spec/requests/{}_spec.rb"
      \   ],
      \ },
      \ "spec/requests/*_spec.rb": {
      \   "alternate": [
      \     "app/controllers/{}_controller.rb",
      \   ],
      \ }}

Running RSpec with a single keystroke in a separate tmux session

This is a tiny update to Running Specs from Vim, Sent to tmux via Tslime from the thoughtbot blog. Go read it, and realize that it's six years old.

vim-rspec has not been updated in two years. It probably still works fine, but there's a new vim plugin that does what vim-rspec does, just better: vim-test.

I finally set this up yesterday:

" Add vim-test and tslime to vim plugins.
Plug 'janko/vim-test'
Plug 'jgdavey/tslime.vim'

...

" Configure vim-test to execute test command using tslime
let test#strategy = "tslime"

" Configure <CR> aka the Return key to run my test file.
nmap <CR> :TestFile<CR>
" I'm still figuring out which test commands make the most sense
" in my workflow. Right now, this feels pretty good.
nmap <leader><CR> :TestLast<CR>

Now, when I'm in normal mode and hit the return key, rspec gets executed for the current file in a different tmux pane (!!). What I didn't understand before using this was how it would select the correct pane. Turns out, it's super easy. On the first run, it asks me for the tmux session, window, and pane (if necessary). After that, it remembers and it always sends the test command there. Super cool!

Unit testing and test parametrization with RSpec?

Last year I made the switch from developing with Python and Django to working with Ruby and Rails. With that, I also had to learn a new test runner. No more pytest :(.

For some reason, a very popular test runner for testing Rails apps is RSpec. Its domain-specific language (DSL) is written with Behavior Driven Development in mind. Hence its tag line:

Behaviour Driven Development for Ruby. Making TDD Productive and Fun.

I'm really struggling with using RSpec, and I wonder why. Most of the time, I'm fighting the framework instead of enjoying its benefits. So I want to explore these questions and hopefully get some answers how to do this better. This is the first post on this topic.

Testing simple functions

Let's take a simple function that adds two numbers. 

# Python
def add(a, b):
    return a + b

# Ruby
def add(a, b)
  a + b
end

Using pytest, it takes 2 lines of code to test this function with one set of parameters.

def test_add():
    assert add(5, 4) == 9

Using RSpec, this takes me at least 5 loc.

describe '#add' do
  it 'returns the sum of 2 integers' do
    expect(add(5, 4)).to eq(9)
  end
end

Or 4 if I use the subject shorthand notation.

describe '#add' do
  subject { add(5, 4) }

  it { is_expected.to eq 9 }
end

That's a lot of words for just a simple assertion.

Test parametrization

Now I want to test not only one set of test parameters, but many. With pytest I use the parametrize marker.

import pytest

@pytest.mark.parametrize("a, b, expected", [
    (5, 4, 9),
    (1, 2, 3),
    (1, 2, 4)
])
def test_add(a, b, expected):
    assert add(a, b) == expected

In case of a test failure, pytest shows me exactly which parameters lead to the error.

======== FAILURES =========
________ test_add[1-2-4] ______

a = 1, b = 2, expected = 4

    @pytest.mark.parametrize("a, b, expected", [
        (5, 4, 9),
        (1, 2, 4)
    ])
    def test_add(a, b, expected):
>       assert add(a, b) == expected
E       assert 3 == 4
E        +  where 3 = add(1, 2)

RSpec 1

A naïve Ruby implementation would be to loop over an array of parameter arrays.

params = [
  [5, 4, 9],
  [1, 2, 3],
  [1, 2, 4]
]

describe '#add' do
  params.each do |p|
    it 'returns the sum of 2 integers' do
      expect(add(p[0], p[1])).to eq(p[2])
    end
  end
end

Failure output is not very helpful as I don't see which parameters caused the error.

Failures:

  1) #add returns the sum of 2 integers
     Failure/Error: expect(add(p[0], p[1])).to eq(p[2])

       expected: 4
            got: 3

       (compared using ==)
     # ./demo02_spec.rb:16:in `block (3 levels) in <top (required)>'

RSpec 2

Another approach, that I found on the internets, includes writing a test function and call that function repeatedly using different parameters.


def test_add(a, b, expected)
  describe '#add' do
    subject { add(a, b) }

    it { is_expected.to eq expected }
  end
end

test_add 5, 4, 9
test_add 1, 2, 3
test_add 1, 2, 4

The test function looks a little obscure, but the last part is very readable. Also, when a failure occurs, we can see the exact line where it happens.

Failures:

  1) #add should eq 4
     Failure/Error: it { is_expected.to eq expected }

       expected: 4
            got: 3

       (compared using ==)
     # ./demo03_spec.rb:12:in `block (2 levels) in test_add'

rspec-parametrized

Lastly, there is the rspec-parameterized plugin that provides a syntax that's very close to pytest.

require 'rspec-parameterized'

describe '#add' do
  where(:a, :b, :expected) do
    [
      [4, 5, 9],
      [1, 2, 3],
      [1, 2, 4]
    ]
  end

  with_them do
    it 'returns the sum of 2 integers' do
      expect(add(a, b)).to eq expected
    end
  end
end

Failure output is actually helpful, even if not as clean as pytest:

Failures:

  1) #add a: 1, b: 2, expected: 4 returns the sum of 2 integers
     Failure/Error: expect(add(a, b)).to eq expected

       expected: 4
            got: 3

       (compared using ==)
     # ./demo05_spec.rb:20:in `block (3 levels) in <top (required)>'

The reason we have not yet introduced this plugin into our test suite at work is that the gem has a few dependencies that have not been updated in many years:

Conclusion?

All-in-all, I am still very confused about how to best use RSpec. Testing simple functions includes a lot of boilerplate code that's hard to write and slow to read. Test parameterization is doable but no clear best practice has emerged, yet.

If you can help me out and give me some pointers, or if you can tell me that I'm going at this completely wrong, please mention me on Mastodon (preferred) or Twitter or contact me another way.

Possible future questions include: