Porting `iterate` to Ruby

I originally posted this as an email to the Ruby Parley mailing list, but I thought it might have broader appeal.

I've been writing some code that deals with time ranges recently.

I've also been playing with functional languages where list manipulation is common when solving pretty much any problem that can be modeled as list manipulation!

List manipulation is desirable when it results in code that is easier to reason about, does not mutate state, avoids unnecessary looping structures (again, often with mutable state), or keeps the door open for solving the problem in a different way.

While Ruby has Enumerator and Enumerable (which are great!), they revolve around methods that yield. But there's a large variety of methods that "iterate" (without side effects) but don't yield. For instance, ActiveSupport provides Time#next_month, but not Time#each_month.

So, I built an enumerator that is modeled after Clojure's iterate. My Iterator assumes you give it a method (free from side effects) that, well, iterates:

class Iterator < Enumerator
  def initialize(object, method)
    super() do |y|
      y << object
      loop { y << object = object.__send__(method) }
    end
  end
end

The code may be a bit dense at first, but Ruby's Enumerator is doing the heavy lifting. I find it pretty great that Ruby gives me the tools to build something like that in a few lines of code.

My original use case was dates and times:

require 'date'

# First 6 months of a given year
Iterator.new(Date.new(2013, 1, 1), :next_month).take(6)
# [#<Date: 2013-01-01>, #<Date: 2013-02-01>, #<Date: 2013-03-01>,
#  #<Date: 2013-04-01>, #<Date: 2013-05-01>, #<Date: 2013-06-01>]

# With ActiveSupport, range of the first 3 months of a given year
Iterator.new(Time.new(2013, 1, 1), :next_month).
  lazy.map(&:all_month).take(3).to_a
# => [2013-01-01 00:00:00 -0500..2013-01-31 23:59:59 -0500,
#     2013-02-01 00:00:00 -0500..2013-02-28 23:59:59 -0500,
#     2013-03-01 00:00:00 -0500..2013-03-31 23:59:59 -0400]

Iterator also adds an elegant way to generate infinite number sequences which can often be good substitutes for loops:

# First 10 3-digit "palindrome numbers"
Iterator.new(100, :next).
  lazy.select { |n| n.to_s == n.to_s.reverse }.take(10).to_a
# => [101, 111, 121, 131, 141, 151, 161, 171, 181, 191]

If you're a fan of freedom patching, you can add an #iter_for method to Object:

class Object
  def iter_for(method)
    Iterator.new(self, method)
  end
end

... then you get:

require 'date'

Date.new(2013, 1, 1).iter_for(:next_month).take(6)
# [#<Date: 2013-01-01>, #<Date: 2013-02-01>, #<Date: 2013-03-01>,
#  #<Date: 2013-04-01>, #<Date: 2013-05-01>, #<Date: 2013-06-01>]

If you're interested in seeing #iter_for in Ruby 2.1, please comment on or watch Feature Request #8506.



My Assets Are No Longer Minified in Rails 4: HALP!

Rails 4.0 and sprockets no longer heed the config.assets.compress directive. Instead, the JavaScript and CSS compressor must be specified explicitly with config.assets.js_compressor and config.assets.css_compressor, respectively.

No error or deprecation warning is raised: assets will simply not be minified if the Rails 3 config.assets.compress = true directive is left intact.

Unminified assets may cause pages to load more sluggishly.

When upgrading a Rails 3 application to Rails 4, make sure to adjust config/environments/production.rb as shown:

Widgets::Application.configure do
  # ...

  # Compress JavaScripts and CSS
  # config.assets.compress = true <- remove the Rails 3 setting
  config.assets.js_compressor  = :uglifier
  config.assets.css_compressor = :sass
end

Both Rails 3 and 4 use uglifier and sass-rails in the default Gemfile, so these compressors are available unless you have removed them.

With the new settings added, assets will again minify when compiled in production.

If you are interested in more upgrading steps and checklists, check out the handbook I am writing: Upgrading to Rails 4.



Upgrading Rails: Gems Extracted in Rails 4

During the development of Rails 4, many features that were present in earlier versions of Rails were removed from Rails itself and extracted to gems.

Extracting features to gems slims down Rails itself, and allows Rails to take a different direction to solve certain problems. For example, strong_parameters in Rails 4 is recommended over attr_accessible and attr_protected from Rails 3 when an application needs to protect itself from mass-assignment vulnerabilities.

Furthermore, some of the extracted gems have new maintainers. These fresh maintainers might respond more quickly to bug and feature requests than the Rails core team feasibly can.

Unfortunately, existing applications that are upgraded to Rails 4 may need to pull in several new gems in order to perform properly. I have compiled a list of these extracted gems and the features they provide. If your application uses any of the listed features, you'll want to pull them into Gemfile while upgrading to Rails 4.

The book I'm writing, Upgrading to Rails 4, goes into more detail about these extractions as well as new features in Rails 4.

Gem Description
protected_attributes Because Rails 4 recommends strong_parameters for mass-assignment protection, attr_accessible and attr_protected have been extracted. I expect that most upgraded applications will need this gem, as the transition to strong_parameters can be tedious and error-prone.
activeresource While the ActiveRecord-like abstraction over a RESTful API has always shipped as a gem, it is no longer included with Rails by default. Include it explicitly if your application requires it.
actionpack-action_caching
actionpack-page_caching
Rails 4 includes many improvements to fragment caching, but action and page caching have been extracted. If your application uses action or page caching, be sure to pull these gems in explicitly.
activerecord-session_store The ability to store session data in a database table has been extracted in Rails 4. If your application uses the ActiveRecord session store, include this gem.
rails-observers Rails no longer encourages the use of observers, separate objects that can react to lifecycle events of ActiveRecord models. If your application uses observers, make sure to include this gem.
actionpack-xml_parser Following security vulnerabilities involving inbound XML, Rails extracts the ability to accept XML input to a gem. If your application accepts XML in a request body (note: this is distinct from responding with XML), you will need to pull in this gem.
rails-perftest Rails 4 extracts ActionDispatch::PerformanceTest. If your application includes performance tests (usually located in test/performance), add this gem to your bundle.
actionview-encoded_mail_to Rails previously included a little-known feature to obfuscate email addresses in hyperlinks, either with HTML entities or JavaScript code. If your application uses the encode option with mail_to, include this gem.

If you're interested in more details about upgrading to Rails 4, please consider buying my Upgrading to Rails 4 handbook.



Decoding Rails Session Cookies

An Upgrading to Rails 4 reader emailed me about this quote from the book:

Rails 3 uses digitally signed cookies as the default store for sessions. Digitally signed cookies cannot be easily tampered with, but users can read the data that is being saved.

He asked:

Would you say that's something trivial to do, or would it involve some decent amount of work? Just by accessing the Rails 3 cookies from my browser's inspector, I can't seem to find a way to read the content from them.

Great question! Let's try.

For this example, I setup a Rails application that sets a session key to a constant value:

class ApplicationController < ActionController::Base
  before_filter :add_value_to_session

  private

  def add_value_to_session
    session[:message] = "Hello World!"
  end
end

If I hit the Rails application with curl -i (include headers), I see:

$ curl -i http://localhost:3000/
HTTP/1.1 200 OK 
Set-Cookie: _railsapp_session=BAh7B0kiD3Nlc3Npb25faWQGOgZFRkkiJWYxZGNlMmNmYjFmYjBkODQ0NTc1ZWE3OTBjZmJmNTZkBjsAVEkiDG1lc3NhZ2UGOwBGSSIRSGVsbG8gV29ybGQhBjsARg%3D%3D--decd8e233744e2ba5a80481426e41d72a6685986; path=/; HttpOnly

The cookie data looks obfuscated, but in reality it's shrouded only by Base64 encoding.

Rack's session cookie methods can easily decode it:

# decode.rb
require 'rack'

puts Rack::Session::Cookie::Base64::Marshal.new.decode(ARGV[0])

Running decode.rb with the cookie's data results in:

$ ruby decode.rb BAh7B0kiD3Nlc3Npb25faWQGOgZFRkkiJWYxZGNlMmNmYjFmYjBkODQ0NTc1ZWE3OTBjZmJmNTZkBjsAVEkiDG1lc3NhZ2UGOwBGSSIRSGVsbG8gV29ybGQhBjsARg%3D%3D--decd8e233744e2ba5a80481426e41d72a6685986
{"session_id"=>"f1dce2cfb1fb0d844575ea790cfbf56d", "message"=>"Hello World!"}

Our message, alongside a generated session identifier, is available without the Rails application's secret session token.

Attempting to tamper with the data, repackage it, and send it back to the server would fail because the digital signature would not check out, but the data is at least readable.

Rails 4 will feature an encrypted cookie store, where data is both digitally verified and unreadable to end users. I write about this (and more, of course!) in Upgrading to Rails 4.

In closing, remember:

  • The default Rails 3 session store (cookie store) allows users to read the contents of the session. They may not, however, change or tamper with the session data.
  • Do not store data in the session that would be sensitive even if the data were read by an end user or an attacker sniffing the traffic.
  • Switch to the encrypted cookie store when Rails 4 ships.


rspec-rails and capybara 2.0: what you need to know

Background

Writing full-stack tests is important. I go into detail about this in my Acceptance Testing slide deck.

In a Rails application, RSpec request specs are often used for the task of writing full-stack tests. In capybara 1.x, request specs both wrap Rails' IntegrationTest framework and mix in capybara, a tool for driving web applications (often via headless web browsers).

Confusion

The fact that request specs both wrap a Rails IntegrationTest and have capybara methods mixed in is confusing. José Valim goes into depth about the problems in his blog post entitled "Improving the integration between capybara and RSpec".

If you've ever confused get (Rails) with visit (capybara) or response with page, or attempted to explain the difference to a newcomer, you know it is odd and often frustrating.

The RSpec and capybara teams worked together to alleviate the confusion in capybara 2.x.

Changes

Upon upgrading to capybara 2.0, capybara will not be available by default in RSpec request specs. Instead, a new type of spec--the feature spec--has been created for use with capybara.

To upgrade to capybara 2.0, you'll need to do a few things:

  • Upgrade rspec-rails to 2.12.0 or greater
  • Move any tests that use capybara from spec/requests to spec/features. Capybara tests use the visit method and usually assert against page.

Alternatively, you can keep using capybara in request specs, but you'll need to manually mix in the methods.

Examples and Recommendations

The rest of this piece has some of my opinions, but opinions I've had good success with in the applications I develop.

Use RSpec request specs to test interactions with your application as a HTTP API. To do so, use methods like get, post, put, delete and assert against response:

# spec/requests/widget_api_spec.rb
describe "widget API" do
  it "allows API clients to create widgets" do
    post widgets_url, widget: { name: "Awesome Widget" }, format: "json"

    expect(response.status).to eq(201) # "Created"
  end
end

Use RSpec feature specs (with capybara) to test your application as a user might interact with it. To do so, use methods like visit and assert against page. You may also use feature instead of describe and scenario instead of it to increase readability:

# spec/features/widget_management_spec.rb
feature "widget management" do
  scenario "creating a new widget" do
    visit root_url
    click_link "New Widget"

    fill_in "Name", with: "Awesome Widget"
    click_button "Create Widget"

    expect(page).to have_text("Widget was successfully created.")
  end
end

Further Reading

Shoutouts

The capybara team, especially Jonas Nicklas, were pivotal in making these changes happen. In fact, my own role was small comparatively. Thank you Jonas!

Thanks to Darren Coxall, Jim Hodgson, and David Chelimsky for reviewing this post.