Subscribe (RSS)In preparation for an article I’m writing on RSpec, I decided to revisit setting up my preferred RSpec development environment in Windows. I was ever so quickly reminded of why I made the switch to Linux in the first place.
The Background
Let me get everyone caught up to speed. RSpec is of course a Behavior Driven Development framework for Ruby and Ruby on Rails. The Behavior Driven Development process itself involves writing a specification (test), running that specification (which will initially fail), writing the functionality to satisfy that specification, and finally running the specification again which should then pass. This process continues until you’ve implemented all the functionality in the application you are developing.
Now, if you run all the specifications manually, this can get really bothersome very quickly. This is particularly aggravated in a Windows environment, since all the common Rails tasks (rakes, generations, migrations, etc.) seem to take twice as long as they do in a Linux or OSX environment. Even under Linux or OSX, though, the frequency of running specs tends to slow down the production cycle incredibly when run manually.
Thankfully, there’s ZenTest and more specifically the autotest functionality of ZenTest. Autotest watches for file changes and then automatically runs any specs or tests that have changed. Combine this with the color-coding provided by Redgreen and a notification system (Growl, KDENotify, Libnotify) and you have a very effective Behavior Driven Development environment.
The Specs
There is strong indication that all of the issues I have noticed are version dependent (i.e. broken in newer versions), so it makes sense, then, to provide the specs of the various gems and plugins I’m using for these tests:
- Windows XP SP2
- Ruby 1.8.6
- Rails 1.2.3
- ZenTest 3.6.1
- RSpec 1.0.8
- Spec:Rails 1.0.8
- Redgreen 1.2.2
- Ruby-Snarl 0.0.8
- Snarl 1.6
The Issues
But then there’s Windows. On the surface, it looks as though the same should be possible here. We can develop Rails apps in Windows. RSpec, Autotest, and Redgreen are operating system agnostic. There’s even a fairly mature notification system for Windows in Snarl. So what’s the problem? Well, when you put all of these together, it explodes.
I did a good bit of trial and error debugging tonight trying to pin down the exact problem, and while I was ultimately unsuccessful, I was able to notice certain trends which, to a more apt Rubyist, may spark an idea for a solution. So, for posterity, I want to post my findings in the hope that someone can figure out what is wrong.
Autotest and Redgreen
A great deal of work has been done to integrate RSpec and Autotest, and in Linux and OSX, the two play very nicely together with little to no configuration. In Windows, however, Autotest initially hangs when trying to read the specs. The only way I could find to get Autotest to actually run the specs through, was to remove everything from the spec.opts file in the spec directory of the app in question.
Unfortunately one of the lines in the spec.opts file is the “–colour” option which sets up the color-coding of passes and failures. I did quite a bit of research online, and found many articles which tell you that you must install the redgreen gem in order to get color-coding on your tests. However, this seems to be outdated information since both Autotest and RSpec seem to come with this functionality out of the box, now (assuming you have installed the win32console gem to make it work in Windows). Indeed, even without the redgreen gem installed, if I simply ‘rake spec’ the color-coding is there.
So, after realizing that I could not rely on this out of the box functionality in Windows, I tried the alternate route of installing the redgreen gem and then requiring it in a ‘.autotest’ file in my RAILSROOT (require ‘redgreen/autotest’ along with require ‘Win32/Console/ANSI’). However, this resulted in autotest erroring-out on load complaining of an ‘Invalid Option: -O’.
I then realized that Autotest has a redgreen plugin of its own, which can be reference as flip-flop of the previous require for the redgreen gem (i.e. require ‘autotest/redgreen’). So I uninstalled the redgreen gem, changed the require in .autotest, and ran autotest again. This time it loads fine, but without color-coding still. Finally, just to be thorough, I reinstalled the redgreen gem in the hopes that it still somehow provided some functionality for the autotest plugin of the same name. However, this two proved fruitless. Autotest still ran successfully, but still without color-coding.
Autotest and Snarl
At this point, I gave up on having nice color-coded test results. Snarl, I thought, can be setup to provide all the visual indication necessary, so I began to work on the Snarl side of things. I already had Snarl installed, so I went ahead and installed the ruby-snarl gem which sets up the connectivity. Then, I opened my .autotest file once more and added the require for snarl (require ‘autotest/snarl’). Time to test it out, so I went to my command line, ran autotest once more, only to see Snarl pop up with a notification window saying all tests had passed, while autotest simultaneously informed me that I had two failures. This is pretty much where I stopped because I have no idea where to even begin to troubleshoot Snarl.
The Summary
In the end, the only thing I could get to work was plain-jane autotest, which is better than nothing but not by much. Through all the digging I have done online, the only thing I can determine definitively is that at one point it was possible to integrate all of this in Windows. However, this was before both Autotest and RSpec added in their own implementations of Redgreen. The Snarl issue is even stranger, and I’m not sure if it is a related issue or something entirely separate.
The ironic part is that it appears that it has been so for quite some time, and no one is either using Windows to notice it, or cares enough to fix it if they have noticed it. Which gives even greater weight in my mind to simply abandon Windows for Rails development, preferring Linux or OSX. Obviously, OSX would be the optimum choice, but for those of us still on PCs, Linux is as close as you can get.
Tags: autotest, bdd, redgreen, rspec, ruby on rails, snarl, windows
Chris Says:
September 14th, 2007 at 4:10 pm
I am also at the ready to abandon Windows for rails development for the same reason. I’ve spent 3 days trying to get this setup working and I’m not even this far. I can’t even get autotest to run. When I try I get an error that makes it seem like autotest can’t find my test files even though they are there and running “rake test” works perfectly.
I uninstalled everything and started with ruby 1.8.6 and installed the same versions of everything else you are using here. I execute “rails test_site” then “ruby script/generate scaffold person” to get some test files and when I run autotest I get the following.
loading autotest/rails
c:\ruby\bin\ruby -I.;lib;test -rtest/unit -e “%w[test/unit/person_test.rb].each { |f| require f }” | unit_diff -u
c:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require’: no such file to load — test/unit/person_test.rb (LoadError)
from c:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require’
from -e:1
from -e:1:in `each’
from -e:1
I’m pleading with my boss to get me a mac, I even told him I would come in and set it up on the weekend, but if that proves fruitless, I would at least like to get autotest working even if I can’t have nice gui notifications or colored text.
chrisdpratt Says:
September 16th, 2007 at 9:39 pm
Since, I wrote this post, I revisited setting up my Windows environment one last time (I purchased e-Text Editor, a TextMate clone for windows, and would love to be able to actually use it). This time, however, I approached it from a different standpoint, using Cygwin.
With Cygwin, I am able to have autotest happily running my specs with colored output. The only thing missing is Snarl integration, but 1) I’m not sure if it’s even possible to integrate Snarl with Cygwin and 2) I’m not sure where to begin if it is.
Nevertheless, this has allowed me to productively use Windows for Rails development, for the most part. Everything still runs slower than in Linux and OSX, but having TextMate like functionality makes up for that.
Anyways, since others seem to be having the same issues setting up Cygwin properly can be a bit tricky, I think I’ll be posting about it real quick here. I’ll try to get something out tonight or tomorrow.
weyus Says:
September 21st, 2007 at 11:02 am
Hmmm. I see the same thing. What I notice is that autotest –rails and rake spec are clearly not doing the same thing. While I am on the verge of buying a Mac anyway because my system is so fscking slow, I’d like to try and power through this because it feels like it should be debuggable. I’ll post any findings.
weyus Says:
September 23rd, 2007 at 3:45 pm
I discovered that my autotest was not actually running the specs because of a problem with the way that they were being called. That is, if I ran autotest, it would construct appropriate spec commands but they they wouldn’t actually be run. Running the exact same spec command in a command window would work fine.
Turns out the command to run the spec commands looks like:
c:\ruby\bin\ruby -S script/spec –options spec/spec.opts spec/helpers/admin_hel
per_spec.rb spec/models/evaluation_request_spec.rb spec/models/right_spec.rb spe
c/controllers/admin_controller_spec.rb spec/models/person_spec.rb
In autotest.rb line 209 (ZenTest 3.6.1), we see:
open(”| #{cmd}”, “r”) do |f| …
This is how autotest actually executes the spec command. Turns out that the single \ characters will causes this command to fail. Testing with irb, I found that changing the \ to / or escaping the \ with another \ would allow this to work.
I don’t know yet whether this is a bug in autotest or whether autotest is relying on something lower level to generate the path to the ruby executable. But either way the command needs to be constructed in such a way that it can be executed.
I will post anything more if I find it here.
Wes Gamble Says:
September 24th, 2007 at 6:19 pm
The bug is in autotest, in a method that it uses to construct the path to the ruby executable.
Jamal Says:
October 7th, 2007 at 6:39 am
Same goes here,
I get too many errors when I tried to get autotest to run on Windows.
Vitor Says:
October 8th, 2007 at 11:53 am
One more to hate windows and autotest (not) working together.
darelf Says:
November 13th, 2007 at 12:04 am
i succeeded in having colored autotest output and snarl popups in Windows (not cygwin) with next raugh steps:
content for .\.autotest
require ‘autotest/snarl’
content for .\ansipipe.rb
require ‘rubygems’
require ‘win32console’
@io = Win32::Console::ANSI::IO.new()
until $stdin.eof? do
line = $stdin.gets
@io.puts line
end
@io.flush
in .\vendor\plugins\rspec\lib\spec\runner\formatter\base_text_formatter.rb hack method colour=
def colour=(colour)
@colour = colour
# winhack
# begin
# require ‘Win32/Console/ANSI’ if @colour && PLATFORM =~ /win32/
# rescue LoadError ;
# raise “You must gem install win32console to use colour on Windows”
# end
end
`snarl` must be installed.
Gems: `win32console`, `diff-lcs`, `ruby-snarl` must be installed.
`Redgreen` gem is not needed.
`ruby` and `rake` must be included in $PATH
run autotest from the root of project in command line as:
autotest | ruby ansipipe.rb
not delicate but it works for me
Chris Foley Says:
January 11th, 2008 at 9:57 pm
Looks like in rspec 1.1.1, darelf’s hack to base_text_formatter.rb has been moved to .\vendor\plugins\rspec\lib\spec\runner\options.rb around line 104
Thomas G. Says:
February 13th, 2008 at 3:03 pm
Hi,
Using rSpec and Zentest as of today’s version, I am having the same problems: no color in the shell, and snarl telling me that all tests pass when they do not… And moreover the tests are quite slow compared to OSX or Linux boxes…
vknightbd Says:
May 30th, 2008 at 6:50 am
I also managed to hack it so that I got colored autotest and snarl to work outside of cygwin for rspec-1.1.4, zentest-3.9.2, win32console-1.0.8
change the require in line 130 from ‘rubygems’ to ‘RubyGems’ in rspec-1.1.4\lib\spec\runner\options.rb (not sure if this is significant)
add the following to the top of ZenTest-3.9.2\lib\autotest.rb (for color)
require ‘Win32/Console/ANSI’ if RUBY_PLATFORM =~ /win32/
Snarl is broken since ZenTest is not parsing Rspec’s output correctly.
replace the following regex in the initialize method in autotest.rb to match the Rspec output.
self.completed_re = /\d+ examples(, \d+ failures)?(, \d+ pending)?/
self.failed_results_re = /^\s*\d+\)\s*[^\n]*(?:Failed|Error)[^\n]*\n([^\n]*)\n([^\n]*):\d+:/i #ugly
replace the consolidate_failures method in autotest.rb (or in my case, merb_rspec.rb) to use the new array generated by the above failed_results_re.
def consolidate_failures(failed)
filters = Hash.new { |h,k| h[k] = [] }
failed.each{|error_message, spec_file| filters[spec_file] << error_message}
return filters
end
I also made some extra changes to ZenTest-3.9.2\lib\autotest\snarl.rb and modified the red hook and added an all_good hook.
Autotest.add_hook :red do |at|
failed_tests = at.files_to_test.inject(0){ |s,a| k,v = a; s + v.size}
failed_files = at.files_to_test.size
snarl “Tests Failed”, “#{failed_tests} tests failed of #{failed_files} specs tested.”, :red
end
Autotest.add_hook :all_good do |at|
snarl “All Tests Passed”, “Tests have fully passed”, :green
end
Anlek Says:
June 11th, 2008 at 9:20 am
vknightbd had the right idea, but my snarl worked fine as is, the only thing I had to do based on what he wrote was:
—–
add the following to the top of ZenTest-3.9.2\lib\autotest.rb (for color)
require ‘Win32/Console/ANSI’ if RUBY_PLATFORM =~ /win32/
—–
Doing the first step with the rubygem actually created 3 errors when running auto test (saying that rubygem was already included or something like that)
and I didn’t do the bottom two steps because like I said, snarl was working fine for me.
Bryan Says:
September 24th, 2008 at 6:50 pm
Following the example above, to get autotest to output with color, without hacking released files:
—–
I put the following in my .autotest:
require ‘Win32/Console/ANSI’
—–
I never had any issues getting autotest or snarl to work.