Continuous Testing (i.e. re-running tests immediately after changing code) really helps improve the edit -> compile -> test loop. Currently doing build & rerunning the unit tests in Scala in IDEA on a typical maven project is pretty slow.
I tried using Infinitest as an IDEA plugin; which looks promising though it is a little slow, only kicks in when you select 'build' and the UI doesn't work too great when using ScalaTests.
I've blogged about how its really easy to use continuous testing with
sbt (
simple build tool); particularly if you already have a maven build...
http://macstrac.blogspot.com/2010/01/using-sbt-on-your-scala-maven-project.htmlso its easy to use sbt in incremental mode; it incrementally compiles, loads & reruns tests really quickly. (Usually only a couple of seconds per change - much faster than just a build takes on IDEA on Scala projects). e.g. to rerun unit tests on my current project its about 20 seconds or so in IDEA using its make; in sbt its about 3 seconds
The only downside is when there are failures, there's no fast way to navigate them from IDEA. I basically want to see the usual unit test window in IDEA, which updates dynamically as sbt reruns tests; then any stack traces get shown in the IDEA UI so I can easily navigate to points in stack traces.
So I was wondering; what if there was some kind of sbt plugin for IDEA; where we can get sbt test runner to try connect to some network port or URI - or write files to some well known localtion which the sbt IDEA plugin could see, and it would just grab the tests run, their results and the console output and render the usual test runner window? i.e. so we can watch the results of sbt inside the IDEA UI and click/navigate to failed tests and stack traces?
Basically its another kind of test runner which actually doesn't do anything other than listen to what sbt it doing (and you run that separately in a shell - then you get smart completion & colour coding in the shell)? The other option is you run sbt from inside IDEA - though not sure the completion would all work etc? Either way - the main issue really is just grabbing the results of running tests from SBT so we can render them nicely to the IDEA UI so scala developer get super fast continuous testing feedback
TeamCity Changes
Then you can create an Application in IDEA where you add
(the last line turns off the colors from the terminal)
Then when running in continuous testing mode by entering the following into the shell "~ test" then you get test failure stack traces nicely formatted so you can click on the lines to get to the code.
You lose tab completion in the shell though.
The main issue is its not as easy to read/navigate the console output as it is the JUnit UI (particularly switching between different failures)
e.g. "== test-start ==" is the start of a test run and "== test-finish ==" indicates the end. Then within there, you just look for "[info] == nameOfClassBeingTested ==" to represent a test class being ran, then a test name working looks like
[info] Test Starting: someName
[info] Test Passed: someName
and a failure
info] Test Starting: someName
[error] Test Failed: someName
[error] Expected someError....
Then rather like the JUnit UI, we can capture the log and put it into a test class/method specific area so you can see the tests ran, the ones that failed and for failed tests zoom in on the console output etc
http://github.com/jstrachan/webbytest/blob/master/src/main/scala/webbytest/HtmlTestsListener.scala
this page describes how to add it to your sbt project....
http://github.com/jstrachan/webbytest
The only downside with this approach is you have to add the listener to your sbt project. I'm not sure if there's a way to add a listener dynamically without changing the project (e.g. specifying the IDEA listener as a command line argument to the sbt command).
Still it'd be an awesome first step, if there was an IDEA TestsListener folks could add to their projects - then we can figure out how to make that plugin an optional thing (so it can be specified as an environment setting or command line argument or something)
It can be used to compile the Scala projects and execute any other SBT commands from within IDEA.