Tryouts - v0.8 BETA

Tryouts is a high-level testing library (DSL) for your Ruby codes and command-line applications.

A tryout is made up of one of more drills. The return value of the drill block is compared to the expectations defined by one or more dreams. The goal is to achieve your dreams.

Terminology

• Tryout: a set of drills (think: basketball tryouts)
• Drill: a single test.
• Drill Sergeant: The class responsible for executing a drill.
• Dream: the expected outcome of a drill. There’s always one or more dream per drill.

Installation

    $ gem install tryouts

Writing Tests

The examples below are a complete overview of Tryouts syntax.

Testing Ruby Codes (:api)

    library :gibbler, "../path/to/gibbler/lib"

    tryouts "Common Usage", :api do

      # This drill block should return 3.
      drill "Maths R Us", 3 do
        12 / 4
      end

      # You can specify a method to execute
      # on the return value of the drill block.
      drill "We want a symbol", :class, Symbol do
        orange.methods.first
      end

      # Dreams can also be specified explicitly which is
      # important b/c it's possible to specify multiple.
      dream :class, Array
      dream [:a, :b, :c]
      drill "Should return a list of 3" do
        Letters.list(3)
      end

      # Drills can pass based on an exception too.
      dream :exception, NameError
      drill "Something failed" do
        raise NameError
      end

      # We can even put simple drills on a single line.
      drill 'Small, fast, and furious', 'Muggsy Bogues', :match, /Mug+sy Bogu?es/

    end

Testing Command-Line Applications (:cli)

You can execute drills on command-line applications.

    command :rudy, "path/2/rudy"

    tryouts "Rudy CLI", :cli do

      # We have specified the name of the command above so
      # we only need to specify the arguments in the drill.
      # $ rudy myaddress -e
      drill "can display my IP addresses", :myaddress, :e

      # The drill returns an Array of lines from STDOUT so
      # we can use Array#grep to check the command output.
      # If any lines match, this test will pass.
      dream :grep, /No machines running/
      drill "can call with no arguments"

      # Drills can also be defined with a block which gets
      # executed via Rye::Box#batch. Methods in the block
      # are executed as commands. The return value of the
      # block is considered the test output.
      dream :match, /\d+\.\d+\.\d+\.\d+/
      drill "can execute a block of commands" do
        ret = rudy :q, :myaddress, :e
        ret.first
      end

    end

Running Performance Benchmarks (:benchmark)

You can also use Tryouts to run benchmarks. The tryouts method takes a second parameter which specifies which drill sergeant to use. Below we specify :benchmark so each drill is timed and executed multiple times.

   tryouts "Benchmark examples", :benchmark do

     # We create an Array and store it in a class
     # variable so it's available to other drills.
     drill "Create test array" do
       @@array = (1..100000).map { rand }
     end

     dream :mean, 3.0        # The mean should be <= 3.0 seconds
     dream :sdev, 0.1        # and the standard deviation <= 0.1
     drill("Array sort!") { @@array.dup.sort! }

     # You can also include a dream inline
     drill("Array sort", :mean, 3.0) { @@array.dup.sort }

     # The 3rd argument is the number of times to
     # execute the drill block. The mean and sdev
     # are calculate based on all iterations. The
     # default is 5 and the maximum is 30.
     dream :sdev, 0.1, 15
     drill("Array sort") { @@array.dup.sort }

   end

The drill blocks in a benchmark Tryout return Tryouts::Stats objects. See: Tryouts::Drill::Sergeant::Benchmark

Running Tests

Tryouts comes with an executable called sergeant. This is the drill sergeant that tells you what percentage of your dreams come true.

    $ sergeant

    Gibbler Gazette

     Basic syntax with SHA1
      "Object"                                                              PASS
      "Class"                                                               PASS
      ...
      "Knows when an Hash has changed"                                      PASS

    All 9 dreams came true

The sergeant looks in the current working directory for a tryouts directory and will automatically load all files ending in _tryouts.rb.

    $ ls -l tryouts/
    01_mixins_tryouts.rb
    10_syntax_tryouts.rb
    20_cli_tryouts.rb
    30_benchmark_tryouts.rb
    50_class_context_tryouts.rb

You can also specify specific files to load:

    $ sergeant any/file.rb

Verbose mode gives you some extra information, including the return values from the drills. The more v’s, the more information:

    $ sergeant -v

In quiet mode, the sergeant returns only a PASS or FAIL message (in the case of a FAIL, it will also return a non-zero exit code):

    $ sergeant -q
    PASS

Screenshots

:api Tryouts

from gibbler

:cli Tryouts (verbose)

from tryouts

:benchmark Tryouts (verbose)

from gibbler

News

2009-07-01: Drills for command-line applications

Support for testing command-line applications has been re-introduced in 0.8. See example code further down in this README.

2009-06-26: DSL Syntax Change

The order of dream arguments has reversed between 0.7.1 and 0.7.2. It is now:

• dream :method, ‘Expected value’

This is a return to the original syntax and I think it’s the right way to go because it reads more naturally in the one-liner format:

• drill ‘test name’, ‘return value’, :class, String

BETA Notice

Tryouts is very new (est. 2009-05-19) and has not been vetted by the scrutiny of time. In particular you can expect:

• The test definition syntax may change in future releases.
• Unexpected errors.
• Bugs! I love fixing bugs so if you find one let me know.

On Threads

Tryouts does some funky stuff to make it simple to write tests. This "funky stuff" means that this library is not thread-safe at definition-time. However, once all tryouts files are parsed (or in OO-syntax, once all objects are created), this class should be thread-safe at drill-time.

More Info

• Check out the sourcecodes
• Read the rdocs
• About Solutious
• Inspiration

Thanks

• Everyone at Utrecht.rb and Amsterdam.rb for putting up with my Ruby questions :]
• Sam Aaron (sam.aaron.name) for the early feedback.
• Kalin Harvey for the encouragement.

Credits

• Delano (@solutious.com)

Related Projects

• Context
• Testy
• Expectations
• Zebra

License

See: LICENSE.txt