A performance testing framework for cucumber io.
Cucumber-Performance is a tool that allows multithreaded performance tests using cucumber as runner specification.
Cucumber is a implementation of Behavior Driven Development (BDD). Which uses simple natural language scripts to define a software feature. These executable specifications are written in a language called gherkin. Example:
Feature: Beer Scenario: Jeff dinks a beer Given: Jeff is of age and has a beer And: Jeff opens his beer. When: Jeff takes a sip. Then: Verify he enjoyed it.
These scripts can be used to develop the features themselves but also drive automated tests.
So, you now have a working functional automation test suite. But you want to run a performance test. This would require either rewriting your existing functional tests or copying a bunch of code. Also, you would need to create a performance test harness.
Most likely each team will end up with something that is project specific and doesn't use the existing functional code base.
Cucumber Performance provides a level of automation on top of Cucumber. And is an implementation of a new concept (as far as I know) called Concurrent Behavior Driven Testing (CBDT).
Cucumber Performance provides a means to use your existing functional tests without writing a single line of code. It provides the ability to run performance simulations with support for common load testing features:
It uses a new type of script called Salad. Salad is a re-implementation of Cucumber Gherkin with the focus on performance simulations.
Plan: Bar visit Simulation: Jeff drinks 3 beers. Group: beer.feature Runners: 1 Count: 3
Here is an example plan
Plan: test Simulation: simulation 1 Group test.feature #slices #these values will replace property "value out" |value out| |changed value 1| |changed value 2| #number of threads Runners: 2 #total number of threads to run. Count: 2 #a optional random wait mean for before thread runs tests. #thread will wait between +-50% of this mean RandomWait: 00:00:02 #Will run all groups for the period below Simulation Period: simulation 2 period Group test.feature |value out| |changed value | Threads: 5 #count is ignored in a simulation period Count: 1 #run time Time: 00:00:30 RampUp: 00:00:10 RampDown: 00:00:10
Concurrent Behavior Driven Testing is the methodology of creating functional automation that can be used in concurrent test scenarios. This means coding with the understanding that each functional test could be used in a multithreaded environment.
CBDT requires an automation team to follow strict guidelines when coding functional test cases. Being careful to avoid static variables and race conditions that will cause failures in a multiple-threaded world. This of course requires a larger understanding of programming or a least team leadership that can enforce these guidelines.
It takes a lot of planning to implement Cucumber Perf.
Your functional automation should follow these rules:
Follow directions in wiki to get up and running.
<dependency> <groupId>com.github.mpinardi</groupId> <artifactId>cucumber-perf</artifactId> <version>4.0.0</version> </dependency>
<dependency> <groupId>com.github.mpinardi</groupId> <artifactId>cucumber-perf</artifactId> <version>3.0.0</version> </dependency>
This project is licensed under the MIT License - see the LICENSE.md file for details