Config to implement BDD using JBehave and Selenium

This post describes the steps to configure the complete environment and tools to be used to automate BDD tests using JBehave and Selenium
[This is a consolidated version of some earlier posts that deal with some of the topics here; refer other posts from this blog for more details]


OS / Software / Libraries / Version -
1. OS name: "windows 7", version: "6.1", arch: "x86", family: "windows"
i. Even though am having 64bit machine, all the software being used is 32-bit
2. Apache Maven 3.3.9
i. Maven home: C:\Program Files [x86]\Apache\Maven\apache-maven-3.3.9
3. Java 8 (32 bit) - 1.8.0_112
i. Java home: C:\Program Files (x86)\Java\jdk1.8.0_112
4. Selenium 3.3.1
5. Eclipse Java EE IDE Version: Neon
6. M2E Plugin [always update to latest version]
7. Log4J 1.2.17
8. Apache POI 3.15
9. OpenCSV 3.9
10. IE 11 - 32 Bit
     i. Though the parent IE process is 64 bit, but the actual instance is of 32 bit, hence, we need to run with the 32 bit driver for selenium
11. Firefox
12. Chrome

Steps -

1. Install the JDK, set the Environment variables
2. Install Maven
3. Download IDE - Eclipse
4. Download JBehave plugin

Installation and Update via the Eclipse Installer:

Help > Install New Software...

Add the new site location http://jbehave.org/reference/eclipse/updates/

Select JBehave Eclipse feature and follow standard Eclipse installation procedure

How to verify:

Go to file > new > other, and then select JBehave > New Story

If you can see the JBehave or 'New Story' then installation was succesfull.

5. Create your JBehave project -

Refer this link for more details - Create a Maven project for JBehave

1. Create a folder - C:\Automation\BDD\
2. Navigate to this folder via maven
3. Run command - Use the latest version of JBehave. Update the value of this parameter -DarchetypeVersion=<latest ver>. This can be checked from Maven Central

mvn archetype:generate -DarchetypeGroupId=org.jbehave -DarchetypeArtifactId=jbehave-simple-archetype -DarchetypeVersion=4.6.3 -DarchetypeRepository=https://nexus.codehaus.org/content/repositories/releases/org/jbehave/jbehave-simple-archetype/

• groupID: com.automation
• artifactID:bdd
• others as default

This would download an empty project based on jbehave-simple-archetype

  4. Go to the pom dir, then run this

mvn dependency:resolve

This could download a bunch of stuff in your local .m2 dir

  5. Go to the pom dir, and then run the command

mvn eclipse:eclipse

This could download a bunch of stuff too.

6. Import this project in eclipse -

 - Go to File > Import > Maven project

You may get a few errors when you first load the project. Try the below things to resolve.

- Go to Project > Clean

This will rebuild the project

- Use Alt+F5 with 'Force update of Snapshots/Releases' checked

This could resolve most of the errors.

Sometimes we get the following error after importing the default jbehave project in eclipse

• ERROR Description - Plugin execution not covered by lifecycle configuration: org.jbehave:jbehave-maven-plugin:4.0.5:unpack-view-resources (execution: unpack-view-resources, phase: process-resources)
• ERROR Resource Path - pom.xml /bddproject
• ERROR Location - line 70
• ERROR Type - Maven Project Build Lifecycle Mapping Problem

Resolution 1 - This is a very common error, associated with M2E plugin, and usually it goes away if we update the M2E to latest version. 

Resolution 2 - Use the QuickFix action in eclipse to mark the goal to be 'ignored'

If not use this link to figure out what needs to be done - https://www.eclipse.org/m2e/documentation/m2e-execution-not-covered.html

If eclipse does not throw any errors with your new jbehave project, then your project setup is successful, but your setup is far from complete!

7. External Dependencies -

At this point you can try adding/installing other software/libraries that you need for your project, as maven dependencies. Search and copy all the dependency coordinates from the Maven Central directly, nowhere else. Add these new dependencies above the 3 jbehave dependencies already present in the pom.xml

Add dependencies for the following

• Selenium
• Log4J
• Apache POI

Then the run the mvn command to resolve the dependencies

mvn dependency:resolve

Then download the required style-files for the reporting via:

mvn generate-resources

This is required only once, unless you run the clean command.

The tests can be executed via:

mvn integration-test

If this step results in BUILD SUCCESS then your basic project setup is correct and complete

Additional configuration can be done as below...

8. Configure Log4J

9. Add custom configurations for JBehave

JBehave Configuration for BDD

Custom Configurations for JBehave

1.   Meta Filters -

·      Meta filtering is most useful when we want to perform selective running of stories/scenarios.

To enable filtering in JBehave based on Meta tags, just add the following 3 lines in the MyStories class, under the Constructor [MyStories() - created as part of the simple archetype]

List<String> metaFilters = new ArrayList<String>();

metaFilters.add(System.getProperty("metaFilters", "-skip"));

configuredEmbedder().useMetaFilters(metaFilters);

·      Refer these links for more info
Enable Meta Filtering in JBehave
Passing and Reading Run Time Variables via Maven

2.   Avoid @Named annotation –

·      This makes jbehave interpret the name contained between the delimiters as the parameter name and look it up in the parameters provided by the Examples table. The default behaviour of parameter lookup is overridden via the ParameterControls.

·      This helps to avoid the use of @Named parameter annotation in the method definitions, and also avoids different aliases of steps

Refer this link - http://jbehave.org/reference/stable/parametrised-scenarios.html

Add this line in the end of configuration() method –

.useParameterControls(new ParameterControls().useDelimiterNamedParameters(true)

    

                             

3.   Change the delimiters to [] –

·      This would define the delimiters to be used for parameter names. This will enable us to use [ ] instead of the default < >, in parameter names.

Add this line in the end of configuration() method -

.useNameDelimiterLeft("[").useNameDelimiterRight("]"));

MyStories.java:

public class MyStories extends JUnitStories {

private final static Logger log = Logger.getLogger(MyStories.class);

public MyStories() {

configuredEmbedder().embedderControls().doGenerateViewAfterStories(true).doIgnoreFailureInStories(true)

.doIgnoreFailureInView(true).useThreads(2).useStoryTimeoutInSecs(300);

// Custom Config >> Added to enable meta tag based filtering

List<String> metaFilters = new ArrayList<String>();

metaFilters.add(System.getProperty("metaFilters", "-skip"));

configuredEmbedder().useMetaFilters(metaFilters);

}

@Override

public Configuration configuration() {

log.info("setting JB configurations");

Class<? extends Embeddable> embeddableClass = this.getClass();

// Start from default ParameterConverters instance

ParameterConverters parameterConverters = new ParameterConverters();

// factory to allow parameter conversion and loading from external resources (used by StoryParser too)

ExamplesTableFactory examplesTableFactory = new ExamplesTableFactory(new LocalizedKeywords(),
new LoadFromClasspath(embeddableClass), parameterConverters);

// add custom converters

parameterConverters.addConverters(new DateConverter(new SimpleDateFormat("yyyy-MM-dd")),

new ExamplesTableConverter(examplesTableFactory));

return new MostUsefulConfiguration()

.useStoryLoader(new LoadFromClasspath(embeddableClass))

.useStoryParser(new RegexStoryParser(examplesTableFactory))

.useStoryReporterBuilder(new StoryReporterBuilder()

.withCodeLocation(CodeLocations.codeLocationFromClass(embeddableClass))

.withDefaultFormats()

.withFormats(CONSOLE, TXT, HTML, XML))

.useParameterConverters(parameterConverters)

// Custom Config >> This makes jbehave interpret the name contained between the delimiters as the
// parameter name and look it up in the parameters provided by the Examples table

.useParameterControls(new ParameterControls().useDelimiterNamedParameters(true)

// Custom Config >> This would define the delimiters to be used for parameter names. This will
// enable us to use [ ] instead of the default < >, in parameter names

.useNameDelimiterLeft("[").useNameDelimiterRight("]"));

}

@Override

public InjectableStepsFactory stepsFactory() {

return new InstanceStepsFactory(configuration(), new MySteps());

}

@Override

protected List<String> storyPaths() {

return new StoryFinder().findPaths(codeLocationFromClass(this.getClass()), "**/*.story", "**/excluded*.story");

}


}

JBehave Results Report format for BDD Story

How and what the JBehave result report displays, and how to update the format

• All the scenarios mentioned with the 'Scenario:' tag in the story file are counted as 'Total' number of scenarios per story file - it does not matter whether they have GWT steps defined/implemented or not, if there is a 'Scenario:' tag, it would be counted.
• Scenarios without the GWT steps defined in the story file would be counted as 'Pending' and would also be marked as 'Successful'. This makes sense because those scenarios have not failed/excluded technically because their steps are not even defined, forget implemented. Having a 'skip' tag for such scenarios does not make sense and is redundant.
• This would be also true for the scenarios with GWT steps defined but not yet fully implemented; these would be counted as 'Pending' as well. In short, if GWT steps are not defined/implemented fully, the scenario would be marked as pending, and in this case, Meta filtering to skip the test is Not necessary.
• Only those scenarios are counted as 'Excluded' which 1. have the GWT steps defined/implemented  and 2. have the skip filter added against them. Scenarios without GWT steps, even though marked as 'skip-ed' would still be counted as 'Pending' and not 'Excluded', which is correct.
• The following sections are not needed and should be removed - how exactly needs to be added here.
• AfterStories
• BeforeStories
• GivenStory Scenarios

• Run Config for IDE/Maven command-line:

-Dbrowser=FIREFOX -DrunEnv=test1 -DmetaFilters=+Regression-skip-Manual

• This will run only the scenarios with the tag "Regression"
• This will NOT run any scenarios with skip and Manual tags
• We can add as many tags as needed with corresponding +/- notations
• Refer these posts for more detail Passing and Reading Run Time Variables via Maven and Enable Meta Filtering in JBehave

How the JBehave tests should be run -

Its a good idea that the final RunConfig should be tested only via maven command-line and not via any IDE. Also, when we want to run the tests, we should fire the mvn command and not use the IDE. 

This will ensure that we are testing with the same command that we want to put on CI, and also, bring to surface any issues when running via just command-line which is how all the tests are supposed to be triggered.

Bug with JBehave results reports -

Issue - Old results are not deleted from the /target/jbehave/ dir, due to which results which are not part of the current also get included in the latest results. 

When we do a selective run of only the scenarios that we want to run [only those scenarios as per the filter provided], and only they do get run, but if we have run any other stories previously [which were not part of the current selective run] then their results keep lingering around and are present in the final result report - we should have a clean run everytime, but not have to download the dependencies or resources everytime.

Solution - For now, this has been taken care by manually deleting the contents of the /target/jbehave/ dir [the 'view' folder also gets deleted, but it gets recreated when the tests are run] as part of the BeforeStories method. 

Also, we should be just deleting the files, in the target and view folders, and not the other folders and files in these 2 folders.

A better solution would be to figure out the jbehave config that can help delete older results before each run.

• The default format that comes with the simple archetype is not good enough, and is very bland. It is sans colours and has a lot of redundant info.
• To update the Report format, the pom from the simple archetype needs to be updated with the one from CCentric, specially the element with **/*Stories.java has to be changed to **/*Scenarios.java.
• This has to be replaced with the one from CodeCentric [LT also used the one similar to CodeCentric] This is much better than the default one, but still has two issues - 1. The 'GivenStory Scenarios' section still appears in the report, and is always blank - This is not the BeforeStories section, as it BeforeStories appear along with regular stories. 2. The 'Totals' showing the number of stories in on RHS, though this is not a deal-breaker, and is a good enough version to start with.
• There is a third report format that I have seen which does not have 'GivenStory Scenarios' section, nor the incorrect Totals section. So, the goal is to have the report in this format.
• But do not try to build this format by editting the FTL files yourself - its much more time consuming and tricky than you think. Instead try to search the config that will help generate the report in the format that you need.
• Also, there are a few extra options that can be included in the reports [in the order of priority] -
• Attach the screenshots
• Have the 'Jenkins Build Console Log' link to display the log file info
• Attach the test data files - input & output - as a link for each scenario result

Info about FTL files -

• FreeMarker (Template) by FreeMarker ProjectFreeMarker is a 'template engine'; a generic tool to generate text output (anything from HTML to autogenerated source code) based on templates. It's a Java package, a class library for Java programmers. It's not an application for end-users in itself, but something that programmers can embed into their products. This association is classified as Source Code.
• This uses the .FTL files that we have under the view/ftl folder for jbehave.

Conventions to write BDD Story files in JBehave

Writing BDD Stories in Gherkin is easy, but Conventions would help us scale -up and manage them much more effectively.

• Do not create too many feature files for each Jira story/task that you have, instead create feature files based on the functional aspects or features of your application. This will prevent proliferation of too many scenarios spread across too many story files.
• Define a unique parent tag for each feature file that you create. This will be used to run all the scenarios in this story file. Define the parent Meta Tag as per the following format - @PROJECTNAME_FEATURENAME
• Define a unique scenario-level Meta Tag as per the following format - @COMPONENT_FEATURE_TC<scenario-num> This will be used to run only the selected test-scenario from that story.
• Along with the Feature, we may also want to add a tag, for the Jira Story, if needed, like @JIRA_<jiraID>. This can come handy when we only want to run tests based on each Jira Story or Epic
• Define a common tag for different Category of tests, like @REGRESSION, @SMOKE, @API, etc. This will help run all the regression or smoke tests across different story files.
• Meta tags can be defined in upper or lower case, but upper case is preferred
• Do NOT use '-' in any of the meta tags. Using '-' in a tag name will actually not execute that test scenario
• Do NOT have spaces in between in any tag's name, use '_' instead
• Now, all my tests are not automated, and they wont run but they sure should be in the story files. So have additional tags for tests that are not automated, are still work in progress, or you just dont want to run, like below. [All these tags would be used to skip the tests]
• @SKIP - denotes that these tests should be skipped and not run at all when running the JBehave Stories.
• @MANUAL - denotes that the tests will not be automated, and have to be executed manually.
• @WIP - denotes that these tests are not ready to be run in an automated manner
• One of the reasons to include manual tests in the stories as well is to ensure that we have just one place to maintain all our tests, and its easier to report status.

Refer this post to put these conventions to use - How to use Meta Filters in JBehave

Add Meta Filters for JBehave Story

Meta filters in JBehave are very useful when we want to run only small subset of test scenarios across multiple Story files. 

For instance when there are 100 tests spread across 5 different Story files and we want to run just 20 tests (4 tests from each of the 5 stories). If we have Meta-tags defined, we just have to specify the tags for the tests we want to run or skip.

To enable filtering via Meta tags in JBehave, just add the following 3 lines in your Stories.java class, under the Stories() Constructor.

public Stories() {

  configuredEmbedder().embedderControls()
  .doGenerateViewAfterStories(true)
  .doIgnoreFailureInStories(true)
  .doIgnoreFailureInView(true)
  .useThreads(2)
  .useStoryTimeoutInSecs(300);

      //Custom Config >> Added to enable Meta tag based filtering
        List<String> metaFilters = new ArrayList<String>();
        metaFilters.add(System.getProperty("metaFilters", "-skip"));
        configuredEmbedder().useMetaFilters(metaFilters);

    }

@SKIP - this is the meta tag that can now be added to any test to skip its execution.

Refer this post to run JBehave stories by specifying Meta filters via Maven

Create a BDD project via JBehave Archetype

The fastest way to start building and automating BDD Stories via JBehave tool is to have the entire Framework/Project structure created via Maven Archetypes using the jbehave-simple-archetype.

This generates a project structure with all the necessary config and guidelines which help in implementing the Story files right away, and saves us a lot of time in writing a lot of boiler-plate-code.

Just navigate to the dir [lets say, C:\BDD\] where you want to keep all your BDD projects, and then enter the below in the cmd prompt to download the project via mvn

mvn archetype:generate -DarchetypeGroupId=org.jbehave -DarchetypeArtifactId=jbehave-simple-archetype -DarchetypeVersion=4.6.3 -DarchetypeRepository=https://nexus.codehaus.org/content/repositories/releases/org/jbehave/jbehave-simple-archetype/


Project Specification - 

• When asked to enter a groupID, enter like: com.company.department
• When asked to enter an atifactID, enter like: bdd-jbehave
• When you are asked to define value for version just press enter to select the default version as 1.0-SNAPSHOT
• Let the default package be same as groupId so press Enter and confirm the details by typing “y” and then enter as seen below.
• After successful completion you will see a folder named after your artifactId in your workspace which contains a pom.xml and also src folder.
• This would also contain a sample MyStories.java and MySteps.java files, with default config

We get the following major classes/files

• MyStories.java - contains all config to run JBehave stories
• MySteps.java - contains implementation of all steps
• my.story - contains all the scenarios for a story
• pom.xml with all the dependencies and build config

That's it, your maven project is created. You can use this as a template to add your own story, steps and other classes.


To open the project in IntelliJ - 
Select the option to import a new project from the IDE, and then specify the pom.xml for the newly created jbehave project.
On the subsequent windows, select the option to automatically download the Source and Documentation. This will automatically download all the packages that you import in your classes, as and when they are used, and will prevent package not found errors.
Ensure that the JBehave plugin is installed in IntelliJ, before importing the project.
Ensure that you select the 'Sources' and 'Documentation' checkboxes while importing, so that all the dependencies get downloaded.

[Optional step] To open the project in Eclipse - 
Now cd into the folder jbehave which contains the pom.xml and then execute the following command:
    mvn eclipse:eclipse
This command will make your project an eclipse project which can be opened through eclipse by adding the relevant .classpath and .project files.

That's it, you are ready to do BDD, just behave!