13.4.17

Evaluation of Automation Tools

Sometimes it feels so good to just past judgments! In fact I wanted to title the post judgement on tools, but people get real sentimental these days.

So, with all due respect to whomsoever it may concern....lets see what some of the OS tools prevalent in the market are good at, or mostly, lack.

  1. External Libraries
    1. Web Scrappers - Can these help read the html page, and give the data you want, in key:value pair?
      1. JSoup
        1. This is the only good option for Java, which is just a jar
      2. BeautifulSoup
        1. This is only for Python, but very capable
    2. XML Parsers
      1. Can these help read the XML, and give the tag:data you want, in key:value pair?
  2. White Framework
    1. supposedly to automate desktop apps, written in Java
    2. http://codoid.com/white-framework-cheat-sheet/
  3. YML Files for Test Data
  4. Protractor for AngularJS 
    1. Works with Se as well, and may have better Sync options. 
    2. Also, it is useful in building object locators for those custom tags in AngularJS
  5. Capybara 
    1. this works only with ruby
  6. Zeus
  7. Liverload
  8. Zephyr 
    1. Looks like this only provides flashy QA metrics, no other real use
  9. TestingWhiz Community edition
  10. Guice with Se
  11. VelocityDep
  12. CodedUI
    1. Comes bunndled with VS from MS, needs C# and OOOPs, can work on Web and Windows forms
    2. May have better Obj ID process, but Obj Management is difficult as no OR
  13. Freemarker library
  14. Wrappers - Discarded [DO NOT WASTE TIME ON WRAPPERS. They do not add anything new to the core Se code base, just propose to make it easier. If the wrapper goes out of business, so do you. Instead, focus on learning new generic and widely used Libraries [jars] that can add more features and make your life easier.]
    1. Robot framework
      1. This is making a lot of noise these days, may be some good, but not keen on using this
    2. SeLion
      1. Enabling Test Automation in Java. 
      2. SeLion builds on top of TestNG and Selenium to provide a set of capabilities that get you up and running with WebDriver in a short time. 
      3. It can be used for testing web and mobile applications. 
      4. Seems to be Free, not sure
      5. http://paypal.github.io/SeLion/html/documentation.html#getting-started
    3. Tellurium Discarded
      1. Its NOT Free!
    4. Fluentlenium - Discarded
      1. A wrapper framework for Se
      2. Looks like it supports only CSS not XPath
      3. http://awesome-testing.blogspot.in/2016/01/introducing-fluentlenium-1.html
    5. Selenide Discarded
      1. This is one more wrapper, and not widely used, has very different commands from my current PO framework
    6. Bromine Discarded
      1. Bromine is an open source QA tool that uses selenium RC as its testing engine. It is a test management tool for Se, and more so only for SeRC Tests, and aims to replace HPQC!
      2. Tests have to be written in java, and then uploaded to Bromine, which provides just the execution env for all the uploaded tests
      3. Bromine is a web application, and needs to be hosted on a separate server - yeah, good luck getting a new server just for this!
      4. http://www.methodsandtools.com/tools/tools.php?bromine

Eclipse shortcuts

If not in life, at least in eclipse, there are some shortcuts!

  • Ctrl M - maximize
  • Ctrl Shift B - breakpoint
  • Ctrl Shift / - collapse all
  • Ctrl Shift * - expand all
  • Ctrl Shift F - auto-format
    • Increase the max line width to 1000 to have all the code in a single line
    • Go to Window > Preferences > Java > Code Style > Formatter > Edit
  • String formatting
    • Go to Window > Preferences > Java > Editor > Typing
      • Escape text when pasting into string
  • Ctrl / - Toggle Line Comment
  • Alt Shift R - Rename Variable
  • Ctrl 1 - Show error resolution
  • Alt Shift Up Arrow - Select entire string
  • Ctrl Shift P - Jump to Opening/Closing Brace
  • Ctrl K - jump to the next instance
  • Alt Shift Up arrow - Select entire string in quotes
  • Go to Source/declaration - F3
  • Enable auto activation of content assist:
    • Go to Window > Preferences > Java > Editor > Content Assist > Auto activation triggers for Java
    • Add the string .abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_
    • This would have to be done individually for editors of each language


Chrome shortcuts -

  • Open chrome browser without authentication/security warnings
    • chrome –ignore-certificate-errors &> /dev/null &

Troubleshooting hacks, Jugaad


1.              Guice Provision Error -
Cause - Happens when Surefire plugin is initiated on Maven 3.0.5 (which is too old now). This usually happens on Jenkins/TC when the default settings for Maven are used.
Resolution - Use latest version of Maven and specify the same in Jenkins' Maven Settings too

2.              SurefireBooterException -          
To resolve, add this config in the POM, to set useSystemClassloader to false:
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <useSystemClassLoader>false</useSystemClassLoader>
    </configuration>
</plugin>

3.              Run Maven commands without changing dir –
We don’t need to ‘cd’ to the directory containing the pom every time we want to run a mvn command, we can fire the mvn command from anywhere as long as we give the path to the pom like below:
Syntax: mvn –f <fullpath-to-pom> <goals> -D<params>
Sample: mvn –f C:/Automation/keyword/pom.xml test –Dthread1=Test1
It would be good to not have any spaces in the path so as to avoid escape char.
Use / instead of \  in the path.

4.              Invoke CMD via VbScript –
If you want to invoke the CMD utility automatically with certain parameters then use the below snippet:
Set oShell = CreateObject(“WScript.Shell”)
cmndToRun = “mvn –f C:/Automation/keyword/pom.xml test –Dthread1=Test1”
oShell.Run “cmd.exe /k “ & cmndToRun
To keep the CMD window open use /k after cmd.exe or use /c to close it.

5.              StackOverflowError –
Happens due to infinite recursion. For example when a method invokes itself during its execution or where one class object is instantiated under another class recursively. We will not get any compile-time Errors, but at runtime we will get this Error.

6.               Maven Compilation Error - package <name> does not exist
Resolution - Sometimes old or un-used packages are not found after updating versions of some libraries, which causes compilation Errors. Fastest solution is to delete these unwanted packages from Java files.

7.              Log4J package is not getting imported in the classes, hence, not able to initiate logging.
Steps to troubleshoot - A combination of these resolved this, after multiple iterations
·       Tried mvn dependency:resolve - It got successfully downloaded when resolving dependencies via mvn.
·       Delete local repo and re-download all dependencies from scratch - Even after deleting the old local repo, and rebuilding the same from scratch, it does not work
·       'Cleaned' the eclipse project - it resolved all Errors, but still Log4J is not getting imported.
·       Delete '.lastUpdated'  files from local repo
·       For Windows cd (change directory) to <user-directory>\.m2\repository and execute this command:
for /r %i in (*.lastUpdated) do del %i
·       Now update dependencies again.
·       You could also get Errors like: [Could not find artifact org.apache.logging.log4j:log4j:jar:2.6.1 in central (https://repo.maven.apache.org/maven2) -> [Help 1]]
·       Run mvn eclipse:eclipse - This could cause the following Error, visible only on eclipse, not in maven: [The project was not built due to "Resource already exists on disk: '/bddproject/target/classes/log4j.properties'.". Fix the problem, then try refreshing this project and building it since it may be inconsistent]. To resolve it, run mvn clean, as it will delete the target folder, where this Error was. Then go to eclipse and do Project > Clean. Now, all Errors should be resolved.

8.              Get 'failed to load jvm' Error when running eclipse
            Try restarting the machine, it gets resolved sometimes

9.              Even though the default story steps[in myStory] have been implemented, while running the MyStories class, they still come up as @Pending in the results.
Cause - The Pending annotation was already imported by default in the default MySteps class, which was marking all the steps as pending.
Resolution - Delete that import statement for Pending, and re run the test, it worked and all steps were Green/Run/Passed
Also, if now I add a Pending annotation but do not use it, it still runs the remaining steps, as it should run.
 
10.          Getting junk lines being reported in the console with the freemarker log -
Like - "Jul 05, 2016 1:17:56 AM freemarker.log._JDK14LoggerFactory$JDK14Logger info"
If Log4J works, then this is not needed

11.          Run via mvn is Erroring out -
mvn clean install - this command Errors out
Error - [Error] Failed to execute goal org.jbehave:jbehave-maven-plugin:4.0.5:run-stories-as-embeddables (embeddable-stories) on project bddproject: Execution embeddable-stories of goal org.jbehave:jbehave-maven-plugin:4.0.5:run-stories-as-embeddables failed: A required class was missing while executing org.jbehave:jbehave-maven-plugin:4.0.5:run-stories-as-embeddables: org/apache/log4j/Priority

12.          The simple-archetype comes with the default jbehave report template, which needs to be fixed

13.          Even though the M2E plugin is downloaded and installed, it does not show up in eclipse - there is nothing for maven

14.          Getting the following Error while running dependency:resolve command
Error - Failed to collect dependencies at org.jbehave:jbehave-core:jar:4.0.5 -> com.thoughtworks.xstream:xstream:jar:1.4.7:
Cause - Looks like the command to download the dependencies was getting timed out, as it worked well when the internet connection was strong
Resolution - Ran the dependency:resolve command again, and it was successful, without any Errors

15.          Getting the following Error when deleting and re-importing the bdd project
Error - unbound classpath variable 'm2_repo
Cause - Eclipse is not able to locate the path of the local mvn repo
Resolution - The below steps work to solve this issues
·       Open the Eclipse Preferences [Window - Preferences]
·       Go to [Java - Build Path - Classpath Variables]
·       Click New and set its name as M2_REPO
·       Click Folder and select your Maven repository folder. For example, my repository folder is C:/Users/user/.m2/repository
·       Rebuild the Project.

16.            No need to run these commands
mvn compile
mvn clean install
mvn clean

17.          Error - archive for required library cannot be read in eclipse
·       This generally happens when you are importing projects, which has external jars [added via maven POM or via direct import]
·       The first thing to try is delete those external jars and their folders, and then re-import them
·       Then in Eclipse, go to Project > Clean Project [Ensure that Build Automatically is checked]
·       If this does not resolve, then see if the jars got corrupted during copy/import, then replace them with original/valid jars

18.          Avoid having multiple versions of the same jars in the projects - only have the required version and delete the rest.

19.          If you get Errors like 'Source Not found' or 'Attach source', or 'NoClassDefFoundException' it generally means some jar is missing and not there in your build path, so find that jar, and just add it to your build path

20.          Split method in Java has a bug!
·       When we use a split function, ideally, if we don’t specify any limit, it should return all the tokens in the string, but it does not, if you have multiple delimiters in the end with empty tokens. 
·       For example in a | delimited message ("ASDAS|ASDASD|AA||||ASS|||||"), the last empty tokens would be ignored.
·       To fix this use the limit as -1
·       String[] token = sampleMsg.split("\\|" , -1);

21.          Always use string.isEmpty() method to check if the string is empty or not. 
1.    Never use null or any other method. Even if the variable is not a String, convert it to String via toString and then use isEmpty().
2.    Though a lot of people would frown upon this idea, but its simple, effective, and very easy to remember and can be implemented by a rookie in your team
22.          Ensure that you use JDK [and not JRE] in your Project Build Path

23.          Apache POI –
When adding apache poi in the dependency tree ensure to add dependencies for "poi-ooxml" and "poi-ooxml-schemas" as well, as some of the base classes for apache poi use these jars, and otherwise we would not be able to use certain classes like XSSF.

24.          If QCUtils is not working in UFT
·       Try checking the Registry values for this key.
·       'HKEY_CURRENT_USER\Software\Mercury Interactive\QuickTestProfessional\MicTest\QEEE'. 
·       This key had the parameter 'ExternalExecutionSupported', so either set it to Yes or delete it.
 
 
Other Hacks -
  • Error: Could not find PKIX Certificate Path when connecting to Artifactory.
    • Problem: When trying to run any maven commands on windows machines, sometimes we get this error where we are not able to connect to Artifactory or any Central Repo in Enterprise setups. This error will not come on your home computer but its one of the perks of working in a big Co.
    • What its not: This problem is not related to your Artifactory credentials or API Keys, or git or bitbucketor even the maven Settings.XML; although thats what you might be lead to think.
    • Cause: The problem is related to outdated Java Security Certificates or the use of incorrect ones. This happens when the java pkg gets upgraded or the one that you currently have installed does not have the required certificates. So the solution really lies in updating the security certificate file (cacertificates file in jdk dir).
    • Sol 1: Manually find and download the latest certificate and then update the cacertificates file, and then import them via the usual 'keystore' import command that you can easily google. The prob with this approach is that it needs Admin rights to edit the cacertificates file, and you will not get that ever in a big Co - another perk. So this method is DOA.
    • Sol 2: If your jdk package has recently been upgraded, then you might be lucky enough to get the latest java cacertificates files which will hopefully have the correct certificates added, and you will have to re-point your JAVA_HOME and M2_HOME and PATH variables to this new jdk pkg.
      • But that also needs Admin rights, so you will not be able to do that also. Sometimes some Cos have support teams that give you temp Admin rights, which might save the day for you, if not, read on...
      • What you can do is reset the Env Variables like JAVA_HOME and M2_HOME and PATH to new values for the current session via CMD prompt. This will work only till the time this CMD prompt is open, and all changes will be lost when you close it, and you will have to re-do these. The steps are:
        • SET JAVA_HOME=<new path>
        • SET JDK_HOME=<new path>
        • SET PATH=<new path>;%PATH%
        • Remember to append to the PATH variable otherwise it will overwrite and remove all the other values in it.
        • No need to change the variables for Maven
        • This should point your current session to the new jdk pkg folder which has the correct certs file.
    • Sol 3: Create a new folder for JDK pkg where you would have admin rights and then re-point all the variables, including Maven based, to this new folder. This approach would be helpful if you are trying to update the existing certs file with the new certs



12.4.17

Config to implement BDD using JBehave and Selenium

This post describes the steps to configure the complete environment and tools to be used to automate BDD tests using JBehave and Selenium
[This is a consolidated version of some earlier posts that deal with some of the topics here; refer other posts from this blog for more details]


OS / Software / Libraries / Version -
1. OS name: "windows 7", version: "6.1", arch: "x86", family: "windows"
i. Even though am having 64bit machine, all the software being used is 32-bit
2. Apache Maven 3.3.9
i. Maven home: C:\Program Files [x86]\Apache\Maven\apache-maven-3.3.9
3. Java 8 (32 bit) - 1.8.0_112
i. Java home: C:\Program Files (x86)\Java\jdk1.8.0_112
4. Selenium 3.3.1
5. Eclipse Java EE IDE Version: Neon
6. M2E Plugin [always update to latest version]
7. Log4J 1.2.17
8. Apache POI 3.15
9. OpenCSV 3.9
10. IE 11 - 32 Bit
     i. Though the parent IE process is 64 bit, but the actual instance is of 32 bit, hence, we need to run with the 32 bit driver for selenium
11. Firefox
12. Chrome

Steps -

1. Install the JDK, set the Environment variables
2. Install Maven
3. Download IDE - Eclipse
4. Download JBehave plugin

Installation and Update via the Eclipse Installer:

Help > Install New Software...
Add the new site location http://jbehave.org/reference/eclipse/updates/
Select JBehave Eclipse feature and follow standard Eclipse installation procedure
How to verify:
Go to file > new > other, and then select JBehave > New Story
If you can see the JBehave or 'New Story' then installation was succesfull.

5. Create your JBehave project -

Refer this link for more details - Create a Maven project for JBehave

  1. Create a folder - C:\Automation\BDD\
  2. Navigate to this folder via maven
  3. Run command - Use the latest version of JBehave. Update the value of this parameter -DarchetypeVersion=<latest ver>. This can be checked from Maven Central
mvn archetype:generate -DarchetypeGroupId=org.jbehave -DarchetypeArtifactId=jbehave-simple-archetype -DarchetypeVersion=4.6.3 -DarchetypeRepository=https://nexus.codehaus.org/content/repositories/releases/org/jbehave/jbehave-simple-archetype/

  • groupID: com.automation
  • artifactID:bdd
  • others as default
This would download an empty project based on jbehave-simple-archetype
  4. Go to the pom dir, then run this
mvn dependency:resolve
This could download a bunch of stuff in your local .m2 dir
  5. Go to the pom dir, and then run the command
mvn eclipse:eclipse
This could download a bunch of stuff too.


6. Import this project in eclipse -

 - Go to File > Import > Maven project
You may get a few errors when you first load the project. Try the below things to resolve.
- Go to Project > Clean
This will rebuild the project
- Use Alt+F5 with 'Force update of Snapshots/Releases' checked
This could resolve most of the errors.
Sometimes we get the following error after importing the default jbehave project in eclipse

  • ERROR Description - Plugin execution not covered by lifecycle configuration: org.jbehave:jbehave-maven-plugin:4.0.5:unpack-view-resources (execution: unpack-view-resources, phase: process-resources)
  • ERROR Resource Path - pom.xml /bddproject
  • ERROR Location - line 70
  • ERROR Type - Maven Project Build Lifecycle Mapping Problem
Resolution 1 - This is a very common error, associated with M2E plugin, and usually it goes away if we update the M2E to latest version. 
Resolution 2 - Use the QuickFix action in eclipse to mark the goal to be 'ignored'
If not use this link to figure out what needs to be done - https://www.eclipse.org/m2e/documentation/m2e-execution-not-covered.html
If eclipse does not throw any errors with your new jbehave project, then your project setup is successful, but your setup is far from complete!


7. External Dependencies -

At this point you can try adding/installing other software/libraries that you need for your project, as maven dependencies. Search and copy all the dependency coordinates from the Maven Central directly, nowhere else. Add these new dependencies above the 3 jbehave dependencies already present in the pom.xml

Add dependencies for the following
  • Selenium
  • Log4J
  • Apache POI

Then the run the mvn command to resolve the dependencies
mvn dependency:resolve

Then download the required style-files for the reporting via:
mvn generate-resources
This is required only once, unless you run the clean command.

The tests can be executed via:
mvn integration-test
If this step results in BUILD SUCCESS then your basic project setup is correct and complete


Additional configuration can be done as below...

8. Configure Log4J

9. Add custom configurations for JBehave