Let us try and fill in some of the background to W3QA's unique business solution, terms used and the market it operates in.
Apart from keeping tests readable what benefits are there to using FIT on Selenium RC?
Selenese is a FIT style specification written in HTML. It is the generic language of the Selenium world. Over 95% of Selenium users do not write code - instead they write Selenese.
Maintaining your tests in Selenese keeps you connected to the growing number of commercial and non commercial members of that user "community". This will make a range of test services and support available that would otherwise be difficult to provide.
Without Hrmes, when you wish to fully extend Selenium it means moving to Selenium RC and abandoning Selenese and its FIT style specifcation. This will involve having to migrate tests into a programming language, such as Java. This is a significant hurdle.
Hrmes preserves the FIT format and structure. Hrmes has been developed to offer "Selenium RC for Humans". It is the only framework that has FIT on a Selenium RC platform.
Why do I need a framework?
Fundamentally the answer comes down to scalability. Automation projects that ignore the need for a framework will fail.
If you are thinking of implementing test automation or have already done so, ensure you fully appreciate what a framework is and make sure you have one that works for all test levels, not just unit testing.
If you do not, your test automation project will pick up a significant technical debt that will have to be paid - and probably when you least can.
Why integrate the Hrmes test specification with Selenium? Surely it's better to make it fully abstracted from any tool?
There are advantages to both. A test specification that is fully integated with a test tool will help you establish effective automation faster. Only a test specification that is fully integrated with Selenium can deliver an execution ready framework.
Full abstraction would have the advantage of delivering test tool independence. However, integrating a highly abstract test specification with an automation tool is expenisve and a diversion, unless test tool independence is a critcial objective.
On balance, W3QA believes that the clarity and readability of FIT is sufficient. Hrmes offers the capability of extending these basic DSL features - through abstraction mappings.
Having a human readable specification that is fully integrated with one of the world's most acclaimed test tools isn't such a bad thing.
What is wrong with a code driven approach?
The answer will depend on the objectives of your test automation project. It will also depend on the profile and skills of the team that are going to deliver it. The vast majority of Selenium RC automation test suites are written in Java and use JUnit or TestNG unit test frameworks. Many of these implementations deliver on some key objectives.
However, code based frameworks will always fail to deliver in critical areas. A key objective of test automation is making clear what is being tested and maintaining the involvement of non-developers, particularly testing and business expertise. This is a real problem with a code based approach.
I have the budget to afford a bespoke automation framework, why should I consider Hrmes?
Hrmes is not just about saving money. Hrmes can be fully extended using all the functionality available to Java 5. Any bespoke functionality that can be implemented in Java can be implemented in Hrmes.
It is worth bearing in mind that most "bespoke" frameworks are often poorly designed and highly idiosyncratic. These frameworks are rarely understood by anyone other than the person who created them. Avoid this pitfall, and focus on building the truly bespoke functionality you need; not trying to recreate what you can get out of the box.
Hrmes enables you to spend your test budget more efficiently and effectively by targeting the development of what you need. Afterall, your test team should be focused on getting tests written and not needlessly developing test frameworks.
What is a test automation framework?
A test framework is not the tests themselves nor even a test suite. It consists of all the functionality required to enable the effective development and use of automated test scripts. It supports test methodology and provides an environment where tests can be developed, run and understood.
A framework is the enabling part of the test automation function, not the tests themselves.