User manual
1. Each test class extends from AbstractTest
- Test class extends tech.testnx.cah.AbstractTest
- Implements method getModuleName()
2. Managed WebDriver
-
Get a default webdriver specified in global-config.properties
WebDriver driver = getDriver();
-
Get a specific webdriver
WebDriver driver = newChromeDriver();
WebDriver driver = newFirefoxDriver();
WebDriver driver = newIeDriver(); -
Customized the WebDriver by listeners
WebDriver driver = getDriver();
WebDriverEventListener listener= new AbstractWebDriverEventListener() {};
driver = registerDriverListener(driver, listener);
driver = unregisterDriverListener(driver, listener); -
Managed the WebDrivers by pool
WebDriver driver1 = requestWebDriver();
WebDriver driver2 = this.requestWebDriver(BrowserType.CHROME);
WebDriver driver3 = this.requestWebDriver(BrowserType.FIREFOX);
releaseWebDriver(driver1);
releaseWebDriver(driver2);
releaseWebDriver(driver3);
3. Managed RESTful API Client
-
Get a RestClient and RestClientV2
RestClient client = RestClient.newInstance("http://endpoint/api/");
RestfulResponse response = client.doGet();
Assert.assertEquals(response.getStatus(), 200);RestClientV2 clientV2 = RestClientV2.newInstance();
RestfulResponse response = clientV2.doGet("http://endpoint/api/", null);
Assert.assertEquals(response.getJsonParser().getStringValue("status"), "success");
Assert.assertEquals(response.getJsonParser().getIntValue("data[2].employee_age"), 65); -
Customized the RestClient by listeners
RestClientV2 client = RestClientV2.newInstance();
RestClientListener listener = new RestClientListener() {};
client.registerListener(listener);
client.unregisterListener(listener);
4. Configure the path of resources directory
CAH configs, data, drivers, logs, reports are stored in resources directory. Now it is possible to customize the path.
- Set up Environment Variable: CAH_HOME=Path; the directory should contain resources directory
- Create cah-home directory which should contain resources directory
- Use current working directory with java code, which should contain resources directory
5. Test Data Management
-
Test data type supported: JSON and XLS
- Test
@Test @Description(testIDs = "T-005", title = "Data Ready", description = "Test plus: ${{1}} + ${{2}} = ${{3}}?") public void testDataPlus(String description, int first, int second, int result) { Recorder.log(description); Assert.assertEquals(first + second, result); } @Test @Description(testIDs = "T-006", title = "Data Ready2", description = "Test minus: ${{1}} - ${{2}} = ${{3}}?") public void testDataMinus(String description, int first, int second, int result) { Recorder.log(description); Assert.assertEquals(first - second, result); } - Json
{ "common": { "commonData1": "value for commonData1" }, "tests": { "BasicTests": { "testDataPlus": [ { "description": "Verify the 5 + 5 = 10?", "first": 5, "second": 5, "result": 10 } ], "testDataMinus": [ { "description": "Verify the 7 - 5 = 1?", "first": 7, "second": 5, "result": 1 } ] } } } - xls
- Test
-
Auto-binding between test and data with zero effort, support types: Boolean, Short, Integer, Long, Float, Double, String, JsonNode, TransformableTestDataBean(interface)
-
Test(JsonNode)
@Test public void test3(String testname, JsonNode node) { System.out.println(testname); System.out.println(node.toString()); System.out.println(node.path("student").path(0).path("name").asText()); } -
Json(JsonNode)
"test3": [ { "testname": "TESTCASE-3", "node": { "student": [ { "id": 1, "name": "Tom", "isNew": true }, { "id": 2, "name": "Nick", "isNew": false } ] } } ] -
XLS(JsonNode)
-
Data Bean(TransformableTestDataBean)
1 public class Test4DataBean implements TransformableTestDataBean<Test4DataBean> { 2 3 public int id; 4 public String name; 5 public boolean isNew; 6 7 @Override 8 public Test4DataBean transformTestData(String content) { 9 10 try { 11 return Utilities.jsonUtility.getMapper().readValue(content, Test4DataBean.class); 12 } catch (IOException e) { 13 Logger.getLogger().error("Failed to transform test data from: " + content); 14 e.printStackTrace(); 15 throw new RuntimeException("Failed to transform test data"); 16 } 17 } 18 19 @Override 20 public boolean equalsValue(Test4DataBean t) { 21 22 if (id != t.id) return false; 23 if (!name.equals(t.name)) return false; 24 if (isNew != t.isNew) return false; 25 return true; 26 } 27 } -
Test(TransformableTestDataBean)
@Test public void test4(String testname, Test4DataBean bean) { System.out.println(testname); System.out.println(bean.id); System.out.println(bean.name); System.out.println(bean.isNew); } -
Json(TransformableTestDataBean)
"test4": [ { "testname": "TESTCASE-4", "Test4DataBean": { "id": 1, "name": "Tom", "isNew": true } } ] -
XLS(TransformableTestDataBean)
-
-
Data driven test
If one test has multiple data groups, the test will be automatically executed multiple times with different data group- Test
@Test public void testSample(String a, String b){ System.out.println(a + " :::: " + b); } - JSON
"DataDrivenTests": { "testSample": [ { "a": "my key", "b": "my value" }, { "a": "your key", "b": "your value" } ] - XLS
- Test
-
Auto-Decrypt secured test data Encrypted test data by Utilities.securityUtility.encrypt() and save them in data file, then the test data will be auto-decrypted for test
6. Share and manage the state across tests by StateManager
When user need share or pass some state or varible across tests, StateManager will provide help.
@Test
public void test1() {
logger.info("Start");
StateManager.setState("test1.stateA", "good");
logger.info("End");
}
@Test
public void test2() {
logger.info("Start");
if (StateManager.getState("test1.stateA").equals("good")) {
logger.info("Take action A");
} else {
logger.info("Take action B");
}
logger.info("End");
}
7. Template test
Template test could be useful in API test. the API payload with the same schema will be used in multiple tests and environments.
CAH framework provides two two Template Composer: CAH-Composer and Freemarker.
The template files could be stored in both module folder and ennvironment folder, but the template files are higher priority in ennvironment folder.
@Test
public void test_CAH_COMPOSER(String name, String age, String isStudent) {
//Test CAH_COMPOSER
Map<String, String> dataModel = new HashMap<>();
dataModel.put("name", name);
dataModel.put("age", age);
dataModel.put("isStudent", isStudent);
ComposeTemplate compoer = TemplateComposerProvider.CAH_COMPOSER.getComposer();
//test1-templates.json could be store in both module folder and environment folder
String content = compoer.compose(dataModel, dataClient.findTemplate("test1-templates.json").get().toFile());
logger.info(content);
JsonNode rootNode = Utilities.jsonUtility.readTree(content);
CahAssert.assertEquals(rootNode.path("name").asText(), name);
CahAssert.assertEquals(rootNode.path("age").asInt(), Integer.valueOf(age));
CahAssert.assertEquals(rootNode.path("isStudent").asBoolean(), Boolean.valueOf(isStudent));
}
@Test
public void test_FREEMARKER(String name, String age, String isStudent) {
//Test FREEMARKER
Map<String, String> dataModel = new HashMap<>();
dataModel.put("name", "Jacky");
dataModel.put("age", "14");
dataModel.put("isStudent", "true");
ComposeTemplate compoer = TemplateComposerProvider.FREEMARKER.getComposer();
//test4-templates-freemarker.json could be store in both module folder and environment folder
String content = compoer.compose(dataModel, dataClient.findTemplate("test4-templates-freemarker.json").get().toFile());
logger.info(content);
JsonNode rootNode = Utilities.jsonUtility.readTree(content);
CahAssert.assertEquals(rootNode.path("name4").asText(), dataModel.get("name"));
CahAssert.assertEquals(rootNode.path("age4").asInt(), Integer.valueOf(dataModel.get("age")));
CahAssert.assertEquals(rootNode.path("isStudent4").asBoolean(), Boolean.valueOf(dataModel.get("isStudent")));
}
@Test
public void test_ALL(String name, String age, String isStudent) {
Map<String, String> dataModel = new HashMap<>();
dataModel.put("name", "Mike");
dataModel.put("age", "20");
dataModel.put("isStudent", "true");
String templateContent = "{\n"
+ " \"name\": \"${{name}}\",\n"
+ " \"age\": ${{age}},\n"
+ " \"isStudent\": ${{isStudent}}\n"
+ "}";
//Test CAH_COMPOSER
logger.info("Test CAH_COMPOSER");
ComposeTemplate compoer = TemplateComposerProvider.CAH_COMPOSER.getComposer();
String content = compoer.compose(dataModel, templateContent);
logger.info(content);
JsonNode rootNode = Utilities.jsonUtility.readTree(content);
CahAssert.assertEquals(rootNode.path("name").asText(), "Mike");
CahAssert.assertEquals(rootNode.path("age").asInt(), 20);
CahAssert.assertEquals(rootNode.path("isStudent").asBoolean(), true);
//Test FREEMARKER
logger.info("Test FREEMARKER");
templateContent = "{\n"
+ "\n"
+ " \"name\": \"${name}\",\n"
+ " \"age\": ${age},\n"
+ " \"isStudent\": ${isStudent}\n"
+ " \n"
+ "}";
compoer = TemplateComposerProvider.FREEMARKER.getComposer();
content = compoer.compose(dataModel, templateContent);
logger.info(content);
rootNode = Utilities.jsonUtility.readTree(content);
CahAssert.assertEquals(rootNode.path("name").asText(), "Mike");
CahAssert.assertEquals(rootNode.path("age").asInt(), 20);
CahAssert.assertEquals(rootNode.path("isStudent").asBoolean(), true);
}
8. SSH tunnel creation/deletion and remote execute SSH commands by JSCH, SSHJ, SSHD
User can choose the provider in global-config.properties. Multiple network tunnels can be created in one machine and network tunnel life cycle is managed by test.
And support both user password and private key.
#SSH Client: JSCH, SSHJ, SSHD
SSH_Client=JSCH
HasSshTunnel sshTunnel = SshClientProvider.provide(host, user, password);
//HasSshTunnel sshTunnel = SshClientProvider.provide(host, user, Paths.get("Path to private key"));
sshTunnel.connect();
logger.info("SSH tunnel is started....");
sshTunnel.setLocalPortForwarding(3000, "100.10.150.60", 80);
logger.info("SSH tunnel is stopped....");
sshTunnel.unsetLocalPortForwarding(3000);
sshTunnel.disconnect();
HasSshExec sshExec = SshClientProvider.provide(host, user, password);
//HasSshTunnel sshTunnel = SshClientProvider.provide(host, user, Paths.get("Path to private key"));
sshExec.connect();
logger.info("Execute the first command");
String result = sshExec.executeCommand("cd ..; pwd; ls");
logger.info(result);
logger.info("Execute the second command");
result = sshExec.executeCommand("ping -c 3 bing.com");
logger.info(result);
sshExec.disconnect();
9. Native Logger
User can choose Logger Level(TRACE, DEBUG, INFO, WARN, ERROR) in global-config.properties
- Global Config
#Log Level: TRACE, DEBUG, INFO, WARN, ERROR Log_Level=INFO - Test code
Logger logger = Logger.getLogger(); logger.trace(""); logger.debug(""); logger.info(""); logger.warn(""); logger.error("");
10. CAH Assert
- CahAssert is 100% compatible with testNG Assert. Actually CahAssert is the delegate of testNG Assert. CahAssert saves all assert detail information into AssertRecord, which will be displayed in reports.
@Test
public void testCahAssert() {
Object[] actual = {1.5, 1.6, 1.7, 1.8};
Object[] expected = {1.9, 1.6, 1.7, 1.8};
CahAssert.assertNotEquals(actual , expected, "Check if object array is equal");
}
- AssertRecord.Extra is information holder if need to save extra information into records, which will be displayed in reports. Extra information only be available and saved into next assert.
@Test
public void testCahAssert() {
Object[] actual = {1.5, 1.6, 1.7, 1.8};
Object[] expected = {1.9, 1.6, 1.7, 1.8};
AssertRecord.Extra.setExtra("This is the first extra information for testCahAssert");
CahAssert.assertNotEquals(actual , expected, "The first extra information is available and saved into this assert");
CahAssert.assertNotEquals(actual , expected, "The first extra information is not available and saved into this assert");
AssertRecord.Extra.setExtra("This is the second extra information for testCahAssert");
CahAssert.assertEquals(actual , expected, "The second extra information is available and saved into this assert");
}
11. Test reporting
-
Annotation for report: @Description
@Test @Description(testIDs = "T-001", title = "Knock Bing By Browser", submodule = "UI testing", description = "Knock bing by keywords ${{0}} and check search result with ${{1}}") public void testKnockBingByBrowser(String keywords, String result) {} -
Recorder: record test flow and status during execution and show in report
Recorder.log(String msg); Recorder.log(Recordable< ? > rec); -
Predefined record types: CommonRecord, ScreenshotRecord, AssertRecord, ActionRecord, StepRecord
- CommonRecord
Take a note - ScreenshotRecord
Take a screenshot - AssertRecord
It is generated automatically during invokde CahAssert methods, and save the assertion detail then show in report - ActionRecord
It is generated automatically if use ECO mode, and save the action detail then show in report - StepRecord
It is generated automatically if use ECO mode, and save the step detail then show in report
- CommonRecord
-
Report type: CONSOLE, EXCEL, HTML_BOOTSTRAP, HTML_VUE_VUETIFY User can choose report types in global-config.properties. Report contains Test ID, Name, Description, Duration, Recording for each case, Errors, Exceptions, Logs and Screenshot for failed cases
#Report type(support multiple reports): CONSOLE, EXCEL, HTML_BOOTSTRAP, HTML_VUE_VUETIFY Report_Type=CONSOLE, EXCEL, HTML_BOOTSTRAP, HTML_VUE_VUETIFY-
Console report
-
HTML_BOOTSTRAP report (ONE page)
-
HTML_VUE_VUETIFY report (all in ONE page)
-
EXCEL report
-
-
Performance monitoring reports: CONSOLE, EXCEL, HTML_BOOTSTRAP
User can choose monitor providers in global-config.properties#Monitor Provider(support multiple provider): Http_Archiver_Monitor, Web_API_Perf_Monitor, Web_UI_Perf_Monitor Monitor_Provider=Web_API_Perf_Monitor, Web_UI_Perf_Monitor #Enable Performance Monitoring: TRUE, FALSE Enable_Performance_Monitor=TRUE- Http_Archiver_Monitor
Recording all http activities for both Web UI test and Restful API test, finally http archive file is generated like 'CAH_AT_Performance.har'. Then view it by http archive viewer
- Web_API_Perf_Monitor
- Console Web API Performance Report
- HTML Web API Performance Report
- XLS Web API Performance Report
- Console Web API Performance Report
- Web_UI_Perf_Monitor
- Console Web UI Performance Report
- HTML Web UI Performance Report
- XLS Web UI Performance Report
- Console Web UI Performance Report
- Http_Archiver_Monitor
12. ECO Design Pattern
- Auto-detect automation accident and auto-retry at right place and right timing within a minimum scope, and no retry code in test
- Roles: Executor, Controller, Operator (ECO)
- Scopes: Test, Step, Action
- Stop retry with two conditions
- Reach max retry threshold
- Face critical errors (java.lang.Error, java.lang.AssertionError, tech.testnx.cah.common.exceptions.StopTestError)
- ECO Model
- ECO Flow
- Example code
- Executor
@Executor
public class AdvancedTests extends TestBase {
private BingController bingController;
private WebDriver driver;
@BeforeMethod
public void beforeMethod() {
driver = getDriver();
bingController = new BingController(driver);
}
@Test
@Description(testIDs = "T-001",
title = "Bing Search",
submodule = "bing",
description = "Search Bing by keywords ${{0}}")
public void testBingSearch(String keywords, String expectedTitle) {
String url = "https://bing.com";
bingController.searchAndCheck(url, keywords, expectedTitle);
}
}
- Controller
@Controller
public class BingController extends ControllerBase {
private BingSearchPO bingSearchPO;
private WebDriver driver;
public BingController(WebDriver driver) {
this.driver = driver;
bingSearchPO = new BingSearchPO(driver);
}
protected BingController() {}
@Step(description = "Search Bing and check result", retry = 2, intervalWaitInSecond = 3)
public void searchAndCheck(String url, String keywords, String expectedTitle) {
driver.get(url);
bingSearchPO.search(keywords);
String result = bingSearchPO.getResultTitle();
logger.debug("Search result title: " + result);
Assert.assertTrue(result.equalsIgnoreCase(expectedTitle));
}
}
- Operator
@Operator
public class BingSearchPO extends PageBase {
@FindBy(name = "q")
private WebElement searchbox;
@FindBy(id = "search_icon")
private WebElement searchButton;
public BingSearchPO(WebDriver driver) {
super(driver);
}
protected BingSearchPO() {}
@Override
public void waitForReady() {}
/**
* Action: Search Bing by keywords
*/
@Action(description = "Search Bing by Keywords: ${{0}}", retry = 2, intervalWaitInSecond = 3)
public void search(String keywords) {
waitForReadyAndCapturePagePerf();
searchbox.clear();
searchbox.sendKeys(keywords);
Utilities.timeUtility.sleep(2);
searchButton.click();
}
/**
* Action: Get result title
*/
@Action(description = "Get result title", retry = 2, intervalWaitInSecond = 3)
public String getResultTitle() {
waitForReadyAndCapturePagePerf();
return driver.getTitle();
}
}