Testing mit Codeception: Full-stack testing PHP framework

$I->wantTo(‘Test our
Software‘);

About me
• Susann Sgorzaly, 30
• Statistician and Software developer
• PHP developer for 3,5 years
• currently: Developer at ITEXIA GmbH
(on maternity leave)
@susgo
@susann_sg

Motivation
• Situation in the middle of 2017:
• quarterly releases
• before each release: weeks of manually testing and bug
fixing
• Problem:
• slow
• expensive
• late
• and …

Motivation
• How to change it?
• analyse the situation (testing strategy):
• project manager clicks his/her way though the software
• NO test scenarios list/plan
à there was no strategy at this point
à first of all: list of test cases (Excel list)
• search a way to automate this and for the moment only this

Motivation
• Searching for know-how:
1. own experiences from previous company (QuoData
GmbH)
• reproduce the test cases by building full navigation
scenarios with CasperJS
• take screenshots of all relevant pages
• product owner can confirm these screenshots as correct
• look at differences of the screenshots in a later run –
confirm if they are correct
• we had a web platform to confirm automatically created
screenshots as correct

Motivation
var login = {
name: 'admin',
pass: 'admin',
};
casper.thenOpen('http://localhost');
casper.then(function() {
this.fill('#user-login-form', login);
});
casper.thenClick('#edit-submit');
casper.waitFor(function() {
return !this.getCurrentUrl().match(RegExp(
'/nach-dem-login'));
}, null, function() {
this.echo('[error] [casper] Failed to log
in with '
+ login.name + ':' + login.pass + '!',
'ERROR');
this.exit(1);
});

Motivation
Before After changes
Differences

Motivation
1. own experiences from previous company (QuoData
GmbH)
• Pros at first sight:
• result is easy to interpret
• notices design breaks
• confirmation by the product owner
• Cons at first sight:
• language / technology differs from product language
• manually confirmation of screenshots needed

Motivation
2. experiences from other company (portrino GmbH)
• Codeception

Motivation
<?php
$I = new AcceptanceTester($scenario);
$I->amOnPage('/site/login');
$I->see('Login', 'h1');
$I->wantTo('try to login as admin with correct credentials');
$I->fillField('input[name="LoginForm[username]"]', 'admin');
$I->fillField('input[name="LoginForm[password]"]', 'admin');
$I->click('login-button');
$I->dontSeeElement('#login-form');
$I->expectTo('see user info');
$I->see('Logout');

Motivation
2. experiences from other company (portrino GmbH)
• Pros at first sight:
• PHP (uses the software language)
• simple to read
• possible simple to write and easy to maintain
• no manually confirmation needed
• Cons at first sight:
• no screenshots and therefore no design (“sexy
frontend”) confirmation by the product owner

What Is Codeception?
• powerful testing framework written in PHP
• inspired by BDD
• only requirements are basic knowledge of PHP and the theory
of automated testing
• kept as simple as possible for any kind of users
• powered by PHPUnit
„Describe what you test and how you test it. Use PHP to write
descriptions faster.“

What Does Codeception Do?
• multiple approaches:
• acceptance tests
• functional tests
• unit tests

What Kind Of Tests? – Acceptance Tests
• browser emulation (Selenium / Webdriver)
• can have no knowledge of the technologies
• can test any website
• testing done from a non-technical person
• readable by humans (managers)
• tests JavaScript / Ajax
• stability against code changes
• SLOW!

What Kind Of Tests? – Functional Tests
• web request and submit to application emulation
• assert against response and internal values
• the person testing knows how the software works
• uses different request variables to ensure the functionality of
the software for nearly all cases
• framework-based
• still readable by humans
• can‘t test JavaScript / Ajax
• less slow

What Kind Of Tests? – Unit Tests
• test the application core functions
• single isolated tests
• the testing person knows the internals of the application
• only readable by IT professionals
• fastest!

Codeception: How to? - Installation
• Install via composer
composer require “codeception/codeception“ --dev
• Execute it as:
./vendor/bin/codecept

Codeception: How to? - Configuration
• example: configure acceptance test suite
• edit configuration file: tests/acceptance.suite.yml
actor: AcceptanceTester
modules:
enabled:
- PhpBrowser:
url: {YOUR APP'S URL}
- HelperAcceptance

Side Note: Actors and Modules?
• test classes use Actors to perform actions à act as a dummy user
• actor classes are generated from the suite configuration
• methods of actor classes are taken from Codeception Modules
• each module provides predefined actions for different testing
purposes
• modules can be combined to fit the testing environment
modules:
enabled:
…

Side Note: PHP Browser?
• doesn’t require running an actual browser
• runs in any environment: only PHP and cURL are required
• uses a PHP web scraper, which acts like a browser: sends a
request, receives and parses the response
• can’t work with JS (modals, datepickers, …)
• can’t test actual visibility of elements
• …
• fastest way to run acceptance tests
• But: not the situation a manager or customer is in

Side Note: Selenium WebDriver
• requires running an actual browser (Firefox, Chrome, …)
• can work with JS (modals, datepickers, …)
• can test actual visibility of elements
• …
• slower
• but: a manager or customer uses it too

Side Note: PhpBrowser vs WebDriver?
• doesn’t matter what you choose at the beginning
• most tests can be easily ported between the testing backends
• PhpBrowser tests can be executed inside a real browser with
Selenium WebDriver
• you only have to change the acceptance test suite
configuration file module and rebuild the AcceptanceTester
class

Codeception: How to? - Configuration
• Example: configure acceptance test suite
• Edit configuration file: tests/acceptance.suite.yml
• minimum: add your application url
modules:
enabled:
- PhpBrowser:
url: {YOUR APP'S URL}
- HelperAcceptance

Codeception: How to? - Generate Test
• Execute
./vendor/bin/codecept generate:cest acceptance Login
àgenerates new php class file
LoginCest.php for class LoginCest in the folder
‚tests/acceptance‘
Note: Cest is the class based format. Codeception also supports Cept which is a
scenario based format (see example from the Motivation slides)

Codeception: How to? - Write Test
<?php
class LoginCest
{
public function tryToLoginAsAdmin(AcceptanceTester $I)
{
$I->see('Logout');
}
}

Codeception: How to? - Write Test
• Actions
• Assertions
$I->see('Logout');

Codeception: How to? - Run Test!
./vendor/bin/codecept run --steps
./vendor/bin/codecept run --debug
./vendor/bin/codecept run acceptance
./vendor/bin/codecept run acceptance
LoginCest:ensureThatLoginWorks
./vendor/bin/codecept run
tests/acceptance/LoginCest.php::ensureThatLoginWorks
./vendor/bin/codecept run --xml

Codeception: How to? - Run Test!

Codeception: Basic Features
• multiple backends, easily changed in configuration
• Selenium, PhpBrowser, PhantomJS
• elements matched by name, CSS, XPath
• data clean-up after each run
• integrates with different frameworks (e.g. Symfony2, Yii2,
Laravel)
• Dependency Injection

Codeception: Basic Features
• executes PHPUnit tests
• BDD-style
• WebService testing via REST and SOAP is possible
• generates reports: HTML, XML, JSON
• Fixtures (known test data)
• Database helpers
• Code Coverage

Codeception - solves all my problems!
YEAH!!!!
Seems to be easy. Let‘s go home and write some tests…

Codeception - solves all my problems?

Codeception - solves all my problems?
NO!
My tests often broke!
My data are unstable!
So many tests,
so less structure!
NO!
NO!
NO!NO!
NO!
NO!
NO!
NO!
NO!
NO!
NO!

Best practice. My tests often broke
• use ids
• use the right / a good selector (CSS, name, XPath, label)
• use constants: PageObjects in Codeception

• PageObjects:
• represents a web page as a class
• the DOM elements on that page are its properties
• some basic interactions are its methods
• example:
./vendor/bin/codecept generate:pageobject Login

class LoginCest
{
public function tryToLoginAsAdmin(AcceptanceTester $I)
{
$I->see('Logout');
}
}

namespace Page;
class Login
{
// include url of current page
public static $URL = '/site/login';
/**
* Declare UI map for this page here. CSS or XPath allowed.
*/
public static $form = '#login-form';
public static $usernameField = 'input[name="LoginForm[username]"]';
public static $passwordField = 'input[name="LoginForm[password]"]';
public static $formSubmitButton = 'login-button';
}

class LoginCest
{
public function tryToLoginAsAdmin(AcceptanceTester $I, PageLogin $loginPage)
{
$I->amOnPage($loginPage::$URL);
$I->seeElement($loginPage::$form);
$I->fillField($loginPage::$usernameField, 'admin');
$I->fillField($loginPage::$passwordField, 'admin');
$I->click($loginPage::$formSubmitButton);
$I->dontSeeElement($loginPage::$form);
$I->see('Logout');
}
}

Best practice. My data are unstable
• try to search for stable elements on the website
• use fixtures instead of database dumps
• fixtures:
• sample data for tests
• data can be either generated, loaded from an array or taken
from a sample database
• usage with Db module:
$I->haveInDatabase('posts', array('title' => My title', 'body' => The body.'));

• use DataFactory module to generate the test data dynamically
• uses an ORM of your application to define, save and
cleanup data
• should be used with ORM or Framework modules
• requires package: "league/factory-muffin“

<?php
use LeagueFactoryMuffinFakerFacade as Faker;
$fm->define(User::class)->setDefinitions([
'name' => Faker::name(),
// generate email
'email' => Faker::email(),
'body' => Faker::text(),
// generate a profile and return its Id
'profile_id' => 'factory|Profile'
]);
• Generation rules can be defined in a factories file

modules:
enabled:
- Yii2:
configFile: path/to/config.php
- DataFactory:
factories: tests/_support/factories
depends: Yii2
• load factory definitions from a directory

• DataFactory module actions
• generate and save record:
$I->have('User');
$I->have('User', ['is_active' => true]);
• generate multiple records and save them:
$I->haveMultiple('User', 10);
• generate records (without saving):
$I->make('User');

Best practice. So many tests, so less
structure
• test code is source code and therefore should follow the same
rules
• reuse code
• extend AcceptanceTester class located inside the
tests/_support directory
• use StepObjects
• use PageObjects

structure – Extend AcceptanceTester
class AcceptanceTester extends CodeceptionActor
{
// do not ever remove this line!
use _generatedAcceptanceTesterActions;
public function login($name, $password)
{
$I = $this;
$I->submitForm('#login-form', [
'input[name="LoginForm[username]"]' => $name,
'input[name="LoginForm[password]"]' => $password
]);
$I->see($name, 'Logout');
}
}

structure - StepObjects
• groups some common functionality for a group of tests
(for testing a part of a software: admin area, …)
• extends AcceptanceTester class
• example:
./vendor/bin/codecept generate:stepobject Admin
à generate a class in tests/_support/Step/Acceptance/Admin.php

namespace StepAcceptance;
class Admin extends AcceptanceTester
{
public function loginAsAdmin()
{
$I = $this;
// do login
}
}

class TestCest
{
function tryToTest(StepAcceptanceAdmin $I)
{
$I->loginAsAdmin();
// test something
}
}

Codeception: Nice to have
• session snapshots (for faster execution)
• share cookies between tests
• e.g. test user can stay logged in for other tests:
• $I->saveSessionSnapshot('login‘);
• $I->loadSessionSnapshot('login');
• group tests with @group annotation for test methods:
• e.g. @group important
• ./vendor/bin/codecept run acceptance -g important

Codeception: Nice to have
• before/after annotations
• example annotations
• dataProvider annotations
• environments
• dependencies
• multi session testing
• …

Codeception: Summary
• easy for developers but difficult for product owner
• layout tests missing
• should we use another tool?

Codeception: Summary
• Codeception is extendable
• solution: use module Visualception
• see https://github.com/Codeception/VisualCeption for more
information

Testing mit Codeception: Full-stack testing PHP framework

Testing mit Codeception: Full-stack testing PHP framework

More Related Content

Similar to Testing mit Codeception: Full-stack testing PHP framework (20)

Recently uploaded (20)

Testing mit Codeception: Full-stack testing PHP framework