Automate Behavior-driven Development | DrupalCon Portland 2022

Automate Behavior-driven Development
Yas Naoi
Chief Architect, Cloud
DOCOMO Innovatins, Inc.

Yas
2
Copyright © 2022 DOCOMO Innovations, Inc. All Rights Reserved.

Yas
3
• Chief Architect, Cloud

Yas
4
• DOCOMO Innovations, Inc. (Palo Alto, CA)
• NTT DOCOMO Inc.

Yas
• DOCOMO Innovations, Inc. (Palo Alto, CA)
• NTT DOCOMO Inc.
• Cloud, DevOps, Agile Software Development
• d.o: 17 years
5

Copyright © 2022 DOCOMO Innovations, Inc. All Rights Reserved. 6
Our Team

/ Cloud Team Members /

CA
Yas
Baldwin
Kumiko
Tatsuya
Hiroki
Tamaki

India
Sachin
CA
Yas
Baldwin
Kumiko
Tatsuya
Hiroki
Tamaki

Japan
Xiaohua
Takumaru
Masami
Ryo
Tomohiro
India
Sachin
CA
Yas
Baldwin
Kumiko
Tatsuya
Hiroki
Tamaki Shota
Makoto

11
Cloud Orchestrator
Search: drupal cloud orchestrator

12
Cloud Orchestrator
Search: drupal cloud orchestrator

Multi-cloud Support
──────────────
13
Cloud Orchestrator

Multi-cloud Support
──────────────
§ AWS
14
Cloud Orchestrator

Multi-cloud Support
──────────────
§ AWS
§ Kubernetes
15
Cloud Orchestrator

Multi-cloud Support
──────────────
§ AWS
§ Kubernetes
§ OpenStack
16
Cloud Orchestrator

Multi-cloud Support
──────────────
§ AWS
§ Kubernetes
§ OpenStack
§ VMware
17
Cloud Orchestrator

Multi-cloud Support
──────────────
§ AWS
§ Kubernetes
§ OpenStack
§ VMware
§ Terraform Cloud
18
Cloud Orchestrator

Agenda
19

Agenda
• Development Model
• What’s Behat for Behavior-driven Development (BDD)
• Templating for Behat Test Scenarios
• Automated Testing for BDD
20

Software Development Lifecycle

Coding
Integration Testing
Component Design
Function Design Unit Testing
System Requirements System Testing
Business Requirements Acceptance Testing
V-model [Boehm 79]

Business Requirements
V-model [Boehm 79]

System Requirements
V-model [Boehm 79]

Component Design
System Requirements
V-model [Boehm 79]

Component Design
Function Design
System Requirements
V-model [Boehm 79]

Coding
Component Design
Function Design
System Requirements
V-model [Boehm 79]

Coding
Component Design
System Requirements
V-model [Boehm 79]

Coding
Integration Testing
Component Design
System Requirements
V-model [Boehm 79]

Coding
Integration Testing
Component Design
V-model [Boehm 79]

Coding
Integration Testing
Component Design
TDD
V-model [Boehm 79]

Coding
Integration Testing
Component Design
TDD
BDD
V-model [Boehm 79]

Coding
Integration Testing
Component Design
TDD
BDD
ATDD
V-model [Boehm 79]

Coding
Integration Testing
Component Design
V-model [Boehm 79]
✔TDD
BDD
ATDD
PHPUnit (Drupal)

Coding
Integration Testing
Component Design
V-model [Boehm 79]
✔TDD
✔BDD (ATDD)
PHPUnit (Drupal)
Behat+
Drupal
Extension

Behavior-driven Development
39

PHPUnit
/**
* Tests CRUD for K8s node.
*
* @throws Exception
*/
public function testNode(): void {
$cloud_context = $this->cloudContext;
$data = $this->createNodeTestFormData(self::K8S_NODE_REPEAT_COUNT);
$this->updateNodesMockData($data);
// Update k8s nodes.
$this->drupalGet("/clouds/k8s/$cloud_context/node/update");
$this->assertNoErrorMessage();
for ($i = 0; $i < self::K8S_NODE_REPEAT_COUNT; $i++) {
$this->assertSession()->pageTextContains($data[$i]['name']);
}
for ($i = 0, $num = 1; $i < self::K8S_NODE_REPEAT_COUNT; $i++, $num++) {
// Make sure the all node listing exists.
$this->drupalGet('/clouds/k8s/node');
$this->assertNoErrorMessage();
for ($j = 0; $j < $num; $j++) {
$this->assertSession()->pageTextContains($data[$j]['name']);
}
}
}

BDD Test Scenario Example by Behat
Feature: Login, Check the Clean State, and Logout
In order to list resources on the clean state
As a user
I need to have both Drupal and AWS IAM permissions
Scenario: Log in as "Anonymous user"
Given I am not logged in
When I visit "/clouds"
Then I should be on "/user/login"
And I should see the heading "Log in"
Scenario Outline: Access cloud service providers
Given I am logged in as a user with the <role> role
And I should not see the link "All" in the "nav_bar"
And I should <verb> "Add cloud service provider" in the "nav_bar"
When I go to "/clouds"
Then I should be on "/admin/structure/cloud_config/add"
And I should get a <code> HTTP response
And I should see the heading <heading>
Examples:
| role | verb | code | heading |
| "Authenticated user" | not see | 403 | "Access denied" |
| "Cloud administrator" | see | 200 | "Add cloud service provider" |
| "Administrator" | see | 200 | "Add cloud service provider" |

Behat + drupal-extension + Chrome Driver
2022 DOCOMO Innovations, Inc. All Rights Reserved. Confidential.
43
Application under Test
(AUT)
Test Framework

44
Test Framework

Test Framework
Features
in Gherkin
45

Test Framework
Features
in Gherkin
Drupal Ext.
Drush Ext.
API Driver
46

Test Framework
Features
in Gherkin
Drupal Ext.
Drush Ext.
API Driver
Mink Ext.
Chrome Ext.
WebDriver
47

Chrome
Driver
Test Framework
Features
in Gherkin
Drupal Ext.
Drush Ext.
API Driver
Mink Ext.
Chrome Ext.
WebDriver
WebDriver
Implementation
48

Chrome
Driver
Headless
Browser
Test Framework
Features
in Gherkin
Drupal Ext.
Drush Ext.
API Driver
Mink Ext.
Chrome Ext.
WebDriver
WebDriver
Implementation
<< Native API >>
49

Application under Test
(AUT)
Chrome
Driver
Headless
Browser
<< Native API >> << HTTP >>
Test Framework
Features
in Gherkin
Drupal Ext.
Drush Ext.
API Driver
Mink Ext.
Chrome Ext.
WebDriver
WebDriver
Implementation
50

Behat (Gherkin)
As a user
Examples:

Behat (Gherkin)
As a user
Examples:
Initial State / Condition

Behat (Gherkin)
As a user
Examples:
Action Take Place

Behat (Gherkin)
As a user
Examples:
Expected Result

I want to use parameters in scenarios!
As a user
Examples:

As a user
Examples:

How it works by natural language?
72

The Tricks
/**
* Log in as the existing user
*
* @Given I am logged in as user :name
*/
public function iAmLoggedInAsUser($name): void {
$user = user_load_by_name($name);
$this->getSession()->visit(user_pass_reset_url($user) . '/login');
}

The Tricks
/**
*
*/
}
Sentence

The Tricks
/**
*
*/
}
Parameter
Variable

The Tricks
/**
* Check if the current page contains no messages of the specified selector.
*
* @Then /^I should see no (?P<selector>(?:success|error|warning)) message$/
*/
public function assertSelectorNotVisible($selector): void {
switch ($selector) {
case 'success':
$selector_id = 'success_message_selector';
break;
case 'error':
$selector_id = 'error_message_selector';
break;
case 'warning':
$selector_id = 'warning_message_selector';
break;
default:
throw new ExpectationException(sprintf("Message type is invalid: %s",
$selector), $this->getSession()->getDriver());
}
$this->assertNotVisible($selector_id);
}
Regular
Expression

But scenario outline is not enough!
77

I want to use parameters for everything!
Scenario Outline: Launch an EC2 instance
Given I am logged in as user "{{ user_name }}"
When I visit "/clouds/design/server_template/"
And I click <instance_name>{{ cloud_context }}
And I click "Launch"
And the url should match "/launch"
And I check the box "Automatically terminate instance"
And I press "Launch"
Then I should be on "/clouds/aws_cloud/{{ cloud_context }}/instance"
And I should see "pending" in the <instance_name> row
And I should see <instance_type> in the <instance_name> row
And I should see "{{ availability_zone }}" in the <instance_name> row
Examples:
| instance_name | instance_type |
| "{{ instance_name_1 }}" | "{{ instance_type_1 }}" |

I want to use parameters for everything!
Scenario Outline: Launch an EC2 instance
Given I am logged in as user "{{ user_name }}"
When I visit "/clouds/design/server_template/"
And I click <instance_name>{{ cloud_context }}
And I click "Launch"
And the url should match "/launch"
And I check the box "Automatically terminate instance"
And I press "Launch"
Then I should be on "/clouds/aws_cloud/{{ cloud_context }}/instance"
And I should see "pending" in the <instance_name> row
And I should see <instance_type> in the <instance_name> row
And I should see "{{ availability_zone }}" in the <instance_name> row
Examples:
| instance_name | instance_type |
Twig Template!

Get Parameterized

Get Parameterized
docker build

Get Parameterized
docker build
RUN composer require
drupal/cloud
drupal/drupal-extension
...

Get Parameterized
Packagist:
drupal/cloud_orchestrator
incl. drupal/cloud
Feature: ...
Scenario Outline: ...
Examples:
docker build
drupal/cloud
...
<< Download >>

Get Parameterized
docker run docomoinnovations/cloud_orchestrator
Packagist:
incl. drupal/cloud
Feature: ...
Examples:
docker build
drupal/cloud
...
<< Download >>
<< docker run >>

Get Parameterized
Run drush cloud:behat:preprocess
• Read a YAML file whih contains parameter mappings
• Replace {{...}} as an input variable
Packagist:
incl. drupal/cloud
Feature: ...
Examples:
docker build
drupal/cloud
...
<< Download >>
<< docker run >>

Get Parameterized
private_params.yml
e.g.
This is a placeholder:
Cloud Orchestrator
Packagist:
incl. drupal/cloud
Feature: ...
Examples:
docker build
drupal/cloud
...
<< Read parameter files>>
<< Download >>
<< docker run >>

Get Parameterized
Feature: ...
...<This is a placeholder>...
Examples:
| This is a placeholder |
| {{This is a placeholder}} |
private_params.yml
e.g.
Cloud Orchestrator
Packagist:
incl. drupal/cloud
Feature: ...
Examples:
docker build
drupal/cloud
...
<< Read Feature template files>>
<< Download >>
<< docker run >>

Get Parameterized
Feature: ...
Examples:
private_params.yml
e.g.
Cloud Orchestrator
Feature: ...
Examples:
| Cloud Orchestrator |
Packagist:
incl. drupal/cloud
Feature: ...
Examples:
docker build
drupal/cloud
...
<< Output actual feature files>>
<< Download >>
<< docker run >>

Get Parameterized
Run behat command
run_behat.sh
Feature: ...
Examples:
private_params.yml
e.g.
Cloud Orchestrator
Feature: ...
Examples:
Packagist:
incl. drupal/cloud
Feature: ...
Examples:
docker build
drupal/cloud
...
<< Read feature files>>
<< Download >>
<< docker run >>

Get Parameterized
Run behat command
run_behat.sh
Feature: ...
Examples:
private_params.yml
e.g.
Cloud Orchestrator
Feature: ...
Examples:
Execute BDD tests
and get Output
Packagist:
incl. drupal/cloud
Feature: ...
Examples:
docker build
drupal/cloud
...
<< Process features >>
<< Read feature files>>
<< docker run >>
<< Download >>

We Test Our Code

We Want This One!

GitLab CI/CD Pipeline

GitLab CI/CD Pipeline Architecture Basics

Developer

Developer
GitLab Master

GitLab Master GitLab Runner
Developer

GitLab Master
GitLab
Repository
Developer
GitLab Runner

GitLab Master
CI/CD
Pipeline
GitLab
Repository
Developer
GitLab Runner

CI/CD
Pipeline
GitLab
Repository
Developer
gitlab-runner 1
gitlab-runner 2
…
gitlab-runner N

GitLab
Container
Registry
CI/CD
Pipeline
GitLab
Repository
Developer
gitlab-runner 1
gitlab-runner 2
…
gitlab-runner N

CI/CD
Pipeline
GitLab
Repository
Developer
GitLab
Container
Registry
gitlab-runner 1
gitlab-runner 2
…
gitlab-runner N

EC2
Our GitLab CI/CD Pipeline Architecture
GitLab Master (SaaS)
CI/CD
Pipeline
GitLab
Repository
Developer
Slack
GitLab
Container Registry
gitlab-runner 1
gitlab-runner 2
gitlab-runner 3
gitlab-runner 4

Slack Integration

How we can build CI/CD pipeline?
111

.gitlab-ci.yml
stages:
- run_tests
- .post
variables:
UPDATE: 20220408a
WEB_ROOT: '/var/www/html/web'
CLOUD_HOME: "${WEB_ROOT}/modules/contrib/cloud"
BROWSER_OUTPUT: "${WEB_ROOT}/sites/simpletest/browser_output/"
IMAGE_TAG: 9.4.x-arm64
.preprocess_template: &preprocess_definition
image:
name: "${CI_REGISTRY_IMAGE}:${IMAGE_TAG}"
stage: run_tests
only:
- branches
# - merge_requests
before_script: |
whoami
sudo service apache2 start
sudo service memcached start
sudo mkdir -p "${WEB_ROOT}/simpletest/${CI_PIPELINE_ID}"
sudo mount -t tmpfs -o size=8G tmpfs "${WEB_ROOT}/simpletest/${CI_PIPELINE_ID}"
sudo chmod 777 -R "${WEB_ROOT}/simpletest"
rm -rf "${CLOUD_HOME}" && cp -ar "${PWD}" "${CLOUD_HOME}”
.artifacts_template: &postprocess_definition
after_script: |
mkdir -p "${BROWSER_OUTPUT}"
if [ -d "${BROWSER_OUTPUT}" ]; then cp -ar "${BROWSER_OUTPUT}" "${CI_PROJECT_DIR}/";
else mkdir -p "${BROWSER_OUTPUT}" && touch "${BROWSER_OUTPUT}/NO_ARTIFACTS.txt"; fi
php -v
drush status
...

.gitlab-ci.yml
• Describe the pipeline definition in one YAML file
• Put root (top) directory of your branch
• Get started
– https://docs.gitlab.com/ee/ci/quick_start/

Demo
• Behat
• GitLab CI/CD Pipeline

Our Custom Extension to drupal-extension
1. Logged in as the existing user without password
– Using drush user:login
2. Take a screenshot, which is very useful in troubleshooting
– Not only with Selenium + @javascript, but Chrome Extension or w/o
@javascript
3. Specifying a region with the row index of a table
– Especially when the same labels are used in multiple rows
4. Assert only the success message shows
– Namely, neither error or warning messages show
116

117
– Not only with Selenium + @javascript, but Chrome Extension
or w/o @javascript annotation

118

119

120

Known Issues: Solutions
1. User login sometimes fails due to "Username" is not found
– Workaround: add a longer sleep after chromium started
2. The text in checkbox/link is visible, but not ready to be clicked
– Add 500 ms sleep before such action
3. Cannot distinguish DOM objects if the same text are set
– For the table rows
4. Cannot login with a role created in the same Behat process
– Workaround: divide a feature file to run Behat for each
– Avoid using login with such role
121

Where to Find Our Solutions
123
• Behat feature files & configurations
– https://git.drupalcode.org/project/cloud/-/tree/5.x/tests/src/Behat
– https://git.drupalcode.org/project/cloud/-/tree/5.x/modules/cloud_service_providers/aws_cloud/tests/src/Behat
– https://git.drupalcode.org/project/cloud/-/tree/5.x/modules/cloud_service_providers/k8s/tests/src/Behat
• Drupal extensions with our custom functions
– https://github.com/docomoinnovations/drupalextension

Resources
124
• Cloud Orchestrator
– https://drupal.org/project/cloud
– https://cloud.airgap.host
• YouTube Channel
– https://bit.ly/2TJssME

Coming together is a beginning,
keeping together is a progress;
working together is a success.
Henry Ford
“ “

Automate Behavior-driven Development | DrupalCon Portland 2022

More Related Content

What's hot (20)

Similar to Automate Behavior-driven Development | DrupalCon Portland 2022 (20)

More from DOCOMO Innovations, Inc. (17)

Recently uploaded (20)

Automate Behavior-driven Development | DrupalCon Portland 2022