Integration tests: use the containers, Luke!

Integration tests:
use the containers, Luke!
Roberto “Frank” Franchini
@robfrankie

whoami(1)
https://www.linkedin.com/in/robfrank/

Former CTO of Arcade Analytics
Joined Activiti team in Alfresco

Same output from different data-sources

New features every week for every connector

Test Feature B
Test Feature A
Test Feature C
Test Feature D

Activiti Cloud is the first Cloud Native BPM framework built to
provide a scalable and transparent solution for BPM
implementations in cloud environments.

Each microservice has its own integration testing suite that
needs RabbiMQ and KeyCloack to be started before

Every morning a developer wakes up and starts containers
docker run -it -p 5672:5672 -p 15672:15672 --rm rabbitmq:management
docker run -it --rm -p 8180:8180 activiti/activiti-keycloak

Then run tests in the IDE
(Poor developer experience)

Don’t start containers manually anymore

Image credits https://magnasoma.com
Monolith era

Unit tests
Integration tests
UI
Tests
Slower
Faster
More
integration
More
isolation

Testcontainers is a Java library that supports JUnit tests,
providing lightweight, throwaway instances of common
databases, Selenium web browsers, or anything else that can
run in a Docker container.

Temporary database containers: specialized PostgreSQL,
MySQL, MsSQL Server, MariaDB, Oracle XE, Virtuoso

RDBMS containers
@Container
public static PostgreSQLContainer container = new PostgreSQLContainer();
@Container
public static MySQLContainer container = new MySQLContainer();
@Container
public static OracleContainer container = new OracleContainer();

Others containers: neo4j, couchbase, toxy-proxy, kafka, redis,
influxDB, elasticSearch, rabbitMQ

Others containers
@Container
public static Neo4jContainer container = new Neo4jContainer();
@Container
public static InfluxDBContainer container = new InfluxDBContainer();
@Container
public static KafkaContainer container = new KafkaContainer();
@Container
public static RabbitMQContainer container = new RabbitMQContainer();

Webdriver containers: run a Dockerized Chrome or Firefox
browser ready for Selenium/Webdriver operations - complete
with automatic video recording

Selenium
private val chrome: BrowserWebDriverContainer<Nothing> = BrowserWebDriverContainer<Nothing>()
.apply {
withDesiredCapabilities(DesiredCapabilities.chrome())
withRecordingMode(RECORD_ALL, File("target"))
start()
}

Generic containers: run any Docker container as a test
dependency

Generic container, image based
@Container
public static GenericContainer container = new GenericContainer("orientdb:3.0.23")
.withExposedPorts(2424, 2480)
.withEnv("ORIENTDB_ROOT_PASSWORD", "rootpassword")
.waitingFor(Wait.forListeningPort());

ROM debian:stretch-slim
LABEL maintainer="NGINX Docker Maintainers
<docker-maint@nginx.com>"
ENV NGINX_VERSION 1.15.5-1~stretch
ENV NJS_VERSION 1.15.5.0.2.4-1~stretch
RUN set -x
&& apt-get update
&& apt-get install --no-install-recommends
--no-install-suggests -y gnupg1 apt-transport-https
ca-certificates
&&
NGINX_GPGKEY=573BFD6B3D8FBC641079A6ABABF5BD827BD9B
F62;
found='';
for server in
ha.pool.sks-keyservers.net
hkp://keyserver.ubuntu.com:80
hkp://p80.pool.sks-keyservers.net:80
version: '2'
services:
elasticsearch:
build:
context: elasticsearch/
args:
ELK_VERSION: $ELK_VERSION
volumes:
-
./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch
.yml:ro
ports:
- "9200:9200"
- "9300:9300"
environment:
ES_JAVA_OPTS: "-Xmx256m -Xms256m"
networks:
- elk
logstash:
build:
context: logstash/
args:
ELK_VERSION: $ELK_VERSION
volumes:
- ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro
- ./logstash/pipeline:/usr/share/logstash/pipeline:ro
ports:
- "5000:5000"
- "9600:9600"
environment:
LS_JAVA_OPTS: "-Xmx256m -Xms256m"
networks:
- elk
depends_on:
- elasticsearch
kibana:
build:
Use a compose
Start from a Dockerfile

Generic container from Dockerfile
private val container: GenericContainer<Nothing> = GenericContainer<Nothing>(
ImageFromDockerfile("robfrank/ngnix")
.withFileFromPath("Dockerfile", Paths.get("./src/main/docker/nginx/Dockerfile"))
).apply {
withExposedPorts(80)
waitingFor(Wait.forListeningPort())
start()
followOutput(Slf4jLogConsumer(log))
}

Whatever is containerized by your team(s)

DB’s containers? Isn’t
enough?

H2 is fast, BUT doesn’t emulate specific features

Testcontainers is slower*, BUT gives 100% db compatibility
*not so slower

Jenkins
DOOD: Docker outside of Docker
DIND: Docker inside of Docker

DOOD: Dockerfile
FROM jenkins/jenkins:lts
USER root
RUN apt-get update
&& apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common
&& curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg
| apt-key add -
&& add-apt-repository
"deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")
$(lsb_release -cs)
stable"
&& apt-get update
&& apt-get install -y docker-ce
&& apt-get clean
&& rm -rf /var/lib/apt/lists/*
RUN usermod -aG docker jenkins
USER jenkins

version: '2'
services:
jenkins:
image: robfrank/jenkins:latest
ports:
- 8080:8080
- 50000:50000
privileged: false
volumes:
- ~/volumes/jenkins_home:/var/jenkins_home
- /usr/bin/docker:/usr/bin/docker
- /var/run/docker.sock:/var/run/docker.sock
- /usr/local/bin/docker-compose:/usr/local/bin/docker-compose

/usr/bin/docker
/usr/bin/docker

Travis: declare the service docker

language: java
service: docker
notifications:
email:
- builder@mycompany.com
before_install:
- docker version
- docker info
- cd ./src/main/docker/orientdb && ./build.sh && cd -
- cd ./src/main/docker/postgresql-dvdrental && ./build.sh && cd -
jdk:
- openjdk8
- openjdk12

Test over different versions of a single database with
parametric test

Test a feature over multiple databases

Test your app against (or supported by) a complex env:
queue, kv-store, log aggregator, search engine

Use a compose file if necessary

PostgreSQL container
@Container
public static PostgreSQLContainer container = new PostgreSQLContainer();
@Test
public void shouldTestSimpleQuery() throws SQLException {
Connection conn = DriverManager.getConnection(container.getJdbcUrl(),
container.getUsername(), container.getPassword());
Statement stmt = conn.createStatement();
stmt.execute("SELECT 1");
ResultSet resultSet = stmt.getResultSet();
resultSet.next();
assertThat(resultSet.getInt(1)).isEqualTo(1);
}

Jdbc url and init method
@Test
public void shouldSelectFromBar() throws SQLException {
String jdbcUrl =
"jdbc:tc:postgresql:9.6.8://hostname/databasename?&TC_INITFUNCTION=io.github.robfrank.testc
ontainers.JavaJdbcUrlTest::sampleInitFunction";
Connection conn = DriverManager.getConnection(jdbcUrl);
stmt.execute("SELECT * FROM bar");
resultSet.next();
assertThat(resultSet.getString("foo")).isEqualTo("hello world");
}

Jdbc url and init method
public static void sampleInitFunction(Connection connection) throws SQLException {
connection.createStatement().execute("CREATE TABLE bar (n" +
" foo VARCHAR(255)n" +
");");
connection.createStatement().execute("INSERT INTO bar (foo) VALUES ('hello world');");
connection.createStatement().execute("CREATE TABLE my_counter (n" +
" n INTn" +
");");
}

Jdbc url script
@Test
public void shouldSelectFromBar() throws SQLException {
String jdbcUrl =
"jdbc:tc:postgresql:9.6.8://hostname/databasename?&TC_INITSCRIPT=initdb.sql";
Connection conn = DriverManager.getConnection(jdbcUrl);
stmt.execute("SELECT * FROM bar");
resultSet.next();
assertThat(resultSet.getString("foo")).isEqualTo("hello world");
}

Generic container, image based
@Test
internal fun `should select beers vertexes`() {
OrientDB("remote:${container.containerIpAddress}:${container.firstMappedPort}", OrientDBConfig.defaultConfig()).use {
orientDB ->
orientDB.open("openbeer", "admin", "admin").use { db ->
db.query("select from Beer limit 10").use { resultSet ->
resultSet.asSequence()
.toList().apply {
assertThat(this).hasSize(10)
}.map { record ->
assertThat(record.isVertex).isTrue()
assertThat(record.hasProperty("name")).isTrue()
assertThat(record.hasProperty("descript")).isTrue()
record.vertex.get()
}.forEach { vertex: OVertex ->
assertThat(vertex.getEdges(ODirection.OUT)).isNotEmpty
}
}
}
}

Add to your project
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<version>1.12.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>MODULE_NAME</artifactId>
<version>1.12.0</version>
<scope>test</scope>
</dependency>

Use in test
@Container
val container = PostgreSQLContainer<Nothing>()
@Test fun `should perform simple query`() {
val conn = DriverManager.getConnection(container.jdbcUrl,
container.username,
container.password)
val stmt = conn.createStatement()
stmt.execute("SELECT 1")
val resultSet = stmt.resultSet
resultSet.next()
assertThat(resultSet.getInt(1)).isEqualTo(1)
}

https://github.com/robfrank/testcontainers-examples

ArcadeAnalytics connectors: https://github.com/arcadeAnalytics/arcade-connectors/
Kotlin and Java
Single container for multiple test classes
Neo4j, Postgres, Mysql, OrientDB, JanusGraph test and custom images
Alfresco Activiti: https://github.com/Activiti/activiti-cloud-query-service
Use of Testcontainers instead of maven plugin for lifecycle
Other examples

Integration tests: use the containers, Luke!

More Related Content

What's hot (20)

Similar to Integration tests: use the containers, Luke! (20)

More from Roberto Franchini (8)

Recently uploaded (20)

Integration tests: use the containers, Luke!