Ruby meetup 7_years_in_testing

7 Years In Testing
Tamás Tompa (Digital Natives)
tamas.tompa@digitalnatives.hu
Budapest Ruby Meetup 2013
11/09/2013
Thursday, September 12, 13

§ 7 years in ruby, rails
§ 7 years in webapp testing
§ 5 years in agile development
§ 5 years in automated testing

Looking for testing
best practices

Short version
INTEGRATE
AUTOMATE

Short version
INTEGRATE
AUTOMATE
(everything)

Human VS Machine
§ Use humans for thinking
§ Use machines for executing
§ Humans can forgive, forget things
§ Machines are strict, consequent

Theory

Development Workflow
Resolved Deployed
Testing /
Working
Testing /
Not
working
In
progress Stable
for feature based development

Automating steps
Resolved Deployed
Testing /
Working
Testing /
Not
working
In
progress Stable
Manual Automatic Manual Automatic

Automated Testing
§ rspec
§ turnip
§ cucumber
§ capybara
§ selenium -> phantomjs
§ factory girl
§ jenkins / cloud CI
TOOLS

Working, bug free software is
priceless...

Working, bug free software is
more expensive...

... but only if you don’t know
how to test
and maintain your code

Write tests for those parts of the
code which are...
§ the most often used
§ changing frequently
§ changing data and working with
financial data
§ more likely to fail

Instrumentation
§ Instrumentation means that the code gets enhanced with
flags, and when the execution passes a certain flag, it
gets set. When a test code coverage measurement is
made then the tool:
§ instruments the code - places these flags
§ runs the test cases - flags get set
§ and finally prints out the result - how many flags have
been set
§ During usage coverage measurement, test execution is
replaced by customer interaction.

Most often used code
§ Instrument the code base and deliver
it to the testers / customer
§ Turn on coverage measurement in your
(staging) application
§ Deploy it
§ Do manual testing of critical functions
§ Summarize and analyze metrics
§ Write tests for highly covered classes

Frequently changing code
§ Use version control system logs
§ Gitinspector
§ StatSVN
§ Find out which files change often!

Working with data
§ Make code reviews
§ Re-prioritize classes and methods
§ Add more priority when data is critical
§ Move to top when working with money

Failing, problematic code
§ Find and analyze bugs in your Issue
Tracker System
§ Write test before fixing the issue
§ Review CI server logs
§ collect often failing tests
§ Refactor unstable code
§ refactor your tests also

Git branching
§ Work only with feature branches
§ Test the branches separately
§ Use Pull Requests to integrate
feature branches to a development
branch
§ Do more (manual) testing on
development branch
§ Merge easily to stable,
than finally to production

Methods
§ TDD - it’s up to you :)
§ BDD - based on TDD + domain
specific acceptance criterias
§ CDD - for prototypes, spikes

In practice...

Manual Testing
§ Plan your test cases
§ at Sprint Planning Meetings
§ Allocate a good Tester to the team
§ If you can’t, test all User Story after
completion, within team, but between
team members
§ Try to measure coverage during
manual testing

Continuous Integration
§ Jenkins server
§ Cloud alternatives: travis-ci, circleci, drone.io
§ Parallel job execution (queues / executors)
§ Main branches VS Feature branches
§ Instant builds VS Daily builds
§ Conflicting databases and shared resources
§ CI rake task in the project
§ Re-run logic for Cucumber tests
§ Long running jobs

CI - Ruby, Rails
§ Jenkins / RVM Plugin
§ RVM-managed environment:
ruby-1.9.3-p392@PROJECTNAME
§ Jenkins / Ruby Metrics Plugin
§ Rcov coverage report
§ Other stats
§ Bundler integration

CI - Feature branches
§ Jenkins / Git Plugin
§ Branches to build:
^(?!.*master|stable|development|origin/HEAD).*$
§ Jenkins / Version Number Plugin
§ Version Number Format String:
#${BUILD_NUMBER}-${GIT_BRANCH}

CI - Build types
§ Instant builds
§ Higher priority
§ Immediately after commit
§ Build trigger: Poll SCM
§ Daily builds
§ Once a day
§ Long running tests
§ Build trigger: Build periodically

CI - Rake
task :commit => :clean do
Rake::Task['db:migrate'].invoke
Rake::Task['db:test:load'].invoke
Rake::Task['db:test:purge'].invoke
Rake::Task['ci:build:rspec_internal'].invoke
Rake::Task['ci:build:cucumber_internal'].invoke
end
rake ci:build:commit
> Example: https://gist.github.com/tompata/6513021

CI - Rerun
§ Re-run only previously failing tests
§ Re-try the randomly failing tests
§ Built-in support in Cucumber
§ Easy integration with Jenkins
§ Save resources and time

CI - Integration
§ Bitbucket
§ Redmine
§ Notifiers (HipChat, Email, Desktop)
§ Deployment (Capistrano)
§ Monitoring (Airbrake, New Relic)

Browser tests
§ integration tests
§ cucumber / capybara
§ responsive design
§ phantomjs / screen sizes
§ cross browser (design)
§ mtest (npm) - multiplatform
§ saucelabs / cloud
§ 162 device/os/browser platforms

Responsive design
§ PhantomJS
§ Screen resolutions (window_size)
§ Cucumber profiles

Stress tests
§ penetration / load testing before
releases
§ use separated, dedicated
environments (staging test)
§ use virtual instances (ec2)
§ monitor instances (new relic)
§ use online tools (loadimpact,
loader.io) or local (jmeter)

Coverage, Quality
§ rcov
§ simple cov
§ the magical 80%
§ code climate - code analysis, quality

INTEGRATE
AUTOMATE

Resources
§ http://zsoltfabok.com/blog/2011/05/narrow-down-what-to-test/
§ http://nvie.com/posts/a-successful-git-branching-model/
§ http://jenkins-ci.org/
§ https://wiki.jenkins-ci.org/display/JENKINS/Ruby+Metrics+Plugin
§ https://saucelabs.com/
§ https://github.com/digitalnatives/
§ https://github.com/digitalnatives/mtest
§ https://gist.github.com/tompata/6513021
§ https://codeclimate.com/
§ http://loadimpact.com/

We’re hiring!
http://digitalnatives.hu/jobs

Ruby meetup 7_years_in_testing

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Ruby meetup 7_years_in_testing (20)

More from Digital Natives (15)

Recently uploaded (20)

Ruby meetup 7_years_in_testing