![Sample Packages & Tools
Category Package/Tool Name Purpose
Deploy & Setup Virtualenv Tool to create Isolated Python env that would provide
consistent library versioning deployment of project
Dev Automation Jinja2.Template Templatize and auto-generate Class definitions, Unit test
module skeleton, Html report format generation etc to
minimize manual repetitive work
Dev Workflow git Quick project setup time.
Source control for collaborative & agile development, code
review processes
Dev Json;
Avro
Input/Output data persistence and movement;
To maintain versioning across releases.
Localized
Development and
Unit testing
[Connectors to data sources to
fetch sample data into project
framework]
Nosetest
For each unit-test “fetch” and persist sample data
representive of Positive & Negative test-cases for unit-test
local execution.
Automatically discover and run tests.
Continuous Testing Jenkins Continuous Integration – Run regression tests with every
check-in. Also run full functional-test with complete run-
time env and complete input data mimic’ing Production
Documentation Pydoc Major functions and util packages should be documented
with pydoc
Documentation &
Reporting
Sqllite database Any documents and report generation (for results analysis,
metadata analysis)](https://image.slidesharecdn.com/agilecollaborativepractices-150926063729-lva1-app6892/85/Agile-collaborative-practices-6-320.jpg)
Agile collaborative practices
- 1. Ingredients of Agile and Collaborative Development Model Python Based Projects [email protected]
- 2. Agenda • Introduction – Why adopt agile & collaborative model • Source Code Structure • Testing Considerations • Sample Packages & Tools
- 3. Introduction • Why Agility needs to be factored in – When you have to respond to adhoc new/changing requirements during the project sustenance lifecycle – When you need fast iterative go-to-prod cycles • Why Collaborative model needs to be factored in – When development needs to be de-centralized and multiple disparate teams contribute – Organizations may have pockets of “SMEs” that could collaborate on functionality in parallel • Testing Considerations – Testing & Automation has to be integrated at several stages – Localized unit testing of framework, utils and contributed modules
- 4. Source Code Structure • Identify and Segregate what constitutes “Framework” from rest of the components – Core Framework component, for instance, should be the only part of your code that follows “closed” development model – i.e a centralized team owns and maintains the “Framework” modules – “Framework” changes could be less frequent than rest of the components – Good example is a Framework that abstracts the “data store” interactions and prepares a Data-model to be consumed by other collaborative developers • Identify and segregate Non-framework modules – Non-framework modules should be “open” for collaborative development – Non-framework modules should be consumers of the Framework and can follow agile development schedules & practices independently – Good example of Non-framework module is “Rules” development. Rules could be developed collaboratively by “SMEs” from disparate teams. They could do so by subscribing and adhering to the core Framework published Interfaces (data-model, or getter APIs for instance) • Testing Considerations – Framework should provide unit-test modules and templates for both core framework and the non- framework modules – Framework should also provide “Integration testing” hooks – where the new changes could be executed on a target environment representing production-like dataset. This is crucial to help identify quality issues early in development considering short agile deployment cycles
- 5. Source Code Structure project_root/ framework/ rules_root/ framework/ src/ utils/ unittest/ conf/ rules_root/ data/ rules/ unittest/ - Closed, waterfall development - separate framework repo - Framework code packages, and runs the rules, abstracts data-store details - Collaborative, agile development - separate rules repo - Rules code only focuses on business logic using data-model/interfaces published by framework runtime
- 6. Sample Packages & Tools Category Package/Tool Name Purpose Deploy & Setup Virtualenv Tool to create Isolated Python env that would provide consistent library versioning deployment of project Dev Automation Jinja2.Template Templatize and auto-generate Class definitions, Unit test module skeleton, Html report format generation etc to minimize manual repetitive work Dev Workflow git Quick project setup time. Source control for collaborative & agile development, code review processes Dev Json; Avro Input/Output data persistence and movement; To maintain versioning across releases. Localized Development and Unit testing [Connectors to data sources to fetch sample data into project framework] Nosetest For each unit-test “fetch” and persist sample data representive of Positive & Negative test-cases for unit-test local execution. Automatically discover and run tests. Continuous Testing Jenkins Continuous Integration – Run regression tests with every check-in. Also run full functional-test with complete run- time env and complete input data mimic’ing Production Documentation Pydoc Major functions and util packages should be documented with pydoc Documentation & Reporting Sqllite database Any documents and report generation (for results analysis, metadata analysis)