Иосиф Иткин, Exactpro - TBA
Download as PPTX, PDF•0 likes•350 views
1) Exactpro is a specialist QA firm focused on testing financial systems that was acquired by the London Stock Exchange Group in 2015. 2) The London Stock Exchange Group is a leading international exchange group that traces its history back to 1698 and has over 5,500 employees. 3) Exactpro uses automated testing tools like Sailfish and ClearTH to test systems, as well as techniques like formal verification, crowd-sourced testing, and machine learning.
Иосиф Иткин, Exactpro - TBA
- 1. Machine Learning and Testing in London Stock Exchange Group Iosif Itkin, Co-Founder and CEO, Exactpro 9 April 2016, AIST Conference Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 2460, +1 415 830 38 49 www.exactpro.com
- 2. 2 Exactpro Systems Company Overview Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com Exactpro is: Clients Worldwide: • A specialist firm focused on functional and non functional testing of systems that process wholesale financial products, particularly market infrastructure • A UK company with operations in the US and four QA & software development centres in Eastern Europe • Part of London Stock Exchange Group as of May 29, 2015 • Incorporated in 2009 with 10 people, our company has experienced significant growth as satisfied clients require more services;
- 3. 3 London Stock Exchange Group Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com • A leading diversified international exchange and infrastructure group with assets across the entire exchange value chain • The Group can trace its history back to 1698 • Headquartered in the United Kingdom, with significant global operations, the Group employs approximately 5,500 people • Post trade and risk management services are a significant part of the Group’s business operations • Through FTSE Russell, the Group is a global leader in financial indexing, benchmarking and analytic services with approximately $10 trillion benchmarked to its indexes
- 4. 4 Tools & Methods of Program Analysis Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com Sailfish: Automated testing Compares intended and received results ClearTH: • Automated end 2 end clearing system testing Dolphin: Market surveillance testing Shsha: • Automated monitoring, analysis and reporting Load Injector: • 75K messages / second from a single CPU core • Measures latencies in microsecond range Minirobots: • A variety of algoes simulating end-clients will run and see how system performs Quality Assurance: test planning and managementProcess audit and test coverage analysis Intelligent Management of Large Data Sets Test data management Protocol level testing using FIX/FAST, SOAP, HTTP, ITCH, SWIFT, MQ, SQL, proprietary binary and text based data formats, etc. Automated regression testing Gathering requirements and test scenario creation (human, message & reporting interfaces) Creating and productizing state-of-the-art test harnesses Test automation Intelligent functional and exploratory testing Latency & capacity testing
- 5. 5 Quality Assurance Trends Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com FrAgile Process Crowd-sourced Testing Formal Verification Methodologies Cognitive Technology
- 6. 6 FrAgile Process Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com Test is Dead Ideas before Products Happy Software Developers Software Engineer in Test
- 7. 7 Crowd-Sourced Testing Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com Immediate Feedback from Diverse User Base Instrumentation is Required to Process Data
- 8. 8 Formal Methods Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com Model Checking Theorem Proving Static Analysis Verification
- 9. 9 What’s Next Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com Can We Learn from Other Industries? Be Both Faster and Safe Who Had the Most Severe Problems with Technology?
- 10. 10 Proposed Approach Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com Build Software to Test Software Test Tool Sophistication should Match Platform’s Complexity A Good Robot is Required to Survive in Robot Apocalypse
- 11. 11 Trading Systems Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com Testing Instruments, Risk Control and Monitoring Tools should have the Same Priority as the Trading Platform Testability is Required for Agile Process and Continuous Integration SET = Building Software to Test Software No Formal Artefacts from Agile Process. Parallel Stream for Test Harness Formal Verification can Improve Quality The Software will Break Anyway The Absence of Adequate Monitoring and Kill Switches is What Turns a Problem into a Disaster
- 12. 12 Passive Testing Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
- 13. 13 Test Tools for Client Onboarding and Certification Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com
- 14. Regular Test Automation 14 Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com Execute a test regression and make it: as extensive as possible as efficient as possible
- 15. 15 Minirobots Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com A set of Minirobots Algorithms allowing to perform a complete test regression of a trading system within a few hours All Robots work concurrently against the test environment Robots are independent from each other, and their actions are based on current conditions on the order books Robots use an abstract Test Library and their main goal is to cover all test cases within the library
- 16. Testing Algo-Trading System 16 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Deploy one or several scalable matching engines to serve as market simulators. Repeat your tests multiple times to assess the quality and efficiency of your software
- 17. Machine Learning 17 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com Select efficient limited subset of tests from an infinite number of possibilities Identify test cases that were actually executed and their status Teach robots to adhere to our values: Integrity, Excellence, Partnership and Innovation
- 18. Human Learning 18 Exactpro Systems, LLC Quality Assurance & Related Software Development for Innovative Trading Systems Tel: +7 495 640 24 60 , +1 415 830 38 49 exactpro.com https://ru.linkedin.com/in/iosifitkin http://extentconf.com