![Business Event Monitoring with
time series
http://d3js.org
https://square.github.io/cubism
“A time series is a sequence of data points, typically consisting of successive
measurements made over a time interval.”
[wikipedia]](https://image.slidesharecdn.com/whatwedowithgomala20150401t1231-190426072611/85/What-we-do-with-Go-18-320.jpg)
What we do with Go
- 1. What we do with Go Marcel Lanz, engineer, proCentric AG
- 2. agenda • how we started with Go • RichClient/JVM Distribution • Business Event Monitoring • using Go, why Go is Fun
- 3. a few problems solved • 2014 RichClient/JVM distribution (groupcache) • 2014 Business Event Monitoring with time series (d3/cubism) • 2014 Business Operation Monitor (ng) • 2015 OCR Solution (omnipage, rewritten (old) C++ server)
- 4. how we started with Go • Insurance Sector • Windows “Servers”, MSSQL, Winddows File Servers, ActiveDirectories… • but JEE/JBoss Java AppServer • eclipse RCP based RichClient, about 200 users • multiple generations (32bit/64bit, old OCR 32bit Solution)
- 5. SOAPCaller • “new” external SOAP Service • Monitoring: no SOAP support • external sensors via exit code and stdout • *.bat, java app, Windows-Script… => why not Go
- 6. SOAPCaller • SOAPCaller • send SOAP request • receive and parse SOAP response (parse XML) • report via exit code & stdout • => one exe-File, starts & runs in 200ms, simple deployment
- 7. network issues (1) • new infrastructure (network and servers) • RichClient App running on SMB Share • ClassNotFound Exceptions in random order • unreliable Network (L3 switches, ProxyARP issues)
- 8. network issues (2) • “the network is slow”, “something is wrong” • blocking Clients • business processes unavailable / blocking • printers unavailable 25% of the time => L3 Switches
- 9. RichClient/JVM distribution • Windows SW distribution not yet available • don’t run apps on “shares” • distribute them to the client • update, staging and multiple locations
- 10. Indexing Server • walk over a given directory by configuration (JSON) • read files into memory, SHA1 hash them and put them into a map • combine configuration and the file index into a boot file • provide the boot file over HTTP
- 12. ClientStarter (1) • ClientStarter.exe + server.json (JAVA Swing App, launch4j EXE) • on every start • download boot.json from Index Server • synchronize index files locally • boot the app
- 13. ClientStarter (2) • a tuesday afternoon • worked more or less in second try • structs, JSON en-/decoding, filepath.Walk(), maps, “net/http”
- 14. Issues • after a few hours, server crashed, dump in windows memory allocation/management • debug.WriteHeapDump(f.Fd()) • https://golang.org/issue/8119 (solved within 2 days) • => switched to Windows 64Bit
- 16. ClientStarter 2.0 • multiple remote locations (DE, IT, FR) • staging, jre6 ➡ GroupCache (github.com/golang/groupcache) (small patch for server selection) ➡ expiring keys for boot.json => boot.json_20140701T165_ ➡ took another afternoon (+ a bit more to polish) ➡ runs over a year without issues
- 18. Business Event Monitoring with time series http://d3js.org https://square.github.io/cubism “A time series is a sequence of data points, typically consisting of successive measurements made over a time interval.” [wikipedia]
- 19. Quotes from Business Support • “its slow” • “at 9am, we had a lot of blocking clients” • “something is wrong” • “we are missing documents in ‘Postfach’”
- 20. what do you see ?
- 25. first try
- 26. a morning
- 27. correlate events
- 28. working day 1
- 29. working day 2
- 30. what is this?
- 31. printing Issues
- 32. Post Mortem
- 33. JEV (Jahresendverarbeitung) • 24 “Node Cluster” JBoss • 1 Batch Master • year before took x hours => distribute JBoss + config with “ClientStarter” => monitor it with EventAgent
- 34. visual perception • colors (Cyntia Brewer) • correlate events • event count is not that important (one, a few, a lot) • see: http://edwardtufte.com/ - http://colorbrewer2.org • “PowerPoint Does Rocket Science”
- 35. • Senior JAVA Developer: ”I can do that with JAVA too" • how to deploy that in a JVM based ops infrastructure? • which IDE do you use for that? using Go? (1)
- 36. • stable and reliable • nice programming environment, also for Windows • easy to learn, docs and spec are enough • POLA, (principle of least astonishment) using Go? (2)
- 37. Why Go is fun ? • Go is boring (kind of)… but thats good • simple, elegant, orthogonal, POLA • interfaces (not the same as JAVA, might be confusing for them) • channels • select
- 38. Questions ?.. Thank you very much 2015-04-01 Marcel Lanz Senior System & Software Architect [email protected]