SlideShare a Scribd company logo

Eclipse Packaging Project Usage Data Collector

Markus Knauer
Markus Knauer

The Eclipse Packaging Project (EPP) and its Usage Data Collector (UDC) are designed to enhance the end-user experience and gather statistics on Eclipse usage in RCP and RAP applications. The UDC collects data such as commands executed and user interactions while ensuring user privacy through anonymization methods. Analysis of the collected data reveals usage patterns, including the most popular commands, views, and editors utilized by Eclipse users.

1 of 42
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
The Eclipse Packaging Project and its Usage Data Collector in RAP and RCP Applications Markus Knauer (EclipseSource) © 2009 EclipseSource | April 2009
Agenda... [0] Introducing EclipseSource [1] The Eclipse Packaging Project [2] The Usage Data Collector in RCP applications [3] Results and Statistics from the Ganymede Release [4] The Usage Data Collector in RAP applications [5] Analysing the results with VUA [6] Non-code aspects of a Usage Data Collector © 2009 EclipseSource | April 2009
EclipseSource - Who we are • Evolved out of Innoopract and Code 9 • Deep involvement in Eclipse from the beginning • Eclipse Strategic member • 2 board members (1 elected) • World class team of recognized experts • Proven delivery track record – Eclipse 1.0, 2.0, 2.1, 3.0, Callisto, Europa, Ganymede • Offices in Karlsruhe, Ottawa, Portland, Sofia Title | © 2009 EclipseSource
A strategic member of the Eclipse Foundation  12 strategic members, 170+ sustaining members  Strategic members commit to 8 full time developers working on the open source Eclipse codebase © 2009 EclipseSource | April 2009
Eclipse solutions are our business integration & delivery technology Yoxos provides security Open Source and tools that enable Projects for tools enterprises to manage and runtimes and deliver Sponsored development eclipse technology Maintenance services Helping corporations to implement Eclipse strategies, offering consulting, development, support and training © 2009 EclipseSource | April 2009
© 2009 EclipseSource | April 2009
EclipseSource Leadership at Eclipse  2 board members  Projects  Various board committees – Equinox co-lead  PMCs – RCP lead – RT PMC leaders – RAP lead – Eclipse PMC leader – EPP lead – Tools PMC member – g-Eclipse co-lead – Technology PMC member – ECF lead  5 Architecture Council – PDE co-lead members – Orbit lead  Mentor many projects – Zest lead © 2009 EclipseSource | April 2009
EPP - What's it all about? Create a better end-user experience. © 2009 EclipseSource | April 2009
EPP - What's it all about? • Eclipse Packages Download  coordinate, maintain, build, configure, ... • Usage Data Collector  How are developers using Eclipse? • Eclipse Dynamic Download  Wizard-based download configurator © 2009 EclipseSource | April 2009
EPP Usage Data Collector Included in all EPP packages  Millions of downloads from eclipse.org/downloads  Currently about 120.000 distinct users uploading every month UDC: Views, Perspectives, Bundles, Editors, Commands view executeCount200901 userCount200901 Average per User org.eclipse.jdt.ui.PackageExplorer 1546984 67514 23 org.eclipse.ui.console.ConsoleView 1084367 72544 15 org.eclipse.ui.navigator.ProjectExplorer 757932 41273 18 org.eclipse.wst.server.ui.ServersView 262302 19159 14 org.eclipse.ui.views.ResourceNavigator 239341 13033 18 Statistics available from http://www.eclipse.org/org/usagedata/results.php © 2009 EclipseSource | April 2009
EPP UDC in Your RCP Product Include the EPP UDC feature in your product But ... [1] The data is uploaded to eclipse.org ...  Only the Eclipse Foundation has access to the raw data  It is possible to change the upload URL with a property -Dorg.eclipse.epp.usagedata.recording.upload-url=http://udc.eclipse.org/upload.php [2] Where is the server application?  Eclipse Foundation uses its own implementation (PHP)  Source code not available  But it is easy to write your own application © 2009 EclipseSource | April 2009
EPP Architecture (Standard RCP) UsageDataService udc.eclipse.org UsageMonitor UsageMonitor UsageDataEventListener UsageDataEventListener UsageMonitor UsageMonitor accept(UDEvent) start/stopMonitoring() http usagedata.gathering usagedata.ui usagedata.recording (service, (preferences, dialogs) (filtering, uploading) monitors, events) Eclipse RCP Platform Client © 2009 EclipseSource | April 2009
UDC – Tracking Information: Monitors Extension Point org.eclipse.epp.usagedata.gathering.monitors • UsageMonitor: start/stopMonitoring(UDCService) • Examples  BundleUsageMonitor  PartUsageMonitor  Views, Perspectives, Editors  LogMonitor – Errors in the log file (IStatus.ERROR)  CommandUsageMonitor  SystemInfoMonitor  OS, architecture, windowing system, locale, JVM, etc. © 2009 EclipseSource | April 2009
UDC – Collecting Data: Event Listeners Extension Point org.eclipse.epp.usagedata.listeners.event • UsageDataEventListener: accept(UsageDataEvent event) • UsageDataEvent: String what, String kind, String description, String bundleId, String bundleVersion, long when • UsageDataRecorder (RCP Standard) stores data in CSV files .metadata/.plugins/org...epp.usagedata.recording/upload0.csv © 2009 EclipseSource | April 2009
UDC – Storing/Uploading Data: Uploaders Extension Point org.eclipse.epp.usagedata.recording.uploader • Interface Uploader: startUpload(), ... • Example:  AskUserUploader asks the user every 5 days to upload data to udc.eclipse.org – see UsageDataRecordingSettings © 2009 EclipseSource | April 2009
UDC – Identifying Users??? • Identifying users is a no-go, at least in open source. • For statistical reasons it is important to associate data to a certain user Solution: Create neutral UUID for user and workspace: • $HOME/.org.eclipse.epp.usagedata.recording.userId • WS/.metadata/.plugins/org.eclipse.epp.usagedata.recording/ .org.eclipse.epp.usagedata.recording.workspaceId © 2009 EclipseSource | April 2009
RCP UDC Example Data what kind bundleId bundleVersion description time started bundle org.polarion.eclipse.team.svn.connector.svnkit15 2.1.0.I20090213-1500 org.polarion.eclipse.team.svn.connector.svnkit15 1237484685840 activated workbench org.eclipse.ui.workbench 3.5.0.I20090129-1430 1237484687277 deactivated workbench org.eclipse.ui.workbench 3.5.0.I20090129-1430 1237484687292 activated workbench org.eclipse.ui.workbench 3.5.0.I20090129-1430 1237484747712 activated view org.eclipse.jdt.ui 3.5.0.v20090128-1300 org.eclipse.jdt.ui.PackageExplorer 1237484751790 started bundle org.eclipse.ui.cheatsheets 3.3.200.v20081205 org.eclipse.ui.cheatsheets 1237484757338 opened editor org.eclipse.pde.ui 3.5.0.v20090202-1400 org.eclipse.pde.ui.featureEditor 1237484758274 activated editor org.eclipse.pde.ui 3.5.0.v20090202-1400 org.eclipse.pde.ui.featureEditor 1237484758535 activated view org.eclipse.jdt.ui 3.5.0.v20090128-1300 org.eclipse.jdt.ui.PackageExplorer 1237484767806 opened editor org.eclipse.ui.editors 3.5.0.v20090125-2000 org.eclipse.ui.DefaultTextEditor 1237484782149 activated editor org.eclipse.ui.editors 3.5.0.v20090125-2000 org.eclipse.ui.DefaultTextEditor 1237484782207 executed command org.eclipse.ui.workbench.texteditor 3.5.0.v20090128-0800 org.eclipse.ui.edit.text.goto.lineEnd 1237484792864 executed command org.eclipse.ui.workbench.texteditor 3.5.0.v20090128-0800 org.eclipse.ui.edit.text.select.wordPrevious 1237484795809 © 2009 EclipseSource | April 2009
Feedback results – number of uploads 140000 120000 100000 80000 Old Users New Users 60000 40000 20000 0 01/04/08 01/05/08 01/06/08 01/07/08 01/08/08 01/09/08 01/10/08 01/11/08 01/12/08 01/01/09 01/02/09 120.000 distinct users are uploading data every month. http://www.eclipse.org/org/usagedata/results.php © 2009 EclipseSource | April 2009
Analysing the Data Here is some data from January 2009. © 2009 EclipseSource | April 2009
Feedback results – Commands (Execution) command executeCount200901 userCount200901 # per user org.eclipse.ui.edit.delete 4228796 75815 56 org.eclipse.ui.file.save 3629929 79836 45 org.eclipse.ui.edit.paste 2756112 84245 33 org.eclipse.ui.edit.copy 1938088 77638 25 org.eclipse.debug.ui.commands.StepOver 1316288 17236 76 org.eclipse.ui.edit.text.goto.lineEnd 1217421 47416 26 org.eclipse.ui.edit.undo 1089139 60178 18 org.eclipse.ui.edit.text.contentAssist.proposals 978616 39627 25 org.eclipse.ui.edit.text.goto.lineStart 966017 39241 25 org.eclipse.ui.edit.text.goto.wordNext 886107 16637 53 org.eclipse.ui.edit.text.goto.wordPrevious 715297 16595 43 org.eclipse.ui.edit.text.select.wordPrevious 493556 15786 31 org.eclipse.ui.edit.text.select.wordNext 485328 15669 31 What's the most used command? Delete??? © 2009 EclipseSource | April 2009
Feedback results – Commands (Execution by User) command executeCount200901 userCount200901 # per user org.eclipse.ui.edit.text.goto.lineDown 59786 427 140 org.eclipse.ui.edit.text.goto.lineUp 49540 395 125 org.eclipse.ui.edit.text.goto.columnNext 41878 399 105 org.eclipse.ui.edit.text.deletePrevious 5383 63 85 org.eclipse.ui.edit.text.goto.columnPrevious 30553 359 85 org.eclipse.debug.ui.commands.StepOver 1316288 17236 76 org.eclipse.ui.edit.delete 4228796 75815 56 org.eclipse.ui.edit.text.goto.wordNext 886107 16637 53 org.eclipse.ui.edit.text.deleteNext 22237 438 51 org.eclipse.ui.file.save 3629929 79836 45 org.eclipse.ui.edit.text.goto.wordPrevious 715297 16595 43 org.eclipse.ui.edit.text.cut.line.to.end 22134 580 38 org.eclipse.ui.edit.findNext 135955 4063 33 org.eclipse.ui.edit.paste 2756112 84245 33 If someone knows this it is used very often. © 2009 EclipseSource | April 2009
Users of the Ganymede Release... • How many different Views are being used?  ~ 120 • How many different Editors are being used?  ~ 80 • How many Perspectives are being used?  ~ 30 © 2009 EclipseSource | April 2009
Perspective – Top 5 perspective executeCount200901 userCount200901 org.eclipse.jdt.ui.JavaPerspective 307220 69176 org.eclipse.jst.j2ee.J2EEPerspective 120420 38447 org.eclipse.debug.ui.DebugPerspective 90679 23079 org.eclipse.team.ui.TeamSynchronizingPerspective 57534 12093 org.eclipse.cdt.ui.CPerspective 39212 9960 • Most people do Java programming  Followed by J2EE and C/C++ • Is creating another Perspective a good idea? © 2009 EclipseSource | April 2009
Views – Top 10 view executeCount200901 userCount200901 org.eclipse.jdt.ui.PackageExplorer 1546984 67514 org.eclipse.ui.console.ConsoleView 1084367 72544 org.eclipse.ui.navigator.ProjectExplorer 757932 41273 org.eclipse.wst.server.ui.ServersView 262302 19159 org.eclipse.ui.views.ResourceNavigator 239341 13033 org.eclipse.debug.ui.DebugView 229363 22349 org.eclipse.search.ui.views.SearchView 216037 21674 org.eclipse.ui.views.ContentOutline 209932 36984 org.eclipse.ui.views.ProblemView 173191 40000 org.eclipse.team.sync.views.SynchronizeView 137594 13251 Package Explorer and the Console View... Who thought this? © 2009 EclipseSource | April 2009
Editors – Top 10 editor executeCount200901 userCount200901 org.eclipse.jdt.ui.CompilationUnitEditor 4878831 78668 org.eclipse.cdt.ui.editor.CEditor 559440 8932 org.eclipse.jst.jsp.core.jspsource.source 438950 16762 org.eclipse.wst.xml.ui.internal.tabletree.XMLMultiPageEditorPart 393683 28678 org.eclipse.ui.DefaultTextEditor 231430 25574 org.eclipse.jdt.ui.ClassFileEditor 120639 20771 org.eclipse.ui.browser.editor 118697 9564 org.eclipse.compare.CompareEditor 115190 12226 org.eclipse.wst.html.core.htmlsource.source 96529 8103 org.eclipse.jdt.ui.PropertiesFileEditor 83958 12187 Very popular: Java, C/C++, XML, ... © 2009 EclipseSource | April 2009
EPP Usage Data Collector UDC available for RCP applications, but... NEW: Extension for RAP-based environments but this is only half of the story... NEW: Widget based monitor ...and how do we analyse the data? NEW: VUA – Visual Usage Analytics © 2009 EclipseSource | April 2009
Inside the EPP Usage Data Collector © 2009 EclipseSource | April 2009
Additional Monitor: Widget based monitor Problem: In order to visualize user interactions, one needs to track the widget (button, etc.) that was responsible for the event. Solution: Widgets that need tracking must be marked manually with a distinct identifier! widgetXY.setData( "org.eclipse.epp.udc", "yourInternalIdentifier" ); © 2009 EclipseSource | April 2009
UDC and VUA Interaction – Reference Architecture © 2009 EclipseSource | April 2009
The Database © 2009 EclipseSource | April 2009
The Setup for the Demonstration [1] Derby as database system export DERBY_HOME=... ./bin/startNetworkServer -h 127.0.0.1 [2] Apache ActiveMQ as queuing service ./activemq http://0.0.0.0:8161/admin/queues.jsp [3] MQC (Message Queue Client) – reads the queue and writes to the database java -jar mqc.jar © 2009 EclipseSource | April 2009
EPP Visual Usage Data Analytics (VUA) • The VUA allows the interactive analysis of the usage data • No code changes • It only needs a connection to the DBMS • It can be used in both, RCP and RAP environments • See DEMO © 2009 EclipseSource | April 2009
VUA – A custom widget Button Example: Button btn = new Button(showcase1, SWT.None); btn.setData( "org.eclipse.epp.udc", "btn1ShowCaseCustomTracking" ); Combo Box Example: Combo comboDropDown = new Combo(showcase1, SWT.DROP_DOWN | SWT.BORDER); comboDropDown.setData( "org.eclipse.epp.udc", "combo1ShowCaseCustomTracking" ); © 2009 EclipseSource | April 2009
VUA – Compare User Interactions (Example) Command is available – by pressing a button Button btn = new Button(parent, SWT.None); btn.setText( "send Message" ); btn.setData( "org.eclipse.epp.udc", "btnSendMessage" ); btn.addSelectionListener( new SelectionAdapter() { ... – from the toolbar Comparison by database query defined in plugin.xml: <extension point="com.innoopract.vua.visualization.queries"> <queries id="Comparison" query_string="select a.ID, a.KIND, a.DESCRIPTION, ..."> <output format="0.00%" quotient="true" /> ... © 2009 EclipseSource | April 2009
EPP UDC VUA Screenshot – The Application © 2009 EclipseSource | April 2009
EPP UDC VUA Screenshot – The UDC Results © 2009 EclipseSource | April 2009
Non-code Aspects: Privacy and User Data © 2009 EclipseSource | April 2009
Other Examples: Debian Popularity Contest You can help out the Debian Project by sending your popularity-contest results to us anonymously. With information from enough people, we can determine which packages are most often used, which are often installed but not used, and which are nearly never used. (...) http://popcon.debian.org/ © 2009 EclipseSource | April 2009
Non-code aspects of the Usage Data Collector • Make it as transparent as possible! • Tell your users what you are going to do! • Let the user decide! - Opt-In • Check for possible legal issues! • Let the users see the results! • Make the source code available! Keep in mind that usage data is sensitive data! © 2009 EclipseSource | April 2009
Further Information EPP Project Webpage  http://www.eclipse.org/epp EPP Usage Data Collector and VUA  http://www.eclipse.org/epp/usagedata/index.php  http://www.eclipse.org/org/usagedata/  http://wiki.eclipse.org/EPP/Framework_for_Visual_Analytics_of_user_tracking_information Mailing List  https://dev.eclipse.org/mailman/listinfo/epp-dev © 2009 EclipseSource | April 2009
Backup © 2009 EclipseSource | April 2009
RCP vs. RAP – Architecture http://www.eclipse.org/rap/ © 2009 EclipseSource | April 2009

More Related Content

Viewers also liked (7)

PPTX
Packaging design ppt
Ishita Sachdeva
 
PPTX
Food Packaging (Food Processing Technology)
Supta Sarkar
 
PPT
Packaging Design Conference 12-09
Paul Vickers
 
PPSX
Packaging & labeling in food industries
naveenaugust
 
PPTX
PACKAGING OF TABLETS: TYPES, MATERIALS AND QC.
Akshay Joshi
 
PPTX
Packaging ppt
Shraddha Kurdekar
 
PPTX
Blister packing
CCL Pharmaceuticals
 
Packaging design ppt
Ishita Sachdeva
 
Food Packaging (Food Processing Technology)
Supta Sarkar
 
Packaging Design Conference 12-09
Paul Vickers
 
Packaging & labeling in food industries
naveenaugust
 
PACKAGING OF TABLETS: TYPES, MATERIALS AND QC.
Akshay Joshi
 
Packaging ppt
Shraddha Kurdekar
 
Blister packing
CCL Pharmaceuticals
 

Similar to Eclipse Packaging Project Usage Data Collector (20)

PDF
P2 Introduction
irbull
 
PPT
Leveraging the Eclipse Ecosystem for the Scientific Community
guestd41014
 
PPT
Eclipse Training - Main eclipse ecosystem classes
Luca D'Onofrio
 
PDF
Eclipse Way
Tomasz Zarna
 
PDF
Eclipse plug in development
Martin Toshev
 
PDF
Who Took The Cookie From The Cookie Jar?
Olivier Thomann
 
PDF
Open Source governance and the Eclipse Foundation, OW2online, June 2020
OW2
 
PDF
Keynote de Mike Milinkovich
EclipseDayParis
 
PPT
ABC of Platform Workspace
Tomasz Zarna
 
PDF
Improve Your IBM Domino Designer Experience
panagenda
 
PPTX
Eclipse Developement @ Progress Software
sriikanthp
 
PPTX
EclipseDay Lyon: What’s happening these days at Eclipse?
Philippe Krief
 
PDF
Focus Group Open Source 28.4.2010 Paolo Maresca
Roberto Galoppini
 
PPT
Together in Eclipse
Tomasz Zarna
 
PPT
Eclipse Training - Introduction
Luca D'Onofrio
 
PDF
Single Sourcing
Ralf Sternberg
 
PDF
Eclipse Rich Client Platform 2nd Edition Jeff Mcaffer
shirkamiahlp
 
PDF
Eclipse Rich Client Platform 2nd Edition Jeff Mcaffer
marijgabari
 
PPT
SIGAda Hibachi Workshop Presentation
Gneuromante canalada.org
 
PDF
Eclipse Day India 2015 - Keynote (Mike Milinkovich)
Eclipse Day India
 
P2 Introduction
irbull
 
Leveraging the Eclipse Ecosystem for the Scientific Community
guestd41014
 
Eclipse Training - Main eclipse ecosystem classes
Luca D'Onofrio
 
Eclipse Way
Tomasz Zarna
 
Eclipse plug in development
Martin Toshev
 
Who Took The Cookie From The Cookie Jar?
Olivier Thomann
 
Open Source governance and the Eclipse Foundation, OW2online, June 2020
OW2
 
Keynote de Mike Milinkovich
EclipseDayParis
 
ABC of Platform Workspace
Tomasz Zarna
 
Improve Your IBM Domino Designer Experience
panagenda
 
Eclipse Developement @ Progress Software
sriikanthp
 
EclipseDay Lyon: What’s happening these days at Eclipse?
Philippe Krief
 
Focus Group Open Source 28.4.2010 Paolo Maresca
Roberto Galoppini
 
Together in Eclipse
Tomasz Zarna
 
Eclipse Training - Introduction
Luca D'Onofrio
 
Single Sourcing
Ralf Sternberg
 
Eclipse Rich Client Platform 2nd Edition Jeff Mcaffer
shirkamiahlp
 
Eclipse Rich Client Platform 2nd Edition Jeff Mcaffer
marijgabari
 
SIGAda Hibachi Workshop Presentation
Gneuromante canalada.org
 
Eclipse Day India 2015 - Keynote (Mike Milinkovich)
Eclipse Day India
 
Ad

More from Markus Knauer (6)

PDF
Web Applications with Eclipse RT and Docker in the Cloud
Markus Knauer
 
PDF
Eclipse RAP 2.0, 2.1, 2.2, and beyond.
Markus Knauer
 
PDF
Sovereign: Migrating Java Threads to Improve Availability of Web Applications
Markus Knauer
 
ODP
Searching The Cloud - The eclipseRT Umbrella
Markus Knauer
 
PDF
Raindance - Tooling for the Clouds
Markus Knauer
 
PDF
Cloud Computing and Eclipse technology - how does it fit together?
Markus Knauer
 
Web Applications with Eclipse RT and Docker in the Cloud
Markus Knauer
 
Eclipse RAP 2.0, 2.1, 2.2, and beyond.
Markus Knauer
 
Sovereign: Migrating Java Threads to Improve Availability of Web Applications
Markus Knauer
 
Searching The Cloud - The eclipseRT Umbrella
Markus Knauer
 
Raindance - Tooling for the Clouds
Markus Knauer
 
Cloud Computing and Eclipse technology - how does it fit together?
Markus Knauer
 
Ad

Eclipse Packaging Project Usage Data Collector

  • 1. The Eclipse Packaging Project and its Usage Data Collector in RAP and RCP Applications Markus Knauer (EclipseSource) © 2009 EclipseSource | April 2009
  • 2. Agenda... [0] Introducing EclipseSource [1] The Eclipse Packaging Project [2] The Usage Data Collector in RCP applications [3] Results and Statistics from the Ganymede Release [4] The Usage Data Collector in RAP applications [5] Analysing the results with VUA [6] Non-code aspects of a Usage Data Collector © 2009 EclipseSource | April 2009
  • 3. EclipseSource - Who we are • Evolved out of Innoopract and Code 9 • Deep involvement in Eclipse from the beginning • Eclipse Strategic member • 2 board members (1 elected) • World class team of recognized experts • Proven delivery track record – Eclipse 1.0, 2.0, 2.1, 3.0, Callisto, Europa, Ganymede • Offices in Karlsruhe, Ottawa, Portland, Sofia Title | © 2009 EclipseSource
  • 4. A strategic member of the Eclipse Foundation  12 strategic members, 170+ sustaining members  Strategic members commit to 8 full time developers working on the open source Eclipse codebase © 2009 EclipseSource | April 2009
  • 5. Eclipse solutions are our business integration & delivery technology Yoxos provides security Open Source and tools that enable Projects for tools enterprises to manage and runtimes and deliver Sponsored development eclipse technology Maintenance services Helping corporations to implement Eclipse strategies, offering consulting, development, support and training © 2009 EclipseSource | April 2009
  • 6. © 2009 EclipseSource | April 2009
  • 7. EclipseSource Leadership at Eclipse  2 board members  Projects  Various board committees – Equinox co-lead  PMCs – RCP lead – RT PMC leaders – RAP lead – Eclipse PMC leader – EPP lead – Tools PMC member – g-Eclipse co-lead – Technology PMC member – ECF lead  5 Architecture Council – PDE co-lead members – Orbit lead  Mentor many projects – Zest lead © 2009 EclipseSource | April 2009
  • 8. EPP - What's it all about? Create a better end-user experience. © 2009 EclipseSource | April 2009
  • 9. EPP - What's it all about? • Eclipse Packages Download  coordinate, maintain, build, configure, ... • Usage Data Collector  How are developers using Eclipse? • Eclipse Dynamic Download  Wizard-based download configurator © 2009 EclipseSource | April 2009
  • 10. EPP Usage Data Collector Included in all EPP packages  Millions of downloads from eclipse.org/downloads  Currently about 120.000 distinct users uploading every month UDC: Views, Perspectives, Bundles, Editors, Commands view executeCount200901 userCount200901 Average per User org.eclipse.jdt.ui.PackageExplorer 1546984 67514 23 org.eclipse.ui.console.ConsoleView 1084367 72544 15 org.eclipse.ui.navigator.ProjectExplorer 757932 41273 18 org.eclipse.wst.server.ui.ServersView 262302 19159 14 org.eclipse.ui.views.ResourceNavigator 239341 13033 18 Statistics available from http://www.eclipse.org/org/usagedata/results.php © 2009 EclipseSource | April 2009
  • 11. EPP UDC in Your RCP Product Include the EPP UDC feature in your product But ... [1] The data is uploaded to eclipse.org ...  Only the Eclipse Foundation has access to the raw data  It is possible to change the upload URL with a property -Dorg.eclipse.epp.usagedata.recording.upload-url=http://udc.eclipse.org/upload.php [2] Where is the server application?  Eclipse Foundation uses its own implementation (PHP)  Source code not available  But it is easy to write your own application © 2009 EclipseSource | April 2009
  • 12. EPP Architecture (Standard RCP) UsageDataService udc.eclipse.org UsageMonitor UsageMonitor UsageDataEventListener UsageDataEventListener UsageMonitor UsageMonitor accept(UDEvent) start/stopMonitoring() http usagedata.gathering usagedata.ui usagedata.recording (service, (preferences, dialogs) (filtering, uploading) monitors, events) Eclipse RCP Platform Client © 2009 EclipseSource | April 2009
  • 13. UDC – Tracking Information: Monitors Extension Point org.eclipse.epp.usagedata.gathering.monitors • UsageMonitor: start/stopMonitoring(UDCService) • Examples  BundleUsageMonitor  PartUsageMonitor  Views, Perspectives, Editors  LogMonitor – Errors in the log file (IStatus.ERROR)  CommandUsageMonitor  SystemInfoMonitor  OS, architecture, windowing system, locale, JVM, etc. © 2009 EclipseSource | April 2009
  • 14. UDC – Collecting Data: Event Listeners Extension Point org.eclipse.epp.usagedata.listeners.event • UsageDataEventListener: accept(UsageDataEvent event) • UsageDataEvent: String what, String kind, String description, String bundleId, String bundleVersion, long when • UsageDataRecorder (RCP Standard) stores data in CSV files .metadata/.plugins/org...epp.usagedata.recording/upload0.csv © 2009 EclipseSource | April 2009
  • 15. UDC – Storing/Uploading Data: Uploaders Extension Point org.eclipse.epp.usagedata.recording.uploader • Interface Uploader: startUpload(), ... • Example:  AskUserUploader asks the user every 5 days to upload data to udc.eclipse.org – see UsageDataRecordingSettings © 2009 EclipseSource | April 2009
  • 16. UDC – Identifying Users??? • Identifying users is a no-go, at least in open source. • For statistical reasons it is important to associate data to a certain user Solution: Create neutral UUID for user and workspace: • $HOME/.org.eclipse.epp.usagedata.recording.userId • WS/.metadata/.plugins/org.eclipse.epp.usagedata.recording/ .org.eclipse.epp.usagedata.recording.workspaceId © 2009 EclipseSource | April 2009
  • 17. RCP UDC Example Data what kind bundleId bundleVersion description time started bundle org.polarion.eclipse.team.svn.connector.svnkit15 2.1.0.I20090213-1500 org.polarion.eclipse.team.svn.connector.svnkit15 1237484685840 activated workbench org.eclipse.ui.workbench 3.5.0.I20090129-1430 1237484687277 deactivated workbench org.eclipse.ui.workbench 3.5.0.I20090129-1430 1237484687292 activated workbench org.eclipse.ui.workbench 3.5.0.I20090129-1430 1237484747712 activated view org.eclipse.jdt.ui 3.5.0.v20090128-1300 org.eclipse.jdt.ui.PackageExplorer 1237484751790 started bundle org.eclipse.ui.cheatsheets 3.3.200.v20081205 org.eclipse.ui.cheatsheets 1237484757338 opened editor org.eclipse.pde.ui 3.5.0.v20090202-1400 org.eclipse.pde.ui.featureEditor 1237484758274 activated editor org.eclipse.pde.ui 3.5.0.v20090202-1400 org.eclipse.pde.ui.featureEditor 1237484758535 activated view org.eclipse.jdt.ui 3.5.0.v20090128-1300 org.eclipse.jdt.ui.PackageExplorer 1237484767806 opened editor org.eclipse.ui.editors 3.5.0.v20090125-2000 org.eclipse.ui.DefaultTextEditor 1237484782149 activated editor org.eclipse.ui.editors 3.5.0.v20090125-2000 org.eclipse.ui.DefaultTextEditor 1237484782207 executed command org.eclipse.ui.workbench.texteditor 3.5.0.v20090128-0800 org.eclipse.ui.edit.text.goto.lineEnd 1237484792864 executed command org.eclipse.ui.workbench.texteditor 3.5.0.v20090128-0800 org.eclipse.ui.edit.text.select.wordPrevious 1237484795809 © 2009 EclipseSource | April 2009
  • 18. Feedback results – number of uploads 140000 120000 100000 80000 Old Users New Users 60000 40000 20000 0 01/04/08 01/05/08 01/06/08 01/07/08 01/08/08 01/09/08 01/10/08 01/11/08 01/12/08 01/01/09 01/02/09 120.000 distinct users are uploading data every month. http://www.eclipse.org/org/usagedata/results.php © 2009 EclipseSource | April 2009
  • 19. Analysing the Data Here is some data from January 2009. © 2009 EclipseSource | April 2009
  • 20. Feedback results – Commands (Execution) command executeCount200901 userCount200901 # per user org.eclipse.ui.edit.delete 4228796 75815 56 org.eclipse.ui.file.save 3629929 79836 45 org.eclipse.ui.edit.paste 2756112 84245 33 org.eclipse.ui.edit.copy 1938088 77638 25 org.eclipse.debug.ui.commands.StepOver 1316288 17236 76 org.eclipse.ui.edit.text.goto.lineEnd 1217421 47416 26 org.eclipse.ui.edit.undo 1089139 60178 18 org.eclipse.ui.edit.text.contentAssist.proposals 978616 39627 25 org.eclipse.ui.edit.text.goto.lineStart 966017 39241 25 org.eclipse.ui.edit.text.goto.wordNext 886107 16637 53 org.eclipse.ui.edit.text.goto.wordPrevious 715297 16595 43 org.eclipse.ui.edit.text.select.wordPrevious 493556 15786 31 org.eclipse.ui.edit.text.select.wordNext 485328 15669 31 What's the most used command? Delete??? © 2009 EclipseSource | April 2009
  • 21. Feedback results – Commands (Execution by User) command executeCount200901 userCount200901 # per user org.eclipse.ui.edit.text.goto.lineDown 59786 427 140 org.eclipse.ui.edit.text.goto.lineUp 49540 395 125 org.eclipse.ui.edit.text.goto.columnNext 41878 399 105 org.eclipse.ui.edit.text.deletePrevious 5383 63 85 org.eclipse.ui.edit.text.goto.columnPrevious 30553 359 85 org.eclipse.debug.ui.commands.StepOver 1316288 17236 76 org.eclipse.ui.edit.delete 4228796 75815 56 org.eclipse.ui.edit.text.goto.wordNext 886107 16637 53 org.eclipse.ui.edit.text.deleteNext 22237 438 51 org.eclipse.ui.file.save 3629929 79836 45 org.eclipse.ui.edit.text.goto.wordPrevious 715297 16595 43 org.eclipse.ui.edit.text.cut.line.to.end 22134 580 38 org.eclipse.ui.edit.findNext 135955 4063 33 org.eclipse.ui.edit.paste 2756112 84245 33 If someone knows this it is used very often. © 2009 EclipseSource | April 2009
  • 22. Users of the Ganymede Release... • How many different Views are being used?  ~ 120 • How many different Editors are being used?  ~ 80 • How many Perspectives are being used?  ~ 30 © 2009 EclipseSource | April 2009
  • 23. Perspective – Top 5 perspective executeCount200901 userCount200901 org.eclipse.jdt.ui.JavaPerspective 307220 69176 org.eclipse.jst.j2ee.J2EEPerspective 120420 38447 org.eclipse.debug.ui.DebugPerspective 90679 23079 org.eclipse.team.ui.TeamSynchronizingPerspective 57534 12093 org.eclipse.cdt.ui.CPerspective 39212 9960 • Most people do Java programming  Followed by J2EE and C/C++ • Is creating another Perspective a good idea? © 2009 EclipseSource | April 2009
  • 24. Views – Top 10 view executeCount200901 userCount200901 org.eclipse.jdt.ui.PackageExplorer 1546984 67514 org.eclipse.ui.console.ConsoleView 1084367 72544 org.eclipse.ui.navigator.ProjectExplorer 757932 41273 org.eclipse.wst.server.ui.ServersView 262302 19159 org.eclipse.ui.views.ResourceNavigator 239341 13033 org.eclipse.debug.ui.DebugView 229363 22349 org.eclipse.search.ui.views.SearchView 216037 21674 org.eclipse.ui.views.ContentOutline 209932 36984 org.eclipse.ui.views.ProblemView 173191 40000 org.eclipse.team.sync.views.SynchronizeView 137594 13251 Package Explorer and the Console View... Who thought this? © 2009 EclipseSource | April 2009
  • 25. Editors – Top 10 editor executeCount200901 userCount200901 org.eclipse.jdt.ui.CompilationUnitEditor 4878831 78668 org.eclipse.cdt.ui.editor.CEditor 559440 8932 org.eclipse.jst.jsp.core.jspsource.source 438950 16762 org.eclipse.wst.xml.ui.internal.tabletree.XMLMultiPageEditorPart 393683 28678 org.eclipse.ui.DefaultTextEditor 231430 25574 org.eclipse.jdt.ui.ClassFileEditor 120639 20771 org.eclipse.ui.browser.editor 118697 9564 org.eclipse.compare.CompareEditor 115190 12226 org.eclipse.wst.html.core.htmlsource.source 96529 8103 org.eclipse.jdt.ui.PropertiesFileEditor 83958 12187 Very popular: Java, C/C++, XML, ... © 2009 EclipseSource | April 2009
  • 26. EPP Usage Data Collector UDC available for RCP applications, but... NEW: Extension for RAP-based environments but this is only half of the story... NEW: Widget based monitor ...and how do we analyse the data? NEW: VUA – Visual Usage Analytics © 2009 EclipseSource | April 2009
  • 27. Inside the EPP Usage Data Collector © 2009 EclipseSource | April 2009
  • 28. Additional Monitor: Widget based monitor Problem: In order to visualize user interactions, one needs to track the widget (button, etc.) that was responsible for the event. Solution: Widgets that need tracking must be marked manually with a distinct identifier! widgetXY.setData( "org.eclipse.epp.udc", "yourInternalIdentifier" ); © 2009 EclipseSource | April 2009
  • 29. UDC and VUA Interaction – Reference Architecture © 2009 EclipseSource | April 2009
  • 30. The Database © 2009 EclipseSource | April 2009
  • 31. The Setup for the Demonstration [1] Derby as database system export DERBY_HOME=... ./bin/startNetworkServer -h 127.0.0.1 [2] Apache ActiveMQ as queuing service ./activemq http://0.0.0.0:8161/admin/queues.jsp [3] MQC (Message Queue Client) – reads the queue and writes to the database java -jar mqc.jar © 2009 EclipseSource | April 2009
  • 32. EPP Visual Usage Data Analytics (VUA) • The VUA allows the interactive analysis of the usage data • No code changes • It only needs a connection to the DBMS • It can be used in both, RCP and RAP environments • See DEMO © 2009 EclipseSource | April 2009
  • 33. VUA – A custom widget Button Example: Button btn = new Button(showcase1, SWT.None); btn.setData( "org.eclipse.epp.udc", "btn1ShowCaseCustomTracking" ); Combo Box Example: Combo comboDropDown = new Combo(showcase1, SWT.DROP_DOWN | SWT.BORDER); comboDropDown.setData( "org.eclipse.epp.udc", "combo1ShowCaseCustomTracking" ); © 2009 EclipseSource | April 2009
  • 34. VUA – Compare User Interactions (Example) Command is available – by pressing a button Button btn = new Button(parent, SWT.None); btn.setText( "send Message" ); btn.setData( "org.eclipse.epp.udc", "btnSendMessage" ); btn.addSelectionListener( new SelectionAdapter() { ... – from the toolbar Comparison by database query defined in plugin.xml: <extension point="com.innoopract.vua.visualization.queries"> <queries id="Comparison" query_string="select a.ID, a.KIND, a.DESCRIPTION, ..."> <output format="0.00%" quotient="true" /> ... © 2009 EclipseSource | April 2009
  • 35. EPP UDC VUA Screenshot – The Application © 2009 EclipseSource | April 2009
  • 36. EPP UDC VUA Screenshot – The UDC Results © 2009 EclipseSource | April 2009
  • 37. Non-code Aspects: Privacy and User Data © 2009 EclipseSource | April 2009
  • 38. Other Examples: Debian Popularity Contest You can help out the Debian Project by sending your popularity-contest results to us anonymously. With information from enough people, we can determine which packages are most often used, which are often installed but not used, and which are nearly never used. (...) http://popcon.debian.org/ © 2009 EclipseSource | April 2009
  • 39. Non-code aspects of the Usage Data Collector • Make it as transparent as possible! • Tell your users what you are going to do! • Let the user decide! - Opt-In • Check for possible legal issues! • Let the users see the results! • Make the source code available! Keep in mind that usage data is sensitive data! © 2009 EclipseSource | April 2009
  • 40. Further Information EPP Project Webpage  http://www.eclipse.org/epp EPP Usage Data Collector and VUA  http://www.eclipse.org/epp/usagedata/index.php  http://www.eclipse.org/org/usagedata/  http://wiki.eclipse.org/EPP/Framework_for_Visual_Analytics_of_user_tracking_information Mailing List  https://dev.eclipse.org/mailman/listinfo/epp-dev © 2009 EclipseSource | April 2009
  • 42. RCP vs. RAP – Architecture http://www.eclipse.org/rap/ © 2009 EclipseSource | April 2009