NetBeans Architecture Answers for Datasystems API module

General Information

Question (arch-what): What is this project good for?
Answer: In summary, the LoadersAPI is responsible for scanning files in a directory on disk, weeding out irrelevant files of no interest to the IDE, and grouping the rest into logical chunks, or just determining what type of data each represents. It does this scanning by asking each registered data loader whether or not the given file(s) should be handled. The first loader to recognize a file takes ownership of it, and creates a matching data object to represent it to the rest of the IDE.
Question (arch-overall): Describe the overall architecture.
Answer:

This module provides API (read more) that works on top of file objects and gives each file a logical behaviour - icon, name, operations, etc.

Question (arch-usecases): Describe the main use cases of the new API. Who will use it under what circumstances? What kind of code would typically need to be written to use the module?
Answer: A lot of usecases is described in the javadoc. Here is the list of some faqs:

Using Scripting and Templating Languages

Often many people require ability to create a "clever" template - e.g. write piece of simple text and at the time of its processing do some advanced changes to it using either scripting or templating languages.

This traditionally used to be a bit complicated task, however since version 6.1 there are new interfaces org.openide.loaders.CreateFromTemplateHandler - can be registered as a services in a lookup and it is reponsible for handling the whole copy of the template file(s) to the destination folder. and org.openide.loaders.CreateFromTemplateAttributesProvider - can be registered as a services in a lookup and it is reponsible for providing "hints" - e.g. map mapping strings to various objects. and these interfaces allow anyone to extend the behaviour during creation of new files without writing new DataLoader and co.

The support was moved to a new module; please see api.templates module for more information.

How to add action to folder's popup menu?

Loaders-folder-any-Actions - The actions that the default folder loader shows in its popup menu are read from a layer folder Loaders/folder/any/Actions so if any module wishes to extend, hide or reorder some of them it can just register its actions there. As code like this does:

    <folder name="Loaders" >
        <folder name="folder" >
            <folder name="any" >
                <folder name="Actions" >
                    <file name="org-mymodule-MyAction.instance" >
                        <attr name="instanceCreate" stringvalue="org.mymodule.MyAction" />
                    </file>
                </folder>
            </folder>
        </folder>
    </folder>
    

As described in general actions registration tutorial.
This functionality is available since version 5.0 of the loaders module. Please use OpenIDE-Module-Module-Dependencies: org.openide.loaders > 5.0 in your module dependencies.

In version 5.8 all the standard loaders were changed to read actions from layer:

• Loaders-text-xml-Actions - The actions that the standard XML loader shows in its popup menu are read from a layer folder Loaders/text/xml/Actions
• Loaders-content-unknown-Actions - The actions that the loader for unrecognized files shows in its popup menu are read from a layer folder Loaders/content/unknown/Actions
• Loaders-application-x-nbsettings-Actions - The actions that the loader for instance and settings files shows in its popup menu are read from a layer folder Loaders/application/x-nbsettings/Actions

How to allow others to enhance actions of your loader?

If you want other modules to enhance or modify actions that are visible on DataObjects produced by your DataLoader and you are either using DataNode or its subclass, you can just override protected String actionsContext() method to return non-null location of context in layers from where to read the actions.
The usual value should match Loaders/mime/type/Actions scheme, for example java is using Loaders/text/x-java/Actions, but the name can be arbitrary.
This functionality is available since version 5.0 of the loaders module. Please use OpenIDE-Module-Module-Dependencies: org.openide.loaders > 5.0 in your module dependencies.
Question (arch-time): What are the time estimates of the work?
Answer:

XXX no answer for arch-time

Question (arch-quality): How will the quality of your code be tested and how are future regressions going to be prevented?
Answer:

XXX no answer for arch-quality

Question (arch-where): Where one can find sources for your module?
Answer:

The sources for the module are in the Apache Git repositories or in the GitHub repositories

Project and platform dependencies

Question (dep-nb): What other NetBeans projects and modules does this one depend on?
Answer: It depends on FilesystemsAPI, LookupAPI, UtilitiesAPI.

The default answer to this question is:


These modules are required in project.xml:

• CommonAnnotationsAPI - The module is needed for compilation. The module is used during runtime. Specification version 1.10 is required.
• Progress - The module is needed for compilation. The module is used during runtime. Specification version 1.59 is required.
• ProgressSwing - The module is needed for compilation. The module is used during runtime. Specification version 1.40 is required.
• org.netbeans.api.scripting - The module is needed for compilation. The module is used during runtime. Specification version 1.0 is required.
• DataSystemsAPI - The module is needed for compilation. The module is used during runtime. Specification version 1.0 is required.
• MimeLookupAPI - The module is needed for compilation. The module is used during runtime. Specification version 1.11 is required.
• GeneralQueriesAPI - The module is needed for compilation. The module is used during runtime. Specification version 1.9 is required.
• ActionsAPI - The module is needed for compilation. The module is used during runtime. Specification version 6.3 is required.
• org.openide.awt - The module is needed for compilation. The module is used during runtime. Specification version 7.51 is required.
• DialogsAPI - The module is needed for compilation. The module is used during runtime. Specification version 7.22 is required.
• ExplorerAPI - The module is needed for compilation. The module is used during runtime. Specification version 6.8 is required.
• FilesystemsAPI - The module is needed for compilation. The module is used during runtime. Specification version 9.0 is required.
• org.openide.filesystem.spi - The module is needed for compilation. The module is used during runtime. Specification version 9.0 is required.
• ModulesAPI - The module is needed for compilation. The module is used during runtime. Specification version 7.19 is required.
• NodesAPI - The module is needed for compilation. The module is used during runtime. Specification version 7.0 is required.
• EditorAPI - The module is needed for compilation. The module is used during runtime. Specification version 6.26 is required.
• WeakListener.setAccessible - The module is needed for compilation. The module is used during runtime. Specification version 9.3 is required.
• LookupAPI - The module is needed for compilation. The module is used during runtime. Specification version 8.0 is required.
• UtilitiesAPI - The module is needed for compilation. The module is used during runtime. Specification version 9.3 is required.
• WindowSystemAPI - The module is needed for compilation. The module is used during runtime. Specification version 6.7 is required.
Question (dep-non-nb): What other projects outside NetBeans does this one depend on?
Answer: It does not depend on any external library.
Question (dep-platform): On which platforms does your module run? Does it run in the same way on each?
Answer: It runs on any platform.
Question (dep-jre): Which version of JRE do you need (1.2, 1.3, 1.4, etc.)?
Answer: JRE 1.3.
Question (dep-jrejdk): Do you require the JDK or is the JRE enough?
Answer: JRE is enough.

Deployment

Question (deploy-jar): Do you deploy just module JAR file(s) or other files as well?
Answer: Data Systems are integral part of openide.jar.
Question (deploy-nbm): Can you deploy an NBM via the Update Center?
Answer: Yes. openide.jar can be deployed via AutoUpdate.
Question (deploy-shared): Do you need to be installed in the shared location only, or in the user directory only, or can your module be installed anywhere?
Answer: openide.jar needs to be in the system directory.
Question (deploy-packages): Are packages of your module made inaccessible by not declaring them public?
Answer: No.
Question (deploy-dependencies): What do other modules need to do to declare a dependency on this one, in addition to or instead of the normal module dependency declaration (e.g. tokens to require)?
Answer:

XXX no answer for deploy-dependencies

Compatibility with environment

Question (compat-i18n): Is your module correctly internationalized?
Answer: Yes.
Question (compat-standards): Does the module implement or define any standards? Is the implementation exact or does it deviate somehow?
Answer: No.
Question (compat-version): Can your module coexist with earlier and future versions of itself? Can you correctly read all old settings? Will future versions be able to read your current settings? Can you read or politely ignore settings stored by a future version?
Answer: No answer
Question (compat-deprecation): How the introduction of your project influences functionality provided by previous version of the product?
Answer:

XXX no answer for compat-deprecation

Access to resources

Question (resources-file): Does your module use java.io.File directly?
Answer: No.
Question (resources-layer): Does your module provide own layer? Does it create any files or folders in it? What it is trying to communicate by that and with which components?
Answer:

Loaders-mime-type-Factories - Loaders are registered in the layer in folder Loaders/mime/type/Factories.

Yes, module creates the folders in layer org/netbeans/core/ui/resources/layer.xml in core-ui.jar. The folders are instrumental to store information about used templates. Provided folders:

• PrivilegedTemplates - A folder Privileged offers to other module possibility add own templates.

• RecentTemplates - A folder Recent stores a set of recently used templates, it's not open to other module.

• Menu -

  The main menu of the application is composed by reading Menu/ folder in the layer. A sub folder is treated as a sub menu. Instances of individual files (usually .instance or .shadow) may then represent Action or JMenuItem or JSeparator.

  Since version 7.44 one can attach property-prefix attribute to every folder. Then all the file attributes are scanned and if some of them start with the specified prefix they are placed a client properties on the JMenu instance (after stripping the prefix off).

None of them is forced to found.

Question (resources-read): Does your module read any resources from layers? For what purpose?
Answer: NewTemplateAction reads a list of templates from folders Privileged and Recent. This list is exposed in a popup menu.
Question (resources-mask): Does your module mask/hide/override any resources provided by other modules in their layers?
Answer: No.
Question (resources-preferences): Does your module uses preferences via Preferences API? Does your module use NbPreferences or or regular JDK Preferences ? Does it read, write or both ? Does it share preferences with other modules ? If so, then why ?
Answer:

org.openide.actions.FileSystemRefreshAction.manual - If NbPreferences.root().node("org/openide/actions/FileSystemRefreshAction").getBoolean("manual", false) returns true, then FileSystemRefreshAction is displayed in popup menu of folders.

Lookup of components

Question (lookup-lookup): Does your module use org.openide.util.Lookup or any similar technology to find any components to communicate with? Which ones?
Answer:
• DataLoaderPool.class
• ActionManager.class
• (in XMLDataObject) Node.Cookie.class
• RepositoryNodeFactory.class
• ModuleInfo.class
• ClassLoader.class

If there is no DataLoaderPool available in default lookup, the fallback is to use a simple loader pool that contains just the fixed system loaders, and any DataLoaders that can be found in default lookup.

DataFolder.FolderNode.setName() looks for one instance of org.openide.loaders.FolderRenameHandler.
Question (lookup-register): Do you register anything into lookup for other code to find?
Answer: Registration of
• DataLoaderPool.class
• Environment.Provider.class
• RepositoryNodeFactory.class
is done in core in META-INF/services.
Question (lookup-remove): Do you remove entries of other modules from lookup?
Answer: No.

Execution Environment

Question (exec-property): Is execution of your code influenced by any environment or Java system (System.getProperty) property? On a similar note, is there something interesting that you pass to java.util.logging.Logger? Or do you observe what others log?
Answer: org.xml.sax.driver - This is a standard way to find a class of a SAX2 driver. See SAX2 documentation netbeans.profile.memory - Boolean.TRUE means to dettach from shared impl of parser, it is static!? org.openide.loaders.FolderList.refresh.interval - The value of type integer determines the number of milliseconds between successive refreshes of contents of a folder. Can be used to tweak performance of folder refresh. Defaults to 10. netbeans.dataobject.insecure.operation - If set to true, the DataObject.copy, move, createFromTemplate are executed in insecure way. That means that other threads can access the products of such operation before it finishes. This is a friend contract with projects, that need to do such strange things. Will be removed when they fix it. Nb.Explorer.Folder.icon - Icon or Image for closed folder. Nb.Explorer.Folder.openedIcon - Icon or Image for opened folder. Tree.openedIcon - Fallback Icon or Image for opened folder. Tree.closedIcon - Fallback Icon or Image for folder. wizard.anything - When TemplateWizard invokes DataObject.createFromTemplate, it passes as argument all its properties to it with prefix wizard.. That way they are available to underlaying scripting and templating engines. org.netbeans.modules.openide.loaders.ASK_OnSaving - Control on save yes/no dialog in DataEditorSupport by setting the ASK_OnSaving key in org/netbeans/modules/openide/loaders/Bundle.properties to yes or no in a branding file in your application. org.netbeans.modules.openide.loaders.ASK_OnClosing - Control on close yes/no dialog in DataEditorSupport by setting the ASK_OnClosing key in org/netbeans/modules/openide/loaders/Bundle.properties to yes or no in a branding file in your application.
Question (exec-component): Is execution of your code influenced by any (string) property of any of your components?
Answer: The data system stores some of its vital information in file attributes. Those attributes are accessible via calls to org.openide.filesystems.FileObject.getAttribute(name) org.openide.filesystems.FileObject.setAttribute(name, value) They can also be stored in the module layer. Please see filesystems for more information about this. NetBeansAttrAssignedLoader - Extended attribute for holding the class of the loader that should be used to recognize a file object before the normal processing takes place. NetBeansAttrAssignedLoaderModule - Extended attribute which may be used in addition to EA_ASSIGNED_LOADER which indicates the code name base of the module that installed that preferred loader. If the indicated module is not installed, ignore the loader request. See #13816. template - If set to Boolean.TRUE the file is recognized as template and its instantiation is allowed. isRemoteAndSlow - If the file attribute isRemoteAndSlow is true on a folder, the New File wizard will avoid asking for its children. templateWizardURL - Attribute that defines a template wizard description page (type URL to HTML). templateWizardIterator - Attribute that defines a custom template wizard iterator (type TemplateWizard.Iterator). originalFile - Path to the target file in its filesystem (type String). originalFileSystem - System name of filesystem of target file (type String; default is same as that of shadow). UseOwnName - if true, the DataShadow name is used instead of original's name, affects DataShadows of filesystem roots only simple - templates and folders under Templates/ folder can be annotated with <attr name="simple" boolvalue="false"< if they are supposed to be hidden in Template Manager. If a folder is annotated with this attribute, it is also hidden in standard New File wizard. PartialOrders - Read the list of intended partial orders from disk. Each element is a string of the form a/b for a, b filenames with extension, where a should come before b. The value of the attribute must be of type Boolean; ignored unless true. OpenIDE-Folder-SortMode - Extended attribute for order of children. The values are "F", "N", "C", "0" (type String). OpenIDE-Folder-Order - Extended attribute for order of children - stores list of file names separated by '/' (type String). org.openide.loaders.FolderChildren.delayedCreation -

Since 7.25 the DataFolder.getNodeDelegate() tries to prevent creation of DataObject in AWT dispatch thread. Rather it creates dummy node with name derived from the name of the file and simplified content of lookup:

• FileObject - the file that the node represents
• Node - the node itself, but without any important properties
• DataObject - created on the fly, very inefficient, if requested from AWT dispatch thread, it prints a warning. Consider using just FileObject.

The creation of real node is scheduled to background and as soon as the DataObject and its Node are ready, the initial dummy node is replaced by the real one.

This whole system is slightly incompatible and may complicate creation of filtered views over the node hierarchy (one needs to be ready to really dynamics changes). That is why it is possible to disable the new delayed system by starting the system with -Dorg.openide.loaders.FolderChildren.delayedCreation=false. Use this property as a temporary fix for your problems, but consider fixing your code to support the delayed mode in the future.

EA-OpenIDE-Connection - Extended attribute to store (ArrayList of Type and Node.Handle). Used by Java synchronization feature at least; generally, ConnectionCookie. DataFolder.Index.reorderable - If set to Boolean.TRUE on a folder not in the system filesystem, make its node reorderable. expectedTime - When the DataObject is moved to new location, we we need to adjust the time to the new file object. CloneableEditorSupport exports special "expectedTime" property for this purpose. Tested in DataEditorSupportTest.testChangeFileWhileOpen.
Question (exec-ant-tasks): Do you define or register any ant tasks that other can use?
Answer:

XXX no answer for exec-ant-tasks

Question (exec-classloader): Does your code create its own class loader(s)?
Answer: No.
Question (exec-reflection): Does your code use Java Reflection to execute other code?
Answer:

XXX no answer for exec-reflection

Question (exec-privateaccess): Are you aware of any other parts of the system calling some of your methods by reflection?
Answer:

XXX no answer for exec-privateaccess

Question (exec-process): Do you execute an external process from your module? How do you ensure that the result is the same on different platforms? Do you parse output? Do you depend on result code?
Answer:

XXX no answer for exec-process

Question (exec-introspection): Does your module use any kind of runtime type information (instanceof, work with java.lang.Class, etc.)?
Answer:

No.

Question (exec-threading): What threading models, if any, does your module adhere to? How the project behaves with respect to threading?
Answer:

XXX no answer for exec-threading

Question (security-policy): Does your functionality require modifications to the standard policy file?
Answer:

XXX no answer for security-policy

Question (security-grant): Does your code grant additional rights to some other code?
Answer:

XXX no answer for security-grant

Format of files and protocols

Question (format-types): Which protocols and file formats (if any) does your module read or write on disk, or transmit or receive over the network? Do you generate an ant build script? Can it be edited and modified?
Answer: Any file format.
Question (format-dnd): Which protocols (if any) does your code understand during Drag & Drop?
Answer: We use following MIME type: application/x-java-openide-dataobjectdnd;class=org.openide.loaders.DataObject;mask={0}
Question (format-clipboard): Which data flavors (if any) does your code read from or insert to the clipboard (by access to clipboard on means calling methods on java.awt.datatransfer.Transferable?
Answer: We use following MIME type: application/x-java-openide-dataobjectdnd;class=org.openide.loaders.DataObject;mask={0}

Performance and Scalability

Question (perf-startup): Does your module run any code on startup?
Answer: Yes. The list of loaders is read from the disk to memory.
Question (perf-exit): Does your module run any code on exit?
Answer: No.
Question (perf-scale): Which external criteria influence the performance of your program (size of file in editor, number of files in menu, in source directory, etc.) and how well your code scales?
Answer: Performance of some operations is lineary proportional to number of files in a given folder times number of registered data loaders in the pool.
Question (perf-limit): Are there any hard-coded or practical limits in the number or size of elements your code can handle?
Answer: No.
Question (perf-mem): How much memory does your component consume? Estimate with a relation to the number of windows, etc.
Answer: Unknown.
Question (perf-wakeup): Does any piece of your code wake up periodically and do something even when the system is otherwise idle (no user interaction)?
Answer: No.
Question (perf-progress): Does your module execute any long-running tasks?
Answer: The module uses its own RequestProcessors to handle long running tasks.
Question (perf-huge_dialogs): Does your module contain any dialogs or wizards with a large number of GUI controls such as combo boxes, lists, trees, or text areas?
Answer: This module contains infrastructure for creating "New" functionality wizards.
Question (perf-menus): Does your module use dynamically updated context menus, or context-sensitive actions with complicated and slow enablement logic?
Answer: No.
Question (perf-spi): How the performance of the plugged in code will be enforced?
Answer:

XXX no answer for perf-spi