SenchaCon 2016: Building Enterprise Ext JS Apps with Mavenized Sencha Cmd - Frank Wienberg

Editor's Notes

• #2: Welcome everybody to the SenchaCon technical session “Building Enterprise Ext JS Apps with Mavenized Sencha Cmd”! Before the actual talk, let me introduce you to someone who motivated us.
• #10: Wouldn't it help to at least use the same build tool for both tasks? So then, when consolidating to one build tool, which one shall it be? Maven or Sencha Cmd? Realistically, at least in our development process, Maven cannot easily be replaced. You might have experienced this situation, probably this is why you are here, in this session, right? After having introduced Joe Fullstack, let me quickly introduce myself. My name is Frank (Wienberg) and I am Senior Software Architect at CoreMedia, located in Hamburg, Germany. One of my main topics is User Interface architecture and development. CoreMedia is a product vendor of a Digital Marketing Platform (simply put, a Content Management System that integrates with eCommerce platforms) and we use Java for the back-end and Ext JS for the editorial UI. For our efforts in Web and UI development tooling, in 2009, CoreMedia founded the Open Source project Jangaroo.
• #11: Jangaroo started with an ActionScript-to-JavaScript compiler that is still in use at CoreMedia today and fully supports Ext JS 6. But that is another story… Maven has always played a central role for the integration of our build tools and thus for Jangaroo, so when we upgraded to Ext JS 6 and had to integrate Sencha Cmd in our tool stack, it was the natural choice to include our Sencha-Maven-tooling into Jangaroo. This means that what I present here is available as Open Source. Of course, you still have to license Ext JS, or use it under GPL if that is possible for you.
• #12: It also makes sense that Maven is the primary build tool, because while Sencha Cmd is a specialist for Ext JS building, Maven is a generalist that is designed to be extensible to integrate arbitrary build tools. There are a few aspects where Maven and Sencha Cmd overlap, for example dependency and artifact management. Because we already have the infrastructure for Maven and also found it to be more mature and flexible in that regard, we chose to let Maven handle artifacts and versions instead of using Sencha remote packages. The slide shows an overview of “who does what”, but of course I’ll go into more detail on that later.
• #13: Last thing to settle before I give an overview of the rest of the talk is the general approach how we integrated Sencha Cmd into Maven. Basically, there are three ways to invoke an external tool from Maven. Both the exec plugin and the antrun plugin are general-purpose and can be used directly in an ad-hoc fashion. While exec can invoke any command line tool through a system call, antrun can invoke ant tasks, even custom ones like Sencha’s ant tasks, and offers more convenience in passing parameters. What we wanted to achieve, however, is minimal boiler plate code for developers who want to use Maven for building Ext JS Apps. The best way to achieve this is to create a custom Maven plugin that automates all standard tasks and puts convention over configuration (the “Maven Mantra”).
• #14: Now these are the three main topics of this session. Each topic is comprised of two parts, where the first recaps the Maven concepts needed to follow the second part. I suppose you are already familiar with the basics of Sencha Cmd. To let you see a working example as soon as possible, the first topic is to present a simple, but complete Sencha Cmd Maven workspace. The second topic gives background on how the integration works, that is how our custom Maven plugin functions. In the third part, I’ll show you how the Ext JS client-side App can be neatly integrated into a Java Web-App. After the inevitable conclusion it’s time for your questions.
• #15: Right, so let‘s start with a short primer about Maven project- and workspace layout.
• #16: What Maven calls a “project” is quite similar to one Sencha package or app. Instead of a package.json (or app.json), the project is described in XML format (the Java world!) in a file always called pom.xml. Maven insists on a clear separation of source, the version-controlled source files, and target, files generated by the build process. The command mvn clean wipes out the target directory, enforcing this rule. Inside the s-r-c directory, there are subfolders for production code called „main“ and for test code, obviously called „test“. Below each of these, there are subfolders for different types of sources, languages or technologies like „java“, „python“, or „resources“.
• #17: Like Sencha Cmd, Maven also has the concept of a workspace. However, Maven also uses projects to define a workspace, namely so-called aggregator projects. An aggregator project collects other projects, which are then called „modules“ and are usually located in its sub-directories. When building an aggregator project, all its modules are built, too. The set of all projects that are built together is called the Maven reactor. All projects in a reactor are always built in the correct dependency order. A project is identified by its Maven-“coordinates“ group ID, artifact ID and version. Usually, group ID and version are the same within a workspace, and each artifact ID should be unique.
• #18: Let me show you a simple Maven POM. POM stands for “Project Object Model” and describes one Maven project. Obviously, it is an XML format in which several elements are predefined. The tags <groupId>, <artifactId> and <version> establish the “Maven Coordinates”. The are other optional elements like <name> and <description> for further project meta-data. <packaging> determines the so-called build lifecycle to execute and the type of resulting artifact – this is going to be important for our Sencha-Cmd-integration. Maybe the second most important section is <build> / <plugins>, where you configure all plugins that contribute to the build. Last but not least, dependencies on other Maven projects are given the <dependencies> section.
• #19: To give a concrete example now, I prepared a very simple Sencha Maven workspace. Let me show you.
• #20: The example scenario is that we have an Ext JS utility class that is to be shared by multiple applications, so we put it into a dedicated Sencha package. We also have an example Sencha app that uses this utility class and thus the package. To implement this simple, but not trivial example, we only need five rather short files, which I’ll show in detail in a minute. Of course we need the two Ext JS JavaScript files for the utility class (MyUtil.js) and the application class (Main.js). Three pom.xml files define the Maven projects that represent the Sencha package, the Sencha app, and the workspace root, respectively. Compare this to Sencha Cmd, where you generate the workspace, package and app and end up with a bazillion JS, JSON, and Ant build XML files.
• #21: Now before we look into details of how the POM XML files look like, here are the commands to generate and build the workspace. The first step only has to be performed once for each Maven workspace. The resulting files should then be checked into your VCS so that anybody checking out can build immediately, using… …the second command: simply “m-v-n install”. When performed in the root directory, this builds both the package and the app (and “installs” their artifacts into the local Maven repository, more on that later). The third command is equivalent to perform “sencha app watch” in the generated App directory. We thought it was a good idea to provide a Maven wrapper for “sencha app watch”, too, so that you can always use the mvn command, and so you do not have to know about the generated workspace structure, which I’ll show later. Then, you can open your Sencha App in the browser as usual. The only unusual thing is the long path under which the generated index.html is located.
• #22: With the commands on this slide, we (re)build the App with the Sencha development profile, resulting in a slightly shorter path for the generated index.html and, like you know from Sencha Cmd, better debugging capabilities. The fourth statement shows you how to update the generated Sencha package and app after changes in source code. This is currently necessary since source files are copied into the target and picked up by sencha app watch there.
• #23: Sencha Cmd has a bad habit of creating files right next to source files or in their parent directory. To keep the strict Maven rule of separate src and target folder, we let Sencha Cmd only see files in target folders. The only exception is the workspace.json file itself. While in a „normal“ Sencha workspace, all local packages are in subfolders of the same folder „packages/local“, in a Maven Sencha workspace, each package has its own „packages/local“ folder. This is only possible because workspace.json allows multiple local packages root folders. Application root folders are also specified in workspace.json. Because the Maven reactor knows about all Maven projects that build a package or app, both lists of folders can be (and are!) generated by our Maven plugin. The file is shown in green on the slide. Actually, there are a few more which I left out for brevity. All target folders are populated when invoking mvn package or mvn install, shown in blue. Here, you can see that a package.json / app.json is generated by our Maven plugin and the source files are simply copied from src to target. So in the target folders, the standard Sencha Cmd file layout is created. Now, how do the POM files look like?
• #24: The workspace root POM is nothing special, just a simple aggregator of the package and the app project.
• #25: Now it gets more interesting: how to we enable the specialties of a Sencha Cmd Maven build? Since you already saw a typical Maven POM, let me just point out the specialties of Sencha-related POMs. POMs that define Sencha Packages use the packaging type “jangaroo-pkg”. (If the plugin ever becomes officially supported by Sencha, this should change to “sencha-pkg” or just “pkg”). In the <plugins> section, the Jangaroo Maven Plugin must be specified, and the “extensions” flag must be set to true to tell Maven that this plugin defines a custom build lifecycle.
• #26: The minimum additional configuration is to add a dependency on the desired version of Ext JS. Jangaroo will provide Maven artifacts of the GPL versions of at least Ext JS 6.0 and 6.2. If you want to use a commercial version, you have to install it to your company’s Maven repository (probably on your own Nexus server), or ask Sencha if they would be willing to set up a Maven server with log-in for Sencha customers!
• #27: POMs that define Sencha Apps use the packaging type “jangaroo-app”. The Jangaroo Maven plugin is applied just as for packages, only that additionally, to generate the correct app.json, you have to specify the application main class in the <configuration> section. Other app configurations can be added here, but <applicationClass> is the only required one.
• #28: The fact that our app is to use our utility package is expressed through a <dependency> on the “my-package” Maven artifact. Since here, app and package live in the same workspace and thus share groupId and version, you can use Maven’s project property expressions (dollar sign with curly braces).
• #29: So while the slides might look a bit detailed, they actually showed all the configuration code you need for a simple workspace. The next section is about how the Jangaroo Maven Plugin works “under the hood”. To explain that, I have to go far afield and talk a bit more about how Maven works and how it can be extended.
• #30: In Ant, the sequence if tasks to execute is defined in an ad-hoc manner in every single Ant build.xml. While this is flexible, it also leads to a lot of boilder plate build configuration code. An example of this is how Sencha Cmd copies a bulk of Ant xml files to your hard disk every time you generate a package or an app. To extend this build process, you actually need to tweak these generated files, which may lead to problems when updating Sencha Cmd. In contrast, Maven is built from the start with the intention of being extensible or pluggable. There are predefined build lifecycle types that can be extended through plugins. But even cooler is the option to have your plugin define a custom lifecycle!
• #31: Let’s see how this works. A Maven project or module is built with the lifecycle that is specified in its POM. The lifecycle defines several build phases and which plugin goals to execute automatically in each phase. So a goal is something that a plugin performs in a certain phase. The concrete POM can explicitly add plugin goal executions to phases, but all plugin goals defined by the lifecycle are executed automatically. This matches the Maven approach “convention over configuration”: you can let Maven execute many plugin goals by just stating a lifecycle!
• #32: Here you can see how a custom lifecycle is selected: Use the plugin that defines the lifecycle with extensions: true and configure the custom packaging. The Maven plugin defines its custom lifecycle in its components.xml. Essentially, components.xml encodes what you saw on the previous slide in XML, namely which phases exists and which plugin goals are executed in those phases.
• #33: To get some actual work done in its goals, a Maven plugin can access the Maven build context. This comprises all information contained in the project’s POM, like source and target path, properties, and dependencies.
• #34: Like I said in the introduction, we incorporated Sencha Cmd integration into CoreMedia‘s existing Jangaroo Maven Plugin. Now let me show you how.
• #35: Maven not only support building your software, but also running and testing it. Like shown in the simple workspace example, with Maven and the Jangaroo Plugin, you can easily set up and build a workspace consisting of packages and apps. These apps can be run through sencha app watch, and later I‘ll tell you about another option. Running unit tests is similar to running an app, so even when building packages, the Jangaroo Maven Plugin generates a light-weight app that executes your unit tests.
• #36: Let‘s dive into the build process. The simple workspace example shows that similar to sencha generate workspace, you can let Maven plus Jangaroo generate a Sencha-Cmd-compatible workspace.json. You can easily build many packages and apps with one command, without any shell magic – something that Sencha Cmd cannot. Owing to Maven aggregator projects, you can build all packages and apps, or you can start at one package or app and build all its predecessors or successors, dependency-wise. When building a package, Jangaroo takes care of generating package.json, analogously an app.json for an app. Here, dependencies are read from the POM and translated to their Sencha names. Static resources are copied from their Maven standard location to their Sencha standard location. The actual task of „compiling“ the generated package or app is then delegated to Sencha Cmd. For apps, the profile or mode that Cmd is supposed to use, development, production, or testing, can be handed through.
• #37: Referring to the figure of a custom Maven lifecycle, here is the (simplified) Jangaroo Maven Plugin lifecycle. To focus on the main tasks, I left out test build and source-zip goals. The cool thing: a custom Maven Lifecycle can reuse (cherry-pick) standard Maven plugins / goals! This is what we actually do for resources, installing and deploying artifacts. For package builds, there are indeed only two new goals: package-pkg and package. The first one generates the standard Sencha Cmd package layout in the target directory, while the second one essentially invokes sencha package build.
• #38: Just for integrity, this slide shows the lifecycle for apps, which is very similar. There is a prepended first step that generates an almost empty app from a custom Sencha App Template. Then, all the packaging heavy lifting is done in a single goal „package-app“.
• #39: The goals „install“ and „deploy“ handle Maven Artifacts. Simply put, an artifact is the outcome of a project build.
• #40: Maven plugin goals can attach an artifact to use as the build result. Since plugin goals are automatically invoked by the build lifecycle, and the lifecycle is determined by the project's packaging, the packaging (indirectly) determines the artifact type. Here, the „package“ goal of Jangaroo attaches a „pkg“ artifact in zip format. The standard Maven install goal copies this file to the local Maven repository. The standard deploy goal uploads the file from the local repository to a remote Maven repository, e.g. on a Nexus server. Open Source artifacts can be deployed to the Maven Central Repository, where everybody can use them without specific configuration.
• #41: To be precise, Maven dependencies do not point to other projects or modules, but to other artifacts. The process to look for an artifact is three-stage: If an artifact is the result of a module in the reactor (current build), it is fetched directly from the target folder of that module. Otherwise, Maven looks up the artifact in the local Maven repository. If it also cannot be found there, Maven queries all configured remote Maven repositories, one after another, and if found, downloads the artifact and stores it in the local Maven repository. Thus, subsequent build will be faster and can even be performed in offline mode.
• #42: The cool thing about Maven is that Maven plugins are also provided as artifacts and thus use the same versioning, look-up and download mechanism as any modules! Especially for plugins, it is very convenient that when they are deployed to Maven's central repository, they can be used by a Maven client without further configuration. This means that to use Jangaroo, you just have to install Java and Maven. The Jangaroo Maven Plugin is downloaded automatically when referred to in your POM! The only exception here is that binary third-party-tools a plugin relies on, here, Sencha Cmd, have to be installed locally. There are ways to even automate platform-dependent downloads with Maven, but we did not yet investigate on that.
• #43: Now back from the plugin artifacts to the concrete Maven Sencha build artifacts. The two build types “package” and “app” lead to two different artifact types: Packages use the original Sencha “pkg” format, which is a zip with a certain structure, essentially a zipped built package. For apps, there is no format defined by Sencha, but they should be zipped up in a format that can easily be deployed. Because we want to integrate our Sencha Apps with Java Web-Apps, we chose to package them as JARs (Java Archives): More about that in a minute.
• #44: „pkg“ artifacts are really only needed when using the module from another workspace. As long as the app that uses the package is in the same workspace / reactor, the build process uses the package target directory directly, not the artifact, thus it is not needed (yet). To speed up the build process especially during development (not in the CI), Jangaroo supports a flag –DskipPkg which skips building the pkg artifact. But using modules really becomes important when they are truly reused. So for platforms or frameworks, where you develop modules used by your customers, building and deploying pkg artifacts absolutely makes sense and has never been easier.
• #45: Some more details on the “app” artifact structure. Since an Ext JS App is comprised of “static” resources, it could simply be a ZIP. But Java Web-Servers support a special format: A JAR, also a ZIP format, may contain static Web ressources under the standard path META-INF/resources. Then, it suffices to drop such a JAR into the WEB-INF/lib directory of your Java Web-App, and the resources will be served, without unpacking! And even deploying the JAR into a Java Web-Server is automated by Maven!
• #46: So her comes the grande finale: Java Web-App integration!
• #47: It needs no more than another Maven project, defined by another POM. Let’s call the module my-webapp. Here is the POM. It features a Maven standard packaing called “war”. This packaging takes care of creating the standard Java Web Archive file layout, for example generating a web.xml. The only thing you need to add to make it actually work is to add the Application Server of your choice, typically Jetty or Tomcat. Here, we chose Jetty. Without further configuration, Jetty is run and serves your Web-App on a standard port (8080).
• #48: A simple dependency on the Sencha App JAR artifact suffices to let Jetty serve the client part of the WebApp. “scope: runtime” tells Maven that this dependency is not needed for compiling and might save you from “enforcer” warnings. Here, you would add a dependency on your back-end Java module, for example a REST service implementation. Then, you can fire up Jetty and let it serve your Ext JS App simply typing “mvn jetty:run”.
• #49: To conclude: The integration of Sencha Cmd into Maven through Jangaroo Maven Plugin lets you take advantage of Maven strengths for Ext JS development. Maven provides modules to structure large code bases. It enforces a clean separation of build source and target. Maven features mature build artifact and version management and supports releases and software deployment. Even if you are not into Java or have separated UI development from back-end development, you may want to consider using Maven for these advantages. For releases, unless you produce Open Source software and release on Maven Central, you additionally need a Maven Repository server like Sonatype Nexus.
• #50: But the highest gain in productivity is for full-stack developers who integrate Java and Ext JS. You provide them with a unified or uniform Java and Ext JS build environment. They can use the same build configuration file syntax and command line syntax for both systems. The Jangaroo Plugin provides a seamless, future-proof integration of Sencha Cmd, because Cmd is used as a black box, not reverse-engineered. Finally, integrating client and (Java) server part of your Web-app is a breeze with this set-up.