JavaScript Rewrite

Tapestry and JavaScript

Tapestry 5 has had a interesting mix of characteristics.

On the one hand, it has had a large number of features that work, and work well, right out of the box, with no special configuration or setup. This includes client-side validation, dynamic content updates, simple animations, progressive enhancement, and other standard Ajax and DHTML use cases.

In addition, Tapestry has evolved, from Tapestry 5.0 through 5.3, into a quite capable provisioning framework:

However, JavaScript support in Tapestry is still unsatisfactory. Too often, Tapestry falls into an uncanny valley where the framework (server-side and client-side) does so much automatically that it becomes accepted that it does everything ... developers later discover, to their dismay, that the last 10% of custom behavior they desire is very hard to implement, because of all the common problems that plague any complex system: insufficient APIs, unexpected leaky abstractions, or just plain bugs.

Common examples of the challenges imposed by Tapestry include implementing a Confirm mixin, customizing behavior when a Zone component is dynamically updated, or any number of issues related to Forms, form elements, and Ajax updates.

This document is a roadmap for how Tapestry 5.4 will revisit the relationship between server-side Java and client-side JavaScript. Ultimately, we hope to convert this relationship from an obstacle to using Tapestry into an essential reason to select Tapestry in the first place.

Tapestry JavaScript Limitations (through 5.3)

Dependence on Prototype/Scriptaculous

Tapestry made an early choice to embrace Prototype and Scriptaculous at a time when this made sense, circa 2006-2007.

The goal was to have Tapestry provide a client-side API, the Tapestry namespace, that in turn would delegate complex behaviors (including DOM element selection, event management, and XmlHttpRequest processing) to a foundational framework. The goal was to isolate all the direct dependencies on Prototype in such a way that it would be possible, in the future, to swap out for a different foundational framework, such as jQuery or ExtJS. Unfortunately, expediency has proven to make this goal even less reachable!

Lack of Documentation

There has not, to date, been an adequate documentation of the T5 and Tapestry namespaces, beyond the code itself.

Lack of Module Structure

Beyond the basic use of namespaces, Tapestry has not embraced modern JavaScript usage; specifically, it makes limited use of hygenic functions to form modules. Hygenic functions are JavaScript functions that exist as a way to encapsulate private properties and functions. Tapestry 5.3 makes more use of this pattern than previous releases.

What modularity is present in the JavaScript is organized around the T5.initializers (Tapestry.Initializers) namespace, and the mechanics of full-page and partial-page renders (described more fully below).

Complex Initialization

Many users are perplexed by how Tapestry performs initialization: in a typical bespoke 13 application, the developer will create a <script> block at the bottom of the page, and do initializations there. In Tapestry, it can be much more complex:

This often feels like overkill, but it is necessary for a number of desirable characteristics:

Despite this, the Tapestry approach can feel very "heavy". In a bespoke page, initialization that may affect many elements of the page often takes the form of a single event handler, attached to the <body> element, that catches events that bubble up from much lower in the DOM. The single handler function identifies the applicable elements using CSS selectors, including those that are based on HTML5 data- attributes. Additional data- attributes will define additional behavior ... for example, a URL for a triggered request. This is "light" because:

By contrast, Tapestry is "heavy":

JavaScript Improvements for 5.4

The goals for Tapestry 5.4 are:

RequireJS

Rather than reinvent the wheel, Tapestry should incorporate a proper JavaScript module loader; RequireJS is an excellent candidate, especially considering the new features provided in its 2.0.1 release.

RequireJS supports the AMD (Asynchronous Module Format), with some additional support for the CommonJS module format (the format used by Node.js). The latter is simpler, but is designed for a server-side environment; AMD is specifically designed to handle asynchronous loading of JavaScript into a web browser.

RequireJS is geared towards bespoke applications; for Tapestry it is expected that some of the pathing and other configuration normally done in the client using the RequireJS API will instead by handled more dynamically on the server, using typically Tapestry configuration and extension mechanisms. For example, RequireJS allows mappings of module names to URLs, which is useful when working with multiple third-party JavaScript libraries that may be organized differently form each other. Tapestry can incorporate such logic on the server side instead, making the interface from the browser to the server uniform, even when the details of where each module is stored is quite variable.

Slow Page Load and Initialization

Tapestry 5.1 and up has support for dealing with slow page loads (especially, slow loads of extenal JavaScript). This is necessary, because in slow page load situations, the user may submit a form or click a link before page initialization has added an event handler for that submit or click; it was common in those cases for the a traditional request to be sent to the server for a link or form that was expected by the developer to only be accessed via an Ajax request. Without a server-side check (via the Request.isXHR() method), the server-side event handler would return a response that can not be handled in a traditional request, and the user would see the Tapestry exception report page.

Tapestry 5.3 and earlier would wait for the page loaded event (by observing Prototype's "dom:loaded" event) before executing any JavaScript initialization functions. Likewise, in a partial page render (Ajax) update, it would ensure that all JavaScript libraries had been loaded before executing any initialization functions.

It is not clear how this same functionality will be supported in Tapestry 5.4 as the asynchronous module loading makes it difficult to know when all modules have been loaded and all initialization functions have been invoked.

Mapping Modules to Assets

Under RequireJS, modules are identified by string that represents a kind of virtual path on the server. The path does not start with a scheme, or a slash, or end with a ".js" suffix: in all those cases, RequireJS will load a JavaScript file but not treat it as a dependency.

On the server side, Tapestry will map the path to a classpath asset.

There must be provisions for the following options:

In addition, it may be reasonable to have Tapestry automatically (or via some configuration) wrap CommonJS modules as AMD modules 17

Modules will be stored on the classpath, in a modulejs package below each library's root package. Modules within that package are referenced by their name relative to the package 18 .

Under this system, module core/pubsub would be the file pubsub.js in the package org.apache.tapestry5.corelib.modulejs, since Tapestry's component library 'core' is mapped to package org.apache.tapestry5.corelib.

Certain key modules, such as Underscore may be mapped at the root level, as they are used so often.

Extensions to JavaScriptSupport

A number of new methods will be added to JavaScriptSupport, to support the following behaviors:

The intent here is to support shifting of client-side behavior from the 5.3 style, an approach that involved monkey-patching functions onto T5.initializers, and move the same logic into modules, preferably with simpler parameters. It is also expected that there will be greater use of data- prefixed HTML5 attributes in place of separate configuration, as outlined above.

@Require annotation

In the (hopefully) common case that a module can operate without additional configuration, the @Require annotation will be analagous to the Tapestry 5.2 @Import annotation. It will allow one or more modules to be required. This is only useful when the modules are stand-alone (not needing any explicit configuration).

Increased Use Of Publish/Subscribe

An inherent limitation of Tapestry 5.3 and earlier is the too-direct connection between DOM events and client behaviors. Coding in terms of "the user clicks and the event handler submits a request" makes it very hard to fine tune behavior. For example, much work and experimentation was needed in order to introduce a Confirm mixin that could be used to introduce a confirmation dialog before allowing an event (clicking a link or submitting a form) to continue.

A better pattern is to convert DOM events into messages in a Publish/Subscribe (PubSub) system. Tapestry 5.3 introduced an initial pass at a PubSub library for this purpose, but more work is needed in this area, especially in terms of preventing unwanted references to DOM elements from impacting garbage collection under Internet Explorer.

Regardless of the exact details, in Tapestry all handling of user input should occur in two stages: a very simple event handler whose job is simply to send a PubSub message, and handlers for those PubSub message.

Avoiding JavaScript Classes

Much of the logic for important operations, such as client-side validation (and input field decoration), are based on the use of client-side JavaScript classes. This has been somewhat valuable in terms of making the behavior controllable via monkey patching. On the other hand, it cam be clumsy to accomplish in practice, as the desired behavior is only described in terms of the implementation.

In addition, these JavaScript class instances are yet more memory for the browser to manage.

By using a fine-grained set of PubSub messages, the logic usually bundled into a single JavaScript class can be assembled (and, in theory, replaced) more easily. In addition, Tapestry can do less. For instance, rather than monkey-patching the Tapestry.ZoneManager class to enable new behavior when a Zone element is updated, relying on a PubSub message to learn when the Zone was updated, and perform the desired updates or animations there.

Expose Global Message Catalog to Client

Tapestry currently maintains two global message catalogs; a global server-side catalog (usually named WEB-INF/app.properties 20 ) and a client-side catalog. The client-side catalog is smaller, more limited, and less extensible.

Allowing the client application to have full access to the entire message catalog would make maintaining the catalog simpler, and make it easier to keep client-side and server-side messages consistent.

For security purposes, it should be possible to exclude some keys from the message catalog exposed to the client. In addition, keys whose values include String.format() productions (for example, %s) should be excluded, as those productions are meaningless in the client.

Partial Page Update Response

A key part of Tapestry's dynamic behavior has been the partial page update; a specific JSON reply to Ajax requests (usually initiated via a Zone component).

The format and behavior of the response has evolved from release to release.

When an Ajax request is processed by the server, the response should handle any of a number of outcomes:

The injected JavaScript libraries and CSS links will often duplicate libraries and CSS links already present on the page; when the page is partially rendered, the server has no way to know what full or partial page renders have already occured 22 .

Tapestry 5.3 first loads any additional JavaScript (usually by adding new <script> tags to the page). Once JavaScript libraries and CSS links have been added, and JavaScript libraries have been loaded, the DOM is updated with the new content. Lastly, any initializations are processed.

For Tapestry 5.4, a number of changes are planned:

To expand on the last note first; the keys that define imported JavaScript and CSS, module requirement and invocation, and content update will not be top-level keys of the JSONObject response: they will be buried inside a tapestry top-level key. An available function will be provided that takes an arbitrary JSONObject, extracts the tapestry key and handles it, then invokes a provided callback before the module requirement and invocation step. The intent is for requests that perform purely data oriented operations, the server-side can not only provide a response, but can piggy back client-side updates in the response.

Maintaining Backwards Compatibility

Backwards compatibility is the greatest challenge here; ideally, applications (and third party libraries) that were written for Tapestry 5.3 will continue to operate unchanged in Tapestry 5.4.

At the same time, much of what Tapestry 5.3 does on the client and server should be deprecated (and hopefully, simplified).

Compatibility mode will be initially enabled, via a symbol value.

In compatibility mode, additional client-side JavaScript will be loaded to provide the same T5 and Tapestry namespaces available in Tapestry 5.3.

The implementations of these namespaces will be reconstructed in terms of the new module system. The loading of the compatibility layer will occur during full page render.

Twitter Bootstrap

In Tapestry 5.3 and earlier, Tapestry automatically includes a default CSS link on all pages. This CSS file acts as a partial CSS reset (normalizing the look of the application across common browsers), and provides a large number of CSS rules that many Tapestry components expect to be present. The CSS rules are all given a "t-" (for Tapestry) prefix.

For Tapestry 5.4, this default CSS link will be changed to be the default Twitter Bootstrap. This will not only refresh the Tapestry look and feel, but will provide a better structure for customizing the application's look and feel.

As with today, it will be possible to override the location of this CSS file (for example, to use a newer version of Bootstrap than is packaged in the application, or an application-specific customized version).

This will entail some changes to some components, to make use of reasonable or equivalent Bootstrap CSS classes, rather than the Tapestry 5.3 classes.

Twitter Bootstrap also includes a number of jQuery-based plugins; these will be exposed in the module system.

Content Delivery Network Integration

Tapestry 5.3 has limited ability to integrate into a content delivery network; it can dynamically rewrite URLs for assets (including JavaScript libraries, CSS files, image files, etc.). However, it assumes that the CDN can "pull" the content, as needed, from the live site.

A desirable feature would be request URL that would produce a JSON-formatted report of all assets that should be mirrored by the CDN: this would include all files that might be exposed to the browser, including virtual assets (such as JavaScript stacks, aggregated modules, and so forth). This could be leveraged by a tool that would use this information to extract the assets from the live application and exported to the CDN.

Determining what assets are available is somewhat problematic as Tapestry mixes server-side only resources (.class files, .tml files, etc.) freely with assets that might be exposed to the browser 24 . Some of those server-side resource may expose details, such as other server hosts and potentially user names and passwords, that should never be exposed to the client.

In addition, a "walk" of the classpath to locate potential exportable assets can be quite expensive (though not considerably more so than what Tapestry already does at startup to identify page and component classes).

ExtJS Compatibility

To be determined. ExtJS inlcudes it own system for dynamically loading ExtJS modules, as well as expressing dependencies between them. Its capabilities overlap what RequireJS offers. It would be nice if, in an ExtJS application, the ExtJS loader could be used instead of RequireJS, or at least, ensure that they do not interfere with each other.

More Thoughts

This is a big undertaking; this document is not a contract, and is certainly not complete, but is only starting point for discussions about what will be forthcoming in Tapestry 5.4.


Footnotes
Reference Notes
13 By "bespoke", we mean a non-component-based, manually created page; a standalone static HTML page.
14 Prior to 5.3, it was the Tapestry.Initializers namespace; both names reference the same object, for backwards compatibility.
15 Tapestry assists with unique id allocation, but it would be much better if unique ids were not necessary.
16 A request for any module should provide the aggregated set of modules; RequireJS will not need to send additional requests for the other modules
17 Traditionally, Tapestry has configured this kind of behavior via service contributions, but there is ample evidence that this could be done using external configuration, perhaps using a JSON file in the module package, to control aggregation, wrapping, and other aspects the process. This would be more agile, as it would not require restarts when the configuration changes.
18 A rarely used feature of Tapestry is that a component library name may be mapped to multiple packages; resolving a module name may require a search among the packages. There is the expectation that the developer will ensure that there are no duplications that would lead to ambiguities.
19 Values passed to module functions may be limited to String and JSONObject.
20 app.properties provides application-specific messages, and overrides of other messages provided by Tapestry and other third-party libraries. The global message catalog is actually a composite of all of these sources.
21 This was greatly enhanced in 5.3 to present the full exception report in a pop-up iframe.
22 It might be possible for the request to include a list of what's already loaded in the browser, so that the server can filter what it sends back; however, given factors such as content compression and typical upload vs. download bandwidth, it is almost certainly more effective for the browser to send too much, and let the client filter out duplicates.
23 Technically, in the order of invocations on JavaScriptSupport.
24 This should never have been the case, but that's hindsight.