Fluid configuration of d3.js bandlines with FRP

Fine-grained FRP interactivity for real-time configuration and live update with d3.js and a simplified version of flyd.js. Now best run on Chrome.

Summary: in lieu of the customary rerendering with d3.js, each interaction cascades through only the necessary blocks in the data flow. For example, changing the sparkstrip circle radius invokes merely one .attr('r', ...) invocation, while the rendering code shape resembles regular d3 code. Also, only the necessary parts of the data model are recomputed. The dependencies are explicit.

MVC is enjoying a renaissance, however it is suboptimal for views and interactions that are of some complexity. Good descriptive or predictive analytics, financial performance analytics, data visualization and even real time behavior and interactivity are best modeled as an explicit DAG, or for practical purposes, a data flow realization inspired by Functional Reactive Programming.

There is a stream of FRP libraries and having used RxJS and kefir.js, I wanted to try flyd for its compatibility with functional programming. The approach should work with any of the libraries but flyd is cool and minimal.

Note that this is just an experiment with certain goals, not an example for any of these: performance optimized code (streams proliferate in this version and is choppy on non-Chrome); idiomatic use of d3.js, FRP or ramda; production quality code; easy to debug code that is complete or free of glitches.

The goal of the experiment was to take the FRP paradigm to an unusually granular level to learn new lessons. In particular, FRP streams (approximations of mathematical variables) are being passed around, as opposed to JavaScript variables. As a result, entire swaths of the d3.js rendering tree are avoided - which has key performance benefits - such a way that it doesn't mandate the unnatural slicing up of complex views into 'containers' or 'widgets'. Modularity is a good idea, best done on natural boundaries, which may or may not coincide with performance considerations. An FRP approach allows only those parts to be updated that need to change. Admittedly there's contention between approaches of d3.js, FRP and react.js for the control of who mutates the DOM, and the current example can be changed to work with vanilla HTML5 mutations, react.js or canvas/WebGL operations. In fact, the biggest problem with the example is that it clashes with d3 transitions. A next version will take care of transitions such that increasing the number of visible points doesn't result in choppiness.

The performance benefits partly hinge on conscious structuring of the scene graph, which was the subject of an earlier experiment. For example, a zooming functionality could rely on using a element as the slippy layer so that scaling is only done on a single element (with vector-effect: non-scaling-stroke).

Important benefits come from FRP not just via incremental updates, and architecture (e.g. safer cacheability of pure functions). Due to its asynchronous nature, one can steer processing steps into a Web Worker (example of that will be posted), which takes advantage of modern hardware and more importantly, helps avoid locking up or framerate issues in the main event loop. With FRP, it's easier to loop in server communication as well.

Input processing, such as new data points, and datavis manipulation (mouse, keyboard) is done naturally with FRP. While this example just uses a dat.gui component for tweaking various levers, the pertaining principles of FRP hold true. For lack of time this example doesn't provide direct manipulation via inverse scales etc. but another small example, for mobile devices, does.

I think the main benefit of reactive programming is that the shape of the code, and the easy ability to trace dependencies in the code (input/output of data flow graph nodes) correspond to the domain (e.g. visualizing some aggregate statistics, or on a more specific example: finance, or healthcare...) rather than a triangle that focuses on the means, not the content, causing object lifecycle, asynchronous communication verbosity, fragility and testability issues.

A larger purpose of the exercise was to make a nod toward direct manipulation as advocated by Bret Victor. While there are other, really interesting efforts in progress, this experiment aimed at sticking to the almost bare-DOM d3.js abstractions rather than tying the experiment to a declarative datavis language the way the awesome Grammar of Graphics inspired tools does. Inventing new, declarative data visualization languages is very promising but on client projects I need to work with the freedom and common standard that SVG, HTML, Canvas and WebGL can give.

In the case of data visualization, there may be an enormous difference between the several seconds latency that code change involves, and the immediacy that direct manipulation yields. Ideally, sizes, constraints, layout and shapes would evolve under the hands of the maker, trying to turn the datavis medium malleable and fluid. It might lead to lower cost of bespoke data visualizations, possibilities in citizen journalism, timeboxed newsroom processes, or accidental discovery of interesting graphics.

Bandlines were chosen for the experiment due to the fact that they're simple and clean, yet of sufficient complexity to serve as demonstration ground for the data flow concept. As seen in Stephen Few's design document, they require the calculation of aggregate statistics. Also, I'm interested in innovative, data-rich approaches such as bandlines when it comes to time series, financial charting, indicators and on-chart annotations in general. Since horizon charts, bandlines and Tufte's banded sparklines are relatively new type of graphics, it's exciting to tweak them and explore the continuity of visualization space among such differing approaches, their edge cases and configuration.

License

forked from monfera's block: Fluid configuration of d3.js bandlines with FRP