Some notes about an implementation of the datavis pipeline

<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title>Datavis Pipeline</title>
</head>
<a>
<h1>An implementation of the visualization pipeline</h1>

<h2>D3.js Charts Library</h2>
<ul>
<li><a href="https://dimplejs.org/">dimple</a></li>
<li><a href="https://code.shutterstock.com/rickshaw/">Rickshaw</a></li>
<li><a href="https://dc-js.github.io/dc.js/">dc.js</a></li>
<li><a href="https://www.taucharts.com/">Tau charts</a></li>
<li><a href="Chartshttps://addepar.github.io/#/ember-charts/overview">Ember</a></li>
<li><a href="https://metricsgraphicsjs.org/">MetricsGraphics.js</a></li>
<li><a href="https://fastly.github.io/epoch/">Epoch</a></li>
<li><a href="https://plottablejs.org/">Plottable.js</a></li>
<li><a href="https://racker.github.io/glimpse.js-website/">glimpse.js</a></li>
<li><a href="https://github.com/trifacta/vega/">Vega</a></li>
<li><a href="https://tenxer.github.io/xcharts/">xCharts</a></li>
<li><a href="https://imaginea.github.io/uvCharts/">uvCharts</a></li>
<li><a href="https://square.github.io/cubism/">Cubism.js</a></li>
<li><a href="https://c3js.org/">c3</a></li>
<li><a href="https://techanjs.org/">TechanJS</a></li>
<li><a href="https://visible.io/">D4</a></li>
<li><a href="https://github.com/novus/nvd3">NVD3</a></li>
<li><a href="https://github.com/Quartz/Chartbuilder/">Gneisschart</a></li>
<li><a href="https://github.com/wq/wq.app/blob/master/js/wq/chart.js">wq/chart.js</a></li>
<li><a href="https://chinmaymk.github.io/angular-charts">Angular-charts</a></li>
<li><a href="https://esbullington.github.io/react-d3-website/">react-d3</a></li>
<li><a href="https://github.com/PatMartin/DexCharts">DexCharts</a></li>
<li><a href="https://github.com/akngs/dviz">DViz</a></li>
<li><a href="https://github.com/sirrice/gg/">gg</a></li>
<li><a href="https://github.com/rameshvs/jsplotlib">jsplotlib</a></li>
<li><a href="https://github.com/bergcloud/lp-chart">LPChart</a></li>
<li><a href="https://github.com/openbrainsrc/Radian">Radian</a></li>
<li><a href="https://github.com/michael/dance">Dance.js</a></li>
<li><a href="https://github.com/mlarocca/Dynamic-Charts">Dynamic-Graphs</a></li>
<li><a href="https://pykcharts.com/">PykCharts.js</a></li>
<li><a href="https://github.com/boundary/firespray">Firespray</a></li>
<li><a href="https://micropolar.org/">Micropolar</a></li>
</ul>

<h2>Design Patterns</h2>
Unstructured examples (D3 alternative gallery)<br>
Boilerplate (NVD3)<br>
Inheritance (Rickshaw)<br>
Composition (d3.chart)<br>
Mixin (D4)<br>
Components (Plottable)<br>

<h2>Grammar of Graphics</h2>
Pipeline<br>
dataset -> variables (e.g., speed) -> transformation (e.g., rank) -> scale (e.g., log) ->
coordinate system (e.g., cartesian) -> element and visual attributes (e.g., rectangle, color)<br>

<h2>D3 implementation</h2>
D3 binds data to graphics by adding a __data__ attribute to DOM objects<br>

dataset: json, array, ajax<br>
variables: lazy accessors<br>
transformation: d3.nest, d3.layout<br>
scale: d3.scale<br>
coordinate system: d3.layout + d3.scale used on accessors<br>
element: DOM elements and attributes (SVG/HTML/CSS)<br>

<h2>What could be different</h2>
Though D3 is a very clever implementation of the vis pipeline, some alternatives could be discussed<br>

dataset: data manipulation (cleaning, format, validation)<br>
variables: doesn't have to be lazy (e.g. d3.layout; interesting discussion <a href="https://github.com/mbostock/d3/issues/1145">here</a>)<br>
transformation: more consistent data formats (see <a href="https://wiki.cfcl.com/Projects/D3/DF">here</a>)<br>
scale: separate data scaling from view scaling<br>
coordinate system: abstracting SVG and canvas coordinates system; abstraction for polar<br>
element: not tied to the DOM<br>

<h2>Assembly implementation</h2>
D3 assembles the graphical element on the DOM tree. One way of getting away from the DOM is to use a pipeline pattern.<br>
From Firespray:<br>

<pre>
    	var pipeline = fy.utils.pipeline(
		fy.setupContainers,
		fy.setupScales,
		fy.setupBrush,
		fy.setupAxisY,
		fy.setupAxisX,
		fy.setupHovering,
		fy.setupStripes,
		fy.setupGeometries
	);
</pre>

<h2>Opportunity with Planet OS charts</h2>
<ul>
    <li>Multi-target rendering (SVG, Canvas, WebGL, HTML)</li>
    <li>Better separation</li>
    <li>Improve the layout pattern (less lazy, more abstract pattern)</li>
    <li>Immutability</li>
    <li>Components (file separation, generators)</li>
    <li>Data layer</li>
    <li>Interaction</li>
    <li>Utilities ecosystem (image exports, tooltip bisects, themes)</li>
</ul>

<h2>Requirements</h2>
<ul>
    <li>Interaction (cross-linking, zoom/pan)</li>
    <li>Large datasets</li>
    <li>Extensible chart types</li>
    <li>Multi-dimensional (3D, multiple Y axes, small multiples, sparklines)</li>
    <li>Streaming</li>
    <li>Multiple data types (time series, ordinal, numerical, categorical)</li>
</ul>

<h2>Links</h2>
<ul>
    <li><a href="https://plnkr.co/edit/QjsCva?p=preview">Planet OS Charts Live Example</a></li>
    <li><a href="https://plnkr.co/edit/uqBDIJsqGlM2E8zNmy5J?p=preview">Firespray Live Example</a></li>
    <li><a href="https://plnkr.co/edit/DTheQmbL9p8eL7IgjNNG?p=preview">Micropolar Live Examples</a></li>
    <li><a href="https://plnkr.co/edit/vh2OkO0EQ7UIyV2fTF6y?p=preview">Streaming Line Charts Demo</a></li>
</ul>

</body>
</html>