Intelligent Tutoring Systems for Mathematics

What Does Learning Look Like?

The primary goal of this interactive visualization is to make the invisible components of learning visible. We look at student problem solving in mathematics in particular because learning math in school is often challenging for students to understand and is considered a gateway to high school graduation and beyond. When students solve a math problem they do a lot more work than just getting an answer right or wrong. They spend time writing down their work, they often try multiple strategies, they might be able to ask for help, and often they make mistakes and try again. Using learning technologies such as Intelligent Tutoring Systems, all of these learning behaviors can be recorded. The goal of this work is to visualize those learning behaviors to paint a more descriptive picture of student learning.

The Data Set

This data set shows a summary of student learning behavior during mathematical problem solving within an Intelligent Tutoring System called ASSISTments . This system records behavior such as correctness, time to solve a problem, number of attempts, and number of hints seen during a problem. This data set also has corresponding MCAS scores for each student (MCAS is a state-wide standardized mathematics test for students in Massachusetts).

Data Sources

To access original data from Feng, Heffernan, and Koedinger (2009) see here.

The Visualizations

Hover over a bar in the grouped bar chart to highlight all corresponding data! For example, if you hover over time (orange) in the bar chart, you will highlight time in all the corresponding bars and all the corresponding data points in the scatterplot.

The scatterplot shows raw data from student problem solving behaviors. It is useful to take a first look at this entire data set as a whole and later dive into specifics. The x-axis shows the number of problems completed by a student. The y-axis shows the corresponding value of one of four categories: the percentage of correct problems (blue), the average time to solve a problem in minutes (orange), the average number of attempts made to solve a problem (green), and the average number of hints requested during a problem (red). You can see a general trend that as more problems are completed, the lower and more consistent these values tend to be. You can especially see this with time (orange). This kind of curve directly resembles learning curves, which are often shown as a decrease of errors over time.

Do learning behaviors change based on the student? One way to find out is to separate the data by student knowledge. To explore this question, the grouped bar chart shows average values for the same categories as the scatterplot, but is grouped by knowledge level (MCAS score). Following the MCAS scoring structure, the highest knowledge level is grouped as Advanced, whereas the lowest knowledge level is labeled as Warning. You can see that on average, advanced students have a higher percentage of correct problems, spend less time solving, solve in fewer attempts, and opt for less hints than all other knowledge levels.

This visualization could be used as a tool to get a first glimpse at student learning from an overall perspective, as well as to further explore the details of how learning behaviors change based on knowledge-level.

For more explanation, you can watch this video.

Code Sources

The top visualization is forked and modified from curran's block: Stylized Scatter Plot with Color Legend

The bottom visualization is forked and modified from mbostock's block: Grouped Bar Chart

Built with blockbuilder.org

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width">
    <script src="https://d3js.org/d3.v4.min.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/d3-legend/2.24.0/d3-legend.min.js"></script>
    <title>ITS Two Views</title>
    <style>
      .domain {
        display: black;
      }
      .title {
  font: bold 30px "Helvetica Neue", Helvetica, Arial, sans-serif;
}
      .subtitle {
  font: italic 14px "Helvetica Neue", Helvetica, Arial, sans-serif;
}
      .tick line {
        stroke: #C0C0BB;
      }
      .tick text, .legendCells text {
        fill: #8E8883;
        font-size: 11pt;
        font-family: georgia;
      }
      .axis-label, .legend-label {
        fill: #635F5D;
        font-size: 12pt;
        font-family: georgia;
      }
      .highlighted {
          cursor: pointer;
          background-color: #FC0 !important;
          fill: #FC0 !important;
      }
      div.tooltip {
  position: absolute;
  text-align: center;
  width: 60px;
  height: 28px;
  padding: 2px;
  font: 12px sans-serif;
  background: lightsteelblue;
  border: 0px;
  border-radius: 8px;
  pointer-events: none;
}
      
    </style>
    <script>
  function highlightStuff(key)
  {
    for(var i=0; i<document.getElementsByClassName(key).length; i++)
    {
     
      document.getElementsByClassName(key).item(i).setAttribute("class", key+" highlighted");
    }
  }
  function unhighlightStuff(key)
  {
    for(var i=0; i<document.getElementsByClassName(key).length; i++)
    {
      document.getElementsByClassName(key).item(i).setAttribute("class", key);
    }
  }
    </script>
  </head>
  <body>
    <svg width="1000" height="500"></svg>
    <script>
      const xValue = d => d.problemsCompleted;
      const xLabel = 'Number of Problems Completed';
      const yValue = d => d.value;
      const yLabel = '';
      const colorValue = d => d.category;
      const colorLabel = 'Learning Components';
      const margin = { left: 90, right: 200, top: 190, bottom: 60 };

const svg = d3.select('svg');
      const width = svg.attr('width');
      const height = svg.attr('height');
      const innerWidth = width - margin.left - margin.right;
      const innerHeight = height - margin.top - margin.bottom;

xAxisG
        .append('text')
          .attr('class', 'axis-label')
          .attr('x', innerWidth / 2)
          .attr('y', 45)
          .text(xLabel);

colorLegendG.append('text')
          .attr('class', 'legend-label')
          .attr('x', innerWidth/2)
          .attr('y', 40)
          .text(colorLabel)
          //.text(function(d) { return d; })
      ;
                  
    var universalScale = d3.scaleOrdinal()
    .range(["correct", "time", "attempts", "hints"]);
           
      const xScale = d3.scaleLinear()
          .range([100, width]);
      const yScale = d3.scaleLinear()
      .range([height, 0]);
      const colorScale = d3.scaleOrdinal()
      .range(d3.schemeCategory10);
       // .domain(["correct", "MCAS Score", "time", "attempts", "hints"])
          //.range(["#ff4800", "#009933" , "#0000FF", "#ff00a9","#ffd000"]);

const xAxis = d3.axisBottom()
        .scale(xScale)
        .tickPadding(10)
        .tickSize(-innerHeight)
          ;

const yAxis = d3.axisLeft()
        .scale(yScale)
        .ticks(5)
        .tickPadding(15)
        .tickSize(-innerWidth)
          ;

const colorLegend = d3.legendColor()
        .scale(colorScale)
        .shape('rect')
//          .orient('horizontal')
//          .labelWrap(30)
//      .shapeWidth(40)
//      .labelAlign("start")
//      .shapePadding(15)
      
        
      colorLegendG.append('rect')
        .style ("cursor", "pointer")
          ;

const row = d => {
        d.problemsCompleted = +d.problemsCompleted;
        d.value = +d.value;
        
        return d;
      };

d3.csv('itsLearninga.csv', row, data => {
        xScale
          .domain(d3.extent(data, xValue))
          .range([0, innerWidth])
          .nice();

yScale
          .domain(d3.extent(data, yValue))
          .range([innerHeight, 0])
          .nice();

g.selectAll('circle').data(data)
          .enter().append('circle')
              .attr('class', d => universalScale(d.category))
            .attr('cx', d => xScale(xValue(d)))
            .attr('cy', d => yScale(yValue(d)))
            .attr('fill', d => colorScale(colorValue(d)))
            .attr('fill-opacity', 1)
                .attr('stroke', 'black')
            .attr('r', 4).on("mouseover", function(d) {
            highlightStuff(universalScale(d.category));
          })
          .on("mouseout", function(d) { unhighlightStuff(universalScale(d.category));                                                     
});
                
        
        xAxisG.call(xAxis);
        yAxisG.call(yAxis);
        colorLegendG.call(colorLegend)
          .selectAll('.cell text')
            .attr('dy', '0.1em');
        
     svg.append("text")
        .attr("class", "title")
       	.attr("x", 250)            
        .attr("y", 40)
        .style("font-size", "30px") 
        .text("Visualizing Student Learning");
     svg.append("text")
       	.attr("class", "subtitle")
       	.attr("x", 190)            
        .attr("y", 70)
        .text("Explore student learning by interacting with the two graphs below. The top scatterplot");
     svg.append("text")
       	.attr("class", "subtitle")
       	.attr("x", 190)            
        .attr("y", 90)
        .text(" shows raw data of learning components during student learning. The bottom grouped"); 
       svg.append("text")
       	.attr("class", "subtitle")
       	.attr("x", 190)            
        .attr("y", 110)
        .text("bar chart shows student learning components by knowledge level. Hover over a color   "); 
        svg.append("text")
       	.attr("class", "subtitle")
       	.attr("x", 190)            
        .attr("y", 130)
        .text("in the bar chart to highlight the corresponding learning component in both charts.   "); 
         svg.append("text")
        .attr("class", "subtitle")
       	.attr("x", 820)            
        .attr("y", 260)
        .style("font-size", "16px") 
        .text("Learning ");
        svg.append("text")
        .attr("class", "subtitle")
       	.attr("x", 820)            
        .attr("y", 280)
        .style("font-size", "16px") 
        .text("Components")
        ;
        
              
       
      
      });
      
//Knowledge-Level Charts
//var svg2 = d3.select("svg2"),
//    margin2 = {top: 20, right: 20, bottom: 30, left: 40},
//    width2 = +svg.attr("width") - margin.left - margin.right,
//    height2 = +svg.attr("height") - margin.top - margin.bottom,
//    g2 = svg.append("g").attr("transform", "translate(" + margin.left + "," + margin.top + ")");

var    chart2 = d3.select("body")
    .append("svg")
        .attr("width", parseInt(width.replace("px", "")) + margin.left + margin.right)
        .attr("height", parseInt(height.replace("px", "")) + margin.top + margin.bottom)
        .attr("style", "margin-top:-350px");
var g2=chart2
        .append("g")
        .attr("transform", "translate(" + margin.left + "," + margin.top + ")");
      
var x0 = d3.scaleBand()
    .rangeRound([0, 700])
    .paddingInner(0.2)
        .paddingOuter(0.1);

var x1 = d3.scaleBand()
    .padding(.07);

var y = d3.scaleLinear()
    .rangeRound([height, 200]);
 
var div = d3.select("body").append("div")
    .attr("class", "tooltip")
    .style("opacity", 0);
      
d3.csv("itsLearning2b.csv", function(d, i, columns) {
  for (var i = 1, n = columns.length; i < n; ++i) d[columns[i]] = +d[columns[i]];
  return d;
}, function(error, data) {
  if (error) throw error;

var keys = data.columns.slice(1);

x0.domain(data.map(function(d) { return d.mcasBin; }));
  x1.domain(keys).rangeRound([0, x0.bandwidth()]);
  y.domain([0, d3.max(data, function(d) { return d3.max(keys, function(key) { return d[key]; }); })]).nice();
var z = d3.scaleOrdinal()
    .range(["#0068f9", "#f97800", "#22992a", "#d1091d"]);
  
  g2.append("g")
    .selectAll("g")
    .data(data)
    .enter().append("g")
      .attr("transform", function(d) { return "translate(" + x0(d.mcasBin) + ",0)"; })
    .selectAll("rect")
    .data(function(d) { return keys.map(function(key) { return {key: key, value: d[key]}; }); })
    .enter().append("rect")
          .attr("class", function(d) {return d.key})
      .attr("x", function(d) { return x1(d.key); })
      .attr("y", function(d) { return y(d.value); })
      .attr("width", x1.bandwidth())
      .attr("height", function(d) { return height - y(d.value); })
      .attr("fill", function(d) { return z(d.key); })
          .on("mouseover", function(d) {
            highlightStuff(d.key);
          })
          .on("mouseout", function(d) { 
    				unhighlightStuff(d.key);})
    		;
       
//  svg.selectAll("rect")
//     .data(data)
//   .enter().append("rect")
     //.attr("r", 5)
//     .attr("cx", function(d) { return x(d.correct); })
//     .attr("cy", function(d) { return y(d.time); })
//  	.attr("cy", function(d) { return y(d.attempts); })
//  	.attr("cy", function(d) { return y(d.hints); })
//     .on("mouseover", function(d) {
//       div.transition()
//         .duration(200)
//         .style("opacity", .9);
//       })
//     .on("mouseout", function(d) {
//       div.transition()
//         .duration(500)
//         .style("opacity", 0);
//  });
 
  g2.append("g")
      .attr("class", "axis")
      .attr("transform", "translate(0," + height + ")")
      .call(d3.axisBottom(x0));

g2.append("g")
      .attr("class", "axis")
      .call(d3.axisLeft(y).ticks(null, "s"))
    .append("text")
      .attr("x", 2)
      .attr("y", y(y.ticks().pop()) + 0.5)
      .attr("dy", "0.32em")
      .attr("fill", "#000000")
      .attr("font-weight", "bold")
      .attr("text-anchor", "start")
      ;
  
  svg.append("text")
        .attr("class", "subtitle")
       	.attr("x", 100)            
        .attr("y", 700)
        .text("Visualizing Student Learning");

}); 
     
      </script>
  </body>
</html>

https://d3js.org/d3.v4.min.js

https://cdnjs.cloudflare.com/ajax/libs/d3-legend/2.24.0/d3-legend.min.js