SmartStep:
A Physical Computing Application to Teach Basic Math Skills

Faculty Advisor:
Tony Scarlatos
Lecturer, Computer Science
SUNY at Stony Brook

Senior Programmer:
Sherranette Tinapunan

Associate Programmers:
Clarence Verceles
Edward del Mundo

Interface Design:
Mike Chen

Photos and Screenshots >>

SmartStep is a multimedia application that helps kindergarten and early elementary students learn and practice basic math skills. Similar to hopscotch or jump rope, this application uses physical activity to reinforce basic math skills (like skip counting); while honing motor skills, pattern recognition, rhythm and coordination.

Rationale:
Multimedia learning software has demonstrated its efficacy for over a decade, and most schools offer some computer-mediated instruction even at the Kindergarten level. But while the multimedia learning applications are child-friendly, the keyboard and mouse interface is not. Children have not developed typing skills yet, and their “hunt and peck” input along with errors slows down the pace of interaction. Computer mice are not ergonomically designed for children’s small hands, and multi-button mice can confuse them and cause input errors. Furthermore, the traditional sedentary interaction with a computer is especially difficult for children, who have higher requirements for frequent exercise. At a young age children are developing motor skills and physical coordination, and associate mental skills with physical activity, such as counting on their fingers.

Recent advances in sensor technology have made it economically feasible for a school to purchase a physical interface to teach math with standard desktop computing systems. But even with the lower cost of hardware the system wouldn’t be efficient if it couldn’t be repurposed for multiple learning tasks. So the SmartStep application is built around the teacher’s interface which allows them to choose among many lessons and to configure interactions for specific learners, such as setting the level of difficulty. These sessions can be saved so a teacher can build a library of activities for individual learners or groups.

The Prototype:
The hardware component consists of nine, differently colored, 12” square floor tiles arranged in a 3 X 3 grid, connected to the computer through a MIDI interface. The student’s interface is a 3 X 3 grid of cells in a table that mirrors the arrangement of the floor tiles. Each cell presents a sequence of numbers required to play the math game. Visual and auditory feedback accompany each hop a student makes on the floor tiles. The teacher’s interface allows them to create math games and set their parameters through easy-to-use forms and pull-down menus.

The hardware consists of nine taptiles from Infusion Systems. They are light and portable and are easily set up. Each tile is connected to a digitizer which converts each sensor’s analog voltages to MIDI channel messages when a tile is stepped on. The MIDI messages are input to the computer using a MIDI to USB interface. The sensor input is saved to a configuration file so recalibration isn’t necessary.

The SmartStep application is written in Macromedia Director MX’s scripting language, Lingo. An extension to Director, XMidi, enables Director MX to receive and process MIDI channel messages in real time.

The prototype runs on either Mac OSX or Windows XP platforms.

Future work includes development of a spelling game; Simon Says (a pattern recognition and memory exercise); and field-testing at local schools.

This project was presented at the CIT conference at Stony Brook (June 2004), and CTE at Stony Brook (January 2005). It will be on display at the NY Hall of Science over the summer.

This work is supported by an NSF CRCD grant (#0203333).