Introduction to Experimentation

Site Index

Purpose

This lab provides direct experience with the design and conduct of rigorous experiments in computer science. In doing this, it helps students understand what the experimental process involves, and the challenges of doing it well.

Prerequisites

Understanding of Chapter 4 of Algorithms and Data Structures: The Science of Computing.

Background

When one of the robots introduced in Chapter 2 of Algorithms and Data Structures: The Science of Computing receives a move message, it does one of two things: either the tile in front of the robot is unobstructed and the robot moves forward, or the tile is obstructed and the robot flashes to indicate a collision. Both of these responses take a noticeable amount of time. This lab involves doing an experiment motivated by the question “which takes longer?”

Experimentation

Good experiments generally require more thought and care than it seems at first. Chapter 4 of Algorithms and Data Structures: The Science of Computing discusses some of the issues that arise in computer science experiments. Sections 4.5 and 4.7, on mathematical tools for controlling and estimating error, and measuring time in Java, respectively, may be particularly useful in this exercise.

Robots

The robot is available as a Java class named Robot (and a supporting class named RobotRoom). Programs that use these classes need to include two Java files: Robot.java and RobotRoom.java. The “Final Details” section of this document explains how to find these files and their documentation.

Any Java source file that refers to the Robot or RobotRoom classes should “import” those classes, via the statement

    import geneseo.cs.sc.*;

at the beginning of the file.

Exercise

Based on informal observations of the robots, intuitions about whether hitting walls or moving freely should be faster, or other criteria, suggest a hypothesis about whether it is faster for a Robot object to handle a move message when the robot is free to move, or when the robot is obstructed. Then design, carry out, and report on an experiment that tests this hypothesis. (Note that the important thing about this exercise is the process of doing the experiment, not whether the hypothesis is right or wrong.)

All experiments follow the same general outline (hypothesis, prediction, procedure, data analysis, conclusions, etc.), but some elements of that outline merit particular note in this experiment:

• The prediction. The hypothesis for this experiment is likely to be a simple statement (e.g., “robots move faster when unobstructed”), and so the prediction will probably be correspondingly simple. But the prediction is still distinct from the hypothesis: the prediction involves some ideas about specific things to do (e.g., “if I try to move one robot that is obstructed, and another that is unobstructed…”), and specific expectations of the results (e.g., “…then the unobstructed robot will take less time to move than the obstructed one will take to flash”).
• Variables. The natural dependent variable in this experiment is the time it takes a robot to respond to a move message. The independent variable isn’t quite so obvious, but (depending on the prediction) will probably be whether the robot is obstructed or unobstructed (in effect, a Boolean independent variable).
• Confounding factors. There are a number of factors in the robot environment that might affect the time it takes a robot to move (or flash): the color of the tile the robot starts on, the color of the tile it moves to, the direction the robot is facing, etc. Such factors should either be treated as additional independent variables in the hypothesis, prediction, and experiment, or as possible environmental effects to be controlled.
• Data analysis. Data analysis for this experiment may be as simple as averaging several measurements of each variable in order to estimate its “true” value. However, if the experiment is elaborate, the data analysis may also be quite elaborate.
• The conclusion. Because this experiment tests a (probably) simple hypothesis, using a (probably) simple prediction, the basic conclusion is also likely to be simple. However, the conclusion (like many experiments’ conclusions) should not only say whether or not the data support the hypothesis, but should also mention additional information that the experiment revealed. For example, the conclusion will probably be able to mention the actual measured times for moving and colliding, may be able to discuss new hypotheses suggested by the experiment’s data, etc.

Final Details

The Robot Class

Both Robot.java and RobotRoom.java can be downloaded from the Web.

Documentation on both classes is also available on the Web. The main documentation page is an index to documentation for all the Java classes written for use with Algorithms and Data Structures: The Science of Computing. To see the documentation for a specific class, click on that class’s name in the left-hand panel of the page.

Submitting Your Work

Turn in a lab report describing your experiment. Format and submit this report as directed by your instructor.

Copyright © 2004 Charles River Media. All rights reserved.

Revised Aug. 8, 2005 by Doug Baldwin

Site Index