Lab 3: Sentence Comprehension

Stage 2 of the lab is now ready to run: see the link to the instructions page below.

In this lab, you will design and run a scaled-down version of a language processing experiment. The lab comes in a couple of stages.

Stage 1: Creating Materials

Extra tips on creating materials (posted 11/30/97)

Posted Monday November 24th, due Tuesday December 2nd. It is fine to work together on creating these materials ... and may be particularly helpful if you are not a native speaker of English. Note: creating experimental items can be quite tricky ... so don't expect to be able to just do it by sitting down for half an hour. Expect to revise your first attempts a couple of times.

I have put in the course mailbox in the Linguistics Department mailroom one copy of a chapter by Edith Kaan (MIT) and Laurie Stowe (U. of Groningen) on "Constructing the Right Materials" from their draft coursebook "Developing an Experiment: Techniques & Design". Although this chapter was written for students working with Dutch, and contains more details than you will be likely to need, it may still be useful. This copy is available for photocopying ... if you take it, then please return it promptly!

The first step, and often the most difficult, is to create experimental stimulus items. If you are testing how people comprehend sentences which are temporarily ambiguous between Structure A and Structure B (where the precise nature of the two structures depends on what you are testing), you will need at least the following experimental conditions:

i. Structure A, ambiguous

ii. Structure A, unambiguous

iii. Structure B, ambiguous

iv. Structure B, unambiguous

You will need to decide what kind of syntactic structure you would like to test, some possibilities are given below (ambiguous passages in boldface):

• Complement clause vs. relative clause temporary ambiguity (e.g. The president told the journalist that the first lady intimidated that he should go jump in the Potomac)
• Double object construction vs. relative clause temporary ambiguity (e.g. The man gave the dog the bone fell on a bath)
• Object NP vs. main clause subject temporary ambiguity (e.g. While Mary was mending the sock fell off her lap)
• Any other ambiguity you care to think of (see many examples on class handout)
• Or you could test the effect of varying the kind of verb used in a given ambiguity (in the manner suggested by the MacDonald et al. paper)
• Or you could measure reading times in (somewhat) difficult to process center-embedded sentences

Number of items needed: each subject sees just one of the versions of each experimental item, so if you create 4 items in an experiment with 4 conditions (i.e. 16 sentences total), then each subject will see just one example of each experimental condition; if you create 8 items (i.e. 32 sentences total) then each subject will see 2 examples of each condition, etc. In a full experiment you would want each subject to see at least 4-8 examples of each condition, but for the purposes of this exercise it will be enough for you to create at least 8 experimental items, which will give you 2 data-points per condition per subject, assuming that you have 4 experimental conditions. [Note that 8 experimental items does not mean 8 sentences total -- the number of sentences required will be 8 times the number of conditions in your experiment.]

Things to look out for:

It can be quite difficult to make good experimental items for sentence comprehension experiments. Here are some things that you need to take care about in creating your experimental materials:

• ambiguity: if you're testing a temporary ambiguity, are the sentences really ambiguous in the regions that you want them to be? Read over your sentences: do you notice any semantic bias to give the ambiguous region one interpretation or another?
• critical word/region: in order to measure whether subjects are experiencing difficulty at some point in reading a sentence, the critical region (often the point of disambiguation) must always occur at the same point in the sentence and must consist of the same kinds of words (e.g. if it is a 3-word PP in one item, then it should be a 3-word PP in all of the items).
• be careful to avoid creating ambiguity or difficulty in parts of the sentence that are irrelevant to your interests.

You will need to submit your experimental items as an electronic text file (on disk or via email), organizing the items in the exact format required by the moving-window program. To see the format, consult the list of sample experimental items. Look carefully at how these items are organized.

Stage 2: Running the Experiment

Once the materials have been submitted, they will be loaded into the moving-window experimental package on one of the computers in the Linguistics Department lab, and we will all run as subjects in the experiment. If the materials are submitted on time, the experiment should be ready to run by Thursday December 4th.

The experiment is now ready to run. For instructions on running yourself as a subject in the experiment, go to the Lab 3 Experiment instructions page.

Stage 3: Analyzing the Results

We'll deal with this once we have the data!