Lab 4: Sentence Comprehension

Due Wednesday, May 1, 2002

Designing Experimental Stimuli

In this lab you will design an experiment on adult sentence comprehension. Your task will be to select a topic and design the experimental stimuli.

Non-native speakers of English: it may be helpful to get the assistance of a native speaker for the fine-tuning of the stimuli. (If you wish to use ambiguities in your own language, please come speak with me.)

Ambiguity

Listeners are typically quite good at understanding sentences as they are spoken. We generally do not need to wait until the end of a sentence before starting to figure out what the sentence means. On the contrary, we have the impression that we build up the meaning of sentences piece-by-piece, as the words are spoken. Intuitively, this process of incremental interpretation seems quite effortless, and we are only rarely aware of difficulty in doing this. However, researchers who have attempted to construct models of how we process sentences so effortlessly have found that is far from trivial to construct such models. Experimental studies of ambiguity resolution have played a substantial role in research on language comprehension in the last 25 years.

A sentence is temporarily ambiguous if part of the sentence is compatible with two syntactic structures. For example,

While Wallace was reading the newspaper....

could be continued in at least the two following ways:

While [Wallace was reading the newspaper] his tea was getting cold.
While Wallace was reading [the newspaper lay unnoticed on the doormat].

In the first continuation, the NP "the newspaper" is the direct object of the verb "read." In the second continuation, the NP "the newspaper" is the subject of the main verb "lay." This means that when a listener hears or a reader reads an NP like "the newspaper" in a context like this, it is impossible to be certain of what structural analysis it requires. This is a classic example of a temporary ambiguity. Research on ambiguity resolution has asked questions such as, (i) how is ambiguity detected, (ii) what do comprehenders do when faced with an ambiguity - do they have a strategy to determine which option to pursue, do they pursue both options, or do they simply wait until the ambiguity goes away?, (iii) what happens if the comprehender makes an incorrect choice - is it possible to recover from mistakes?

Reading Time Measures

A common way of studying sentence comprehension is to measure the time that it takes subjects to read the words of a sentence. To a (very rough) approximation: longer reading times are associated with greater processing difficulty. For this lab we will be using one version of this approach, known as self-paced reading. Subjects read sentences on a computer screen which are displayed one word at a time. Each time the subject presses the space bar on the computer keyboard, a new word appears and the preceding word disappears. This presentation method is sometimes referred to as a 'moving window display' (see demonstration in class).

A sequence of presses might give rise to the following sequence of trials (one line is displayed at a time, in the middle of the computer screen):

----- ------- --- ------- --- --------- --- -------. [space-bar press]

While ------- --- ------- --- --------- --- -------. [space-bar press]

----- Wallace --- ------- --- --------- --- -------. [space-bar press]

----- ------- was ------- --- --------- --- -------. [space-bar press]

----- ------- --- reading --- --------- --- -------. [space-bar press]

----- ------- --- ------- the --------- --- -------. [space-bar press]

----- ------- --- ------- --- newspaper --- -------. [space-bar press]

----- ------- --- ------- --- --------- was -------. [space-bar press]

----- ------- --- ------- --- --------- --- burning. [space-bar press]

Because the space-bar presses determine the length of time that each word is displayed, the subject has full control over the rate at which the sentence is presented. The computer measures the time between each space-bar press, and this gives a measure of how long each word was displayed to the reader.

It is standard practice to present subjects with a simple yes/no question after each sentence, which asks about the content of the sentence. This task helps to ensure that subjects are paying attention to the sentence, rather than just pressing the space bar without reading the sentence.

There are of course many factors which can contribute to ease or difficulty of processing a sentence, many of which are tangential to the goal of understanding how temporary ambiguities are resolved. In order to isolate the contribution of ambiguity to reading times, it is standard to compare the word-by-word reading times for a temporarily ambiguous sentence to the reading times for exactly the same words in an unambiguous control sentence. For example, word-by-word reading times to a temporarily ambiguous sentence like:

The boat floated down the river sank without a trace. (temporarily ambiguous)

could be compared to word-by-word reading times to the unambiguous control sentence:

The boat that was floated down the river sank without a trace. (unambiguous control)

If we were comparing the sentences above, we would most likely find that the word "sank" is read much more slowly in the ambiguous condition compared to the unambiguous condition. This reflects the fact that in the ambiguous condition this word may force readers to give up a prior analysis of "The boat floated down the river" as a simple clause rather than as a complex NP.

Important: The fact that readers encounter difficulty at the word "sank" in the ambiguous condition is the crucial evidence that they chose the incorrect main verb interpretation of "floated down the river."

Therefore, one of the most important aspects of designing stimuli for a sentence comprehension experiment is the careful design of appropriate control sentences.

If you are testing how people comprehend sentences which are temporarily ambiguous between Structure A and Structure B (where the precise nature of the two structures depends on what you are testing), you will need at least the following experimental conditions:

Ambiguous, resolved to Structure A
Unambgiuously Structure A
Ambiguous, resolved to Structure B
Unambiguously Structure B

If you find that at the disambiguation point, conditions (1) and (2) show no difference in reading times, but that at the point of disambiguation condition (3) takes much longer to read than condition (4), then this suggests that when subjects are faced with an ambiguity between Structure A and Structure B, they tend to select structure A. Therefore, when Structure A is confirmed as the correct analysis, subjects do not need to change any of the structures already built, but if Structure B is confirmed as the correct analysis, subjects will get into difficulty, since they have been assuming Structure A.

Example:

The lookout spotted the sniper and alerted his commander.
The lookout saw the sniper and alerted his commander.
The lookout spotted by the sniper was chastised.
The lookout who was spotted by the sniper was chastised.

Examples of Temporary Ambiguities

For this lab, you must select a single ambiguity to test. Here is a list of possibilities which you might want to choose from. If you have another idea, then please talk with me (BB) for advice.

You will need to decide what kind of syntactic structure you would like to test; some possibilities are given below (ambiguous passages underlined):

Complement clause vs. relative clause temporary ambiguity

The president told the journalist that the first lady annoyed that he should go jump in the Potomac.
Object NP vs. main clause subject temporary ambiguity

While Wallace was reading the newspaper was burning.
Direct Object vs. Embedded Subject ambiguity:

The smart student remembered the answer was probably misleading.
Prepositional phrase attachment ambiguity: NP-modifier vs. VP-modifier or VP-argument:

The cook placed the cake in the oven on the table.
Relative clause attachment ambiguity: relative clause as modifier of lower or higher NP:

The reporter interviewed the daughter of the colonel who had had an accident.

Or you could measure reading times in (somewhat) difficult-to-process center-embedded sentences.

What You Need to Create

Conditions: Appropriate experimental conditions (e.g., Structure A ambiguous, Structure A unambiguous, Structure B ambiguous, Structure B unambiguous). Speak with me if you are uncertain about how to create your experimental conditions. You should aim for your different conditions to be maximally similar, except for the differences which are crucial for your experimental manipulation.
Number of items needed: enough items such that all subjects will see at least 2 examples of each of the four conditions. This will require a total of 32 sentences, for the following reason...

Each subject sees just one of the versions of each set of sentences, so if you create 4 sets of items in an experiment with 4 conditions (i.e., 16 sentences total), then each subject will see just one example of each experimental condition; if you create 8 sets of items (i.e., 32 sentences total) then each subject will see 2 examples of each condition, etc. In a full experiment you would want each subject to see at least 4-8 examples of each condition, but for the purposes of this exercise it will be enough for you to create at least 8 sets of experimental items, which will give you 2 data-points per condition per subject, assuming that you have 4 experimental conditions, as outlined above. That is, you need 8 sets of four, i.e., 8 sets like the lookout spotted set above.
Comprehension questions: You will also need to create comprehension questions to accompany your sentences.
Note: in constructing your experimental sentences and comprehension questions, it is important not to introduce factors that will unfairly bias your results. For example, if you create an ambiguity, but one of the two meanings is vastly more natural or plausible than the other, then this is a cause for concern. Similarly, in creating your comprehension questions, you should make sure that the correct answer is clear, and does not depend on information not present in the stimulus sentence.

Unambiguous Controls

You may find that it can be difficult to create the unambiguous controls. However, you can use tricks like different verbs (as in the lookout example above), different forms of pronouns, different prepositions, or different beginnings to clauses to disambiguate:

Object/Subject of main clause ambiguity:
1. While Wallace was visiting you ruined my china.
2. While Wallace was visiting me he ruined my china.
3. While Wallace was visiting I ruined my china.
PP attachment ambiguity:
1. The cook put the cake in the oven on the table.
2. The cook put the cake into the oven by the table.
Relative/complement clause ambiguity:
1. The president told the journalist that the first lady annoyed that he should go jump in the Potomac.
2. The president told the journalist why he should go jump in the Potomac.

So your full 4-sentence set for the object/subject ambiguity might look like the following:

Subj, ambiguous: While Wallace was visiting you ruined my china.
Subj, unambiguous: While Wallace was visiting I ruined my china.
Obj, ambiguous: While Wallace was visiting you he ruined my china.
Obj, unambiguous:While Wallace was visiting me he ruined my china.

The underlined regions are where we compare the reading times; they should be as similar as possible to each other.

If you have questions, email Benjamin Bruening (bruening@udel.edu).

This lab was created by Colin Phillips and modified slightly.

-----	-------	---	-------	---	---------	---	-------.	[space-bar press]
While	-------	---	-------	---	---------	---	-------.	[space-bar press]
-----	Wallace	---	-------	---	---------	---	-------.	[space-bar press]
-----	-------	was	-------	---	---------	---	-------.	[space-bar press]
-----	-------	---	reading	---	---------	---	-------.	[space-bar press]
-----	-------	---	-------	the	---------	---	-------.	[space-bar press]
-----	-------	---	-------	---	newspaper	---	-------.	[space-bar press]
-----	-------	---	-------	---	---------	was	-------.	[space-bar press]
-----	-------	---	-------	---	---------	---	burning.	[space-bar press]