Stimulus Control of Operant Behavior

A famous story in baseball (probably apocryphal) has a rookie talking to Mickey Mantle about hitting. The rookie asks Mantle, "I just don't understand it. I swing the bat just like you do, but don't seem to have as much success. What's my problem?" The Mick replied, "Oh, you know how just fine. Now you need to learn when."

Stimulus control is all about us learning to pay attention to, and eventually responding differentially to, things we perceive in the environment (stimuli) that give us information about the effectiveness our behavior: what behavior is likely to be effective, the conditions under which it will be effective, and what we are likely to get for it. There is a great deal in that opening sentence, and let's unpack it by looking at a simple demonstration of a rat in a Skinner Box learning a simple discrimination task. As we do this, we'll emphasize some of the common terminology of stimulus control. Please attend (this is one of those common terms used in stimulus control) to the phraseology of the next few paragraphs.

Through successive approximations, a rat is trained to press a bar to obtain food (a primary reinforcer, or S^R). During the training process, a red light above the bar is constantly lit. Since this light provides little information for the rat, it is largely ignored. Then, at some point, the experimental conditions are changed so that the light is on for a few minutes, off for a few minutes, etc. When the light is on, bar pressing is followed by the presentation of food (usually on some schedule of reinforcement), and when the light is off, bar pressing has no consequence. In a relatively short period of time, the rat comes to respond when the light is on, and not when it is off. This basic relationship is diagrammed below:

When the rat begins to respond differentially in the presence of the two stimuli, we say that it has come under the control of the stimuli. This phrase is instructive, but requires some discussion. The term control does not imply that the rat has no choice--it is merely a probabilistic statement about the likelihood of responding in the presence of the red light, more technically known as the discriminative stimulus (S^D). In other words, probability of responding is an operational definition of control--a very likely response means that the stimulus has a high degree of control over the rat's behavior. Technically, this situation in which conditions change periodically, and each condition is signalled by a specific S^D, is referred to as a multiple schedule of reinforcement.

What determines the degree of control that an S^D has over our behavior?

• the potency of the S^D ,
• the reliability of the S^D in predicting S^R,
• the immediacy of the S^R that the S^D predicts, and
• the cost of attaining the S^R that the S^D predicts.

• The more potent the reinforcer, the more the S^D will control your behavior. Let's say you are nuts about Elton John (a little aside here--Dr. Behavior just spent the weekend with some people who were at an Elton John concert in Fargo, ND, and considered it a peak experience in their lives! Elton John in North Dakota--the mind boggles!). If so, the S^R (an Elton John concert) is a BIG DEAL (potent S^R), and an S^D signalling an Elton John concert will, to a large extent, control your behavior.
• The more reliably the S^D predicts reinforcement, the more the S^D will control your behavior. OK, how did you hear about the Elton John concert? Did you overhear snippets of a conversation before class? Did you read about it in The Independent? How well do each of these predict that the S^R is real? Frankly, I'd trust The Independent (I thought I'd never say that!). The point is, we've all come, based on our past experience, to trust some souces of information more than others because those sources are more accurate, in that they better signal the availability of reinforcement, and therefore exert more control over our behavior.
• The more immediate the reinforcement, the more the S^D will control your behavior. When is the concert? If it's next week, hearing about it will probably prompt you to immediate action. If it's next semester, it probably will control little behavior right now. Short-term vs. long-term control of our behavior is one of the most important principles of conditioning. Immediate reinforcement has inordinate control over our lives, and for good reason--paying attention to short-term events has obvious survival value. It's a good idea to plan for your retirement, but if you're looking at the Wall Street Journal as you step off that curb, and don't attend to that truck bearing down on you, you'll never get to enjoy your big portfolio.
• The greater the cost, the less the S^D will control your behavior. So, these tickets-- are they free for CSB/SJU students? If so, you may well go even if you aren't an Elton John groupie. Let's say they are $20. Well, I'm a fan, so it's probably worth it. Crowd control time--$100. Yikes! Serious bucks here. That's twelve cases of Old Milwaukee--two nights entertainment with my friends. How much do I like this guy, anyway? OK, I really like the guy, but I think I'll pass. Notice we haven't changed the basic potency, immediacy or reliablitiy, but we've started to question the potency relative to the cost.

We make these decisions all the time, and they're usually pretty automatic. In the next unit, we will discuss the role that economic princples play in determining the extent to which an S^D will control our behavior.

The Interaction of Pavlovian and Operant Conditioning.

As we've been discussing throughout the semester, Pavlovian and operant conditioning are parts of an integrated, seamless response we have to the environment. We've artificially been separating them so that we could understand the components of our responsiveness to the environment, but this artificial separation is ultimately problematic, since people start looking at a response as either Pavlovian or operant, when in fact both are usually occurring. Muffy's confusion in Muffy cleans up big time. of how stimulus control of operant behavior and Pavlovian conditioning interact illustrates the point. Pavlovian and operant conditioning only make sense evolutionarily when seen as acting together. Indeed, as we've been talking about recently, even separating stimulus-response (S-R) pairings is artificial, in that the flow of our behavior/changes in the environment are pretty seamless. S-R interactions are iterative, in that what we do changes what stimuli come ou t way, and what stimuli come our way changes what we do, etc. So, with that, it's time to look more closely at how the components of our behavior come together.

Stimulus Control, Conditioned Stimuli, and Conditioned Reinforcement. In one sense, Pavlovian and operant conditioning come together in the way stimuli come to control our behavior. Take the case of the rat in the example above. The red light acts as an S^D, in that the rat comes to press the bar only in the presence of the red light. But is that all that is going on? Clearly not. If we were watching the rat and recording physiological responses, we would get measures indicating arousal when the red light comes on, indicating that the red light is also acting as a CS. In fact, the CR is no doubt occurring immediately before any operant behavior occurs. It's difficult to imagine an operant response occurring without a Pavlovian response also occurring simultaneously (or slightly before).

The final part of the puzzle comes in the form of conditioned reinforcement (S^r). This is a concept that seems to cause a fair amount of confusion, so let's go at this slowly. In the example of our friend the rat, above, the S^D and CS components seem pretty straight-forward. But the red light, because it has been associated with food in the past, can be used as a reinforcer to maintain behavior as well. Like a CS, the reinforcing properties of the red light exist because of the unique past history of this individual rat--rats which have not been trained in the presence of red lights signaling the availability of food show little interest in red lights. Bar pressing can be maintained, at least to some extent, by making the appearance of the red light contingent upon some response of the rat, such as pressing another bar. Now, where S^r gets complex (sorry!) is that there are two basic types of conditioned reinforcement--one type (token reinforcement) in which the S^r is a necessary step along the way to receiving Primary reinforcement (S^R), and one in which S^r is not a necessary condition for obtaining reinforcement but has reinforcing properties because it has been associated with powerful reinforcing events in the past. Most discussions of S^r are about token reinforcement. The term refers to the fact that with this type of reinfrocement, the S^r must be accumulated (like money, or tokens) before the Primary reinforcement (S^R) can be obtained. For example, getting points on a test, a grade in a class, etc., are examples of token reinforcement in that a student needs to pass a test (get points, or tokens), get a passing grade in a class, etc., to eventually graduate and get a diploma (another S^r, but more on this later). In this case, S^r makes perfectly good sense that it will maintain behavior, since the bigger contingency out there can't be had without picking up this one along the way. That's not to say that the S^r is unimportant; it does maintain the behavior by providing a small, but immediate, reinforcement for the behavior.

The other kind of S^r is usually much more subtle. We don't need to get it, like we do with the "necessary step" type. But it is still very important in our lives. Getting a smile when we've said an encouraging word to a friend, or receiving a smile when we really need it, may not be necessary (in that it is not tied into a larger reinforcement system), but it still feels good. And why does a smile have reinforcing properties? Because, like the red light for the rat, it has been present when other good things have happened to us.

An example showing the interaction of S^r, S^R and CS. It's sometimes difficult to make a distinction between behavior ("responses," in simple parlance) and environmental events ("stimuli"). Stimuli act as S^Ds, S^rs, and CSs, depending upon how we're looking at them, and what we are doing in a scientific analysis is seeing how these environmental events interact with our behavior. I think we can view primary reinforcement, or S^Rs, as different, in that, what is ultimately reinforcing is doing something primal--eating, drinking, breathing, etc. Ethologists refer to these acts as consumatory behavior. Note the similarity of this view to the Premack Principle. So, the end of the chain, then, is getting to engage in behavior that is reinforcing.

Let's look at an example. It's a beautiful June afternoon, you're driving down a country road, when you spot a raspberry patch. You stop, pick a bucket of raspberries, take them home, bake a pie with them, and enjoy a great dessert. We can diagram this example this way:

Let's look at the example in detail:

• Seeing the raspberry patch. Your first response is something like, "Oh, boy! Raspberries!" That's Pavlovian (CS). It also reinforces you having driven down the country road (S^r), and you are more likely to do it again in the future. And, it sets up what to do next (S^D)--You stop along the side of the road.
• In the raspberry patch. You have all of these memories of your childhood, picking raspberries, etc. (CS). You see which berries are ripe (S^D), and pick them. Being able to pick them reinforces (S^r) your stopping along the roadside.
• Having the cookbook and ingredients together. This sets the occasion (S^D) for baking a raspberry pie. Having the berries available reinforces your having stopped and picked them (S^r). You also have a strong Pavlovian response (CS) in preparation for having a pie ready to eat.
• A completed pie. This clearly reinforces baking the pie (S^r), you clearly have strong Pavlovian responses to seeing a raspberry pie, and it also tells you that you can now eat it (S^D)!
• Eating a nice berry pie a la mode. This is the consumatory response (S^R) that the whole sequence ultimately depends upon--it completes the chain.

I would like to take this opportunity to clarify some of the conventions we have been using, and will continue to use this semester:

• S^D--a stimulus that sets the occasion for certain forms of behavior. The stimulus acquires its control over your behavior through your past experience with it. For example, the jiggling keys of your boss leads you to get back to work when you hear them because in the past, when you heard the keys, your boss was soon to appear, and if you were working, things went fine, but if you were talking, you were punished, so you now start working as soon as you hear them.
• S^R--an environmental event that increases the likelihood that the response it follows will increase in frequency in the future. In the vernacular of the peasantry, a reinforcer. By convention, the capital ^"R" is reserved for what are commonly referred to as Primary reinforcers-- those things that will act as reinforcers without prior experience. As a general statement, anything that functions as an appetitive UCS will also function as an S^R.
• S^r--an environmental event that, due to its association with an S^R, will come to act as a reinforcer. Keep in mind that these environmental events will only continue to function as an S^r if they continue to maintain at least occasional connection with an S^R, and are known as Conditioned Reinforcement.
• S^-R and S^-r--these are similar to S^R and S^r, but are involved with aversive stimuli rather than appetitive stimuli.

Before going on, be sure to Make Up Your Own Question!

Email Tom

Last modified on October 13, 1998.