In this page, I attempt to impose some order on my diverse blogs about research in industrial and organizational psychology. I use the AQAL version of Integral Theory, as articulated by Ken Wilber and our colleagues at the Integral Institute, as an organizing mechanism. Naturally, many of my blogs will not fit neatly into a particular dimension of Integral Theory. Reasonable minds no doubt will differ with me about my taxonomy. I welcome your comments about how things ought to be rearranged.
Behind the hyperlinks listed below, you'll find blogs addressing organizational research that has implications for the "All Quadrants" dimension of Integral Theory. Blogs that clearly fit into any of the quadrants will be found behind the hyperlink bearing that quadrant's name, while blogs that consciously cross quadrant boundaries will be found behind the hyperlink named "Integral".
Sports fans have heard commentators talk about certain players "understanding the team concept." But what does that mean in the organizational world? Is there such an understanding, and can you measure it in job applicants (and thereby select employees based on it) and does it predict job performance once hired into a team?
Mumford and his colleagues have come up with a measure of this very thing, and have provided evidence that it is a valid measure. They call the aspect of this understanding of teams that they focused on "team role knowledge." They define it as "the knowedge an individual possesses about the nature of team roles and the situational contingencies govening their use." They go on to explain that the term "encompasses the declarative and procedural knowledge of role types and contingencies that is needed to effectively perform team roles."
What are these roles? The researchers identified 10 main functional roles that seem to cover the bulk of the role waterfront identified in the scientific literature on the subject:
Through a series of studies, they developed a test of knowledge of these roles and how they function within teams, the Team Role Test (TRT), a 90-item measure. They validated the test in pilot studies, then took it to the field to see how it worked in a real-life situation. As it turns out, the TRT does several things that psychologists like:
So, it's an encouraging start to a new field of study -- the extent to which individuals' knowledge of how team roles function predict their performance in teams. If these results hold up in future research, employers will have a new and powerful tool to help them figure out who will be the most productive team members.
From: Mumford, T. V., Van Iddekinge, C. H., Morgeson, F. P., & Campion, M. A. (2008). The team role test: Development and validation of a team role knowledge situational judgment test. Journal of Applied Psychology, 93(2), 250-267.
Personality tests are increasingly popular tools in hiring practices. Despite widespread suspicion that performance on such tests can be faked, a growing number of employers are using them, because some personality traits, like conscientiousness, seem to predict performance pretty well.
Shawn Komar and his colleagues would put a warning label on these tests: Faking on personality tests is likely to diminish the predictive validity of those tests where the trait of effective faking matches up with ineffective work behaviors. Note, however, that where the ability to fake might help one with job performance (e.g., some sales jobs), faking on such tests won’t affect the test validity negatively. In fact, it might even help!
So, when you’re administering tests that can be faked, be sure you want to hire people who can fake effectively.
(From "Faking and the Validity of Concientiousness: A Monte Carlo Investigation", by Komar, Brown, Komar, and Robie, 2008 JAP 93(1), 140-154.)
For years, it has been known that there is some sort of relationship between how much a job matters (task significance) and job performance. The problem has been that because, as the saying goes, “correlation does not equal causation”, it is not necessarily the case that if a person sees that the task matters, they’ll be more likely to be perform well. The causation piece has been missing.
Adam Grant’s study attempted to provide that missing piece. He thinks he has. Across a series of studies designed to get beyond correlation to causation, he’s found: “the findings that task significance increased job performance across different occupations, samples, manipulations, lend credibility to long-held assumptions about the significance of task significance in shaping employees’ behaviors.” The key seems to be in how employees perceive the social impact and social worth of their jobs. Thus, “interventions designed to enable employees to gain a deeper understanding of how their work benefits others, not only themselves, may play an important role in increasing work motivation and performance.”
(From "Does Intrinsic Motivation Fuel the Prosocial Fire? Motivational Synergy in Predicting Persistence, Performance, and Productivity," by Grant (2008), JAP 93(1), 48-58.)
There is a widely held belief, among both entrepreneurs and researchers, that the world of entrepreneurship is best navigated by instinct, not careful, elaborate planning. However, Frese and his colleagues at Justus-Liebig-University, in Giessen, Germany, would disagree. As it turns out, “elaborate and proactive planning” is significantly related to entrepreneurial success.
For some, this may seem like an obvious conclusion. It is not. A significant number of people have argued over the years that it is a combination of brains, motivation, and sheer luck that determines the fate of a small business. Certainly, this study confirms that cognitive ability and motivational factors are significant contributors to small business success. However, it is quite clear that plans, defined as “mental simulations of actions used to develop for thought and control future actions,” play a very large role, as well.
This is particularly important in at least a couple of respects. First, since it is very difficult to improve cognitive ability while planning, on the other hand, can be taught, this is no small finding. Second, the findings of this study suggest that planning does not, contrary to the belief of some, impede small business improvisation and experimentation. Rather, it seems more likely that “planning and experimentation can actually complement each other. … Planning may actually improve the use of experimenting and improvising; for example, owners may be able to recognize more effectively whether they are on track in their experiments.”
Thus, it seems there are at least three a major “buckets” of factors in the AQAL model’s “upper-left”, or Intentional quadrant that contribute to small business success: brains, motivation, and the knowledge of how to make plans.
(From “Business Owners’ Action Planning and Its Relationship to Business Success in Three African Countries,” by Frese, Krauss, Kieth, Escher, Grabarkeiwicz, Luneng, Heers, Unger, & Friedrich (2007), JAP 92(6), 1481-1498.)
Most organizations (91%) evaluate their training programs with satisfaction surveys. Far fewer evaluate what trainees learned (54%) or how well their training transferred to their work (23%). Does this make sense? Are we training employees so that they can be satisfied, or so that they can learn how to do their jobs better?
One answer is that it is too hard or too expensive to assess how much people learned in the training. The argument then goes, “if they’re satisfied with the training, perhaps that means they learned something, and therefore we’ll be able to use satisfaction as a substitute for learning.”
But is that true? Sitzmann and her colleagues set out to find out the answers not only to these questions, but to other questions, like what characteristics of the trainee or the course will predict how the trainee rates the course?
The conducted a meta-analysis of 136 studies to address their dozen hypotheses about trainee reactions. Here is an all-too-brief assessment of their key findings:
So, what are the practical take-aways? Here is the one that matters most to me:
Post-training satisfaction surveys are good for telling you about how motivated employees are likely to be as a result of the training, but not about how much they learned. Organizations are much better off asking a trainee whether he/she feels confident about their ability to use what they learned in a training than how much the liked the training. Of course, if you really want to measure how effective at teaching the trainees your training is, measure their learning, not their satisfaction with the course.
From: Sitzmann, T., Brown, K. G., Casper, W. J., & Ely, K. (2008). A review and meta-analysis of the nomological network of trainee reactions. Journal of Applied Psychology, 93(2), 280-295.
In addition, no matter what kind of employees you’ve got, you’re more likely to encourage taking charge behavior if you run your organization in such a way in which the employees believe that the procedures used to make decisions are basically fair.
(From "Me or We?" The Role of Personality and Justice as Other-Centered Antecedents to Innovative Citizenship Behaviors Within Organizations", by Moon, Kamdar, Mayer, and Takeuchi, 2008, JAP 93(1), 84-94.)
Van Kleef and Cote have looked at the question of whether there is a pattern to when anger works and when it does not, and have concluded that it has something to do with the appropriateness of the expression of anger and the relative power of the two involved in the conflict.
So, what is “appropriate” expression of anger? An appropriate expression of anger is “correct for the situation and in correct proportion to the evoking circumstances.” The researchers in this study let the observer be the judge of propriety.
Here are the key findings:
People with a notable power advantage can be angry all they want – inappropriately or appropriately – with less powerful people and, at least in the short run, get what they want. Low-power people are likely to concede ground in a conflict to high-power people who are angry.
Low-power people, on the other hand, need to be careful about the appropriateness of their anger. If their expression of anger is appropriate, they have a decent shot at gaining some negotiating advantage. If the expression of anger is inappropriate, the more powerful person is more likely to dig his or her heels in and actually give less ground than they might otherwise have given.
Naturally, there are moral implications, not just questions of efficacy, when it comes to expressions of anger. The authors cite at least one – the immunity of high-power people to the negative consequences of inappropriate expressions of anger can be seen as fostering leader tyranny: “our model indicates how anger may be employed as a tool by unethical leaders to achieve their aims.”
What is left for other studies is the long-term impact of the choice to consistently use anger – inappropriate and appropriate – as a negotiating tactic. I’m betting that it isn’t good.
We assume that doing a better job will, in a fair universe (and workplace), result in a higher appraisal from one’s supervisor. What has been less obvious to researchers is whether you can also influence that appraisal by voluntarily helping coworkers with their workplace problems and/or speaking up with innovative suggestions for change.
Steven Whiting and his colleagues have moved passed mere correlation to the world of causation and have concluded that “helping” and “voice” behaviors have an independent, significant impact on the boss’s performance evaluation, above and beyond mere task performance.
So, work hard, be helpful, and speak up (helpfully) if you want a raise.
(From "Effects of Task Performance, Helping, Voice, and Organizational Lyoalty on Performance Appraisal Ratings", by Whiting, Podsakoff, and Pierce, 2008, JAP 93(1), 125-139.)
In what will be absolutely no surprise to practitioners of mindfulness in all traditions, researchers have found evidence that meditation may be able to increase one's compassion.
![]()
While meditators have reported this for several millenia, what makes it interesting is that the study is focusing on the neuroscientific aspects of the phenomenon, or what Integral Theorists might call the "upper-right quadrant". Like it or not, meditators, Western culture requires this sort of inquiry, in addition to individual ("upper-left") and collective ("lower-left") reports that validate this finding from the inside of our experience.
Lutz, Brefczynski-Lewis, Johnstone, and Davidson examined brain activity of expert meditators and novices in meditative and non- meditative states in response to various stimuli selected to evoke empathy. Using neuroimaging, the researchers examined activity in those portions of the limbic region of the brain that have been found to be associated with empathic responses to another's pain.
Not only did the expert meditators' limbic regions show greater sensitivity to the sounds of pain of others than those of the novices, but they also showed a greater overall sensitivity to human sounds revealing emotion than did the novices. The researchers concluded that "the mental expertise to cultivate positive emotion alters the activation of circuitries previously linked to empathy and theory of mind in response to emotional stimuli."
Not being longitudinal, this study did not directly assess whether compassion can be learned. However, the evidence does point in that direction: "The fact that large and systematic changes in brain function were observed in response to auditory emotional stimuli presented during the meditative practice of compassion, and the fact that robust differences were observed between experts and novices, suggests that the next steps to evaluate the behavioral impact of this training and to longitudinally assess its effects are warranted."
That's geek-speak for "let's go get some funding for a longitudinal study to demonstrate that you can learn how to be compassionate by meditating." If we are to move compassion-building out of places of worship into the secular world, this kind of research will be essential.
It is widely understood that the more your organization's employees are oriented to customer needs, the more likely is your organization to receive high ratings of performance from those customers. This is true not only in the service sector, but also the manufacturing sector.
What are the factors that increase employee customer orientation ("ECO")? Liao and Subramony took at look at this issue and found a number of factors, the most obvious being proximity to the consumer. Customer-contact employees, such as customer service and sales personnel, had a greater ECO than did those in production roles (e.g., product design and production), who, in turn, had a greater ECO than support staff (HR, IT, and accounting). No surprise there, certainly, but it's not all that useful a finding on its own, because it doesn't tell an organization what it can do to improve its ECO.
Liao and Subramony also looked at the role of leaders, calling the extent to which an senior leadership is oriented toward customer needs "leader customer orientation" ("LCO"). While they found a positive relationship between the senior leadership team's customer orientation and ECO across all employee groups, regardless of proximity, they found a stronger connection between LCO and ECO among support staff (i.e., those farthest from customers). This was a predicted result, as support staff, being farther from the day-to-day expression of customer needs, will benefit more from leadership on this topic than those who are much more involved with consumers.
Thus, those organizations seeking to focus strategically on consumers will do well to hire customer-oriented senior leaders, then training them on how to develop customer orientation among employees. Some of the recommended mechanisms suggested by these authors and others include:
Anybody who has ever designed and attempted to implement an organizational practice to accomplish an objective such as higher performance, improved customer satisfaction, or higher quality, can tell you that not all of them work on all of these objectives. Gibson and her colleagues took a closer look at organizational practices and come away with a much more fine-grained understanding of which organizational practices do accomplish which objectives most predictably.
This blend of Texas and California researchers examined three kinds of practices: information sharing, boundary setting, and team enabling. They define these terms as follows:
No single set of practices among those that they examined predicted financial performance, customer service, and quality. The researchers found a clear relationship between:
The moral of the story is, share information if you want to make more money, establish clear policies about desired behavior to improve customer service, and empower your teams to raise the quality of your products or services.
(From "What Results When Firms Implement Practices: Differential Relationship Between Specific Practices, Firm Financial Performance, Customer Service, and Quality," by Gibson, Benson, Porath, and Lawler, 2008 JAP 92(6), 1467-1480).
In a meta-analysis of 46 studies of telecommuting, involving over 12,000 employees, Gajedran and Harrison of Pennsylvania State University concluded that existing research generally supports the well-publicized claims that telecommuting is good for employers and employees alike. However, “telecommuting intensity”, or the extent of scheduled time that employees spend doing tasks away from a central work location, can play a role in the impact of telecommuting.
“Telecommuting is mainly a good thing,” the authors concluded, positively associated with employee perceptions of autonomy, lower work-family conflict, and the quality of employee-supervisor relations. As a result, it is associated with increased job satisfaction, lower turnover intent, lower role stress, higher supervisor ratings, and higher objective ratings of job performance.
The study also highlighted perceived autonomy as the key “conveyor” of the benefits of telecommuting. In other words, it is the employees’ sense of autonomy that is the largest psychological contributor to job satisfaction, lower turnover intent, and the rest of the “good news” about telecommuting.
Finally, the intensity of telecommuting did have the effect on the employee’s role-stress. Women in particular benefit for more intense telecommuting than do men, particularly in terms of family-work conflict. About the only significant downside to intense telecommuting was a negative impact on coworker relationships.
Thus, the implications of this meta-analysis for employers are:
(Gajendran, R.S. and Harrison, D. A., “The good, the bad, and the unknown about telecommuting: Meta-analysis of psychological mediators and individual concequences,” (2007). Journal of Applied Psychology, 92(6), 1524-1541.)
If we see Barack Obama in the midst of an excited crowd of admirers, are we more likely to see him as charismatic?
According to Pastor and his colleagues, we are. But don’t get excited, you uncharismatic leaders out there. Simply persuading your supporters to get excited won’t substitute for your own charisma – you’ve got to have that, first. Only then, with your own charisma established, will the sight of your excited supporters increase observers’ belief that you are charismatic.
Most of the research on charismatic leadership has focused on the characteristics of the leaders. There has been a growing tendency to look at the followers, too. In this study, the researchers focused on the extent to which followers were aroused, i.e., the intensity of their emotional response to the leader.
One of the theories tested in this study, “misattribution theory”, suggests that a phalanx of aroused followers can, in and of itself, be interpreted as the sum and substance of the leader’s charisma. In other words, might somebody watching a non-charismatic leader mistake the followers’ excitement for charisma? The conclusion was that the leader must initially have some significant level of charisma in order for the followers’ excitement to generate more. The researchers quote Klein and House:
“Charisma is a fire, a fire that ignites followers’ energy and commitment, producing results above and beyond the call of duty.” The researchers extended that metaphor by saying that “followers’ arousal increases th flammability of followers by raising the propensity of followers to perceive the leader as charismatic and, therefore, giving a boost to the fire.”
Thus, through the lens of Integral Theory, my internal (upper-left) belief about the leader's charisma is triggered by observing not only the behavior of the leader (upper-right), but also the behavior of the followers (lower-right) which in turn evokes a shared belief between myself and the followers that the leader is charismatic (lower-left).
(Pastor, J.C., Mayo, M., & Shamir, B. (2007). Adding fuel to fire: The impact of followers' arousal on ratings of charisma, Journal of Applied Psychology, 92(6), 1584-1596.)
The researchers found that, not surprisingly, incivility can affect job satisfaction, which can affect employees’ desire to leave the organization. Furthermore, they found that this rudeness can affect group members’ mental health, which in turn affects their physical health.
Thus, rude employees can not only make their colleagues unhappy and more likely to quit, they can literally make them sick, both mentally and physically.
I'll call your attention to the interaction among the various quadrants in the AQAL model -- individual employee behavior affecting attitudes, at both the individual and group levels, which in turn affect physical and mental well being, again at both the group and individual levels.
More wisdom into the interactions between interiors and exteriors and individuals and collectives of the workplace.
(From "Personal and Workgroup Incivility: Impact on Work and Health Outcomes,( by Lim, Cortina, and Magley, 2008, JAP 93(1), 95-107).
I had so much fun with the last blog, in which I gave some very quick summaries of articles from the current issue of the Journal of Applied Psychology (“JAP”) (Volume 93, Number 1, January 2008), that I thought I would do some more. I have decided, though, to submit my summaries one at a time. So, here's a summary of my favorite one from that issue, one that beckons me to probe more deeply, as you'll see:
In a far-reaching examination on how a group’s conflict resolution style affects both its performance and the satisfaction of members of the group, Behfar and her colleagues developed an elegant group-level theory about conflict resolution strategies ("The Critical Role of Conflict Resolution in Teams," by Behfar, Peters,on, Mannix, and Trochim). This theory posits two broad dimensions: pluralistic-vs.-particularistic strategies and reactive-vs.-preemptive strategies. Here’s the short version of what they concluded:
Groups with conflict resolution strategies that are both particularistic (aimed at satisfying or containing individual members) and reactive (applied in reaction to existing problems) are more likely to do poorly on both performance and satisfaction. Strategies are designed to react to previous problems by focusing on minimizing individual misery, to include divide and conquer, avoiding debate, choosing the easy solution, avoiding group meetings, and trial and error to correct processes.
Not a happy place to be.
You can make people happier by adopting preemptive (planned to preempt negative effects of conflict), as opposed to reactive strategies. So, a particularistic and preemptive strategy is characterized by anticipating how group decisions impact individual feelings, including work assignments based on volunteers, displacing analysis with the inclusion of all ideas, and a strong focus on cohesion. You’ll get happier people, but you won’t improve performance.
Performance is addressed by moving from particularistic strategies to pluralistic strategies (conflict approaches that apply to all in the group). So, if you are using reactive and pluralistic strategies, your group will react to previous disruptions by restructuring and clarifying expectations, including the use of written rules and punishments, majority rules at decision time, and an arbitration approach to conflict. Performance in such groups is up, but satisfaction is low.
You get to conflict management nirvana by combining pluristic and preemptive strategies. In this approach, the group foresees problems and preemptively organizes to eliminate disruptions, including work assignments based on skill, forecasting scheduling and work-load problems, understanding the reasons behind compromises, and focusing on content over delivery style.
What’s interesting to me about this model is that I recognize some developmental themes here from Integral Theory and Spiral Dynamics. The particularistic/reactive looks pre-rational, or "red" in both the Integral Theory and the Spiral Dynamics color schemes. The pluralistic/reactive approach looks more conventional-rational to me, or Integral Theory's "amber and Spiral Dynamics' “blue”. Particularistic/preemptive, with its all-inclusive, individualistic approach looks post-rational, “green”, and pluralistic/preemptive seems almost second-tier, or integral (turquoise or yellow). This goes beyond the authors’ research, but their conclusions sure seem familiar to me. Taking Integral Theory’s developmental approach, it would suggest that groups learn how to be pluralistic before they learn how to be preemptive, since post-modernity follows modernity in most individuals and organizations.
This is worth a closer look, and maybe even a submission to the AQAL Journal. Stay tuned.
In recent years, employers have paid increasing attention to their employees’ struggle to balance work and family responsibilities. Often, this effort has taken the form of providing more family-friendly benefits and policies. However, it now seems that work redesign may be one tool available to employers. Valcour’s study of service employees and their work-family satisfaction concluded that, while the more hours a person works, the less likely that person is to be satisfied with the work-family balance, job complexity and control over work hours is much more important.
The lessons employers seeking to increase employee satisfaction with work-family balance can take away from this study are:
Certainly, increasing job complexity and providing employees with more control over work time is no small matter. However, it seems that such efforts are more likely to produce positive results in employee satisfaction, by a long shot, then reducing work hours will accomplish. High involvement work practices, such as substantive employee involvement in decision-making, the use of autonomous workings, training and mentoring, and pay for performance, are likely candidates for such job redesign efforts.
This is a classic example of how paying attention to the systems (the lower right) quadrant of AQAL Integral Theory can yield benefits in the intentional (upper-left) quadrant. The systems in which we are embedded have a profound effect on our interior experience of the world, in this case, satisfaction with a fundamental issue, the way we are balancing our work and family lives. While Dr. Valcour didn’t directly address of culture within these organizations (lower-left quadrant), it takes no great leap of imagination to suggest that the job redesign by her findings would have a significant impact on the culture of an organization.
(From "Work-Based Resources as Moderators of the Relationship Between Work Hours and Satisfaction With Work-Family Balance", by Valcour (2007), JAP 92(6), 1512-1523.)People who have a really tough job, one that may involve being exposed to danger to themselves or others (like a firefighter), sometimes drink to cope, particularly after critical incidents that cause extraordinary levels of stress. No surprise there. Other than individual interventions, is there anything an employer can do to prevent that from happening?
Samuel Bacharach and his colleagues believe so. Their study found a relationship between the adequacy of the resources in the workplace and employee drinking to cope. This resource adequacy showed up in both the connection between the critical incidents and the resultant stress and between the stress and the drinking behavior. Resource inadequacy seems to make these front-line workers vulnerable both to stress and to maladaptive coping behaviors.
The flip side of this is a cautionary tale for managers of such people – deny needed resources to your employees not only at the risk of their ability to perform, but at the risk of their health.
This close connection between the systems quadrant (lower-right) and the intentions quadrant (upper-left) of the "all-quadrants" component of the AQAL Integral Model is a familiar one in the world of organizations -- the systems in which we are imbedded have a direct and sometimes profound intent on our inner lives, one to which integrally-informed managers will attend.
(From "Firefighters, Critical Incidents, and Drinking to Cope: The Adequacy of Unit-Level Performance Resources as a Source of Vulnerability and Protection", by Bacharach, Bamberger, and Doveh, 2008 JAP 63(1), 155-169).
As even Snow White knew, good moods in the workplace are to be encouraged. It is not news, either to Walt Disney or to organizational psychologists, that positive moods not only feel good, but also improve task performance. What is not fully understood is why that is so. Why is it that positive mood generally improves performance? Tsai, Chen, and Lieu think that it is a combination of factors both intrapersonal (motivational) and interpersonal (giving help to and receiving help from co-workers) that provide the key link.
"In addition to its relationship with coworker helping and support via helping other coworkers, positive mood was directly associated with coworker helping and support." In other words, a good mood not only makes you more helpful and therefore more likely to attract help, but your very mood attracts help. Additionally, "positive moods can be contagious," also increasing the likelihood your coworkers will help you do your task successfully.
These interpersonal factors are not the only link, however. Employees with positive moods "may perform better through higher self-efficacy and task persistence." If we're cheerful, we're more likely to think we can do our job well, and are more likely to see the job through to a successful conclusion.
So, from the manager's perspective, what's the take-away? "Managers who hope to increase employee task performance could take actions to enhance employee positive moods [such as] making employees understand that the result of their work may have a significant effect on the well-being of other people [or] demonstrate charismatic or transformational leadership behaviors to create a positive affective tone in groups through the process of emotional contagion between leaders and group members." In short, if you're a manager, do what you can to cheer them up and letting them see the broader impact of their work in the world.
Again, we see the infiltration of integral thinking into organizational behavior -- the linkage between the intrapersonal (individual motivational factors) and the social (interpersonal dynamics), and between the interior (moods, individual and shared) and the exterior (behavior) is increasingly significant to researchers.
(From Tsai, W, Chen, C., and Liu, H., (2007). Test of a model linking employee positive moods and task performance. Journal of Applied Psychology, 92(6), 1570-1583.)
Personality tests are valid – to a point. It’s almost undisputed that the major personality tests generally tell us something useful about a person. But how much they tell us is usually pretty small – they make us only slightly smarter about the person taking the test.
So, how do you change the tests so they make us more than just slightly smarter? Context, context, context! One name used for context is “frame of reference.” So, instead of a context-free item like “I pay attention to details,” you go with an item like “I pay attention to details at work.”
Lievens and his colleagues challenged the conventional wisdom about why context works. It has been widely argued that contexts reduces the variability among test-takers, making it less likely that some of the test takers will answer a question across all contexts and others will self-select a context in which they interpret the test questions.
Lievens et al suggest – then test – an alternative explanation. They do not deny that context reduces between-person variability, but they suggest that the impact is minimal. They point to within-person variability as the key threat to validity that frame-of-reference reduces. The thinking behind this explanation is that, when responding to an item on a test, a person will conduct a selective search of his/her memory to help them respond to the item, and that providing the context for that memory search helps assure that the person will be more consistent with himself/herself in the responses. Thus reliability, and therefore validity, of the measure increases, not because different people are more likely to answer questions consistently with one another, but because each person is more likely to answer them consistently with themselves.
In a pair of studies, the researchers found the alternative explanation – focusing on within-person variability – to provide a better explanation of how context helps improve tests. Besides giving us a clearer theoretical understanding of how people think when they take tests, this research will help designers of test make sure the tests are not only reliable, but also valid, by making sure they select the correct frame of reference. The researchers found that using an incorrect frame of reference – one that is conceptually irrelevant to the criterion -- actually decreases validity.
From: Lievens, F, De Corte, W., & Schollaert, E. (2008). A closer look at the frame-of-reference effect in personality scale scores and validity. Journal of Applied Psychology, 93(2), 268-279.
As I write this in the midst of the battle for the Democratic nomination for the presidency of the U.S., there is a debate going on about whether it's harder for an African-American man or a Caucasian woman to gain the presidency. It is impossible to tell, of course, from a study of this election (I mistrust studies where n=1, Freud notwithstanding).
I did come across an article, however, that bears on the topic, if only to the extent that it tells us a bit about how sexism can operate where a traditionally male job is at stake. In "Motherhood: A Potential Source of Bias in Employment Decisions, by Heilman and Okimoto (2008, JAP 93(1), 189-198), the authors present significant evidence that, when people think about fitness for a traditionally masculine job, not only is there a bias against women, but there is a stronger bias against those women that more fully epitomize women. This study demonstrated that motherhood can "hinder the career advancement of women and that it is the heightened association with gender stereotypes that occurs when women are mothers that is the source of motherhood's potentially adverse consequences."
Certainly, this doesn't answer my original question about race vs. sex. However, it does shed some light on the nature of sexism in employment decisions -- it is womanliness itself, and how much an individual woman evokes that sense of womanliness in those assessing her fitness for a particular job, that can interfere with her chances at a traditionally masculine position.
Pregnant women have a special status, but “special” is not necessarily good, particularly when it comes to the workplace. In a fascinating exploration of the Theory of Ambivalent Sexism, Hebl and her colleagues examined how pregnant women are treated differentially from non-pregnant women in two kinds of settings: one consistent with a sexist view of the “proper”, or more traditional role for women (e.g. shopping), and one inconsistent with that view (e.g. employment in a traditionally masculine job). The short answer to their questions is this: through a series of subtle manifestations of sexism, pregnant women are more likely than non-pregnant women to be treated rudely and evaluated harshly as job applicants, and are more likely to be treated in an overly benevolent, even patronizing way than nonpregnant women when shopping.
It seems that pregnancy, as the purest and most visible marker of a woman’s traditional role, that is childbearing, activates our attention to the fact of her womanliness, and makes her more likely to be a target of our sexist attitudes and behaviors. The sexism cuts both ways. When a woman is engaged in a traditional role-appropriate behavior such as shopping, people are more likely to touch and engage in other aspects of over friendliness (such as patronizing smiling, eye contact, nodding, helpfulness) toward pregnant versus nonpregnant customers. On the other hand, when a woman is challenging those traditional notions of role-appropriate behavior for women by applying for or working in traditionally masculine jobs, pregnant women are significantly more likely to be the object of discrimination, both formal and informal, than are pregnant women. Formal discrimination shows up in a number of ways, including a decreased likelihood of job offers. Informal discrimination was manifested in a variety of rude behaviors, including hostility, anxiousness, staring, and furrowed brows.
Thus, it seems, that while these subtle manifestations of sexism are not universal, there are enough instances of these manifestations that it is reasonable to conclude that there exists a sort of a “carrot-and-stick” system of interpersonal rewards and punishments that seems designed to steer pregnant women away from employment and toward their more traditionally “proper” role in society.
(From “Hostile and Benevolent Reactions Toward Pregnant Women: Complementary Interpersonal Punishments And Awards That Maintain Traditional Roles,” (2007), by Hebl, King, Glick, Singletary, and Kazama, JAP 92(6), 1499-1511).