experiments have shown that in selecting personnel for a job

Employee Selection and Assessment , Strengthening Organizations

Effective Employee Selection Methods

Why would we ever want to use something like social media profiles to inform selection decisions when there are much much more accurate ways to evaluate applicant skills and fit? One reason, we thought, might be due to the overwhelming number and type of selection tests available. The purpose of this article is to help deal with that clutter by presenting three of the most effective and universal employee selection tools along with the outcomes and specific requirements that you can expect when implementing each. Although no method will ever be without drawbacks, the key is to find the one that best fits your hiring strategy and can most easily be aligned with your existing processes and procedures.

The of the most effective, valid methods of employee selection will be described below in detail. They include:

• General Mental Ability
• Structured Interviews
• Situational Judgment Tests

1. GENERAL MENTAL ABILITY (GMA)

GMA (a.k.a., cognitive ability or g ) is possibly the single most effective tool for selection. In fact, this approach is effective at predicting future performance in every type of job, at all job levels (from entry-level to CEO) and in every industry. GMA can be assessed in a variety of ways, from 30 minute paper and pencil tests like the Wonderlic, to more expensive online computer adaptive tests. Both computer and paper & pencil tests are equally valid, allowing organizations to select the approach that fits best.

Additional Information on GMA:

• Article. Return-on-Investment of using GMA as an employee selection method

2. STRUCTURED INTERVIEWS

These are not your standard interviews that start with “So tell me about yourself…” In structured or behaviorally-based interviews, applicants are asked a series of specific, predetermined, job-related questions while their responses are scored using detailed criteria (often presented in a scoring guide that provides detailed descriptions on what constitutes each rating). An “interview panel” approach is often used, where 2-3 trained managers ask the questions and score each response separately. After the interview, their ratings are compared to determine the consistency or interrater reliability. When responses are scored inconsistently, interviewers discuss their rationale and come to consensus.

Additional Information on Structured Interviews:

• Article. Adding Structure to Unstructured Interviews
• Reference Guide. A guide to Structured Interviews (.pdf guide from US Bureau of Human Resources)

3. SITUATIONAL JUDGMENT TESTS (SJT)

These tests have been described as the multiple-choice equivalent to structured interviews. In SJTs, applicants are asked to choose how they would respond to a variety of hypothetical situations that are relevant to the target job. Results indicate how that particular applicant will behave when faced with particular situations and decisions. The ability of this method to predict how applicants will respond to complicated decisions makes SJTs one of the best approaches for managerial and technical positions.

WHAT ABOUT COMBINING THESE METHODS?

YES! It is important to note that combining more than one instrument or method can greatly improve the predictive validity of your hiring process . For example, combining GMA tests with structured interviews will be much more effective than using either of them alone. Also, using any of these three methods would be better than evaluating applicant resumes and giving unstructured interviews or non-validated off-the-shelf tests.

There are countless tools, methods, and approaches to making good selection decisions. However, according to decades of applied organizational research the ones described above are the most successful, accessible methods for finding those diamonds in the rough. It is important to note that other valid methods were intentionally left out: Assessment Centers were not described because they are not a realistic approach for many jobs and organizations.

We have designed, developed, and validated selection processes for numerous public and private clients over the years. To leave you, reader, with a final thought, we have found that when job analyses are used a foundation to develop (or select appropriate off-the-shelf) selection tests, they pay off big in terms of improved performance, productivity, environment, and retention .

– Scontrino-Powell

References:

Christian, M. S., Edwards, B. D., & Bradley, J. C. (2010). Situational judgment tests: Constructs assessed and a meta-analysis of their criterion-related validities. Personnel Psychology , 63 , 83-117.

Schmidt, F. L., & Hunter, J.E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274.

Gatewood, R. D., Feild, H. S., & Barrick, M. (2011). Human Resource Selection (7th ed.), South-Western Publishing.

Related Stories

Personality and job performance, improve job interviews: add some structure, ready to take your organization to the next level.

Get In Touch

• Leadership Development
• Team Building
• Operational Excellence
• Case Studies

• Search Menu
• Sign in through your institution
• Advance articles
• Author Guidelines
• Submission Site
• Open Access
• About European Sociological Review
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, conclusion and discussion, acknowledgements, appendix 1: occupations according to the required levels of education, sex ratios, and degree of decision-making: number of vacancies and applications (in parentheses) in each of the sampled occupations, appendix 2: samples of rresumes for a tax advisor*.

• < Previous

The Role of Gender Stereotypes in Hiring: A Field Experiment

These authors M. José González, Clara Cortina and Jorge Rodríguez have contributed equally to this article.

• Article contents
• Figures & tables
• Supplementary Data

M José González, Clara Cortina, Jorge Rodríguez, The Role of Gender Stereotypes in Hiring: A Field Experiment, European Sociological Review , Volume 35, Issue 2, April 2019, Pages 187–204, https://doi.org/10.1093/esr/jcy055

• Permissions Icon Permissions

Using correspondence testing, we investigate if employers discriminate against women based on stereotypes or prejudices. We sent four (two pairs of fictitious man–woman) résumés to 1,372 job offers from a broad selection of occupations. In one pair, candidates had equivalent curriculum vitae (CVs) except for their sex and their qualifications (meeting standards or higher). In the second pair, candidates differed by sex and parenthood status (with or without children). We interpret the observed differences in favour of men as signalling gender bias in recruitment. This bias is reduced when women have higher qualifications and increases when they have children. We interpret employers’ openness to modify their decisions when candidates’ personal characteristics differ from the group norm, and the absence of discrimination among highly qualified non-mothers, as evidence that gender bias in recruitment is largely grounded in employers’ stereotypes rather than in prejudices.

In this study, we use a correspondence testing approach to investigate whether employers discriminate against women based on stereotypes or prejudices. The correspondence test is an experimental technique that consists of sending pairs of résumés for job offers that are very similar in everything except the trait to be analysed—gender, in our case. Discrimination is established if candidates with the trait—females—have a lower probability of being selected for further screening. We sent two pairs of résumés to 1,372 real job advertisements appearing in two large local labour markets (Madrid and Barcelona) in Spain in 2016.

This research design allows us to address some issues insufficiently explored in the literature, namely the extent to which women experience subtle forms of discrimination based on the order in which they are selected for further screening. More importantly, unlike in many previous studies, we additionally estimate if gender discrimination varies by an applicant’s parenthood status and level of qualifications. This allows us to assess to what extent employers’ discriminatory practices are grounded in stereotypes about males and females’ typical qualifications and prescribed roles as mothers and workers, rather than on unswerving prejudice against, or aversion towards, female workers. Finally, and as our research strategy combines within- and between-job comparisons of candidates’ appeal to employers, we add important controls to the analyses pertaining to job characteristics, like the degree of masculinization of the occupation, the level of authority required in the job, and the tightness of the labour market within which the job offer is posted.

Do Employers Favour Men?

The gender discrimination hypothesis states that gender bias in recruitment, that is, employers’ conscious or unconscious preference for male candidates, is pervasive and it may arise for different reasons. According to economic theory, discrimination may be explained by employers’ imperfect information on applicants’ human capital characteristics, which are relevant for the job and also difficult to standardize on a résumé ( Becker, 1985 ; Heckman, 1998 ). Employers use group-level statistical summaries (i.e., group averages) as proxies for these variables, and this explains why individuals from these group are treated differently. Thus, in selecting candidates for a job opening, employers not only consider the observed and standardizable qualifications shown in candidates’ curriculum vitae (CVs), but also rely on stereotypes about the typical level and dispersion of other difficult-to-standardise qualifications. This practice leads to forms of statistical discrimination based on rational assessments of productivity and risk regarding potential employees ( Baumle and Fossett, 2005 ). Ascribed characteristics such as gender and age are used as potential proxies for traits that are difficult and expensive to measure in real contexts.

Some stereotypes are descriptive and based on knowledge of men and women’s typical abilities. This knowledge can be direct, based on experience, or indirect and transmitted by trusted third parties. Thus, in our societies, in which the division of labour is gendered, men typically appear as possessing greater agentic and leadership qualities than women, and higher aspirations and commitment at work. In contrast, women are assumed to possess greater communal qualities associated with caring behaviours ( Cuddy et al. , 2004 ). Other stereotypes are prescriptive, that is, based on cultural beliefs about what men and women ought or ought not do. They are often justified with reference to higher communal values that reinforce a system of patriarchal authority favouring men ( Connell, 1995 ; Rudman and Glick, 2001 ). Prescriptions are accompanied by sanctions when someone violates them. Thus, working mothers who behave agentically may be perceived as lacking femininity and be subjected to a variety of sanctions ( Connell, 1995 ; Benard and Correll, 2010 ).

Sociologists have repeatedly shown how the patriarchal character of organizational life reinforces the idea of separate spheres for women and men, and contributes to the perception that being an ‘ideal worker’ is incompatible with being a ‘good mother’ ( Fuegen et al. , 2004 ; Ridgeway and Correll, 2004 ; Benard and Correll, 2010 ; Glass and Fodor, 2011 ; Byron and Roscigno, 2014 ). The ideal worker is a ‘committed’ employee who sacrifices his most personal concerns, such as those derived from family responsibilities, for the sake of his career; a worker who is expected to drop all current engagements when a new and important work demand arises, devotes many hours to ‘face time’ at work when needed, or works late nights or weekends if necessary ( Correll et al. , 2007 ). The ideal of the ‘good mother’, instead, places the child at the centre of the family and assumes that child care is the chief responsibility of the mother—an emotionally absorbing, labour-intensive, and time-consuming responsibility ( Hays, 1996 ).

Descriptive stereotypes contribute to the generation of prescriptive views or beliefs about men and women’s proper roles in society; and prescriptions typically lead to the desired outcomes. Hence, under the ‘motherhood mandate’, women are expected to be more family oriented and less committed to paid work, and thus, less productive than similarly qualified male workers ( Russo, 1976 ; Hays, 1996 ). They are also expected to have higher rates of absenteeism, which, according to employers, eventually affects their productivity at work ( Correll et al. , 2007 ). Employers’ expectations may even be harmful to some employees, such as when pregnant women are determined to appear ‘well’ and remain at work even when feeling sick ( Gatrell, 2011 ).

According to the theories discussed above, both descriptive and prescriptive gender stereotypes may influence hiring processes. We develop two hypotheses to test these possibilities. We expect that discrimination against female candidates will be smaller for candidates with higher levels of qualifications for the job. The reduction in the level of employers’ discrimination against female candidates will indicate that they allow any perceived handicaps in women’s unobservables to be compensated with candidates’ higher observables. We will interpret a reduction in discrimination among applicants with higher standardized qualifications, as evidence that employers engage in statistical discrimination based on descriptive stereotypes or on shared beliefs about the typically different traits and abilities of men and women, which can be counteracted when candidates show higher qualifications.

Furthermore, we expect that discrimination against female candidates will be higher for candidates with children. As before, the change in employers’ level of discrimination against women will indicate that they are willing to adapt their stereotypes on women and men’s qualifications when receiving further information on candidates’ characteristics. However, the increase in discrimination will in this case show that their behaviour is based on prescriptions about the proper roles of men and women in society, punishing non-compliant women with higher discrimination when the stereotype of the good mother and worker is challenged. Hence, if discrimination increases when we restrict our sample to applications by job candidates with children, we will interpret this finding as evidence for statistical discrimination based on prescriptive stereotypes .

Instead of (or in addition to) statistically discriminating women, employers, particularly males, may rely on gender prejudices, based on negative feelings about women, that similarly result in higher barriers to women’s employment, especially in high-status occupations ( England, 1994 ; Jaret, 1995 ). Prejudices are negative judgements about groups that carry a stigma. These judgements are irrational because—unlike statistical discrimination—they are not based on expectations about groups’ productivities ( Becker, 1985 ). It can arise at different stages of an individual’s work history (pre-selection, job interview, or promotion), which makes it difficult to tackle the subject empirically ( Baumle and Fossett, 2005 ). Psychologists have studied in detail the basis of prejudicial attitudes—see Hodson and Dhont (2015) for a recent review of this work. They have shown that prejudicial individuals are more likely to display automatic emotional responses of animosity or antipathy towards members of other groups, based on faulty and inflexible generalizations of their inferior qualities ( Allport, 1954 ). Prejudices are often acquired at an early age and as a consequence, in the case of gender, of segregated socialization ( Glick and Hilt, 2000 ). We propose to test this hypothesis of prejudicial discrimination by assessing if discrimination is significant also in the group of female candidates with the lowest probability of being statistically discriminated—the group of highly qualified non-mothers. In other words, if discrimination remains for the group of most clearly competent female candidates, we will interpret this finding as evidence for discrimination based on prejudice .

Gender Discrimination in Recruitment Decisions

Much of the evidence on gender discrimination derives from field experiments such as correspondence studies. These studies are considered to be the most reliable methods to reveal unequal treatment in hiring ( Riach and Rich, 2002 ), because, unlike observational studies, they can control for selection effects and problems associated with endogeneity. These effects occur, for example, when women themselves make employment and occupational choices that lead them to disadvantaged positions ( Skyt Nielsen et al. , 2004 ; Sahni and Paul, 2010 ). While these choices may also respond to the same stereotypes and prejudices affecting employers, or anticipate these attitudes, they can generally only indirectly be attributed to discrimination ( Lundberg and Startz, 1983 ).

In correspondence studies, fictitious individuals who have nearly identical résumés except for certain traits such as sex, apply for the same jobs, and differences in outcomes—usually callback rates—are interpreted as reflecting discrimination. The difficulty in correspondence studies is generally not with detecting discrimination but with identifying its sources—stereotypes or prejudices (Neumark et al. , 1996). To accomplish the latter, in many correspondence studies the experimenter varies the personal characteristics of the fictitious applicants to determine if differences in employers’ rates of response vary accordingly between men and women ( Larribeau et al. , 2013 ). For example, the experimenter may vary the parenthood status of the applicant within or across jobs, and ascertain if women are more discriminated against when they have children ( Correll et al. , 2007 ; Albert et al. , 2011 ; Bygren et al. , 2017 ). If they are, this is attributed to employers’ reliance on prescriptive stereotypes when making hiring decisions, based on beliefs that mothers should not be given a job because their place is at home. In other studies, the marital or age status of the applicant is modified, and used as an indicator of how likely they are to become a parent in the near future ( Petit, 2007 ).

Still in other studies, qualities that are typically unobserved in a CV, like personality traits, are subtly added to candidates’ résumés, and employers’ reactions are identified and explored to reveal if women who display ‘masculine’ traits are penalized more than others ( Weichselbaumer, 2004 ). Finally, in some studies what is varied is the level of standardized qualifications of the applicant, in the expectation that employers relying on stereotypes may consider that women’s typical handicaps in unobservable traits are smaller (i.e., can be compensated) when they have higher standardizable qualifications—perhaps because only women who do not have these handicaps can achieve such higher qualifications (Larribeau et al. , 2013). In all cases, what allows distinguishing stereotypical from prejudicial discrimination is employers’ disposition to change their hiring decisions against women when applicants’ personal characteristics diverge from gender stereotypical norms. Prejudicial discrimination, in contrast, is residually established as any discrimination left and exercised against the most favoured sub-group of women.

The evidence provided by correspondence studies on gender discrimination in hiring, and on its sources, is rather mixed. A few studies conclude that there is no discrimination (e.g., Albert et al. , 2011 ; Bygren et al. , 2017 ). Some others maintain that discrimination occurs only for some subgroups of female applicants, as expected under the hypothesis of statistical discrimination based on stereotypes ( Petit, 2007 ). The remainder suggest that it occurs only in some contexts for some age groups ( Albert et al. , 2011 ). Correspondence studies differ also in terms of the contexts that they choose to study. Some are located in countries with strong family policies promoting and facilitating mothers’ employment, like Sweden or France ( Petit, 2007 ; Bygren et al. , 2017 ); others in more traditional institutional contexts like Spain ( Albert et al. , 2011 ; León and Pavolini, 2014 ); and still others, in societies with mixed gender equality records, like the United States or the United Kingdom ( Neumark et al. , 1996 ; Correll et al. , 2007 ; Larribeau et al. , 2013).

We sent sets of fake résumés to a wide-ranging sample of 1,372 job openings/vacancies between June and November 2016 in the two largest Spanish cities. Madrid and Barcelona are two of the most economically dynamic cities in the country, with a larger supply of job offers and employment rates far above the national mean, which makes them particularly suitable for our experimental research. Spain has recently experienced a rapid and massive incorporation of women into the labour market and the educational system, but it still exhibits large gender inequalities. According to the Spanish Statistical Office, in 2016 the employment rate for the population aged 25–54 (typical motherhood ages) was 65.6 per cent for women and 77.4 per cent for men, a difference of almost 12 points; the prevalence of unemployment in the same age group was higher for women than for men (20.3 per cent vs. 16.3 per cent); part-time work was overwhelmingly done by women (24.1 per cent vs. 7.8 per cent); women earned 14.2 per cent less than men when comparing their average gross hourly earnings, and they represented only a third (31 per cent) of workers in managerial positions (Eurostat). Gender inequalities in the labour market can be partly attributed to the limited support of the welfare state for working parents and mothers’ difficulties for reconciling paid work and caring responsibilities ( León and Pavolini, 2014 ). This explains why the presence of women in the labour market decreases with the number of children (according to Eurostat, in 2016 the employment rates for women aged 20–49 without children, with one or two children, and with three or more children were, respectively, 72, 67, and 52 per cent).

The sampling of jobs was designed to reflect the diversity of the labour markets in terms of the following: (i) the tightness or level of unemployment, which was proxied with the variable ‘city’ (the Spanish Statistical Office estimated the unemployment rates in Barcelona and Madrid in 2016 to be, respectively, 12, 5, and 14.8); (ii) the typical sex ratio of the occupations to which each job pertained, that is, male-dominated occupations (15–40 per cent women), mixed occupations not dominated by either sex (41–61 per cent women), or female-dominated occupations (62–83 per cent women); (iii), whether or not the tasks to be performed at work were managerial; and (iv) the average number of years of education of workers in the occupation, which we interpret as signalling the required level of qualifications for the job, that is, low, medium, or high. 1

However, it should be noted that the sample is far from being representative of all job openings in the two cities during the period of the experiment. First, the online Internet service we used to access the job openings, while widely used in both cities by job seekers, cannot capture other vacancies filled through more informal channels of recruitment ( Fernández-Muñoz and Blasco-Camacho, 2012 ). Second, by design, we decided to send applications to an approximately equal number of job openings in each of the 18 types of jobs (occupations) that resulted from multiplying the three levels of education by the three sex ratios by the two decision-making categories in which we divided all job openings—see Appendix 1 for a detailed list of the 18 occupations and the number of jobs applied for in each occupation (an extended version of the methodology, the data used in this study and the Stata do-file to completely replicate this study are available as Supplementary Data ).

In contrast to most previous correspondence studies, for each job opening we sent four fake applications. 2 The four applications consisted of two sets of matched CVs, with each set containing a CV from one male and one female with equivalent characteristics. The two sets differed in either candidates’ level of skills/qualifications for the job or their parenthood status (see the four sets of résumés in Appendix 2). As noted in the previous section, we were interested in assessing the effect of these two factors on employers’ gender discrimination practices. Thus, in approximately half of the job openings, we sent four applications consisting of two sets of matched-paired male–female applications differentiated by candidates’ skills. Skill differentiation was introduced in the CVs by making adjustments to the résumés. In half of the applications the candidate met the strict requirements for the job offer (i.e., a shop assistant had the typical educational level for this position, up to secondary education according to the labour force survey, and a short work experience). In the other half of the applications, the candidate also met the strict requirements for the job offer, up to secondary education for a shop assistant, but additionally he/she reported speaking a foreign language, having longer work experience and holding supervisory roles in previous jobs. In each of these four applications, the parenthood status of the candidates was fixed to either a ‘with two children’ or ‘without children’ status, with this status alternating across the job openings. In the other half of openings, we sent four applications consisting of two sets of matched-paired male–female applications, this time differentiated by candidates’ parenthood status, that is, by whether or not they had children. In these four applications, candidates’ skills were fixed to either low or high, with each skill level alternating across job openings.

By sending two pairs of applications differentiated by candidates’ skills to half of the job openings and two further pairs differentiated by parenthood status to the other half, we considerably reduced the sample of job openings needed to detect significant differences in discriminatory practices against women based on candidates’ qualifications and parenthood status. Additionally, this research design allowed us to test if variations in gender discriminatory practices according to candidates’ characteristics occur within the same job opening (e.g., when the same hiring agent discriminates against women when two candidates display low qualifications or when they have children, compared to when they do not) or across job openings (e.g., when two agents from two different companies display differential treatment towards women), or in both cases. 3

There is an ongoing debate about the pros and cons of sending match-paired applications to the same employer or single applications to multiple employers. We took this debate into consideration when choosing our research design and opted for the matched-pairs design. This design allows us to use the order in which the candidates were called back for an interview, thus helping to understand the multiple and subtle ways in which discrimination may be practised. The matched-pair approach is well established in the sociological literature—see, for instance, Quadlin (2018) or Bygren et al. (2017) . Unmatched designs based on random allocation of fictitious candidates across jobs guarantee that the treated and controlled arms are equivalent only in the expectation (on average across multiple replications of an experiment). A matched-pair design replicates the experiment on the same unit/employer, making the experiment more efficient and minimizing the risk of making Type II errors ( Bruhn and McKenzie, 2009 ).

Our decision to send four rather than two résumés to the same employer helped solve a problem often overlooked in matched-paired designs that send only two CVs. In the latter, the decision to decline a first invitation to attend a job interview issued to one of the fake candidates may increase the probability that the second fake candidate is selected, thus potentially obscuring the extent of discrimination. If employers’ hiring strategy consists of calling a fixed number of potential candidates for an interview, the probability of being on this list could be affected by other candidates’ (also fake candidates’) decisions to decline the invitation, giving an opportunity to a female candidate to be interviewed who otherwise might have never had this opportunity (especially in jobs with few applicants). By sending four résumés and by considering the order in which the candidates are selected, we avoid this problem and are better able to assess the extent of discrimination.

The age of all candidates varied randomly across job openings, but it was always within the range of 37 to 39 years. This age range was selected to reduce employers’ uncertainty about potential maternity or paternity leaves linked to future births. Each application used one of over 350 individual profiles created specifically for the study. The creation of such profiles and their online storage was required by the Internet service as a condition to apply for any of the job openings. To save resources, we decided to use the same profile for four different job openings. Each profile included a photograph randomly assigned to the corresponding fake male or female candidate and a résumé, standardized in the format required by the Internet service, without any cover letter. The standardized form of the résumé helped reduce any potential biases stemming from its layout.

Responses to the applications from interested employers, usually for the purpose of setting up an interview with the candidate, arrived in the form of either a call to a cell phone number or a message to an email account setup for groups of candidates never applying for the same opening. The fieldwork team used eight different cell phones to receive the phone calls and checked the email accounts of each of the fake candidates daily. Application rejections arrived in the form of messages sent explicitly by the web service or were inferred from the absence of any contact from the prospective employer 4 weeks after the application was submitted.

Our research design raises several ethical concerns as it imposes an additional burden on employers who review fictitious job applications and participate in an experiment without giving informed consent. Deception and absence of informed consent are serious issues which in our case required the approval of an Ethical Consortium before the field work could start. The Consortium decided that detecting and understanding how gender discrimination operates within the Spanish labour market outweighed these ethical concerns.

Analytical Strategy

To assess the presence of discrimination, we analyse differences in response rates by sex in multilevel models, in which applications (level 1) are nested within job vacancies (level 2). This model allows us to take into account the auto-correlation of decisions made by the same employer regarding the set of applications we sent to them for evaluation. Multilevel models are common in matched-pair designs because they are more efficient, and the standard errors they estimate are unbiased (see Uggen et al. , 2014 ). We estimate two types of models, according to whether the dependent variable is one of the following: (i) a binary variable measuring whether the candidate was or was not called back by the employer for further screening; or (ii) a limited interval variable reflecting the call order in which the candidates were called back, scaled from 0 (he or she was never called back), to 1 (he or she was called back in fifth place 4 ), to 2, 3, 4, and 5 (he or she was called back before any other candidate). For the first set of analyses (a), we use multilevel logit models; for the second set (b), we use multilevel ordered logit models, which are more appropriate than OLS regression to estimate models in which the interval dependent variable’s distribution is strongly skewed and bounded within a limited set of discrete, positive values ( Greene, 2012 ).

We use random intercepts because two characteristics used to distinguish the male–female matched pairs were forced to vary randomly. In half of the job openings, the two pairs differed in skill level and had the same parenthood status, which changed randomly for all four candidates across jobs. In the other half, they differed in parenthood status and had the same level of skills, which changed randomly across jobs. Hence, there were two possible sources of variation in skills and parenthood statuses: variations among fake candidates applying for the same job and variations among fake candidates applying for different jobs.

In all models, the independent variables of substantive interest are candidates’ sex, qualification level, and parenthood status. In all models, we test whether differences in callback probabilities between men and women differ for candidates with alternative skills and parenthood status via two-way interaction effects between gender and each of the other variables of interest. In all models, we also use two types of controls, linked to the characteristics of either the applications or the job openings. 5 For the former, the characteristics are: the order in which the application was sent within the set, and whether it was within a set of two, four, or six applications (and for the four-application sets, whether the trait that was fixed for all candidates was their level of skills or parenthood status). Among the latter controls, we included the number of other applicants that had applied for the job at the time we sent our first application, whether or not the job bestowed supervisory and decision-making power; the level of education required by the job; whether the three-digit occupational category to which the job belonged was male or female dominated, or mixed; and the city in which the job was offered.

Number of applications and callback probabilities by candidates’ sex, skill level, and parenthood status

The overall callback rate was 9.3 per cent 6 (see lowest right-hand corner of Table 1 ), which is within the standard of previous audit studies ( Bertrand and Mullainathan, 2004 ; Quadlin, 2018 ) despite the high unemployment rate of the country. As expected, candidates with higher qualifications on their résumés were called in larger numbers than candidates with lower qualifications (11.6 per cent vs. 6.7 per cent). Additionally, childless candidates received more callbacks than candidates with children (10.2 per cent vs. 8.4 per cent). More importantly for this study, men were called back in higher proportions than women (10.9 per cent vs. 7.7 per cent). Finally, Table 1 shows that even in the sub-group of women subject to lower discrimination (highly qualified non-mothers) men were called back in higher proportions than women (13.6 per cent vs. 12.4 per cent).

Multilevel logit of being called back for an interview, by candidates’ sex, level of qualification, and parenthood status, and controlling for key characteristics of the application and the job opening

Notes: Baseline: male, low skilled, childless, mixed occupation, managerial, mid-education job, Madrid, 0 previous applicants, sent first, only two applications sent.

Significant at the 0.01 level.

Directional test.

Focussing on the variables of substantive interest, the results in Table 2 suggest that women are significantly more discriminated against by employers than men. These gender discriminatory practices are not significantly different among candidates with lower and higher qualifications or among candidates with and without children. 7 However, the direction of the differences is as expected in our two hypotheses concerning statistical discrimination—higher gender discrimination among less qualified applicants and among candidates with children.

Multilevel ordered logit of being called back for an Interview, by candidates’ sex, level of qualification, and parenthood status, and controlling for key characteristics of the application and the job opening

Significant at the 0.05 level.

The cut points at the bottom of the table indicate the thresholds on the latent variable governing the probability across all jobs that a candidate would be chosen with a different priority or not at all. Cut point 1 is the estimated threshold on this latent variable used to distinguish candidates chosen in the last (fifth) place compared to candidates who were not selected for an interview (whose cut point is set to 0). Cut point 2 is the estimated threshold on the latent variable used to distinguish candidates chosen in fourth place from those chosen in fifth place; and so on until cut point 5, which is the estimated threshold that separates candidates chosen in first place from candidates chosen in second place.

The values of the cut points give a sense of the distance (in terms of probability) of falling into each category of the dependent variable. The ‘distance’ between the score 0 assigned to the baseline and cut point 1 is much larger (3.683) than the distance between cut points 1 and 2 (3.691–3.683 = 0.008). The scaling of the cut points in the dependent variable allows the coefficients in Table 3 to be interpreted as per a regular multilevel model with an interval dependent variable, as the change in the mean of the dependent variable per unit change in each independent variable (for continuous independent variables), or as the difference in the predicted mean of the dependent variable between any category of interest and the baseline category represented by candidates who were never called back (for categorical independent variables).

The effects of the controls on the mean of the rescaled variable trend in the same direction as those estimated previously for the logit model. Application characteristics have no impact on whether or not candidates are selected for an interview and the order in which it occurs. In terms of job characteristics, those that typically have a mixed male–female composition, that do not require much decision-making, that require medium levels of education, that were offered in Barcelona, and that received lower numbers of applications from non-fake candidates, all display higher scores in the optimally rescaled dependent variable, meaning that these candidates were given priority over other candidates by employers in the selection process.

The coefficients for the variables of substantive interest also trend in the same direction as in the multilevel logit model. The main difference is that the interaction effects between candidates’ gender and skills and between candidates’ gender and parenthood status are now significant (directional tests). The results show that applications by women are given significantly less priority by employers than applications by men, thus confirming the hypothesis on gender discrimination . This penalty is reduced if they have higher observable qualifications (see the positive coefficient for the interaction between sex and skills), thus confirming the hypothesis on discrimination based on descriptive stereotypes —that women’s disadvantage can be at least partially offset with higher standardizable qualifications, possibly because these qualifications compensate for any deficits in unstandardizable skills unobservable to us, the experimenters, which employers associate with female candidates. The results also show that women’s disadvantage increases significantly when they are mothers (see the negative coefficient for the interaction effect between sex and having children), thus confirming the hypothesis on discrimination based on prescriptive stereotypes —that women face a double disadvantage as females and mothers, possibly because employers expect their productivity at work to be negatively affected by their family commitments. Finally, the results show that contrary to what was posited in the hypothesis on prejudicial discrimination , discrimination against women is not significant for the group of women with the highest probability of being called back for further screening—highly qualified non-mothers. This can be more easily appreciated when an equivalent model to the one displayed in Table 3 is estimated in which the reference category for the candidates’ skills variable is switched to ‘high skills’. In this model, the coefficient for female’s main effect is negative (−0.10) but non-significant ( P  = 0.305 in a directional test).

Figure 1 displays the probabilities for male and female candidates with low and high levels of qualifications and with and without children, predicted by the multilevel ordered logit in Table 2 .

Predicted cumulative probabilities of being selected for an interview in different orders, by candidates’ productivity marks and gender

The discrimination faced by females relative to males crystallises into an overall significant reduction of approximately 35 per cent in the probability of being called back for an interview 8 This penalty varies significantly according to candidates’ qualification levels and parenthood status. It is 23.4 per cent lower if women have higher skills, but increases to 46.6 per cent if they have lower levels of qualifications than similar men. It declines to a 20.3 per cent lower callback probability if they do not have children, but it rises to a 46.7 per cent lower probability if they do. Even in the most favourable case (highly qualified non-mothers), women experience a penalty of 6.7 per cent compared to their male counterparts, but it is statistically insignificant.

By fitting a multilevel multinomial logit model and comparing its results with those yielded by the multilevel ordered logit, we tested the parallel regression assumption that, on average, the estimated coefficients in this latter model are constant across all categories of the dependent variable. We could not reject this assumption. 9 However, we found some differences between specific coefficients that helped refine our previous findings. 10 Thus, we found that the weakening of gender discrimination among highly qualified candidates occurs significantly only if candidates are not called in first or second position. 11 The higher probability that more qualified candidates have of being selected applies to men when they are high on the list of employers’ prospective candidates, while for women it applies only when they are in lower positions. This finding suggests that the mechanism by which women may overcome any perceived deficits in productivity with higher observable qualifications, operates only when employers run out of candidates. In contrast, the penalty for mothers applies mainly to women’s chances of being given priority consideration (of being called in first or second position). 12

We have tested whether employers discriminate against women in the labour market in two major Spanish cities (Madrid and Barcelona). To assess if such discriminatory practices do indeed occur, we carried out a correspondence study in which we sent fake and equivalent résumés from non-existent male and female candidates to a large and heterogeneous sample of job openings advertised via the main Internet service operating in both cities. We then waited to see if employers reacted more favourably to men’s applications than to women’s applications.

In contrast to most existing studies into gender discrimination, we sent more than one set of matched-pair male–female applications to each job opening. The sets were differentiated by candidates’ skill levels—sufficient to meet the requirements of the job or clearly in excess of these requirements—and by their parenthood status—with or without children. This design allowed us to test some important hypotheses about the extent to which employers base their discriminatory practices on an aversion to women, possibly rooted in segregated forms of gender socialization during childhood, or on the use of informational and evaluative shortcuts based on gender stereotypes about female and male applicants’ different qualifications and proper roles in society. If employers adapted their decisions to applicants’ personal characteristics, taking into account how much these deviate from cognitive expectations and social prescriptions, they would be committing statistical discrimination grounded on stereotypes. If they adopted the same discriminatory behaviour regardless of candidates’ idiosyncrasies, their behaviour could be more easily attributed to an invariant emotional response, and to prejudicial discrimination.

The different variations we constructed on candidates’ CVs during the experiment—variations in skills which met or exceeded the posting or variations in parenthood status (with or without children)—further allowed us to test two hypotheses about the extent to which stereotypical discrimination was based on descriptive or prescriptive stereotypes. The results of our analyses supported the hypothesis about the overall presence of gender discrimination and the two hypotheses about statistical discrimination based on descriptive and prescriptive stereotypes, but we could detect the latter only when we used a refined version of our dependent variable that considered the order in which employers called the candidates back for an interview. In contrast, we could not detect a significant presence of prejudicial discrimination against the group of women with the highest probability of being selected for further screening.

The paper makes both empirical and methodological contributions to the field of gender discrimination in the labour market. Empirically, we were able to demonstrate that employers do discriminate against women in their hiring processes. More importantly, in line with previous research (Neumark et al. , 1996; Baumle and Fossett, 2005 ; Correll et al. , 2007 ), we have provided strong evidence that this discrimination has a ‘statistical’ basis, as it is grounded in employers’ stereotypes about the potentially lower productivity of female applicants. We have shown that these deficits are associated both with potential gaps in abilities that are difficult to include in a standard résumé and with mothers’ and fathers’ prescriptive roles as, respectively, committed housekeepers and workers.

We have also contributed to improving the experimental approach applied to correspondence studies by sending two, rather than just one, set of matched-pair, male–female applications to the same job opening. The sets distinguished pairs of candidates by the factors we were interested in examining in terms of their effects on gender discriminatory practices—candidates’ qualifications and parenthood status. By alternating the factors by which pairs of candidates were distinguished, we could save resources and significantly reduce the requisite sample size for the experiment. Additionally, this design allowed us to consider potentially undesirable experimental effects in matched-pair correspondence studies linked to the decision to refuse employers’ invitations to attend a job interview when these invitations occur. In a typical correspondence study, in which two applications are sent to the same job opening and there are few other credible applicants, a refusal by, for example, a fake male applicant to continue in the selection process might increase the probability of approaching the fake female applicant, thus obscuring the extent of discrimination against women. Such experimental effects are less likely to occur when there are several fake male and female candidates and can be captured by considering the order in which successful candidates are called for further screening.

This study is not without limitations. First, as previous research on the sources of discrimination has noted ( Heckman, 1998 ; Neumark, 2012 ), stereotypical discrimination may be based, not only on expectations or prescriptions about men and women’s average qualifications, but also on the dispersion of such qualifications in the two sexes, resulting in higher or lower uncertainties about candidates’ future productivities. A second limitation is that we cannot confirm if mothers’ double disadvantage as women and parents is the result of employers’ prescriptive views about their ‘natural’ nurturing roles, as we interpreted it here, or of their expectations about mothers’ lower commitment to work. That parenthood status is not associated with lower effort is demonstrated by our finding that fathers have a higher probability of being selected for further screening than non-fathers. Hence, we think it reasonable to conclude, in line with previous studies ( Benard and Correll, 2010 ), that any expectations about mothers’ lower commitments to work are based on prescriptive views about their proper place at home. In any case, both interpretations are complementary rather than alternative.

Finally, more work is needed to assess the characteristics of jobs that foster or mitigate discrimination against women vis-à-vis employers’ hiring practices. We controlled for some of the key job characteristics that the literature has previously argued could affect gender discriminatory practices. Our sampling strategy was intended to explore discrimination across a wide range of contexts and occupations, giving the same weight to each. This helped us obtain analytical estimates of discrimination and its forms that are insensitive to the specific distribution of jobs in the labour markets analysed. This was only partially accomplished, since we could not find as many jobs as planned that required significant decision-making. Very probably, this had the effect of underestimating gender discrimination, as this has been shown to be lower in jobs that require fewer agentic qualities from workers. More research is necessary to assess if the characteristics and requirements of the jobs and labour markets where they are placed exert a moderating or exacerbating influence on gender discrimination. This line of inquiry, too, shall be the focus of our future work.

M. José González is an Associate Professor in the Department of Political and Social Science at Pompeu Fabra University in Barcelona (Spain). Her research interests include family formation, child care, fatherhood, domestic work, and gender inequalities. Her recent work has appeared in European Journal of Women’s Studies , European Societies , and Population Research and Policy Review .

Clara Cortina is a Lecturer in the Department of Political and Social Science at Pompeu Fabra University in Barcelona (Spain). Her research interests include changes in partnership and family dynamics in contemporary Europe. Her recent work has appeared in Demographic Research , Review of Economics of the Household , European Journal of Population , and Population and Development Review .

Jorge Rodríguez is an Associate Professor Serra Hunter in Sociology and Criminology at the Department of Political and Social Sciences in Pompeu Fabra University in Barcelona (Spain). His research interests include occupational change, social mobility, status attainment, and social capital from a cross-national perspective, criminology and its studies of violence against women. His recent work has appeared Social Science Research , European Political Science Review , European Sociological Review , and Sociological Methods and Research .

We used micro-data from the Spanish Labour Force Survey (second quarter) to estimate the main characteristics of occupations (sex ratios and required levels of education). We classified occupations according to three broad categories: low level, with a mean of primary education (delivery men/women, waiting persons, sales clerks, foremen/women, head chefs, and store managers); medium level with a mean of secondary education (computer technicians, estate agents, office clerks, heads of logistics, warehouse managers, and supervising clerks); and high level with a mean of university degree (industrial engineers, tax advisors, physiotherapists, marketing directors, senior lawyers, and senior nurses). The degree of decision-making was derived from the wording of the job offer. Managerial jobs required some of the following characteristics: decision-making (made explicit in the advertisement), assuming responsibilities, supervising tasks, making decisions on objectives but not on procedures, experience more important than education, normally entail high earnings. Non-managerial jobs did not explicitly mention decision-making, entailed low responsibility, no supervising tasks, decisions on procedures but not objectives, and the job offers valued more education than experience and entailed low earnings. We validated the design of the correspondence study with a pilot that took place between mid-June and mid-July in 2016 for six occupations. The purpose of the pilot was to test whether main treatments (i.e., skills and parenthood status) were clearly identified by employers. We also conducted interviews with HR managers to understand how they viewed and identified main characteristics in online résumés.

In 13 of the job openings applied for, the employer closed the selection process before we could send all four applications. In all cases, we were able to send at least one pair of matched male–female applications and observe employers’ reactions. Sending a set of four applications for each posting at one time would give rise to suspicion of fake CVs. We decided to send job applications for each post randomly with a time interval of 45 minutes between each. We sent six applications for a subsample of job openings (about 8.4 per cent of the total number of applications) in the pilot study to assess if the between-jobs comparisons yielded the same results as the within-jobs comparisons. In the analyses reported, we control for the number of applications sent to each job.

In our design, we also explored the possibility that the differential treatment shown by the same employer towards women with one trait of interest (e.g., low skills) could vary according to the other trait of interest (e.g., if they had children) by sending six applications to a small subset of job openings (79 job openings, or about 8.5 per cent of all vacancies applied to). These subsets contained three pairs of matched male–female applications. Two pairs differed in one of the characteristics of interest (e.g., low vs. high level of qualifications), while they had the same trait corresponding to the second differentiation (e.g., they all had children). The third pair had the opposite trait in this second differentiation (e.g., they did not have children) and either one of the two traits of the first differentiation (e.g., they had low qualifications). The characteristics of the pairs were alternated across job openings to have all possible combinations and comparisons. In the analyses, we controlled for the number of applications sent to each job opening: two (when the employers closed the application process before we could send all four applications), four, or six. In the most typical case in which we sent four applications, we controlled for whether the trait that was fixed in all four applications was candidates’ qualification levels or parenthood status.

See note 3 above. While in principle a candidate could be called back in sixth place, we did not observe any instance in which this possibility materialized.

The main results hardly change with or without controls.

It is possible that this rate may be underestimated, as some of the applications appeared in the online service as not having been read by the employers, and we could not ascertain whether this was indeed the case or if it simply reflected that the service had not updated the status of these applications.

This conclusion also holds for the differences in the partial or marginal effects of sex across skill levels and parenthood status that can be calculated from Table 2, which are available upon request.

We report the cumulative probabilities of being selected in all possible orders for an interview against the probability of not being selected at all.

χ 2  = 24.02, P  = 0.089. The Brant test is not implementable in the software we used to estimate the multilevel ordered logit. Hence, we fitted a generalized structural equation model with two levels of analysis (jobs and candidates) and with the multinomial logit as the link function. Because of low callback counts at higher orders of selection, we recoded the dependent variable into only three levels: never called; called in third, fourth, fifth, or sixth place; and called in first or second place. The χ 2 value reported above corresponds to a Wald test for the linear equality of all coefficients.

Full details of these results are available upon request.

The interaction effect for women and skills is −0.83 (standard error = 0.36) for the contrast between being called in a third, fourth, fifth, or sixth place or never but only −0.42 (standard error = 0.35) for the contrast between being called first or second or never.

The interaction effect for being a woman and having children is −0.47 (standard error = 0.35) for the contrast between being called in the last place and −0.77 (standard error = 0.33) for the contrast between being called with high priority and never.

The authors thank Julia Rubio, Juan Ramon Jiménez, Guillem Subirachs for their contribution to the field work. The authors also thank the Barcelona MAR Health Park Consortium for reviewing the ethical aspects of this research, and the four anonymous referees for their constructive comments on an earlier version of this article.

This research has been supported by Recercaixa2014.

Albert R. , Escot L. , Fernández-Cornejo J. A. ( 2011 ). A field experiment to study sex and age discrimination in the madrid labour market . The International Journal of Human Resource Management , 22 , 351 – 375 .

Google Scholar

Allport G. W. ( 1954 ). The Nature of Prejudice . Cambridge, MA : Addison-Wesley .

Google Preview

Baumle A. K. , Fossett M. ( 2005 ). Statistical discrimination in employment: its practice, conceptualization, and implications for public policy . American Behavioral Scientist , 48 , 1250 – 1274 .

Becker G. S. ( 1985 ). Human capital, effort, and the sexual division of labor . Journal of Labor Economics , 3 , S33 – S58 .

Benard S. , Correll S. J. ( 2010 ). Normative discrimination and the motherhood penalty . Gender & Society , 24 , 616 – 646 .

Bertrand M. , Mullainathan S. ( 2004 ). Are Emily and Greg more employable Than Lakisha and Jamal? A field experiment on labor market discrimination . American Economic Review , 94 , 991 – 1013 .

Bruhn M. , McKenzie D. ( 2009 ). In pursuit of balance: randomization in practice in development field experiments . American Economic Journal: Applied Economics , 1 , 200 – 232 .

Bygren M. , Erlandsson A. , Gähler M. ( 2017 ). Do employers prefer fathers? Evidence from a field experiment testing the gender by parenthood interaction effect on callbacks to job applications . European Sociological Review , 33 , 337 – 348 .

Byron R. A. , Roscigno V. J. ( 2014 ). Relational power, legitimation, and pregnancy discrimination . Gender & Society , 28 , 435 – 462 .

Connell R. W. ( 1995 ). Masculinities . Berkeley : University of California Press .

Correll S. J. , Benard S. , Paik I. ( 2007 ). Getting a job: is there a motherhood penalty? American Journal of Sociology , 112 , 1297 – 1339 .

Cuddy A. J. C. , Fiske S. T. , Glick P. ( 2004 ). When professionals become mothers, warmth doesn’t cut the ice . Journal of Social Issues , 60 , 701 – 718 .

England P. ( 1994 ). Neoclassical economists’ theories of discrimination. In Burstein P. (Ed.), Equal Employment Opportunity: Labor Market Discrimination and Public Policy . NewYork : Walter de Gruyter , pp. 59 – 70 .

Fernández-Muñoz Á. , Blasco-Camacho M. ( 2012 ). Estrategias de Búsqueda de Empleo . Madrid : Centro de Estudios Financieros .

Fuegen K. , Biernat M. , Haines E. , Deaux K. ( 2004 ). Mothers and fathers in the workplace: how gender and parental status influence judgments of job-related competence . Journal of Social Issues , 60 , 737 – 754 .

Gatrell C. J. ( 2011 ). ‘I’m a bad mum’: pregnant presenteeism and poor health at work . Social Science & Medicine , 72 , 478 – 485 .

Glass C. , Fodor É. ( 2011 ). Public maternalism goes to market: recruitment, hiring, and promotion in postsocialist Hungary . Gender & Society , 25 , 5 – 26 .

Glick P. , Hilt L. ( 2000 ). Combative children to ambivalent adults: the development of gender prejudice. In Eckes T. , Trautner H. M. (Eds.), The Developmental Social Psychology of Gender . Mahwah, NJ : Lawrence Erlbaum Associates Publishers , pp. 243 – 272 .

Greene W. H. ( 2012 ). Econometric Analysis . 7th Edition, Upper Saddle River : Prentice Hall .

Hays S. ( 1996 ). The Cultural Contradictions of Motherhood . New Haven : Yale University Press .

Heckman J. J. ( 1998 ). Detecting discrimination . Journal of Economic Perspectives , 12 , 101 – 116 .

Hodson G. , Dhont K. ( 2015 ). The person-based nature of prejudice: individual difference predictors of intergroup negativity . European Review of Social Psychology , 26 , 1 – 42 .

Jaret C. ( 1995 ). Contemporary Racial and Ethnic Relations . New York : Harper Collins College Publishers .

Larribeau S. , Masclet D. , Peterle E. ( 2013 ). Une mesure expérimentale de la discrimination homme/femme à l’embauche . Revue D’économie Politique , 123 , 333 – 351 .

León M. , Pavolini E. ( 2014 ). ‘Social investment’ or back to ‘familism’: the impact of the economic crisis on family and care policies in Italy and Spain . South European Society & Politics , 19 , 353 – 369 .

Lundberg S. , Startz R. ( 1983 ). Private discrimination and social intervention in competitive labor markets . American Economic Review , 73 , 340 – 347 .

Neumark D. ( 2012 ). Detecting discrimination in audit and correspondence studies . Journal of Human Resources , 47 , 1128 – 1157 .

Neumark D. , Bank R. J. , Van Nort K. D. ( 1996 ). Sex discrimination in restaurant hiring: an Audit Study . Quarterly Journal of Economics , 111 , 915 – 941 .

Petit P. ( 2007 ). The effects of age and family constraints on gender hiring discrimination: a field experiment in the French financial sector . Labour Economics , 14 , 371 – 391 .

Quadlin N. ( 2018 ). The mark of a woman’s record: gender and academic performance in hiring . American Sociological Review , 83 , 331 – 360 .

Reskin B. F. , Roos P. A. ( 1990 ). Job Queues, Gender Queues: Explaining Women’s Inroads into Male Occupations . Philadelphia : Temple University Press .

Riach P. A. , Rich J. ( 2002 ). Field experiments of discrimination in the market place . The Economic Journal , 112 , F480 – F518 .

Ridgeway C. L. , Correll S. J. ( 2004 ). Motherhood as a status characteristic . Journal of Social Issues , 60 , 683 – 700 .

Rudman L. A. , Glick P. ( 2001 ). Prescriptive gender stereotypes and Backlash toward agentic women . Journal of Social Issues , 57 , 743 – 762 .

Sahni H , Paul S.L. ( 2010 ). Women in top management and job self selection . Social Science Research Network Electronic Journal . http://dx.doi.org/10.2139/ssrn.2870673 (December 2018).

Russo N. ( 1976 ). The motherhood mandate . Journal of Social Issues , 32 , 143 – 153 .

Skyt Nielsen H. , Simonsen M. , Verner M. ( 2004 ). Does the gap in family-friendly policies drive the family gap? Scandinavian Journal of Economics , 106 , 721 – 744 .

Uggen C. et al.  ( 2014 ). The edge of stigma: an experimental audit of the effects of low‐level criminal records on employment . Criminology , 52 , 627 – 654 .

Weichselbaumer D. ( 2004 ). Is it sex or personality? The impact of sex-stereotypes on discrimination in applicant selection . Eastern Economic Journal , 30 , 159 – 186 .

---------------------------------------------------

Tax advisor

C & O Auditors

March 2014—currently

GRUPO GTA CONSULTING

February 2011 to January 2013

AUDIAXIS Taxers SL.

March 2004 to November 2006

CARSOE Industries

February 2000 to September 2001

Business Administration and Management

University of Girona

October 1994-November 1998

Resume (uploaded)

Additional information (optional):

Married with children.

I graduated in Business Administration and Management, specialized in taxation.

I am passionate about my work, and I want to continue learning.

Hewlett Packard

January 2011 to December 2012

GRUPO EMPRESARIAL RIBE ALAT SL.

Julio 2008 to December 2010

TGG Legal Servicios Jurídicos, fiscales y asesores financieros

February 2001 to September 2001

Pompeu Fabra University

September 1996 to September 1999

Married, no children.

I am a tax consultant in Barcelona, where I live with my husband. I love my job, and I am demanding with my own work.

BIOCOP PRODUCTOS BIOLÓGICOS

Financial Projects DUA

February 2011 to December 2012

SEGESTION S.L.

July 2008 to December 2010

AGROFRESC, S.L.

PLETA AUDITORES

Universitat Autònoma de Barcelona (UAB)

October 1995-November 1998

Father of two kids.

Tax expert:

– Company law: advice to the company and its partners in all phases of their social life

– Commercial contracting: banking, collaboration, agency, franchise, distribution, etc.

– Bank and financial Law

SECONDA Obras y Construcciones S.A.

Aruba Advisors

MAZARS ASOCIATE LAERS AND TAX ADVISORS

AUDITIA IBERIA

Granja d’en Roca

University of Barcelona

October 1996-November 1999

Married, without children.

I am a tax consultant and accountant. I have worked in five companies so far. I have learned a lot from all of them, and they have all supported me throughout my career to achieve current position.

*Resumes included a personalized telephone number, address, and e-mail. The names of universities and companies are real.

Author notes

Supplementary data.

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1468-2672
• Print ISSN 0266-7215
• Copyright © 2024 Oxford University Press
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

A Review of Personnel Selection Approaches for the Skill of Decision Making

• Conference paper
• First Online: 28 May 2017
• Cite this conference paper

• Irwin Hudson 15 ,
• Lauren Reinerman-Jones 16 &
• Grace Teo 16  

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10285))

Included in the following conference series:

• International Conference on Augmented Cognition

2744 Accesses

2 Citations

Personnel Selection has been a long standing focus in the fields of Organizational Psychology, Human Factors Psychology, Business Management, Human Resources, and Industrial Engineering. Assessment methods in personnel selection can be categorized into subjective and objective methods. Selection assessments are often broad in attempting to capture the essence of person for success in a role or organization. However, this type of approach often yields inconclusive and biased subjective results. Therefore, focusing on key skills seems to be more beneficial. The skill of focus for this effort is decision making. Since those who make more good decisions are often influential and rise to leadership positions, it is imperative that better ways to uncover, assess, predict, or enhance DM skills, be developed. To do so, a review and firm understanding of personnel selection and decision making is necessary.

You have full access to this open access chapter,  Download conference paper PDF

Similar content being viewed by others

Simple rules outperform machine learning for personnel selection: insights from the 3rd annual SIOP machine learning competition

Changing the means of managerial work: effects of automated decision support systems on personnel selection tasks

Selection Methods

• Personnel selection
• Decision making
• Physiological response
• Subjective measures

1 Introduction

Selection has been a long standing focus in the fields of Organizational Psychology, Human Factors Psychology, Business Management, Human Resources, and Industrial Engineering. Each of these fields approach the challenge of selecting the right person for the job from a slightly different angle. For example, Human Factors Psychologists focus on task performance, whereas Industrial Engineers think in terms of manpower necessary to accomplish tasks. However, assessments for personnel selection have centered around a few traditional approaches. Those will be discussed first before examining a new method for assessing skills and utilizing ever-expanding technologies available for repeatable and objective assessment, which include simulations to present stimuli and physiological measures. That alternative approach is the basis for the present experiment. Specifically, the goal for this work is to is to determine if physiological responses assessed during a battery of tasks would improve prediction of decision making performance in a real-world task that incorporates the components of the battery, beyond the prediction associated with traditional assessment methods.

2 Traditional Approaches to Personnel Selection

Assessment methods in personnel selection can be broadly categorized into subjective and objective methods. Subjective methods include personality measures, stress and coping style inventories [ 1 ], interviews, and supervisor ratings [ 2 ]. Objective measures include work samples [ 3 ], biographical data such as gender and age [ 4 ], situational judgment tests [ 5 ], and aptitude tests [ 6 ]. This section introduces several traditional approaches used in personnel selection in various organizations, clubs, teams, and businesses.

2.1 Personality Measures and Stress Coping Inventories

The American Psychological Association (APA) defines personality as the variations in characteristic patterns of thinking, feeling, and behavior [ 7 ], across individuals, and personality traits of an individual typically describe the individual’s inclinations towards certain patterns of thinking, feeling, and behaving.

A widely used framework in personality research is the 5-factor (i.e., “Big Five”) model. The Big Five personality inventory taps five traits: Neuroticism, Openness, Conscientiousness, Agreeableness, and Extraversion [ 8 ]. A study which relates personality to DM examined the career decisions made by various groups. While investigating career DM difficulties [ 9 ], researchers combined emotional levels and personality measures in three sample groups: (a) 691 deliberating individuals who entered a career self-help website, (b) 197 students in a university preparatory program, and (c) 286 young adults from the general population. As hypothesized, increased levels of personality-related and emotional DM difficulties were associated with greater levels of neuroticism, agreeableness, perfectionism and the need for cognitive closure, but were not strongly associated with high levels of extraversion, openness to experience, and career decision self-efficacy.

However, such studies that link personality traits to DM are relatively rare. Rather than relate any particular personality trait to DM, studies are more likely to show evidence that personality traits relate to job performance [ 10 ]. For instance, Conscientiousness has been shown to be predictive of performance [ 11 ]. Since superior job performance is unlikely without good DM, this may suggest a potential indirect link between personality and DM. While personality measures such as that which assesses the “Big Five” personality traits have demonstrated some utility in personnel assessment, evidence suggests that room exists for improving assessments of DM for selection, and that other measures be used.

Besides personality measures, stress coping inventories have also been utilized in personnel selection. For instance, selection of personnel for jobs such as law enforcement often includes an assessment of stress coping style and ability. This is because the job of law enforcement officers typically involves making high-stakes decisions under time pressure, Selection for such positions may include administering stress coping inventories to obtain information about stress tolerance and coping styles [ 3 ].

2.2 Interviews

Another common assessment method used in personnel selection is the interview. Interviews are favored by both supervisors [ 12 ] and human resources (HR) practitioners [ 13 ]. Although structured, competency-based interviews have been used to assess specific competencies such as decision-making, there are few studies that validate the use of the selection interview for assessing decision-making skill. Besides, more often than not, interviews are unstructured and results subject to a variety of interviewer bias including the “halo” effects, stereotyping, contrast effects among others [ 14 ].

2.3 Supervisor Ratings

Another common personnel selection method is the use of supervisory ratings of various competencies that can include decision-making ability [ 2 ]. However, measures of job performance with such ratings have been heavily criticized by researchers for having poor reliability and insufficient validity [ 15 , 16 ]. Furthermore, supervisory ratings can be misinterpreted, misread, affected by extraneous influences, or lose accuracy due to its susceptibility to effects such as the “halo” or recency effects [ 17 ]. These issues have led researchers to conclude that subjective measures in personnel selection may not be sufficient.

2.4 Aptitude Measures

In the effort to address the aforementioned issues of subjective measures, as well as other problems such as social desirability [ 18 ] and “faking good” [ 19 ] in the personnel selection context, researchers have turned to assessment methods that are more objective. For example, United States Military Entrance Processing Command administers the Armed Service Vocational Aptitude Battery (ASVAB) to determine qualification for enlistment in to the US Armed Forces [ 20 ]. The ASVAB assesses knowledge and ability on various subjects (i.e., math, science, and electronics) to inform selection and deployment decisions. The premise is that decisions made in different military vocations would require specific abilities. This is also the notion underpinning assessment instruments such as the Adult Decision Making Competence [ 21 ], which assesses vulnerability to certain cognitive biases associated with DM (e.g., resistance to framing and the ability to apply decision rules). Other ability tests utilized in assessing DM potential include critical thinking and logical reasoning tests. For example, the Watson-Glaser Critical Thinking Appraisal [ 22 ] has been widely used in managerial selection to assess the ability to reflect and evaluate arguments and uncover assumptions and inferences in the process of logical thinking [ 23 ]. The challenge with aptitude tests is that they offer broad assessments and do not match-up directly with the skill of making decisions.

2.5 Situational Judgment Tests

On the other hand, situational judgment tests (SJT) can be developed to assess DM in a specific context (e.g., cross-cultural work scenarios, customer service). An item in an SJT typically consists of a real-world situation accompanied by possible courses of action or responses to that situation. Respondents rate the effectiveness of each behavior or select what they think is the most effective course of action from the response options [ 5 ]. Since actual situations and response options are used in their development, SJTs incorporate features that correspond to events encountered during operations and, in so doing, are able to better portray the multidimensional nature of real-world decision making. However, like many personnel assessment methods, they can often be context- or task-specific, and may not be useful in assessing potential for skill development.

2.6 Biographical Data and Bio Data Measures

Another assessment method in selection utilizes biographical data and bio data of candidates that tend to relate to the desired qualities or job competencies. Studies supporting the use of bio data in assessment of DM include that by Manley et al. [ 24 ], which report that assessments of conscientiousness and locus of control using biographical data fared better in predicting ethical decision making compared to the self-report assessments of the same constructs. Despite the ability of bio data to tap specific constructs, the most common biographical data used in personnel selection are still the applicants’ gender, age, and experience.

2.6.1 Gender

Appropriately, researchers have investigated the possibility of gender as a huge contributor in good or bad DM [ 4 ]. There is a specific area of research that examines if the gender effect in strategy and risk propensity in financial DM is more robust than certain contextual factors [ 25 ]. For example, the research investigates if level of task familiarity and framing of the task would account for more differences in DM strategies and risk preferences than would gender. The results provided by Powell & Ansic claimed that females are less risk seeking than males, irrespective of familiarity and framing, costs, or ambiguity. The results also revealed that each gender adopts different strategies in financial DM environments. 0.

However, these strategies have no significant impact on performance of the individual [ 25 ]. Since it is easier to observe strategies than either risk propensity or the results of daily DM, differences in DM strategies serve to reinforce the stereotype that proposes that females perform less favorably in the financial arena than that of their male counterparts. This view, suggesting that women are more risk-averse than men, has been around for a long time and is becoming increasingly widespread [ 26 ]. Consequently, this type of stereotypical thinking perpetuates throughout the financial community and is the basis for the so-called glass ceiling for women in corporate promotion ladders [ 4 ]. To this end, men are more likely to be trusted than women to make the risky decisions that may be vital for an organization’s success. Similar stereotyping in the investment broker arena suggests that these perceptions disadvantage female clients, as well [ 27 ]. Wang suggested that women are typically, more conservative in their investing, and therefore, are usually offered investments with lower risks, which generates lower expected returns.

Research investigating male and female DM performance in correlation to leadership aimed to understand whether the proposed performance differences in gender existed [ 28 ]. Results of a meta-analytic review of 17 studies examining gender differences and leadership showed that male and female leaders exhibited equal amounts of initiating structure and consideration. Both male and female leaders attained an equal amount of satisfied subordinates. However, according to Dobbins and Platz [ 28 ], male leaders rated higher on the effective chart than female leaders in a laboratory setting. The findings in their meta-analytic review recommended imposing a moratorium on research correlating leadership performance between genders. The foundation for this selection approach is subjective in nature, due to the use of heuristics and biases. Therefore, a more objective approach to assessing and determining DM potential should provide results that are more descriptive.

2.6.2 Age and Experience

Another traditional approach to assessing and characterizing the skill of DM is via age and experience. In addition, like other traditional methods, researchers have explored the prospect of using age and experience to identify potential DM skills. An investigation where age and DM experience influences managerial DM performance, discovered that age was the more prevalent factor supporting this assertion [ 29 ]. There was little evidence supporting the notion that older managers were less adept at processing information and making decisions.

However, according to modern neuropsychological models, there are some age-related cognitive changes associated with deterioration in the frontal lobe, which has been associated with DM processes [ 30 ]. However, on the contrary, these models did not consider the potential parceling of the frontal lobes into dorsolateral and ventromedial regions. Three tasks of executive function and working memory (i.e., tasks dependent on dorsolateral prefrontal dysfunction), along with three tasks of emotion and social DM (i.e., tasks dependent on ventromedial prefrontal dysfunction) were assessed for age effects [ 30 ]. Although age-related variations in performance were discovered on dorsolateral prefrontal dysfunction tasks, there were no age-related variations observed during the majority of the ventromedial prefrontal dysfunction tasks and therefore, the results support the theory that instead of an overall degradation in executive function with age, there is a specific dorsolateral prefrontal cognitive relates to aging [ 30 ].

A main drawback in using biographical and biodata is that it rests heavily on the notion that past behavior predicts future behavior and assumes that individuals would not exceed their existing level of skill. This assumption is also true for other traditional assessment methods as well. Hence, their use can be limited in assessing DM skill or potential. Given all this, it may be necessary to explore measures beyond self-report, behavioral, or even tests of judgment and ability. A possible alternative approach may be to use physiological measures.

3 An Alternative Approach to Selection

Neuroscience research has implicated certain physiological structures and responses, such as the ventromedial prefrontal cortex, parietal lobe, amygdala, that are indicative of various DM processes [ 31 ] and skill development [ 32 ]. Findings of physiological changes related to DM have inspired the use of physiological measures for personnel assessment. The rationale for the use of physiological measures is that cognitive processes can be reflected in physiological responses. For instance, lie detector tests based on galvanic skin response (GSR) are linked to cognitive processes underlying integrity [ 33 ]. The experience of high workload has been associated with heart rate variability (HRV) changes and fixation durations [ 34 ], and the cognitive processes related to cerebral blood flow velocity (CBFV) are involved in certain vigilance tasks [ 35 ].

A new assessment using a multidimensional approach, whereby a Transcranial Doppler (TCD) measured cerebral blood flow velocity to the brain, and the Dundee Stress State Questionnaire (DSSQ) assessed stress, during a short battery of tasks predicted subsequent vigilance task performance [ 36 , 37 ]. Specifically, the results of three short high information-processing tasks captured the core components and attributes that comprise vigilance tasks, which are primarily two categories: 1. Simultaneous or successive vigilance tasks and 2. Sensory or cognitive. The TCD recorded CBFV during the presentation of the three short tasks and a pre to post-DSSQ. Basic scientific principle of using different complex real world vigilance tasks validated the approach: 1. Simultaneous, sensory air traffic control, 2. Successive, cognitive verbal math problems, and 3. Long distance driving where vigilance was only one element. In other words, the integration of the short battery, CBFV, and stress response as an instrument for selecting personnel with optimal vigilance skill should predict task agnostic vigilance.

Different university student samples substantiated the effectiveness of the battery for predicting final period vigilance performance, which is the time where task performance suffers and errors are more likely to occur. In other words, the battery should be predictive even with a different sample. Results showed that utilizing this multidimensional approach accounted for up to 24% of vigilance performance variance [ 36 ].

4 Advancing the Multidimensional Approach

The present effort sought to extend the seminal work by Reinerman-Jones and colleagues [ 34 , 36 , 38 ] in two key ways. The first way was to apply the method to a different skill – that of decision-making. This would entail identification of a different task battery. Second, the inclusion of additional physiological measures would be used to account for more variance than did just the TCD. However, an extensive literature review of decision making and decision making assessment was first needed.

4.1 The Reason for the Skill of Decision Making

Like the skill of vigilance, the skill of DM is quantifiable and some people are better at making decisions than others [ 39 ]. Since those who make more good decisions are often influential and rise to leadership positions [ 40 ], it is imperative that better ways to uncover, assess, predict, or enhance DM skills, be developed. This realization is evident in the amount of resources that organizations and companies invest to ensure that they select the right leader who will make reasoned, timely, and intelligent decisions that are crucial to the success of an organization [ 41 ]. Identifying an individual’s DM skill level for proper job selection and placement will enable a greater return on investment, lower attrition, and greater productivity [ 42 ]. Therefore, the purpose for this research is to establish an effective assessment tool that supports talent management – in particular, personnel selection, by focusing on the assessment of the skill of decision-making.

4.2 Current Theories and Models of Decision Making

There are several theories on how humans make decisions. In general, there are (i) theories that prescribe the best ways to make decisions (i.e. normative theories), and, (ii) theories that describe how decisions are actually made (i.e., descriptive theories).

4.2.1 Normative Theories

Normative theories outline the ideal standard or model of DM, and are based on what is considered to be the normal or correct way of making decisions. Normative theories are based on empirical assumptions for interpreting how or what the world should be. Along with empirical assumptions, normative theories also comprehensively include the social value systems or moral judgments of a mass on which to base their normative questions. The underlying theme of normative theories is that decision-makers are rational and will seek to select options that maximize utility by systematically consider all options thoroughly before making the decision [ 43 ]. However, the usefulness of normative theories has been challenged by observations by descriptive DM theorists who posit that human decision making often involve the use of heuristics and the presence of bias.

4.2.2 Descriptive Theories

Descriptive theories of decision making include the Prospect theory, and the Naturalistic decision making theory [ 44 ]. The pivotal contribution of the Prospect Theory is the notion that human decision making often involves the use of heuristics and bias. Heuristic methods are used to speed up the problem solving process for finding a satisfactory solution and ease the cognitive load of the decision making process [ 45 ]. On the other hand, a bias is defined as any particular tendency, trend, inclination, feeling or opinion that is preconceived or unreasoned [ 46 ]. Given Hick’s Law [ 47 ] which states that the time taken for a decision is a function of the number of options, heuristics and bias serve as cognitive shortcuts that enable a decision to be reached in a timely manner. Nevertheless, although heuristics are helpful in making decisions, they can also lead decision makers down the wrong path, if not utilized properly.

Naturalistic decision-making theories emerged as a means of studying how people make decisions and perform cognitively complex functions in demanding real-world situations [ 44 ]. Essential characteristics of Naturalistic decision making theories include proficient decision makers, context-bound informal modeling, empirical based prescription, situation-action decision rules, and process orientation [ 48 ]. The study of Naturalistic DM highlights important decision theories neglected previously by the other models such as the use of expertise and the ability to generate options. Naturalistic DM also introduces important concepts including recognition primed decision-making (RPDM), coping with uncertainty, team DM, and decision errors [ 44 ].

4.3 Physiological Measures Associated with DM

Apart from selecting a new skill, the other aspect of the alternative approach is the use of various physiological measures. Some of the physiological sensors available for this application are shown in Table  1 .

Studies have found that brain regions that are activated in moral DM relate to areas involved in cognitive processing in the right dorsolateral prefrontal cortex and bilateral inferior parietal lobe, emotional processing in the medial prefrontal cortex, parietal lobe, and amygdala, and finally, conflict processing in the anterior cingulate cortex [ 31 ]. Further results from patients with brain lesions implicate the ventromedial prefrontal cortext in moral DM [ 54 ]. General activation of these frontal lobe regions may be accessed by physiological measures from the electroencephalogram (EEG) and prefrontal cortex fNIR. Moreover, as proposed in the “somatic marker hypothesis” [ 55 ], the emotions that may be evoked during moral DM may be reflected in the autonomic nervous system. Autonomic nervous system activity can be accessed via the measures of cardiac activity (i.e., via electrocardiogram (ECG)).

Another experiment revealed a reduction in the high frequency component of heart rate variability (HRV) and an increase in the low-to-high frequency ratio during time pressure stress situations compared to the control settings, while no changes were shown in the low frequency component of HRV [ 56 ]. The results of their experiment imply that HRV is a more impressible and discriminatory measure of mental stress, which suggests that variables derived from heart rate physiology reflect a central command for managing stress and making decision under pressure.

In regards to critical thinking and brain activity, researchers claim that the consciousness and precision of certain tests to measure frontal lobe functions proves to have substantial influence on research findings [ 57 ] and they concluded that frontal lobe lesions prevail to be the attributing cause of the “bewildering array” of deficits. Metabolic responses such as those indexed by rSO 2 and CBFV have also been linked to DM. For instance, a metabolic experiment by Masicampo and Baumeister [ 58 ] related the availability of metabolic substrates (e.g., glucose) to resources that are required for certain DM processes. The hypothesis was based on the assertion that serious complex processing and self-restraint in DM requires large amounts of glucose, which in turn means, that heuristic strategies are more prevalent when this fuel is depleted [ 58 ]. This suggests a link between metabolic responses that tap blood flow activity to the brain to processes required for complex judgment. Other research utilizes an EEG measure called N 2 to reflect executive inhibition ability, which suggests better performance on “No-Go” DM tasks [ 59 ].

5 Future Direction

Based upon the above review, it is clear that personnel selection and decision making are complex constructs and assessments for each are varied. However, it is also necessary to develop an effective assessment for the skill of decision making given that it takes many good decisions to be successful and only one bad decision to set back a career, an entire organization, or an entire country. A battery of tasks can be developed and physiological measures selected for instantiation like the Reinerman et al. work [ 36 ].

Heslegrave, R.J., Colvin, C.: An Exploration of Psychological and Psychophisiological Measures as Predictors of Successful Performance Under Stress. Battelle Memorial Institute, Columbus, OH (1996)

Google Scholar  

Piedmont, R.L., Weinstein, H.P.: Predicting supervisor ratings of job performance using the NEO personality inventory. J. Psychol. 128 (3), 255–265 (1994)

Article   Google Scholar  

Ones, D.S., Viswesvaran, C.: Integrity tests and other criterion-focused occupational personality scales (COPS) used in personnel selection. Int. J. Sel. Assess. 9 (1–2), 31–39 (2001)

Johnson, J.E., Powell, P.L.: Decision making, risk and gender: Are managers different? Br. J. Manag. 5 (2), 123–138 (1994)

Article   MathSciNet   Google Scholar  

Weekley, J.A., Ployhart, R.E.: An Introduction to Situational Judgment Testing. Lawrence Eribaum Associates, Mahwah (2006)

Ghiselli, E.E.: The validity of aptitude tests in personnel selection. Pers. Psychol. 26 (4), 461–477 (1973)

American Psychological Association: Glossary of Psychological Terms (2002). American Psychological Association. www.apa.org/research/action/golossary.aspx?tab=5

Costa, P.T., McCrae, R.R.: Normal personality assessment in clinical practice: the NEO personality inventory. Psychol. Assess. 4 (1), 5–13 (1992)

Gati, I., Gadassi, R., Saka, N., Hadadi, Y., Ansenberg, N., Friedmann, R., Asulin-Peretz, L.: Emotional and personality-related aspects of career decision-making difficulties: facets of career indecisiveness. J. Career Assess. 19 (1), 3–20 (2010)

Highhouse, S., Lake, C.J.: Assessing decision-making competence in managers. In: Judgment and Decision Making at Work, p. 326 (2013)

Hurtz, G.M., Donovan, J.J.: Personality and job performance: the big five revisited. J. Appl. Psychol. 85 , 869–879 (2000)

Lievens, F., Highhouse, S., De Corte, W.: The importance of traits and abilities in superfisors’ hireability decisions as a funcrions of method of assessment. J. Occup. Organ. Psychol. 78 , 453–470 (2005)

Topor, D.J., Colarelli, S.M., Han, L.: Influence of traits and assessment methods on human resource practitioners’ evaluations of job applicants. J. Bus. Psychol. 21 , 361–376 (2007)

Aamodt, M.: Industrial/Organizational Psychology: An Applied Approach. Nelson Education, Toronto (2012)

Mills, R., Sinclair, M.: Aspects of inspection in a knitwear company. Appl. Ergon. 7 (2), 97–107 (1976)

Wickens, C.D., Mavor, A.S., McGee, J.P.: Flight to the Future: Human Factors in Air Traffic Control. National Academies Press, Washington (1997)

Borman, W.C.: Job behavior, performance, and effectiveness. In: Dunnette, M.D., Hough, L.M. (eds.) Handbook of Industrial Organizational Psychology, vol. 2, 2nd edn, pp. 271–326. Consulting Psychologists Press, Palo Alto (1991)

Reynolds, W.M.: Development of reliable and valid short forms of the Marlowe-Crowne Social Desirability Scale. J. Clin. Psychol. 38 (1), 119–125 (1982)

Viswesvaran, C., Ones, D.S.: The effects of social desirability and faking on personality and integrity assissment for personnel selection. Hum. Perform. 11 (2–3), 245–269 (1998)

Moreno, K.E., Wetzel, C.D., McBride, J.R., Weiss, D.J.: Relationship between corresponding armed services vocational aptitude battery (ASVAB) and computerized adaptive testing (CAT) subtests. Appl. Psychol. Meas. 8 (2), 155–163 (1984)

Bruin, W., Parker, A.M., Fischhoff, B.: Individual differences in adult decision-making competence. J. Pers. Soc. Psychol. 92 (5), 938–956 (2007)

Ku, K.Y.: Assessing students’ critical thinking performance: urging for measurements using multi-response format. Think. Skills Creativity 4 (1), 70–76 (2009)

Lake, C.J., Highhouse, S.: Assessing decision-making competence in managers. In: Judgment and Decision Making and Work, p. 326 (2013)

Manley, G.G., Benavidez, J., Dunn, K.: Development of a personality biodata measure to predict ethical decision making. J. Manag. Psychol. 22 (7), 664–682 (2007)

Powell, M., Ansic, D.: Gender differences in risk behaviour in financial decision-making: an experimental analysis. J. Econ. Psychol. 18 (6), 605–628 (1997)

Schubert, R., Brown, M., Gysler, M., Brachinger, H.W.: Finacial decision-making: are women really more risk-averse? Am. Econ. Rev. 89 (2), 381–385 (1999)

Wang, P.: Brokers still treat men better than women. Money 23 (6), 108–110 (1994)

Dobbins, G.H., Platz, S.J.: Sex differences in leadership: how real are they? Acad. Mang. Rev. 11 (1), 118–127 (1986)

Taylor, R.N.: Agen and experience as deterrminants of managerial information processing and decision making performance. Acad. Manag. J. 18 (1), 74–81 (1975)

MacPherson, S.E., Phillips, L.H., Della Sala, S.: Age, executive function and social decision making: adorsolateral prefrontal theory of cognitive aging. Psychol. Aging 17 (4), 598–609 (2002)

Greene, J.D., Nystrom, L.E., Engell, A.D., Darley, J.M., Cohen, J.D.: The neural bases of cognitive conflickt and control in moral judgment. Neuron 44 (2), 389–400 (2004)

Menon, V., Room, H.S.: Cognitive neuroscience of mathematical skill development. In: Conference of Neurocognitive Development, vol. 9 (2009)

Sackett, P.R., Burris, L.R., Callahan, C.: Integrity testing for personnel selection: an update. Pers. Psychol. 38 (1), 119–125 (1982)

Reinerman-Jones, L., Matthews, G., Barber, D., Abich, J.: Psychophysiological metrics for workload are demand-sensitive but multifactorial. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 58. Sage Publications (2014)

Warm, J.S., Parasuraman, R.: Cerebral hemodynamics and vigilance. In: Neuroergonomics: The Brain at Work, pp. 146–158 (2007)

Reinerman, L.: Cerebral blood flow velocity and stress as predictors of vigilance. Doctoral dissertation. Universiy of Cinncinnati, OH (2006)

Matthews, G., Warm, J.S., Reinerman-Jones, L.E., Langheim, L.K., Washburn, D.A., Tripp, L.: Task engagement, cerebral blood flow velocity, and diagnostic monitoring for sustained attention. J. Exp. Psychol. Appl. 16 , 187 (2010)

Reinerman-Jones, L.: Assessing multidimensional complex decision making with situational judgment tests. In: International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pp. 49–55. IEEE (2016)

Becker, S.W., Brownson, F.O.: What price ambiguity? Or the role of ambiguity in decision-making. J. Polit. Econ. 72 , 62–73 (1964)

Myatt, M.: Self-Made Man - No Such Thing. Forbes Magazine, New York (2011)

Handbook, US Army: Military Leadership. US Government Printing Office (1973)

Higgins, E.T.: Making a good decision: value from fit. Am. Psychol. 55 (11), 1217 (2000)

Einhorn, H.J., Hogarth, R.M.: Behavioral decision theory: processes of judgment and choice. Ann. Rev. Psychol. 19 (1), 1–31 (1981)

Klein, G.: Naturalistic decision making. Hum. Factors: J. Hum. Factors Ergon. Soc. 50 , 456–460 (2008)

Grunig, R., Kuhn, R.: Successful Decision-Making: A Systematic Approach to Complex Problems. Springer Science & Business Media, Heidelberg (2013)

Book   Google Scholar  

Plous, S.: The Psychology of Judgment and Decision Making. McGraw-Hill Book Company, New York (1993)

Hick, W.E.: On the rate of gain of information. Q. J. Exp. Psychol. 4 (1), 11–26 (1952)

Lipshitz, R., Klein, G., Orasanu, J., Salas, E.: Taking stock of naturalistic decision making. J. Behav. Dec. Making 14 (5), 331–352 (2001)

Berka, C.L.: Real-time analysis of EEG indexes of alertness, cognition, and memory acquired with a wireless EEG headset. Int. J. Hum.-Comput. Interact. 17 (2), 151–170 (2004)

Henelius, A., Hirvonen, K., Holm, A., Korpela, J., Muller, K.: Mental workload classification using heart rate metrics. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 1836–1839. IEEE (2009)

Ayaz, H., Shewokis, P.A., Brunce, S., Izzetoglu, K., Willems, B., Onaral, B.: Optical brain monitoring for operator training and mental workload. Neuroimage 59 (1), 36–47 (2012)

Aaslid, R.: Transcranial Doppler Sonography. Springer Science & Business Media, Heidelberg (2012)

Batista, J.: A drowsiness and point of attention monitoring system for driver vigilance. In: IEEE Intelligent Transportation Systems Conference, pp. 702–708 (2007)

Mendez, M.F., Anderson, E., Shapira, J.S.: An investigation of moral judgment in frontotemporal dementia. Cogn. Behav. Neurol. 18 (4), 193–197 (2005)

Damasio, A.R., Everitt, B.J., Bishop, D.: The somatic marker hypothesis and the possible functions of the prefrontal cortex [and discussion]. Philos. Trans. R. Soc. London B: Biol. Sci. 351 (1346), 1413–1420 (1996)

Hjortskov, N., Rissen, D., Blangsted, A.K., Fallentin, N., Lundberg, U., Sogaard, K.: The effect of mental stress on heart rate variability and blood pressure during computer work. Eur. J. Appl. Physiol. 92 (1–2), 84–89 (2004)

Reitan, R.M., Wolfson, D.: A selective and critical review of neuropsychological deficits and the frontal lobes. Neuropsychol. Rev. 4 (3), 161–198 (1994)

Masicampo, E.J., Baumeister, R.F.: Toward a physiology of dual-process reasoning and judgment: lemonade, willpower, and expensive rule-based analysis. Psychol. Sci. 19 (3), 255–260 (2008)

De Neys, W., Novitsky, N., Ramautar, J., Wagemans, J.: What makes a good reasoner? Brain potentials and heuristic bias susceptibility. Proc. Ann. Conf. Cogn. Sci. Soc. 32 , 1020–1025 (2010)

Download references

Author information

Authors and affiliations.

U.S. Army Research Laboratory – Human Research and Engineering Directorate Advanced Training and Simulation Division, Orlando, FL, USA

Irwin Hudson

Institute for Simulation and Training, University of Central Florida, Orlando, FL, USA

Lauren Reinerman-Jones & Grace Teo

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Irwin Hudson .

Editor information

Editors and affiliations.

SoarTech, Orlando, Florida, USA

Dylan D. Schmorrow

Design Interactive, Inc., Orlando, Florida, USA

Cali M. Fidopiastis

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper.

Hudson, I., Reinerman-Jones, L., Teo, G. (2017). A Review of Personnel Selection Approaches for the Skill of Decision Making. In: Schmorrow, D., Fidopiastis, C. (eds) Augmented Cognition. Enhancing Cognition and Behavior in Complex Human Environments. AC 2017. Lecture Notes in Computer Science(), vol 10285. Springer, Cham. https://doi.org/10.1007/978-3-319-58625-0_34

Download citation

DOI : https://doi.org/10.1007/978-3-319-58625-0_34

Published : 28 May 2017

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-58624-3

Online ISBN : 978-3-319-58625-0

eBook Packages : Computer Science Computer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Traditional Behavioral
• Gamified Behavioral
• Job Knowledge Skills
• Realistic Job Previews
• Work from Home
• Reference Checking
• Hiring Process Optimization

By industry

• Professional Services
• Tech, Media & Telecom
• Financial Services
• Hospitality
• Manufacturing
• Government, Education & Non-Profits

By use case

• Hourly Hiring
• Professional Hiring
• Talent Development
• Diversity and Inclusion
• Campus Recruiting
• Blog articles
• Whitepapers
• Case studies
• LIFT Awards

11 Effective Employee Selection Methods To Implement Today

F rom automated CV screening and reading cover letters to holding whiteboard interviews and asking trick questions, the range of methods employers use to assess candidates is immense and overwhelming.

So, how can you tell which employee selection methods will be most effective in your recruitment process?

We’ll detail the topic below, and we’ll show you some practical examples of pre-employment assessments that you can use in the hiring process to make candidate selection bias-free, less time-consuming, and more aligned with your company culture and goals.

What’s in?

What is employee selection.

Employee selection, also known as candidate selection, is the process of finding a new hire best suited for the role in question.

The steps in the employee selection process depend on the role you’re hiring for, your recruiting budget, the seniority of the position, available resources, and your organizational needs.

Rather than relying on one method as the sole criterion for selecting or rejecting candidates, your selection methods should be combined to make the most informed decision possible.

For example, you can combine pre-employment tests with other employee selection methods like job interviews or trial days to accurately predict job success and cultural fit. 

Booking.com ‘s employee selection process and methods for different roles.

What are the types of candidate selection methods?

Before we dive into the topic and detail the different selection methods, let’s quickly look at how they’re classified or grouped.

External vs internal recruitment

First off, you can recruit candidates from external sources, and this method is called external recruitment . Secondly, you can look inside of your company and source candidates for new or open roles from your existing employees. This is called internal recruitment and helps to enable internal mobility , or filling open reqs with internal job seekers.

Sourcing channel classification

Another way to classify the different candidate selection methods is to group them based on the sourcing or recruitment channel used. For example, you can use advertising to source new candidates; this includes direct advertising , where you place job adverts on job boards or your career site, and social advertising , where you source your candidates through job adverts shared on social media platforms.

Talent pool and referral recruitment

Another channel for finding new employees can be your existing talent pool or database of candidates who have previously applied to roles within your company and were suitable but were not hired. Then, you can rely on referral recruitment as another selection method. Here, as the name implies, you’re asking your existing employees to refer potential candidates. Tip: Harver’s automated reference checking solution regularly converts 50% of references into passive candidates .

Internships and apprenticeships

If the roles you’re hiring for are entry-level, as is often the case in high volume recruitment, then a good employee selection method is to offer internships and apprenticeships. Both options ensure that you can act on how to get candidates with the right skills and potential, and the “trial” period gives you the perfect opportunity to develop their skills, while assessing their culture fit.

Boomerang employees

This method doesn’t apply to all roles, but it can be a solution if, for example, you’re hiring a lot of seasonal workers. Boomerang employees are basically people who have worked for your company before, and have left on good terms. You already know they have the right skills, and they know the ins and outs of the job already, so they’re a good option to consider.

In the next section, we’ll focus on the external recruitment methods and referral recruitment , and we’ll detail some of the types of assessments you can use to make sure you’re selecting the top candidates for your open roles.

Like what you see?

Don’t miss out. Subscribe to our monthly digest to get the latest TA and TM resources delivered right to your inbox.

Best employee selection methods for choosing top talent

1. assess cognitive ability.

Cognitive ability is the number one predictor of job performance across all employment levels and industries.

Cognitive ability assessments are a form of pre-employment assessing used to evaluate how well candidates use a wide range of mental processes, such as working with numbers, abstract thinking, problem-solving, reading comprehension, and learning agility .

When implemented and administered correctly, a cognitive skills assessment is a highly effective method for predicting job success. However, one potential pitfall is the risk of adverse impact , which is the negative effect that a biased selection process has on a protected group of people.

To prevent it from hindering your recruitment efforts, you should make it a point to regularly measure the adverse impact of your cognitive ability tests.

For example, our platform provides recruiters with a variety of recruitment dashboards that track the most important hiring metrics. Among others, you can see if there’s any bias in your recruitment process, in which stages it occurs, and whether one location is more prone to bias than others.

If you’d like to see the Harver cognitive assessment in action, you can book a demo today.

Cognitive Ability Testing

2. Evaluate learning agility

Evaluating learning agility is another effective employee selection method to build into your recruiting process. American author, Alvin Toffler, broke down learning agility well when he said:

“The illiterate of the 21st century will not be those who cannot read and write, but those who cannot learn, unlearn and relearn.”

In other words, learning agility is the ability to be in a new situation, not know how to handle it, and then figure it out anyway. An agile learner can apply his or her past learnings to new scenarios that they have yet to experience.

Learning agility is a crucial ability that you can measure to gain a true understanding of how applicants function and adapt in ever-evolving work environments. For instance, think of how important learning agility is whenever reskilling or upskilling employees .

That said, it’s challenging to develop a well-constructed, reliable assessment for gauging learning agility. Challenging, but not impossible. If you’d like to see what this assessment looks like in practice, book a demo below.

3. Situational judgement test (SJT)

Another great way to enhance employee selection is to assess situational judgement capabilities. Situational Judgement Tests (SJT) present candidates with various scenarios that they might experience if they’re selected for the specific role they’re applying for.

Here’s for example an SJT developed by our People Science team for a client in the BPO/Contact Center vertical. This type of pre-employment assessment asks the candidate to choose the best and the wrost way to handle a job situation.

The scenarios are strategically chosen in collaboration with your recruitment or talent team, to illustrate the critical incidents that an employee might deal with once hired.

Behind the scenes, these hiring assessments evaluate how well applicants prioritize client inquiries, follow instructions, and handle situations that crop up in the workplace.

They’re highly predictive of job performance and culture fit and can provide candidates with a realistic job preview early on in the application process.

With that said, Situational Judgement Tests can be costly and sometimes difficult to construct and implement by a recruitment team alone. This is because they typically require input from an industrial-organizational (IO) psychologist, as well as a production team and designer.

In addition to offering tailor-made SJTs based on a customer’s profile, company culture, and hiring needs, Harver also offers out-of-the-box Situational Judgment Tests for faster implementation. If you’d like to see industry-specific examples, and understand how they can help your company hire better candidates, you can book a demo below.

Ready to transform your hiring process?

4. Measure employee integrity

Of course, you want to hire honest, reliable employees for your organization—but how exactly do you measure something like that?

Employee integrity tests allow you to collect insights into candidates’ honesty, dependability, and work ethic. Integrity and other relevant soft skills are typically assessed via a digital personality questionnaire . 

Of organizations that work with pre-employment testing

use integrity tests.

Source: CriteriaCorp

However, there are potential legal issues to be mindful of before jumping on the integrity testing bandwagon. Some have been challenged in court for requiring candidates to rate statements that could be seen as discriminatory. Some areas, such as Massachusetts in the U.S., have even banned employee integrity testing altogether.

To avoid legal problems, it’s important to be sure your test complies with applicable laws, does not have an adverse impact, and demonstrates validity.

5. Test job knowledge

Does the candidate have the actual knowledge needed to do the job in question?

While most of the time that knowledge can be learned on the job and other factors are more important, there are certain roles that require applicants to possess specific job knowledge and skills already.

Each job family requires a different jobs skills and knowledge assessment ; however, some skills like multitasking, typing, and language proficiency can be useful across various roles and industries.

With that said, skills assessments are still an extra step for candidates. That’s why timing is everything. Introducing a skills test too early in the recruiting process can scare away applicants. To avoid this, keep your skills assessment short, allow an adequate amount of time to complete it and wait until you have a small, shortlisted pool of candidates to move forward.

Stop guessing, Start data-driven hiring.

Learn how you implement a modern candidate selection process, that is: streamlined, experience-driven and backed by data.

6. Give a test work assignment

Test assignments or work sample tests are an excellent way to help with employee selection. They let top candidates get a sense of what they would do on the job, while you can understand the skills they bring to the table.  

The downside to test assignments is that if they’re administered too early or are too long, candidates might drop out of the recruitment process. Keep your test assignments brief, provide clear direction and introduce the test later in the recruiting process, such as after an initial interview, when candidates are more invested and likely to follow through.

For instance, for our Contact Center and BPO clients, we’ve developed a Live Chat Support Simulation module that allows recruiters to assess the most important skills of a chat agent while providing candidates with an engaging experience.

Experience our best-in-class Live Chat Assessment first-hand!

Perfect for remote hiring, our live chat assessment makes it easier than ever to hire live chat agents. Candidates experience the job, while you get actionable data to drive hiring decisions.

7. Organize an assessment center

Assessment centers allow employers to see candidates’ hard and soft skills in action. Rather than places that candidates go to take a specific test, in this context assessment center refers to a testing process that analyzes each candidate’s social, analytical and communication skills.  

These tests consist of simulations and exercises designed to evaluate how an individual would perform in real on-the-job scenarios. They are usually given in a stipulated amount of time and typically last for one day.

This approach helps recruiters and hiring managers make fully informed hiring decisions. Also, it provides candidates with a clearer picture of the realities of the job they’re applying for.

One shortcoming is that these tests need to be evaluated by trained people and can be costly to administer and rate. That’s why assessment centers are most cost-effective for large-scale recruitment efforts.

Because grading is manual, results can also be subjective. If you’re thinking about incorporating an assessment center, your assessors should be thoroughly trained to administer tests and evaluate candidates.

8. Structure your interview process

Structuring your interview process ensures that everyone is treated fairly and asked the same pre-determined questions. Doing so allows recruiters to compare candidates’ responses and to be more objective, keeping hiring biases to a minimum.

A structured interview process is especially helpful when you have several qualified candidates for the same role. With that being said, interviewers might evaluate candidates subjectively, which can make it difficult to take what they say at face value.

To get the most out of your interviews, create a standardized interview guide that includes both pre-determined questions and detailed scoring criteria. This will ensure you ask every candidate the same questions and remain as objective as possible when making your new hire selection.

9. Conduct peer interviews

It’s always a good idea to involve the team you’re hiring for, as they know the day-to-day responsibilities and current skills gaps better than anyone else. Peer interviewing helps ensure that you remain objective in your hiring efforts, as well as on the same page as the hiring manager and internal team. 

By letting team members interact with the candidates, you can gain extra insights that the hiring manager might have not been able to yield. Peer interviews help with deciding cultural fit and help candidates feel more at ease. Chances are, they’ll open up more than they would if they were interviewed by a potential boss!  

Potential pitfalls are that it’s necessary to train the interviewers to be sure they ask the right questions. It can also be distracting, sometimes getting in the way of daily responsibilities. You can combat this by choosing the right interviewers, creating a standard interview structure, providing comprehensive interview training and making the job requirements clear.

10. Check candidate references

Reference checks are more than just a formality. They’re a way of revealing valuable insights that can help you identify top candidates and better understand how an applicant would transition into the new role. Checking references allows you to find out more about candidates and how they work and can bring red flags to light before you make your employee selection.

That said, checking references can be time-consuming—especially when you’re hiring for countless open roles at a time. However, you can make it worth your while by asking the right questions. To save an hour of recruiter effort per check, consider an automated reference checking tool .

Focus on questions that provide more insight into performance, accomplishments, and weaknesses. Avoid asking closed questions that only require a yes or no answer, as those are less likely to offer useful information. 

Instead, here are examples of relevant, open-ended questions to ask during a reference check : 

• How long did the candidate work for your organization?
• What were the candidate’s roles and responsibilities?
• What was their biggest strength? Biggest weakness?
• Why did the candidate leave your organization?
• Would you rehire them if the opportunity arose?

of the referees prefer to take an online assessment when compared to more traditional reference-checking methods.

Source: SmartRecruiters

11. Host a job trial day

Like any long-term investment, it’s wise to try things out before making your selection. Inviting shortlisted candidates to a job trial day is a great way to see how they react to common situations that they make encounter in the role if they’re selected.

Think of it as an audition to assess both job and cultural fit. You can see how candidates and potential colleagues get along and help set expectations about both the specific role you’re hiring for and the company overall.

The downside to job trial days is that they are time-consuming for candidates. You should always discuss and agree on the conditions (e.g. compensation, a full/half-day or a couple of hours, etc.) ahead of time. This ensures the candidate experience remains positive—you don’t want to lose your top choice along the way! 

In conclusion

The employee selection process is as unique as an organization itself, and the best format for yours depends on many things. You need to find the selection methods that are reliable, predictive and best suited for the roles you need to hire for.

And also, don’t rely on just one method to make your decision. Instead, build a combination of several employee selection methods that get the job done—and put the right people in the right roles at your company!

Questions about employee selection? Schedule a call with our experts to get answers and tips based on your organization’s business challenges and goals.

• Harver Team

Recommended Articles

Why 2024 is the year of content moderation, world’s first content moderator assessment for $7.5b content moderation services market, how to reduce manufacturing no-shows, quick quits & late-stage attrition, ask the experts: how can talent teams get started with data.

Learning more about making better talent decisions faster?

Get the answers you need to optimize your TA and TM processes and results.

• Check References
• Optimize Your Hiring Process
• Diversity & Inclusion
• Customer Login
• Case Studies
• Integrations
• Privacy policy
• Security Policy Statement
• 2024 Harver B.V. All rights reserved.

Outmatch is now part of Harver

Ready to serve you with our full suite of talent solutions and a fresh look!

3 Most Effective Personnel Selection Methods For Hiring

Ben is a Content Marketing Manager at Test Partnership, where he is responsible for creating and promoting engaging content.

Cognitive Ability Tests

Personality questionnaires, (structured) interviews.

Of the many challenges facing organisations, few are considered more prescient than finding (and retaining) great employees. Personnel selection is perhaps the only challenge which is ubiquitous among all employers regardless of industry, sector, company size, or maturity, as all organisations endeavour to hire the best employees possible.

However, this is easier said than done, as human potential is a particularly difficult and elusive construct to measure or predict. Moreover, a plethora of personnel selection tools exist on the market today, making it exceedingly difficult to decide which to incorporate into selection processes.

Fortunately, a tremendous body of academic research is available to answer this question, and highlight the most effective selection tools available to organisations today. To summarise these findings, here are the three most effective personnel selection methods for hiring:

Want to watch a video instead?

If you would prefer to watch a video, here Ben outlines the 3 best personnel selection methods for hiring:

When it comes to predicting overall job performance, cognitive ability is king. Research clearly shows that ability tests are the strongest predictors of performance in complex professional, technical, and managerial roles, especially when it comes to “Task Performance” (as opposed to Contextual Performance). The reasons behind their effectiveness are easy to understand. Cognitive ability allows people to learn effectively, solve problems, and make decisions, greatly aiding task performance in any form of knowledge work.

Ability tests themselves usually measure specific aspects of cognitive ability, such as numerical reasoning, verbal reasoning, or inductive (logical) reasoning, and when used as a battery, form an overall measure of general cognitive ability. These assessments are usually completed online, making them highly scalable and convenient for HR teams to utilise. You simply invite your candidates to complete them online, wait for the results, and then screen out the low performers, progressing the successful candidates to the next stage. This makes cognitive ability tests the perfect short-listing tool, especially for volume recruitment processes.

Personality questionnaires allow you to measure the key behavioural indicators which predict contextual performance, role-fit, culture-fit, and engagement in the workplace. Few personnel selection methods are as versatile as personality questionnaires, allowing organisations to measure the key aspects of human character which are make or break in occupational settings. These assessments are also usually completed online, meaning they rank among the most convenient and scalable personnel selection tools available to organisations today.

The traditional employment interview is perhaps the most commonly utilised late-stage selection method, with almost all organisations using interviews at some stage. Although the academic research does suggest that interviews are incredibly useful tools to predict performance, it also reveals that only structured interviews display this level of predictive validity. This makes intuitive sense, as unstructured interview inevitably produce unreliable results, as candidates may be asked different questions, some being easier than others.

Although interviews are powerful predictors of performance, unlike ability tests and personality questionnaires they are not scalable selection tools. For example, if each interview requires three hours of preparation, planning, conducting, and then providing feedback, interviewing 1,000 candidates will require 3,000 hours of work. However, with psychometric assessments, inviting 1,000 candidates isn’t much more work than inviting 10, as the process is largely automated online. Because of this, structured interviews are highly effective late-stage selection tools, but make for very ineffective short-listing tools.

"For example, if each interview requires three hours of preparation, planning, conducting, and then providing feedback, interviewing 1,000 candidates will require 3,000 hours of work." - Ben Schwencke

Despite the proliferation of employee selection tools on the market today, these three personnel selection methods represent the pinnacle of effectiveness at predicting performance. All other selection tools should be considered mere supplements to these three methods, and are likely to offer only modest benefits over and above what ability tests, personality questionnaires, and interviews can offer. However, due to the high-stakes nature of employee selection, even modest benefits yield commercially significant results, and organisations are well advised to experiment with personnel selection tools in hopes of maximising the quality of hire.

Attract Only The Best

Make your hiring tests have the best possible candidate experience to ensure the very best candidates apply for your roles.

Other articles you may be interested in

How to Measure Quality of Hire

Learn of our 5 great ways to measure quality of hire which makes organisations make informed decisions about future hires.

Candidate Selection: A Definitive Guide

Learn of candidate selection to improve your candidate selection process and build a high-performing workforce.

Skills Based Hiring: An Ultimate Guide

Learn what skills-based hiring is and why it's an important innovation.

What is a sten score?

Sten scores are standardised 1-10 scores commonly used in psychometric testing...

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Preferences for work arrangements: A discrete choice experiment

Roles Conceptualization, Data curation, Formal analysis, Methodology, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Department of Sociology, University of Bamberg, Bamberg, Germany

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing

Affiliation Department of Sociology, Bielefeld University, Bielefeld, Germany

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – review & editing

Affiliations Department of Sociology, University of Groningen, Groningen, The Netherlands, Department of Sociology, Radboud University Nijmegen, Nijmegen, The Netherlands

• Peter Valet, 
• Carsten Sauer, 
• Jochem Tolsma

• Published: July 12, 2021
• https://doi.org/10.1371/journal.pone.0254483
• Reader Comments

This study investigates individual preferences for work arrangements in a discrete choice experiment. Based on sociological and economic literature, we identified six essential job attributes—earnings, job security, training opportunities, scheduling flexibility, prestige of the company, and gender composition of the work team—and mapped these into hypothetical job offers. Out of three job offers, with different specifications in the respective job attributes, respondents had to choose the offer they considered as most attractive. In 2017, we implemented our choice experiment in two large-scale surveys conducted in two countries: Germany (N = 2,659) and the Netherlands (N = 2,678). Our analyses revealed that respondents considered all six job attributes in their decision process but had different priorities for each. Moreover, we found gendered preferences. Women preferred scheduling flexibility and a company with a good reputation, whereas men preferred jobs with high earnings and a permanent contract. Despite different national labor market regulations, different target populations, and different sampling strategies for the two surveys, job preferences for German and Dutch respondents were largely parallel.

Citation: Valet P, Sauer C, Tolsma J (2021) Preferences for work arrangements: A discrete choice experiment. PLoS ONE 16(7): e0254483. https://doi.org/10.1371/journal.pone.0254483

Editor: Srinivas Goli, University of Western Australia, AUSTRALIA

Received: July 13, 2020; Accepted: June 24, 2021; Published: July 12, 2021

Copyright: © 2021 Valet et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The Dutch data were collected as part of the LISS panel and are freely available for non-commercial research at https://www.dataarchive.lissdata.nl/study_units/view/915 . Users have to sign a statement confirming that information about individual persons, households etc., will not be released to others. Data will only be made available after a signed statement has been received through https://statements.centerdata.nl . The German data were sampled on basis of social security records. Therefore, data are restricted by the German law for potentially sensitive data. For re-analysis purposes, interested users must apply for data access at the Research Data Center of the German Institute for Economic Research ( https://portal.fdz-bo.diw.de ). Eligible researchers can access the data on site at DIW Berlin.

Funding: CS acknowledged a grant of the Dutch Research Council (NWO). Veni grant number: 4510-17-024. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

In recent decades, increasing international competition, emerging markets, and shorter economic cycles, coupled with demographic changes, high unemployment rates, and restrictive employment regulations, have put many Western economies under pressure [ 1 ]. Some have responded by implementing deregulation policies. Intended to improve international competitiveness, deregulation allows employers to use their workforce more flexibly under changing market conditions [ 2 , 3 ]. Consequently, the overall share of non-standard work arrangements—such as part time work, temporary employment, on-call work—has increased considerably [ 4 ]. At the same time as employers are shifting to atypical contracts for economic reasons, employees are demanding new and often more flexible work arrangements that align with their individual circumstances [ 5 , 6 ]. Reasons for shifting employee demands include the ongoing individualization of the life-course [ 7 , 8 ], the emergence of new family or non-work arrangements [ 9 ], and a growing interest in personal development and life-long learning [ 10 ]. Recent literature suggests, people often do not consider high earnings to be the main reason for choosing a job [ 11 ]. Instead, they are increasingly looking for a job that provides flexible working hours [ 12 ], a job they consider meaningful [ 13 ], a job that gives them a sense of belonging [ 14 ], or one that provides possibilities for personal development [ 15 ].

Today’s employers try to offer work arrangements that fit employees’ demands. Thus, a profound understanding of those demands, including and beyond earnings, is essential for employers as they compete with others for qualified personnel. A broader knowledge of the relative importance of job attributes is also crucial for research on labor processes and organizational inequality. While there seems to be broad consensus that “good” jobs are not only a matter of high earnings and job security [ 16 ], there is little evidence on the preference order of various job attributes and the extent to which employees focus more on flexible working hours or further training opportunities and less on high earnings. It would also be useful to know if such preferences are shared by most people or are subgroup specific. For example, theories on gendered socialization and traditional gender roles suggest men and women differ considerably in their preferences for extrinsic and intrinsic job traits [ 17 , 18 ].

Thus, we set out in this study to investigate people’s preferences for specific job attributes using a discrete choice design. To learn which job attributes drive people’s job decisions, we constructed sets of three hypothetical job offers in which we experimentally varied the job attributes. Out of these three offers, respondents chose the one they considered most attractive. Based on various strands of the literature, we identified six job attributes that people are likely to consider when opting for or against a job: level of earnings, job security, options for further training, scheduling flexibility, the reputation of the company, and the gender composition of the work team. We implemented our choice experiment in two large-scale surveys in 2017—one in Germany and the other in the Netherlands.

Our study contributes to the literature in several ways. Discrete choice designs are particularly suitable to investigate how people deliberate between different alternatives. Thus, our discrete choice experiment provided an empirical test of people’s preferences for different job attributes. We studied the importance of these attributes for all respondents in concert and separately for men and women to explore how women and men differ in their decision processes. Moreover, the implementation of our choice experiment in surveys in two countries—Germany and the Netherlands—allowed us to investigate the consistency of evaluation patterns, not only between social groups, but also in different macro-structural contexts. The largely parallel results speak for their robustness and provide solid evidence of people’s preference structure for different job attributes.

Individual job preferences

To understand which job attributes people consider more salient when opting for or against a job alternative, it is necessary to identify the most pertinent job attributes in a more general context. For this, the sociology of work literature provides a good starting point [ 16 , 19 ]. This literature offers comprehensive theoretical and empirical evidence of the potential importance of specific job characteristics and builds on insights from economic and sociological theory and research.

From this literature, we identified six aspects that people consider most relevant: (1) earnings, (2) job security, (3) opportunities for further training, (4) scheduling flexibility, (5) prestige of the company, and (6) the gender composition of the work team. In the following, we elaborate on these six key job attributes and discuss their expected impacts on job decisions. We complement this review with a discussion of why men and women might differ in how they value a particular job attribute. Our gendered expectations mostly draw on theories of gendered socialization and traditional gender roles [ 15 , 17 , 18 ].

First, the level of earnings is a main concern of job seekers [ 20 ]. Classical micro economic theory defines the level of earnings as the most important aspect of a job, and people consistently try to maximize their earnings [ 21 ]. High earnings not only contribute directly to employees’ material well-being; they also provide social status. Research on subjective well-being suggests individual life satisfaction increases with the level of earnings [ 22 , 23 ]. More recent studies, however, show positive but declining marginal effects of earnings on subjective well-being [ 24 – 26 ]. While this might indicate that the level of pay is not as important as previously assumed, meta-analytic studies of motivational initiatives point to the level of pay as the most effective employee motivator [ 20 , 27 ]. Some scholars discuss gendered preferences for high earnings and argue that due to early gendered socialization, women are more intrinsically oriented, and men are more extrinsically motivated. Accordingly, men value high pay more than women, who consider other aspects of a job more important [ 17 , 18 , 28 ]. This argument aligns with a care-rationale reasoning, as women more often work in the low-paying care-service sector [ 29 ]. Overall, the various strands of literature suggest jobseekers value high earnings but are somewhat ambivalent about whether high earnings are the most important job characteristic. Based on the literature, we expected the opportunity to earn more in one job than another would be a crucial factor in the decision-making process for a specific job offer. Moreover, we expected that high pay would be more important for men than for women.

Second, the sociology of work literature argues job security determines perceived job quality. While being more individualistic and flexible than employees in the past, people still value the power to decide on their own whether they want to keep or change a job. Many national and comparative studies show employees frequently rank job security as most important when asked to state their preferences for different aspects of work [ 15 , 30 – 33 ]. Yet the implementation of deregulation policies has resulted in a deterioration of job security for many employees [ 34 ], with severe consequences. Studies show the fear of job loss is comparable to the psychological distress people experience when they actually lose a job [ 35 , 36 ]. Apart from psychological consequences, insecurity affects behavior. Individuals with insecure or temporary jobs tend to postpone important life events such as family formation [ 37 ] or capital investment [ 38 ]. Job insecurity is especially prevalent among newly hired employees who may not have a permanent contract. For example, in Germany in 2018, 35 percent of all newly hired employees in the private sector and about 50 percent in the public sector had a temporary contract [ 39 ]. As permanent contracts provide more job security, we expected people’s sense of the value of a job offer would increase with contract duration, and they would therefore prefer longer-term offers. As the male breadwinner norm remains dominant in many Western countries—especially in conservative welfare states such as Germany [ 40 ]—we expected job security would be more important for men than for women.

Third, opportunities for training and improved qualifications are attractive to employees, as they facilitate their advancement in both internal and external labor markets. Some evidence indicates the detrimental effects of a temporary job are weaker if employers provide their employees with opportunities for further training [ 41 ]. Other studies report decreasing preferences for high earnings associated with higher levels of learning opportunities [ 42 ]. From a human capital perspective, there are two types of further qualifications: (a) general human capital that is transferable to positions outside the actual job, which, in turn, makes the employee more attractive for employers outside the current workplace; (b) specific human capital which is predominantly company specific and therefore mostly beneficial for the internal labor market. Investing in specific human capital increases promotion opportunities and chances for higher pay within the company but has limited value for other employers. Accordingly, we expected that people would generally value opportunities for further training. Moreover, we expected that both men and women would value general human capital over specific human capital, as it would increase their options in the larger labor market.

Fourth, scheduling flexibility allows people to better reconcile the work and non-work spheres of their lives. The decline of the male breadwinner model has led to new arrangements of aligning work schedules with household needs. Many studies suggest women are predominantly faced with these challenges [ 43 – 45 ], especially if there are young children in the household [ 46 ]. Employers are increasingly reacting to these flexibility demands and offering more family friendly work arrangements. While jobs traditionally had a fixed start and end (mostly nine to five), nowadays employers are increasingly offering more flexible schedules to provide better compatibility [ 47 ]. Some are also offering extra time off if needed. Based on the literature, we expected that extended scheduling flexibility would be important for many people when opting for or against a job offer. As women still do most of the reconciling of work and family needs, we expected scheduling flexibility would be more important for women than for men, especially with children in the household.

Fifth, beyond these individual job attributes, employees’ identification with a job is often driven by the reputation of the company they work for [ 48 , 49 ]. The sociology of work literature highlights that pride and dignity are important for employees’ well-being and commitment [ 50 ]. Feelings of pride stem from their work and from the reputation of their company. This also resonates with work on organizational citizenship and extra-role behavior [ 51 ]. In contrast, employees may behave dysfunctionally and show more counterproductive work behavior in organizations with lower prestige [ 52 ]. Accordingly, we expected people’s preferences for a job would increase when the reputation of the company was higher. As the reputation of a company can be considered an intrinsic job preference or a signal of prestige, we had no expectation of whether it would be more important for women or for men.

Sixth, many employees work in gender segregated occupations [ 53 ] or on gender segregated teams [ 54 ]. Various studies in work and in social psychology have investigated the functionality and performance of teams and their gender composition, yielding mixed results [ 55 , 56 ]. Studies focusing on the interactions at the workplace or the formation process in formal teams highlight the importance of the gender composition of work groups and teams [ 57 , 58 ]. There is ample evidence of gender homophily, suggesting that women prefer working with other women and men prefer working with other men [ 59 , 60 ]. One explanation is that people expect others who are like them to act more predictably, have similar interests, and be more trustworthy [ 61 ]. Other studies have not found a homophily bias, and some have even found the contrary [ 62 ]. We expected information on the gender composition of the work team would be important for people’s job decisions. Following the mainstream research, we expected a gender homophily bias in preferences, whereby people would choose a job offer when most co-workers were the same gender as they were.

The present study

Our literature review revealed a number of job characteristics beyond earnings that attract people. We also learned that women and men may value these characteristics (e.g., scheduling flexibility) differently. In our choice experiment, we presented three job descriptions with varying job attributes side by side to respondents. In 2017, we implemented our choice experiment in two large-scale surveys, one conducted in Germany and the other in the Netherlands. Germany and the Netherlands are similar in many respects. Both are welfare states offering various social security (health, unemployment) and family (allowance for children, maternity leave) benefits. At the same time, there are remarkable differences. For example, paternity leave (or father-specific parental leave) benefits are far more generous in Germany (9 weeks) than in in the Netherlands (2 days) [ 63 ]. General labor market statistics show quite low unemployment rates in both countries in 2017 (GER: 3.8; NL: 4.9) [ 64 ] and a similar inequality in disposable income—measured by a Gini coefficient of 0.294 in Germany and 0.285 in the Netherlands [ 65 ]. However, there are remarkable differences in the countries’ temporary and part-time employment patterns. In 2017, 12.8 percent of all employment in Germany was temporary; in the Netherlands, this share was considerably higher, at 21.8 percent. Despite this dissimilarity, both countries show a huge age gradient in temporary employment. Among the employees younger than 25, more than 50 percent (GER: 52.6; NL: 58.8) had temporary contracts; for those between 55 and 64, it was below 10 percent (GER: 3.4; NL: 7.5) [ 66 ]. Patterns in part-time employment are even more dissimilar than those for temporary employment. In Germany, the part-time employment rate in 2017 was 22 percent, in the Netherlands 37 percent. In both countries, more women than men worked part-time. In Germany, 36 percent of all working women were part-timers, compared to 9 percent of all working men; in the Netherlands 56 percent of all working women were part-timers, compared to 19 percent of all working men [ 67 ].

The implementation of our choice experiment in countries with both basic similarities and considerable differences allowed us to investigate people’s preferences for certain job attributes and differences between social groups and also to verify our results in a cross-country replication.

Materials and methods

Respondents.

The data for our study were collected in 2017 as part of two independent national surveys. The German data stem from the second wave of the employee panel survey “Legitimation of inequalities over the life-span” (LINOS-2). The target population of the first wave (LINOS-1) conducted in 2013 was employees subject to social security contributions. This included most private and public sector employees but excluded the self-employed and civil servants. Respondents for LINOS-1 were randomly sampled all over Germany. About 76 percent of the LINOS-1 respondents who consented to stay in the panel also participated in LINOS-2—representing 2,741 respondents. The LINOS survey was conducted as a multi-mode survey with random allocation to one of two modes: self-interviewing (by mail or online, depending on respondent’s preference) or personal computer-assisted interviewing. A detailed description of the data, the sampling procedure, and the materials used in the survey can be found in the technical reports [ 68 – 70 ]. As the discrete choice experiment was implemented only in LINOS-2, we restricted our analyses to this wave. For re-analysis purposes, the full dataset is available under the restriction of the German law for potentially sensitive data. Interested users must apply for data access, and the data can only be accessed on-site at the German Institute for Economic Research (DIW Berlin).

The Dutch data come from the Family Survey Dutch Population (FSDP) [ 71 ]. The FSDP is a large-scale survey that began in 1992 and has since been conducted at five-year intervals by the Sociology Department of Radboud University Nijmegen. All citizens of the Netherlands irrespective of their employment form the target population of the FSDP. Our discrete choice experiment was implemented in the 2017 wave. The 2017 FSDP wave consisted of 3,099 respondents who were members of the Longitudinal Internet Studies for the Social Sciences (LISS) Panel. The total LISS panel consisted of 4,500 households, comprising 7,000 individuals and was based on a probability sample of households drawn from the population register by Statistics Netherlands. The 2017 FSDP wave data is accessible through the LISS Data Archive [ 71 ].

Table 1 shows the arithmetic means, standard deviations (SD), minimums (Min.), and maximums (Max.) for key variables describing the two samples. Overall, we see that the samples differed only slightly. Looking at demographics, we see a slightly larger share of women, somewhat older respondents, a higher share of college degrees, and a higher share of respondents in the Dutch data. Obviously, the key difference between the two samples is the share of currently employed which, due to the discussed differences in the sampling, was much higher in Germany than in the Netherlands. In line with the OECD data on part-time employment [ 67 ], weekly working hours were higher in Germany than in the Netherlands.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0254483.t001

In discrete choice experiments, respondents choose (at least) one option out of multiple alternatives presented simultaneously to them within a so-called choice set. In these choice sets, different attributes vary experimentally in their levels. Therefore, discrete choice experiments are multi-factorial survey-experiments. The random allocation of the choice sets to respondents ensures independence of the respondent’s characteristics. Moreover, the composition of fictitious alternatives allows researchers to investigate people’s preferences for job offers independent of their current situation. This might potentially compromise the external validity; the extent to which results from discrete choice experiments can be generalized to actual job choice situations. Yet studies specifically investigating external validity of preferences or attitudes expressed in multi-factorial survey experiments conclude they are very similar to the preferences and attitudes respondents show in their real lives [ 72 ].

To create the three alternative job offers of our choice sets, we used D-efficient sampling strategies. This ensures efficient estimates of potential effects, as attributes are uncorrelated and balanced in their levels. For the sampling of the German choice sets, we used the user-written Stata ado dcreate [ 73 ]. We sampled 36 job offers from which we created 12 sets with alternative job offers. Respondents in the German survey were randomly faced with one of the 12 choice sets. Accordingly, we had a between-subjects design. Moreover, the order of the choices within each choice set was random; a specific job description was presented as the first alternative to one respondent and as the second or third alternative to another. Thus, we avoided primacy and recency effects [ 74 ].

To generate the experimental set-up for the Dutch survey, we used the sampling procedures implemented in SAS [ 75 ]. Again, we used the D-efficiency criterion to sample the alternatives and to combine them into choice sets. In this case, however, we developed a design in which every respondent answered three instead of one choice set. Therefore, we first sampled 90 alternative job offers. From these, we created ten sets of three choice sets. As before, each choice set consisted of three alternative job offers. Choice sets were once again randomly allocated, but this time, the order of appearance of the three sets every respondent rated was also random. This ensured that the levels of choice attributes and respondent characteristics were uncorrelated, and there were no order effects within the sets.

In both surveys, respondents could skip the task. In the German case, only 76 of the 2,741 respondents did not make a decision on a job offer. Thus, 97 percent of all choice sets were answered. Among the Dutch respondents, only 86 percent of all displayed choice sets were answered. This might indicate that the differences between job offers were more subtle in the Dutch case—due to the higher number of overall experimentally varied job offers (90 vs. 36 in the German data)—thus increasing the difficulty of the decision-making process.

In the German version of the survey, our hypothetical job offers comprised five experimentally varied attributes: earnings, job security, opportunities for further training, scheduling flexibility, and prestige of the company. The Dutch version included a sixth attribute: the gender composition of the work team. All attributes had three levels (see Table 2 ). We varied earnings as average earnings, slightly above average earnings, and far above average earnings. We opted against job offers with under-average earnings as we assumed preferences for avoiding underpayment would dominate all other attributes. We also decided against concrete amounts of earnings to prevent respondents from comparing their actual earnings with those in the job offer. We measured job security by the employment duration specified in the job contract. Again, we varied three levels: a permanent contract, a 5-year contract, and a 2-year contract. In the third attribute, training opportunities, we distinguished between general training (e.g., distance learning, language courses) and work-specific training, both paid by the employer. General training measured investments in general human capital; work-specific training measured investments in specific human capital. The third level was no opportunities for further training. To capture preferences for family or care arrangements, we distinguished three types of scheduling flexibility that are commonly advertised in German job openings: flexible working hours with short-notice time off if needed, flexible working hours without extra time off if needed, and no flexibility in working hours at all. The reputation of the company offering the job was described as very good, average, or rather bad. The gender composition of the work team (only included in the Dutch data) varied between more men, more women, and about an equal share of men and women. S1 Table shows a choice set with all six attributes.

https://doi.org/10.1371/journal.pone.0254483.t002

Data analytical approach

As people’s decisions on a job offer also depend on the displayed alternative offers, we estimated conditional logit models for the German and Dutch data. The conditional logit model (sometimes called multinomial logit model [ 78 ]) is appropriate for unlabeled and randomly ordered choices, as in our case (in many discrete choice applications, choices are labeled—e.g., choice of a specific mode of transportation or preference for a specific brand—and thus require strategies that allow estimation of alternative specific intercepts) [ 79 , 80 ]. To test if respondent characteristics—such as gender—explained decisions for or against certain job offers, we included interaction terms. By doing this, we could test the theoretical assumption of gender differences for certain job attributes.

The structure of the Dutch survey was somewhat more complicated than the German one, as every respondent evaluated three choice sets. Therefore, we had 8,034 decisions nested in the 2,678 survey respondents. Re-analyses with panel-data mixed conditional logit models accounting for the data structure yielded results similar to those of the first choice set only. Therefore, we decided to report results the same way for the German and Dutch data.

In the following, we report all results as average semi-elasticities [ 81 ] and provide the corresponding tables with all estimates, along with information on model fit statistics in the Supporting information. For our data analysis, we used Stata 16.1, for the estimation of the average semi-elasticities, we used the user written ado aextlogit [ 82 ], for the presentation of results, we used the user-written ados coefplot [ 83 ], estout [ 84 ], and the scheme plotplain [ 85 ].

Preferences for job characteristics in Germany and the Netherlands

Fig 1 shows the results of the discrete choice experiment for the German sample. The figure displays the average semi-elasticities for all levels in contrast to the grand mean. The grand mean reflects the decision probability for one of the three job offers if the respondent has no preferences for one job offer over the others. This base probability of about 1/3 is set to zero in the figures. Accordingly, a positive effect—displayed to the right of the zero line on the x-axis—indicates that people are, on average, more likely to choose a job offer if it includes the respective job attribute. An effect displayed to the left of the zero line indicates people are less likely to choose a job offer if it includes the respective job attribute. The whiskers represent the 95% confidence intervals. A confidence interval overlapping zero means the effect of the respective job attribute is statistically not significant.

The figure shows the relative decrease/increase in probabilities (with 95% confidence intervals) of choosing a job due to the respective job attribute.

https://doi.org/10.1371/journal.pone.0254483.g001

Looking at the dimension earnings first, we see that the percentage increase in the (base) probability of choosing a job offer was 24.2 percent if earnings were far above average and if the job offered slightly above average earnings, the probability increased by 7.3 percent. However, if the job offer included only about average earnings, the (base) probability of choosing this offer decreased by 31.5 percent. Wald tests on the equality of these effects indicate that people significantly preferred earnings far above average over earnings slightly above average (χ 2 = 19.11; p < .001), as well as far above average earnings over average earnings (χ 2 = 146.78; p < .001), and slightly above average earnings over average earnings (χ 2 = 72.40; p < .001). These results are in line with our theoretical expectations on the increasing utility of higher wages. Model 1 of S2 Table shows the corresponding semi-elasticities in the traditional way with a reference category—meaning that coefficients in all tables in the Supporting information must be interpreted with respect to the reference category instead of the base probability. For example, the coefficient of .557 for far above average earnings means that people are on average 55.7 percent more likely to choose a job offer displaying far above average earnings compared to a job offer with about average earnings (the reference category).

Second, people were, on average, 60.8 percent more likely to choose a job with a permanent contract and 56.9 percent less likely to choose a job lasting only 2-years. The effect of a 5-year contract was also slightly negative but not significant. Again, all differences between these levels of job security were statistically highly significant (permanent contract vs. 5-year contract: χ 2 = 184.15; p < .001; 5-year contract vs. 2-year contract: χ 2 = 80.54; p < .001).

Third, people preferred opportunities for further training over no training opportunities. Yet they did not differentiate between opportunities for general or specific training (χ 2 = 1.80; p = .179) as theory would suggest, given the generalizability and convertibility of the former.

Fourth, scheduling flexibility was important. People were, on average, about 30 percent more likely to choose a job offering scheduling flexibility. Yet it seemed to make little difference if in addition to a flexible schedule, time off if needed was explicitly granted (χ 2 = 0.01; p = .969). In contrast, people were 59.1 percent less likely to choose a job offer with no flexibility. This is in line with research suggesting employee preferences for more individualistic and flexible schedules.

Lastly, the reputation of the company made a difference in the decision process. People, on average, were much more likely to choose a job with a company with a very good reputation over one with a company with only an average reputation (χ 2 = 68.18; p < .001) or a rather bad reputation (χ 2 = 440.83; p < .001). Surprisingly, a very good company reputation was even more important than high earnings (χ 2 = 49.53; p < .001), specific training opportunities (χ 2 = 50.67; p < .001), or scheduling flexibility (χ 2 = 20.62; p < .001).

Fig 2 shows the results for the Dutch sample. The graphical presentation is similar to that for the German sample. Model 1 of S3 Table shows the respective coefficients and standard errors of the semi-elasticities when using specific reference categories.

https://doi.org/10.1371/journal.pone.0254483.g002

Comparing Figs 1 and 2 , we observe that the overall effect sizes in the Dutch data were smaller than in the German data, but the directions and the relative sizes of the effects were quite similar. Additional analyses restricting the survey to employed respondents, to identical choice sets across surveys, and to only the first of each respondent’s three decisions led to very similar results. This indicates that differences in study design did not account for the smaller effect sizes we found in the Dutch data. The preference structure was again very straightforward in terms of earnings, contract duration, further training, family/care arrangements, and reputation of the company. Yet in the Dutch case, the most important aspect driving the choice of a job offer was not job security but the reputation of the company. Nonetheless, job security was more important than high pay (χ 2 = 172.97; p < .001) or scheduling flexibility (χ 2 = 125.61; p < .001). Finally, with respect to the additional attribute included in the Dutch choice set—the gender composition of the work environment—people preferred working in workplaces with equal shares of men and women to working in male-dominated (χ 2 = 21.38; p < .001) or female-dominated (χ 2 = 30.88; p < .001) workplaces. People also showed a tendency to prefer male-dominated workplaces to female-dominated workplaces, but this difference was not statistically significant (χ 2 = 0.53; p = .466).

Gender differences in job preferences

Fig 3 shows the results separately for men and women in the German sample. Again, we estimated the displayed average semi-elasticities in contrast to the grand mean from the coefficients of S2 Table (Model 2 and Model 3). We estimated separate models for men and women and a full interaction model to formally test for gender differences (Model 1 of S4 Table ). The inspection of Fig 3 suggests men and women based their decisions for or against a job offer on the same job attributes, but they had slightly different priorities. For men, the option to receive wages far above average was a stronger predictor of choosing a job than for women. While men were more likely to choose jobs with high earnings than those with slightly above average earnings (χ 2 = 23.42; p < .001), this differentiation did not matter as much to women (χ 2 = 1.58; p = .208). Men and women did not seem to differ considerably in their preferences for jobs with average earnings. While this resonates with our theoretical expectation that men value extrinsic aspects of a job more than women, we observed no statistically significant gender differences in preferences for earnings far above average (χ 2 = 1.92; p = .166) or slightly above average (χ 2 = 0.63; p = .428). Again, the most important aspect of a job offer for both men and women was job security, and we observed no gender differences in preferences for job security. Turning to the further training opportunities, we observed a slight leaning of women towards general human capital and of men towards specific human capital. However, these gender differences were not statistically significant. The possibility of having a flexible work schedule was significantly more important for women than for men. Women were more likely to choose job offers with flexible working hours (χ 2 = 5.88; p = .015) and flexible job offers with time off if needed (χ 2 = 8.48; p = .004). This is largely in line with the literature highlighting different responsibilities of men and women in the arrangements of work and family life. Lastly, the reputation of the company seemed to be somewhat more important for women. As the reputation of the company can be considered an intrinsic job characteristic, this result resonates with our theoretical expectations of the gendered valuation of extrinsic and intrinsic job attributes. Yet formal tests showed no significant gender differences (χ 2 = 2.33; p = .127).

https://doi.org/10.1371/journal.pone.0254483.g003

When we compared the importance of the different job attributes separately for men and for women, we had some surprising results. Job security was the most important job attribute for men and women alike, but the preference order of the other job attributes was less clear. For men, a very good company reputation was even more important than high pay (χ 2 = 10.04; p = .002). Surprisingly, for men, having a flexible schedule (χ 2 = 0.46; p = .498) and specific training opportunities (χ 2 = 2.36; p = .124) were as important as high pay. For women, jobs with a flexible schedule with (χ 2 = 14.00; p < .001) or without (χ 2 = 9.42; p = .002) time off if needed were more attractive than high-paying jobs. These results are in line with the theoretical expectation that women do most of the reconciling of work and family but are less supportive of the expectation that men will show preferences for extrinsic job attributes.

Fig 4 shows the respective effects separately for male and female respondents. Model 2 and Model 3 of S3 Table display the respective estimated semi-elasticities and standard errors. Model 2 of S4 Table shows the full-interaction models that test for gender differences. The overall patterns shown in the German data in Fig 3 and the Dutch data in Fig 4 reveal remarkably similar preferences. As we found for the German respondents, among the Dutch respondents, high earnings and a permanent contract seemed to matter more for men. The formal test showed robust significant gender differences for a permanent contract (χ 2 = 13.61; p < .001) and marginal significant differences for high earnings (χ 2 = 3.37; p = .066). Scheduling flexibility (χ 2 = 15.14; p < .001; with time off if needed: χ 2 = 25.61; p < .001) and the reputation of the company (χ 2 = 3.54; p = .060) were more important to women. With respect to family/care arrangements, it seemed women differentiated between the two types of scheduling flexibility. The probability of choosing a job increased by 12 percent if it included a flexible schedule with additional time of if needed, whereas it increased by 9 percent when “only” a flexible schedule was offered. This difference was not statistically significant, however (χ 2 = 1.30; p = .253). Preferences for further training opportunities again resembled those we found in the German data. Yet the opportunity for further training did not affect women’s job choices significantly, and it affected men’s choices only slightly for general training. Results for preferences for the gender composition in the work environment were largely parallel for men and women: both seemed to prefer working with about equal shares of men and women. This result contradicts our theoretical expectation of homophily, namely, that men would prefer to work in male dominated and women in female dominated workplaces. There was a slight tendency among men to favor job offers from companies with more men over job offers from companies with more women and vice versa among women, but neither the difference among men (χ 2 = 1.80; p = .179) nor the difference among women (χ 2 = 0.09; p = .763) was statistically significant.

https://doi.org/10.1371/journal.pone.0254483.g004

There were some remarkable cross-country gender differences. For Dutch men, high earnings were as important as a permanent contract (χ 2 = 2.02; p = .155), a very good company reputation (χ 2 = 0.03; p = .860), and a balanced gender composition of the work team (χ 2 = 0.16; p = .686). In contrast to German men, for Dutch men, high pay was more important than training opportunities (χ 2 = 16.27; p < .001) and scheduling flexibility (χ 2 = 10.64; p < .001). For Dutch women, a very good company reputation was more important than high earnings (χ 2 = 17.65; p < .001) and a permanent contract (χ 2 = 16.69; p < .001). All other aspects, except for further training opportunities, were equally important for Dutch women. These results are again largely in line with the theoretical expectation that women do most of the reconciling of work and family and also speak to the greater importance men place on extrinsic job attributes.

Our theory section suggests that in Germany and the Netherlands, women are more likely than men to face the challenge of aligning work schedules with household needs. Much of the debate on the high shares of part-time working women has revolved around women’s need for scheduling flexibility if they care for children [ 46 ]. Therefore, we checked if children in the household moderated women’s preferences for jobs with scheduling flexibility. The respective full-interaction models are displayed in S5 and S6 Tables. For the German respondents (see S5 Table ), the presence of children increased preferences for scheduling flexibility. However, this effect was entirely driven by women’s preferences for flexibility. German men showed no significantly increased preferences for flexible schedules if children were in the household (Model 3 of S5 Table ). For the Dutch respondents ( S6 Table ), preferences for scheduling flexibility were also higher if children lived in the household. This effect seemed to be mostly driven by male respondents and likely reflected the less generous parental leave policies for fathers in the Netherlands.

Robustness checks

To corroborate our results, we complemented our analyses with sensitivity analyses and robustness checks. As Dutch respondents completed three different choice sets, we re-analyzed the data with panel-data mixed conditional logit models to allow for correlations not only within but also across choice situations. In addition, we investigated whether the results changed when we restricted the data to the first choice per respondent. These sensitivity analyses yielded results similar to those presented above.

We checked whether the general patterns of our results held in two subgroup analyses. First, we investigated whether currently employed respondents differed in their job preferences from respondents who were not currently employed. The idea was that certain job characteristics, such as job security or flexibility, might be more important to employed respondents, as they might be more salient in their daily lives. S7 Table shows the full interaction model. Among the German respondents (Model 1), none of the interactions was significant, suggesting current employment status did not influence job preferences. However, two interactions were significant for the Dutch respondents (Model 2). For employed Dutch respondents, a permanent contract was somewhat more important, and a flexible schedule was somewhat less important than for the unemployed. Nevertheless, the general pattern of job preferences remained largely the same.

Second, we investigated if, among the employed, current job satisfaction affected the results. The idea was that those who were dissatisfied with their current job were more likely to be in the process of a job search and therefore would be faced with similar choices in their real lives. S8 Table shows a full interaction model with current job dissatisfaction as a moderator. We defined all employees who rated their job satisfaction on an 11-point scale below the country mean as dissatisfied. The interactions indicated that our reported results were solid. We only detected one significant interaction, whereby dissatisfied respondents had somewhat higher preferences for training opportunities.

In this study, we investigated which job attributes people consider more important when choosing a job. For this, we designed a discrete choice experiment in which we experimentally varied the levels of six job attributes. We expected to find positive effects of high earnings, permanent contracts, further training opportunities, scheduling flexibility, company reputation, and same gender working groups on the probability of choosing a job offer. In 2017, we implemented our discrete choice experiment in large-scale surveys in Germany and the Netherlands.

Even though the experimental design and the survey samples of the two studies differed considerably, our analyses revealed largely similar job preferences among German and Dutch respondents. More specifically, people’s decisions on a job offer were mostly driven by high job security in terms of a permanent contract, by high earnings, and by a very good company reputation. Yet we detected some notable cross-country differences in which job attribute people considered most important. While German respondents thought job security was the most important attribute, Dutch respondents pointed to the company reputation. Arguably, a permanent contract means more in terms of employment protection in Germany than the Netherlands. As German employment regulations set higher barriers for employers to fire employees without due cause, initial employment in Germany is increasingly on a fixed term basis. Because of the less strict employment regulations in the Netherlands, Dutch respondents might have associated permanent contracts less strongly with job security than German respondents. Somewhat surprisingly, in both countries, people thought high pay was less important than we had expected. Admittedly, we did not vary absolute pay levels; we only specified whether a specific job paid much more, slightly more, or about the same as similar jobs. In Germany, a permanent contract was more important than high pay; in the Netherlands, a good company reputation and a permanent contract were more important than high pay. This is in line with sociological theory’s suggestion that people’s job preferences are increasingly driven by other job aspects than high pay [ 11 – 15 ].

In more detailed analyses, we looked for gendered job preferences. As expected, while all respondents valued scheduling flexibility, it was more important for women than men. This supports our theoretical reasoning on traditional gender roles and is in line with recent research finding a considerable gender gap in household responsibilities [ 43 – 45 ]. To corroborate our interpretation of the finding, we investigated the moderating role of children in the household on job preferences. In both countries, we found more preference for scheduling flexibility if there were children in the household. Notably, in Germany, children in the household increased women’s preferences for scheduling flexibility but did not play a significant role in men’s preferences. This accords with recent research showing gender differences in scheduling flexibility and care responsibilities [ 86 ]. Interestingly, in the Netherlands, children in the household increased men’s but not women’s preferences for scheduling flexibility. In the Netherlands, women with children are more likely to work part time than men (and also more likely to work part time than German women), and for part-timers, scheduling flexibility may be less important. But this is only speculative. In-depth analysis and a thorough theoretical discussion are required to gain a better understanding of these country specific differences.

While our analyses confirmed our expectation that women would have stronger preferences for scheduling flexibility than men, we had mixed results for our broader expectations of men’s stronger preferences for extrinsic job attributes and women’s stronger preferences for intrinsic job attributes. Dutch men considered high pay more important than further training opportunities or scheduling flexibility, but German men showed no statistically significant preferences for either of these job attributes.

Lastly, the gender composition of the team, measured only in the Dutch sample, revealed that men and women both preferred to work in teams with a balanced share of men and women. This did not support our expectation of a preference for homophily among both men and women and might indicate that people dislike being in a minority position. Future studies should look into this in greater detail.

Limitations

Our study contributes to the knowledge on preferences of employees for specific job aspects. Nonetheless, we must acknowledge some limitations. The first limitation is the lack of generalizability of our results. As discrete choice experiments are survey experiments with fictitious choices, it remains unclear whether people would use the same criteria and in the same way when making real-life decisions. Instead of actual behavior, we measured behavioral intentions. A recent study with a discrete choice design looking at job preferences of undergraduates found these preferences were largely related to actual job choices reported four years after graduation [ 11 ].

A second limitation was the difference in effect sizes for the two samples. The effects of the job attributes on the decision on a job offer were much stronger for the German than the Dutch sample. We believe the direction and relative importance of the specific dimensions are solid, and we offer a couple of possible explanations of the differences in effect sizes. First, these differences could reflect the different methodological approaches. We used more dimensions, more job offers, and more decisions per respondent in the Dutch survey. Consequently, the decisions might have been harder for the Dutch respondents, thereby leading to smaller effect sizes. The higher number of missing values on job choices among the Dutch respondents suggests this possibility. Second, the differences might indicate that the phrasing of the different levels of attributes captured the theoretical ideas better in the German context. Third, there may be actual country differences in people’s preference structures related to national norms and/or labor market structures. We encourage scholars to investigate job preference structures from a cross-country comparative perspective to probe this issue.

Conclusions

In this paper, we have shown how the use of discrete choice sets might shed light on people’s preferences for certain job attributes. While our results are largely in line with findings from previous observational studies, they suggest job-seeking behavior is not solely driven by a desire to maximize earnings. Moreover, our finding of gendered preferences suggests women value different job characteristics than men.

We have simply given an overview, and many more questions could be answered with our design. A promising approach would be to focus more specifically on a single job attribute or the social context of respondents. For example, our robustness checks revealed that respondents who were dissatisfied in their current job had a higher preference for training opportunities. Intuitively this makes sense; dissatisfied people may seek avenues to leave their present job. Corroboration of this interpretation requires a sound theoretical foundation and the formulation of specific hypotheses that take different potential causes of job dissatisfaction into account. Our complementary analyses revealed that children in the household moderated respondents’ preferences for scheduling flexibility but affected men and women differently in Germany and the Netherlands. Future studies with a strong theoretical basis focusing on country level explanations and using in-depth analyses could develop our understanding of these country specific peculiarities.

Supporting information

S1 table. example choice set with six job attributes..

https://doi.org/10.1371/journal.pone.0254483.s001

S2 Table. Main effects for German respondents choosing a job offer.

https://doi.org/10.1371/journal.pone.0254483.s002

S3 Table. Main effects for Dutch respondents choosing a job offer.

https://doi.org/10.1371/journal.pone.0254483.s003

S4 Table. Full-interaction models w/ respondent’s gender as moderator.

https://doi.org/10.1371/journal.pone.0254483.s004

S5 Table. Full-interaction models for German respondents w/ children in household as moderator.

https://doi.org/10.1371/journal.pone.0254483.s005

S6 Table. Full-interaction models for Dutch respondents w/ children in household as moderator.

https://doi.org/10.1371/journal.pone.0254483.s006

S7 Table. Full-interaction models w/ respondent’s current employment status as moderator.

https://doi.org/10.1371/journal.pone.0254483.s007

S8 Table. Full-interaction models w/ respondent’s current job satisfaction as moderator.

https://doi.org/10.1371/journal.pone.0254483.s008

• View Article
• Google Scholar
• 8. Mills M. Individualization and the Life Course: Toward a Theoretical Model and Empirical Evidence. In: Howard C, editor. Contested Individualization. New York: Palgrave Macmillan; 2007. pp. 61–79.
• 9. England P, Farkas G. Households, Employment, and Gender: A Social, Economic, and Demographic View. New York: Aldine Pub. Co.; 1986.
• PubMed/NCBI
• 16. Kalleberg AL. Good Jobs, Bad Jobs: The Rise of Polarized and Precarious Employment Systems in the United States, 1970s-2000s. New York: Russell Sage Foundation; 2011.
• 19. Edgell S, Granter E. The Sociology of Work. Continuity and Change in Paid and Unpaid Work. 3rd ed. London: Sage Publications Ltd; 2020.
• 21. Stinchcombe AL. On creating efficient industrial administration. New York: Academic Press; 1974.
• 37. Nolan JP, Wichert IC, Burchell BJ. Job insecurity, psychological well-being and family life. In: Heery E, Salmon J, editors. The insecure workforce. London: Routledge; 2000. pp. 193–221.
• 39. Sauer C, Valet P, Roscigno VJ, Wilson G. Neoliberalism’s Public-Sector Impact: The U.S. and Germany Compared. In: Dwyer R, Warhurst C, Mathieu C, editors. The Oxford Handbook of Job Quality. Oxford: Oxford University Press; 2020. pp. 1–14; in press.
• 54. Charles M, Grusky DB. Occupational Ghettos The Worldwide Segregation of Women and Men. Stanford, CA: Stanford University Press; 2004.
• 61. Kanter RM. Men and Women of the Corporation. New York: Basic Books; 1977.
• 63. OECD. Parental leave: Where are the fathers? Men’s uptake of parental leave is rising but still low. OECD Policy Br. 2016.
• 64. OECD. Unemployment rate (indicator). https://doi.org/10.1787/52570002-en
• 65. OECD. Income inequality (indicator). 2020. https://doi.org/10.1787/459aa7f1-en
• 66. OECD. Temporary employment (indicator). https://doi.org/10.1787/75589b8a-en
• 67. OECD. Part-time employment rate (indicator). 2020. https://doi.org/10.1787/f2ad596c-en
• 72. Auspurg K, Hinz T. Factorial Survey Experiments. Thousand Oaks, CA: Sage Publications Ltd; 2015.
• 76. Amaya-Amaya M, Gerard K, Ryan M. Discrete Choice Experiments in a Nutshell. In: Ryan M, Gerard K, Amaya-Amaya M, editors. Using Discrete Choice Experimentsto Value Health and Health Care. Dordrecht: Springer; 2008. pp. 13–46. https://doi.org/10.1007/978-1-4020-5753-3 _1
• 78. Greene WH. Economitric Analysis. 7th ed. Essex: Pearson; 2012.

Tel: 024.7300.7989 - Phone: 1800.6947 (Thời gian hỗ trợ từ 7h đến 22h)

VIDEO - Lộ trình SUN - Ôn luyện ĐGNL, ĐGTD - 2025

LIVE - Ôn luyện ĐGNL, ĐGTD - 2025

LỚP 12 - LUYỆN THI TN THPT - ĐH, ĐGNL & ĐGTD

T Chính T Tùng T Đăng T Huy T Toản T Nguyên

C Hương Thuỷ C Quỳnh Anh C Phương C Thu Thuỷ

C Xuân C Ngọc Anh T Kiều C Thuỷ C Nga C Phương C Thắng C Phượng

T Toản T Vinh T Bích C Loan T Long

T Tùng T Chất T Duy

T Hải C Kim Anh C Châu T Hiếu

C Thu T Hiển T Quang

T Nam T Phong C Linh C Thanh

Nắm trọn kiến thức lớp 12 Toán, Lí, Hoá

Lớp 12 – Luyện thi TN THPT & ĐH 2024

T Chính T Chí T Cường T Thắng T Nguyên

C Phương T Linh C Thủy

C Xuân C Phượng C Thắng C Phương

T Toản T Thế Anh

T Chất T Tùng

T Hải T Mạnh

T Hòa T Nam T Phong

Lớp 11 - 2K8

T Chính T Tùng T Đăng T Huy T Toản T Chí T Nguyên

T Linh C Thủy C Quỳnh Anh C Loan

C Xuân C Ngọc Anh C Linh C Diễm

T Toản C Loan T Vinh T Long T Bích

T Hải C Châu T Hoạch

T Phong C Thanh C Linh

Tiết kiệm đến 61%

Lớp 10 - 2K9

T Chính T Toản T Nguyên C Hương

C Phương C Hương Thủy C Loan

C Xuân C Huế C Kiều Thắng

T Toản C Loan T Vinh T Bích

T Tùng T Duy T Chất

T Hải T Hoạch

T Nam C Linh C Thanh

NỀN TẢNG LỚP 9 - LUYỆN THI VÀO 10

T Bảo C Loan C Hương T Hải T Hiếu C Yến | C Hải

C Tạ Thuỷ C Lan C Giang C Hương Thuỷ

C Hương C Thảo C Hà C Hoàn C Thắng C Linh | C Nga

T Tùng T Chất T Duy T Sơn T Hoạch C Châu

T Hiển T Quang C Kiều Anh C Thanh

Tiết kiệm đến 66%

Lộ trình 3 bước Học Tốt lớp 9 - Luyện Thi vào 10

Học livestream môn Toán trực tiếp cùng thầy Đỗ Văn Bảo

Lớp 9 – Luyện thi vào 10 - 2023

T Cường T Đông T Bảo

C Hòa C Tạ Thủy C Hương Thủy C Hà

C Hoàn T Long C Linh C Lan

C Loan T Vinh

Lớp 8 - 2K11

T Bảo C Loan C Hải C Yến

C Lan C Hương C Giang

C Hương C Hà C Phương C Thảo C Hoàn

T Quang T Hiển C Thanh C Vân Anh

T Tùng T Chất T Bích C Châu

Tiết kiệm đến 64%

Lớp 7 - 2K12

T Bảo C Nhung C Trang C Yến

C Tạ Thủy C Mai Hương

C Hương C Hà Phương C Vũ Xuân

T Bích T Hoạch T Hải T Tùng T Chất C Trang C Hương

Tiết kiệm đến 68%

Lớp 6 - 2K13

C Thủy C Giang

C Hương C Ngọc Anh C Sinh C Xuân

T Quang T Hiển T Phong Thầy Nam

C Loan T Vinh T Hải T Tùng T Chất

Lớp 5 - 2K14 - 2025

C Mai C Liên T Độ C Nhuần T Thường

T Thảo C Trang C Thuỷ

C Hương C Linh C Hoàn

Tiết kiệm đến 63%

Lớp 4 - 2K15

C Mai C Liên C Ly T Thường

C Thảo C Trang

C Hương C Ngọc

Tiết kiệm đến 58%

Lớp 3 - 2K16

C Ly   C Thủy   T Thường

C Thủy   C Thảo   C Hoa

Tiết kiệm đến 56%

Lớp 2 - 2K17

C Huyền   C Thảo

Tiết kiệm đến 41%

Lớp 1 - 2K18

• LỚP 12 - ÔN THI ĐGNL, ĐGTD
• Lớp 12 - TN THPT&ĐH/ĐGNL/ĐGTD
• Lớp 12 - Luyện thi TN THPT&ĐH
• Mã kích hoạt
• HOT! Khuyến mãi Mua 1 Tặng 1 Toàn Bộ Khoá Học (Từ 18-20/06/2024)
• 2K7! Chính thức Ra mắt Lộ trình Sun 2025! Luyện Thi TN THPT - ĐGNL - ĐGTD
• 2K8! Bứt Phá Lớp 11 2025! Chương trình mới (VOD+LIVE)
• Tuyensinh247 giải nóng đề thi vào lớp 10 năm 2024 - Tất cả các tỉnh
• 2K6! Thi thử Miễn Phí TN THPT 2024
• 2K10! Học trực tuyến lớp 9 và ôn thi vào 10 năm 2024-2025
• 2K14! Khoá học bứt phá lớp 5 năm học 2024-2025
• Khoá học trực tuyến cấp Tiểu học và THCS năm học 2024-2025
• Khoá học trực tuyến cấp THCS năm học 2024 - 2025
• Khoá học trực tuyến cấp Tiểu học năm học 2024-2025
• 2K9 Chú ý! Khoá Học Bứt Phá Nền Tảng Lớp 9, Công Phá Thi Đỗ Lớp 10
• 2K11 Ơi! Bứt Phá Lớp 7 Năm Học 2023 - 2024
• 2K12! Bứt Phá Lớp 6 Năm Học 2023 - 2024
• Chương Trình Học Tốt Trung Học Cơ Sở Năm Học 2023-2024
• Lộ Trình Học Bứt Phá Lớp 2-9 Năm Học 2023-2024
• 2K7! Bứt Phá Lớp 11 2024! Chương trình mới (VOD + LIVE)
• 2K10! Bứt Phá Lớp 8 Năm Học 2023 - 2024
• 2K8! Bứt phá lớp 10! Chương trình mới (VOD + LIVE)
• Chương trình học tốt tiểu học năm học 2023-2024
• 2K13! Bứt Phá Lớp 5 Năm Học 2023 - 2024
• 2K14! Bứt Phá Lớp 4 Năm Học 2023 - 2024
• 2K15! Bứt Phá Lớp 3 Năm Học 2023 - 2024
• 2K16! Bứt Phá Lớp 2 Năm Học 2023 - 2024
• 2K6! Lộ Trình Sun 2024 - Ba bước luyện thi TN THPT - ĐH ít nhất 25 điểm
• Học trực tuyến lớp 11 đủ môn cùng Thầy Cô giỏi, nổi tiếng
• Khai giảng các khóa lớp 9 Toán - Lý - Hóa - Văn - Anh năm 2018
• Khai giảng khóa Ngữ văn 7 - xây nền vững chắc cho tương lai!
• Luyện thi vào lớp 10 môn Toán, Văn, Hóa, Anh, Lý với giáo viên giỏi và nổi tiếng

Dạng bài Đọc hiểu lớp 12

Mark the letter a, b, c or d on your answer sheet to indicate the correct answer to each of the following questions . experiments have shown that in selecting personnel for a job, interviewing is at best a hindrance , and may even cause harm. these studies have disclosed that the judgments of interviewers differ markedly and bear little or no relationship to the adequacy of job applicants. of the many reasons why this should be the case, three in particular stand out. the first reason is related to an error of judgment known as the halo effect. if a person has one noticeable good trait, their other characteristics will be judged as better than they really are. thus, an individual who dresses smartly and shows self-confidence is likely to be judged capable of doing a job well regardless of his or her real ability. interviewers are also prejudiced by an effect called the primacy effect. this error occurs when interpretation of later information is distorted by earlier connected information. hence, in an interview situation, the interviewer spends most of the interview trying to confirm the impression given by the candidate in the first few moments. studies have repeatedly demonstrated that such an impression is unrelated to the aptitude of the applicant. the phenomenon known as the contrast effect also skews the judgment of interviewers. a suitable candidate may be underestimated because he or she contrasts with a previous one who appears exceptionally intelligent. likewise, an average candidate who is preceded by one who gives a weak showing may be judged as more suitable than he or she really is. since interviewers as a form of personnel selection have been shown to be inadequate, other selection procedures have been devised which more accurately predict candidate suitability. of the various tests devised, the predictor which appears to do this most successfully is cognitive ability as measured by a variety of verbal and spatial tests.  .

Mark the letter A, B, C or D on your answer sheet to indicate the correct answer to each of the following questions .

Experiments have shown that in selecting personnel for a job, interviewing is at best a hindrance , and may even cause harm. These studies have disclosed that the judgments of interviewers differ markedly and bear little or no relationship to the adequacy of job applicants. Of the many reasons why this should be the case, three in particular stand out. The first reason is related to an error of judgment known as the halo effect. If a person has one noticeable good trait, their other characteristics will be judged as better than they really are. Thus, an individual who dresses smartly and shows self-confidence is likely to be judged capable of doing a job well regardless of his or her real ability.

Interviewers are also prejudiced by an effect called the primacy effect. This error occurs when interpretation of later information is distorted by earlier connected information. Hence, in an interview situation, the interviewer spends most of the interview trying to confirm the impression given by the candidate in the first few moments. Studies have repeatedly demonstrated that such an impression is unrelated to the aptitude of the applicant.

The phenomenon known as the contrast effect also skews the judgment of interviewers. A suitable candidate may be underestimated because he or she contrasts with a previous one who appears exceptionally intelligent. Likewise, an average candidate who is preceded by one who gives a weak showing may be judged as more suitable than he or she really is.

Since interviewers as a form of personnel selection have been shown to be inadequate, other selection procedures have been devised which more accurately predict candidate suitability. Of the various tests devised, the predictor which appears to do this most successfully is cognitive ability as measured by a variety of verbal and spatial tests.

Câu 1: This passage mainly discusses the _______ .

A.  inadequacy of interviewing job applicants

B. techniques that interviewers use for judging job applicants

C. effect of interviewing on job applicants

D. judgments of interviewers concerning job applicants 

• Xem lời giải

Giải chi tiết:

Mẹo : Viết lời giải với bộ công thức đầy đủ tại đây

0 Bình luân

Câu 2: The word ‘ hindrance ’ is closest meaning to _______ .

A. procedure

B.  encouragement   

C. assistance 

D. interference

Câu 3: The word ’they ’ refers to _______ .

A. characteristics       

B. judgments 

C. applicants         

D. interviewers

Câu 4: According to the passage, the halo effect _______ .

A.  takes effect only when a candidate is well dressed

B. exemplifies how one good characteristic colors perceptions

C.  stands out as the worst judgmental error

D. helps the interviewer’s capability to judge real ability 

Câu 5: According to the passage, the first impression _______ .  

A.  has been repeatedly demonstrated to the applicant

B. is unrelated to the interviewer’s prejudice

C. is the one that stays with the interviewer

D. can easily be altered

Câu 6: The word ‘ skews ’ is similar to _______.

A. improves  

B. distinguishes

C. biases   

Câu 7: The word ‘ this ’ refers to_______ .

A. predict candidate suitability

B. devise personnel selection

C. devise accurate tests

D.  measure cognitive ability 

Câu 8: The author mentions all of the following reasons why interviewing is not an accurate way to predict candidate suitability EXCEPT the _________ .

A. primacy effect  

B. contrast effect  

C.  halo effect   

D. cognitive effect

Câu 9: The paragraphs following the passage most likely discuss which of the following?

A. More information on cognitive ability tests              

B.  Other selection procedures included in interviewing

C. Other reasons for misjudgments of applicants  

D. More information on the kinds of judgmental effects

Câu 10: Where in the passage does the author discuss the effect of comparing two candidates?  

A. paragraph 3  

B. paragraph 4  

C.  paragraph 2

D. paragraph 1

Tham Gia Group Dành Cho 2K7 luyện thi Tn THPT - ĐGNL - ĐGTD

>> Lộ Trình Sun 2025 - 3IN1 - 1 lộ trình ôn 3 kì thi (Luyện thi TN THPT & ĐGNL; ĐGTD) tại Tuyensinh247.com. Đầy đủ theo 3 đầu sách, Thầy Cô giáo giỏi, 3 bước chi tiết: Nền tảng lớp 12; Luyện thi chuyên sâu; Luyện đề đủ dạng đáp ứng mọi kì thi.

Câu hỏi trước Câu tiếp theo

Hỗ trợ - Hướng dẫn

• Tel: 024.7300.7989

(Thời gian hỗ trợ từ 7h đến 22h ) Email: [email protected]

Đăng ký tài khoản

Nạp tiền vào tài khoản

Cơ quan chủ quản: Công ty Cổ phần công nghệ giáo dục Thành Phát

• CS mua khóa học
• CS trả và đổi khóa học
• CS dữ liệu cá nhân
• CS bảo đảm sản phẩm
• Hình thức thanh toán
• CS bảo đảm Live Pro 9+

Tel: 024.7300.7989 - Hotline: 1800.6947

Email: [email protected]

Văn phòng: Tầng 7 - Tòa nhà Intracom - Số 82 Dịch Vọng Hậu - Cầu Giấy - Hà Nội

Giấy phép cung cấp dịch vụ mạng xã hội trực tuyến số 337/GP-BTTTT do Bộ Thông tin và Truyền thông cấp ngày 10/07/2017. Giấy phép kinh doanh: MST-0106478082 do Sở Kế hoạch và Đầu tư cấp ngày 05/04/2023 (Lần 5). Chịu trách nhiệm nội dung: Phạm Đức Tuệ.

Gọi ngay 18006947

Chat trực tiếp với tư vấn viên

Chat qua facebook Messenger

Chat với chúng tôi qua Zalo

Chapter 6: Experimental Research

6.3 conducting experiments, learning objectives.

• Describe several strategies for recruiting participants for an experiment.
• Explain why it is important to standardize the procedure of an experiment and several ways to do this.
• Explain what pilot testing is and why it is important.

The information presented so far in this chapter is enough to design a basic experiment. When it comes time to conduct that experiment, however, several additional practical issues arise. In this section, we consider some of these issues and how to deal with them. Much of this information applies to nonexperimental studies as well as experimental ones.

Recruiting Participants

Of course, you should be thinking about how you will obtain your participants from the beginning of any research project. Unless you have access to people with schizophrenia or incarcerated juvenile offenders, for example, then there is no point designing a study that focuses on these populations. But even if you plan to use a convenience sample, you will have to recruit participants for your study.

There are several approaches to recruiting participants. One is to use participants from a formal subject pool —an established group of people who have agreed to be contacted about participating in research studies. For example, at many colleges and universities, there is a subject pool consisting of students enrolled in introductory psychology courses who must participate in a certain number of studies to meet a course requirement. Researchers post descriptions of their studies and students sign up to participate, usually via an online system. Participants who are not in subject pools can also be recruited by posting or publishing advertisements or making personal appeals to groups that represent the population of interest. For example, a researcher interested in studying older adults could arrange to speak at a meeting of the residents at a retirement community to explain the study and ask for volunteers.

The Volunteer Subject

Even if the participants in a study receive compensation in the form of course credit, a small amount of money, or a chance at being treated for a psychological problem, they are still essentially volunteers. This is worth considering because people who volunteer to participate in psychological research have been shown to differ in predictable ways from those who do not volunteer. Specifically, there is good evidence that on average, volunteers have the following characteristics compared with nonvolunteers (Rosenthal & Rosnow, 1976):

• They are more interested in the topic of the research.
• They are more educated.
• They have a greater need for approval.
• They have higher intelligence quotients (IQs).
• They are more sociable.
• They are higher in social class.

This can be an issue of external validity if there is reason to believe that participants with these characteristics are likely to behave differently than the general population. For example, in testing different methods of persuading people, a rational argument might work better on volunteers than it does on the general population because of their generally higher educational level and IQ.

In many field experiments, the task is not recruiting participants but selecting them. For example, researchers Nicolas Guéguen and Marie-Agnès de Gail conducted a field experiment on the effect of being smiled at on helping, in which the participants were shoppers at a supermarket. A confederate walking down a stairway gazed directly at a shopper walking up the stairway and either smiled or did not smile. Shortly afterward, the shopper encountered another confederate, who dropped some computer diskettes on the ground. The dependent variable was whether or not the shopper stopped to help pick up the diskettes (Guéguen & de Gail, 2003). Notice that these participants were not “recruited,” but the researchers still had to select them from among all the shoppers taking the stairs that day. It is extremely important that this kind of selection be done according to a well-defined set of rules that is established before the data collection begins and can be explained clearly afterward. In this case, with each trip down the stairs, the confederate was instructed to gaze at the first person he encountered who appeared to be between the ages of 20 and 50. Only if the person gazed back did he or she become a participant in the study. The point of having a well-defined selection rule is to avoid bias in the selection of participants. For example, if the confederate was free to choose which shoppers he would gaze at, he might choose friendly-looking shoppers when he was set to smile and unfriendly-looking ones when he was not set to smile. As we will see shortly, such biases can be entirely unintentional.

Standardizing the Procedure

It is surprisingly easy to introduce extraneous variables during the procedure. For example, the same experimenter might give clear instructions to one participant but vague instructions to another. Or one experimenter might greet participants warmly while another barely makes eye contact with them. To the extent that such variables affect participants’ behavior, they add noise to the data and make the effect of the independent variable more difficult to detect. If they vary across conditions, they become confounding variables and provide alternative explanations for the results. For example, if participants in a treatment group are tested by a warm and friendly experimenter and participants in a control group are tested by a cold and unfriendly one, then what appears to be an effect of the treatment might actually be an effect of experimenter demeanor.

Experimenter’s Sex as an Extraneous Variable

It is well known that whether research participants are male or female can affect the results of a study. But what about whether the experimenter is male or female? There is plenty of evidence that this matters too. Male and female experimenters have slightly different ways of interacting with their participants, and of course participants also respond differently to male and female experimenters (Rosenthal, 1976). For example, in a recent study on pain perception, participants immersed their hands in icy water for as long as they could (Ibolya, Brake, & Voss, 2004). Male participants tolerated the pain longer when the experimenter was a woman, and female participants tolerated it longer when the experimenter was a man.

Researcher Robert Rosenthal has spent much of his career showing that this kind of unintended variation in the procedure does, in fact, affect participants’ behavior. Furthermore, one important source of such variation is the experimenter’s expectations about how participants “should” behave in the experiment. This is referred to as an experimenter expectancy effect (Rosenthal, 1976). For example, if an experimenter expects participants in a treatment group to perform better on a task than participants in a control group, then he or she might unintentionally give the treatment group participants clearer instructions or more encouragement or allow them more time to complete the task. In a striking example, Rosenthal and Kermit Fode had several students in a laboratory course in psychology train rats to run through a maze. Although the rats were genetically similar, some of the students were told that they were working with “maze-bright” rats that had been bred to be good learners, and other students were told that they were working with “maze-dull” rats that had been bred to be poor learners. Sure enough, over five days of training, the “maze-bright” rats made more correct responses, made the correct response more quickly, and improved more steadily than the “maze-dull” rats (Rosenthal & Fode, 1963). Clearly it had to have been the students’ expectations about how the rats would perform that made the difference. But how? Some clues come from data gathered at the end of the study, which showed that students who expected their rats to learn quickly felt more positively about their animals and reported behaving toward them in a more friendly manner (e.g., handling them more).

The way to minimize unintended variation in the procedure is to standardize it as much as possible so that it is carried out in the same way for all participants regardless of the condition they are in. Here are several ways to do this:

• Create a written protocol that specifies everything that the experimenters are to do and say from the time they greet participants to the time they dismiss them.
• Create standard instructions that participants read themselves or that are read to them word for word by the experimenter.
• Automate the rest of the procedure as much as possible by using software packages for this purpose or even simple computer slide shows.
• Anticipate participants’ questions and either raise and answer them in the instructions or develop standard answers for them.
• Train multiple experimenters on the protocol together and have them practice on each other.
• Be sure that each experimenter tests participants in all conditions.

Another good practice is to arrange for the experimenters to be “blind” to the research question or to the condition that each participant is tested in. The idea is to minimize experimenter expectancy effects by minimizing the experimenters’ expectations. For example, in a drug study in which each participant receives the drug or a placebo, it is often the case that neither the participants nor the experimenter who interacts with the participants know which condition he or she has been assigned to. Because both the participants and the experimenters are blind to the condition, this is referred to as a double-blind study. (A single-blind study is one in which the participant, but not the experimenter, is blind to the condition.) Of course, there are many times this is not possible. For example, if you are both the investigator and the only experimenter, it is not possible for you to remain blind to the research question. Also, in many studies the experimenter must know the condition because he or she must carry out the procedure in a different way in the different conditions.

Record Keeping

It is essential to keep good records when you conduct an experiment. As discussed earlier, it is typical for experimenters to generate a written sequence of conditions before the study begins and then to test each new participant in the next condition in the sequence. As you test them, it is a good idea to add to this list basic demographic information; the date, time, and place of testing; and the name of the experimenter who did the testing. It is also a good idea to have a place for the experimenter to write down comments about unusual occurrences (e.g., a confused or uncooperative participant) or questions that come up. This kind of information can be useful later if you decide to analyze sex differences or effects of different experimenters, or if a question arises about a particular participant or testing session.

It can also be useful to assign an identification number to each participant as you test them. Simply numbering them consecutively beginning with 1 is usually sufficient. This number can then also be written on any response sheets or questionnaires that participants generate, making it easier to keep them together.

Pilot Testing

It is always a good idea to conduct a pilot test of your experiment. A pilot test is a small-scale study conducted to make sure that a new procedure works as planned. In a pilot test, you can recruit participants formally (e.g., from an established participant pool) or you can recruit them informally from among family, friends, classmates, and so on. The number of participants can be small, but it should be enough to give you confidence that your procedure works as planned. There are several important questions that you can answer by conducting a pilot test:

• Do participants understand the instructions?
• What kind of misunderstandings do participants have, what kind of mistakes do they make, and what kind of questions do they ask?
• Do participants become bored or frustrated?
• Is an indirect manipulation effective? (You will need to include a manipulation check.)
• Can participants guess the research question or hypothesis?
• How long does the procedure take?
• Are computer programs or other automated procedures working properly?
• Are data being recorded correctly?

Of course, to answer some of these questions you will need to observe participants carefully during the procedure and talk with them about it afterward. Participants are often hesitant to criticize a study in front of the researcher, so be sure they understand that this is a pilot test and you are genuinely interested in feedback that will help you improve the procedure. If the procedure works as planned, then you can proceed with the actual study. If there are problems to be solved, you can solve them, pilot test the new procedure, and continue with this process until you are ready to proceed.

Key Takeaways

• There are several effective methods you can use to recruit research participants for your experiment, including through formal subject pools, advertisements, and personal appeals. Field experiments require well-defined participant selection procedures.
• It is important to standardize experimental procedures to minimize extraneous variables, including experimenter expectancy effects.
• It is important to conduct one or more small-scale pilot tests of an experiment to be sure that the procedure works as planned.
• Practice: List two ways that you might recruit participants from each of the following populations: (a) elderly adults, (b) unemployed people, (c) regular exercisers, and (d) math majors.
• Discussion: Imagine a study in which you will visually present participants with a list of 20 words, one at a time, wait for a short time, and then ask them to recall as many of the words as they can. In the stressed condition, they are told that they might also be chosen to give a short speech in front of a small audience. In the unstressed condition, they are not told that they might have to give a speech. What are several specific things that you could do to standardize the procedure?

Guéguen, N., & de Gail, Marie-Agnès. (2003). The effect of smiling on helping behavior: Smiling and good Samaritan behavior. Communication Reports, 16 , 133–140.

Ibolya, K., Brake, A., & Voss, U. (2004). The effect of experimenter characteristics on pain reports in women and men. Pain, 112 , 142–147.

Rosenthal, R. (1976). Experimenter effects in behavioral research (enlarged ed.). New York, NY: Wiley.

Rosenthal, R., & Fode, K. (1963). The effect of experimenter bias on performance of the albino rat. Behavioral Science, 8 , 183-189.

Rosenthal, R., & Rosnow, R. L. (1976). The volunteer subject . New York, NY: Wiley.

• Research Methods in Psychology. Provided by : University of Minnesota Libraries Publishing. Located at : http://open.lib.umn.edu/psychologyresearchmethods . License : CC BY-NC-SA: Attribution-NonCommercial-ShareAlike

Privacy Policy

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Psychol

Predictive Validity of Interviewer Post-interview Notes on Candidates’ Job Outcomes: Evidence Using Text Data From a Leading Chinese IT Company

Shanshi liu.

1 School of Business Administration, South China University of Technology, Guangzhou, China

2 Business School, Guilin University of Technology, Guilin, China

Yuanzheng Chang

Jianwu jiang.

3 College of Management, Shenzhen University, Shenzhen, China

4 Tencent Holdings Limited, Shenzhen, China

Huaikang Zhou

Associated data.

The data will not be available to public since they are proprietary data that belong to the company.

Despite the popularity of the employment interview in the employee selection literature and organizational talent selection process, few have examined the comments interviewers give after each interview. This study investigated the predictability of the match between interviewer post-interview notes and radar charts from job analysis on the candidate’s later career performance using text mining techniques and data from one of the largest internet-based technology companies in China. A large sample of 7,650 interview candidates who passed the interviews and joined the company was obtained to show that the number of job-related capabilities interviewers mentioned in their notes was positively related to candidate’s job performance, the number of promotions, and negatively related to turnover. Moreover, the dimensions of the radar chart from job analysis covered in the interview moderated the predictability of interview post-interview notes. Our results indicated that a smaller number of radar chart dimensions by which interviewers assessed the candidates in the interview positively moderated candidates’ promotion for product development jobs, and negatively moderated turnover for technical jobs. The implications of these results for structured interview research in both theory and practice are discussed.

Introduction

The employment interview is a proven and popular selection method that has drawn continuous attention from researchers for more than 100 years ( Macan, 2009 ). It is often used to assess the fit of the candidate to the employer and is shown to have high predictive validity for job performance ( Huffcutt et al., 2001 ). One of the most consistent findings of selection interviews is that structured interviews are more reliable and valid than unstructured interviews ( Levashina et al., 2014 ). Extant literature focuses on the factors that influence interviewers’ adoption of interview structure and the structured interview process itself, trying to give better guidance in terms of how to increase its adoption and the boundary of when to use it. For example, Chapman and Zweig (2005) find that interviewer training, interview focus, interviewer reactions to interview structure, and applicant reaction to interview structure lead to different levels of interview structure and how interviewees react to them. Campion et al. (1997) identified 15 components of the interview structure and divided them into two categories: what to ask and how to evaluate. The reliability, validity, and user reactions of these components were assessed. Huffcutt and Arthur (1994) proposed a framework that defined interview structure with similar categories: standardized interview questions, and standardized response evaluation. Their framework divided interview structure into four levels, with level 1 being structure with no formal constraints to level 4 being asking the same questions with no modifications allowed. Particularly, level 2 was defined as “limited constraints, typically standardization of the topical areas to be covered.” However, interview structure is still often utilized dichotomously in many studies, mostly comparing high interview structure with no structure. Despite this framework, few studies tried to explore the level 2 interview structure on the outcome of a candidate’s job performance.

Among the many components of interview structure, job analysis is utilized most frequently in interview structure research for interview questions content dimension ( Levashina et al., 2014 ). Job analysis serves as a foundation of selection in human resources and one of the two main goals of job analysis is to find out what kind of worker fits best to certain jobs ( Sanchez and Levine, 2012 ). For the job analysis component, finding accurate job requirements is the key to develop relevant interview questions to assess a candidate’s capabilities. Collecting critical job incidents with job experts, such as managers and interviewers, is the most common method ( Morgeson et al., 2016 ). However, the development of job-related structured interview questions in previous research is often carried out by researchers, not professionals in the field ( Campion et al., 1997 ), therefore not reflecting the full picture of this process in real situations.

On the other hand, interview structure can be grouped by how note-taking is required in the interview process, such as whether notes are taken during or after the interview or what to write in the notes. Campion et al. (1997) suggest that taking notes during the interview is considered a higher structure level than after the interview. On the other hand, taking notes in great detail during the interview is considered the highest level, while note-taking without specific instructions about when and how to write notes being the less constrained structure. Although studies have shown that taking notes during or after an interview helps interviewers organize their thoughts, thus making better judgments about the candidate, few have tried to explore the content of these notes and how it relates to the candidate’s job performance.

Apart from the interview structure, the Person-Job (PJ) fit theory suggests that finding the match between the job candidate’s attributes and the job requirements is the cornerstone of a successful selection ( Barrick and Parks-Leduc, 2019 ). Hu et al. (2016) show that supervisors’ Person-Organization fit perceptions are positively related to new hires’ performance for executives in a Fortune 500 company. How the fit between the candidate and the job predicts a candidate’s future job performance remains an unanswered question ( Posthuma et al., 2002 ). Interview validity is shown to increase if questions asked during a structured interview are job-related ( Wiesner and Cronshaw, 1988 ) because it can better assess if the candidate fits the job requirements. However, only indirect effects have been found between job analysis and interview validity ( Conway et al., 1995 ). Extant research mostly uses cognitive test results as the dependent variable to show the validity of using job analysis in structured interviews ( Thorsteinson, 2018 ). These cognitive performance scores are then linked to the candidate’s job performance. However, whether using job analysis as an interview component directly affects interview validity requires further exploration.

This paper aims to investigate the direct effect of interviewer notes on interview validity, while focusing on the role of job analysis as an interview structure component. The notes we use in our study are the narrative comments interviewers write after each interview about what they think of the candidate for later reference as required by the company. What interviewers say about the candidate reflects the questions interviewers asked or cared the most about during the interview because the comments consist of the evaluation of the answers candidate provided during the interview. This is shown by the appearance of job-related keywords in their notes. For example, if an interviewer wants to know whether the candidate has good leadership skills, he/she may ask the candidate through direct or indirect questions, such as “how do you lead a team” or “tell me an event you organized.” The interviewer would write down how he/she thought about the candidate’s performance in the note with the keyword leadership skills. Examining these keywords allows us to get insights about what the interviewer what to know during that interview. Specifically, we examine: (1) whether the match between the interviewer’s post-interview note about the candidate’s interview performance and the job requirements of the position will predict the candidate’s job performance, promotion, and turnover and (2) how interviewers could utilize the radar chart from job-analysis to better assess interviewees for different types of jobs.

Our study extends the interview literature in three ways. First, we draw from interviewers’ narrative comments in their post-interview notes in actual scenarios to reflect the questions interviewers go through during the interview, which shows the direct effect of interview notes on interview validity. Moreover, while PJ fit is shown to predict higher job satisfaction, better job performance, and lower turnover rate ( Kristof-Brown et al., 2005 ), our study also includes candidates’ number of promotions, making it a well-rounded measurement of fit and interview validity.

Second, the practices the company used in our study required interviewers to assess the candidate using their way of question-asking and evaluation without following a strict structure. The questions interviewers asked during an interview were either self-reported or pre-determined in previous studies, which might not reveal the situation in an actual interview. Our study puts less emphasis on the structure of interviews in favor of what interviewers know about the candidate’s job-related capabilities by examing the content of interviewer narrative comments in post-interview notes with the radar chart from job analysis rather than what structure interviewers follow during the interview. This is similar to the level 2 of interview structure proposed by Huffcutt and Arthur (1994) such that radar charts serve only as a framework for interview questions without providing exact questions to be asked. With the results of this study, interviewers and practitioners could use the scores and attributes of the job analysis result in interviews as a reference for their questions without sticking to pre-determined questions to evaluate candidates. This approach extends structured interview research by showing how level 2 of both dimensions proposed by Huffcutt and Arthur (1994 , p. 186), “characterized by limited constraints, typical standardization of the topical areas to be covered,” affects interview validity.

Third, despite the emergence of the big data movement in other management contexts ( Kobayashi et al., 2018 ), studies on employment interviews have mostly maintained the use of traditional datasets and methods. Besides, many studies on interview validity use mock interviews or experiments to test their hypotheses, which have been shown to have lower validity than real job interviews ( Posthuma et al., 2002 ). Our study uses text analysis and natural language processing methods with data generated in real practices to explore the content of what interviewers say after the interview, which extends the use of text mining in explanatory research/hypotheses testing in employment interview literature ( Kobayashi et al., 2018 ).

Conceptual Background and Hypotheses Development

The match between interviewers’ narrative comments and radar charts dimensions on the interview’s predictive validity.

Whether interviews can predict a candidate’s job performance has long been of interest to researchers and professionals. Huffcutt et al. (2001) find from multiple meta-analyses that interview is a valid selection method for candidates’ job performance prediction. One of the main research areas in selection interviews is the effect of interview structure on interview validity. Job analysis is one of the components used most frequently in structured interviews ( Levashina et al., 2014 ). One of the essential roles of job analysis is to figure out the work attributes required to perform well on the job ( Sanchez and Levine, 2001 ). A job analysis reveals the requirements of the job and therefore helps identify a matched candidate that fits the role complementarily ( Muchinsky and Monahan, 1987 ). It serves as the foundation on which interviewers and professionals can base their interview questions, which is shown to increase interview validity ( Wiesner and Cronshaw, 1988 ). Radar charts with job requirements and their relative weights were developed by experts from the company as references for employee selection and improvement.

In our study, job analysis is done by company human resource business partners who understand both the business objectives and human resource practices. They first discuss the goal of each position in the business unit with the unit leader to determine the capabilities needed and a weight is assigned to each of these capabilities. Greater weight means a higher likelihood to contribute to a better performance grade. A radar chart of each of the positions is then generated to make it easier for others to understand and utilize ( Figure 1 ). More specifically, the radar chart here in our study is a chart with capabilities needed for the position and their weights arranged around a circle with lines plotted from the center to the edge. The longer the line the higher the weight of the capability with the name indicated at the edge. One of the advantages of radar charts is that they allow users to evaluate the capabilities and weights simultaneously, which is supposed to help interviewers form their interview questions. By using the radar chart together with proper interview training on how to ask questions, interviewers are more likely to focus on the capabilities needed for the position and ask the pivotal questions.

Job-analysis diagram example downloaded from company website.

Another important component of the interview structure is notes written by interviewers. Interviewer notes contain rich text data about what they think of the candidate’s performance during the interview. Text data, a new kind of qualitative data, has become available to researchers in human resources (e.g., Platanou et al., 2018 ), thanks to the development of information technology. In particular, narrative comments from supervisors have shown their validity in performance appraisal literature as a new data source. Comparing to traditional performance ratings, comments in notes offer more contextual information and require a higher cognitive process to write so that are less vulnerable to social and cultural rating impacts ( Brutus, 2010 ). Moreover, some articles have shown the validity of using text data in personnel selection study, such as candidate essays (e.g., Campion et al., 2016 ) and admission interviews (e.g., Elam et al., 1994 ). Given the proximity between performance appraisal and selection interview in human resource management research under the broad category of assessing people ( Markoulli et al., 2017 ), interviewers’ narrative comments in their notes after the interview about a candidate’s performance provide a different and unique perspective on selection interview research.

One of the purposes of selection interviews is to assess the fit, where PJ fit refers to the compatibility between the candidate’s characteristics and the requirements of the job ( Kristof-Brown et al., 2005 ). The framework of PJ fit proposed by Edwards (1991) consists of two classes: needs and supplies (NS) fit, and demands and abilities (DA) fit. NS fit focuses on the desire of the employee and what the job can provide. DA fit focuses on job demands and what the employee’s capabilities have to offer. Interviewers assess candidates’ job-related capabilities by asking job analysis related questions, such as using capabilities chart or opinion from experts as references. Although the narrative comments in the notes may not reveal the actual opinion of the interviewer on the aspect of the job requirements radar chart, it is reasonable to assume that the candidate has met the criteria since our data only have the candidates who passed the interview. Because of the different conceptualization, NS fit is often utilized in studies to examine its relationship with job satisfaction, job stress, and motivation, while DA fit is utilized as a predictor of job performance, promotion, and retention ( Edwards, 1991 ). Boon and Biron (2016) find that higher DA fit predicts higher turnover intention when leader-member exchange (LMX) quality is high. Kristof-Brown et al. (2005) show in their meta-analysis that PJ fit is a valid predictor for job performance and turnover, where the correlation coefficient is positive and negative, respectively. There are three common measurement methods for PJ fit: perceived fit, subjective fit, and objective fit. Interviews serve as a crucial tool to assess the fit of a candidate before they are selected into the organization. Interviewers can assess the match between a candidate’s values and the organization with high accuracy ( Cable and Judge, 1997 ).

Interview narrative comments in our study are the content interviewers write down in their notes after an interview to record what they think of the candidate. They serve as a tool for interviewers to give a review of the candidate’s response to the questions the interviewer asked during the interview. Since interviewer narrative comments are a kind of performance appraisal that assesses people’s behaviors, they can be scored into different performance dimensions ( Speer et al., 2018 ). Interviewers in our research are required to write notes after each interview by their human resources system. The notes are without format or content restriction. Interviewers write down their impression of the candidate’s interview performance and answers to their questions to support their decision on whether the candidate pass or fail. They are expected to write in more detail. By investigating the dimensions covered in the comments, we will be able to know what qualities of the candidate the interviewer has examined during the interview. If the interviewer asked job-related questions during the interview and was able to assess the fit between the candidate and job requirements, the interview will better predict the candidate’s job performance, promotion, and turnover. Therefore, we proposed the following hypotheses:

• H1 : The match between the job requirements radar chart aspects and the interviewer’s narrative comments on the candidate positively relates to the candidate’s (a) job performance, (b) the number of promotions, and negatively relates to the candidate’s (c) turnover.

The Effect of the Breadth of Interview Questions on the Matching Score Between Interviewers’ Narrative Comments and Radar Chart Dimensions on the Interview’s Predictive Validity

Although using the radar chart from job analysis to construct interview questions increases interview validity, how to use the chart more effectively needs further investigation. Since longer interviews cost the organization more ( Thorsteinson, 2018 ), interviewers are not likely to spend much time on an interview. Within a limited length of interview time, interviewers can either go broader to cover more aspects of the radar chart from job analysis and get a comprehensive impression of the candidate or select certain aspects to get a deeper understanding of the candidate’s capabilities. Due to the trade-offs between time and effort, it would be hard for individuals to have broad knowledge domains and be specialized in all of them at the same time. The breadth and depth of employee knowledge are utilized as the two ends of one dimension ( Schilling et al., 2003 ). Since the job requirements across positions vary, and the depth and breadth of knowledge affect how employees perform on the job, the choice of interviewers’ questioning strategy may have different effects on interview validity.

To compare the breadth effect across different job groups, we used all the entry-level jobs of an internet-based technology company that were different in roles and job requirements. The company has divided its jobs into five categories, called Clans: Technical, Product/Project, Marketing, Specialist, and Design. As these names suggest, jobs are grouped into different Clans based on their skill requirements. For example, jobs in the Technical Clan are mainly engineers that require programming and computer skills, whereas jobs in the Marketing Clan are jobs that require business skills for the company’s sales of products and business expansion. Boh et al. (2014) found in their research that the breadth of employee expertise leads to a higher number of inventions, while the depth of expertise is more helpful for inventions that are more complicated and technical. Employees with deeper knowledge would be able to recombine their existing knowledge to have a significant impact on technological change ( Fleming, 2001 ).

Technical jobs often involve abstract problem solving and logical reasoning, therefore a deeper understanding of the tools and logic of completing the tasks will allow employees to perform better. Since our data were from an internet-based technology company whose products are mainly social media platforms and games for consumers where competition is intense, the company needs the products they develop to stand out and differentiate themselves from other competitors. This product development process requires product managers to have a comprehensive and deep understanding of the market and user base, where the depth of knowledge is crucial. Positions in the Specialists Clan include many functions such as legal, investor relations, and enterprise management. These jobs are highly specialized and often require employees to participate in training courses or to obtain licenses before being allowed to perform. For jobs in these Clans, interviewers should focus more on the specialized capability requirements and narrow down their attribute dimensions on the radar charts to get a more detailed understanding of the candidate’s proficiencies to complete specific tasks. Therefore, we proposed the following hypotheses:

• H2 : The relationship between the appearance of radar chart related keywords in the interviewer’s narrative comments and the candidate’s job performance is moderated by the number of dimensions the interview covered, such that the fewer the dimensions, the (a) better performance score, (b) higher number of promotions, and (c) lower turnover probability of the candidate for the Technical, Product/Project, and Specialists Clans.

On the other hand, the breadth of employees’ knowledge enables them to restructure their expertise in innovative ways ( Boh et al., 2014 ). Simonton (2003) finds in his study that the higher number and variety of different cognitive elements available for association increase the probability of generating creative combinations. His result indicates that breadth of knowledge helps individuals to have new ideas and be more creative. This is especially important for employees in the Design Clan, whose jobs are drawing and graphic design for games and social media applications. The design process usually involves recombining different elements and features from different domains to create something new. Thus, employees’ breadth of knowledge could help expand their search domains and come out with different features and styles to fit the themes of different applications.

Jobs in the Marketing Clan are mostly roles in the business operation of the company, such as marketing, sales, and strategic planning. These jobs do not require very technical or specialized skills but do call for the ability to come out with new ideas for the changing business environment. Besides, business operations often involve different parties and their interests. Solving business problems requires building connections with these parties to communicate effectively. Since these parties are different and their interests vary, the breadth of knowledge will help employees to increase the variety of ideas to handle different scenarios and novel business problems ( Greve and Taylor, 2000 ). Because the nature of the jobs in the Design and Marketing Clans is different and requires more innovation but less technical know-how, the breadth of knowledge is more critical here than the other three Clans. Therefore, we proposed the following hypotheses:

• H3 : The relationship between the appearance of radar chart related capabilities keywords in the interviewer’s narrative comments and the candidate’s job performance is moderated by the number of dimensions the interview covered, such that the more the dimensions, the (a) better performance score, (b) higher number of promotions, and (c) lower turnover probability of the candidate for the Design and Marketing Clans.

Materials and Methods

Data and procedure.

We tested our hypotheses using human resources data from one of the top internet-based technology companies in China. This company has a well-built data infrastructure that supports data from many human resource departments and functions, including recruitment, assessment, training, compensation, and turnover management. We were able to link these different systems using the unique id number of each candidate that has been selected into the company after the interviews.

All the jobs in the company are divided into five Clans mentioned above. The hierarchy is Clan – Class – Position. For example, a typical hierarchy could be Technical Clan – Technology Research and Development Class – Web Front-End Development Position. Also, each position can be further classified into different Grades and Ranks. There are 6 Grades in total, which follow the hierarchy as Entry – Intermediate – Specialist – Expert – Master – Fellow, and are labeled 1–6. Furthermore, each Grade has three Ranks: Basic, Regular, and Professional. For any employee to be promoted to the next Grade, he/she has to go through the Ranks one by one, from Basic to Professional. For each Rank, there is a dedicated capability radar chart from job analysis that is used as a training reference for both the supervisor and employee to understand the requirements of the position at that level. An example of the diagram is given below ( Figure 1 ).

All candidates seeking a job in this company have to be interviewed. This company has an interviewer scheme program where employees have to go through a training class before being allowed to interview job applicants. The training class aims to familiarize employees with the interview process and remind them to not ask interview questions that may impose legal issues such as questions related to gender, marital status, and age. In the training materials, we obtained from the company’s internal knowledge-sharing platform, it specified that interviewers should adopt the Situation, Task, Action, Result (STAR) interview method to assess interviewees’ capabilities and experiences. This method requires interviewers to be less specific and ask more open-ended questions so that they can listen to and observe more carefully the answers given by the interviewee. After each interview, interviewers are required to write down their thoughts about the candidate. There is no pre-specified structure of the comments; interviewers can determine what to write and in what detail, although giving clear and concise comments is encouraged. This requirement together with the STAR interview method makes the interview comments reasonable evidence of what capabilities and experiences the interviewer assessed the candidate during the interview and in what detail.

The sampling frame for this study consisted of 7,650 candidates who had gone through the selection process and had become a formal employee of the company from 01 July 2016 to 01 July 2018. These candidates applied for various positions, including 3660 (47.8%) candidates for Technical Clan, 1825 (23.9%) candidates for Product/Project Clan, 1268 (16.6%) candidates for Marketing Clan, 449 (5.9%) candidates for Specialist Clan, and 447 (5.9%) candidates for Design Clan. The average age of the candidates is 31.16, with 75.4% male. The distribution of the job applications for each Clan reflects the actual employee composition of the company, as programmers and product managers are the most needed talents in an internet-based company whose products are mainly social media applications and online games.

Performance Rating

Employees receive a job performance rating from their immediate supervisor biannually at the team level. This rating is called “star” and is given out on a five-point scale, ranging from 1 star being unacceptable to 5 stars being exceptional. We used the first term rating as the measure of employee performance rating because it was the first rating the candidate received after the interview, which reflected the interview assessment of the fit between the candidate and the dimensions from the radar chart more accurately, as the predictability decreases over time ( Barrick and Zimmerman, 2009 ).

A promotion is defined as a move to a higher Rank in the hierarchy described above. The number of promotions is recorded by counting the upward change in employee position level during the time he or she is in the company.

Turnover is defined as the employee’s exit from the company. It is treated as a dichotomous variable, where 1 means the employee left the company within the observed timeframe, and 0 means he or she did not leave.

Matching Score

Before calculating the matching score, we needed to clean and process the text data of interviewers’ narrative comments as well as the dimensions from each radar chart. Because the interviewers’ narrative comments were written in Chinese, and the Chinese words are not delimited, word segmentation was a necessary step ( Peng et al., 2017 ). First, we used a Chinese stopword list to remove symbols and common words that did not provide any insights (e.g., “the,” “that,” “here,” etc.) from the content. Second, Python programming language and a Chinese Natural Language Processing package HanLP ( Han He, 2014 ) was used to segment sentences in the comments and job-analysis dimensions. Sentences in comments were broken down into meaningful words, which were then to match with the words from the job-analysis dimensions. The matching score was calculated according to the formula:

where D referred to the dimensions from the radar chart, S i referred to the weight of i th dimension, and O c c u r i referred to the number of occurrence of i th dimension in the comments. Since the radar chart for each job has a different number of dimensions with different weights, we normalized the matching score by dividing the sum of all the dimension weights.

Breadth Coverage

The breadth of dimensions from the radar chart interviewers covered during the interview is calculated by adding how many dimensions appeared in the comments. More specifically, the algorithm goes through all the phrases in the comments one by one. Whenever a new dimension that has not been seen in the previous phrases appears, the breadth coverage number adds one. Then the number is divided by the total number of dimensions for normalization since each radar chart has a different number of dimensions. Therefore, the number represents the total number of unique dimensions the interviewer has been able to assess during the interview. Although the breadth number is calculated using the comments written after the interview, it is still a relatively accurate measure. The company requires interviewers to write comments after each interview, which will become a company record for both the interviewer and the interviewee if the candidate is hired. Since this is part of interviewers’ job, they are responsible to write it to their best knowledge. With different writing styles, the length of the comments may vary across interviewers. However, these comments should reflect the ideas and dimensions interviewers assessed during the interview, at least what impressed the interviewer the most which led to his/her decision of whether hire or pass.

A candidate’s performance after he or she is selected into the company may be influenced by many factors. Therefore, several control variables were included in the analyses for better estimates of our hypotheses. We used a dichotomous variable as an indicator of the employee’s gender, where 1 indicated male and 0 indicated female because male employees may be favored more than female employees, especially in a male-dominated work environment such as in our case. Employee education level is added as a control variable because different education levels will have different impacts on the employee’s ability to process complex information, therefore affect his or her job performance ( Wally and Baum, 1994 ). Workplace age stereotypes are getting more prevalent in affecting performance evaluation, promotion, and retention of employees ( Posthuma and Campion, 2009 ). Thus, we included employee age as a control variable.

Moreover, the average team member rating was included as a control variable, where we aggregated all team members’ performance ratings and divided them by the total number of members. Team sizes were controlled because they affect how well team members know each other for performance grading. Different job clans may have different performance rating criteria, such that employees in some clans may more easily take credits for outstanding outputs, while others may not be so obvious. Therefore, we controlled the job position as well. A dummy variable of the year of hire was also added to reflect the changing environment of the labor market as well as the business the company was running in.

Descriptive statistics, the means, SD, and correlations among all variables are presented in Table 1 . To test our hypotheses, we used employees’ age, education level, gender, the size of their team, the year of hire, and the team’s average performance score as control variables. The matching score between interviewer narrative comments and dimensions on the radar chart was used as the independent variable to test its effects on performance, promotion, and turnover. We used a three-way interaction between matching score, breadth coverage, and the Clan of the candidate to test the moderating effect of breadth coverage on matching score in different job Clans ( Mitchell, 2012 ). Tables 2 – 4 provide the results of multivariate tests of our hypotheses for the dependent variables of performance, promotion, and turnover, respectively. Specifically, model 1 only took the control variable as an independent variable, while model 2 tested the direct effect of matching score on performance and promotion using Ordinary Least Squares (OLS) regression analysis, and turnover using Logit modeling. In addition to model 2, model 3 added a three-way interaction to test hypotheses 2 and 3. The coefficients are shown in Tables 2 – 4 , with the asterisk indicating the significance level.

Means, SD, and correlations continued.

Statistics analysis results for dependent variable performance.

Statistics analysis results for dependent variable turnover.

For candidates’ performance, model 1 in Table 2 showed that age and average team performance were positively related to performance rating, while the year of hire was negatively related to performance with significance. When the matching score was added to the model, the performance rating coefficient was positive and statistically significant ( β = 0.012, p < 0.05) with increased Adjusted R 2 value, supporting Hypothesis 1a. In addition, an F-test was carried out in model 2 with the result of a significant F-change, meaning that matching score significantly improved the prediction of the variables. One of the problems when dealing with a large sample size is that p -values approaching zero, giving statistical significance but impractical effects. Zhang et al. (2010) calculated the marginal effect in their study to show practical significance. Following their method, we calculated the marginal effect of matching scores on performance. When the matching score increases by one SD (SD = 1.685), performance would increase by 2 percent.

For the promotion dependent variable, model 1 in Table 3 showed that age and gender were positively related to the number of promotions, while the year of hire was negatively related to promotion with significance. In model 2, where the matching score was added, the coefficient was positive and statistically significant ( β = 0.016, p < 0.05) with increased Adjusted R 2 value, supporting Hypothesis 1b.

Statistics analysis results for dependent variable promotion.

Model 1 in Table 4 showed that team size was positively related, and average team performance was negatively related to turnover with significance. Supporting Hypothesis 1c, model 2 with the matching score added indicates turnover indicator coefficient was negative and statistically significant ( β = −0.233, p < 0.01), suggesting that the higher the matching score, the less likely the candidate would leave the company. Since we used Logit modeling for the turnover dependent variable, no Adjusted R 2 value was reported.

To test Hypotheses 2 and 3, we used a three-way interaction between matching score, breadth coverage, and Clan. High breadth coverage indicates that the interviewer covered more dimensions from the radar chart in the questions asked during the interview, while low breadth coverage indicates fewer dimensions covered. The analysis was performed using Stata, following the examples and instructions in Mitchell (2012) for continuous by continuous by categorical interactions. Results are shown in model 3s in Tables 2 – 4 for the three dependent variables analyzed respectively. For candidates’ performance ratings, our results indicated that none of the coefficients for the three-way interaction terms was significant for all five Clans, thus failed to support our Hypotheses 2a and 3a.

Results of model 3 in Table 3 showed that the coefficients of the interaction term on candidates’ number of promotions were negatively significant for the Product/Project Clan ( β = −0.171, p < 0.01) and the Specialists Clan ( β = −0.099, p < 0.1). Figures 2 , ​ ,3 3 showed the marginal prediction of the breadth coverage for different levels of matching score on the number of promotions for the Product/Project Clan and the Specialists Clan, respectively. As seen in Figure 2 , the fewer dimensions covered with higher matching score gave a better prediction on candidates’ number of promotions. However, Figure 3 shows that low breadth resulted in negative marginal prediction for Specialists Clan, suggesting that low breadth coverage caused fewer promotions. These results suggested that the fewer job-analysis dimensions interviewers covered during the interview, the better the matching score predicted candidates’ number of promotions, partially supporting Hypothesis 2b but failing to support Hypothesis 3b.

Moderating effect of breadth coverage on promotion for Product/Project Clan.

Moderating effect of breadth coverage on promotion for Specialists Clan.

Results of model 3 in Table 4 showed that the coefficients of the interaction term on candidates’ turnover were negatively significant for the Specialists Clan ( β = −9.913, p < 0.01) and negatively significant for the Technical Clan ( β = −3.225, p < 0.01). Figures 4 , ​ ,5 5 showed the marginal prediction of the breadth coverage for different levels of matching score on turnover for the Specialists Clan and the Technical Clan, respectively. As observed from Figure 4 , low breadth coverage was comparatively better than high breadth coverage for Technical Clan, such that candidates interviewed with fewer dimensions were less likely to leave the company. However, Figure 5 shows that low breadth coverage predicted higher turnover than high breadth coverage as the matching score increased for Specialists Clan. These results partially supported our Hypotheses 2c but failed to support our Hypotheses 3c.

Moderating effect of breadth coverage on turnover for Technical Clan.

Moderating effect of breadth coverage on turnover for Specialists Clan.

The results of our study show that the match between interviewer’s narrative comments and capability dimensions on the radar chart had a significant effect on interview validity in the candidate’s job performance, promotion, and turnover. The candidate’s job performance after being selected is the common measure of interview validity ( Huffcutt and Arthur, 1994 ; Kleinmann et al., 2011 ). However, most of the results were drawn using mock interviews and experimental data (e.g., Ingold et al., 2015 ; Huffcutt et al., 2017 ). Our study further confirmed the current literature empirically by using data from a real work environment. While promotion is often considered a result of higher performance ratings ( Lyness and Heilman, 2006 ), our results indicate a direct relationship between interview qualities and the number of promotions. Despite the theoretical implication that DA fit influences employee turnover ( Kristof-Brown et al., 2005 ), whether interview qualities have a direct effect on interviewee turnover was hard to analyze because of the lack of data from real scenarios. Our results showed and confirmed this theoretical implication empirically, that is, that DA fit decreased turnover rate. Together with the effect on interviewee performance and promotion, we contributed to the PJ fit literature, specifically DA fit, the finding that interview is an effective method for assessing DA fit, and thus a valid predictor for interviewees’ job performance, promotion, and turnover.

Moreover, we found that the breadth of job-analysis dimensions coverage moderates the effect of the match between interviewers’ narrative comments and job-analysis dimensions on interview validity for job categories that require deeper and narrower knowledge. Even though radar charts from job analysis is a valid tool for interview question generation, it is often treated dichotomously as to whether it is used for interview questions or not, showing a failure to appreciate the extensive amount of information it may contain. Our findings suggested an explicit usage of radar charts from job analysis in selection interviews, therefore filled the gap in knowledge of how job analysis could be better utilized to assist interviewers and extended the literature on job analysis in human resources management.

One of the reasons that our findings did not support the moderating effect of the number of job-analysis dimensions on candidates’ job performance in Hypotheses 2a and 3a was that the number of dimensions might not reflect how good the choice of dimensions by the interviewer was. Speer et al. (2019) find in their research that interviewers’ social intelligence and general mental ability are important factors that help interviewers choose more suitable interview questions and rate prospective employees accurately. Including interviewers’ personality and intelligence data may help fill this gap, and show how the different aspects of a radar chart should be chosen to predict candidates’ performance more accurately.

The moderating effects of breadth coverage on candidates’ number of promotions were significant on Specialists Clan and Product/Project Clan in Hypothesis 2b, but not on Technical Clan. On the other hand, the moderating effects were significant on Specialists Clan and Technical Clan but not on Product/Project Clan in Hypothesis 2c. This was probably because some of the jobs in the Technical and Product/Project Clan required not very deep knowledge, but broad interaction skills like jobs in Design and Marketing Clans. For example, there were jobs in the Product/Project Clan that dealt with game operations and overseas collaboration management. In the Technical Clan, jobs such as services management and operation planning were less technical but required more personal skills to coordinate with other departments. Therefore, it would be better to further test Hypotheses 2b and 2c with more specific job positions to see more clearly how the breadth coverage would moderate the results.

The failure to support Hypotheses 3b and 3c, where we argued that the higher the number of dimensions covered in the interview, the better its predictability, indicated that for jobs that required broader knowledge, the use of job analysis in interviews might require more attention. Though the job analysis is a valid component for a selection interview, as shown in Hypotheses 1a, 1b, and 1c, the number of dimensions covered might not be good enough to reveal how interviewers could utilize it in interviews for jobs in Design and Marketing Clans. Future research could focus on the content of the dimensions and how they relate to performance evaluation or PJ fit of the candidate, rather than just DA fit.

Apart from the theoretical contributions mentioned above, our results extend the literature on selection interviews by using a new type of data – interviewers’ narrative comments – as well as employees’ actual performance data within the company, to illustrate the direct effect of asking job-related questions on interview validity. Moreover, the use of text analysis on interview narrative comments is a method that has not been used in the selection interview literature, even though it is gaining more attention in other areas of human resources management research such as performance appraisal (e.g., Brutus, 2010 ; Speer, 2018 ; Speer et al., 2018 ), applicants’ justice perceptions ( Walker et al., 2015 ), and training of interviewers ( Shantz and Latham, 2012 ). Our study extends selection interview literature by introducing a new method proven in another field of study. In addition, using interviewer comments and radar charts from job analysis is a more objective method for interview candidates’ fit assessment, since it does not require employees to give ratings of the fit they perceive, which are likely to be inaccurate because of the lack of knowledge or understanding of the job requirements. This method showed a more promising fit assessment method and confirmed the extant research findings.

Even though Huffcutt and Arthur (1994) proposed a framework for different levels of interview structure in terms of the level of standardization and restriction, many studies still treat interview structure dichotomously without considering the different effects imposed by different structure levels. On the other hand, although structured interviews are believed to outperform unstructured interviews in many aspects by academic researchers, they are not well adopted by professionals and practitioners ( Van der Zee et al., 2002 ). One of the most prominent reasons that interviewers are reluctant to use structured interviews is the lack of autonomy ( Nolan and Highhouse, 2014 ). The contribution of our study here is practical. Our results indicate that interviewers could use job-analysis as a structure during the interview without using pre-determined questions, as long as they can form a conclusion about the candidate’s capabilities related to the job-analysis. This relaxed interview structure similar to level 2 of question standardization in framework of Huffcutt and Arthur (1994) was a valid predictor for interviewees’ performance. Using this relaxed structure is itself a contribution because most of the structured interview research has only examined whether structures are utilized in interviews but not to what degree they are utilized ( Chapman and Zweig, 2005 ). Our study further confirms the validity of this framework with large sample size and real-world data.

Practically, our results indicate that interviewers could choose a less restricted interview structure and be more autonomous in their interviews without compromising their interview validity. Moreover, the choice of interview questions in our study was decided by interviewers without prespecified rules, which made our result more relatable to other interviewers in practice. As Zhang et al. (2018) point out, the adoption of interview structure depends on how well practitioners and professionals understand the validity of this practice. In addition, the study by Van der Zee et al. (2002) shows that a subjective norm is one of the factors that motivate interviewers what interview techniques to use. Therefore, by using data and comments from actual interviewers and how they behaved during interviews, our results would improve interviewers’ understanding and appreciation of the superior quality of a structured interview so that they are more willing to adopt this method.

Limitations and Future Research

This study has several limitations that may need further exploration. Despite the innovative usage of interviewers’ narrative comments and real-world data generated from daily human resources activities in a real business entity, our data were drawn from one single organization, which might not generalize well to other organizations. Since different organizations may have different rules for conducting interviews and selection standards differ across industries, it would be of theoretical and practical value to compare interviewer narrative comments from other organizations and industries to see the boundary and effectiveness of how well interviewers’ narrative comments can predict interviewees’ job performance.

One of the main factors in our study is the interviewer. Whether the personality, training, and other backgrounds of the interviewer affect how he/she writes the comments and conducts the interview may have a direct effect on how well the matching score predicts the job outcome as well as the moderator effects of the breadth coverage. This issue needs to be addressed in future research from the interviewers’ perspective to provide a full scope of the study.

The requirement of the existence of high-quality radar charts from job analysis for every position might be a challenge for other organizations, especially smaller enterprises that do not have enough budget for job-analysis development. This requirement may limit the generalizability of our findings. However, what attributes interviewers are most interested in may be revealed through a thorough analysis of their narrative comments. Future research may explore the content of interviewer narrative comments using more sophisticated natural language processing techniques such as machine learning (e.g., Campion et al., 2016 ) to discompose the comments into different job-related attributes and explore their predictability on interviewees’ job performance. This approach will further contribute to the job-analysis literature by showing a new method of job-analysis development for selection interviews. Through the content analysis of interviewer narrative comments, organizations can figure out what attribute has the best predictive validity and construct radar charts accordingly, which also requires less time and budget.

While, we showed that a relaxed interview structure similar to level 2 in Huffcutt and Arthur (1994) was a valid predictor for interviewee job performance, we did not compare the validities of different levels of structure, given the promising usage of narrative comments data. Different interview structure levels may lead to different comments interviewers give after each interview. It is worth analyzing these differences in comment content and styles and how they impact the predictive validity of interviews. Future research could extend this study by controlling the interview structure levels to compare how they influence the comments interviewers write and which level gives the highest predictive validity.

Despite these limitations, the present study contributes to the selection interview literature by taking an important step toward addressing calls for research that investigates the validity of structured interviews, using new techniques and data that meet the shifts in selection literature that are affected by tremendous changes in business ( Ployhart et al., 2017 ). Our sample contained employees from various positions with large sample size, as well as data from real selection interviews and employees’ job performance ratings in an actual organization setting, thus increasing our confidence in both theoretical and practical contributions. Also, our use of text analysis techniques allowed us to investigate how the match between interviewers’ narrative comments and radar chart dimensions predicts candidates’ performance and career in the organization. To our knowledge, this is the first empirical examination of how the breadth of job-analysis dimensions asked during an interview influences the interview validity. Thus, the findings of our study offer important insights into how interviewers should conduct their interviews using radar charts from job analysis for better interview validity.

Data Availability Statement

Ethics statement.

The studies involving human participants were reviewed and approved by South China University of Technology. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

YC carried out the study, analyzed the results, and finished the manuscript. JJ contributed to the design and conception of the study. SL and HM helped with data acquisition and provided valuable feedback on the drafts. HZ helped with data processing and modeling. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank Qiuping Peng for her helpful feedback and support on data handling. We also like to thank Jingming Feng and Bo Sun for their helpful comments.

Funding. This study was supported by the National Natural Science Foundation of China (71832003 and 71672116).

• Barrick M. R., Parks-Leduc L. (2019). Selection for fit . Annu. Rev. Organ. Psych. Organ. Behav. 6 , 171–193. 10.1146/annurev-orgpsych-012218-015028 [ CrossRef ] [ Google Scholar ]
• Barrick M. R., Zimmerman R. D. (2009). Hiring for retention and performance . Hum. Resour. Manag. 48 , 183–206. 10.1002/hrm.20275 [ CrossRef ] [ Google Scholar ]
• Boh W. F., Evaristo R., Ouderkirk A. (2014). Balancing breadth and depth of expertise for innovation: a 3M story . Res. Policy 43 , 349–366. 10.1016/j.respol.2013.10.009 [ CrossRef ] [ Google Scholar ]
• Boon C., Biron M. (2016). Temporal issues in person–organization fit, person–job fit and turnover: the role of leader–member exchange . Hum. Relat. 69 , 2177–2200. 10.1177/0018726716636945, PMID: [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Brutus S. (2010). Words versus numbers: a theoretical exploration of giving and receiving narrative comments in performance appraisal . Hum. Resour. Manag. Rev. 20 , 144–157. 10.1016/j.hrmr.2009.06.003 [ CrossRef ] [ Google Scholar ]
• Cable D. M., Judge T. A. (1997). Interviewers' perceptions of person–organization fit and organizational selection decisions . J. Appl. Psychol. 82 , 546–561. 10.1037/0021-9010.82.4.546, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Campion M. C., Campion M. A., Campion E. D., Reider M. H. (2016). Initial investigation into computer scoring of candidate essays for personnel selection . J. Appl. Psychol. 101 , 958–975. 10.1037/apl0000108, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Campion M. A., Palmer D. K., Campion J. E. (1997). A review of structure in the selection interview . Pers. Psychol. 50 , 655–702. 10.1111/j.1744-6570.1997.tb00709.x [ CrossRef ] [ Google Scholar ]
• Chapman D. S., Zweig D. I. (2005). Developing a nomological network for interview structure: antecedents and consequences of the structured selection interview . Pers. Psychol. 58 , 673–702. 10.1111/j.1744-6570.2005.00516.x [ CrossRef ] [ Google Scholar ]
• Conway J. M., Jako R. A., Goodman D. F. (1995). A meta-analysis of interrater and internal consistency reliability of selection interviews . J. Appl. Psychol. 80 , 565–579. 10.1037/0021-9010.80.5.565 [ CrossRef ] [ Google Scholar ]
• Edwards J. R. (1991). “ Person-job fit: a conceptual integration, literature review, and methodological critique ” in International review of industrial and organizational psychology. Vol. 6 eds. Cooper C. L., Robertson I. T. (Oxford, England: John Wiley & Sons; ), 283–357. [ Google Scholar ]
• Elam C. L., Johnson M., Wiese H. J., Studts J. L., Rosenbaum M. (1994). Admission interview reports: a content analysis of interviewer comments . Acad. Med. 69 ( Suppl. 10 ), S63–S65. 10.1097/00001888-199410000-00044, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Fleming L. (2001). Recombinant uncertainty in technological search . Manag. Sci. 47 , 117–132. 10.1287/mnsc.47.1.117.10671 [ CrossRef ] [ Google Scholar ]
• Greve H. R., Taylor A. (2000). Innovations as catalysts for organizational change: shifts in organizational cognition and search . Adm. Sci. Q. 45 , 54–80. 10.2307/2666979 [ CrossRef ] [ Google Scholar ]
• Han He (2014). HanLP: Han language processing. Available at: https://github.com/hankcs/pyhanlp (Accessed September 13, 2019).
• Hu J., Wayne S. J., Bauer T. N., Erdogan B., Liden R. C. (2016). Self and senior executive perceptions of fit and performance: a time-lagged examination of newly-hired executives . Hum. Relat. 69 , 1259–1286. 10.1177/0018726715609108 [ CrossRef ] [ Google Scholar ]
• Huffcutt A. I., Arthur W. (1994). Hunter and hunter (1984) revisited: interview validity for entry-level jobs . J. Appl. Psychol. 79 , 184–190. 10.1037/0021-9010.79.2.184 [ CrossRef ] [ Google Scholar ]
• Huffcutt A. I., Conway J. M., Roth P. L., Stone N. J. (2001). Identification and meta-analytic assessment of psychological constructs measured in employment interviews . J. Appl. Psychol. 86 , 897–913. 10.1037/0021-9010.86.5.897, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Huffcutt A. I., Culbertson S. S., Goebl A. P., Toidze I. (2017). The influence of cognitive ability on interviewee performance in traditional versus relaxed behavior description interview formats . Eur. Manag. J. 35 , 383–387. 10.1016/j.emj.2016.07.007 [ CrossRef ] [ Google Scholar ]
• Ingold P. V., Kleinmann M., König C. J., Melchers K. G., Van Iddekinge C. H. (2015). Why do situational interviews predict job performance? The role of interviewees’ ability to identify criteria . J. Bus. Psychol. 30 , 387–398. 10.1007/s10869-014-9368-3 [ CrossRef ] [ Google Scholar ]
• Kleinmann M., Ingold P. V., Lievens F., Jansen A., Melchers K. G., König C. J. (2011). A different look at why selection procedures work: the role of candidates’ ability to identify criteria . Organ. Psychol. Rev. 1 , 128–146. 10.1177/2041386610387000 [ CrossRef ] [ Google Scholar ]
• Kobayashi V. B., Mol S. T., Berkers H. A., Kismihok G., Den Hartog D. N. (2018). Text mining in organizational research . Organ. Res. Methods 21 , 733–765. 10.1177/1094428117722619, PMID: [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kristof-Brown A. L., Zimmerman R. D., Johnson E. C. (2005). Consequences of individuals'fit at work: a meta-analysis of person–job, person–organization, person–group, and person–supervisor fit . Pers. Psychol. 58 , 281–342. 10.1111/j.1744-6570.2005.00672.x [ CrossRef ] [ Google Scholar ]
• Levashina J., Hartwell C. J., Morgeson F. P., Campion M. A. (2014). The structured employment interview: narrative and quantitative review of the research literature . Pers. Psychol. 67 , 241–293. 10.1111/peps.12052 [ CrossRef ] [ Google Scholar ]
• Lyness K. S., Heilman M. E. (2006). When fit is fundamental: performance evaluations and promotions of upper-level female and male managers . J. Appl. Psychol. 91 , 777–785. 10.1037/0021-9010.91.4.777, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Macan T. (2009). The employment interview: a review of current studies and directions for future research . Hum. Resour. Manag. Rev. 19 , 203–218. 10.1016/j.hrmr.2009.03.006 [ CrossRef ] [ Google Scholar ]
• Markoulli M. P., Lee C. I. S. G., Byington E., Felps W. A. (2017). Mapping human resource management: reviewing the field and charting future directions . Hum. Resour. Manag. Rev. 27 , 367–396. 10.1016/j.hrmr.2016.10.001 [ CrossRef ] [ Google Scholar ]
• Mitchell M. N. (2012). Interpreting and visualizing regression models using Stata. College Station, Tex: Stata Press. [ Google Scholar ]
• Morgeson F. P., Spitzmuller M., Garza A. S., Campion M. A. (2016). Pay attention! The liabilities of respondent experience and carelessness when making job analysis judgments . J. Manag. 42 , 1904–1933. 10.1177/0149206314522298 [ CrossRef ] [ Google Scholar ]
• Muchinsky P. M., Monahan C. J. (1987). What is person-environment congruence? Supplementary versus complementary models of fit . J. Vocat. Behav. 31 , 268–277. 10.1016/0001-8791(87)90043-1 [ CrossRef ] [ Google Scholar ]
• Nolan K. P., Highhouse S. (2014). Need for autonomy and resistance to standardized employee selection practices . Hum. Perform. 27 , 328–346. 10.1080/08959285.2014.929691 [ CrossRef ] [ Google Scholar ]
• Peng H., Cambria E., Hussain A. (2017). A review of sentiment analysis research in Chinese language . Cogn. Comput. 9 , 423–435. 10.1007/s12559-017-9470-8 [ CrossRef ] [ Google Scholar ]
• Platanou K., Mäkelä K., Beletskiy A., Colicev A. (2018). Using online data and network-based text analysis in HRM research . J. Organ. Effect. People Perform. 5 , 81–97. 10.1108/JOEPP-01-2017-0007 [ CrossRef ] [ Google Scholar ]
• Ployhart R. E., Schmitt N., Tippins N. T. (2017). Solving the supreme problem: 100 years of selection and recruitment at the journal of applied psychology . J. Appl. Psychol. 102 , 291–304. 10.1037/apl0000081, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Posthuma R. A., Campion M. A. (2009). Age stereotypes in the workplace: common stereotypes, moderators, and future research directions . J. Manag. 35 , 158–188. 10.1177/0149206308318617 [ CrossRef ] [ Google Scholar ]
• Posthuma R. A., Morgeson F. P., Campion M. A. (2002). Beyond employment interview validity: a comprehensive narrative review of recent research and trends over time . Pers. Psychol. 55 , 1–81. 10.1111/j.1744-6570.2002.tb00103.x [ CrossRef ] [ Google Scholar ]
• Sanchez J. I., Levine E. L. (2001). “ The analysis of work in the 20th and 21st centuries ” in International handbook of work and organizational psychology. Vol. 1 eds. Anderson N., Ones D. S., Sinangil H. K., Viswesvaran C. (Thousand Oaks, CA: Sage; ), 71–89. [ Google Scholar ]
• Sanchez J. I., Levine E. L. (2012). The rise and fall of job analysis and the future of work analysis . Annu. Rev. Psychol. 63 , 397–425. 10.1146/annurev-psych-120710-100401, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schilling M. A., Vidal P., Ployhart R. E., Marangoni A. (2003). Learning by doing something else: variation, relatedness, and the learning curve . Manag. Sci. 49 , 39–56. 10.1287/mnsc.49.1.39.12750 [ CrossRef ] [ Google Scholar ]
• Shantz A., Latham G. P. (2012). Transfer of training: written self-guidance to increase self-efficacy and interviewing performance of job seekers . Hum. Resour. Manag. 51 , 733–746. 10.1002/hrm.21497 [ CrossRef ] [ Google Scholar ]
• Simonton D. K. (2003). Scientific creativity as constrained stochastic behavior: the integration of product, person, and process perspectives . Psychol. Bull. 129 , 475–494. 10.1037/0033-2909.129.4.475, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Speer A. B. (2018). Quantifying with words: an investigation of the validity of narrative-derived performance scores . Pers. Psychol. 71 , 299–333. 10.1111/peps.12263 [ CrossRef ] [ Google Scholar ]
• Speer A. B., Christiansen N. D., Laginess A. J. (2019). Social intelligence and interview accuracy: individual differences in the ability to construct interviews and rate accurately . Int. J. Sel. Assess. 27 , 104–128. 10.1111/ijsa.12237 [ CrossRef ] [ Google Scholar ]
• Speer A. B., Schwendeman M. G., Reich C. C., Tenbrink A. P., Siver S. R. (2018). Investigating the construct validity of performance comments: creation of the great eight narrative dictionary . J. Bus. Psychol. 34 , 747–767. 10.1007/s10869-018-9599-9 [ CrossRef ] [ Google Scholar ]
• Thorsteinson T. J. (2018). A meta-analysis of interview length on reliability and validity . J. Occup. Organ. Psychol. 91 , 1–32. 10.1111/joop.12186 [ CrossRef ] [ Google Scholar ]
• Van der Zee K. I., Bakker A. B., Bakker P. (2002). Why are structured interviews so rarely used in personnel selection? J. Appl. Psychol. 87 , 176–184. 10.1037/0021-9010.87.1.176, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Walker H. J., Helmuth C. A., Feild H. S., Bauer T. N. (2015). Watch what you say: job applicants’ justice perceptions from initial organizational correspondence . Hum. Resour. Manag. 54 , 999–1011. 10.1002/hrm.21655 [ CrossRef ] [ Google Scholar ]
• Wally S., Baum J. R. (1994). Personal and structural determinants of the pace of strategic decision making . Acad. Manag. J. 37 , 932–956. 10.2307/256605 [ CrossRef ] [ Google Scholar ]
• Wiesner W. H., Cronshaw S. F. (1988). A meta-analytic investigation of the impact of interview format and degree of structure on the validity of the employment interview . J. Occup. Psychol. 61 , 275–290. 10.1111/j.2044-8325.1988.tb00467.x [ CrossRef ] [ Google Scholar ]
• Zhang D. C., Highhouse S., Brooks M. E., Zhang Y. Y. (2018). Communicating the validity of structured job interviews with graphical visual aids . Int. J. Sel. Assess. 26 , 93–108. 10.1111/ijsa.12220 [ CrossRef ] [ Google Scholar ]
• Zhang Y., Li H., Li Y., Zhou L. A. (2010). FDI spillovers in an emerging market: the role of foreign firms' country origin diversity and domestic firms' absorptive capacity . Strateg. Manag. J. 31 , 969–989. 10.1002/smj.856 [ CrossRef ] [ Google Scholar ]