Part 5: Comments on Testing by Dr. R.P. Ascano

This resource has multiple chapters. Use the navigation on the side or below to browse each chapter.

What are the criteria for evaluating psychological testing?

Many examining psychologists are not aware of the existence of 29 CFR § 1630.13, titled “Prohibited medical examinations and inquiries.” Even more importantly, section 1630.10, “Qualification standards, tests, and other selection criteria,” discusses the types of tests that can be used. Generally, tests may not screen people for disabilities, and should be “job related for the position in question and . . . consistent with business necessity.”

What does this mean? It would be difficult for a testing to be “job related for the position in question” if the mission organization does not have a job analysis/job description, or the job description that it does have does not meet the ministerial exception standard, or if the job description does not have sufficient behavioral criteria.

The job description and behavioral criteria are needed to provide the examining psychologist with the needed data to ascertain what psychometric would be appropriate to use.

The federal government has provided further guidance in 29 CFR § 1607.5, that discusses “General standards for validity studies” at some length.¹

What is the professional standard to evaluate job testing?

Standards for Educational and Psychological Testing 14.8 provides that to analyze the validity of test content, you must carefully define the domain of interest of the test content. In other words, what are you trying to test? If you are testing employment-related issues like “selection, classification, and promotion,” then the content of the test “should be based on job analysis.”

What is job analysis and description? Job analysis is a “detailed statement of work behaviors and other information relevant to the job.”² A job description is a “general statement of job duties and responsibilities.”³

Likewise, the federal regulations define “content validity” as requiring data that shows that “the content of a selection procedure is representative of important aspects of performance on the job.”⁴

This means that you cannot do employment-related testing without a careful job analysis and/or description.

Can testing a person be invasion of privacy?

The mission must keep in mind the issue of “invasion of privacy.” Courts have held that a personality test can infringe on the privacy of the test taker.⁵ For example, if a psychologist used the Multiphasic Sex Inventory – II, which involves the examiner accessing clinical data regarding the examinee’s sexuality, this would be an invasion of privacy, because this information is not relevant during the pre-hire phase.

In order to avoid invasion of privacy, each and every psychometric used MUST be related to the job as defined by the job description. An example of a test that is likely to be relevant would be the Workplace Personality Profile. The scales of this test are: Drive to Achieve, Self-confidence, Assertiveness, Service Orientation, Flexibility and Reliability. These are all traits that will likely be relevant to many job descriptions; also, this test does not assess disabilities.

The EEOC applies its uniform guidelines on employee selection to “tests and other selection procedures which are used as a basis for any employment decision.”⁶ Because of this, mission organizations should be well aware of these uniform guidelines and the standards applied for testing in employment.

What if I use the Myers-Briggs test?

The Myers-Briggs test was not developed for employment selection. Instead, it was developed for use in training and development. If a mission psychologist uses the Myers-Briggs as a pre-employment test, it could create professional liability, as the test is being misused. It would also probably violate content validity as defined above.

If the mission uses it for training and development, as it was intended, that might be acceptable. Myers-Briggs is a poplar measure of normal personality. Some researchers express optimism in its utilization in various business milieus.⁷

But I have some further concerns about the test. For one thing, the results are not consistent. Analysis reflects that a large portion of their participants received different type profiles when retested.⁸ This finding was replicated. Across a 5-week test-retest interval, 50% of the participants received a different classification on one or more of the scales.⁹

Another research finding reports that 35% of individuals had a different four-letter type score after a 4-week interval.¹⁰

This research raises questions about the appropriateness of using type categories for individuals who do not evidence a clear preference for a specific Myers-Briggs Type Indicator.

There are no scales built into the Myers-Briggs to detect the effects of random responding, response sets such as social desirability, or either conscious or unconscious response distortion. Social desirability response set seems to influence scores on the EI and JP scales.¹¹

In essence, the response distortions range from lack of self-insight to deliberate faking, as well as the effects of response sets in general, which is applicable to self-reporting questionnaires.¹²

In litigation, this test would not meet the (i)Daubert(i) standard for scientific evidence, so it should be used cautiously. In my opinion, mission organizations are being placed at risk by their consulting psychologists who use the Myers-Briggs as a pre-employment psychometric.

What should a psychologist know about reasonable accommodations?

Bear in mind that if an employment decision is challenged on the basis of the accuracy or validity of the testing or the reasonableness of the offered accommodations, the credentials of the examining psychologist will be important. In a lawsuit, the psychologist’s background in organizational psychology, or whether the psychologist is certified in vocational psychology, may impact whether the selected accommodation is considered reasonable. A plaintiff may also challenge whether a proposed accommodation is empirically supported by research data.

Because of this, if there is any question whether the evaluating psychologist has sufficient credentials in a complex situation, it might be wise to make an external referral to an expert.

What are the spiritual intersections of testing?

As a forensic psychologist, I advocate that the mission organization focus on a “spiritual resiliency” assessment, which would be consistent with the “ministerial exception.” For example, clinical vulnerability, such as susceptibility to depressive disorder, has almost no effect when religiosity levels are high. However, when religiosity is low, vulnerability has a substantial effect on high-risk behaviors.¹³

A spiritual resiliency assessment can be both very effective in identifying potential problems, and be unlikely to be challenged because the mission is a religious organization.

What about post-employment testing?

I advocate that a mission organization perform an annual “medical/psychological” evaluation as a prevention protocol. I recommend using an instrument such as the Maslach Burnout Inventory as a general screening tool. If the result is positive for burnout, then a more comprehensive evaluation would be justified and desirable.

_________________________________________

^1. Please see EEOC Enforcement Guidance, # 915.002 [May 19, 1994], Part V (B) (2), Pre-employment disability related inquiries and medical examinations under the ADA (see ADA Manual) (BNA Subsect. 70: 1103 et. seq.)

^2.29 CFR § 1607.16(K)

^3. 29 CFR § 1607.16(L)

^4. 29 CFR § 1607.16(D)

^5. See Osborn v. U.S., 385 US 323, 342, 1966

^6. 29 CFR § 1607.2

^7. McCauley, M., 2000, “Myers-Briggs Type Indicator: A bridge between counseling and consulting,” Consulting Psychology Journal: Practice and Research, 52, pp 117-132

^8. Howes, R. and Carskadon, T., 1979, “Test-retest reliabilities of the Myers-Briggs Type Indicator as a function of mood changes,” Research in Psychological Types, 2, pp 67-72

^9. McCarley, N. and Carskadon, T., 1983, “Test-retest reliabilities of scales and subscales of the Myers-Briggs Type Indicator and of criteria for clinical interpretive hypotheses involving them,” Research in Psychological Type, 6, pp 24-36

^10. Myers, I., et al., 1998, Manual: A Guide to the Development and Use of the Myers-Briggs Type Indicator, Palo Alto, CA: Consulting Psychologist Press

^11. McCauley, M., 1981, “Jung’s theory of psychological types and the Myers-Briggs Type Indicator,” In McReynolds, P., (Ed.) Advances in Personality Assessment, V. 5, pp 294-352

^12. Boyle, G., 1985, “Self-report measures of depression: Some psychometric considerations,” British Journal of Clinical Psychology, 24, pp 45-59

^13. Korn, L., et al., 2014, “Problem behaviors among Israeli undergraduate students: Applying Jessor’s problem behavior theory among young adult students,” Frontiers in Public Health, 2(December, article 273), pp 1-8

Disclaimer: not official legal or psychological advice or opinion