In the first part of this edition, How Proverbs Connect Language, Culture and Cognition, we explored the relationship between proverbs and human experience, and why interpreting them requires a mental leap of varying difficulty. Their use in neuropsychological and psychiatric assessments – and adapting them for use across different languages and cultural contexts – opens a bigger can of worms, or Pandora’s box.
To adapt an assessment so that the target population in another country experiences the assessment at the same level of difficulty, finding a proverb used in the target country with a similar message is not enough. If the replacement is easier, harder, more familiar or less abstract than the original, it may no longer contribute to the measurement in the same way. Getting it right is partly a linguistic matter but more prominently about measurement equivalence.
Proverbs in Psychometrics
Let’s return to the Positive and Negative Syndrome Scale (PANSS) since it is widely known across multiple therapeutic areas.
The Difficulty in Abstract Thinking section (N5 item in the PANSS Negative Scale) contains a series of proverbs, organized into broad bands of increasing interpretive difficulty for native speakers of American English.
The selection of those proverbs drew on professional judgement and the limited published evidence available at the time on proverb abstraction and intellectual ability. Within the PANSS interview, responses to these prompts help the clinician evaluate the severity of difficulty in abstract thinking.
In psychometric terms, the proverb is a measurement stimulus; a prompt designed to elicit observable evidence about a latent construct. Through this lens, adapting the proverb is about preserving the prompt’s ability to elicit comparable evidence of abstract thinking in another language and culture.
In the previous part of this edition, we established that although the wisdom may be universal, the imagery in proverbs is culturally bound, which is directly linked to its phrasing.
This shift in perspective is what distinguishes translation from transadaptation.
Preserving the Measurement
Let’s use one of the PANSS proverbs as an example.
People who live in glass houses should not throw stones at others.
Suppose we need to adapt this item for another language and culture. We could begin by looking for a proverb in the target language that conveys a similar underlying principle.
For the PANSS, however, semantic similarity alone is not sufficient because the proverb forms part of an assessment of abstract thinking. The adapted version must also place a reasonably comparable cognitive demand on the respondent. Data collected in France must remain interpretable alongside data collected in Egypt, Singapore or India.
Exposure to proverbial speech varies significantly across cultures. In some societies, proverbs remain part of everyday conversation, storytelling, ceremony and literature. In others, they are encountered far less frequently. Hence, respondents with the same level of abstract reasoning may differ simply because one population has greater lifelong familiarity with proverbial language.
A different problem occurs when the original proverb is translated instead of replaced. Its imagery may depend on an unfamiliar cultural reference, or the translated metaphor may require a more complicated chain of inference than the source item.
Without addressing these differences, how will we know during data analysis if the responses reflect differences in abstract thinking, or differences in the difficulty, familiarity, and interpretive demand?
Replacing items in clinical assessments is an accepted psychometric practice when there is sufficient evidence that the original item is unlikely to function the same way in the target language or culture. The objective is to preserve the evidence the item generates about the intended construct. Preserving the wording is secondary, as long as the item remains functionally equivalent.
Over time, however, a misconception has become entrenched within parts of the industry: once an instrument has been validated or standardized, its items cannot be changed. In reality, the opposite is often true. When an item no longer works the same way because its language, imagery or cultural context does not resonate with the target population, preserving the original wording may introduce more measurement error than replacing it.
This principle is not unique to cross-cultural adaptation. Items are routinely refined, replaced or discarded during instrument development, and they may require further modification when the same instrument is validated in another language, another country or even another linguistic community within the same country. The objective is to ensure that the adapted item continues to generate valid evidence about the same construct.
A suitable replacement must therefore preserve the intended level of interpretive difficulty while continuing to generate evidence about the same construct. If interpretation of the imagery travels through a substantially simpler or more complex reasoning process, the adapted stimulus may contribute differently to the clinician’s rating. And when the scores contain these differences, measurement comparability has not been adequately protected.
What Needs to Be Preserved Depends on What Is Being Measured
There is no universal rule for adapting assessment items.
The type of equivalence that must be preserved depends on the construct the assessment is measuring.
What, exactly, is the assessment trying to measure?
If the construct is abstract thinking, as in the PANSS proverb task, the objective is to preserve the cognitive demands of the stimulus. The specific moral lesson conveyed by the proverb is incidental to the construct. What matters is that the proverb elicits a comparable level of abstract reasoning.
If the assessment is designed to evaluate proverb comprehension, preserving the intended meaning becomes essential because understanding that particular proverb forms part of the construct being measured.
If the assessment evaluates cross-cultural understanding, the objective may be to preserve the underlying principle but allow the imagery and wording to change.
The adaptation strategy therefore always follows the construct that is being measured.
Once the construct has been identified, then how do we determine if an adapted stimulus is likely to preserve it?
The Stimulus Equivalence Framework
In part 1 of this edition, I hinted that adapting proverbs requires more than intuition. This is the approach I have gradually developed over years of adapting culturally embedded assessment items.
When adapting complex items like proverbs, the main challenge is in determining if the replacement continues to generate comparable evidence about the construct being measured.
At Santium, we approach this through what we call the Stimulus Equivalence Framework (SEF). Instead of starting with language, the framework begins with the construct and works backwards.
Step 1. Construct
Before adapting any assessment items, and depending on the scope of work, identify exactly what the assessment, subscale and/or item is supposed to measure.
This information should already exist. If it doesn’t, ask for it. If the construct cannot be established, adaptation decisions become assumptions instead of evidence-based judgements. Such limitation should be documented before the project begins.
In the case of the PANSS proverb task, the construct remains abstract thinking.
Everything that follows must support that objective.
Step 2. Build the Team and Identify Preservation Target
Once the construct has been defined, determine which characteristics of the stimulus must be preserved.
In our abstract thinking example, the replacement proverb does not have to express the same moral lesson. It does, however, need to place a comparable cognitive demand on the respondent. In that sense, the adaptation target is the proverb’s position on the continuum of abstract interpretive difficulty.
Because very little published evidence exists on the relative abstract difficulty of proverbs across languages, the adaptation team relies on developing a working hypothesis.
This is where the choice of expert reviewers becomes critical.
For culturally embedded stimuli such as proverbs, the review team should include bicultural subject-matter experts. These are individuals who are fluent in both languages and have lived long enough in both cultures to develop emic fluency. Language proficiency alone is not enough. They must understand how native speakers in each culture naturally recognize, interpret and reason through proverbial language.
Their task is to estimate the apparent difficulty band occupied by the source proverb. They then identify several target-language candidates that appear to occupy the same band. Then, determine empirically which candidate best preserves the measurement. Some of these activities are demonstrated in step 3 below.
At this stage, these remain expert judgements rather than empirical evidence. Their purpose is to identify the strongest candidate stimuli for testing with respondents from the target population.
Step 3. Select the Adaptation Strategy
Only after the preservation target has been identified should adaptation decisions be made.
Let’s look at examples from the English PANSS being adapted to Czech. Starting with the easier proverbs from the English version.
Example 1 – Direct Equivalent (Item 12)
One swallow does not make a summer.
Candidate
Jedna vlaštovka jaro nedělá.
One swallow does not make a spring.
Now, let’s look at the reasoning pathway and estimate the level of difficulty.
Response process
Source
– Observe one swallow.
– Infer that the season may be changing.
– Recognize that one observation is insufficient.
– Generalize: one example does not establish a pattern.
Candidate
– Observe one swallow.
– Infer that spring may be arriving.
– Recognize that one observation is insufficient.
– Generalize: one example does not establish a pattern.
Decision
Although the season changes from summer to spring, the reasoning pathway appears very similar. The proverb therefore becomes a strong candidate for empirical testing.
Example 2 – Functional Replacement (Item 16)
Source proverb
People who live in glass houses should not throw stones at others.
Candidate
Nezametej před cizím prahem, když máš nepořádek před vlastním.
Don’t sweep in front of someone else’s doorstep when your own is untidy.
Response process
Source
– Glass houses break easily.
– Throwing stones and someone retaliating puts you at risk.
– Your own weakness mirrors the person you criticize.
– Do not criticize faults you also possess.
Candidate
– Another person’s doorstep is dirty.
– Your own doorstep is also dirty.
– You ignore your own faults.
– Do not criticize faults you also possess.
Decision
The imagery is very different, but the moral principle is the same. However, the Czech metaphor and the wording used appear to be less obscure and hence may be easier than the English one. Consider the following options:
- Retain as a candidate, but continue searching.
- Since this is the most difficult proverb in the English version, the Czech candidate could be moved up to an easier position; perhaps to become one of the items in the #5–8 grouping. This retains a proverb that shares the same meaning as another in the English version.
- Find another Czech proverb that is less transparent, and compare again.
Example 3 – No Meaning Equivalent (Item 2)
Source proverb
Carrying a chip on your shoulder.
Candidate
There is no Czech proverb that conveys the idea that someone is persistently resentful, defensive, or convinced that they’ve been treated unfairly. Czechs have multiple idioms for this idea, but they are not metaphoric. They reveal the idea fairly easily, when not directly. So, we must go straight to finding another candidate that has a comparable cognitive leap as this proverb.
Response process
Source
– A person is carrying an object.
– The proverb is frequently used, hence commonly understood.
– The object is a metaphor for an enduring wound, injury, emotional scar or behavioural state.
To find a comparable candidate, the underlying message can be completely different. The most challenging aspect appears to be the inference of the underlying psychological characteristic and the metaphoric object causing it.
The adaptation team would now look for Czech proverbs or idiomatic expressions that require respondents to navigate a similar sequence of mental operations, but not necessarily in the same order. The order or the nature of the physical object, or whether a person is involved in the original, does not matter.
Candidate 1
Dělá z komára velblouda.
He makes a camel out of a mosquito.
Response process
– Metaphorical transformation
– Concrete image representing metaphorical exaggeration; a behavioural trait
Also a short inferential chain.
Decision
Probably comparable.
Candidate 2
Malovat čerta na zeď.
To paint the devil on the wall.
Response process
– Concrete image with metaphorical meaning implying a psychological tendency.
– The person is imagining disaster before it happens.
– Generalized to pessimism or catastrophizing.
Decision
Probably comparable
Candidate 3
Chodit kolem horké kaše.
To walk around hot porridge.
Response process
– Walking around hot porridge is odd.
– Concrete image with metaphorical meaning.
– Infers avoidance or indirect communication; a behavioural tendency.
Decision
Probably slightly more difficult for younger generations.
Only empirical testing can determine which hypothesis is correct.
Step 4. Test the Hypothesis
At this stage, the adaptation team has identified the candidate proverb most likely to preserve the intended level of interpretive difficulty. It is important to remember, however, that this remains a hypothesis. Expert review can narrow the field and identify the strongest candidates, but it cannot establish measurement equivalence.
The first opportunity to test that hypothesis comes during cognitive debriefing. Respondents from the target population are asked to interpret the adapted proverb in their own words. The goal is to understand how they arrive there. Do they process naturally from the literal image to an abstract concept? Is the proverb so familiar that the interpretation is immediate? Or does unfamiliar imagery or cultural context create a barrier that makes the task more difficult than it’s meant to be?
Cognitive debriefing provides valuable evidence about the respondent’s reasoning process, but it addresses only part of the challenge.
The next step is to administer the adapted assessment under conditions that resemble its intended use. Only then can researchers examine whether the adapted proverb contributes evidence about abstract thinking in a manner comparable to the source version. Does it appear to occupy the same position within the hierarchy of proverb difficulty? Does it contribute appropriately to the clinician’s rating of difficulty in abstract thinking? And where study design and sample size permit, does the assessment behave similarly across different languages and cultural groups?
No single study proves equivalence. In good practice, evidence accumulates. Expert judgement identifies the strongest adaptation hypothesis. Cognitive debriefing examines how well respondents interpret the adapted stimulus. Repeated administration followed by psychometric evaluation determines if longitudinally observed performance supports the hypothesis.
That is why psychometric preservation cannot be achieved through translation alone. It is achieved by developing the strongest possible adaptation hypothesis and then gathering evidence that the adapted stimulus continues to generate comparable evidence about the construct being measured.
We just have to test it.
How Santium Contributes to This Space
Santium contributions to the Life Sciences industry are designed to support the global implementation of patient-centered endpoints and patient-facing research materials. We do this through multilingual patient and stakeholder interviewing, cross-cultural evidence gathering, local subject-matter expert engagement, and measurement-aware linguistic and cultural adaptation.
Drawing on expertise in psychometrics, thousands of linguistic validation projects, multilingual interviewing, and multinational study operations, Santium helps sponsors and researchers evaluate whether patient-facing concepts remain relevant, clinically meaningful, culturally appropriate, and operationally usable across languages, countries, and modes of administration.
Curious About the World’s Languages?
If you found this article interesting, follow me on LinkedIn and join our newsletter to receive future editions. In the Santium Language Series, I explore interesting traits of the world’s languages, from the most widely spoken to obscure, introducing their structure and word formation to cultural nuance and their role in social norms.
Stay connected and don’t miss the next edition!
Are you interested or work in the COA-related area of clinical research?
If you found this article interesting, follow me on LinkedIn or subscribe to this Adapting COAs the Right Way series, where we explore the psychometrics hiding under the hood and the translation methods for clinical outcome assessments – from simple rating scales to complex neuropsychological batteries. We introduce their constructs, design rules you need to know about, the importance of phrasing, concepts being measured, and how cultural nuance and age-based norms impact the translation and cultural adaptation process.
Join me in elevating the quality of COA translation through a psychometrically-driven perspective.
Stay connected and don’t miss the next edition!
About the Author
Monika Vance | Managing Director | SANTIUM
My work sits at the intersection of linguistics, scientific and medical translation, psychometric measurement, and multilingual operations, where terminology, usability, end-user experience, and regulatory context must align. I write about scientific and medical translations, psychometrics, languages, patient-centred research and the operational challenges that inevitably come with them. I also teach translators how to properly translate and validate complex psychometric instruments to hone their expertise in linguistic validation.
Monika Vance
Managing Director | SANTIUM
My work sits at the intersection of linguistics, scientific and medical translation, psychometric measurement, and multilingual operations, where terminology, usability, and regulatory context must align. I write about scientific and medical translations, psychometrics, languages, patient-centred research and the operational challenges that inevitably come with them. I also teach translators how to properly translate and validate complex psychometric instruments to hone their expertise in linguistic validation.
