BLOG
OTHER ARTICLES
What are the discrepancies between perceived vs true levels when language testing?
We have just finished an English evaluation session for around 50 employees of the same company. The idea for this article comes from this recent experience.
A prevailing question - What does the placement test provide?
What are our placement tests? We have written an article about our tests.
But here we are going to look at various perceived discrepancies with the test and why we are keeping our protocol.
First, a little background. When I introduce our test, I explain the following:
The test is relative and not absolute.
Our test is practical, not theoretical. We measure what participants use, not what they know. The nuance is important. If a person knows how to conjugate a verb in the future tense, but when they speak, only the present tense is used, we might then classify them in a level where the future tense will be learned. (Note: this is one factor of many).
The test is used to form relatively homogeneous groups of learners. This classification will also give us grammatical themes for the first lessons. Thereafter, the teacher will be able to adapt to the needs of the individual or the group.
We do not measure the same thing that a test like "Cambridge" would, which lasts a few hours, in only 15-20 min. We do not test for academic comprehension, theoretical knowledge, text comprehension, and writing.
The test is carried out one-on-one, and it is done remotely. Depending on the region of the learner and that of the teacher, it can be done by telephone or videoconference.
If it is done by telephone, no visual support will be present, an element that can sometimes prove to be useful, but generally does not impact the overall result.
So, in this context, what could be the sources of discrepancies between our test results and the perceived level of a learner?
Our test assesses the practical component (conversation and comprehension). Very often, we find speaking is weaker than reading, theoretical knowledge, and comprehension. So, sometimes, learners may feel that they are better than they are at speaking. Although not a discrepancy per se, we sometimes get questions about the ranking with comments such as "I can watch movies and TV in English". Moreover, I speak with my colleagues in the target language. So why am I at this level?
Not surprisingly, listening comprehension is very often found to be stronger than the speaking level for English second language students -- we are constantly bombarded with English information from various sources including movies, TV series, videos, and other media.
Fluidity is not synonymous with fluency. As far as conversation, there is a big difference between a person having the confidence to express themselves in the target language, but with errors and mastering verb tenses and structures correctly. We regularly find extroverts who speak fluently, but they only use the present tense.
Teachers are aware that some speakers need a transition period to switch to the target language. Although we make this transition a bit at the beginning of the test, for many it will not be long enough. This can make some participants less comfortable, both in terms of understanding and speaking. The result is, then, to undervalue a participant.
Some people are nervous about getting tested. Our test is used to establish a starting point, determine the educational objectives, and form groups with common learning needs. However, the fact remains that some will feel the process is unnatural and experience nervousness. Because of this, they may feel that they underperformed, and the result does not represent the true picture.
Although our pedagogy has ten levels for most languages, participants do not always "fall" precisely into a level. We have several cases that overlap two levels. Moreover, we have some people who can use advanced structures and have problems with weaker grammar points. What do we do with these learners? How can we classify them? Generally, we will tend to place them in the lower level of the two options considered, but not always. This will depend on fluidity, accuracy, vocabulary, etc., all factors to consider. It is often with these people that discrepancies occur. They do not exactly fit into one box.
There will also be discrepancies at the tester level - we are not immune to this. Did the learner say his "ed" at the end of the verb? Did I understand what the learner meant? Usually, the teacher will revisit this structure with a similar question to confirm or reject the first impression. As our test is done orally, the accuracy of the answers is based on the tester's understanding which can sometimes be influenced by a strong or a less common accent, a bad phone line or internet connection, low audio volume, or just someone who does not speak very loudly or who is shy. It should be noted that the teachers are trained in specific methods to create a relaxed atmosphere, to put the speaker at ease, to double-check when in doubt - all this in a brief 20-30 minutes.
Why do we keep our test format? For several reasons.
Although the subject of the article is about discrepancies in testing, the test itself is very good. Although no test is infallible, the fact remains that it is excellent for the vast majority of cases.
Our pedagogy is adapted. If we make a mistake by one level, for example, the teacher adapts by moving on to the next level. We are advised of the situation and the change is made. We do not have to stay with the initial assessment.
Initially, our tests were longer to further limit the discrepancies we just described. But companies have asked us to make groupings between different levels. So, in-depth testing became practically useless.
In training programs for companies with several groups at different levels, mobility is relatively simple. If a learner's level is not what it should be, they can move to a group that will be closer to their level. Although rare, these situations do exist.
Another important point about evaluation; everyone learns at a different pace. We have talked about those who are learning a third or fourth language, and there are those who are very good at acquiring a language, those who do work outside of school hours, those who are not afraid to practice with colleagues, friends, and neighbors, and those who are learning a language from the same root (Spanish for French speakers, for example). There are several reasons why the homogeneous group at the first hour is no longer homogeneous at the 10th hour. So in this context, it becomes counterproductive to put more effort into greater accuracy when testing.
Have we stopped refining our test? No. We regularly review our way of doing things to improve the process.
The takeaway: Our job is to deliver an honest assessment, as we seek to develop quality skills at each level. This is how we build confident speakers.
If you have any questions or comments, please do not hesitate.