El Guiniguada, 31 (2022), pp. 91-100
Print ISSN: 0213-0610 – eISSN: 2386-3374
Servicio de Publicaciones y Difusión Científica Universidad de Las Palmas de Gran Canaria
Servicio de Publicaciones y Difusión Científica (SPDC), Universidad de Las Palmas de Gran Canaria, Parque Científico-Tecnológico, Edificio Polivalente II, C/ Practicante Ignacio Rodríguez, s/n
Campus Universitario de Tafira, 35017 Las Palmas de Gran Canaria, Spain
eISSN: 2386-3374
10.20420/ElGuiniguada.2013.333 (doi general de la revista)
Journal information, indexing and abstracting details, archives, and instructions for submissions: http://ojsspdc.ulpgc.es/ojs/index.php/ElGuiniguada/index
Investigando la Implicatura Conversacional en el Examen de TOEIC
DOI (en Metadatos y en Sumario Revista)
Recibido el 05/08/2021 Aceptado el 30/12/2021
El Guiniguada is licensed under a Creative Commons ReconocimientoNoComercial-SinObraDerivada
4.0 Internacional License.
Investigando la Implicatura Conversacional en el Examen de TOEIC
Andrea Martínez Celis andmar07@ucm.es Carmen Maíz Arévalo cmaizare@ucm.es
Universidad Complutense de Madrid
RESUMEN
La incorrecta inferencia de la implicatura conversacional puede conducir a malentendidos entre los interlocutores que comparten la misma primera lengua (L1), y conlleva un mayor esfuerzo al aprender una segunda lengua (L2). Por esta razón, debe abordarse en los cursos de segundas lenguas (L2). Sin embargo, enseñar a los alumnos a inferir el significado pragmático correctamente cuando se enfrentan a la implicatura conversacional plantea la cuestión de cómo probar si la han inferido debidamente. Este artículo se centra en el examen TOEIC (Test of English for International Communication) en un grupo de cincuenta estudiantes universitarios que han realizado la última versión del examen, donde se incluyen implicaturas conversacionales en las secciones de Listening y Reading. Los resultados muestran que los estudiantes parecen obtener mejores resultados en comprensión auditiva que en lectora gracias a las señales paralingüísticas. La comprensión lectora podría mejorarse mediante una instrucción explícita y una mayor competencia lingüística.
PALABRAS CLAVE
COMPETENCIA PRAGMÁTICA, ENSEÑANZA DE LENGUAS, EXÁMENES OFICIALES
ABSTRACT
The incorrect inference of conversational implicature might lead to misunderstandings among interlocutors who share the same L1, and it entails an extra degree of effort when learning an L2. For this reason, it should be tackled in L2 courses. However, teaching learners to infer pragmatic meaning correctly when faced with conversational implicature raises the question of how to test whether they have inferred it properly. This paper focuses on the TOEIC examination (Test of English for International Communication) in a group of fifty university students who have taken the latest version of the exam, where conversational implicatures are included in the Listening and Reading sections. Results show that learners seem to perform better in the Listening than in the Reading thanks to paralinguistic cues. Reading comprehension could be improved through explicit instruction and a higher level of linguistic competence.
KEYWORDS
PRAGMATIC COMPETENCE, LANGUAGE TEACHING, OFFICIAL EXAMINATIONS
INTRODUCTION
Grice (1975) firstly defined conversational implicature as the inferencing process by means of which the meanings of utterances are interpreted in relation to their context of use (Bouton, 1994). Further developed by Sperber and Wilson (1995) in their Relevance Theory, it can be argued that conversational implicature may often lead to communicative breakdown among interlocutors who share the same L1. Hence, understanding implicit meanings might entail an extra degree of effort on the part of L2 language learners, who can find it particularly demanding independently of their linguistic proficiency (Bouton,1994a; Taguchi, Li & Liu, 2013; Köylü 2018). As a result, it should be tackled in L2 classes so that learners are prepared to engage successfully in realistic communication. Furthermore, learners should be taught how to make use of their pragmatic competence in their L1, where positive transfer might also help to reach suitable inferences. Nonetheless, teaching learners to infer pragmatic meaning correctly when faced with conversational implicature raises the question of how to test whether these learners have managed to infer such pragmatic meaning.
Considering the above, it has been argued that the higher the linguistic proficiency in the L2, the more likely an L2 learner will successfully infer conversational implicature, not only in English but also in other languages like Spanish, Chinese or Japanese (Bouton 1988, 1992, 1994a, 1994b, 1999; Shively, Menke, & Manzón-Omundson, 2008; Taguchi 2005, 2008; Taguchi et al., 2013; Köylü, 2018). Most research, however, has been built on tests specifically designed to check L2 learners’ pragmatic competence (see Roever, 2013; Pratama, Nurkamto, Rustono & Marmanto, 2017), rather than using existing textbooks or official examinations that can determine students’ future career, as is the Test of English for International Communication (TOEIC henceforth). Although there are some exceptions like Wang (2011) on the CET exam or Li (2016) on the use of implicature in textbooks for Chinese EFL learners, the TOEIC examination has received less scholarly attention than other official examinations like TOEFL (cf., Zhang, 2006). Thus, this paper intends to contribute to the existing literature on the comprehension of conversational implicature by L2 learners as part of the latest version of the official examination TOEIC. More specifically, we aim to answer the following research question: Does linguistic competence correlate with the successful inference of conversational implicature?
THEORETICAL FRAMEWORK ARTICULATING THE RESEARCH
Regarding the classification of conversational implicatures, in this paper we shall use the taxonomy put forward by Bouton (1994a) and further developed by Pratama et al., (2017). Nevertheless, for the sake of clarity, we will rely just on what these authors consider the two macro-categories: formulaic and idiosyncratic or contextual implicatures. As they point out:
formulaic implicatures can be explained as implicatures based on certain templates. The templates can be identified by the presence of lexical, semantic, syntactic or pragmatic markers. For example, POPE-Q can be recognized by the presence of rhetorical question and scalar implicatures by the presence of modals. On the other hand, idiosyncratic implicatures are template-less and rely heavily not on semantic or pragmatic markers but on context. […] No lexical, semantic, syntactic or pragmatics markers can reliably help the listener to guess the speaker’s intent in such instance. (Pratama et al., 2017, p. 54)
CONTEXT AND METHODS
The students of this public Spanish university are required to finish their university degrees with, at least, a B1 level according to the Common European Framework of Reference for Languages (CEFR henceforth) of Listening and Reading skills in any of these four languages: English, French, German or Italian. Those who do not possess any certificate that proves, at least, a B1 level must attend a course and pass a total of two exams during the semester. These exams account for 40% and 60% of the final mark, if test-takers achieve the average score, which depends on the language, they will be able to take an official examination in order to obtain the certificate.
The participants of this specific research were a total of 50 English learners. The course that the university provided was guided to obtain the maximum possible mark in the TOEIC official exam. When enrolling in the course, they are required to take an online test and obtain a minimum of A2.3, according to the CEFR, so that they are entitled to take the course. If they do not have this level at that moment, they cannot take the course and are forced to wait until they are able to prove the minimum A2.3 level. It must be mentioned that these students can be at the second, third or fourth course of their bachelor’s degree and that they all come from a wide range of university degrees. The participants of the present study started the course in mid-September and took the most recent version of the TOEIC exam in early November. By then, they had attended four hours a week, around 24 hours, of “standard” English classes. The participants did not receive any kind of instruction on conversational implicature during that time. The data that have been used in this study were collected from the total results of this test (i.e., final marks), Listening and Reading test results separately, and from the specific answers provided by the participants to some selected questions in which conversational implicatures can be found.
RESULTS AND DISCUSSION GENERAL RESULTS
Before presenting in more detail the analysis of conversational implicature in the listening and reading sections, Figures 1 and 2 below offer a visual overview of the participants’ progress according to their initial CEFR level (before starting the course) in the TOEIC examination (during the course). In the case of the listening examination, and except for A2, B1.2 and B2.2 students, whose performance in the test is worse, the other levels show a clear improvement, especially so in the case of B1.3, B2.3 and C1 levels. All in all, there is improvement (as shown by the tendency line) in correlation with the students’ linguistic level (except for B1.2 and B2.2 levels), whose results are better in TOEIC exam.
Figure 1
Students’ progress in the TOEIC exam according to their initial CEFR level (Listening).
0%
0%
0%
C1 B2.3 B2.2 B1.3 B1.2 B1.1 A2
Note. N = 50
100%
100%
37,5%
62,5%
100%
8%
82%
33,5%
66,50%
20%
80%
Decrease Improvement Progress
Interestingly, the reading section of the examination shows a different picture. As Figure 2 shows, in general, students’ answers worsen in the reading section. This seems to pose an extra challenge for most students, even those at higher levels. For example, all the students at the B1.3, B2.2 and B2.3 levels perform worse in the TOEIC exam. A plausible explanation might be that students are not used to this new type of questions involving conversational implicature and rely on the same bottom-up strategies they used to answer the questions in standard English exams. Surprisingly, students at the lowest level (A2) are those who improve more than the rest of the groups. This might be explained as a result of their positive transfer of reading strategies (both bottom-up but especially top-down) from their L1.
Figure 2
Students’ progress in the TOEIC exam according to their initial CEFR level (Reading)
Note. N = 50
In the coming sections, we focus on the detailed analysis of those questions in each section that involve conversational implicature, as part of the latest version of the
TOEIC exam. We will analyze the type of implicature and how students deal (successfully or not) with its interpretation.
CONVERSATIONAL IMPLICATURES IN THE LISTENING SECTION
Four questions in the listening section rely on the inference abilities of learners’ to correctly interpret implicature. Notwithstanding, for the sake of space, just one example will be shown. In this example, an formulaic implicature is found. More specifically, this implicature is a scalar type of implicature based on epistemic modality (“I’m sure…”):
[Context: After mentioning a successful agreement recently achieved, a woman in an office asks her boss for permission to leave earlier on Friday because she wants to participate in a sport event organized by the company knowing that there is still a lot of work to do.]
Why does the boss say, “I’m sure that Carrie can help out on Friday”?
To disagree with a colleague’s opinion
To express dissatisfaction with the woman’s actions
To confirm an approaching deadline
To approve the woman’s request ll
As Figure 3 shows, the tendency line demonstrates that the number of right answers grows proportionally to the level of the students according to the CEFR. The higher the level, the more correct their answer is. This seems to show that there is a relationship between linguistic proficiency and the capacity to infer correctly, despite exceptions like the case of B2.2 level students, where 37.5% of the participants have given a wrong answer to the question.
Figure 3
Students’ results according to CEFR level
Wrong Right Progress
C1.1 B2.3 B2.2 B1.3 B1.2 B1.1 A2
100%
100%
62,5%
33,5%
8,0%
37,5%
66,5%
100%
82%
80,0%
20%
Note. N = 50
In the case of wrong answers, the most common tendency is to answer C, especially by B1.1 participants, as can be seen in Figure 4.
Figure 4
Distribution of "answer C" according to CEFR level
B1.2 25%
B2.2 13%
A2 [PORCE NTAJE]
B1.1 37%
Note. N = 50
Learners seem to mistake the meaning of “I’m sure” as a synonym of “confirm” rather than an indirect approval of the prior request to leave the office earlier than usual. This shows participants’ lack of attention to the rest of the context. Their answer is based on establishing quick lexical relationships or even on literal translation from their L1.
CONVERSATIONAL IMPLICATURES IN THE READING SECTION
As in the listening section, there were four questions where the correct answer depends on the successful inference of a conversational implicature. In the reading section, however, students have more linguistic support as they can go back to the text, if they consider that they have enough time to do it, whilst the listening is only played once during the examination. For the sake of space, we shall focus on the analysis of one of these questions, as we did in Section 3.2. This question is based on a chat conversation where students have to correctly infer that Ms. McGuigan’s problem has been solved, being (A) the correct answer.
[Context: Two colleagues are talking about a company's volleyball tournament that will take place soon. Speaker A asked Speaker B about her department's team, but Speaker B says that she is three players short. Speaker A suggests she should join another department to form a team together.]
At 4:58 P.M., what does Ms. McGuigan most likely mean when she writes, “That’ll take the pressure off”?
She will not have to find more players in her department. ll
She will not have to postpone the tournament.
She will not have to change the tournament rules.
She will not have to play in the tournament.
In this case, students are faced with an idiosyncratic or contextual conversational implicature, as the correct inference depends not so much on linguistic items as on background knowledge. Surprisingly, and except for B2.2 and B2.3 levels, students seem to find this item more complex to infer and, even if there is a certain correlation between their initial level and their level of success, we can also observe that, even at higher levels like C1.1, they seem to encounter difficulties (see Figure 5). It can be argued that the linguistic level alone does not ensure a correct interpretation of
contextual conversational implicatures although this might be the case when dealing with formulaic ones, which rely more heavily on language itself.
Figure 5
Students’ results according to CEFR level
Wrong Right Progress
C1.
B2.
B2.
0,0%
0,0%
40,0%
60,0%
100,0%
60,0%
100,0%
60,0%
50,0%
B2. | 40,0% | |
B1. | 40,0% | |
B1. B1. A2 | 25,0% |
50,0%
50,0%
50,0%
75,0%
Note. N = 50
The most selected wrong answer to this question was C: ‘‘She will not have to change the tournament rules’’, as illustrated by Figure 6:
Figure 6
Distribution of "answer C" according to CEFR level
B1.3 17%
B2.1
8%
A2 [PORC ENTAJ
E]
B1.2 25%
B1.1 25%
Note. N = 50
Finding the same wrong answer in most levels seems surprising considering that virtual chats are most probably the most used channel of communication nowadays, especially among people at these participants’ age. A possible reason is that chat conversations need more context than any other category to be fully understood because the text is normally too short, thus, readers need to go beyond words. The reason why a great number of participants have chosen option C may be that they automatically related the word ‘‘pressure’’ to the word ‘‘rules’’ due to an unconscious lexical connection. It is common to find these words in the same context, for example, the existence of rules may cause pressure on people. Therefore, their general failure
may be related to lack of time to reread the text and, consequently, make quick lexical relationships instead of trying to establish a logical context.
DISCUSSION AND CONCLUSIONS
This paper examined the results obtained from a total of fifty students in the latest version of the TOEIC exam, which includes conversational implicatures. In addition to this, the answers provided by the participants to some specific questions in which conversational implicatures are found have been analyzed in order to hypothesize about the possible reasons for their most common wrong answers.
With the purpose of answering the previously formulated research question, it must be mentioned that the results of this study show that, in accordance with previous findings (Shively et al., 2008; Taguchi 2005, 2008), linguistic proficiency does help in the correct inference of conversational implicature, and therefore in the improvement of their L2 pragmatic competence, but not necessarily so. Furthermore, in general terms, and in agreement with previous studies (cf. Bouton 1994; Taguchi et al. 2013; Köylü, 2018), participants do not seem to be aware of conversational implicatures at least without receiving explicit instruction. Nevertheless, students perform better in the Listening section of the TOEIC exam most probably due to paralinguistic cues. In addition to this, even if students have the skill to infer conversational implicatures — especially idiosyncratic ones— in their L1, they do not always transfer positively this skill to the L2 because they translate literally. This may even result in negative consequences as shown in the first example of the Listening section. As a result, they fail to answer correctly, doing it at random or by establishing quick lexical relationships rather than taking context into consideration and hence, inferring the implicit meaning (Grice, 1975). The causes may be also related to the fact that test- takers are not used to this type of questions in which pragmatic skills are tested, and they are mostly trained to understand and use grammatical rules rather than to interpret beyond words. As a conclusion, the new version of the TOEIC examination seems to be more useful for real communication, but not “easier” for Spanish students.
The limitations of this study must be also mentioned. The initial idea was to compare the results of two different groups of fifty students each, taking both the former and the new version of the TOEIC exam. One of the groups would have received explicit instruction on pragmatics, while the other would have continued learning according to the standard curricula (and without explicit pragmatic instruction). However, due to logistics and as a first step, we eventually focused on the answers provided by just one group of fifty students who had taken the latest version of the TOEIC exam without having received any kind of instruction on pragmatics.
Additionally, it is important to point out that the participants’ initial level is based on an online test which might not render wholly accurate results. Moreover, because of the nature of the subject, which agglutinates students from different academic courses, there were not the same number of participants per level of English according to the CEFR. All things considered, it is also assumed that a limited number of participants from a determined Spanish university cannot represent the whole situation.
To conclude, the researchers’ hypothesis that derives from the present results is that explicit instruction on pragmatics and, more particularly, on conversational implicature, both formulaic and idiosyncratic or contextual, might lead to better
results. Notwithstanding, for future research, explicit pragmatic instruction — independently of linguistic competence— is required in addition to a more solid methodological design, where both a control and test group can actually be compared.
REFERENCES
Bouton, L. F. (1988). A cross,cultural Study of Ability to Interpret Implicatures in English. World Englishes, 7(2), 183-196.
Bouton, L. F. (1992). The Interpretation of Implicature in English by NNS: Does It Come Automatically--Without Being Explicitly Taught? Pragmatics and Language Learning, 3, 53-65.
Bouton, L. F. (1994a). Conversational implicature in a second language: Learned slowly when not deliberately taught. Journal of pragmatics, 22(2), 157-167.
Bouton, L. F. (1994b). Can NNS Skill in interpreting implicature in American English be improved through explicit instruction? A pilot study. Pragmatics and Language Learning 5, 88-108.
Bouton, L. F. (1999). Developing Nonnative Speaker Skills in Interpreting Conversational Implicatures in English. Culture in second language teaching and learning, 30(1), 47-70.
Grice, H. P. (1975). Logic and conversation. In P. Cole & J. Morgan (Eds.), Syntax & Semantics (Speech Acts ed., Vol. 3, pp. 41–58). Academic Press.
Köylü, Y. (2018). Comprehension of conversational implicatures in L2 English. Intercultural Pragmatics, 15(3), 373-408.
Li, Q. (2016). Conversational implicature in English listening comprehension teaching. Theory and Practice in Language Studies, 6(10), 2044-2051.
Pratama, H., Nurkamto, J., Rustono, R., and Marmanto, S. (2017). Second Language Learners' Comprehension of Conversational Implicatures in English. 3L: Language, Linguistics, Literature®, 23(3), 50-66.
Roever, C. (2013). Testing implicature under operational conditions. In Assessing Second Language Pragmatics (pp. 43-64). Palgrave Macmillan.
Shively, R. L., Menke, M. R., & Manzón-Omundson, S. M. (2008). Perception of irony by L2 learners of Spanish. Issues in Applied Linguistics, 16(2).
Sperber, D., & Wilson, D. (1986). Relevance: Communication and cognition (Vol.
142). Cambridge, MA: Harvard University Press.
Taguchi, N. (2005). Comprehending implied meaning in English as a foreign language. The Modern language journal, 89(4), 543-562.
Taguchi, N. (2008). Pragmatic comprehension in Japanese as a foreign language. The Modern Language Journal, 92(4), 558-576.
Taguchi, N., Li, S., & Liu, Y. (2013). Comprehension of conversational implicature in L2 Chinese. Pragmatics & Cognition, 21(1), 139-157.
Wang, H. (2011). Conversational Implicature in English Listening Comprehension. Journal of Language Teaching & Research, 2(5).
Zhang, S. (2006). Investigating the relative effects of persons, items, sections, and languages on TOEIC score dependability. Language Testing, 23(3), 351-369.