In one of our 510 course meeting, we talked about vocabulary learning which reminds me of these two articles on vocabulary learning. I would like to evaluate these two articles reporting two studies which employed two different experimental designs.
Gettys, Imhof, and Kautz (2001) writing in Foreign Language Annals titled Computer-Assisted Reading: The effect of Glossing -Format on Comprehension and Vocabulary Retention targeting language teaching practitioners as their readers. The piece has a clear purpose which is to compare two glossing formats and to determine which one is more beneficial for reading comprehension and vocabulary retention. Since the target reader of this article is language teaching practitioners, I think this article’s research questions are interesting for them as they promise answers to the fundamental inquiry in vocabulary learning: what is the best way to learn vocabulary. Another good thing about the research questions is that the are measurable which makes it easy to track whether or not they are answered thoroughly and in fact, they are answered thoroughly in the later parts of the article.
The design of this experimental study responds well to its purpose and research questions. Both the experimental and the qualitative components of the study on students’ perception of their learning serve the pursuit of the research questions’ answers well. However, the number of the participant might be too small,only 22 participants, that might threat the statistical significance of the results. A crossover method was performed which means both experimental dan control group had both treatments, but with different sequences. I think this crossover method is suitable to assure higher validity result in experimental studies. To get data about students’ reading comprehension and vocabulary retention, the authors conducted recall protocols, multiple-choice, short-term-memory vocabulary test, questionnaire were employed which I think were suitable for the desired comprehension and word retention constructs.
The results of the study showed that the group exposed to basic dictionary forms demonstrated a significantly greater overall retention of vocabulary. The tests on comprehension of the text yielded not so clear results and do not show a significant statistical difference. The result is surprising as I expected that sentence level equivalent glossing would help students better with word retention. Apart from the surprising result, what I like about this article and study is that all the questions are answered and the article is organized well. Also, the author used relevant sources which were published relatively current to the date of this study was conducted.
The second article by Tozcu and Coady (2004) titled Successful Learning of Frequent Vocabulary Through CALL also Benefits Reading Comprehension and Speed clearly states the research questions which are inline with the purpose of this experimental study. This study investigated the effect of direct vocabulary learning using Computer Assisted Language Learning (CALL) on vocabulary knowledge, reading comprehension, and speed of word recognition. The design of the study is straightforward and responding well to the research questions. This study has a sufficient number of subjects, 56 intermediate level students studying English full time for university academic preparation and randomly assigned to treatment (28 students) and control groups (28 students).The treatment group studied at a frequency level (of 1,000words) with vocabulary software (Tutorial CALL). The control group was required to read two 2-page articles per week, and answer four comprehension questions based on the articles. Both the treatment and control groups were given the same tests which were used as pre-and posttests in vocabulary, reading comprehension, and reaction time.
I think the results of this study are useful even if not so surprising as research has shown that CALL benefits language learning. The result shows that students who used tutorial CALL did learn a significantly larger number of words than those in a control group. They decreased their reaction time for frequent word recognition as compared to the control group and they exhibited significantly better reading comprehension than a control group. Although both groups showed increases in vocabulary gain, and reading comprehension, and a significantly greater gains than the control students.
Both articles evaluated here have a pretty straight forward structure which makes it easy to navigate around the article. The headings and subheadings are clear and the overall face validity is good. Both studies have well-constructed designs that respond well to the need of the research inquiries. In addition, the second article includes pedagogical implication part which I think teachers can relate to. Even though both studies are experimental studies, but the designs are different in terms of how treatments are given for the experiments and control groups; Gettys (2001) used cross over and Tozcu (2004) used conventional method. The crossover method provides reader with insights on how to increase the validity through changing the sequence of the treatments. On the other hand the conventional experimental method might be familiar for the reader therefore it is easier for them to follow the logic of the design.
Here are some resources for Crossover designs:
- Power point presentation on crossover-analysis.
- Penn State web page on Crossover design
- Youtube video on Cross Over design.
Refferences
Gettys, S., Imhof, L.A, Kautz, J.O. (2001).Computer-assisted reading: the effect of glossing – format on comprehension and vocabulary retention. Foreign Language Annals, 34(2), 91-99.
Tozcu, A., & Coady, J (2004). Successful learning of frequent vocabulary through CALL also benefits reading comprehension and speed. Computer Assisted Language Learning, 15(55), 473-495.