ICT4LT Module ICT4LT Module
The aims of this module are for the user to consider key issues in assessing language skills through ICT in order to be able to:
This Web page is designed to be read from the printed page. Use File / Print in your browser to produce a printed copy. After you have digested the contents of the printed copy, come back to the onscreen version to follow up the hyperlinks.
Terry Atkinson, Freelance Educational Consultant, UK/France.
Graham Davies, Thames Valley University, UK (Editor & Section 6).
Computer Aided Assessment (CAA) covers a range of assessment procedures and is a rapidly developing area as new technologies are harnessed. In essence, CAA refers to any instance in which some aspect of computer technology is deployed as part of the assessment process. Some of the principle examples of CAA in language learning are:
CAA is more than just a list of possible applications, however. Its importance is intimately bound up with raising achievement, since it can be argued that the role of ICT in raising achievement cannot be fully measured unless ICT is also used in the assessment process.
Just as oral skills cannot easily be assessed by a written test, so there are ICT-specific language skills that cannot easily be assessed through pencil and paper exercises and tests. Consider the following examples:
If students spend time practising the above activities on computers the intention is to raise achievement in language learning at a general level, and this might well be picked up in a conventional test or examination. However, as the balance moves further and most of students' reading is online and most of their writing is computer-based, using paper technology is unlikely to enable students to demonstrate fully their skills, particularly ICT-related ones such as using writing tools - spellchecker, thesaurus, etc. An integrated approach to teaching, learning and assessment is always likely to be more successful than a random approach. Thus, teaching methods, learning methods and assessment methods need to cohere if learners are to learn successfully and if valid and reliable results are to be achieved from assessment procedures. This is true whether or not ICT is to be used - extensive oral practice activities and no oral exam is still a problem in many classes, and so too is extensive use of computers and no opportunity to use computers as part of the assessment process. Moreover, the incentive for teachers to incorporate learning technologies into classwork is reduced if the examinations do not use these technologies or expressly prohibit their use.
Formative assessment: In general, CAA is used mainly for formative assessment rather than summative assessment because it is excellent for giving immediate feedback, e.g. in tests designed to measure students' progress in specific areas, either for self-assessment purposes or for the teacher, e.g. as in placement tests (see Section 1.4).
Summative asssessment: There is a good deal of discussion at present regarding the use of CAA - also dubbed e-assessment - in examinations at the end of a course and in national examinations such as the GCSE examinations in England. It's a controversial topic and has been subjected to a good deal of media hype, with outrageous claims being made regarding its possible uses. To what extent do you think CAA can be used to carry out summative assessment? Jot down your immediate thoughts on this question and the concerns that you might have if your own students were to be assessed in this way. Then read Section 1.3, Which skills can be assessed? Go back to your list of concerns and see which ones have been answered fully, partly or not at all. Send a message to us, using our Feedback Form, if you still have concerns that have not been answered.
Use of CAA for summative assessment is increasing, especially in higher education. Trainee language teachers in England and Wales are now expected to pass computer-based skills tests in literacy, numeracy and ICT, so some of the readers of this module have already experienced summative CAA and will be familiar with the procedures that make it possible. These include:
The above measures can ensure that there is as much security involved as in paper-based exams and the process is not too dissimilar to that used for carrying out orals. Perhaps a more important question is whether computer-based tests can really assess language skills and, if so, which skills are best assessed through CAA and which through other formats. These questions are addressed in the next section. Summative assessment is not likely to be widely implemented in the near future as there are still a number of concerns about its reliability, but it is likely that ingenious solutions and new technologies will bring about a much greater degree of summative assessment than is currently possible. Placement testing and adaptive testing are more likely to be introduced in some areas: see Section 2.3.
| Skill | Assessment by computer | Assessment of electronic output by human being |
| Listening | Limited range of responses and response modes to test comprehension. | Listening tests can be presented on a computer, students' answers can be stored electronically and assessed by a teacher. Self-assessment and peer assessment are also possible. |
| Speaking | Very limited as yet. Automatic Speech Recognition (ASR) software is developing rapidly but it is still too unreliable to be used in accurate testing. See below under Speaking. | Students can record their own voices on a computer for assessment by a teacher. Self-assessment and peer assessment are also possible. |
| Reading | Limited range of responses and response modes to test comprehension. | Reading tests can be presented on a computer, students' answers can be stored electronically and assessed by a teacher. Self-assessment and peer assessment are also possible. |
| Writing | Very limited as yet, but spell checking and grammar checking are developing. See Section 6.1, Module 1.3. | Students' answers can be stored electronically and assessed by a teacher. Self-assessment and peer assessment are also possible. |
There are many CD-ROMs and websites that provide listening activities that assess comprehension. In its simplest form, the student listens to an item and then chooses an answer from some form of multiple-choice menu. If well designed, such tests work effectively and they can offer instant feedback to the student, which has a beneficial effect on learning. The main limitations of CAA for testing listening skills can be summarised as follows:
Despite these objections, assessment of listening comprehension by computer can be of great value to students, offering a form of comprehensible input (Krashen 1985). Moreover, computer-based listening comprehension can combine sound with text, still images, video, animation and on-screen interactivity which creates thereby a much richer environment than is otherwise possible: see Module 2.2, Introduction to multimedia CALL, and Module 3.2, CALL software design and implementation. A measure of student control to allow ease of navigation, options to retry or move to a different section, to attempt different tasks or roles etc are vital to ensure active participation. Good equipment is also vital: headphones, fast network/Internet access and/or networked CD-ROMs.
To assess speaking is a very complex task and, as yet, computers are not fully adequate in either Automatic Speech Recognition (ASR), but research in this area is developing rapidly. For further information on ASR see:
Limited assessment of speaking skills is possible. Self-assessment and peer assessment can be managed if microphones are available to allow students to record themselves and listen to the playback. A number of multimedia CD-ROMs have this feature: see Module 2.2.The Encounters series of CD-ROMs allows students to take part in a role-play by recording themselves and then saving the whole role-play on disk, with their own voices slotted into the appropriate positions in the role-play, for assessment by a teacher: http://www.camsoftpartners.co.uk/encounters.htm
See the software for automated testing of spoken English produced by Versant.
At a basic level it is simple to assess reading comprehension in much the same way as it is possible to assess listening comprehension, e.g. with multiple-choice, drag-and-drop and fill-in-the-blank tests: see Listening. There are many CD-ROMs and websites that offer interactive exercises and tests with these functions. More extended reading tasks are harder to set on computer. On-screen reading is difficult with longer texts. Research indicates that people read around 25%-30% more slowly from a computer screen. Web guru Jako Nielsen writes:
"Reading from computer screens is about 25% slower than reading from paper. Even users who don't know this human factors research usually say that they feel unpleasant when reading online text. As a result, people don't want to read a lot of text from computer screens: you should write 50% less text and not just 25% less since it's not only a matter of reading speed but also a matter of feeling good. We also know that users don't like to scroll: one more reason to keep pages short. [...] Because it is so painful to read text on computer screens and because the online experience seems to foster some amount of impatience, users tend not to read streams of text fully. Instead, users scan text and pick out keywords, sentences, and paragraphs of interest while skipping over those parts of the text they care less about." Be Succinct! Writing for the Web, Alertbox for March 15, 1997: http://www.useit.com/alertbox/9703b.html
Extended texts are more likely to be print-based unless they are in hypertext format, i.e. separate pages linked together as on the Web, or CD-ROM reference materials. In the case of hypertext, the computer may be a suitable medium for assessing information-gathering techniques. The skills needed to track down documents, follow links within and between them and find specific extracts are of increasing importance in academic life, in commercial settings and in leisure time. These skills are, to an extent, generic rather than specific to any given language although learners do need to know key terms involved in searching for information. Many teachers are aware of the need to ensure that learners are equipped with the appropriate language skills for Web browsing, e.g. foreign language terms for help, search, next page, OK etc.
As for speaking (see Speaking), the language processing capacity of computers is as yet too limited to enable computers to mark students' written work, apart from single words and short sentences typed at the keyboard. Work on parsing students' typed responses and error diagnosis is in progress - see Section 8 in Module 3.5, headed Parser-based CALL, and see Heift & Schulze (2003). However, there are features in modern computers that can be used within the assessment process such as spellchecking to enable self-assessment of spelling and even grammar and style checking, which although very imperfect, does pick up some errors that students can use to self-correct, such as errors of gender or number: see example below which is a screenshot from Microsoft Word. See Section 6.1, Module 1.3, headed Spellcheckers, grammar checkers and style checkers.
Figure 1: Screenshot, Microsoft Word Spellchecker
The main use of computers in the assessment of writing is, currently, the use of on-screen marking as described in Section 3.
You can read more about grammar checkers in:
Computer-based exercises and tests often take the same kind of format. The essential difference between an exercise and a test is the purpose to which it is put. An exercise usually offers instant feedback to the learner and an opportunity to correct any errors that are made, whereas a test may offer little feedback to the learner apart from a raw score at the end of the test, or no feedback at all, e.g. where the results of the test might be stored for analysis by a teacher or examination body. Exercises are usually designed to offer the learner practice in specific areas and to motivate and encourage, whereas tests are usually designed to assess the leaenr's progress in specific areas, i.e. for self-assessment purposes, for the teacher or for an examination body. But sometimes these distinctions become blurred. The main kinds of tests include:
See the website of Linguanet Europa. This project has recently undergone expansion to incorporate an interface in a number of new languages and addresses in particular the needs of adult learners and independent learners. The site includes advice on ways of assessing and improving one's current ability in different languages (including links to websites that offer diagnostic and placement tests), communicating electronically with other language learners and finding appropriate resources. A substantial online catalogue of language learning resources is also being built up here: http://www.linguanet-europa.org
When reading this section bear in mind the distinction that was made earlier between exercises and tests. See Section 1.4, headed Exericise or test?
The feedback loop refers to the process through which the learner is giving an opportunity to produce an answer and to receive feedback on his/her input. Feedback is essential to learning, but the nature of the feedback is also important and the computer has certain advantages and disadvantages in providing that feedback. See:
See also: Bangs (2003).
The two main advantages of feedback are that it can be given (i) instantly and (ii) in a non-judgemental way. The major disadvantage is that the feedback given may not necessarily be adequately discriminatory or differentiated. It is possible to provide detailed, comprehensive feedback but this is very time-consuming. Consider a multiple-choice test in which four answers are given for each question. There are lots of software packages that allow the authoring of multiple-choice tests of this sort and for specific feedback to be written in for each answer. However, constructing such a test takes a significant amount of time, both in thinking through the details of the feedback and in typing it all in: see Module 2.5 , Introduction to CALL authoring programs. Despite this disadvantage, the two main advantages of feedback do bring excellent learning opportunities which can be authored by anyone with basic computer literacy. Heather Rendall has shown how a very simple program in which jumbled sentences have to be re-ordered can be used to develop grammatical insight: see the report on her research project in Section 5, Module 1.4. See also Rendall (2001).
The immediate assessment that the learner receives provides the key to learning. Consider the following example:
the talks boys teacher to the
The task is to enter the text in the correct order, i.e.
The teacher talks to the boys
Consider what the learner must know in order to get this correct: Meaning? Word order? Verb forms? If the learner gets the answer wrong, he/she will be presented with the correct answer immediately. Whilst this is not the most sophisticated form of feedback, Rendall's research points towards the power of this type of feedback in developing an instinctive grasp of the working of the language. Her subjects justify their thinking by saying "it sounds right", so the effect of feedback is to develop a sense of what is linguistically correct. This takes lots of practice but with sentence forms like the one in the above example, banks of exercises can be quickly generated. In fact, it works best at short sentence or phrase level because that eliminates ambiguity. Here are some grammar points than can be tested through this simple procedure:
Can you draft some examples of the above points for the language(s) that you teach? Can you think of other points that could be tested in this way?
The most common exercise types are listed below:
It should be borne in mind that since the advent of the multimedia computer tests and exercises can contain a variety of different kinds of presentation of the stimulus, student input and feedback, e.g. a multiple-choice test could consist of
The possibilities are enormous. See Section 5, Module 3.2, headed Template examples.
Collections of ready-made exercises exist for the more commonly taught languages. These can be found in software catalogues and are increasingly available online via websites: see the Websites list (below). Many exercises have been developed by teachers and are free for non-commercial use. As an alternative to buying ready-made exercises, it is possible to use authoring programs to develop your own exercises: see Module 2.5, Introduction to CALL authoring programs. The following authoring programs are all in common use.
Hot Potatoes
This software was
especially developed to create Web-based language exercises, including multiple
choice, gap-filling, matching, jumbled sentences, crosswords and short text
entry. It is used extensively on websites that provide interactive exercises
and tests. Visit the Hot Potatoes website to find out more, download
the software and see lots of examples: http://hotpot.uvic.ca.
See Winke & MacGregor (2001) for a review of Hot
Potatoes.
Quia
This is an online provider of exercises. There are many language learning exercises
which are free to use. It is also possible to develop your own exercises. It
is very easy to do this and to store the exercise on the Quia website,
although subscription charges may apply: http://www.quia.com
Fun With Texts
Fun With Texts
is one of the most widely used authoring packages for Modern Foreign Languages
in UK secondary schools and there are many ready-made exercises available as
well as full authoring functions within the program. Its use is primarily in
total text reconstruction - so-called "total Cloze" - but it includes
other exercise types too. Version 4.0 includes multimedia enhancements that
allow you to integrate images and audio and video recordings into the exercises.
See Section 8, Module 1.4, headed Text
manipulation, and see http://www.camsoftpartners.co.uk/fwt.htm
Question Mark
The Question Mark
company develops software for presenting interactive exercises and gathering/collating
test information: http://www.questionmark.com
TaskMagic:
Authoring tool for the creation of a variety of exercise types, including.text
match, picture match, sound match, picture-sound match, grid match, mix and
gap, exercises based on dialogues. See http://www.mdlsoft.co.uk
and see http://www.camsoftpartners.co.uk/taskm.htm
Websites
See the Websites
list (below).
The Council of Europe's Common European Framework of Reference for Languages (CEFR or CEF) is a scheme of assessment based on research conducted by the Council of Europe dating back to the 1970s.
The following description of the Common European Framework of Reference was taken from the Council of Europe's website:
Developed through a process of scientific research and wide consultation, this document provides a practical tool for setting clear standards to be attained at successive stages of learning and for evaluating outcomes in an internationally comparable manner.
It is the result of extensive research and ongoing work on communicative objectives, as exemplified by the popular 'Threshold level' concept.
The Common European Framework of Reference (CEFR) for Languages provides a basis for the mutual recognition of language qualifications, thus facilitating educational and occupational mobility. It is increasingly used in the reform of national curricula and by international consortia for the comparison of language certificates.
A European Union Council Resolution (November 2001) recommended the use of this Council of Europe instrument in setting up systems of validation of language competences.
The CEFR is a document which describes in a comprehensive manner
- the competences necessary for communication
- the related knowledge and skills
- the situations and domains of communication
The CEFR is of particular interest to course designers, textbook writers, testers, teachers and teacher trainers - in fact to all who are directly involved in language teaching and testing. It facilitates a clear definition of teaching and learning objectives and methods and provides the necessary tools for assessment of proficiency.
The CEFR has become a key reference document and valuable tool for educational and professional mobility. It is available in nearly thirty language versions.
The CEFR proposes six common reference levels, each requiring a certain number of guided learning hours (GLH). The Council of Europe's publications specify the number of guided learning hours required for the six CEFR levels, e.g.
The Waystage publication (1990 edition): "The learning-load involved is estimated to be about half of that required by Threshold Level 1990, which means that we think that, with proper guidance, the average learner should be able to master it in some 180-200 learning-hours, including independent work." (p. 2 of this 154-page publication: ISBN 92-871-2002-1)
The Threshold publication (1990 edition): "If pressed to give a general indication, nevertheless, we can only say, at this stage, that we assume the learning-load for Threshold Level 1990 to be similar to that for its predecessor [i.e. the earlier publication on Threshold] and that there is some evidence that, with adequate guidance, absolute beginners need an average of c. 375 learning hours - including independent work - to reach the older objective." (p. 6 of this 252-page publication: ISBN 92-871-2003-X)
See the Cambridge University ESOL website at: http://www.cambridgeesol.org, especially:
See also the website of the Association of Language Testers in Europe (ALTE) at http://www.alte.org . ALTE also provides a comprehensive list of "can do" statements.
Appendix C of the CEFR document contains the descriptors for self-assessment ("can do" statements) and reporting back to learners in relation to the CEFR levels adopted by DIALANG.
Like the CEFR, Asset Languages offers sets of "can do" statements that describe what the learner is capable of doing in the different skills. The "can do" statements can be downloaded from the Asset Languages website in PDF format, e.g.
Writing Grade 3: (Stages 1-3 = Breakthrough Level = CEFR A1 = NQF Entry Level)
I can write a few short sentences with support using expressions which I have already learned.Listening Grade 4: (Stages 4-6 = Preliminary Level = CEFR A2 = Foundation GCSE)
I can understand the main points and some of the detail from a short spoken passage.Speaking Grade 5: (Stages 4-6 = Preliminary Level = CEFR A2 = Foundation GCSE)
I can give a short prepared talk, on a topic of my choice, including expressing my opinions.Speaking Grade 6: (Stages 4-6 = Preliminary Level = CEFR A2 = Foundation GCSE)
I can give a short prepared talk, on a topic of my choice, expressing opinions and answering simple questions about it.Reading Grade 7: (Stages 7-9 = Preliminary Level = CEFR B1 = Higher GCSE)
I can understand longer texts and recognise people's points of view.
The CEFR "can do" statements are a bit more explicit, e.g. Level A2 Listening (2 "can do's" are listed here out of a list of 10) = Foundation GCSE:
The complete text of the Common European Framework document can be downloaded in PDF format from the Council of Europe's website. It is also available in book format from Cambridge University Press.
The following table shows the correspondences between the CEFR Levels, Cambridge General English Exams, the UK Languages Ladder Levels, National Curriculum (England) Levels, National Examinations (England) and the number of Guided Learning Hours required for each level. Note: "England", not Wales, Scotland and Northern Ireland, which have different curricula and examination schemes - although they are broadly similar in most respects.
|
CEFR Level
|
Cambridge General English Exams |
Languages
Ladder |
National
Curriculum (England) |
National
Exams (England)
|
Guided
Learning Hours |
| A1 Breakthrough |
1-3 |
1-3 | NQF Entry Level 1-3 | Approx 90-100 hours | |
| A2 Waystage |
Key English Test (KET) | 4-6 Preliminary |
4-6 | Foundation GCSE | Approx 180-200 hours |
| B1 Threshold |
Preliminary English Test (PET) | 7-9 Intermediate |
7-8 & EP* | Higher GCSE | Approx 350-400 hours |
|
B2 |
First Certificate in English (FCE) | 10-12 Advanced |
AS / A / AEA Level | Approx 500-600 hours | |
| C1 Proficiency |
Certificate in Advanced English (CAE) | 13 Proficiency |
University Degree Level and above | Approx 700-800 hours | |
| C2 Mastery |
Certificate of Proficiency in English (CPE) | 14 Mastery |
University Degree Level and above | Approx 1000-1200 hours |
The learning hours required may vary, of course, depending upon other factors such as the learner’s mother tongue, other languages already learned, the intensity of the course being followed, the inclination and age of the learner, and the amount of exposure to the language outside lesson times, particularly if the course takes place in a country where the language is spoken.
What level of competence have students reached and how can it be assessed? As learners move through the system the need for accurate information on the level reached is vital. Increasingly, computers are being used for diagnostic testing.
DIALANG is a major EU-funded project aiming to provide effective diagnosis of language competence in 14 EU languages, based on the Common European Framework of Reference (CEFR) for Languages: see Section 2.2 (above). It uses online tests, including placement and self-assessment tests, as key tools in this process: http://www.dialang.org
DIALANG is for use by the general public and can be downloaded from the DIALANG website. Tests are available in Listening, Writing, Reading, Structures and Vocabulary for 14 different languages: Danish, Dutch, English, Finnish, French, German, Greek, Icelandic, Irish, Italian, Norwegian, Portuguese, Spanish and Swedish.
Figure 2: Screenshot, DIALANG
The EU-funded WebCEF project (2006-2009) aims to enable the collaborative assessment of oral language proficiency through a Web-based environment: http://www.webcef.eu
Language learners and language teachers will be able to evaluate their own video and audio samples together with colleagues and peers across Europe. Like DIALANG this project is also based on the descriptors of the Common European Framework of Reference (CEFR): see Section 2.2 (above).
Registered members of the WebCEF community can create, upload, rate and comment on video and audio samples. Using low-threshold technology, they can create their own video and audio samples to be assessed by a community of language learners across Europe.
WebCEF gives:
Adaptive tests aim to assess language competence by asking questions based on the student's response to previous ones. If the student gets the answer right, a harder question is asked and vice-versa. This process claims to quickly find the level of a student in order to assist schools in placing the student in a suitable class. Brigham Young University in the US has pioneered web-based adaptive language testing for placement using it's WebCAPE system: http://webcape.byu.edu. An example in Spanish is shown below:
Figure 3: Screenshot, WebCAPE
For a full discussion of computer adaptive testing in language learning see Chalhoub-Deville (1999).
This form of testing may become much more common in the future. As noted earlier, Examination Boards are already experimenting with computer-based examining for public examinations such as GCSE and A level. It may soon be commonplace for schools to decide on setting of students by such methods. Teacher training departments may use this form of testing for assessment or self-assessment of trainees' language skills.
BBC Languages: Basic tests in a variety of languages to enable learners to assess their knowledge, plus a range of online courses at different levels: http://www.bbc.co.uk/languages/
The Council of Europe European Language Portfolio provides an attractive, portable and motivating way for individual learners to maintain an up-to-date record of their language learning experience and to track their changing levels of performance. It also contains can-do statements which help learners to assess their language skills: http://www.coe.int/t/dg4/linguistic/
LearnDirect: Language tests and courses are available in French, German and Spanish from the LearnDirect website. Select the language in which you are interested and then download the language diagnostic software to assess you level of knowledge. LearnDirect is a UK government-sponsored online training initiative: http://catalogue.learndirect.co.uk/browse/languages/
Goethe-Verlag: Free tests in 24 languages and 552 language combinations: http://www.goethe-verlag.com/tests/
Mahoney Training Consultants: Many employers are using computer-based testing to assess language competence. Catherine Mahoney's Bright assessement tests are available on CD-ROM or online in a range of languages: http://www.mtcbright.com
TOEFL: A large market clearly exists for testing English language competence and there is already a thriving computer-based testing industry. See the Test Of English as a Foreign Language website: http://www.toefl.org
Versant Tests: Formerly known as the Spoken English Tests (SET), the Versant Tests are delivered over a telephone or on a computer and scored by computer. Once administered, numeric scores and performance levels that describe the test-taker's ability to understand and speak the selected language are generated within minutes and can be viewed online. Tests can be taken anytime, anywhere: http://www.versanttest.co.uk
See also the website of Linguanet Europa. This project has recently undergone expansion to incorporate an interface in a number of new languages and addresses in particular the needs of adult learners and independent learners. The site includes advice on ways of assessing and improving one's current ability in different languages (including links to websites that offer diagnostic and placement tests), communicating electronically with other language learners and finding appropriate resources. A substantial online catalogue of language learning resources is also being built up here: http://www.linguanet-europa.org
See also Graham Davies's
Favourite Websites page at:
http://www.camsoftpartners.co.uk/websites.htm
This page contain links to many websites that offer interactive exercises and
tests.
Contents of Section 3
Once upon a time students wrote in exercise books in blue pen and teachers collected these up and marked them in red pen. Although it is true that much learning occurs through considering mistakes, the problem with the above system was that the students did not always have any real incentive to look back at their mistakes except to see their overall mark.
As the story moves on, students began to use computers to type their work, print it and hand it in to their teacher who would then mark the work, again with a red pen. The main gain here is to the eyesight of the teacher, who now marks print rather than handwriting, which is not always easy to decipher.
As word-processing software has developed, new tools are now available that make it possible for students to type their assignments and submit them to their teacher electronically (via the Web, email or over a local network). The teacher can use tools present in modern word-processing packages to mark the work electronically - no need to print. In the example below, the teacher has marked a composition in Spanish. The red marks are corrections:
Figure 4: Microsoft Word screenshot, showing corrections inserted by teacher
The yellow highlighted areas in the above screenshot are comments that pop up as post-it notes when the mouse is passed over them. In the screenshot below, you can see the mouse pointer over the work muy and the post-it note from the teacher:
Figure 5: Microsoft Word screenshot, showing pop-up comments
The teacher can then return the work electronically to the student. The process does not end there, for there is now an incentive for the student to consider the teacher's comments in that the work can be readily corrected leading to a final error-free product.
Figure 6: Microsoft Word screenshot, showing student's corrected work
Marking using the revision facilities in Word
You have seen above a piece of written work in Spanish, with the teacher's corrections and comments already entered. Have a look at the corrections and the comments - note that positioning the mouse pointer over each comment enables the student to read the comment. Now choose one of the following pieces of work by students of a range of languages to mark electronically or, if you have samples from your own students, use one of them. Instructions are given below on how to mark electronically using Microsoft Word 2000.
When you have marked the work, it can be sent back to the student by email or across the school's network. The student can redraft the text incorporating your suggestions.
As an alternative to using word-processing to mark students' work, there is specialist software that offers additional facilities, such as Markin: http://www.cict.co.uk/software/markin/index.htm. The website for this package describes it as follows:
"Markin is a Windows program which runs on the teacher's computer. It can import a student's text for marking by pasting from the clipboard, or directly from an RTF or text file. Once the text has been imported, Markin provides all the tools a teacher needs to mark and annotate the text. When marking is complete, the teacher can export the marked text as an RTF file for loading into a word-processor, or as a Web page so that students can view the marked text in a Web browser. Marked work can even be emailed directly back to the student, all from within the Markin program."
Figure 7: Screenshot from Markin
This software might be easier to use for those not experienced with word-processing and email since it is a ready-made solution.
Imagine that you are a student whose
work has been marked electronically, you receive the work back from your teacher
and go through the following process. You can use any of the files that you
corrected in the previous exercise (see above):
How did this seem from a student perspective? Does it engage the student with the corrections more than conventional paper and pen marking?
Marking using word-processing may seem slow at first and if you can't type quickly, it may take longer than conventional marking. One way to overcome this, especially if you frequently make the same comments, is to use the autotext in Word or its equivalent in other packages. This is a feature which allows you to set up coded comment, e.g. sp to stand for spelling mistake. Of course, you may have trained your pupils to recognise such codes but using autotext allows you to type in SP and have the computer convert it to spelling mistake. Here's how to do it:
See Bishop (2004). Graham Bishop covers the use of the following tools that are available in Microsoft Word:
What is ethically acceptable?
Automatic generation of student reports by computer carries with it obvious dangers in regard to quality assurance and security. For that reason, there is not much actual implementation of such schemes. However, computers can be used to support report writing, to provide templates, a bank of comments and automated comments without surrendering the teacher's normal responsibility for the task.
What will be possible within an overall school policy?
Where schools have adopted computerised reporting systems, it may not be possible to use standard features of word-processing as described below. Inevitably, any system allows only specified features.
What does the technology offer?
There are a number of facilities built into standard word-processing packages that can be exploited for report writing. These include:
You will find a cateogrised Comment Bank for Report Writing here which was contributed to by student teachers: comment.htm
You can experiment with using this to write imaginary reports. Just copy and paste the comment you want into a word-processed document. You can make up your own comment bank using similar categories and adapting the comments for your own students. You can use autotext to automate the process of typing in the most commonly used comments.
Recording progress electronically can help make the process more efficient and can make the information collected more versatile. The downside of this is security: Is the information stored as securely, at least, as it would be in a conventional markbook and is it protected from loss, erasure, etc? There is also the question of being able to use the information meaningfully. Some school-wide systems demand a lot of energy for the inputting of information that cannot readily be extracted by teachers. Sophisticated, custom-designed systems may be less helpful to use than a simple spreadsheet, for which the following examples apply.
Using a spreadsheet as an electronic mark book enables you to:
add up marks and do percentages, averages, etc
Figure 8: Screenshot from an electronic mark book
The file for this was created in Microsoft Excel and can be fully viewed by downloading the XLS file below. The file is also available in CSV format for loading into other spreadsheets. Note in the screen shot the bottom row of the window showing different pages of the markbook for different skills, attendance, etc. This facility is not accessible via the CSV file.
Sample markbook files:
Open the file and have a look at how it works. Try some of the following:
Make a list of the advantages and disadvantages to you in your current teaching situation of an electronic markbook compared to a paper-based one. Which are you going to use in future? Note that even if you don't use a spreadsheet for a markbook, it still provides a very useful format for generating a paper based loose-leaf markbook.
Keeping individual pupil records is a facility available in various computer assisted language learning programs. Multimedia CD-ROM's often have this facility and it is usually possible then for the teacher to interrogate the records to check on pupil progress. Some software incorporates an email facility through which student records are automatically forwarded to the teacher via email at the end of a session. This is available as an option with registered copies of Hot Potatoes. These individual records are important to track what students have been doing in CALL sessions since there is often no other record available.
Increasingly, modern languages departments have their own website or section at their school or college site. To what extent can a website be used to manage the assessment of language students? For example:
Take a Hot Potatoes test on Computer Aided Assessment by clicking here: CAATest.htm. The answers can all be found in Module 4.1!
There is an increasing number of websites that offer distance learning materials, including whole courses delivered via the Web and email, using so-called Virtual Learning Environments (VLEs) such as WebCT, Blackboard or Moodle:
Blackboard and WebCT announced an agreement to merge in October 2005.
A VLE is a Web-based package designed to help teachers create online courses, together with facilities for teacher-learner communication and peer-to-peer communication. VLEs can be used to deliver learning materials within an institution or within a local education authority. They may even address a wider constituency, and may be used on a worldwide basis. VLEs have certain advantages in terms of ease of delivery and management of learning materials. They may, however, be restrictive in that the underlying pedagogy attempts to address a very wide range of subjects, and thus does not necessarily fit in with established practice in language learning and teaching. Moodle appears to be gaining in popularity among language teachers. Moodle is open source software which, as the Moodle website states: "means you are free to download it, use it, modify it and even distribute it". Such systems are also referred to as a Course Management System (CMS) a Learning Management System (LMS), and a Learning Support System (LSS). See the definitions of Managed Learning Environment (MLE) and Learning Platform in the Glossary.
See the following modules that deal in more detail with the Web, distance learning and VLEs:
It takes many hours of study and practice to acquire a reasonable level of proficiency in a foreign language. This represents a substantial investment for any organisation considering the selection of people for language training. It would therefore be useful to predict with reasonable accuracy to what extent an individual who had never studied a language before would benefit from attending a language course. Businesses often underestimate how long it takes to acquire a reasonable level of proficiency in a foreign language. The Council of Europe has calculated, for example, that it requires 180-200 learning hours to achieve Waystage (A2) level and 375-400 hours to achieve Threshold (B1) level with the Common European Framework of Reference (CEFR) for Languages scheme, Threshold level being the level at which the learner begins to communicate with a certain degree of confidence. See Section 2.2 for more information on the CEFR.
It can take a lot longer to reach an operational level for a particular profession. Airline pilots have to be able to communicate in English, and other staff employed in the airline industries are expected to be be proficient in foreign languages. An international railway operator requires its English-speaking train drivers to follow a 600-hour course (plus homework) in French to get them from scratch to operational level. The training includes a 3-week residential course in France. An important factor to bear in mind here is the safety factor. A driver who misunderstands an instruction in French while he is driving a train at 300 kilometres per hour is simply not acceptable as a member of the profession. Language tests in such professions are therefore rigorous, and it would obviously be useful to predict individuals' aptitude for learning a foreign language before sending them off to follow an expensive course.
Success in learning a foreign language depends on a number of basic aptitudes, for example:
But factors other than aptitude often make a successful linguist, e.g. attitude to learning, a good learning style, interpersonal communication skills, intercultural competence, etc: see Skehan (1989) & Skehan (1998).
There is a useful collection of articles on Modern Language Aptitude Testing (MLAT) in Volume 9, Nos. 1 & 2 (1998) of Applied Language Learning, a journal published by the Defense Language Institute Foreign Language Center, Monterey, California, USA: http://www.dliflc.edu/academics/academic_materials/all/allissues.htm
There is comprehensive information on Modern Language Aptitude Tests and other types of language tests at the website of Second Language Testing Inc (SLTI): http://www.2lti.com
Two different types of computer-based aptitude tests are described at the website of the Centre for Applied Language Studies, University of Wales Swansea:
The Language Aptitude Tests CD-ROM
and accompanying manual are published by Express Publishing:
http://www.expresspublishing.co.uk/catalogue_details/details.php?Country=uk&id=1153
http://www.expresspublishing.co.uk/catalogue_details/details.php?Country=uk&id=1155
The LEX tests can be downloaded from: http://www.swan.ac.uk/cals/calsres/free/free.htm
See also:
Language Aptitude Test for Entrance in Classics, Oxford University, 1995:
http://www.rhul.ac.uk/classics/CUCD/test.html
Most teachers have been confronted at some time with a piece of work that contains elements that have been copied from another source without due acknowledgement. Arguably, plagiarists are sophisticated cheats, but at secondary school language learning level they are more commonly naive incompetents who think that copying a large chunk of text from a website or CD-ROM is somehow going to get them a good mark. New technologies have made plagiarism easier, but they have also made its detection easier. Responding to plagiarism requires a two-tier strategy: how to catch the cheat and how to train learners to use source material appropriately. Academic plagiarism is the subject of a publication by Decoo & Colpaert (2002).
Detecting plagiarism
The advent of the information age and the course requirement of GCSE and A-level call for new skills for language learners. It is important for them to be able to evaluate information, select appropriately and acknowledge sources that they have quoted. Providing examples and practice in this can be a proactive way of preventing naive plagiarism. However, there is a dearth of good advice for school students reflecting the relative novelty of the problem. Teachers will probably be able to detect readily any occurrence of plagiarism in foreign language work because they know their students' level of language, but it is also useful for them to know how to use a search engine to detect copying from websites and to find the precise site from which the copying has occurred. An easy strategy is to type in a section of the suspected text into the search line of a search engine such as Google, enclosing the text in speech marks. Five to ten words should be sufficient. With luck, the search engine will find the text on the website it was copied from.
Software packages that detect plagiarism already exist:
Joint Information Systems Committee (JISC): There are particular problems with detection of plagiarism by university students, and even by academic staff, which have created the need for detection software. The university funding councils in the UK to have set up a project looking at the whole issue. The JISC website is worth a visit for anyone wishing to find out more about this issue: see the JISC Plagiarism Advisory Service at http://www.jiscpas.ac.uk
See also the website of JISC's plagiarism detection service TurnitinUK, which is hosted at http://www.submit.ac.uk. This enables institutions and staff to carry out electronic comparison of students' work against electronic sources, including electronic sources containing other students' work. The service is based in the UK and accessed via standard Web browsers. However, detection is only one approach. As the above site points out:
Detection should be used as part of a wider approach to prevention. With this in mind, the JISC are also supporting a plagiarism advisory service based at Northumbria University.
See also the section on plagiarism at Bernard Moro's Web Literacy website: http://www.ecml.at/projects/voll/literacy - follow the Web traps route. See Section 3, Module 1.5 for further information regarding the Web Literacy site.
When downloading materials or copying from another website, it is most important that you pay attention to copyright. Above all, don't assume that just because material is publicly available on the Web you can do whatever you like with it.
Copyright infringement is a growing problem, which we refer to in:
See our General guidelines on copyright, which is a general introduction to copyright, drawing on a variety of sources.
See also:
Bangs P. (2003) "Engaging the learner - how to author for best feedback". In Felix U. (ed.) Language learning online: towards best practice, Lisse: Swets & Zeitlinger.
Bishop G. (2004) "First steps towards electronic marking of language assignments", Language Learning Journal (ALL) 29: 42-46.
Brown J. (1997) "Computers in language testing: present research and some future predictions", Language Learning and Technology 1, 1: 44-59. Available at: http://llt.msu.edu/vol1num1/brown/
Chalhoub-Deville M. (ed.) (1999) Issues in computer adaptive testing of reading proficiency, Cambridge: Cambridge University Press.
Chalhoub-Deville M. (2001) "Language testing and technology: past and future", Language Learning & Technology 5, 2: 95-98. Available at: http://llt.msu.edu/vol5num2/deville/default.html
Council of Europe (2001) Common European Framework of Reference for Languages, Cambridge: Cambridge University Press. ISBN: Hardback 0521803136, Paperback: 0521005310. See also the Common European Framework of Reference for Languages (CEFR) Web pages.
Decoo W. & Colpaert J. (2002) Crisis on campus: confronting academic misconduct, MIT Press / Bradford Books.
Department for Education and Skills - now known as the Department for Children, Schools and Families (2002): Languages for all: languages for life - a strategy for England: http://www.dcsf.gov.uk/languagesstrategy/
Dunkel P. (1999) "Considerations in developing or using second / foreign language proficiency computer-adaptive tests", Language Learning & Technology 2, 2: 77-93. Available at at: http://llt.msu.edu/vol2num2/article4/
Godwin-Jones R. (2001) "Language testing tools and technologies", Language Learning & Technology 5, 2: 8-12. Available at: http://llt.msu.edu/vol5num2/emerging/default.html
Heift T. & Schulze M. (eds.) (2003) Error diagnosis and error correction in CALL, Special Issue, CALICO Journal 20, 3.
Krashen S. (1985) The input hypothesis: issues and implications, London: Longman.
Language Learning and Technology 5, 2 (2001) Special Issue, Computer Assisted Language Testing. Available at: http://llt.msu.edu/vol5num3/default.html
Pachler N. & Byrom K. (1999) Assessment of and through ICT . In Leask M. & Pachler N. (eds.) Learning to teach using ICT in the secondary school, London: Routledge .
Rendall H. (2001) Developing a sense of gender in French: ICT integration at initial learner level. In Atkinson T (ed.) Reflections on ICT, London: CILT.
Roever C. (2001) "Web-based language testing", Language Learning and Technology 5, 2: 84-94. Available at: http://llt.msu.edu/vol5num2/roever/default.html
Skehan P. (1989) Individual differences in second-language learning, London: Edward Arnold.
Skehan P. (1998) A cognitive approach to language learning, Oxford: OUP.
Winke P. & MacGregor D. (2001) Review of Hot Potatoes, Language Learning and Technology 5, 2: 28-33. Available at: http://llt.msu.edu/vol5num2/review3/default.html
CAA Centre website: A site that has been designed to provide general information and guidance on the use of Computer Aided Assessment (CAA) in higher education: http://www.caacentre.ac.uk
Common European Framework of Reference (CEFR) for Languages: See the following Web pages for detailed information: http://www.coe.int/t/dg4/linguistic/ http://www.coe.int/t/dg4/linguistic/CADRE_EN.asp
DIALANG: A major European Commission funded project aimed at providing effective diagnosis of language competence in 14 EU languages. It uses online tests, including placement and self-assessment tests, as key tools in this process: http://www.dialang.org
Graham Davies's Favourite Websites: For a selection of sites relating to language learning and teaching, see Graham's page at: http://www.camsoftpartners.co.uk/websites.htm. This page contain links to many websites that offer interactive exercises and tests. See the entry under Language Testing on Graham's page.
Language Testing: A very interesting set of Web pages maintained by Glenn Fulcher at the University of Leicester. Extensive information about all aspects of language testing: http://www.le.ac.uk/education/testing/ltr.html
If you wish to send us feedback
on any aspect of the ICT4LT website, use our online Feedback
Form or visit the ICT4LT blog at:
http://ictforlanguageteachers.blogspot.com
Document last updated 21 April 2008. This page is maintained by Graham Davies.
Please cite this
Web page as:
Atkinson T. & Davies G. (2007) Computer Aided Assessment (CAA) and language
learning. Module 4.1 in Davies G. (ed.) Information and Communications Technology
for Language Teachers (ICT4LT), Slough, Thames Valley University [Online].
Available from: http://www.ict4lt.org/en/en_mod4-1.htm
[Accessed DD Month YYYY].
© ICT4LT Project 2008 under a Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 UK: England & Wales Licence.
