Abstract

This article describes the development, implementation, and evaluation of fill in multiple blanks questions (FIMBQs) in an online auto-graded homework assessments. Different forms of questions that are either numerical or words or both were developed. Results analysis provided feedbacks for formative assessment of teaching and learning.

1. Introduction

A student’s success in Chemistry course is strongly correlated to the on-task studying of material through solving homework problems ^{1, 2}. Online homework has the advantages over handwritten, due to increased efficiency in distributing, collecting, grading ³, and timeliness of feedback ⁴.

Computer based testing programs adapted items rarely beyond multiple-choice questions ⁵, even though there are indications that this form of assessment predominantly tests only lower levels of cognitive skills ⁶. Fill in Multiple blanks questions (FIMBQs), have been found to be comparable tool to MCQ, which takes off the guessing element and provide more reliable student’s competency test ⁷.

FIMBQs can be found among other type questions in many different online homework software packages ⁸ and on supplemental websites provided by Textbook publishers ⁴. Blackboard is a leading example of a virtual learning environment, which many universities have chosen as their e-learning platform ⁹.

This paper focused is on how to create and implement auto-graded online FIMBQs on a Blackboard, without using any additional software package and the kind of feedbacks that can be a useful tool for formative assessments.

Homework in General Chemistry, Jubail Industrial College, constitute 5% of the total grade on the course, for the last 3 years, we have been using blackboard successfully to deploy our homework. Initially, we deployed 25-30, randomized different types to the students. Lately, we adopted the use of questions sets from a database containing pools of questions to minimized cheating. In the last homework each student received 13 MCQs, 3 true or false questions, 6 matching questions, 7 FIMBQs delivered in 8 questions sets. The evaluations were collected by running the item analysis at the end of the assessments, the results as indicated in Table 1, below showing the percentage of students the got the various questions correct in some of the FIMBQs. The values in the table are subjective and may vary for different years and semester, but on the overall, they provided useful feedback for both teachers and students. Questions without evaluation are taken from online tutorials (2.3, 2.4 and 2.8).

2. Fill in Multiple Blanks Questions (FIMBQs)

For FIMBQs, feature blank spaces, to be filled with appropriate words, phrases or numbers. Student’s answers are graded automatically, by matching with answers already provided by the instructor ¹⁰. FIMBQ were successfully applied to short answers questions, naming of compounds, writing chemical formulas, balancing equations, stoichiometry calculations, numerical calculation questions etc.

The naming of inorganic compounds were assessed using FIMBQs. The grading rubrics has about four correct answers for the names of HgS, this is because of the differences between the new and old method of naming Hg²⁺ as mercury (II) ion or mercuric ion and also the two names of S^2- sulphide ion (British system) and sulfide ions (American system). It is important to select for the evaluation method ‘contains’ rather than ‘exact match’ or ‘pattern match’ to accommodate the wide range of answers, which may result from spaces between the oxidation number and names. The variations in the grading rubrics are less in naming main group metals (as the oxidation number of the metal is not required in the name) and also in the naming of molecular compounds.

The item analysis shows that the question has good discrimination index (0.62 and 0.78, for sections A and B respectively). The comparatively low percentages of pass for HgS, was because some of the students ignored the oxidation of Hg.

In writing the chemical formulas from the names, ionic compounds should be represented by their correct chemical formulas, which are written on the basis of valences ¹¹. For chemical formulas involving polyatomic ions such as nitrate, and phosphate, parentheses followed by appropriate subscripts are used ¹². Blackboard has a limitation of the inability of the examinees to write the number of atoms in the subscript forms.

When using short answer questions to test student knowledge of definitions, consider having a mix of questions, some that supply the term and require the students to provide the definition. These type of questions can be structured as FIMBQs. This mix format will better test student knowledge because it doesn’t rely solely on recognition or recall of the term ¹³.

The feedback from the item analysis shows that most students in both sections did very well and the discrimination factors are 0.32 and 0.79 for section A and B respectively.

In FIMBQs it is very important to keep the answers simple and limited to as few words as possible to avoid mismatched answer due to extra spaces or order of answer. In Figure 3, questions i and ii are ideal types of question that has simple answers compare to question iii that have a long answer with numbers, words and dashes.

Drawing structural formula is only possible by using external applet such as MarvinSketch or Chemwindow ^{3, 14}. Condensed structural formulas can be used and answers such as CH₃CH=CH₂, CH₂CHCH₃, CH₃-CH=CH₂ can be part of the rubric for the answers. Ph- used as substituted benzene.

Calculation questions such as calculation of molar masses, moles, temperature conversions, concentrations etc. can be also be asked and graded automatically using FIMBQs. The advantage of FIMBQs over calculated numeric option on a Blackboard is the ability to ask the students to write both numeric answer and alphabetical unit Figure 4.

Multi-steps calculation such as determination of limiting reaction and calculation of mass of the products formed was deployed using FIMBQ. Partial credit was awarded even if the final answer is not correct.

The feedback from these type of questions enable the instructor to know the percentage of students that understood the first step and have problems in other steps, 55 % and 44% of section A and B respectively students that got the first step, fail the second step. (Table 1, question 3).

The answers of the some questions are either very big or very small as the result they needed to be represented with scientific notations, such as 1.03 x (10^24) or 1.03e+24 or any other forms. Such technical difficulties can be eliminated by using two multiple blanks, one for digits and others for the exponential value as in the Figure 6.

The relatively lower percentage of pass for this question compare to other questions, was found to be some student did not pay attention to the order of the question (items 1, 2) in Arabic enable Window is display as 2, 1 from left to right (Table 1, question 4).

For all the calculation problems, it is important to select for the evaluation method ‘contains’ rather than ‘exact match’ or ‘pattern match’ to accommodate a wide range of answers.

Balancing chemical equations, completing products, converting words equations to chemical equations and classification of chemical were successfully assessed using FIMBQs ¹⁵. However, partial credit to the answer may be checked to encourage students, even though chemical equations needed to be completely balanced as there is nothing as a partially balanced equation.

Balance the following equation and indicate the type of reactions. Do not leave any blank empty, if the number of moles are 1, you must write 1 in the blank.

The item analysis indicated that all the students did very well and slightly lower percentages for the classification reaction, compared to balancing equation, may not be unrelated to the requirement of recall knowledge (Table 1, question 5).

FIMBQs was used to mimic the paper homework by implementing questions in the tabular form. Numerical calculations and short words or phrases could be applicable to this types of questions.

3. Conclusions

This article shows that FIMBQs can mimic most of the forms of questions that can be admitted on the paper-based homework and it’s also an excellent test tool for examining students understanding, comprehension, knowledge of absorption and retention. The chances for guessing are minimal as predetermined answers sets are not provided. All the questions with items analysis were found to have good discrimination index, which is also an indication for its effectiveness.

Acknowledgments

Facilities provided by Jubail Industrial College of Royal Commission for Jubail and Yanbu are gratefully acknowledged. The author is grateful to Dr. Yunusa Umar for his help during preparation of the manuscript.

References