英语论文网

Chapter 3 Research Methodology........... 24-28
    3.1 Participants ...........24-25
    3.2 Instruments ...........25
        3.2.1 ESL Writing........... 25
        3.2.2 The Rating Scale........... 25
        3.2.3 Computer Equipment ...........25
    3.3 Rating Procedure........... 25-26
    3.4 Data Analysis ...........26-28
        3.4.1 Quantitative Data Analysis........... 26-27
        3.4.2 Qualitative Data Analysis........... 27-28
Chapter 4 Data Analysis ...........28-41
    4.1 Rater and Examinee Measurement Analysis........... 28-36
        4.1.1 General Analysis........... 28-29
        4.1.2 Rater and Examinee Measurement Analysis........... 29-36
    4.2 Bias Analysis for Rater-Examinee...........36-40
    4.3 Summary ...........40-41
Chapter 5 Discussion ...........41-46
    5.1 Basic Differences on Raters ...........41-42
    5.2 Severity ...........42-43
    5.3 Rater Consistency........... 43-44
    5.4 Rater Bias........... 44-45
5.5 Summary........... 45-46

Conclusion

The present study focuses on comparing rater performance between the NNEraters and the NE raters based on Multi-facet Rasch model in an attempt to elaboratethe differences of rater severity, intra-rater consistency, inter-rater consistency, andraters’ bias towards different levels of examinee ability. It is necessary to investigatethe fairness and construct validity of ratings awarded by different groups of ratersusing holistic scale in evaluating ESL essays, because holistic ratings are morepreferable and considered as the final reports of student abilities.FACETS software makes it possible exploring the differences of rater severity,intra-rater consistency, inter-rater consistency, and raters’ bias towards different levelsof examinee ability between the two groups of raters. What’s more, the bias analysisis first investigated between the two groups of raters. In sum, the results and findingsof the present study are as follows. Firstly, the study discovers the noticeabledifferences among all raters in the scoring of the 447 essays, which indicates they cannot be interchangeable. However, they except rater 6 maintain high self-consistency.Secondly, it elaborates that the NE raters are more severe than the NNE raters inholistic ratings. Thirdly, it is found that self-consistency and inter-rater reliabilityamong the NNE raters are both greater than that among the NE raters. At last, the twogroups of raters exhibit different bias interactions towards examinees with differentlevels of ability estimate. The two groups of raters have no difference of biasinteractions towards those examinees with extreme high ability higher than 2.00 logitsand extreme low ability less than -2.00 logits, but towards those examinees withability logits within the range of 1.00 to 1.99, the NE raters tend to be more 本论文由英语论文网提供整理，提供论文代写，英语论文代写，代写论文，代写英语论文，代写留学生论文，代写英文论文，留学生论文代写相关核心关键词搜索。