Scientific thinking and communication skills are vital for 21st century STEM careers. From research we know the most effective way for students to develop these skills is through multiple rounds of writing drafts, getting actionable feedback and coaching, then making further refinements until a final product emerges. Ideally, students would be writing and revising routinely, but this is too time- and labor-intensive to be practical in high-enrollment STEM courses. Also, college teachers may not know how best to make high-impact writing instruction part of their classes.
We have a multi-year classroom research project that is funded by the National Science Foundation. We are testing a research-based curriculum that develops scientific reasoning skills through writing. Our approach combines five elements:
- Evidence-based teacher training
- Active learning through text annotation
- Automated student feedback and support
- Bins-based grading
- Emphasis on “reflective coaching” over copy editing
This approach has important instructional benefits beyond just direct effects on student learning:
- Elements 1,4,5 provide actionable data on instructor performance that can guide instructors working in teaching teams, or teaching supervisors working with several assistants.
- Elements 2-5 provide actionable data on students’ knowledge, skills that can be used to improve or revise instructional strategies in real time.
We work with multi-disciplinary group of college English writing instructors, biology teachers, data scientists, and computational linguists who want to understand how undergraduates in learn to write in biology, and help them develop good writing skills sooner. Our first goal is to identify specific patterns of errors that students make as novice scientific writers, then understand WHY students make them. Our second goal is to use those insights to find better ways to train and support student writers in biology and other STEM fields. We also are evaluating how fast graduate teaching assistants master these training methods. Finally, we are looking directly at technology use: how much automated feedback do students use, and how does that translate into progress towards writing mastery?
We believe can help students develop writing skills faster through this approach.
SAWHET Provides Feedback Support
SAWHET (STEM Automated Writing Help Tool) is the technology platform that we developed to collect and analyze student lab reports. Students enter their written assignments into our online web form powered by Qualtrics®. Each section of the paper has a unique entry space (Title, Abstract, Introduction, Material and Methods, Results, Discussion, Figures and Tables, Legends and Literature Cited). While writing, the web form makes sure word requirements are met and a complete report is submitted. If students need assistance, info about how to write each section is provided on the form so students have help at their fingertips.
Once a student report is submitted SAWHET automatically analyzes it using natural language processing to check for the presence of specific requirements, such as presence of a hypothesis and whether statements are backed up by citations. SAWHET also performs an automatic readability test for each section of the report, counts the percentage of scientific terminology and checks for potential plagiarism based on similar citations of reports already stored in the database. The student’s work together with the analysis report is send to the instructor assessing the student’s work. Analytics of each report is stored in the database together with a copy of the student’s original work.
With NSF funding, we are expanding SAWHET’s abilities, and evaluating whether pre-grading feedback to students (with an opportunity to revise and resubmit their work without penalty) can increase the quality of students’ work.