Harken back to your stat courses. As you’ll remember, p is the symbol for significance. When you calculate a p score that is less than or equal to .05, that is a really good result. That means that there is a 95% change that the changes in test performance were a result of your intervention. 5% chance that something else was at play.
When you get significance at a .01 level or less, that’s really, really good.
That means that there is a 99% chance that the results were a result of what you did. 1% chance that something else was going on.
But when you achieve significance at a .001 level, that is AMAZING.
That means there is a 99.9% chance that you were the causal agent of change. This is the closest a statistician will come to telling you that you are it!!
- For Kindergarten, there were significant greater improvements on all three Test of Handwriting Skill measure by the experimental group over the control group.
- And in chart form, it looked like this. Now remember, we were comparing the Change Scores for writing the upper-case alphabet. Airplane was writing by memory; Butterfly was a dictated alphabet; Tree was copying 12 printed letters. Memory, dictation, copying.
- In first and second grades, there were significantly greater pretest to posttest improvements on all THS and Minnesota scores for the experimental group versus the control group… Excepting… first grade Minnesota spacing. Regarding this quality-spacing, both groups had identical change scores over time.
- But the even bigger difference for both first and second graders was the SIGNIFICANTLY LOWER Minnesota scores for Rate! At post-test time, both the first and second graders in the experimental group were SIGNIFICANTLY SLOWER than the control group.
Now, while this is certainly a movement in the opposite direction and one that could be cautionary, I am still obligated to report it to you. However… both site managers reported to me that the children in the experimental groups were SOOOO careful when it came time for the post-testing that they were measurably slower.
I do feel that this difference would even out in time, but at the 8-week mark, printing was still at a very conscious level.
- And this is what their Minnesota post-test scores looked like. Note that Rate has a negative t-value. The results favored the Control group and were really strong. Legibility and Form both had strong results. Alignment and Size had Amazing Results. Spacing was a Push.
- The THS for first graders showed amazing results for Bus, Frog, Truck, and Lion - all tests that required printing lower case letters from memory or dictation. But when it came to copying… their scores, while good, were not as strong.
- In the Minnesota for second grade, again note the negative t value for Rate. Just the same, the difference was not significant. Not so for legibility, form, alignment, size, or spacing. All of these change scores were HUGE.
- And look at the Test of Handwriting Skills. Every test. From memory, Dictation, whether copying individual letters, words, or sentences. Upper or lower case. Every subtest… UNBELIEVABLE.
The feedback from the teachers was overwhelming and undeniable
- The kids grasped the concepts, especially the Star-Worthiness,
- Improvements were immediate
- The Dice Game was very motivating
- The reference points were easy
- The materials were fun
- Carryover happened throughout the day
- Kids wrote slower and took pride in their work
- The teachers were looking forward to using it the next year… which in fact they did. The control teachers were particularly glad when the study was over because they were promised the same materials and couldn’t wait to get started.
Thrilling results.
And because I’m sure you’re wondering… Just what was the control group doing…? They were doing what many of your teachers are also doing. They were engaging in an Incidental Approach toward handwriting instruction. In other words, they were not NOT teaching, but neither were there singular methods, philosophies, materials, or verbal cues. Among the own personal styles…
- Teachers demonstrated letter formations
- Had a variety of visual cues, including writing posters, around the room
- Many had alphabet strips on the students desktops or above the front board
- Children were encouraged to print neatly. This still was a value.
- But there was no uniform or formal instruction
Sound familiar? Compelling evidence not only that the Size Matters Handwriting Program works, but also that kids DO NOT acquire handwriting competence through osmosis.
Of course, one research study does not a body of evidence make. We now have several more in various stages.
- A longitudinal study is in development in New Jersey. We’ll be looking at the impact of handwriting on academic performance, the number of referrals to OT, and other cognitive measures. This study is planned as a multi-year investigation. We will be starting with kindergarten only in several schools, but then looking at these same students the next year in first grade, then second, third, and so on. Certainly, we’ll have enough information to look at what happens to rate in the long haul.
- We have several comparative studies in various stages. One is already underway in Seattle. We’re looking to measure the differences between SMHP and other instructional approaches. It will be interesting to note legibility scores as well as speed of acquisition.
- In the talking stages are studies examining the use of the Size Matters Handwriting Program in addressing specific special needs populations.
And this is just the beginning. If research interests you, please contact me. It takes years to get a study off the ground and even more years to get it into the journals. Right now, I’m collecting names of therapists who would like to participate as site managers as well as those would like to be principal investigators. It helps to have a university behind you, and I’m lucky to have a few. If you are in a doctoral or other research program, access to your libraries would provide the needed Literature Reviews.
Let’s keep talking, but let’s move on. And let’s learn why the Size Matters handwriting Program could have achieved this level of success.
There are several theories. Here is an actual dialogue I repeat many times with my students. Pink is me. Blue are the kids.
- In other words, this is a CHILD-CENTERED approach. When you are child-centered, you build child-empowerment. As you all know from the OT practice framework, including our patients, clients, or students in their treatment plans helps them become invested in the goals. This is the root basis for FOLLOW THROUGH AND CARRY OVER.
- Polatajko says that (using cognitive-based approaches, (in other words, including students in the decision-making and goal setting, children improved their performance on their three self-identified goals, maintained their skills after intervention, and reported satisfaction on all their goals.
- In short, when kids know why they’re doing what they’re doing, there is a buy in.