Observation
I found taking part in the activity (the Pictionary style game) and presenting the resource (the glossary) all-consuming. I was too distracted to make observations and field notes that could make any claim to being thorough or objective. Feedback received from colleagues was very helpful, but also generalising and interpreting (Tjora, 2006), such as the observation by a colleague that “the students were having fun”. The feedback, together with informal chats with students and my general impression, gave a sense of how the resources where received. But this methodology was not “systematic and rigorous” enough to “produce validated evidence” (McNiff, 1995) to support any claim of having improved practice.
I considered testing the students’ knowledge of vocabulary at the beginning and the end of the term. However, there wasn’t enough time to prepare such a test and get ethical approval from my tutors before the arrival of the students. I was also worried that a test, especially before you have taught the students the content, could be stressful and perceived as judgmental. I could have made the test anonymous, but then it would be difficult to identify participants and analyse their development. In addition, such as test would not ascertain if knowledge was gained from the new resources or by some other means.
Designing the questionnaire
Instead, I created a short questionnaire to investigate which resources the students had found helpful. Convers and Presser (2011) provided guidance:
Having “a clear set of research purposes”(Ibid):
My research question provided direction. I focused on finding out what would make it easier for the Bespoke Tailoring students to learn specialist vocabulary.
Pretesting:
I checked the questions with my tutor and peers to avoid confusing or leading language, and to make sure they were easy to answer. In my first draft, I phrased several questions in a leading manner and provided answers to choose from, unintentionally steering the responses.
Avoiding repetition:
While designing the questions, I balanced format variation with “the ‘start-up’ cost of explaining a new question format”(Ibid.). While open questions can be demanding, “[c]losed survey questions inevitably simplify and stylise the life and thought of individuals”(Ibid.). I provided ‘other’ and ‘maybe’ as answers for the closed questions, and followed with several open-ended questions and provided space for comments.
The risk of misremembering:
While “retrospective measures are sometimes essential”, as in the case of the question about the Pictionary style game played several weeks earlier, they should be asked “without illusion that we [will get] exact measures of the past”(Ibid.).
Placing ‘background’ questions at the end:
I reduced the background questions to one: Is your first language English? I don’t want to imply that certain identities might have more limited vocabularies, and I think that making students tick boxes about social background, disability, nationality or ethnicity after a questionnaire about learning vocabulary would do exactly that.
Presentation
Before asking the students to fill in the consent form and answer the questionnaire, I presented the project to them and made sure they knew which resources the questionnaire referred to. In addition to asking for feedback about the image based glossary and Pictionary style game, I also wanted to investigate their views on other means of learning specialist vocabulary. I prepared a short Powerpoint presentation for this purpose. When presenting the project and asking the students to answer the questionnaire, it was important to me that the exercise be as fast as possible (see appendix 1). I gave the students the option to choose if they preferred to answer the questionnaire on paper or online (see appendix 2).
I was surprised to find myself uncomfortable asking the students for help with this project, and slightly embarrassed. I ask them to do things all the time in class, but never for a personal project. Though this project aims to improve their learning experience, I had to make it very clear it was voluntary and not part of their assessment. It was humbling and changed the power balance for a moment. It was also ultimately rewarding, because many of the students engaged in the activity.
References:
Converse, J. M. & Presser, S. (2011) The Tools at Hand. Survey Questions, pp 48-75. Thousand Oaks: SAGE Publications
Tjora, A. H. (2006) Writing small discoveries: an exploration of fresh observers’ observations. Qualitative Research. Vol. 6(4), pp 429-451. Thousand Oaks: SAGE Publications
McNiff, J. (1995) Action research for professional development. Concise advice for new action researchers.
Appendix 1: Keeping it brief
Since participation was voluntary and not part of the students assessment, I did not want the presentation to interfere with their tailoring sessions or free time. A short introduction during class, however, would be beneficial to the students, reminding them of the technical resources available, and introducing them to the basics of a small research project. I stayed for an hour after the end of the session to make up for any missed teaching time, and to make sure no one left feeling that they did not get all their tailoring questions answered that day.
I was also hoping that keeping it short and presenting it during class would improve my chances of getting participants. Just as with the Pictionary style game, it was difficult to find the time during my sessions to present the project, despite having prepared a presentation that would take less than 10 minutes. Ultimately I did it towards the end of a busy day. The students were polite, but tired after absorbing a lot of information. It was helpful that I had made both presentation and questionnaire short and to the point.
Finally, by keeping the presentation as brief as possible I hoped to avoid giving away my own assumptions and prejudices. In hindsight, I suspect the briefness of the introduction to the new resources might also have effected the answers. I discuss this further in blogpost 6.
Appendix 2: Paper vs digital
I gave the students the option to choose if they preferred to answer the questionnaire on paper or online. A questionnaire on paper gives you more freedom to give a nuanced answer. For example, you can tick between two boxes and scribble an explanation in a margin. Such answers are more difficult to analyse, but “simplify[…] the thought of individuals” (Converse and Presser, 2011) less. On the other hand, some students prefer typing to writing by hand. Also, translation tools are easier to use digitally.
A couple of students asked me to email the presentation to them so they could fill in the questionnaire later, which I did.