Measuring Literary Reading Motivation: Questionnaires Design and Pilot Testing

This study aims to present the design and pilot testing procedures of the two specific self-report questionnaires were used to measure the two key aspects of reading motivation, self-efficacy and intrinsic motivation in the field of literary (narrative) reading, and the partial factors that jointly shape them. These instruments were outlined in advance, tested on a small scale and finally administered in a pre-post (quasi)experimental-control group research study, in order to investigate the effect of an intervention reading program to 6 graders. The measurement tools have good validity and reliability evidence, but further construct validity analysis should be done.


Introduction
Student engagement in learning is an important and fully documented predictor of school performance in general, but also in specific domains, including reading (Guthrie & Wigfield, 2000). According to Baker and Wigfield (1999), reading engagement involves and presupposes reading motivation, the text meaning making through the effective use of (meta) cognitive strategies and the interaction of students in reader communities. The reading engagement, therefore, refers to those readers who can coordinate reading strategies, text comprehension, and prior knowledge (cognitive dimension) in a reader community (social dimension) to fulfill personal goals, desires and intentions (motivational dimension) (Guthrie & Wigfield, 2000).
Self-efficacy is one of the most important aspects of developing reading motivation. In particular, reading self-efficacy refers to the personal cognitive judgments and beliefs of each reader about her capabilities to cope with specific reading activities (Walker, 2003;Schunk & Pajares, 2002). These beliefs relate to more content and situational specific judgments of the reader about her potential (e.g. "in the literature test, I believe I will write excellent''). Moreover, they prompt her to realize her overall literary competence, thus influencing her further motivational behavior in relevant fields (Alexander, 2005).
Even though learners believe they can accomplish a task, they will not probably spend time and effort, because they do not perceive the reason or the energy (passion) for completing this activity (Pintrich & Schunk, 2002). This behavior energy depends on the degree of the learner needs satisfaction. Three innate psychological needs determine the level of intrinsic motivation (Deci & Ryan, 1985), while their satisfaction can lead the individual to higher levels of personal development . Intrinsic motivation refers to the innate tendency of a learner to exercise and expand her potential, to explore and search for innovations and challenges . This innate tendency emerges only when her learning engagement becomes an end in itself and simultaneously ensures the satisfaction of "competence, autonomy and relatedness with important others' (Niemiec & Ryan, 2009: 135).
In the field of reading, intrinsic motivation expands on the reader's disposition to read purely for fun, interest and enthusiasm for the reading act (Guthrie & Wigfield, 2000), while co-formulating a strong sense of reading self-efficacy (Wigfield, Eccles, & Pintrich, 1996). Furthermore, the intrinsic motivation for (literary) reading involves both the reader's in vivo participation (pleasure, interest, enthusiasm) and her inherent intention and desire to participate in such an activity, which consists of the only reward (Pintrich & Schunk, 2002). Such readers exhibit self-determined behavior, and choose to read for curiosity, absorption, social interaction and emotional satisfaction (Gambrell, Palmer, Codling & Mazzoni, 1996).
Self-efficacy beliefs, as a key component of reading motivation, influence the reader's decision to engage in a particular task, to overcome any obstacles and perform successfully. These beliefs development affects, among other environmental influences, her universal needs for competence, autonomy and relatedness, and the reasons for their satisfaction.

Research Goal
This study aims to present the design and pilot testing of the two specific self-report questionnaires were used to measure the two key aspects of reading motivation, self-efficacy and intrinsic motivation in the field of literary (narrative) reading, and the partial factors that jointly shape them. Fifty seven sixth grade students participated in an eleven weeks reading intervention, as members of four naturally defined classrooms in two randomly selected primary schools of the city of Kalamata (Greece). In order to investigate the effect of this intervention reading program, repeated measurements were designed in advance, tested on a small scale and finally administered.

A. The Self-Efficacy questionnaire in Literary Reading (Appendix A)
The development of this questionnaire relied on research guidelines concerned with the construction standards of selfefficacy scales (Bandura, 2006;Pajares, 2006); that is, such a tool should not be "an all purpose" measure, but it should reflect specific task-activities of a particular domain (e.g. narrative reading) and be phrased in terms of 'can do' (capability statement) rather than 'will do' (intention statement).
It followed mainly the item and factorial structure of the Reader Self-Perception Scale (RSPS) (Henk & Melnick, 1995) which consists of statements about reading process, significantly, vocabulary recognition, readability and text comprehension. The Self-Efficacy questionnaire in Literary Reading (see Table 1) was based on the self-efficacy theory and consists of a total of 31 items, which are divided into four (4) dimensions (factors) that influence differently the development of self-efficacy (Bandura, 1995;Pajares, 2006;Pintrich & Schunk, 2002). The "Progress" factor items reflect the reader's direct perception of her literary success or failure history during her personal [e.g. "I express my personal opinion (what I saw, I heard, I felt, or I thought), verbally or in writing, about a short story or a novel better now than I could before"] or cooperative processing (e.g. "I share and support more strongly my opinion in the class literature discussion than I used to"), meaning making (e.g. "I understand the messages of a short story or a novel better than I could before") and/or the interpretation-evaluation [e.g. "I am getting better at explaining why someone would read (or not) the short story or the novel I have just read"] of a short story or a novel. It also includes her beliefs of overcoming any difficulties putting much effort (e.g. "When I find difficult to understand something during a short story or a novel reading, I can overcome it more easily now than I could before"). This factor is considered to be the most important for the achievement of high self-efficacy; therefore it was made up of 9 items.
The "Social Feedback" factor items capture the direct [e.g. "My teacher encourages me to freely express my personal opinion (what I saw, I heard, I felt or I thought) about a short story or a novel"] or implicit (e.g. "My teacher likes to see me share and support my opinion when talking about a short story or a novel in the class") comments and opinions of the teacher (5 items), classmates (3 items) (e.g. "My classmates think I do pretty well at reading a short story or a novel") and family (1 item) (e.g. "People in my family like to see me read a short story or a novel"), when the student (reader) processes, meaning makes or interprets-evaluates a short story or a novel as well as overcomes any difficulties.
The "Observation Comparison" factor items are intended to provide the reader's implicit perception of how much better, in comparison to her classmates, she achieves during a personal [e.g. "I express my personal opinion (what I saw, I heard, I felt, or I thought), verbally or in writing, about a short story or a novel better than other kids"] or cooperative processing (e.g. "I share and support more strongly than other kids my opinion in the class literature discussions"), meaning making [e.g. "I need less effort to find the basic elements (series of events, theme, space, time, characters) of a short story or a novel I read than other kids"] and/or interpretation-evaluation [e.g. "I can explain to someone else why to read (or not) the short story or the novel I have just read better than other kids"] of a short story or a novel, overcoming any difficulties (e.g. "When I find difficult to understand something during a short story or a novel reading, I can overcome it more easily than other kids").
Finally, the "Physiological State" factor items represent the physical and emotional responses of the reader during a personal [e.g. "I feel confident, when I am going to express my personal opinion (what I saw, I heard, I felt, or I thought) about a short story or a novel"] or cooperative processing (e.g. "I feel comfortable, when I share and support my opinion while talking about a short story or a novel in my class") meaning making (e.g. "I like to understand a short story or a novel I read in my own way") and/or the interpretation-evaluation (e.g. "I think reading a short story or a novel is relaxing of a short story or novel") of a short story or a novel, overcoming any difficulties (e.g. "I feel calm every time I do not understand something when reading a short story or a novel").
These factors are inevitably related and interact adequately with one another, demonstrating the complex and social origin of the reading process (Alvermann & Guthrie, 1993) and, in particular, literary reading.

B. The Intrinsic Motivation questionnaire in Literary Reading (Appendix B)
The development of this questionnaire was based on the a) Intrinsic Motivation Inventory (IMI) (McAuley, Duncan & Tammen, 1989), b) Motivation for Reading Questionnaire (Wigfield & Guthrie, 1997), c) Flow State Scale (Jackson & Marsh, 1995), (d) Basic Need Satisfaction Work Scale (Deci, Ryan, Gagne, Leone, Usunov & Kornazheva, 2001) and (e) the Need for Relatedness Scale (NRS) (Richer & Vallerand, 1996). It also followed the fundamental principles of the selfdetermination theory (Deci & Ryan, 1985 as well as the flow state theory (Csikszentmihalyi, 1990); within this context, we identified the conceptual construct of intrinsic motivation for that study.
The multifaceted questionnaire (see Table 2) comprises a total of 24 items, divided into four (4) factors. The first factor refers to an empirical manifestation of the literary intrinsic reading motivation, while the others correspond to the three innate needs of the individual as a reader. According to the self-determination theory (Deci & Ryan, 1985, the satisfaction of these needs functions as predictor in the self-regulated reading behavior of the individual reader.  (10) The "Enjoyment/Involvement" factor items outline the reader's subjective 'lived-through' experience of pleasure (e.g. "I like to read a short story or a novel very much"), flow state (e.g. "My mind often wanders, when I read a short story or a novel I like") and involvement (e.g. "I make pictures in my mind, when I read a short story or a novel") when reading a short story or a novel. The statements, illustrating the content of these experiences, were carefully selected by representative variables (factors) of three distinct questionnaires oriented toward literary reading. In particular, the two statements of pleasure experience were adapted from the corresponding factor items (Enjoyment/Interest) of the Intrinsic Motivation Inventory (McAuley et al., 1989), the two statements of flow state experience were modified slightly from the corresponding factor items (Transformation of Time) and (Autotelic Experience) of the flow state scale (Jackson & Marsh, 1995), while the last two statements of involvement experience copied the corresponding factor items (Reading Involvement) of the Motivation for Reading Questionnaire (Wigfield & Guthrie, 1997). The "Enjoyment/Involvement" factor highlights the multifaceted conceptual construct of the reader's intrinsically motivated behavior, which has been little valued by the existing scales.
The "Perceived Competence" factor items capture the reader's general perception of her ability to read a short story or a novel (e.g. "I think I am pretty good at reading a short story or a novel"). This factor is an adaptation of the corresponding factor items (Perceived Competence) of the Intrinsic Motivation Inventory (McAuley et al., 1989). Competence is an inherent need of the individual, the satisfaction of which is related to a more general, affective experience of self-efficacy when accomplishing easily a task. Others believe that a person's overall competence can improve her self-efficacy on specific tasks, but, instead, others point out that the latter is more relevant to the successful task performance than the individual's competence (Bong & Skaalvik, 2003). Despite the initially apparent conceptual identification of 'self-efficacy' with 'competence' and the risk of overlapping, the assessment of the competence, as a universal human personality trait, diverged from that of self-efficacy; this represents rather a body of differentiated self-beliefs, directly linked to the individual functionality.
The "Perceived Autonomy" factor items reflect the reader's sense that she is free to choose [e.g. "I feel like I am free to use multiple ways (e.g. sketching, theatre, group discussion) to process a short story or a novel"], take control of (e.g. "I feel like I can participate in class session to deciding how to read a short story or a novel") and undertake willingly the responsibility of (e.g. "I am free to express and discuss my opinion with others in class about a short story or a novel") her behavior in reading a short story or a novel. This factor emerged from the selective modification of the corresponding factor (Autonomy) of the Basic Need Satisfaction at Work Scale (Deci et al., 2001). The need for autonomy refers to the innate desire of the individual to feel completely free to decide for self-regulated action when performing a task .
Finally, the "Perceived Relatedness" factor items reflect the reader's sense that she experiences acceptance in her relationships with both the classmates (5 items) (e.g. "When we read a short story or a novel in class, I feel my classmates support me") and her teacher (5 items) (e.g. "When we read a short story or a novel in class, my teacher makes me feel safe") when reading a short story or a novel. This factor was modified slightly from the corresponding factor (Acceptance) of the Need for Relatedness Scale (Richer & Vallerand, 1996), including the concepts of understanding, active listening, value, safety and support. Support is actually the most important element for teacherstudent or student-student relationships development. Making a distinction, 'academic support' refers to the help students receive via the learning process; while 'personal support' refers to the experiences of sympathy, friendship and collaboration they feel (Babalis, Galanaki & Stavrou, 2007).
According to the self-determination theory, the simultaneous satisfaction of the three inherent needs for competence, autonomy and relatedness is a prerequisite for the development of the self-regulated reading behavior of students (Deci & Ryan, 1985), even if they come from different age or cultural groups and seek to meet these needs differently. Indeed, the a priori satisfaction of both the reader's competence and autonomy could contribute more to activating and maintaining her participation in the field of literary reading, than the relatedness . Instead, Lapp and Fisher (2009) found that the prior student interaction as well as the reading for pleasure had a more positive impact on the competence of the readers involved.
Most self-efficacy or intrinsic motivation measurement tools in (literary) reading prefer to use the Likert scale. The readers (respondents) are usually asked to rate how much they agree or disagree with each statement using a 5-point Likert system [1=strongly disagree, 2=disagree, 3=neutral or undecided, 4=agree, 5=strongly agree]. This ordinal scale, however, puts the subjects of a research study in a difficult position, because they need time and special effort to understand the subtle semantic differences among different point graduations; namely, the meaning difference among "strongly agree", "almost agree" or "agree". To overcome this problem, students (readers) could record their selfefficacy beliefs -as well as their intrinsically motivated reading behavior -on a 100 (or 10) point scale using 10-unit (or single unit) intervals from 0-100 (or 0-10), accordingly (Bandura, 2006); hence, the replacement of these response formats with the "Bar" (Kambaki-Vougioukli & Vougiouklis, 2008), which is based on the 'fuzzy sets' theory. The subjective processing, meaning making and interpretation-evaluation of a literary work by each reader are fuzzy areas per se. The 'bar', a continuous one, offers the possibility of such a fuzzy reading behavior, since it essentially aims for its representation in single unit intervals ranging from 0-1 (actually, from 0-10), instead of an absolute response 0 or 1.

Pilot Study
Prior to the self-report questionnaires administration, a pilot test was accomplished to check their face validity as well as internal coherence-reliability; furthermore, to estimate the completion time required and highlight the degree of readability, thus to make the most recent corrections. Face validity is achieved mainly through the opinion of nonexperts with the study subject; that is, they come from the population sample of interest.
The no probability 'convenience sampling' method was used to select the sample subjects (n) of the pilot study. It is used to choose such "captive audience" (e.g. class or college students) as units in research studies (e.g. a pilot study) that rely on data collection from population members who are conveniently available to participate (Cohen & Manion, 1994:130). In other words, an intact group-class (20 students in grade 6) of the 9 th Primary School of Kalamata was asked to participate willingly. This group had a high degree of constitution similarity and proportion with intervention groups. A teaching hour was granted by the school principal and the class teacher. The two instruments were then administered in random order to 18 students, 11 boys and 7 girls, since that day a boy and a girl were absent. Before completing the questionnaires, a brief introduction was made on the pilot study mission, the conceptual definitions for each questionnaire and their completion guidelines as well as the student anonymity and data confidentiality protection. In case a respondent student had questions, the researcher intervened in private. The process lasted a maximum of 20 minutes per each questionnaire. Students were then asked to comment on the wording and content understanding of the items, but also on the measuring tools presentation (font size, measurement scale, spelling-syntax, completion guidelines clarity).
Based upon feedback received from the pilot study sample, modifications were made to the item pool. For example, in the first questionnaire, question 14 ("I can explain to someone else if it is worthwhile to read the short story or a novel I have just read better than other kids") was rephrased to "I can explain to someone else why to read (or not) the short story or the novel I have just read better than other kids"', while question 13 ("I understand the meanings and ideas of a short story or a novel better than I could before") changed to "I understand the messages of a short story or a novel better than I could before". Similarly, in the second questionnaire, question 7 ("I often feel like time stops or alters, when I read a short story or a novel I like") was modified to "My mind often wanders, when I read a short story or a novel I like", while question 12 ("'I feel that I can choose alternative ways and strategies to read a short story or a novel) changed radically to "I feel like I am free to use multiple ways (e.g. sketching, theatre, group discussion) to process a short story or a novel".
In the preliminary phase of the two instruments development, content validity was additionally checked out, considering both the bibliographic overview of literary self-efficacy-intrinsic motivation concepts and the discussions with experts (see Figure 1). Indeed, such experts can recommend addenda and important points of the underlying theoretical construction, for which the researcher may probably be unaware of. Therefore, on the one hand, a teacherlinguist (with experience in language teaching in primary education) examined whether the statements wording fits for the cognitive developmental stage of middle schoolers. On the other hand, two university professors (experts in literature teaching and reading psychology, accordingly) examined the factors construction, especially if each item in its factor category seems to fit best, and made recommendations. It was felt that some factor items "Enjoyment/Involvement" & "Perceived Competence" of the second questionnaire might subsume the majority of the factor items "Physiological State" of the first questionnaire. Thus, the factors were clearly redefined and, those items that placed in risk of overlapping, were modified or eliminated (e.g. the item "I feel that I can meet the requirements of reading a short story or a novel", included partly in factor 'Enjoyment/Involvement', moved away).
Apart from the revisions made, the internal consistency-reliability across items and factors of each questionnaire was determined via the Cronbach's alpha numerical coefficient (>0.70). When a questionnaire is multifaceted with few items divided into factors, it is recommended to determine the reliability coefficient (α) using SPSS statistics. The «Self-Efficiency questionnaire in Literary Reading» displayed a Cronbach's value of α = 0.937, while the «Intrinsic Motivation Questionnaire in Literary Reading» showed Cronbach α = 0.954, very high values, thus the items of each questionnaire tend to measure the same construct.
Further reliability analyses confirmed the high correlation between the factors of each questionnaire. Moreover, the «Self-Efficacy Questionnaire in Literary Reading» displayed again a very high Cronbach's α = 0.894 (pre) & α = 0.909 (post), and even marginally increasing between the two administration periods. Besides, the «Intrinsic Motivation Questionnaire in Literary Reading» had a very high Cronbach's value α = 0.918 (pre) & α = 0.901 (post), with a minimal decrease between pre and post testing. The internal consistency of the four factors per questionnaire was (see Table 3-4):  The Cronbach's alpha coefficient remained high (> 0.70) in almost all the factors of the two questionnaires, although each consisted of a small number of items (<10). Even the reliability limit values of the factor "Perceived Autonomy" have not been a problem, because, from the one hand, the discarding of an item did not increase the reliability value of this particular factor (column: Cronbach's Alpha if Item Deleted) and, on the other hand, it is confirmed that each item measured the same to the remaining items through the higher correlation value of each item in that factor with the sum values of the remaining items (column: Corrected Item-Total Correlation) (see Table 5). In addition, this factor was indispensable in the conceptual construction and measurement of intrinsic motivation in literary reading. The aforementioned measurement tools design and pilot testing formed part of a broader pre-post (quasi)experimental-control groups research study that got involved in the planning, implementation and evaluation of a literary reading self-efficacy and intrinsic motivation enhancement program for 6 graders. The specific intervention training package consisted of whole fictional, autobiographical scripts (3 short stories & 1 novel), selected from modern Greek literature for children, and followed reader response-based instruction.
The first self-report questionnaire «Self-Efficacy Questionnaire in Literary Reading» is a tool for recording the subjective beliefs and judgments that a middle age-school reader makes before, during and after the reading of a short story or novel, as a domain specific through specific task-activities of narrative text processing, meaning making and interpreting-evaluating. The item composition of this questionnaire was originally grounded in the "Reader Self-Perception Scale" (Henk & Melnick, 1995). However, the need to define a more specific domain of interest (narrative reading) with predetermined reading phases (before, during, after reading) and gradual difficulty activities emerged. In other words, the questionnaire was adapted to measure the readers' beliefs in their capabilities to fulfill different levels of task demands on processing, meaning making and interpreting-judging of a short story or novel, primarily during reading phase and secondly before reading phase (Pajares, 2006;Schunk & Rice, 1993). Bandura (2006) indicated that efficacy beliefs differ in level, generality, and strength, so raising the challenge level or impediments via the items could increase the difficulty as well.
The second self-report questionnaire «Intrinsic Motivation in Literary Reading» is a tool for assessing the selfdetermined behavior variance when a reader deals with a short story or novel. Furthermore, this instrument captures the degree of her innate desire to read a short story or novel for pleasure per se, absorption and emotional satisfaction (Gambrell et al., 1996). Therefore, this questionnaire was designed to distinguish the reader's individual differences when reading a short story or novel, according to the degree of satisfaction of their three psychological needs for competence, autonomy and relatedness (Deci & Ryan, 1985. Indeed, it was decided to administer it immediately after completing the 'Self-Efficacy Questionnaire in Literary Reading' so that respondents have already perceived their capabilities during different literary reading phases or activities, before proceeding to an emotional review of their reading experience. To score both questionnaires, respondents drew a vertical line at that point on the 'bar' format where they felt it determined intuitively their choice at that moment; hence, the responses were distributed over a good part of the range of alternatives, making the measurement tools more sensitive and reliable (Bandura, 2006) as well as stronger predictors of performance (Pajares, Hartley & Valiante, 2001). As regards the data processing, it gave the possibilities of multiple, numerical operations (like ratio scales do) of the questionnaires.

Discussion and Conclusion
There is good statistical evidence for the validity and reliability of the self-efficacy and intrinsic motivation measures in the field of narrative reading, and the partial factors that jointly shape them. These questionnaires were based on the predefined structure of similar self-report tools and reflect the ideas of the theoretical construct of interest. Further work should consider their construct validity, confirming that the selected factor items or the factors per questionnaire are conceptually and statistically interrelated. Ultimately, the quantitative data obtained from the measurement tools should be further combined with qualitative information (e.g. teacher reflective diary, observation, readers task achievement, etc), which would strengthen the validity and effectiveness of the intervention program.
However, it is important that both the self-report questionnaires removed from the usual practice of similar tools to measure reading self-efficacy or intrinsic motivation in general (Guthrie, McRae & Klauda, 2007;Wigfield, Guthrie, Tonks & Perencevich, 2004) and were oriented to situational (narrative) literary reading, while it is likely that different literary genres (short story, poem, argumentation, etc.) prerequire different capabilities or different degree of reader needs satisfaction. Although many researchers have recently begun to engage in the development of such instruments in particular academic domains (e.g. language, maths, science, etc.) (Guthrie et al., 2007), they are little involved in the literary reading field per se, because literature instruction is rather integrated in Language-Arts Curriculum.
These instruments could be used for both whole literature groups and individual reader assessments or interventions, helping teachers to detect their literary reading skills mastery and feelings throughout the school year (at the beginning, at the midpoint, at the end). When the "I should" changes gradually to "I can" and "I love to", the potential for reading engagement behavior becomes a reality; that is, children, who have not read even one book until now, begin to consider literary reading an alternative means of entertainment.

GUIDELINES
The following questionnaire consists of items that refer to the personal judgments a student (reader) makes when reading (= processing, meaning making, interpreting, evaluating) a short story or a novel. This anonymous questionnaire is NOT a test or school assignment. There are NO right or wrong answers. If there is something you do not understand, ask for help from the researcher who administers it. To answer, draw a vertical line at a point on the bar that exists after each item.
The more you disagree with the content of an item the closer to the left end of the bar draw the line. For example:

STRONGLY DISAGREE STRONGLY AGREE
The more you agree with the content of an item the closer to the right end of the bar draw the line.

GUIDELINES
The following questionnaire consists of items that refer to the feelings and experiences a student (reader) learn when reading (= processing, meaning making, interpreting, evaluating) a short story or a novel. This anonymous questionnaire is NOT a test or school assignment. There are NO right or wrong answers. If there is something you do not understand, ask for help from the researcher who administers it. To answer, draw a vertical line at a point on the bar that exists after each item.
The more you disagree with the content of an item the closer to the left end of the bar draw the line. For example:

STRONGLY DISAGREE STRONGLY AGREE
The more you agree with the content of an item the closer to the right end of the bar draw the line.