Resources Available for Student Ratings

Fresno State Student Rating of Instruction (FSSRI)

Explorance Blue Website Learn more about Explorance Blue

Instrument Guides

  • FSSRI Items and Categories (last updated 03/2021) pdf
  • Guideline for Proctors - last updated fall 2011 pdf
  • DAA PowerPoint overview - April 2020 ppt

Answers to Faculty Questions 

Why did we stop using IDEA?

When our contract with IDEA ended, the Academic Senate decided to look into other options rather than automatically renewing it. IDEA was about to make major changes to its administration, including eliminating the option for paper questionnaires. Proposals were solicited from companies that provide this service, and a task force was created of Fresno State experts to propose an instrument created specifically for our needs. In the end, the Senate voted that the Fresno State instrument was a better match to our needs than others in the running, including the IDEA instrument.

What is the new instrument?

This new instrument, the Fresno State Student Ratings of Instruction (FSSRI), was designed by a task force comprised of faculty representatives from all 8 colleges, some with professional expertise related to the assessment of learning as well as scale construction. It is based on the principle that we should ask students to report on directly observable instructor behaviors for which there is published empirical evidence that the behavior promotes learning.

The specific items are available for review on the faculty affairs website, http://fresnostate.edu/academics/facultyaffairs/documents/teaching-effectiveness/FSSRI_Question_Items.pdf, subject to approval by the Academic Senate and the Provost, which we anticipate in September 2019.

Does it meet the requirements of our APM?

 Absolutely. The APM requires several things:

1.  Assess these three dimensions of teaching: instructional design, instructional delivery, and assessment. The FSSRI includes four items about each dimension for a total of 12 items. We selected the topics of the four items by reviewing the published empirical literature on effective teaching and learning and identifying those practices that are known to promote learning.

2.  Allow departments to customize questionnaires by choosing from a pool of items approved by the Academic Senate and the Provost. The FSSRI has essentially 12 small pools of questions. This assures that all of the mandated dimensions (see point #1) are included. Each pool of items has a default item, but faculty who use an online questionnaire may customize by replacing that item with another from the same pool of items, or by adding even more items from that pool. 

The instrument must be reliable and valid. We conducted extensive pilot testing in Spring 2019 on the new instrument. That project firmly established the reliability and validity of this tool. While additional research can (and should) be done to investigate additional forms of reliability and validity and to make improvements, we have done more than most proprietary student ratings instruments. Full details are available upon request. 

How will I customize my student ratings questionnaire? What is the process?

Approximately two months before the end of each semester, instructors of every course in the system will receive an email notification that the system is open for customization. You can follow the link in the email, go into the      Course Ratings link that will appear in your Canvas courses, or go directly to the Explorance portal (csufresno.bluera.com/csufresno.edu). Once you get to your Explorance portal, (Explorance is the company that administers the questionnaires for us) you have three choices for each course:

  1. Opt OUT of evaluation for that course if it is not necessary by your department policy. Please consider opting out of labs that are attached directly to lectures if they have the same instructor. Please consider opting out of independent study or thesis units, especially if there is only one or two students, such that they could never be anonymous. 
  2. Select the PAPER option, in which case you will not be able to customize the questionnaire. When you select paper, a PDF of your survey will be sent to the Print Shop.  It will be printed on your behalf, expenses charged to your Dean’s Office (which will charge it back to your department) and you will receive a delivery of the printed surveys.
  3. Select the dates/times that your evaluation should be active. Please consider opening it on the day you intend to provide time in class to complete the survey, and leaving it open until the last day of instruction.  The period has been extended to include the week of Final Exams.

You will have only two weeks to make these selections. For classes that will be evaluated online, you will receive notification two weeks later that you can customize your questionnaire using the available items. Again, you can follow the link in the email, go to the Course Ratings link in Canvas, or go directly to the Explorance portal (csufresno.bluera.com/csufresno). You must have at least one item in each of the 12 categories, so at least 12 items. You may add items, up to a limit of 24 total. You may add written questions in addition to the standard items.      

 

Can I still administer my student ratings on paper, or do I have to go online?

Paper questionnaires are currently still an option, but the extra cost of paper questionnaires is prohibitive, and we may eliminate this option in the future. For now, when an individual faculty member chooses to administer questionnaires on paper, that person’s department office will be responsible (both financially and in terms of labor) for printing the questionnaires. Also, it will not be possible to customize paper questionnaires by swapping out or adding items. Some departments have decided to require online administration of surveys. 

What is the timeline? For Spring 2020

 

For Spring 2020:

March 16-29: For each course, you will choose one of the following:

  1. Opt-out: Obviously, you should check your department evaluation policy or talk to your chair if you’re not sure if you are allowed to opt-out of a course. You should opt-out of lab sections if you’re already evaluating the lecture and don’t want the lab evaluated separately. 
  2. Choose the paper option: Courses selected for paper will automatically be sent to the Print Shop this semester and charged to your Dean’s Office, who will then charge the cost back to your department.
  3. Select dates for online rating forms: Rating forms can be available anytime from April 13 at 12:01 am until May 15 at 11:59 pm. You can select any timeframe within these dates. Please consider opening your surveys on the date you plan to administer them in class, and leaving them open for at least a few days so that students can finish them at home if they don’t have enough time in class. 

 March 30- April 12: Faculty administering rating forms online can customize the student rating instrument by choosing at least one item from each of the categories in the item bank. They are posted here if you want to look in advance: https://www.fresnostate.edu/academics/facultyaffairs/documents/teaching-effectiveness/FSSRI_Question_Items.pdf

Default items are highlighted in yellow and will be automatically selected if you do not make any changes.

April 13-May 15: Student rating instruments will be available for those who did not narrow down the dates of availability.

What if I teach a lab class? Or something else that doesn’t fit the typical classroom setting?

Each of the 12 pools of items include some that are designed specifically to capture those educational settings. Specifically, we have items for labs, and items for off-campus placements such as internships and practica.  

Are there special items for online classes?

No. Online courses will be evaluated by the same criteria as face-to-face classes. We did verify that the wording of most items will work on the online context. 

How will students access the online student ratings questionnaires?

Students will be invited by email to complete their questionnaires. They will also have reminders pop up in Canvas. Or students can go directly to the Explorance dashboard to access their surveys. csufresno.bluera.com/csufresno  

How will I get a good response rate with online student ratings questionnaires?

The approach that yields high response rates is to administer the student ratings during protected time in class, just as faculty do with paper questionnaires. Students will be able to access the questionnaires on their phone/laptop/tablet in class, when you tell them it’s time to do so. Anyone not in class that day can go online later to complete the questionnaire if they wish. When this method is used, the drop in response rates is minimal.

Also, the student rating system is automatically embedded into Canvas. Therefore, whenever a student goes to Canvas, he or she will see an automatic pop-up reminder to complete the questionnaires that have not yet been completed. Furthermore, faculty will be able to monitor, in real time, what their response rate is. (Not WHO has responded, or see the completed questionnaires of course, but just how many students have responded.) These strategies will be very useful to those who are not able to administer the questionnaires in class. 

Can I identify the date when student ratings questionnaires are available to my students?

Yes. When you go to your Explorance Dashboard to select online administration, you will be able to designate dates, so long as they are between the dates designated for student ratings (generally the last month of the semester.   Ideally, you would open them the day that you intend to provide time in class, and keep them open for students who missed class that day.

Do online student ratings administered in class still need a proctor?

According to the APM, yes. If the Senate wants to reconsider the policy (which is already nonsensical for online classes), they can consider this issue at that time. But with online questionnaires completed in class, the job of the student “proctor” will probably be just to alert the instructor when everyone is finished. 

What will happen if I do nothing?

If you do nothing, then your course will have an online instrument created that will consist of the 12 default items. Your students will be notified by email and in Canvas when the survey is waiting to be completed, and it will close automatically at 11:59 pm on the last day of instruction.  Afterward, the report will be generated and will be available to you and your Dean’s Office, through the Explorance dashboard, to be placed in your personnel file. 

What if I don’t need to administer student rating surveys in my course this semester?

The new system is going to automatically generate questionnaires for every course in the system. So you will have to Opt-OUT if you do not want to evaluate a class. Please note that this is the opposite of what used to happen. In the past, packets were only prepared if you opted IN. If you are going to opt OUT, look for the email that should arrive about two months before the end of the semester alerting you that it’s time to customize your student rating questionnaire. Follow the link, or go into Canvas to find the link, or go directly to the Explorance Dashboard and turn OFF evaluations for the specific course that does not need to be evaluated. Note that department chairs will be notified about who has opted out to verify that it is allowed by the university and department policy. 

How will I get my reports after the semester is over?

Score reports will be available in the faculty Explorance portal one week to 10 days after the grades due deadline. Reports for those who administered their surveys on paper will take longer. Chairs will have access to written comments to review, and deans offices will have access to the reports for placement in personnel files. Go to my.fresnostate.edu (Faculty Self Service).

How are my scores likely to compare to my IDEA ratings?

Like IDEA ratings, the FSSRI ratings are skewed positively (mean score is 4.1 on a 5-point scale). The correlation with IDEA is .80. So your scores are likely to be very similar. Out of the 67 classes in the pilot study with both IDEA and FSSRI data, 24 had identical scores on the two instruments.  Another 21 had an FSSRI score that was lower by 0.2 of a point, and 11 had an FSSRI score that was higher by 0.2 of a point. Only 2 classes had a score that differed by 1 point (on the 5-point scale) and those both scored higher on FSSRI than on IDEA. So we don’t expect any surprises with the data produced by the new instrument. We just expect that they will be more useful because instructors will see exactly what they can do to improve because it’s obvious from the items.

What is known about the validity of online student ratings?

Who can answer questions about this?

Talk to your college representative on the Task Force, or direct questions to the chair of the committee, Katie Dyer.

  • Kathleen Dyer (Chair) (College of Social Sciences)
  • Bernadette Muscat (Ex-officio)
  • William Arce (College of Arts and Humanities)
  • Monica Billen (Kremen School of Education and Human Development)
  • Nagy Bengiamin (Lyles College of Engineering)
  • Jennifer Adame-Walker (College of Health and Human Services)
  • Dermot F. Donnelly-Hermosillo (College of Science and Mathematics)
  • Mariyah Yukhymenko - (Expert in Student Ratings) 

What are best practices for the interpretation and use of student ratings data?

Link to a literature review and analysis focused on how administrators and personnel committees should use student rating data.

https://www.sciencedirect.com/science/article/pii/S0191491X16300232

Where can I see the questions I can ask?

You should be able to see them in Canvas for the courses which you chose to be evaluated, during the evaluation period. Go to Canvas account > Course Ratings. You can access them at all times through my.fresnostate.edu. (Faculty Self Service)

Where can I see what questions I chose?

You should be able to see them in Canvas for the courses which you chose to be evaluated, during the evaluation period. Go to Canvas account > Course Ratings. You can access them at all times through the Faculty Affairs page. Go to Teaching Effectiveness > Resources for Available for Student Ratings > FSSRI categories and items.

Where can I check my response rates?

 

You should be able to see them in Canvas for the courses which you chose to be evaluated, during the evaluation period. Go to Canvas account > Course Ratings. You can access them at all times through my.fresnostate.edu (Faculty Self Service)

I co-teach with another instructor - how do we do that?

Both instructors need to be listed as primary instructors in People Soft. If you are not listed as the primary instructor, please contact your DAA and Jolynne Blake (jblake@csufresno.edu)

Is there a way to reset a student's attempt? I had a student who thought they were rating another instructor but used my evaluation.

Contact Jolynne Blake (jblake@csufresno.edu).

How come only some of my students have access to the evaluation through Course Ratings in Canvas? 

If you have a cross-listed course in Canvas then some of your students may not be able to access evaluation through the Course Ratings link. Have them click on their Canvas Account section to access evaluation there or through the email they received from Explorance.

I want to add more questions but it's not letting me.

You can only add questions during the customization period. You must be in EDIT mode. If all your options are grayed out, log out and log back in. If the administration period begins, you can no longer edit or add questions.

Are my students' responses anonymous? 

 

Responses are anonymous. Through the Explorance Student Rating portal (available in Canvas or my.fresnostate.edu),  you will be able to see total class response rates, but not a specific student's response. After grades are due, faculty will see typed comments.