Blog

Perusall Scoring Options: Annotation Only

December 13, 2021
Guest post by Lauren Barbeau, Ph.D.

When I first started using Perusall, “annotation content” was the only grading option available, which made it fairly easy to establish grading criteria. As Perusall has grown and adapted to respond to user needs, so too have the grading options. Users can now choose between “Annotation Content Only” and “Holistic” grading schemas. While the “Content Only” option focuses specifically on the quality of student comments as determined by Perusall’s algorithm, the “Holistic” grading option allows you to reward students for engaging in effective reading behaviors. Under “Holistic” grading, you’ll find options to give students points based on

• the number of times they open an assignment;
• how much of the document they read;
• how much time they spend actively engaging with content (reading, viewing, commenting);
• how many responses their posts garner from other students;
• how many upvotes their comments receive from peers.

One of the questions I most frequently get from faculty is, “Which grading schema should I use and why?” The answer here really depends on your preferences and pedagogical needs. To help you make an informed decision, this two-post series offers a deep dive into the why, what, and how of both grading options. In this post, I’ll focus on “Content Only” grading and provide an overview of my “quick and easy” tips for setup. If you’re more interested in “Holistic” grading, stay tuned for my next post!

Why use “Content Only” grading?

This option is good for instructors new to Perusall who do not have the bandwidth to determine their “Holistic” grading preferences just yet. While there are many pedagogical benefits to the holistic settings (which I’ll discuss in a future post), first-time users sometimes find these settings a little intimidating to process, though the recent updates to the “Scoring” page layout significantly lower the barrier to entry. If you’re learning Perusall on a crunch, “Content Only” grading can reduce the time you spend on setup. Veteran users may choose the “Content Only” option if their sole purpose is to get students to critically engage with reading content. These are the most common reasons I hear from faculty. You may have other reasons why this option best suits your needs.

What is “Content Only” grading?

This setting is exactly what it sounds like: the content of student annotations is the criteria used to establish their grades. Students need to complete the minimum number of annotations before the deadline to receive full credit. You also have the option to add a penalty for clustering comments rather than evenly distributing them, but student grades will otherwise be determined by what the algorithm deems the quality of their annotations, low, medium, or high. Each annotation receives an individual score based on quality. If a student writes more than the minimum annotations required for full credit, the algorithm averages the highest scoring annotations to arrive at the grade. Perusall’s “How It Works” guide provides more details about the nuts and bolts of how it works.

How do I set up “Content Only” grading?

To reach this scoring option,

1. Select “Settings” in the left menu pane.
2. Select “Scoring” in the “Settings” ribbon menu at the top of the page.
3. Select the “Annotation content only”.

When you select this option, the “annotation content score target” should be set to 100%, as in the screenshot below.

Once you expand the “Content Only” accordion menu, you will have a few more settings to configure. I offer some suggestions for quick set up below, but obviously, you can configure these settings to best meet your needs.

Annotations to Grade

The first decision you will need to make is how many annotations Perusall should score. If students complete more than the required number of annotations, (set to 5 in the screenshot below), the algorithm will average their top five annotations to arrive at the final grade. Students who do not complete the required number of annotations will be docked according to the number of missing comments.

To determine the number of annotations you should require for full credit, consider the average length of your readings. Let’s say you average twenty pages of reading per assignment. How frequently do you want students to leave comments? I typically recommend somewhere around half of the average number of pages for the required comments, in this case ten annotations, but anywhere between seven and fifteen would be appropriate. The idea is to encourage students to comment regularly throughout the reading.

Pro tip: On occasion, I’ve found that I assigned a reading significantly shorter than average, maybe only two or three pages. The brevity of the reading makes it nearly impossible for students to reach a seven to ten comment requirement. In such cases, you will want to override the course pre-set grading options when you create the assignment to lower the number of annotations required for full credit to something more reasonable for the assignment length. I learned this the hard way. To override your scoring presets:

1. Add a new assignment.
2. In the new assignment window, select “Scoring” from the top ribbon menu.
3. Select the “Specify custom scoring settings for this assignment” bubble.
4. Set the “Annotation content score target” to 100% to maintain “Content Only” grading.

As you scroll down, you will see the “Annotations to Grade” option, which will be set to your course standard. Here you can update this number to something more reasonable for the length of the assignment.

Relative Point Value

The “Relative Point Value” option allows you to customize points the algorithm will assign to comments that do not meet, meet, or exceed quality expectations. By default, the system assigns each comment a 2, 1, or 0 based on the quality of the content.

I typically recommend that faculty change the “relative point values” to 3, 2, 1 and reserve 0 for students who simply do not do the assignment, but this change only matters if you check the “Quality scores are absolute box.”

Additional Options

Depending on your pedagogical purposes for using Perusall, you may want to consider whether a “post-deadline reply window” or “late annotation period” is right for you. For the sake of simplicity, you can say no to both of these options. However, if your goal is to foster ongoing dialogue outside the classroom, you may find that allowing students to receive credit for responses posted after the assignment deadline allows you to reward students for continuing the conversation. Giving credit communicates your value for that kind of activity. If you enable this option, students will only receive credit for responses to other students’ annotations. The “late annotation period” gives students some flexibility and grace by allowing them to receive partial credit for new annotations posted after the deadline.

The last option available for “content only” grading is the distribution penalty. This setting determines what percentage of a student’s grade will be docked if their annotations are clustered too closely together. In the early development stages of Perusall, users found that students would read and heavily annotate the first few pages of a text. Once they met the required number of annotations, they would stop reading. With that in mind, you will likely want to establish some level of penalty for this behavior as it’s usually indicative that students are not engaging with the reading as desired. I go with something between 25-50%, but there’s no right answer.

Advanced

Once you’ve chosen your “content only” settings, pop over to the “advanced” tab where you will find a few more grading-related options. To calculate the top of my “assignment score range,” I multiply the number of annotations for full credit by the maximum score on an assignment (5 annotations x 3 max score on each = 15 points each assignment).

You don’t have to do it this way, but it’s the easiest option. I set the bottom of my score range to 0 and check the box that says “assign a score if a student completes no work.” In my grading schema where students receive a 1 for a low quality comment, only students who do not submit any work receive a 0 on the assignment. I do this because I want to encourage students to engage in Perusall, so in that regard, even an emoji response is worth something.

The “threshold grading” option is particularly useful for instructors who want to use Perusall as a completion grade. Think of threshold grading as pass/fail. Once students reach the threshold, they receive credit for completing the assignment. For example, the maximum point value for my assignments as calculated above is 15; I set the threshold to 10.

Students will automatically receive credit for completing the assignment once they reach 10 points. Anything lower than 10 points will receive a 0. Threshold grading may be particularly appealing to anyone concerned about how well an algorithm can grade student responses as this setting does away with the grade nuance. You can focus on giving students a completion grade instead of assessing the quality of their content through the grade.

These are my recommendations for quick and easy auto scoring set up. As you get more comfortable with Perusall, you may want to explore the holistic grade settings to encourage your students to engage in certain types of beneficial behaviors. In a future post, I’ll explore the holistic settings along with potential benefits to students.

Managing the Perusall Workload

November 10, 2021
Guest post by Lauren Barbeau, Ph.D.

A few weeks ago, I received the following email from a colleague: “Now that I'm solidly committed to using Perusall, I'm drowning. How do you keep up with all of those annotations? I know it was supposed to save me time, but I feel like I'm spending hours each week responding to students. Am I doing this right?”

If you’re anything like me and my colleague, promises of a lightened grading and feedback load may have convinced you to adopt Perusall. A technology that gets my students to do the reading, autogrades their posts, and eliminates the need for discussion boards and reading quizzes? Sign me up! Then the posts started flooding in. Students embraced Perusall and started posting far more than the required number of comments for full credit (an ideal situation!). As a conscientious teacher, I wanted to read all of my students’ comments and respond to as many of them as possible, but with an already full plate of teaching responsibilities, the time I began investing in Persaull just to keep up the comments became unmanageable.

I know I’m not alone; I’ve heard from several instructors who experienced a similar trajectory in their use of Perusall, some of them now so demoralized by the workload that they’re questioning whether using Perusall was a mistake. Before we throw the proverbial baby out with the bathwater though, let me offer you hope. I’ve been using Perusall since 2017, and in that time, I’ve developed the following strategies to scale back the time I spend engaging with students in Perusall.

1. Don’t read all the comments. No, really, don’t read all the comments. If you teach a small course or if your students haven’t started engaging to a point where you feel overwhelmed by their comments, reading all the comments might be an option for you. But be warned: you’re establishing a habit you’ll find it difficult to break if/when student engagement takes off.
2. While you’re at it, step back from responding. I know the personal pressure of feeling like you need to read and respond to most student comments to make it worth their while, but one of the best things you can do with Perusall is let the dialogue flow. If your students are writing tons of comments, that’s a good sign! They're talking to each other, which is what you want. If you respond too much, you can actually hamper discussion because students learn to rely on you instead of each other for answers. Think of this like a classroom activity. Can you be there to hear and respond to what every group says? Would you want to be? Probably not. Instead, you'd circulate, pick up pieces of conversations, and maybe chime in with a comment or probing question. You wouldn't expect yourself to hear and respond to everything. This is no different even if medium makes you think it is.
3. Instead of reading all the comments, use Perusall features to direct your attention.
   1. Confusion report: The confusion report can offer you invaluable insight into students’ most common questions in the assignment. This report provides you three areas of confusion for students as indicated by keywords gleaned from their comments. It will also provide you a few example annotations related to the keywords alongside the areas of the text students highlighted. If you’re not getting a confusion report, you will need to teach your students to use the question flag feature, but that’s a blog post for another day.
   2. All Comments: Access the “All Comments” pane for an assignment to see a quick overview of all comments, the number of upvotes a comment received, Perusall’s score for the comment, and the number of replies to a comment. When you see all of the comments in aggregate, you can make more informed decisions about how you spend your time engaging with students. See a comment with a lot of replies or upvotes? You will definitely want to read those threads more in depth. Notice that a particular student never gets any replies? You might review some of their comments and use the @ feature to call other students with relevant interests to engage with the thread.
4. Engage strategically. Research tells us that it’s important for instructors to demonstrate their presence in online platforms. That means engaging with our students online, but the best discussion board pedagogies teach us that we should not consider ourselves responsible for responding to every single student post. While I’ll often argue that we need to let go of some outdated discussion board pedagogies when working with Perusall, this is not one of those cases. Instead of responding to every student post, use these Perusall features to save you time and demonstrate your engagement.
   1. Upvoted by Instructor: This button should be your new best friend. When you upvote something, you provide formative feedback to all of your students by letting them know that something about this comment was worthy of your upvote. If you upvote everything, obviously the button loses its meaning, so use it wisely.
   2. @Name: Summarize/synthesize multiple comments in thread or across threads by tagging students by name with an @. Not only will you save time responding to comments, you’ll also help students engage in the more difficult work of synthesizing multiple perspectives on the content. Using the @ ensures that any student you tag will receive a push notification that will direct them to your comment.
   3. @conversation or @everyone: Perusall now allows you to tag @conversation to notify everyone who commented on a thread or @everyone to send push notifications to the entire class. Using these features can demonstrate to students that you are present in the course and draw their attention to important conversations while also saving you time for more pressing matters.
   4. Star comment: If you’re using Perusall in a synchronous course, let’s not forget that you can engage with student comments during your meeting time. The “star comment” feature allows you to quickly save comments for easy reference. I typically pull up my starred comments on my device to use during class time.

Hopefully, these strategies will help you make Perusall the time-saving platform it was intended to be. However, if you find you’re still struggling to rein in the time you spend engaging in Perusall, my last recommendation is to set a timer to limit your time in Perusall. Once the timer rings, commit to signing out of Perusall. This can be hard to do, especially if you’re seeing some thought-provoking discussions, but if it saves your sanity, it’s worth it! You can always return to read more comments as a reward for finishing all that other grading you’ll catch up on now that you’re not bogged down in Perusall posts.

Introducing Multimodal Assignments and Student Uploads

May 27, 2021

Multimodal assignments

Instructors can use a variety of materials in their Perusall course: a book from the Perusall catalog, PDFs, EPUBs, Microsoft Word and Excel documents, source code files, web pages, videos, and podcasts. Previously, instructors could assign multiple sections from the same text as one assignment, but could not assign multiple PDFs, Word/Excel documents, videos, podcasts, etc. (or a combination of these) together as one assignment. Assigning a document and a video entailed creating two assignments, one for each material. In this case, the result was two columns in the Perusall gradebook, two grade items in the LMS, the instructor spending time and energy figuring out how much weight to give each assignment, creating potential student confusion.

Instructors can now create multimodal assignments to address these issues. Multimodal assignments use more than one Perusall library item but only generate a single column in the gradebook. Multimodal assignments do not limit the number of items that can be included in one assignment, nor the kinds of materials that can be included in one assignment. To create a multimodal assignment, simply follow the same steps as creating a regular assignment, but click + Add another part in the create/edit assignment menu.

Student uploads

Peer review used to be a complicated maneuver in Perusall. Now, with the introduction of student uploads, instructors can enable students to upload materials to their course and then assign them to the class for a peer review exercise. (Instructors can use a differentiated assignment to assign each student a different document but generate only one column in the gradebook.)

Enabling student uploads entails creating a “Student upload folder” (Course home > Library) that students see in their library tab and upload to by clicking the button Upload to this folder. If it is an LMS-integrated course, students can see the folder by launching Perusall through a generic Perusall link (rather than an assignment-specific link).

Other features recently released

Podcast search

Instructors can now search for a podcast by title or description, rather than having to search and paste into the RSS feed URL.

Hide old assignments

Students and instructors can now hide old assignments on the course home page so the most current is always at the top.

Folder descriptions

Instructors can now add descriptions to folders that will appear for students.

5 Tips for Creating Meaningful Reading Assignments with Perusall

March 15, 2021
Guest post by Lauren Barbeau, Perusall Faculty Advisory Panel Member & Assistant Director for Faculty Development & SoTL, University of Georgia

How do we get students to engage meaningfully with our course texts? If you’ve been using Perusall for a while, you already know that this technology solves one of the most fundamental challenges we face as teachers—getting students to do the reading—but you may still be struggling to meaningfully incorporate Perusall assignments into your course. Here are some of the strategies I’ve learned after four years of developing Perusall assignments both as a faculty member and as a faculty developer helping other instructors use Perusall.

1. Make time to explain what Perusall is, how it works, and why you’ll be using it throughout the course.

If you’re familiar with the Transparency in Learning and Teaching (TiLT) framework, you might recognize the similarities here to defining the purpose, task, and criteria (or, as I prefer to say, the “why, what, and how”) for any given assignment.

On the first day of the semester, or in the welcome module of my online/hybrid courses, I make time to explain what I like about Perusall and why we’ll be using it: the metacognitive benefits of seeing what peers have to say about the reading, the community-building benefits, the intertextuality offered by Perusall’s integrative video and hyperlink comment features, the reading skills Perusall encourages students to develop for long-term success, etc. I also provide brief written and video tutorials on how to use Perusall, making sure to highlight any particular features that I want students to use. In my case, I tend to emphasize the @name and #hashtag options because they support my pedagogies, but that’s a story for another day. Finally, I review the grading criteria I’ve established for my course and discuss examples of annotations at various quality levels so students know how they will be evaluated.

2. Set up a practice assignment in Perusall to allow students to explore the tool on their own.

Over the past four years, I’ve given over thirty Perusall workshops, presented on Perusall at more than seven conferences, and conducted more one-on-one Perusall trainings that I can count. I consider this to be my greatest tip of all time: put your syllabus in Perusall. Not only will this give students an opportunity to acclimate to the Perusall environment, it will also ensure that they’ve read (and engaged with!) your syllabus.

I do this in my own courses and am always pleasantly surprised by the depth of student questions and comments; I certainly never had that experience with a traditional “syllabus day.” One faculty member I worked with commented that she has never gotten such insightful questions about her syllabus in all her decades of teaching. The types of questions she received helped her identify and revise areas of the syllabus for greater clarity. As she stated, she would never have known these aspects of the syllabus were unclear had she not posted it for students review in Perusall. Posting the syllabus in Perusall turned out to be mutually beneficial to both the instructor and her students.

3. Embed Perusall readings in Learning Management System (LMS) modules and frame readings in context of other course content.

While Perusall can be integrated into most mainstream LMS platforms, not all of us have access to this integration. Not to worry! You can still create a linked page that ports students directly to their Perusall assignment by pulling the assignment URL. Doing so makes it easier for students to access the right reading at the right moment, but it also appropriately frames Perusall readings as a part of your course modules rather than an additional thing that they need to do outside of the LMS. Just be sure to set the link to open in an external window.

Linking the Perusall assignment in your module allows you to frame the reading. I always end the previous page by stating that “The next page will link you out to this week’s Perusall reading” so students expect to jump to a new tab when they click the forward button. I usually provide a few framing thoughts and questions or highlight important ideas from the reading to which students should pay special attention.

4. Engage with students through Perusall.

The beauty of Perusall is that it’s not a traditional discussion board, which means you’re no longer constrained to traditional discussion board pedagogies. My advice: ditch discussion prompts and let students lead.

If you’ve followed the first three steps, students should be well primed to engage with Perusall sans prompts. Instead of posting prompts, let students know you’re present and reading their comments by posting follow up probing questions, synthesizing multiple comments and tagging students who should join the conversation, upvoting model responses, hashtagging emerging thread themes, or posting relevant links and videos to comment threads. These are only a few of the ways you can meaningfully engage with students through Perusall beyond discussion prompts. This is a new technology, and we need a new pedagogy to go with it. Get creative!

5. Follow up with students.

I expect you’ll be pleasantly surprised by the level of organic student engagement you see through Perusall. However, this does mean that you’ll likely be unable to read every comment on every reading, and that’s ok! Perusall’s analytics make it possible for you to engage with students in aggregate rather than having to read every single comment. The confusion report is particularly helpful here, but only if you’ve taught students to use the question flagging feature. If you’re teaching synchronously, you can use the confusion report to identify top areas to address in your next course meeting. One instructor I knew who was teaching online posted a video each week to address questions that arose from the confusion report.

I also find the “star comment” feature useful for identifying comments I’d like to address further. This feature creates a curated list of comments you can easily return to in a synchronous meeting or in a follow-up video. While I may not be able to respond to every thread (nor should I—I want to encourage students to become co-creators of knowledge rather than cultivating reliance on me as the giver of knowledge), I can acknowledge and expand on useful conversations by quickly skimming through comments and adding them to my curated list.

Now that you’ve discovered the secret to getting your students to do the reading, I hope you can use these tips to more fully incorporate Perusall into your pedagogy to create meaningful reading assignments for your students.

Social Annotation and Community Building

January 22, 2021
Guest post by Nicholas Miller, Perusall Faculty Advisory Panel Member & Assistant Professor of English, Valdosta State University

Although I began using social annotation practices before the COVID-19 pandemic hit, I have found myself increasingly relying on Perusall to guide and structure my classes now that I am primarily teaching online. A number of folks have written about the value of social annotation in the past year, so I do not want to rehash the many great arguments in favor of using platforms like Perusall . The benefits in terms of reading comprehension, close textual analysis, and critical thinking are pretty clear. What I do want to think about for a moment, however, are the more intangible benefits of such platforms when it comes to building community.

One of my biggest concerns when teaching online has always been the inability to replicate my charismatic personality in a learning management system. I rely heavily on tone, visual cues, and enthusiasm to steer and direct conversations in my face-to-face classes—something that can be hard to make visible online. This is especially true when teaching asynchronously. Yet the move to using social annotation tools, and using them heavily, has allowed me new modes of creating community. They are not mirrors of the work I do in-person, but they cover a lot of ground. More specifically, by encouraging formal and informal responses to the readings, I have started using all of the multimodal writing tools available in Perusall to communicate personality online. For example, the use of emojis, GIFs, hyperlinks, memes, and hashtags in my own comments creates a number of the side conversations (about cultural references, popular media, and rhetorical strategies) that can go missing when in-person classes move online.

Using such features allow me—and my students—to present a bit more of ourselves in virtual spaces than one typically sees on a discussion board or in a quiz. These micro-relationships that form, then, become foundational to our class community and eventually enable students to more freely and eagerly respond not only to me online—but to their peers as well. And one striking revelation in doing this has been how social annotation platforms, in some ways, become more egalitarian spaces for my students. The introverts are not left behind when a frenetic conversation does not leave them time to think through their ideas. Students who might struggle to speak up when seeing themselves occupying a minority identity or ideology next to their classmates is often able to speak up better in these virtual spaces (and can use anonymity tools for some posts)—which enriches and complicates our discussions of the texts. And for me, as somebody who often teaches visual texts, the ability for students to come together to make sense of small details in a comic or piece of art can forge bonds that remind me of the discoveries that get made when we tackle passages closely together in a face-to-face class. If I have taken anything away from this year spent teaching entirely online, it is that social annotation and collaborative reading practices will be a hallmark of my teaching for years to come—regardless of the classroom modality. If you have not taken the opportunity to try out platforms like Perusall yet, I highly recommend it. And if you have practices and/or assignments tied to social annotation that you want to share, please do!

Course affected by COVID-19? Publishers now offering free textbooks through Perusall this spring

March 23, 2020

[Note: This post refers to a publisher offer from Spring 2020 and is no longer active.]

We designed Perusall to support learning no matter the mode of delivery: if you’re moving your class online due to the Coronavirus epidemic, Perusall will create an engaging virtual learning experience for your students.

If your students previously purchased books outside of Perusall, and you’d like to move to Perusall now, we may be able to help. We have arranged with an increasing number of our publishers to offer free textbook rentals in Perusall for students in ongoing courses affected by COVID-19, when your students previously purchased access elsewhere. Those currently participating include:

• Broadview Press
• Cambridge University Press
• Hackett Publishing Company
• McFarland
• McGraw-Hill Education
• Oxford University Press
• Princeton University Press
• SAGE
• Springer
• Taylor and Francis (including Routledge and CRC Press)
• University Science Books
• Wiley
• W. W. Norton & Company

(Some restrictions may apply, depending on the publisher.)

If you’d like to take advantage of this offer for your course, email us at support@perusall.com and let us know what book(s) you are using in your course and how many students you have. If permitted to by the publisher, we’ll provide you with an access code your students can use to bypass the paywall for your book(s) until at least May 25. (Let us know if your semester extends beyond that date.)

If you don’t see your publisher on the list above, contact your local publisher sales representative for alternatives that may be available.

Please feel free to share this information with colleagues who are moving their classes online, or with your administration.

How Perusall helps classes moved online due to Coronavirus

March 8, 2020

Perusall is a free platform that guarantees students are prepared by managing the entire out-of-classroom experience for you, automatically. We designed Perusall to support learning no matter the mode of delivery: if you’re moving your class online due to the Coronavirus epidemic, you can count on Perusall to create an engaging virtual learning experience for your students. In this time of crisis, when faculty are being required to move in-person courses online, will you share this information with your colleagues or administration?

As the only truly social e-reader, Perusall keeps students motivated and learning even without an instructor standing in front of a classroom. Because setup is so fast, with or without integration into your school’s learning management system, Perusall can be quickly deployed mid-semester into an existing course. Research shows that Perusall increases student reading preparation from an average of 20-30% to >90%. By combining collaborative discussion and content annotation with automated grading that keeps students on track, Perusall creates a community among your students -- and improves their learning outcomes -- even when no one meets them face-to-face. No matter how you innovate teaching during this crisis, Perusall can help you maintain academic rigor and guarantee students learn.

Perusall is always free when used with your own content or open educational resources; there is no limit to the number of courses you can create or the number of students you can use Perusall with. Create a new course now, or email us at support@perusall.com. We’ll get you set up, answer your questions, or discuss any pedagogical issues that arise.