Tag Archives: Assessment

Coffee Talk: Class Discussion as Formative Assessment

When we talk about formative assessment, what usually comes to mind is a quiz, a ticket out the door, or a temperature-taking tactic like "thumbs-up/thumbs down."  Sometimes when I am feeling a little extra, I break out a Kahoot.

However, my favorite kind of formative assessment is the kind where students don't even realize that it's happening. I love it when students let their guard down, and instead of trying to produce The Correct Answer, they show me what they know. This is why I am a fan of using class discussion as a form of formative assessment.

Whole-class discussion works well enough on the fly when I am tired of the sound of my own voice, but it's far from perfect. Foremost, there are always a few students who regularly chime in, but there are many more students who are all too happy to sit back silently and let others do all the chatting.  I also find that many students don't feel comfortable in this setting to ask for clarification of a point, and so, again, they may sit back silently, hoping that the clouds will part and a ray of sun will illuminate whatever is confounding them. Another issue I experience with whole-class discussion is that, because I am the one leading it, students are less likely to take risks with the ideas they share. Instead, they are inclined to say what they think I want to hear.

A strategy I find myself relying on more often than when I first started teaching college is small group discussion.  Think-pair-share is a great strategy because it takes very little preparation. Sometimes I'll step up my game and grab the Student Sorting Pencils from the CTLE. These allow me to keep the groups moving in unexpected ways by breaking them into groups by color, numeral, or symbol. This method takes a few minutes of prep time, but it feels more exciting for everyone than being asked to turn and talk to a neighbor.

In these small group discussions, I am able to hang back and listen to what students are saying. Students seem to feel more comfortable asking classmates for clarification on a finer point that they might not ask the whole class. They also feel more comfortable calling me over for 'official' clarification. In small group discussions, I can overhear not only whether or not they understand the material, but I can listen to their thought process as they arrive at an answer. Furthermore, if I am tired of the sound of my own voice, surely they are too. The small group discussions break up the class time and allow students to relax a little.

Even in these small groups, when I sidle up to them, sometimes the conversation will cool, like I've removed the lid from a boiling pot. I don't know if they are worried that I might overhear The Wrong Answer and frown disappointedly? Or maybe they think I am there because I want to chime in, so they are making space for my interjection? Either way, I regularly find myself assuring students that I am just listening in and to continue as though I am not there.

One discussion strategy that has been collecting dust for the last semesters is the Socratic Seminar. Although this strategy takes more than a few minutes of preparation--not only on my part, but on the part of the students as well--it solves a few issues that whole-class and small-group discussions present. In a Socratic Circle (where an inner circle of students discusses prepared questions, and an outer circle of students thoughtfully observe and jots notes), I am not the leader of the group. This clears the way for students to strengthen their connections to one another as they share ideas and moderate their own discussion. Furthermore, I don't have to meander around the room to eavesdrop--I can take a seat at the back of the room and listen. They all but forget I am there when the discussion gets rolling. This is a safer space for students to share their thought processes and to take risks than in an instructor-lead discussion. Finally, students seem to like it. Win-win.

With each of these small(er) group strategies, I am able to use the authentic information I gather from students to shape future lessons or to know when to slow down and revisit a concept that seems to be tripping them up.

The opportunity to compose this post has set the gears in my mind turning. Most instructors use Socratic Seminars to facilitate a discussion about something the class was assigned to read, but I don't see why this strategy wouldn't work in a writing lesson. Maybe students could examine a sample essay or could talk about small portions of student drafts in a modified peer review. I'll let you know what comes out of these lesson plan ponderings.

I'd love to hear your tips and tricks for using discussion as a formative assessment. Do you use Socratic Seminars in your courses? What is your favorite strategy for class discussion?


Faking it

When was the last time you saw a film or television show where someone was supposed to play an instrument or sing well?  When that moment of reckoning occurs I always hold my breath and wait for the tell.  The tell is the point where it is clear that the actor is faking it.  That actor may be faking it successfully or poorly – or, of course, the actor may actually be a musician, as many are, and is not faking it at all.   But if faking does occur, an editor often gets involved to fake it further.  We see, we listen, we constantly assess.

When we assess students isn’t this ultimately what we are trying to determine? Are they faking it, or do they know and understand the material? As a musician, do they know how to play musically or are they simply playing the notes?

When we assess aren’t we also looking for those who fake it well?

One of the jokes among instructors of applied music (performing music) is when the teacher corrects the student and the student says, “Well, I just don’t understand. It sounded perfectly fine in the practice room.”

What that means is that most of the time (not all) the student can’t tell the difference and is, ultimately, faking it.

Fake It ‘Til You Make It?

I remember a student who was excellent at mimicry. I learned never to play a piece for her because she had too good an ear and could fake it. The problem was that she could not read music. She was a good pianist but when asked to play something that she misheard or ostensibly misread because it was incorrect she could not “replicate the results.”

I have tremendous respect for her because she had been playing for many years and had to face the fact that not only was she faking it but she had to face the degree of how much she was faking it. If she wanted to continue lessons she had to re-learn how to read music after many years of classical piano lessons, her chosen genre.

There is a part of me that thinks she always knew how much she was faking it but she had choices moving forward. She could have stayed at the level where she was because she could fool many. She could have gone on her merry way and continued to play the way she did. She could have walked away and given up. Music was not her major so that might have been the easiest choice. She chose instead to go back to the basics and learn how to read music. It was a daunting task and I commend her for her perseverance. It was a lesson in patience because, in this case, one does not fake it until you make it. She’d already been down that path.

Two Studies that May Surprise You

I love surprising students (not that you are students) so I leave you with two listenings of people who are not faking it – or are they? In the first example, Bence Peter’s Fibonacci series moves to a video image which allows him to re-sequence the series so that it can go backward, using digital editing. This video clip often offers my students a new “take” on music and they are surprised because they are hearing something different. If you are using speakers, turn them up. In the classroom, I usually turn off the lights.

The second video clip shows an interesting twist on talking and singing where he includes the spoken word. Is he having trouble or is he faking it? Here is Al Jarreau.

 

The Assessment Wheel Goes Round and Round

I would like to share a few thoughts on assessment, if I may.

I have to begin with a slight clause to this week’s blog by saying two things.  First, I am into assessment. When Julie Morrison came out in her “Get Your Assess in Gear” t-shirt, I had the biggest smile on my face.  What a great way to promote the positive aspects of assessment and how important it really is!  

Second, I am a big believer in refreshing one’s self. I make a point of regularly attending different professional development opportunities when they are available, in part because keeping my knowledge up to date is important, but also in part because I know I have forgotten some of the ideas I have picked up over the years as I have tried new things, ideas that work and that I want to try again. Last week, I attended the CTLE’s workshop on “Keys to Unlocking Effective Assessment” as a refresher on effective assessment approaches and to ensure that my understanding of assessment is compatible with GCC’s understanding of assessment. Therefore, assessment is right at the forefront of my mind.

It was a great workshop.  If you haven’t participated, allow me to encourage you to do so.  It is a workshop, not a lecture series, so it is involved and interactive, and I really enjoyed it. Thanks, Meghan!

Thinking back, I can recall horror stories about how hard the assessment course was in my Master’s program, but the class was not only interesting, it was fun and I really understood the way assessment worked as an intentional aspect of course implementation and how it operated in a way to promote student success.  The teaching practicum in my MA required students to come together and collaborate on ideas for incorporating formative assessments as well as the “big” summative assessments and using indirect and direct measures for different feedback in the composition courses we taught. I learned to think of assessment as a living breathing part of the teaching approach rather than a separate entity, a consistent feedback loop that allows performance to engage with working towards strategic objectives or learning goals and encourages self-awareness.  

As a result, I don’t see assessment Data (big D) as numbers. As an English faculty, my mind lends itself to words more easily than it does to numbers. Numbers don’t scare me, but I find it easier to work my way through information in a linguistic fashion. My experiences with assessment in my MA program approached the data more holistically in a way that encouraged my ‘English’ mind to weave the information garnered from the mixed assessments into the teaching and learning process as a whole. Not numbers, concepts and ideas. Data isn’t the numbers, it is what the numbers mean. 

So, in my experience, assessment is continuous. It enables adjustments to teaching-strategies and methods. It provides a constant feedback cycle that allows the students to become self-aware and encourages self-confidence as they learn what they know and what they don’t know and start asking the right questions to expand that knowledge. It helps me as a teacher properly support the student at the right time and in the right way.  It helps me to make adjustments to what I am doing in the classroom to make sure the students are achieving their goals as well as my own. 

The continuous assessment cycle doesn’t just impact my students, it impacts me. All that feedback, all that ‘Data’, provides me with the opportunity to improve, to grow, and to adapt as the educational environment changes around me.   With my students, I have a moment in time to “assess for success”, and yes that can have a huge impact. On the other hand, the continuous assessment also provides me with ‘Data’ on what is working for me in the classroom. What approaches are working? What methodologies are working for the current student population? What activities are providing the best interactions with the content? What areas do I need to approach differently? Which projects are the students embracing as a vehicle for understanding? To put it simply, what works and what doesn’t?  

The Keys to Unlocking Effective Assessment began with a quote, and I am going to end with part of it here.  “Perfection is always just out of reach, but continually striving for perfection contributes to keeping both our instruction fresh and our interest in teaching piqued.”  -E.S. Grassian

And that is why I embrace continuous assessment.

This week’s genially infographic on assessment can be found at this link if the embed code isn’t working.

 

Assessment Success in the Mathematics Department

Written by Dr. Ashley Nicoloff and Dr. Krysten Pampel

As the department assessment coordinator (DAC), I have the opportunity to help my entire department assess each of their sections in the fall semester and analyze the assessment results in the spring semester. I would like to use this opportunity to share the yearly assessment process that our department goes through.

In the Fall we assess every section taught in the math department using google forms. This means that our 300+ sections of MAT and CSC courses are assessed. The course level assessments that we give are built by the course coordinators and the team of instructors that teach that course. Each assessment is roughly 7 questions in length and is projected to take no more than 15 minutes of class time.

As the faculty are giving the course level assessment through the fall, I as the DAC, record which sections have taken the assessment. I then send out reminder emails about the course level assessment with the number of completed sections so far. Many of the instructors like to use the course level assessment as a quick review near the final which makes me nervous since it always feels like there is less time near the end of the semester. During finals week, I send out the results of the number of sections that took the course level assessment.

In the Spring, I meet with all the course coordinators during the week of accountability to clean and review the data. We also take the opportunity to report the findings of the data if time allows. During the spring semester, the course coordinators meet in person or virtually to discuss the results with their instructional teams and how they want to proceed for next year. Sometimes there are rewrites to an entire course-level assessment, sometimes we change the placement of answer choices, and occasionally we leave everything alone in order to collect more data. I take all the changes that are requested and I update all the course level assessments in digital and google form format.

Before the fall semester, I meet with the course coordinators to have them verify the changes to the course level assessments and ensure that the assessments are ready for responses. This meeting also allows me the opportunity to update them on any changes in assessment for the academic year. This could be anything that I learn from the DAC meetings or something that comes down from the district.

I am very proud of the math department sticking with this assessment cycle and being willing to give up some class time to assess their sections of students. This information has helped us guide instructional moves and department-wide strategies to provide our students at GCC with the best MAT and CSC instruction across all sections we teach. The data we collect also assist the college in keeping the accreditation status with the higher learning commission.

 

Pain & Suffering or Just Assessment & Evaluation?

That’s how many instructors and students feel about assessment and evaluation. It’s a lot of needless pain and suffering. It always seems so punitive to students who struggle. But assessment doesn’t have to be that way. Many instructors have found ways to teach and use assessments in a way that encourage students to do better the next time. The key is that there is a next time, and that can be the challenge.

In writing courses, instructors can get overloaded with grading. The more a student writes the better that writing becomes, but who has time to grade all that writing. Apparently writing instructors do. However, there are ways to break down the concepts and skills needed to write well and have students practice those concepts and skills without the need of instructor grading. For instance, much of the bad writing that I see, stems from poor sentence structure. Students love a good run-on sentence, with a few fragments thrown in for good measure. It drive me crazy. “Use a comma or a period somewhere, please,” I beg.

Lucky for us at GCC, we’ve found an adaptive learning tool to help us teach students the grammar and mechanics skills, including sentence structure that they struggle with. If you’re not familiar with adaptive learning, it “is an educational method which uses computer algorithms to orchestrate the interaction with the learner and deliver customized resources and learning activities to address the unique needs of each learner” (Wikipedia). The tool we adopted from McGraw-Hill is called Connect, which includes LearnSmart Achieve. LSA provides an adaptive learning system designed to identify students’ areas of weakness. It uses supplementary content, such as videos, interactive activities, additional readings, and even a time management feature, all intended to guide students through content and resources at an appropriate pace. You can see an example below.

The beauty of this type of tool is students are being assessed all through out the process, and the system is adapting to their needs. If they’re struggling with the content they get more resources and more practice. If a student clearly understands, they hit mastery sooner and complete the lesson. So instead of a lot of pain and suffering, students get what they need. Missing a question doesn’t seem like a punishment. It becomes and opportunity to learn why and try again until they get it right. And as an instructor, I don’t have to grade any of that work. That’s the real beauty. My assessment comes when they put those skills to the test on an essay assignment.

Unfortunately, we can’t eliminate all the pain and suffering. At some point students have to write an essay, and instructors have to grade it. Well, more like grade 100+ of them (24 students x 5 classes). And we assign 3-4 essays in each course, so it’s still a lot of grading. But I digress. Once a student submits a finished essay, eager with anticipation of a passing grade, it takes some time to get that feedback back to students. During that span (1-2 weeks on occasion), students forget all about that paper and the effort or lack of effort they put into it. And when the paper is return, the process often ends there. There’s no motivation to do better. We teach that writing is a process, yet we make the process end when we’re ready. I believe with a C paper and especially an F paper, the process is not over yet. The student needs to continue to work on that essay, not the next one, in order to improve his/her writing.

So my assessment technique involves giving students an opportunity of a rewrite. Yep, more pain essays for me to grade. But it works because students have to tell me what it is they did to improve the essay. What skills did they work on? What help did you seek? Did you work in LearnSmart Achieve? Did you visit the Writing Center? Did you schedule a conference with your instructor? So the process doesn’t have to end with an F paper crumpled and thrown in the trashcan as the student walks out the door (clearly that’s an old reference to times gone by). Writing is a process and the only way to get students to write better is to keep the process going for as long as they need.

Example of McGraw-Hill LearnSmart Achieve

Changes that Lead to Student Success

After years of doing assessments and submitting the results before the end-of-term deadline, I finally realized I could actually be using the data. I have finally made some consistent changes that have led to greater levels of understanding and success in my classroom. Here are my top three.

Change # 1
Every single Exercise Physiology class starts with music and movement. Not just some classes when I feel like it. All classes. You might be thinking to yourself, “well of course, it’s an exercise class, why wouldn’t you be doing exercise with them?” I am teaching the science of exercise, so they are basically learning anatomy and physiology and how that applies to the acute and chronic adaptations to exercise. So, it is highly plausible that I could lecture for 75 minutes straight. Zzzzzzzz.

But no more! I have physical and visual evidence that my students are more engaged following a three minute bout of movement to music that will last for at least 30 minutes.

Change # 2
I have Included the arts in my sciences. I make my students draw pictures in their notes. The art lovers in class really enjoy this, and the non-artsy people appreciate that I bring coloring pencils and I teach them how to draw in a very simplified manner. I also give them visuals to think about to really break down the parts of their drawing. For example, the cell body of a neuron looks a lot like an egg after you have thrown it onto a hot oily frying pan. And the central nervous system (brain and spinal cord) looks just like a lollipop.

It is much easier to review your notes when you have pictures depicting what your words are telling you. Just like I am more likely to read a textbook that has helpful pictures rather than all text and tables.

Change # 3
Less words on slides. I can actually watch their cortisol levels rise when I put up a slide that looks like it has 250 words on it. The serious note takers go into panic, wondering how they will ever jot down all these words. No matter how many times I tell them they have access to the slides, they still feel the need to write everything down, just in case it is on the test. So if you remove all that text and put down two key words that have an emotional impact, they are forced to think for themselves and jot down their own notes.

That is another opportunity to draw images on the board, give examples and simply explain the topic as it relates to their world. Then they give me their examples, we all nod in universal acceptance and we can move on to the next topic. Making an emotional connection will have a greater impact on memory compared to a slide full of words.

So just to recap: move to music for three minutes, encourage the arts, and post impactful words, not paragraphs.

 

Dogs and CATs

I’m a dog guy. I didn’t really know it until later in life. Our family had cats when I was growing up. I remember Frisky and Misty, but those memories are somewhat cloudy as I was fairly young. After I got married and moved to Arizona, my wife and I adopted our first dog, Virginia, named after the state in which we met. She was a beautiful black lab, but cancer took her from us too soon. She did get to both of our kids; however, she was not around long enough for them to have any vivid memories. But, after having Virginia, we quickly became a dog family. Flash forward to today, and we have three wonderful dogs at home. Hero is a loving, carefree Golden Retriever, who we have owned since he was eight-weeks old. We also have two yellow Labrador Retrievers, Ginger and Obi. Both are rescue dogs, and both are incredibly sweet and loving in their own way. Three dogs in the house is “a lot of dog” as we like to say, but we wouldn’t have it any other way.

As I reflect on my love of dogs and my tolerance of cats, it conjures up some connections to our roles as educators. First, I believe effective teachers mirror some of the characteristics of dogs. When I come home from work, our three dogs are absolutely overjoyed to see me – a barrage of wagging tails, playful jumps, and flops at my feet. With a greeting like this, the worries and stresses of the day can quickly disappear. With teaching, I am always impressed with those teachers who provide that warm, positive greeting as students enter the room. Granted, I’m not sure we want teachers jumping playfully and flopping on the ground; however, students do respond positively when teachers take those brief moments before class to welcome them and to show excitement and gratitude that the student has come to class.

Second, dogs express an unconditional love and support of you, no matter the situation. I have met many teachers who have this unconditional love and support for students, the belief that all students can succeed. There may be times when students will let us down, possibly with the choices they make or with the effort they give. But, effective teachers have an unconditional and unwavering belief that all students can learn and achieve.

As an educator, I’ve grown to love cats too – but in this case, I am referring to Classroom Assessment Techniques (CATs). I was first introduced to CATs over a decade ago while working at the Maricopa Center for Learning and Instruction (MCLI). My supervisor at the time handed me a copy of the Angelo and Cross foundational text, Classroom Assessment Techniques: A Handbook for College Teachers. To this day, I still refer to this book as I interact with faculty during classroom observations. CATs are quick and easy informal strategies to measure student learning in the classroom. Some instructors at GCC have completely embraced CATs, using minute-papers or the muddiest point strategy to gauge how well students learned the content and objectives for the class session. My personal favorite CAT is the ticket-out. With this strategy, instructors provide students with a brief question or two at the end of class. Students must write their answers on a note-card or slip of paper and that is their ticket out from class. These informal techniques allow instructors to get a sense of what students learned from the class and what students may have missed, with the ultimate goal of providing additional instruction the next time the class meets or to even provide additional content in Canvas to fill in any gaps. These low-stakes, quick assessment strategies are an effective way to measure student learning and an excellent teaching strategy to help students to achieve.

I am a dog guy – there is no question about that. However, there is definitely a special place for CATs in my teaching heart as well.

 

Effective Assessment and Reflection

I think effective assessment has a lot to do with reflection. Writing instructors often ask students to reflect on work they’ve completed. This helps spark insight about existing strengths and which areas could benefit from further development. It also allows them to consider the amount of time and effort they put into the assignment and how that shaped the outcome. I think assessment can and should be the same way for instructors. Assessments, for me anyway, are tools that are fluid and often change between semesters. I think many of us are always trying to perfect each assignment, so it tests the competencies we want students to demonstrate and also engages them enough to facilitate effective writing. If my students fare poorly on some aspect of an assessment, whether that’s an in class activity, or something more elaborate like a 750 word paper, I typically ask myself what I might have done differently. Should I have covered the concepts invoked in that problematic section more thoroughly in class? Are the assignment directions for that part of the paper confusing? Or is this just a particularly challenging concept that students need multiple exposures to across several assignments before they really perfect it? Sometimes I might even make changes to the assessment itself. Is the thing I’m looking for mission critical in terms of the competencies students need to demonstrate or is it something I do just because I’ve been doing it that way a long time?

I try to ensure my approach to student learning is evidence based. Certainly that means I like to try to keep up with current literature on teaching and learning, but that also means looking at the evidence from the students themselves. My experience with GCC students has been that, on the whole, they are quite hardworking and, if you give them a sufficiently stimulating assignment prompt, they’ll put in substantial work despite the many outside pressures they and most other community college students face (work, family obligations, etc.) — I once had a student that rollerbladed to my 10 am class because his car broke down, but that’s another story. Given that experience, I tend to look for alterations and improvements I can make if a substantial portion of students are struggling with a certain aspect of an assessment. Please don’t read this as “dumbing things down” or “making the assignments easier.” That would be an awfully reductive interpretation of what I’m trying to get at here. The competencies are the competencies. The rubric is the rubric. Effective teaching is about finding ways for as many students as possible to find success in demonstrating those competencies so they can find success here at GCC and beyond.

 

Let’s Get Critical

Last year I went in depth on one of the most overlooked assessment tools, rubrics. My feelings and thoughts on that important tool have not changed, but rather than repeat myself this year I want to talk about a different type of assessment. Specifically, I want to talk about assessing the critical thinking skills of students.

The specificLightbulb critical thinking ability I have been working on is the ability to analyze and attack a strongly held personal belief. The idea being that a good critical thinker should be able to understand opposing viewpoints.

I have done this through a series of writing assignments in various forms over many semesters. The most recent iteration is a “Devil’s Advocate” series of assignments where students are required to write a defense of a personal belief one week and write a defense of the opposing viewpoint the next.

The reason I always do this type of assignment is because of my core belief that critical thinking is a skill that will be useful to students no matter their future profession. It is also a skill that is sometimes overlooked in the test-driven performance-centric world of secondary level education.

Think Outside the Box

A word of warning, these types of assignments do have issues that will arise and need to be planned for ahead of time. There inevitably is always a group of students who absolutely detest this type of work. I had a student go as far as claim I was trying to “force my liberal beliefs” on them through my position of power. That complaint didn’t go anywhere, but it is an example as to how difficult this can be for some individuals. It also is very insightful as to the ability of students to critically think.

I have only recently started to tabulate the data in any real form, and the number of students that are able to successfully “think from the opposing viewpoint” has varied over semesters. The one constant I have noticed in the last decade is that there is always a significant portion of the class (30-50%) that must change their topic or take a sarcastic tone to complete the task, which shows a lack of developed critical thinking ability.

No matter what the final numbers and assessment show, the need to reinforce critical thinking skills at the college level is, well, critical. There are elements of critical thinking that can be taught in any discipline or class, and if every course made an effort to include tasks that require critical thinking skills, the end result would be students who will be better prepared to handle the unknown, problem solve, and appreciate (or at least respect) the “other”.

Education prepares the workforce of the future, politicians, nurses, teachers, managers, everyone that has a job that requires more than a High School diploma. In a world of percentages, having the majority with a solid foundation of critical thinking skills will result in a better world for everyone.

Graduation Photo

If you have assignments that assess critical thinking, or have thoughts about critical thinking in the classroom, I would love to hear about it. Feel free to comment below or send me an e-mail!

 

Frequent Formative Assessment

     At the end of class one day, one of my students uttered, “I learned that I didn’t know what I thought I knew.” It was such a perfect statement that I actually scrawled it down on some scrap paper, so I wouldn’t forget it. The statement came at the conclusion of a round of Kahoot (thank you, Caryn) on APA formatted in-text citations. Students had already been assigned some readings and a SoftChalk lesson on APA.

     The game was low stakes, and they played on teams–the same teams they are in all semester. They were currently working in the final days before their paper was due, so the game was supposed to be review with a few special circumstances thrown in that I knew would come up in their papers–things like a source within a source, the ampersand in parentheses for two authors, the title of an article with no author.

     The more frequent formative assessment I’ve been adding in to my courses with intention comes on the heels of having read Make it Stick: The Science Behind Successful Learning by Peter Brown, Henry L. Roediger III, and Mark A. McDaniel. One of the points made in the book is that frequent, low stakes assessment lets students check what they know and don’t know prior to a summative assessment. It gives them insight into their learning. This review that I used did exactly that for almost all the students. The student who spoke the phrase which could have been quoted from the book recognized that he thought he knew more than he did. He now had a starting point to work from while editing his paper. He got a chance to make corrections to his knowledge and application prior to the summative assessment.

     I have the sentence taped to my computer now. I want to remember the value that more frequent assessment has for my students. I’m using it as reminder to give my students more opportunities to check their own understanding prior to finding out they “didn’t know what [they] thought they knew” on a more significant test or essay.