Happy New Year, data stalkers....
At the beginning of the year, we like to do our planning. It's part of a larger cycle of Plan, Budget, Do, Measure, Evaluate...repeat. We do this so we have a record of our progress over time. It helps us keep from re-inventing the wheel and makes it easier to explain to our outside stakeholders what we do and how we do it. It also drives our budgeting...that alone is worth its own essay at some point!
What's a plan? We know that conditions have a way of changing on us when we least expect it, so plans are flexible layouts of what we'd like to do, and sometimes how we think we will get there. Yes, we need to revise them sometimes. But for those of us with short attention spans and busy jobs and lives, a plan can help keep us on track toward the achievement of at least some of our goals for the year.
The first law of planning? Keep it simple!!
Start very small, very specific, and have regular checkups or benchmarks you can use to see how you are making progress. Especially the first year, give yourself a very short list of goals that you are pretty sure you can achieve. Why? Because you are practicing how to use these tools. You can get fancy once you have confidence from experience.
Parts of a Plan:
1. Mission/Vision or Values
2. Goals
3. Objectives
4. Proposed Measures
5. Timeline
1. Missions are the overall statement of what your organization or unit wants to accomplish. For example, you might have the following mission: "Orvileille will be the best maker of kettle chips the world has ever known." You might focus more on your constituents for a service organization: "Our clients will become fully proficient in atomic fusion for the home."
Visions are broad-spectrum pictures of where you want to go: "We envision a home fusion reactor in every home and office providing reliable low cost energy to our delighted customers."
Values...do I really have to tell you?? One suggestion, though: choose values you can live by, and live by them.
2. Goals are theoretical statements about what you are going to accomplish. They are specific yet vague. Save the specific measures for your objectives...
Here is a goal: We will significantly increase the market share of FusionReactors, Inc.
3. Objectives are more specific about how you will get to that goal: "Over the next three years we will use marketing on internet sites and on billboards to drive an annual 3% minimum increase in market share for our reactors." Specific, and achievable. And look how neatly that will drive your marketing budget discussion!
4. How will you tell you are successful at meeting your objectives and goals? Measurement. It's not so difficult. I would presumably have my market share data from prior years, or be starting from zero (nowhere to go but upward!). Let's call that a pretest measure or a benchmark. After I have put my marketing in place, I would measure the market share at the end of the annual cycle, and compare back. Hopefully, my graph trends upward...but if it doesn't you have an opportunity to try something else in the next cycle by amending your plan.
5. Many people forget the timeline section of their plan. It's helpful to have this section detailed out as much as possible, although almost always you will find yourself adjusting it mid-process. It is a nice record of how you organized your implementation efforts based on your planning.
Does all of this rock and roll, or are there parts you'd like more detail about?
See also this really nice entry on a related topic: http://www.alex-reid.net/2013/01/university-futures.html
Monday, January 14, 2013
Monday, December 17, 2012
Assessment Overload!
Is there such a thing as too much assessment? Yes!
There is also the idea of appropriate or mindful assessment. You don't want this stuff to take over to the point that you are having to do more, more, more assessment and not get your primary work done.
So part of what we do is evaluate the effectiveness of our assessment designs. Yes, it's recursive, but useful.
Once the cycle of assessment is done and you've had time to review those results, you also want to get your gang to reflect upon the process itself. What went well? Badly? Not at all? Where did you get good clear information to support your decisions, and where did you wish you had asked different questions to begin with?
You have time now to devise clever fixes or improvements for the next cycle, and you will get credit for all the diagnosing and building up. Be sure to keep your meeting minutes and notes about how you decided to amend your assessment processes, and why. Those are evidence of good practice.
Likewise, if you are assessing some things and then NEVER using the results for decision making...uh....perhaps you could let that go for now, and only bring it back later if you find a practical need for that information. Again, document the basis for the decision...
Over time, as your needs change, you can focus and refine your measurements so that they are manageable and targeted to the specific goals and outcomes that matter most to your team and unit.
In the new year, I'll talk about how this can integrate with your strategic planning...happy holidays to everyone!
There is also the idea of appropriate or mindful assessment. You don't want this stuff to take over to the point that you are having to do more, more, more assessment and not get your primary work done.
So part of what we do is evaluate the effectiveness of our assessment designs. Yes, it's recursive, but useful.
Once the cycle of assessment is done and you've had time to review those results, you also want to get your gang to reflect upon the process itself. What went well? Badly? Not at all? Where did you get good clear information to support your decisions, and where did you wish you had asked different questions to begin with?
You have time now to devise clever fixes or improvements for the next cycle, and you will get credit for all the diagnosing and building up. Be sure to keep your meeting minutes and notes about how you decided to amend your assessment processes, and why. Those are evidence of good practice.
Likewise, if you are assessing some things and then NEVER using the results for decision making...uh....perhaps you could let that go for now, and only bring it back later if you find a practical need for that information. Again, document the basis for the decision...
Over time, as your needs change, you can focus and refine your measurements so that they are manageable and targeted to the specific goals and outcomes that matter most to your team and unit.
In the new year, I'll talk about how this can integrate with your strategic planning...happy holidays to everyone!
Tuesday, November 27, 2012
OUTCOMES (and other strange critters...)
Many moons ago, when I went to graduate school the second time (I took the wandering path), we all had to take a teaching seminar. I learned a lot. Still, when I got into actually teaching, I discovered I knew very little about academic outcomes measurement, assessment, and process evaluation. I had to study up fast.
I was convinced by what I read and then practiced. Even though it appears at first glance that all this assessment stuff is going to add to your workload, done right, it actually makes it easier in the long run. That was a good surprise. Call me a convert.
Often the first sign of building good assessment into our practice is being asked to provide measurable outcomes for our classes, and ultimately our programs.
Outcomes?? What are those?
Outcomes are brief statements about what students will be able to do as a result of your class or program.
We approached crafting ours by asking "what knowledge base and skill set would ideally be expected of a graduate from our class or program?" Also, "how will we tell if our students have acquired the knowledge and skills we expect?"
Later, after you've had a chance to get more proficient, you can get fancy, but you probably want to start with a short list of 3-6 measurable outcomes that are central to your topic area or your field.
You can set yourself up for success by using measurable action words in your outcomes. For example, if I have an outcome that says I want you to "be able to understand and appreciate aesthetic value" I'm going to have a devilish time measuring that convincingly. However, if I use an outcome that says that students will "be able to identify aesthetic and artistic styles and describe their historic context, including key elements and famous practitioners" then I can more easily link that outcome to specific questions or assignments in my instructional design.
What are some other good examples of murky outcomes turned into measurable ones? Sometimes we had to write badly stated ones first, and then fix them...a good brainstorming exercise.
This stuff is marvelous, because when you gather information about these outcomes later you have the diagnostics to test and improve your own processes, and to support a claim of what you are doing well.
One tool to help you is a set of action words that are associated with Bloom's Taxonomy, a wonderful teaching resource that looks at different levels of learning and measurement. Here is one source for background on those materials, and there are probably dozens of others:
http://www.clemson.edu/assessment/assessmentpractices/referencematerials/documents/Blooms%20Taxonomy%20Action%20Verbs.pdf
Does setting specific measurable outcomes limit what you will teach and what students will learn? No. It gives you a way to present a framework that your students can use as a clear foundation for their learning in your class and beyond.
In a future post I'll tackle how to match measures to outcomes, and other adventures. You can also suggest topics, and I'll see what I can do to explain them clearly.
Many moons ago, when I went to graduate school the second time (I took the wandering path), we all had to take a teaching seminar. I learned a lot. Still, when I got into actually teaching, I discovered I knew very little about academic outcomes measurement, assessment, and process evaluation. I had to study up fast.
I was convinced by what I read and then practiced. Even though it appears at first glance that all this assessment stuff is going to add to your workload, done right, it actually makes it easier in the long run. That was a good surprise. Call me a convert.
Often the first sign of building good assessment into our practice is being asked to provide measurable outcomes for our classes, and ultimately our programs.
Outcomes?? What are those?
Outcomes are brief statements about what students will be able to do as a result of your class or program.
We approached crafting ours by asking "what knowledge base and skill set would ideally be expected of a graduate from our class or program?" Also, "how will we tell if our students have acquired the knowledge and skills we expect?"
Later, after you've had a chance to get more proficient, you can get fancy, but you probably want to start with a short list of 3-6 measurable outcomes that are central to your topic area or your field.
You can set yourself up for success by using measurable action words in your outcomes. For example, if I have an outcome that says I want you to "be able to understand and appreciate aesthetic value" I'm going to have a devilish time measuring that convincingly. However, if I use an outcome that says that students will "be able to identify aesthetic and artistic styles and describe their historic context, including key elements and famous practitioners" then I can more easily link that outcome to specific questions or assignments in my instructional design.
What are some other good examples of murky outcomes turned into measurable ones? Sometimes we had to write badly stated ones first, and then fix them...a good brainstorming exercise.
This stuff is marvelous, because when you gather information about these outcomes later you have the diagnostics to test and improve your own processes, and to support a claim of what you are doing well.
One tool to help you is a set of action words that are associated with Bloom's Taxonomy, a wonderful teaching resource that looks at different levels of learning and measurement. Here is one source for background on those materials, and there are probably dozens of others:
http://www.clemson.edu/assessment/assessmentpractices/referencematerials/documents/Blooms%20Taxonomy%20Action%20Verbs.pdf
Does setting specific measurable outcomes limit what you will teach and what students will learn? No. It gives you a way to present a framework that your students can use as a clear foundation for their learning in your class and beyond.
In a future post I'll tackle how to match measures to outcomes, and other adventures. You can also suggest topics, and I'll see what I can do to explain them clearly.
Thursday, November 15, 2012
Welcome readers, commenters, and contributors!
I don't know if the world really needed another blog about data, but when I made a list of all the things I'd like to share with my colleagues, it looks like I have enough material to either write a book or share over time as blog entries. I think, given my attention span, that the blogging might do the trick.
I hope you will find yourselves comfortable to join in.
My day job is in institutional effectiveness at a two-year college. I have the best job! Okay, maybe you do, too. I'm no expert by any means, so I'll provide references for some things and seek your collective wisdom on others.
I'll spend this month getting the page set up, and shortly I'll post something more topical so we can kick off some discussion.
Regards,
Leslie
I don't know if the world really needed another blog about data, but when I made a list of all the things I'd like to share with my colleagues, it looks like I have enough material to either write a book or share over time as blog entries. I think, given my attention span, that the blogging might do the trick.
I hope you will find yourselves comfortable to join in.
My day job is in institutional effectiveness at a two-year college. I have the best job! Okay, maybe you do, too. I'm no expert by any means, so I'll provide references for some things and seek your collective wisdom on others.
I'll spend this month getting the page set up, and shortly I'll post something more topical so we can kick off some discussion.
Regards,
Leslie
Subscribe to:
Posts (Atom)