Previous Section | Chapter 2 Table of Contents | Next Section
Judith A. Arter and
Vicki Spandel, Northwest Regional Educational Laboratory
The term portfolio has become a popular buzz word. Unfortunately, it as not always clear exactly what is meant or implied by the term, especially when used in the context of portfolio assessment. This training module is intended to clarify the notion of portfolio assessment and help users design such assessments in a thoughtful manner. We begin with a discussion of the rationale for assessment alternatives and then discuss portfolio definitions, characteristics, pitfalls, and design considerations.
Educators and critics are currently reciting a litany of problems concerning the use of multiple-choice and other structured format tests for assessing many important student outcomes. this has been accompanied by an explosion of activity searching for assessment alternatives (French, 1991; Jongsma, 1989; McLean, 1990; Mills, 1989; Myers, 1987; Stiggins, 1991; Valencia, 1990; and Wolf, 1988) that will accomplish the following:
1. Capture a richer array of what students know and can do than is possible with multiple-choice test. Current goals for students go beyond knowledge of facts and include such things as problem solving, critical thinking, lifelong learning of new information, and thinking independently. Goals also include dispositions such as persistence, flexibility, motivation, and self-confidence.
2. Portray the processes by which students produce work. It is important, for example, that students utilize efficient strategies for solving problems as well as getting the right answer. It is also important for students to be able to do such things as monitoring their own learning so that they can adjust what they do when they perceive they are not understanding.
3. Make our assessments align with what we consider important outcomes for students in order to communicate the right message to students and others about what we value. For example, if we emphasize higher order thinking in instruction, but only test knowledge because testing thinking is difficult, students figure out pretty fast what is really valued.
4. Have realistic contexts for the production of work, so that we can examine what students know and can do in real-life situations.
5. Provide continuous and ongoing information on how students are doing in order to chronicle development, give effective feedback to students, and encourage students to observe their own growth.
6. Integrate assessment with instructions in a way consistent with both current theories of instruction and goals for students. Specifically, we want to encourage active student engagement in learning, and student responsibility for and control of learning. We also want to develop assessment techniques that, in their use, improve achievement and not just monitor it.
Using portfolios of student work for assessment, already an instructional tool in many places, is seen as one potential way to accomplish these things. But, using portfolios will only have these desired effects if we plan them carefully.
DEFINITION OF A PORTFOLIO
In consideration of the above values, we offer a definition that is adapted from that developed and refined over a period of 2 years by a consortium of educators under the auspices of NWEA (Northwest Evaluation Association). (For more discussion, see Arter & Paulson, 1991; Paulson et al., 1990; NWEA, 1990.) Our adaptation defines a student portfolio as a purposeful collection of student work that tells the story of the student's efforts, progress, or achievement in (a) given area(s). This collection must include student participation in selection of portfolio content; the guidelines for selection; the criteria for judging merit; and evidence of student self-reflection. This definition supports the view that assessment should be continuous, capture a rich array of what students know and can do, involve realistic contexts, communicate to students and others what is valued, portray the processes by which work is accomplished, and be integrated with instruction.
Purposeful
Without purpose,
a portfolio is just a folder of student work. Different purposes could
result in different portfolios. For example, if the student is to be evaluated
on the basis of the work in the portfolio (e.g., for admission to college),
then he or she would probably choose the final version of his or her best
work. If the portfolio is to be used to see how students go about doing
a project, a complete
record of all activities, drafts, revisions, etc., might be kept. Sometimes
the purpose for doing a portfolio is to celebrate what has been accomplished.
This is a keepsake purpose, and might include personal favorites. On top
of all this, add the purposes of large-scale assessment, which may require
more standardized samples of work. Because of the potential differences
in content and approach, it is essential that users have a clear idea
of the purpose of the portfolio.
Student Self-Reflection
Self-reflection
is seen as necessary because of the purposeful nature of the selection
of work or other displays for the portfolio. To satisfy a purpose, there
needs to be a rationale for the selection of the items to be included;
this requires an analysis of the work and what it demonstrates. Recording
this self-reflection in the form of a "metacognitive" letter
or oral report not only documents this type of student performance, but
also encourages it. Thus, self-reflection is one thing that makes
a portfolio instructional.
Criteria for Judging
Merit
In some teachers'
minds, portfolios and assessment do not mix well, perhaps because assessment
seems reductive. Doesn't it seem ironic, they will argue, to go to all
the trouble of expanding and humanizing our view of student performance
via portfolios only to shrink that view back down via some rating scale?
No one wants portfolios used to trivialize student performance, but it
is easy to take an overly simplistic view of what assessment is and does.
When the decision is made to include or exclude some item from the portfolio, that decision is based on criteria of one kind or another. The question is, "Are the criteria fully and carefully defined and open to all or are they nebulous and guarded so that students must guess what is being sought?" Event if students select their own pieces for inclusion, they are probably using some sort of internal criteria, however intuitive or fuzzily defined. Why not put those criteria in writing and share them as a way of identifying and discussing what is most valued by students and teachers alike?
In fact, criteria give us a schema for thinking about student performance. In the absence of all criteria, how do we know what sort of work a student has accomplished through the year? How does the student know whether to be satisfied, ecstatic, or dismayed? How does the student or the teacher know what goals to set for next time? And how do various audiences know what to make of the performances as a whole?
There are potential benefits for clear criteria. For one thing, those who set the criteria must think very carefully about what it is they value in strong performance, and this helps clarify instructional goals and expectation. Also, to the extent that criteria are shared, students are made part of the evaluation and receive the power that goes with that specialized knowledgepower to recognize strong performance, and power to use criteria to change and improve performance. Finally, clear criteria are a means for us to judge performance.
Guidelines for Selection
Guidelines for
selection provide direction on what to place in the portfolio. Such guidelines
can represent anything from an extremely structured procedure (e.g., everyone
will include an essay comparing the characters in Romeo and Juliet
to those in Great Expectations) to a completely unstructured
procedure (students can choose whatever they want for their portfolios).
A more moderate position would be to specify categories of entries (e.g.,
everyone will select one research report, one multimedia project, one
"best" piece, one paper with all rough drafts, etc.) with students
free to select work for each category.
Student Participation
in Selection
Although it is
possible for someone else (for example, a teacher) to assemble a student's
work into a portfolio, the true instructional value and power of doing
portfolios comes when students use criteria and self-reflection to make
decisions about what they want to show about themselves and why. This
implies self-selection of portfolio content.
Portfolios as Assessment
Devices
The definition
of a portfolio presented above implies assessment. Students cannot assemble
a portfolio without using clearly defined targets (criteria) in a systematic
way to paint a picture of their own efforts, growth, and achievement.
This is the essence of assessment. Thus, portfolios used
in this manner provide an example of how assessment can be used to improve
achievement and not merely monitor achievement.
Please note that our definition of a portfolio does not preclude the use of portfolios for monitoring achievement, as in large-scale assessment. It does suggest, however, that any use of portfolios for large-scale assessment not interfere with their primary use for instruction. In fact, portfolios contain several features that might make them very attractive for large-scale assessment. For example, portfolios usually contain more than one sample of student work, thus providing a more complete picture of a student's achievement than the typical one-shot essay or speech in an end-of-term performance assessment. Also, since portfolios are generated during the process of instruction, their content might represent work produced in a more realistic context.
Integration of Assessment
and Instruction
The process of
assembling portfolios of student work has the potential of both encouraging
and document in critical thinking, problem solving, and independent thinking.
Portfolios include actual work samples and can be designed to include
drafts; therefore, not only can they contain samples of work that reflect
real tasks, but they can be used to look at the processes students go
through when doing these tasks.
Portfolio as a Story
A useful way
to think about a portfolio is as a story-telling device (Arter & Paulson,
1991; Paulson & Paulson, 1991). The purpose of the portfolio is to
make sense of student work, to communicate about student work, and to
make sense of the work in the portfolio in terms of a larger context.
The student work included in the portfolio is that which best tells the
story one wants to tell. This requires justification and a rationale for
the conclusions drawn, which again imply self-reflection, self-selection,
and criteria.
Composite Portfolios
A composite portfolio
is parallel to an individual student portfolio except that it tells the
story for a group. In its simplest terms, a composite portfolio contains
more than one student's work. A composite portfolio might be one way
to aggregate information for demonstrating what impact a school or program
is having on students in general, to demonstrate what is being taught,
etc. The topic of composites has been addressed by several NWEA work groups,
which developed this definition: a composite portfolio is a purposeful
collection of student work that tells the story of a group's efforts,
progress, or achievement. This collection must include criteria for selection,
criteria for judging merit, and evidence of self-reflection (Arter &
Paulson, 1991). Because of the nature of composites, requirements for
their compilation are a little different than for individual student portfolios.
This training module does not specifically address composites. For more
information, see Arter & Paulson (1991).
POTENTIAL PROBLEMS WHEN USING PORTFOLIOS AS ASSESSMENT DEVICES
Just because the use of portfolios can have the instructional and assessment advantages listed above, it does not mean that use of portfolios automatically will have these effects. In actuality, if not done well and interpreted properly, portfolios can mislead as much as, if not more than, the results of fixed-choice tests. For example, consider a situation in which the task was to evaluate how well an instructional area was being taught. A committee of teachers gathered samples of instructional materials and student products to demonstrate what teachers teach and students learn. The resulting collection was very impressive. In fact, it was so impressive that one might not think to ask critical questions such as "Do all teachers do this?" and "Do all students learn this much?" In other words, since the content of the portfolio looked so real, stakeholders might have been mislead about what story the portfolio actually told. (Fortunately, the staff members doing the evaluation in this example had the commitment to ensure that results were interpreted properly.)
It is essential that portfolio systems be designed carefully to ensure that stakeholders draw accurate conclusions about what the portfolios show. Portfolios, as performance assessment devices, can run into all the following problems: the work in the portfolio may not really be representative of what the student knows and can do, the criteria used to critique the product may not reflect the most relevant or useful dimensions of the task, the work that a student puts in the portfolio may make the viewer wonder what is authentic about it, there may be aspects of the portfolio process that make a student unable to really demonstrate what he or she knows or can do, and the conclusions drawn from the portfolio can be heavily influenced by the person doing the evaluation (Arter, 1989, 1991; Rothman, 1990; Valencia, 1989).
Representativeness
The example above
about "Do all teachers teach this?" illustrates the issue of
representativeness. We have to be sure that what is included in the portfolio
provides a complete picture of the phenomenon we are trying to portray.
For example, we can't make statements about students' ability to communicate
in general if all we've collected are the best lessons from the best teachers.
We should encourage a variety of tasks and formats for possible inclusions
in a portfolio so that students have full opportunity to demonstrate performance
(Valencia, 1989).
Criteria
We have already
made the case for having clear criteria. However, not just any criteria
will do; they need to be good criteria. For example, consider the
Informal Writing Inventory (Giordano, 1986) where the writing sample
is assessed by counting the number of errors in conventions. Is this an
adequate measure of being able to write? Good criteria represent a conception
of what is valued in an expert performance; to develop good criteria,
one needs a great deal of content expertise.
Authentic Work and Extraneous
Response Requirements
One reason cited
for assembling portfolios of student work is that they provide a more
authentic view of what students know and can do. However, this authenticity
depends on several factors:
1. What is meant by authentic? The content of a portfolio will mirror the emphasis in the curriculum and classroom. For example, if the curriculum emphasizes phonics and teachers concentrate on phonics, then the samples of work for the portfolio are likely to reflect phonics. Is this authentic? Authentic to what? An authentic reflection of classroom work or an authentic representation of ability to read on real life? One must come to grips with this issue
before even beginning to discuss authentic tasks.2. The work assigned to students, and therefore available to be selected for the portfolio, must match the target. For example, if students only do computational work-sheets in math, work samples might not be available that show math problem-solving ability. Or, what if students usually do prompted writing in which topics are assigned by the teacher? Do these tasks really represent the target of being able to write in daily life? Would students be motivated to perform on this task the same way as they would for a writing purpose of their own design?
3. The portfolio system must not be viewed as an add-on to the "real" instruction taking place in the classroom. If teachers view the portfolio as not making instruction faster, easier, or better, it is hard to predict what the content might be like.
4. Sometimes students are unable to demonstrate what they really can do because of some part of the task that requires skills that really do not have any thing to do with the abilities being examined. Take as an example an "exhibition" in which a group of high school students demonstrated their ability to conduct a symposium discussion. This discussion required reading two very difficult articles and getting words in edgewise in a fairly big group. Would students be at a disadvantage if they are shy or would be able to discuss the issues if the reading were a little simpler? Should personality and reading ability, extraneous to the ability being measured (ability to think), affect performance? These extraneous response requirements can affect the authenticity of the products selected for the portfolio. The point is that portfolios do not automatically imply authenticity.
Differences in Interpretations
Finally, the
perception of the significance of a portfolio can change depending on
who is doing the analysis. For example, consider a developmental portfolio
project for kindergarten students, in which teachers were to select student
work samples that demonstrated growth in developmental stages in writing,
reading, and spelling. What happens if there is no systematic training
in how to do this? Different teachers could come to different conclusions
about the growth of students.
Conclusion
Portfolios have
the potential to tell detailed stories about a variety
of student outcomes difficult to tell using other methods. However, we
all have the obligation to make the story reflect reality and not use
the format to distort reality. Thus, we must take in to account the technical
issues described above and admit what stories can be told from a given
portfolio and what stories are not possible to tell. It is also important
to note that these issues are as important when teachers are using portfolios
for classroom purposes as when portfolios are used for some kind of larger
scale assessment.
SUMMARY OF CURRENT PORTFOLIO EFFORTS
The bibliography at the end of this article lists references to many portfolio projects. In general, the existing portfolio systems appear to have the following characteristics:
1. They cover a full range of structures from specific items being required for all portfolios (e.g., a particular attitude survey or list of books read) to totally open-ended systems in which students can choose anything they want for their portfolios.
2. Purposes for the portfolio systems vary broadly. The main purposes appear to be instructional (either for monitoring of student progress or as an instructional tool in itself) and communication with parents. Other purposes have include college admission, minimum-competency testing, a celebration of what has been accomplished, passing on information to the next teacher, grading, high school credit, and program evaluation.
3. In terms of content, most portfolio systems are currently in the area of communicationwriting or integrated language arts (writing, reading, speaking, and listening). There are some examples of mathematics systems (Equals, 1989; Mumme, 1990) and some discussion of portfolios in other areas such as science (Collins, 1990).
4. Current systems appear to have either the teacher or student as the main stakeholder and parents as another important audience. Other audiences have included school board members, district evaluation staff, state assessment staff, and the general public.
5. Finally, in terms of criteria, some systems describe criteria for assessing the individual entries in the portfolio (such as analytical trait models for writing and math problem solving), but fewer discuss criteria for assessing the portfolio as a whole (e.g., Elliott & Harriman, 1989, and Vermontsee Hewitt, 1989, and Vermont State Department of Education, 1990) or for assessing the student self-reflection in the portfolio.
PORTFOLIO DESIGN QUESTIONS
To avoid the pitfalls discussed above, the remainder of this training module will consider the various design issues that should be addressed when setting up a portfolio system (Arter & Paulson, 1991; Collins, 1990; Macintosh, 1989; Murphy & Smith, 1990; NWEA, 1989; Roettger & Szymezuk, 1990; Vavrus, 1990).
Issue 1: Design Responsibilities
Who should design
the portfolio system? Can a portfolio system be mandated, or even designed,
from the top down? Or, if it is to work at all, should it be generated
by and for the people who will be responsible for assembling it?
There is a considerable amount of concern right now about preempting portfolios for use primarily in large-scale assessment (NCTE, 1991). Putting together innovative and representative portfoliosthe sort that really tell who students are as artists, writers, readers, or mathematiciansdemands a serious commitment of time and energy. Therefore, if portfolios are mandated from on high, they are likely to be seen as an enormous imposition on both students' and teachers' time, and content is not likely to be valid. But if the driving force behind the project comes form the students and teachers themselves, those same portfolios may be seen as an innovative way to showcase, preserve, or celebrate what's already taking place in the classroom. Thus, a grass-roots effort not only has the potential to improve instruction, but also to produce the rich and valid sources of information needed for better large-scale assessment.
Although teachers are correct in feeling protective about the use of portfolios, they are not the only stakeholders in their use; district staff and the public have a legitimate right to see how students are growing. If teachers do not take the initiative in making better achievement information available for large-scale assessment, then someone also will design the systems and tell teachers what to do. An additional argument for having a more centralized portfolio development project is that there are advantages in having common conceptions across grade levels and schools as to what acceptable performance looks like (i.e., standardization of criteria). Everyone benefits form discussions of what performance criteria should bethe students because they know that the same targets will be described in the same way as they progress across grades, the teachers because they have a clearer view of learning targets, and the district/ state because common criteria allow aggregation of information.
Therefore, the question is not really whether it is better to design a portfolio system from the bottom up or the top down. In actuality, it is in the best interests of students, teachers, and district/ state staff to actively work together to (a) preserve the instructional power of portfolios, and (b) see how the potentially rich source of information from portfolios can be summarized at higher levels to show others what students are learning.
Issue 2: Purposes
What is the purpose
of the portfolio? Who are the audiences? Can portfolios be used for more
than one purpose, for example, classroom instruction and large-scale assessment?
Purpose is all-important. As noted above, it affects everything else, including the design of the portfolio, the content, the link to instruction, and even (on some level) how students feel about creating portfolios. Several different purposes are possibleall of them valid. What is important is that the purpose be clearly defined at the outset so that other important decisions will be appropriate.
Can portfolios be used for
both classroom instruction/ assessment and large-scale assessment? It
might seem on first blush that criteria for defining what goes into the
portfolio would need to be highly restrictive for large-scale assessment,
but this is not necessarily so. Portfolios could be standardized at various
levels. Suppose, for instance, that a math portfolio were to be the basis
for large-scale assessment. It might be desirable for students to include
one example of strong performance on a timed test, one example of a practical
application, one example of creative problem solving, and a sample project
linking math to
another content areasay, science. In this example, the portfolio is standardized
as to the types of items to be included and the criteria for assessing
them, but the specific samples of performance chosen for inclusion could
be as creative and different as the students themselves. Thus, it may
very well be possible to impose enough standardization to ensure the equity
and comparability needed for large-scale assessment, and still give enough
leeway to promote the flexibility needed in the classroom.
Issue 3: The Link to
Instruction
What is the relationship
between curriculum, instruction, and portfolios? How will students reflect
on their work? There are several natural links to instruction. First,
we've discussed how the process of assembling a portfolio is a great instructional
exercise in using criteria, taking audience into account, self-reflection,
etc. Second, the process of developing criteria is an instructional activity
because it forces us to think about and articulate what we value.
Third, if we have criteria (targets) for judging performance, we must be able to show where, during instruction, we taught students what they need to know to hit our targets. For example, if we are going to have students self-reflect, we have to help students to develop the skills they need for doing this meaningfully. If their thoughts and comments are to go beyond "I think I did pretty well" or " I think I have more to learn," they need some experience in developing and working with sound criteria that can help them spot strengths and weaknesses in their own self-reflection. They also need to see samples of good self-reflection so that they know what it looks like, and can begin to look beneath the surface to the behaviors and practices that affect such performance.
Fourth, the self-reflection involved in reviewing work to produce a portfolio provides information to students about what they've learned, how they've grown, and what their next target is. Finally, the review of portfolio content and the ongoing conferencing surrounding the production of portfolios provide a great deal of information to teachers for instructional planning.
Considerations
Even though there are some natural links to instruction, there are still
some issues to consider concerning
how portfolios will fit into the curriculum:
1. Portfolios should reflect attention to the same broad curricular goals that drive everyday instruction.
2. The criteria used to evaluate performance of projects or products included in the portfolio should be the same as those used every day in the classroom.
3. The definition of a portfolio provided in this module implies certain values in instruction that need to be congruent with local values, such as active learning and students' taking responsibility for their own learning.
One Example of Use in Instruction.
There is also the consideration of how, functionally, portfolios will
be integrated into instruction. Here is one example: Criteria are developed
that articulate what is valued in a student performance (students can
be involved in this process). For example, in writing you might develop
an analytical trait assessment model that defines what good performance
looks like in the areas of content, organization, voice, work choice,
sentence fluency, and conventions. Criteria for self-reflection might
be those listed under Issue #5, below. These criteria are then systematically
reinforced with students by defining traits, showing examples of good
and poor work on each trait, giving practice on each trait, and having
students critique their own (and other students') work and self-reflections.
Students also use these criteria and experiences to select entries for
a portfolio and justify their choices. Students then use the portfolio
in various ways: deciding periodically whether to replace one entry with
another that is a better example of some accomplishment, examining the
content to look at progress over time, sharing with parents, etc.
Examples of Questions to
Prompt Student Self-Reflection.
Another common question is how to prompt students to self-reflect.
Many people have come up with questions to prompt them to reflect on their
work (Thompson, 1985; Lewis, 1989; EQUALS, 1989; Rief, 1990; Howard, 1990;
Kilmer, 1990; Eresh, 1990). Some of these are the following:
- Describe the process you
went through to complete this assignment. Include where you got ideas,
how you explored the subject, what problems you encountered, and what
revision strategies you used.
- List the points made by
the group review of you work. Describe your response to each pointdid
you agree or disagree? Why? What did you do as the result of their feedback?-What
makes your most effective piece different form your least effective
piece?
- How does this activity relate
to what you have learned before?
- What are the strengths of your work? What still makes you uneasy?
Issue 4: Content
What subject
area(s) will be covered by the portfolio? Will there be any guidelines
for the types of items? When will work be chosen for inclusion? Who makes
these decisions? How will you check that tasks are realistic?
After deciding on the general subject area, you need to decide the level at which you will specify the types of things that will go in the portfolio. The issues in deciding on content are the following: (1) what degree of structure or standardization do you want to impose in order to (2) ensure that you get good evidence for what you want to show about student achievement, while still (3) keeping in mind what impact these decisions will have on primary value of the portfolio as a student-owned instructional device? On the one hand, if others besides the teacher and student control content, the sense of ownership diminishes and both students and teachers may begin to view the portfolio as more of an intrusion than a help. On the other hand, if students have complete control, they may not choose items that really show that they know and can do.
Probably the best compromise, therefore is for students teachers, and other stakeholders to work together in determining what will be included and to make their decisions in light of some nonrestrictive guidelines. For example, one integrated language arts portfolio system for elementary students requires that each student portfolio contain four self-selected reading samples (one per quarter) assessed using a reading developmental continuum, a speaking/ listening checklist completed by the teacher, and a student statement (written or oral) explaining why certain pieces were selected for the portfolio and how the student sees him- or herself as a reader and writer.
Another example is a high school writing portfolio for which each student must select five samples of writing of various types (e.g., a poem, personal narrative, persuasive piece, timed writing sample, and literary analysis); each student must write a cover letter explaining why these pieces were chosen and what they show about the student as a writer; and the teacher must include a letter certifying that the work is the student's own. Guidelines like these should not inhibit. They should suggest potential directions that will help the student show what he or she can do in many contexts for many purposes and audiences.
Valencia (1989) suggests that all levels of standardization could occur in the same portfolio: some entries would be required (e.g., an attitude survey, standardized test scores, an essay on a specific topic); some would be student/ teacher selected but fall into general categories (e.g., a poem, one piece with drafts, one multimedia piece); and some would be open-ended (students or teachers could add anything they wanted).
Avoiding Pitfalls
There are two pitfalls regarding tasks that were discussed in the introductory
sectionnot having authentic tasks and extraneous interference. A nice
checklist for reviewing the tasks assigned to students in order to avoid
these problems is from Region 15 schools in Southbury, Connecticut (Hibbard,
1991). Some of the entries include the following guidelines: the task
and process parallel tasks in the larger world; the quantity and quality
of time and resources for the task are similar to what would be used when
the task was done in the larger world; and the task is engaging for the
student. One could also add this guideline: the task does not require
the student to use skills exraneous to those being assesssed for example,
a lot of reading in order to do a math problem).
When Will Work Be Selected
for the Portfolio?
Will you have requirements for when work will be selected for the portfolio?
In some portfolio systems that are intended to show growth, for example,
students might be asked to record themselves reading at the end of each
quarter. In other systems, the portfolio might be integrated into instruction
in such a way that students are continually reviewing portfolio content
to see whether new work should replace work previously chosen for the
portfolio.
Issue 5: Assessment
What criteria
are used to assess individual portfolio entries and who develops them?
Should there be criteria for assessing the portfolio as a whole? Who assesses?
How can information be aggregated for large-scale assessment?
Criteria for Individual
Entries
The case for explicit criteria was made earlier. Here we would like to
point out that not everything within a portfolio is likely to be assessed
in the same way. For instance, a reading portfolio might include a book
reviewand criteria could be developed for assessing the completeness,
originality, organization, and insight reflected in that review. But is
might also contain a list of books read outside of school; we might or
might not wish to attach criteria (e.g., minimum number) to development
of such a list.
Criteria for the Portfolio
as a Whole.
Assessing individual pieces within a portfolio is not the same thing as
assessing the portfolio itself. For instance, in a writing a portfolio,
one criterion for assessing the quality of an individual essay might be
development of ideas or clarity of the organization. But criteria for
judging the portfolio itself might include such things as variety in mode
or format, diversity of audiences addressed, and dispositions such as
perseverance, flexibility, and self-confidence. Criteria will need to
be generated for the portfolio as a whole if the portfolio is considered
a product in its own right in addition to being a vehicle for collecting
products and materials that reflect performance, skill, and attitudes.
Criteria Versus Standards
One caution about criteria is that they are not necessarily the same as
standards. Criteria state the characteristics of performance that we value.
Standards state the level of performance that we expect for various grades
and ages of students. For example, we can use exactly the same criteria
to assess writing at grades 5 through 12, but a score of "5"
in grade 5 does not mean the same thing as a score of "5" in
grade 12. This is because our "standards" differ for those two
grades; we expect more from students in grade 12. We often need both criteria
and standards. For example, using our criteria we can trace how much a
student has grown, but we might not know how this relates to how much
a student should grow. Or, we can know there a student falls developmentally,
but we don't know how "good" this is (whether the student is
on grade level).
Aggregation
If you decide you want to aggregate information across students for purposes
of large-scale assessment, how will you do this? Possibilities for aggregation
portfolio information range form aggregation numbers generated from use
of performance criteria and illustrating the data with sample student
performances) to student collaborative efforts in which they develop a
composite portfolio that reflects what they, as a group, learned during
some period of time. For more discussion of the possibilities and issues,
see Arter & Paulson (1991).
Issue 6: Management/
Logistics
Who selects the
actual work that goes in to the portfolio? how are portfolios stored and
moved from teacher to teacher? Who has access to portfolio content? To
whom does the portfolio belong?
Who Selects?
Our definition of a portfolio requires that students have the
responsibility for selecting at least some of the portfolio entries. There
is some evidence that even very young students are capable of selecting
work and reflecting on it (Buell, 1991). In fact, these are the activities
that give the portfolio its power. Depending on the purpose for the portfolio
and the age of the students, students might need more or less teacher
guidance.
Storage and Transfer
A portfolio that incorporates videotapes, audio tapes, photographs, posters,
and so on will quickly outgrow the traditional manila folder. Growing
portfolios may require storage in boxes or files. Also, what gets sent
to the next teacher? Here's one idea: students keep an instructional portfolio
during the school year. Then at the end of the school year they develop
from this a transfer portfolio for next year's teacher. This could be
viewed as having a different purpose and audience than the instructional
portfolio and, therefore, might have different guidelines for what to
include. (Having students develop portfolios for different purposes and
audiences is also a valuable instructional activity.) Then, periodically,
content could be retired form the transfer portfolio, say at the end of
grades 3,6, and 9.
Ownership and Access.
It is also important to decide early on who "owns" the portfolio
and who will have access to it. Ownership implies some control over what
goes into the portfolio and, probably, over where and how it is moved;
so this is not a small decision. It is, of course, desirable for students
to feel some sense of ownership since the portfolios they create area
very real extensions of themselves, but it may also be desirable for schools
or districts to retain some control over how portfolio use is managed.
As an example of this issue, what happens when teachers want to pass student
work on form year to year, but parents also want to keep it? Making photocopies
can become overwhelming.
Several groups may feel that they have a right to or a need for information contained in portfolios. Parents will nearly always feel a vested interest, as will next-year teachers. Other potential audiences include counselors, testing specialists, and administrators. In addition to ownership, a key question is the extent to which a portfolio is private versus public information. Again, this is not a trivial question. It has major implications for how portfolio information is gathered and used and what students feel free to include. Will they share private, personal thoughts through writing, surveys, or personal reflection if these things are available on request to anyone who asks? Clearly, if students feel inhibited about what they're willing to put into the portfolio, its potential to reflect the student's capabilities fully is compromised.
Finally, what about use of student portfolios in training? For example, if we are to develop criteria about what a "good" portfolio looks like, we need to have samples. Ethics and the right to privacy demand t hat if we use student work as samples, we need to remove all students identifying information and, ideally, get student permission for such use.
Issue 7: Staff Development
What types of
training for teachers and administrators will be needed to prepare them
to implement and use portfolios? Teachers need time to explore the possibilities
of portfolio development, to get some notion of what portfolios can or
should be. Much of this information may come from other teachers who are
using portfolios and who have
stories of successes and pitfalls to share. In addition, teachers (or
others working with portfolios) need to be well-grounded in the development
and use of performance criteria so that they can recognize strong performance
in writing, reading, science, mathor any areaand effectively work with
students in selecting what will be most representative of their performance.
they also need to have a great deal of content expertise so that they
can develop good criteria and know what to expect from students at various
grade levels. Finally, they need to be knowledgeable in the area of assessment
so that they can avoid the pitfalls mentioned in previous sections of
this module.
CONCLUSIONS
When designed and used well, portfolios can be very beneficial for student learning, teacher professionalism, communication with parents, and measuring certain types of student learning (Buell, 1991). Many places are currently experimenting with such systems at the classroom, district, and state levels with the promise of exciting results.
There is no one "right" way to design a portfolio system because it depends on context, purpose, and audience. In fact, it would be a great mistake to adopt wholesale a portfolio system designed elsewhere because one of the most beneficial effects of designing a system is the bringing together of staff to think through the issues of audience, purpose, content, and criteria. Allowing teachers the time and support to discuss and articulate what is valued in a performance is almost the single, most beneficial part of the process.
When using portfolios for assessment and instruction, we need to be cautious that such assessments are developed and used properly. We can be misled by work portfolios because the content looks so right. We might not notice that the material was not generated in a way to show what students can do or that it is not representative of student work, etc.
Additionally, there is the danger that if we allow users to rush into use of portfolios for instruction and assessment purposes without thinking through their assessment needs, how a portfolio fits into these needs, and what potential problems they might encounter, they could very likely be confused and disappointed when the portfolio assessment does not fulfill their expectations of "fixing" all assessment problems. We want to avoid having people rush headlong into portfolio assessment and reject it later because it didn't work. Portfolios have the potential to be too useful a part of our assessment and instructional arsenal to allow this to happen.
SELF-TEST
1. For your portfolio system, who will be involved in planning? Who will have primary control over the decisions to be made? What leeway will there be for experimentation? (refer to "Issue 1" in the module discussion.)
2. Which of the following purposes are of particular importance for the portfolio system you are developing? (Refer to "Issue 2.")
______ To show growth or change over time
______ To show the process by which work is done as well as the final product
______ To create collections of favorite or personally important work
______ To trace the evolution of one or more projects/ products
______ To prepare a sample of best work for employment or college admission
______ To document achievement for alternative credit for coursework
______ To place students in the most appropriate course
______ To communicate with students' subsequent teacher
______ To review curriculum or instruction
______ Large-scale assessment
______ Program evaluation
______ Other:
3. What are two major instructional goals for your program? (Refer to "Issue 3.")
How will portfolios be used for classroom instruction/assessment in the system you are designing? what problems (if any) do you anticipate? What issues need to be resolved?
What questions would you consider asking students in order to prompt them to self-reflect on the work they are choosing for their portfolios?
4. What is the general curricular focus of the portfolio system you are planning? (Refer to "Issue 4.")
______ Reading
______ Math
______ Writing
______ Integrated Language Arts
______ Science
______ Social Studies
______ Fine Arts
______ Cross-disciplinary or interdisciplinary
______ Other:
Keeping in mind the classroom goals for students you listed in #3, consider the kinds of things that might go into the portfolios you are designing in order to promote the attainment of those goals and, at the same time provide good evidence of the achievement of those goals. First, what might be required to be included in all portfolios, if anything? Second, list four categories of things that should be included in the work students select for their portfolios. How many samples of each of these things should student select?
Will you allow open-ended choices for the portfolio? How many open-ended items will be allowed?
Who will you get to assist you in finalizing these decisions?
What requirements will you have for when entries are selected for the portfolio, if any?
5. For the portfolio system you are developing, choose one of the types of products that students will be asked to place in their portfolio. What should a good performance look like? What does a poor performance look like? In other words, what are your criteria for judging performance? (Refer to "Issue 5.")
For your portfolio system, which of the following considerations do you think are likely to be important in assessing the portfolio as a whole product?
______ Amount of information included
______ Quality of individual pieces
______ Variety in the kinds of things included
______ Quality and depth of self-reflection
______ Growth in performance, as indicated in products or materials included
______ Apparent changes in attitude or behavior, as indi cated on surveys, questionnaires, etc.
______ Other: