The Evaluation the Team Does Itself
The neighborhood's eco-brick cooperative has spent three months making PET bricks for the benches in the community garden. Some days it turns out lots of bricks; other days almost half of them break. Nobody really knows why. How can the team itself figure out what works and what doesn't, without waiting for someone else to tell them?
Evaluate to Improve
A technical system brings together forms of organization, tools, materials, energy, knowledge, and know-how to meet a need. Evaluating it means checking whether it delivers what it promised and what can be improved.
There are two perspectives:
-
Internal evaluation: done by the team working on the project. They know the process from the inside and can measure it every day.
-
External evaluation: done by the community, the users, and other groups. It is the topic of the next page.
Internal evaluation is fast and detailed, but a team can get used to its own flaws. That is why it is complemented by external evaluation.
Four Questions for Evaluating
Four criteria are used to review a technical system. The examples are from the eco-brick cooperative:
| Criterion | Question | Example |
|---|---|---|
| Efficiency | Does it use resources well (time, materials, energy, money)? | 12 L of water are used per batch; another team uses 8 L |
| Effectiveness | Does it achieve the goal it set? | The goal was 40 bricks a day and 36 were made |
| Reliability | Does it work equally well consistently? | Some days 2 bricks come out broken and other days 15 |
| Feasibility | Is it possible to carry out with what you have? | Is there enough PET, people, and tools to keep going? |
The diagram shows the work cycle of a community venture. Explain it in your own words: what is done in each phase and why the last arrow goes back to the first one. Then…
The diagram shows the work cycle of a community venture. Explain it in your own words: what is done in each phase and why the last arrow goes back to the first one. Then answer: who takes part in the external evaluation, marked with «?», and how is their view different from that of the internal evaluation?
Measurable Indicators
An indicator is a piece of data that is always measured the same way to know how one aspect of the system is doing. A good indicator is:
-
Measurable: it has a number and a unit.
-
Clear: the whole team knows how it is obtained.
-
Useful: it is tied to a decision (if it goes up or down, we do something).
| Indicator | How it is measured | Aspect it checks |
|---|---|---|
| Bricks per day | Count the finished bricks at closing time | Effectiveness (did we reach the goal?) |
| Rejection percentage | $\dfrac{\text{piezas rechazadas}}{\text{piezas hechas}} \times 100$ | Reliability and quality |
| Liters of water per batch | Measure the water with a graduated bucket | Efficiency (use of resources) |
| Work hours per brick | Total hours divided by bricks made | Efficiency in time |
The eco-brick cooperative wants to review its own work without waiting for someone from outside to point things out. Propose two measurable internal indicators (for…
The eco-brick cooperative wants to review its own work without waiting for someone from outside to point things out. Propose two measurable internal indicators (for example, pieces made per day, percentage of rejected pieces or liters of water per batch) and explain which aspect of the system each one would improve and how it would be measured.
The Cycle Closes: Continuous Improvement
Measuring is not enough: data is useful only if it changes something. That is why the work cycle ends with an arrow that goes back to the first phase.
-
Measure with the indicators.
-
Compare with the goal or with the previous week.
-
Look for the cause of what didn't go well.
-
Adjust the plan (more drying time, a different mix, a different split of tasks).
-
Measure again to check whether the change worked.
This is called continuous improvement. In addition, when evaluating, a new need may appear (for example, a shortage of PET), and so the cycle starts again from "identify the need."
The diagram shows the work cycle of a community venture. Explain it in your own words: what is done in each phase and why the last arrow goes back to the first one. Then…
The diagram shows the work cycle of a community venture. Explain it in your own words: what is done in each phase and why the last arrow goes back to the first one. Then answer: who takes part in the external evaluation, marked with «?», and how is their view different from that of the internal evaluation?
Go back to the beginning: the cooperative can find out why some days almost half of the bricks break if it measures the rejection percentage every day, compares it with what happened that day (weather, mix, drying), and adjusts the process.