A panel mixing a technical judge with a business, design or domain-expert judge naturally scores execution differently — a technical judge spots a fragile demo instantly; a non-technical judge may not, and vice versa for market fit or usability.
Making a mixed panel work
- Assign rubric criteria roughly to expertise during calibration — the technical judge weighs in more heavily on execution, the design judge on usability, both discuss impact together.
- During calibration, have each judge explain what they're actually looking for in their area, so the others can spot it too.
- Resist the urge to have the most confident voice dominate every criterion — the point of a mixed panel is the different lenses, not one lens amplified.
Why mixed panels are worth the coordination cost
A purely technical panel over-rewards clever engineering with no real user; a purely business panel under-rewards genuine technical difficulty. Together, properly calibrated, they score closer to what actually matters.






