visualizing items in categories

Someone on an IA mailing list I'm on was asking a visualization question. They wrote:

I'm planning a project where participants will grade example items according to how well they fit their idea of what a certain category is. The grading will be on a 1-10 scale, similar to the (in)famous hot-or-not grading system.

I anticipate about fifty or more items to be tested, and for there to be a range of agreement. What data visualisations formats could I consider for showing this?

Compelled to reply, I wrote:

Will you want to show items spanning multiple categories? I'm trying to sus out what you want to get out of this. I'm guessing you want to present back to someone(s) evidence for what categories make sense re: their set of items.

Of the 50 or more items, perhaps another dimension could be the importance of the item to the stake holders. Ex: the items that either make the most $ for the stakeholders, or are the most valuable for their mission, etc. would be "higher" value than other items.

If you could determine that, then you could make that the up/down dimension, with the top being "higher" value items. If you showed the categories as columns on a 2D surface, all with the same width, then placed all items on that surface in top/down order (could think of them as sorted rows on the sheet), leaving space to one side of each category... ah I should just make a picture... OK, attached is a picture.

The picture just shows the top items, but I presume your final visualization would include all items. Each "row" is an item. If it shows up multiple times, that's because it was considered to be more than 50% within the category (or some rule you will come up with).

You could show items that fell further outside that threshold, but only show them dimly. Or show all items with their transparency set to the percentage they were within the category. (if you do that, still show the physical placement of the item in relation to the category, as that is easier for us humans to pull data from.)

An important indicator of success for the categories would be if the top priority/value items all fit snugly into one (or more) categories. It might be ok if lower value items didn't fit the categories as well.