Now that we’ve wrapped up the work of getting the second Atlas of Design off the ground, we wanted to take a minute to explain how our final selections were chosen.
We received almost 300 entries, and it was no easy task to reduce this great collection down to only 32 finalists. To select the maps for the book, we went through three steps.
Step One: Pre-Judging
In the first step, each member of the editorial team looked at each entry, and then recommended whether it should advance to the second step and be reviewed by a panel of judges. This pre-selection was done to ease the burden on the panel, reducing the number of entries that they would have to score.
We erred strongly on the side of inclusiveness in this step. If any one of the three editors recommended that a map should move on to the next step, it advanced. About 200 entries remained at the end of this step.
Step Two: Panel of Judges
Early this year, we recruited a panel of judges to help make the final selections. Each judge reviewed the remaining ~200 entries and gave each a score between 1 and 5, with 5 being the highest. Scoring of each map was highly subjective; we don’t believe there’s any way to really be objective about something like this. Instead, we ensured that we had a broad panel of judges representing a variety of approaches to design. The judges were often in disagreement; almost every map was scored well by at least one judge and poorly by another. This disagreement was exactly our goal in bringing the panel together, because our aim was to ensure that the final selection was not dominated by one style or taste, but held something for everyone.
Several of the judges were also entrants to the competition, and a number of them are featured in the final volume. Judges recused themselves from scoring their own work, and in cases of any other conflicts of interest (such as an instructor scoring a former student).
Step Three: Editor Selection
The editorial team next took the maps and ordered them by their average judge score. We then started looking over the top-ranked ones and selecting the finalists for the book. We used the scores and judges’ comments as a guide, but the ultimate authority rested with us. We kept largely to the judges’ order, but did sometimes exercised our editorial judgment in order to create a book that showed off a wide variety of map styles and approaches. Some examples of how we deviated from the base scores:
1) A number of mapmakers had multiple top-ranked entries. We wanted to limit each mapmaker to only one entry in the final book. But instead of including their best-ranked map, we sometimes chose a different (still highly-ranked) map to include, if it improved the variety of the book. So, let’s pretend the same person made 2 of the judges’ favorite maps. The most-liked was a classic terrain map, and the less-liked was a stylish abstract thematic map. If we think we have a large number of terrain maps, and very few thematic maps, we might choose to include the latter, even if the judges didn’t quite like it as much, as long as it was still highly-ranked. This allows the book to demonstrate a wider range of cartographic styles.
2) Just as we limited authors to one entry, we took a similar approach to organizations, applying a cap to make sure that some of the larger mapmaking enterprises didn’t dominate the book.
3) To produce a book that represented a wide variety of cartographic approaches, we sometimes also placed caps on the number of maps that we accepted that used a particular style.
Throughout, we were guided by the judges’ scores and deviated only rarely where we thought it would make a better book in the end. The judges were asked to appraise each map, one at a time, whereas our job as editors was to look at the big picture of how they all fit together.
Selecting maps in any sort of competition is a highly subjective process, and we want to be transparent about that process. Our aim was to make a book that honored a number of beautiful maps and which came together in a collection that appealed to a wide variety of tastes. We hope that you’ll find us to have met that goal.