After nearly two years of debate, the Department of Education has backed away from its plan to produce public ratings of colleges and universities, deciding instead to produce a data dashboard with no ratings. Users of the dashboard will be able to organize the data into custom reports of their own design, and there may be some contextualizing information, such as a comparison to the national average on certain measures. But the Department will include no judgment as to whether each measure, categories of measures, or the institution’s statistics as a whole are good, bad, or middling.

The increased transparency about costs and student experiences at colleges and universities that will be provided in the dashboard—with ratings or without—is an unambiguously good thing. And on balance, the decision not to include an overall rating for each institution is wise. As I have written elsewhere, while such ratings have remarkable power to focus attention, summarizing a complex organization with multiple goals in a single rating can be misleading and create perverse incentives.

But there are several consequences of the decision to forego evaluation that are worth considering as the Department builds its data dashboard this summer, as colleges and universities prepare for their data to be scrutinized through this new platform, and as parents, students, and others prepare to use it.

First, an evaluation-free set of data means that each user is on her own to decide what is important and how to make comparisons. The Department has pitched this as a positive, consumer-choice-promoting design decision. But there are some well-documented implications of presenting information to consumers with limited interpretive scaffolding. Two of particular relevance to the higher education context are that it disfavors consumers who lack experience with the products—such as low-income, first-generation students—and that consumers tend to prioritize the most salient information rather than the most important information. Examples of the latter might include focusing on average net price rather than net price by income quintile, or focusing on labor market outcomes without accounting for differences by major.

Second, the presence of a summative rating tends to drive behavior that improves the rating—essentially, improving the measures that go into the rating, starting with those that carry the most weight—for better or worse. The reaction to a set of unrated measures will be idiosyncratic. Some institutions will seek to maximize the dashboard measures that their most influential constituents prioritize. For others, the diffuse measures of the dashboard may not motivate very much at all, and public datasets that do include prominent ratings—like the U.S. News rankings—will remain their North Star.

Finally, with U.S. News in mind, just because the Department will not rate its measures doesn’t mean that they won’t be rated. The Department (rightly) intends to make the dashboard accessible to third-party developers, opening the door to creative, flexible, and niche uses of the data that the Department itself couldn’t support (or wouldn’t even consider). It also opens the door to multiple, potentially conflicting evaluations of the measures based on assumptions, methodologies, and agendas that may or may not be revealed and may or may not align with student or societal interests, or even with sound arithmetic.

None of these considerations mean the Department was wrong to abandon the idea of assigning ratings to colleges and universities. But they do point toward some design challenges the Department will have to address, and some principles for addressing them.

While sophisticated users should have the flexibility to customize, the dashboard should include default reports with the measures, comparisons, and contextualizing information that will be most relevant to most users, and explain why they are. The Department should ensure those default reports are meaningful to colleges and universities—by promoting them to students, parents, and guidance counselors, for example, or by utilizing them in its own regulation and oversight. And while the Department should not limit third-party development, it could review apps for criteria of trustworthiness and endorse those that meet them.

Moving away from a rating system sidesteps some tricky issues and raises others. In a sense, the risk of too much focus is traded for the risk of too little. But with attention to the challenges and careful design, the Department can create a nuanced tool that is both useful and impactful.