An AI Policy Tool for Today: Ambitiously Invest in NIST

We believe that sensible artificial intelligence (AI) policy requires, among other things, the ability to accurately describe and quantify the capabilities and risks of AI systems. This ability is both an enabler and a prerequisite to effective regulation, as measurement tools allow us to objectively assess systems and ensure they meet appropriate safety thresholds. In this post, we propose a policy intervention for how to do this: ambitiously fund the National Institute of Standards and Technology (NIST) to support its AI measurement and standards efforts.

We were heartened by the bipartisan support for maintaining American leadership in the development of critical technologies, as expressed during the April 18 budget hearing on the 2024 Request for the Department of Commerce (and by extension, NIST). We think one of the best ways to channel that support is through an increase in federal funding for NIST so that it is well placed to carry out its work promoting safe technological innovation.

In this post, we give an overview of why we think this, and we also share a policy proposal for what an ambitious funding program for NIST could look like in practice. This proposal is readily actionable and builds on a solid foundation of existing work at the agency; we view it as a complement to a suite of policy levers for stronger AI governance.

Background

In the course of just a few years, we’ve seen rapid technological progress in the kinds of tasks AI systems can complete, and a significant rise in the number of publicly deployed, AI-powered products. Researchers have also discovered that these systems display both significant capabilities and risks, some of which can emerge abruptly through training larger-scale models, and some of which are only discovered after deployment. In an attempt to address this, a variety of ideas have been put forward for how best to govern both the development and use of AI systems.

There have been calls for new federal agencies to regulate AI, multinational legislative proposals to mandate compliance requirements, and even calls from researchers to temporarily “pause” AI development. While these ideas all have meritable qualities, we believe there is also a simpler, “shovel ready” idea available to policymakers in the U.S. that receives comparatively little attention: increasing funding for NIST to build and refine measures of AI assurance.

NIST has for many years diligently worked on the science behind measuring AI systems and the development of associated technical standards. Some highlights of NIST’s work in this area include the Face Recognition Vendor Test and AI Risk Management Framework. AI researchers are also deeply familiar with MNIST, a handwriting recognition dataset which helped drive progress in computer vision in the 1990s and was based on NIST databases of handwritten characters and numbers.

Though the art of measurement may sound dry or possibly inconsequential, we believe it is both critical and urgent for helping us as a society better understand the advances and potential drawbacks of increasingly capable AI systems. As we’ve discussed in our research, open-ended AI systems can act in unpredictable ways — we can’t reliably anticipate all potential risks during the development process. Even if we could, the field lacks widely-agreed upon methods to comprehensively measure and assess those risks.

Investing in Safety and Innovation

Despite NIST’s foundational work in measurement, we’ve seen a general under-resourcing and concerning stagnation in funding for AI-related programs at NIST over the past few years. This lack of support is especially acute against the backdrop of recent technological progress and the widespread adoption of AI systems. With an ambitious investment, NIST could build on fundamental measurement techniques and standardize them across the field. Additional resourcing would also allow NIST to build much-needed community resources, such as testbeds, to assess the capabilities and risks of today’s open-ended AI systems.

Amount of funding (in millions) for AI-related programs at NIST from 2020-2024. Dashed black lines represent Anthropic recommendation of a $15 million increase over FY 2023 (inclusive of NIST’s $5 million requested increase).

The ability to effectively evaluate AI systems for both performance and risks is a precursor to smart AI regulation — after all, it’s hard to manage what you can’t measure. NIST has more than a century of experience building measurement infrastructure, and is thus the natural place to continue this work in the AI domain. Beyond informing smarter regulation, this work can help society chart a safer and more productive course for future AI progress by:

• Enhancing the safety of AI systems through rigorous testing to identify and mitigate risks before they arise in public settings
• Increasing trust among the public that the systems they interact with have been validated by an independent, third-party source
• Providing the government with confidence that advanced systems are safe for the general public
• Promoting innovation as AI developers work to build better technology and push the current state of the art
• Creating a market for system certification and positive incentives for developers to participate

We certainly don’t view the development of measurement techniques and technical standards as a panacea for all potential risks of AI systems, but we view it as a pragmatic approach we can act on today. Similar to our thinking on technical AI safety, we believe in a “portfolio approach” to AI governance that can enhance the safety of AI systems when multiple tools are applied in combination. Rigorous forms of system evaluation can complement other proposals, ranging from robust internal controls and governance practices for developing labs, to regular audits by independent organizations, to regulatory and legislative frameworks that are grounded in the public interest.

This proposal builds on our past advocacy for a more well-resourced NIST, including our testimony in front of the U.S. Senate Committee on Commerce, Science, & Transportation, as well as our formal comments in response to government requests for information.

This blog post follows a policy memo we wrote that looks at past funding patterns and some specific proposals for what NIST might build with more resourcing. As Congress considers appropriations for FY 2024, we urge it to consider additional funding of NIST.

View the memo here.