The National Software Reference Library Banner

The National Software Reference Library Logo

GENERAL INFORMATION

VOTING

TECHNICAL INFORMATION

DOWNLOADS

Partner Projects

Privacy Policy/Security Notice
Disclaimer | FOIA |USAGov

NIST is an agency of the
U.S. Commerce Department

Date created: 8/20/2003
Last updated:

Technical comments: nsrl@nist.gov
Website comments: web897@nist.gov

Search NIST website

 

 

 

Welcome to the National Software Reference Library

(NSRL) Project Web Site.

This project is supported by the U.S. Department of Homeland Security, federal, state, and local law enforcement, and the National Institute of Standards and Technology (NIST) to promote efficient and effective use of computer technology in the investigation of crimes involving computers. Numerous other sponsoring organizations from law enforcement, government, and industry are providing resources to accomplish these goals, in particular the FBI who provided the major impetus for creating the NSRL out of their ACES program.

The National Software Reference Library (NSRL) is designed to collect software from various sources and incorporate file profiles computed from this software into a Reference Data Set (RDS) of information. The RDS can be used by law enforcement, government, and industry organizations to review files on a computer by matching file profiles in the RDS. This will help alleviate much of the effort involved in determining which files are important as evidence on computers or file systems that have been seized as part of criminal investigations.

The RDS is a collection of digital signatures of known, traceable software applications. There are application hash values in the hash set which may be considered malicious, i.e. steganography tools and hacking scripts. There are no hash values of illicit data, i.e. child abuse images.

The National Software Reference Library is a project in Software and Systems Division supported by The Special Programs Office.

Query the Hash Set Online

There is a project called NSRLquery developed by Rob Hansen of RedJack Security LLC, which has two subprojects: nsrlsvr, which provides a server that yields NSRL RDS information on request, and nsrllookup, a simple command-line application that queries the server.
Jesse Kornblum has established a beta testing NSRLquery server at nsrl.kyr.us, and NSRL has been supplying release-day data to support this server.

There is a NSRL Hash Search Engine hosted at hashsets.com which currently (Jan 2014) supports searches via MD5 or file name.

NSRL Products and Research

Sub-Project Description
RDS Hashsets The core product of the NSRL, the hashsets and metadata used in file identification.
This data can be used with third-party digital forensics tools.
The NSRL RDS is released four times each year - in March, June, September and December -
according to the schedule below.

Download the current RDS DVD ISO now
Product List A 500KB tab-delimited sorted list of all manufacturers' products and versions that are included in RDS 2.50.
Software Diskprints The NSRL now augments the metadata published in the RDS with data that catalog
modifications of installed software on known systems under controlled conditions.
Mobile Devices NSRL is collecting mobile apps (currently iOS- and Android-based) for inclusion in the RDS,
and to catalog functionalities.
Approximate Matching Approximate matching provides a means to assess/quantify the relationship
between two files beyond same/not same.
Exemplar tools include ssdeep and sdhash (for which data sets exist).
Digital Forensics XML (DFXML) Schema for DFXML objects.
SWIDtags NSRL is preparing a release of metadata in support of Software ID tagging.
An experimental set of 14,400 .swidtag files is available for comments.
Installed Software NSRL is investigating network-based, dynamically installed software
to include metadata from applications acquired via this delivery mechanism.
Archival Collaborations The National Institute of Standards and Technology (NIST) and Stanford University Libraries (SUL)
project to catalog the data contained in about 15,000 software releases from the early days of microcomputing.
External links: NIST article, Stanford article.
Research Environment NSRL has a research computing environment containing some 32,000,000 unique original files,
along with a database containing metadata about the files (filename, bytesize, etc.).
There are specific conditions for access to the research environment.
Other Algorithms Data sets that use hashing or digest algorithms not contained in the RDS release.
Block hashes, SHA-2, SHA-512, etc.
Test Data Test data for SHA-1, MD5 or SHA-256 implementations,
and links to other validation data.
NSRL Perl Modules How NSRL hashes every file encountered and recursively hashes the contents of any
"archive" type of file (e.g. zip file, tar file, cab file, uuencoded file).
Voting Software RDS This data set contains information that can be used to verify
that voting software files have not been modified.

NSRL RDS Annual Release Schedule

Date Task Notes
Feb 16-28 Build and QC of master RDS Software arriving now goes in next release
Mar 1 Deliver master RDS to NIST SRD contact SRD duplicates and mails the RDS
Apr 1 Subscribers should receive RDS in mail  
Apr 28 ISO images of RDS available as free downloads  
May 16-28 Build and QC of master RDS Software arriving now goes in next release
Jun 1 Deliver master RDS to NIST SRD contact SRD duplicates and mails the RDS
Jul 1 Subscribers should receive RDS in mail  
Jul 28 ISO images of RDS available as free downloads  
Aug 16-28 Build and QC of master RDS Software arriving now goes in next release
Sep 1 Deliver master RDS to NIST SRD contact SRD duplicates and mails the RDS
Oct 1 Subscribers should receive RDS in mail  
Oct 28 ISO images of RDS available as free downloads  
Nov 16-28 Build and QC of master RDS Software arriving now goes in next release
Dec 1 Deliver master RDS to NIST SRD contact SRD duplicates and mails the RDS
Jan 1 Subscribers should receive RDS in mail  
Jan 28 ISO images of RDS available as free downloads