The Transaction Log (SQL Server)

 

Updated: January 4, 2017

Every SQL Server database has a transaction log that records all transactions, and the database modifications made by each transaction.

The transaction log is a critical component of the database. If there is a system failure, you will need that log to bring your database back to a consistent state. Never delete or move this log unless you fully understand the ramifications of doing so.

Fun fact! Known good points from which to begin applying transaction logs during database recovery are created by checkpoints. For more information, see Database Checkpoints (SQL Server).

The transaction log supports the following operations:

  • Individual transaction recovery.

  • Recovery of all incomplete transactions when SQL Server is started.

  • Rolling a restored database, file, filegroup, or page forward to the point of failure.

  • Supporting transactional replication.

  • Supporting high availability and disaster recovery solutions: Always On Availability Groups, database mirroring, and log shipping.

Individual transaction recovery

If an application issues a ROLLBACK statement, or if the Database Engine detects an error such as the loss of communication with a client, the log records are used to roll back the modifications made by an incomplete transaction.

Recovery of all incomplete transactions when SQL Server is started

If a server that is running SQL Server fails, the databases may be left in a state where some modifications were never written from the buffer cache to the data files, and there may be some modifications from incomplete transactions in the data files. When an instance of SQL Server is started, it runs a recovery of each database. Every modification recorded in the log which may not have been written to the data files is rolled forward. Every incomplete transaction found in the transaction log is then rolled back to make sure the integrity of the database is preserved.

Rolling a restored database, file, filegroup, or page forward to the point of failure

After a hardware loss or disk failure affecting the database files, you can restore the database to the point of failure. You first restore the last full database backup and the last differential database backup, and then restore the subsequent sequence of the transaction log backups to the point of failure. As you restore each log backup, the Database Engine reapplies all the modifications recorded in the log to roll forward all the transactions. When the last log backup is restored, the Database Engine then uses the log information to roll back all transactions that were not complete at that point.

Supporting transactional replication

The Log Reader Agent monitors the transaction log of each database configured for transactional replication and copies the transactions marked for replication from the transaction log into the distribution database. For more information, see How Transactional Replication Works.

Supporting high availability and disaster recovery solutions

The standby-server solutions, Always On Availability Groups, database mirroring, and log shipping, rely heavily on the transaction log.

In a Always On Availability Group scenario, every update to a database, the primary replica, is immediately reproduced in separate, full copies of the database, the secondary replicas. The primary replica sends each log record immediately to the secondary replicas which applies the incoming log records to availability group databases, continually rolling it forward. For more information, see Always On Failover Cluster Instances

In a log shipping scenario, the primary server sends the active transaction log of the primary database to one or more destinations. Each secondary server restores the log to its local secondary database. For more information, see About Log Shipping.

In a database mirroring scenario, every update to a database, the principal database, is immediately reproduced in a separate, full copy of the database, the mirror database. The principal server instance sends each log record immediately to the mirror server instance which applies the incoming log records to the mirror database, continually rolling it forward. For more information, see Database Mirroring.

Following are the characteristics of the SQL Server Database Engine transaction log:

  • The transaction log is implemented as a separate file or set of files in the database. The log cache is managed separately from the buffer cache for data pages, which results in simple, fast, and robust code within the Database Engine.
  • The format of log records and pages is not constrained to follow the format of data pages.
  • The transaction log can be implemented in several files. The files can be defined to expand automatically by setting the FILEGROWTH value for the log. This reduces the potential of running out of space in the transaction log, while at the same time reducing administrative overhead. For more information, see ALTER DATABASE (Transact-SQL).
  • The mechanism to reuse the space within the log files is quick and has minimal effect on transaction throughput.

Log truncation frees space in the log file for reuse by the transaction log. You must regularly truncate your transaction log to keep it from filling the alotted space! Several factors can delay log truncation, so monitoring log size matters. Some operations can be minimally logged to reduce their impact on transaction log size.

Log truncation deletes inactive virtual log files from the logical transaction log of a SQL Server database, freeing space in the logical log for reuse by the Physical transaction log. If a transaction log were never truncated, it would eventually fill all the disk space that is allocated to its physical log files.

To avoid this problem, unless log truncation is being delayed for some reason, truncation occurs automatically after the following events:

  • Under the simple recovery model, after a checkpoint.

  • Under the full recovery model or bulk-logged recovery model, if a checkpoint has occurred since the previous backup, truncation occurs after a log backup (unless it is a copy-only log backup).

For more information, see Factors That Can Delay Log Truncation, later in this topic.

NOTE! Log truncation does not reduce the size of the physical log file. To reduce the physical size of a physical log file, you must shrink the log file. For information about shrinking the size of the physical log file, see Manage the Size of the Transaction Log File.

When log records remain active for a long time transaction log truncation is delayed, and potentially the transaction log can fill up.

IMPORTANT!! For information about how to respond to a full transaction log, see Troubleshoot a Full Transaction Log (SQL Server Error 9002).

Log truncation can be delayed by a variety of reasons. You can discover what, if anything, is preventing log truncation by querying the log_reuse_wait and log_reuse_wait_desc columns of the sys.databases catalog view. The following table describes the values of these columns.

log_reuse_wait valuelog_reuse_wait_desc valueDescription
0NOTHINGCurrently there are one or more reusable virtual log files.
1CHECKPOINTNo checkpoint has occurred since the last log truncation, or the head of the log has not yet moved beyond a virtual log file. (All recovery models)

This is a routine reason for delaying log truncation. For more information, see Database Checkpoints (SQL Server).
2LOG_BACKUPA log backup is required before the transaction log can be truncated. (Full or bulk-logged recovery models only)

When the next log backup is completed, some log space might become reusable.
3ACTIVE_BACKUP_OR_RESTOREA data backup or a restore is in progress (all recovery models).

If a data backup is preventing log truncation, canceling the backup operation might help the immediate problem.
4ACTIVE_TRANSACTIONA transaction is active (all recovery models):

A long-running transaction might exist at the start of the log backup. In this case, freeing the space might require another log backup. Note that long-running transactions prevent log truncation under all recovery models, including the simple recovery model, under which the transaction log is generally truncated on each automatic checkpoint.

A transaction is deferred. A deferred transaction is effectively an active transaction whose rollback is blocked because of some unavailable resource. For information about the causes of deferred transactions and how to move them out of the deferred state, see Deferred Transactions (SQL Server).
 
Long-running transactions might also fill up tempdb's transaction log. Tempdb is used implicitly by user transactions for internal objects such as work tables for sorting, work files for hashing, cursor work tables, and row versioning. Even if the user transaction includes only reading data (SELECT queries), internal objects may be created and used under user transactions. Then the tempdb transaction log can be filled.
5DATABASE_MIRRORINGDatabase mirroring is paused, or under high-performance mode, the mirror database is significantly behind the principal database. (Full recovery model only)

For more information, see Database Mirroring (SQL Server).
6REPLICATIONDuring transactional replications, transactions relevant to the publications are still undelivered to the distribution database. (Full recovery model only)

For information about transactional replication, see SQL Server Replication.
7DATABASE_SNAPSHOT_CREATIONA database snapshot is being created. (All recovery models)

This is a routine, and typically brief, cause of delayed log truncation.
8LOG_SCANA log scan is occurring. (All recovery models)

This is a routine, and typically brief, cause of delayed log truncation.
9AVAILABILITY_REPLICAA secondary replica of an availability group is applying transaction log records of this database to a corresponding secondary database. (Full recovery model)

For more information, see Overview of Always On Availability Groups (SQL Server).
10For internal use only
11For internal use only
12For internal use only
13OLDEST_PAGEIf a database is configured to use indirect checkpoints, the oldest page on the database might be older than the checkpoint LSN. In this case, the oldest page can delay log truncation. (All recovery models)

For information about indirect checkpoints, see Database Checkpoints (SQL Server).
14OTHER_TRANSIENTThis value is currently not used.

Minimal logging involves logging only the information that is required to recover the transaction without supporting point-in-time recovery. This topic identifies the operations that are minimally logged under the bulk-logged recovery model (as well as under the simple recovery model, except when a backup is running).

NOTE!! Minimal logging is not supported for memory-optimized tables.

ANOTHER NOTE! Under the full recovery model, all bulk operations are fully logged. However, you can minimize logging for a set of bulk operations by switching the database to the bulk-logged recovery model temporarily for bulk operations. Minimal logging is more efficient than full logging, and it reduces the possibility of a large-scale bulk operation filling the available transaction log space during a bulk transaction. However, if the database is damaged or lost when minimal logging is in effect, you cannot recover the database to the point of failure.

The following operations, which are fully logged under the full recovery model, are minimally logged under the simple and bulk-logged recovery model:

When transactional replication is enabled, BULK INSERT operations are fully logged even under the Bulk Logged recovery model.

  • SELECT INTO operations.

When transactional replication is enabled, SELECT INTO operations are fully logged even under the Bulk Logged recovery model.

  • Partial updates to large value data types, using the .WRITE clause in the UPDATE statement when inserting or appending new data. Note that minimal logging is not used when existing values are updated. For more information about large value data types, see Data Types (Transact-SQL).

  • WRITETEXT and UPDATETEXT statements when inserting or appending new data into the text, ntext, and image data type columns. Note that minimal logging is not used when existing values are updated.

    The WRITETEXT and UPDATETEXT statements are deprecated; avoid using them in new applications.

  • If the database is set to the simple or bulk-logged recovery model, some index DDL operations are minimally logged whether the operation is executed offline or online. The minimally logged index operations are as follows:

    • CREATE INDEX operations (including indexed views).

    • ALTER INDEX REBUILD or DBCC DBREINDEX operations.

      The DBCC DBREINDEX statement is deprecated; Dont use it in new applications.

    • DROP INDEX new heap rebuild (if applicable). (Index page deallocation during a DROP INDEX operation is always fully logged.)

Managing the transaction log

Backing Up the Transaction Log (Full Recovery Model)

Restoring the Transaction Log (Full Recovery Model)

SQL Server Transaction Log Architecture and Management Guide
Control Transaction Durability
Prerequisites for Minimal Logging in Bulk Import
Back Up and Restore of SQL Server Databases
Database Checkpoints (SQL Server)
View or Change the Properties of a Database
Recovery Models (SQL Server)

Community Additions

ADD
Show: