From Fedora Project Wiki
(initial version of most sections)
(filled in remaining parts, changed preferred email)
Line 6: Line 6:
 
== Owner ==
 
== Owner ==
 
* Name: [[User:kengert| Kai Engert]]
 
* Name: [[User:kengert| Kai Engert]]
* Email: kengert@redhat.com
+
* Email: kaie@redhat.com
 
* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
 
* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
 
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
 
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
Line 98: Line 98:
  
 
== Documentation ==
 
== Documentation ==
<!-- Is there upstream documentation on this change, or notes you have written yourself? Link to that material here so other interested developers can get involved. -->
+
There are couple of upstream documents, which have been produced a couple of years ago, which might provide background on this change of technology.
 +
* https://wiki.mozilla.org/NSS_Shared_DB
 +
  * https://wiki.mozilla.org/NSS_Shared_DB_And_LINUX
  
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+
(Those documents partially focused on an additional property, which could potentially get achieved by migrating to the new file format: Having a common database for each system user, shared by all applications one user uses.
N/A (not a System Wide Change)  
+
However, this potential consolidation is more complicated to achieve. I suggest to EXCLUDE the consolidation from this change. The reason is the unresolved issue of merging databases from two sources to a single destination, when the sources used different database protection passwords. Migration will fail, unless application provides a user interface to prompt for the second database password.)
  
 
== Release Notes ==
 
== Release Notes ==
<!-- The Fedora Release Notes inform end-users about what is new in the release.  Examples of past release notes are here: http://docs.fedoraproject.org/release-notes/ -->
+
Yes, we should provide detailed release notes.
<!-- The release notes also help users know how to deal with platform changes such as ABIs/APIs, configuration or data file formats, or upgrade concerns.  If there are any such changes involved in this change, indicate them here.  A link to upstream documentation will often satisfy this need.  This information forms the basis of the release notes edited by the documentation team and shipped with the release.
 
 
 
Release Notes are not required for initial draft of the Change Proposal but has to be completed by the Change Freeze.  
 
-->
 
  
 
[[Category:ChangePageIncomplete]]
 
[[Category:ChangePageIncomplete]]
Line 116: Line 114:
 
<!-- After review, the Wrangler will move your page to Category:ChangeReadyForFesco... if it still needs more work it will move back to Category:ChangePageIncomplete-->
 
<!-- After review, the Wrangler will move your page to Category:ChangeReadyForFesco... if it still needs more work it will move back to Category:ChangePageIncomplete-->
  
<!-- Select proper category, default is Self Contained Change -->
+
[[Category:SystemWideChange]]
[[Category:SelfContainedChange]]
 
<!-- [[Category:SystemWideChange]] -->
 

Revision as of 18:41, 3 July 2017

NSS Default File Format SQL

Summary

Change the NSS library default to use the sqlite based data storage, when applications don't specify their preferred storage file format.

Owner

  • Name: Kai Engert
  • Email: kaie@redhat.com
  • Release notes owner:

Current status

  • Targeted release: Fedora 27
  • Last updated: 2017-07-03
  • Tracker bug: <will be assigned by the Wrangler>

Detailed Description

Applications that use the NSS library often use a database for storage of keys, certificates and trust. NSS supports two different file formats, one called DBM (based on berkeley DB files) and another one called SQL (based on sqlite DB files).

Today's default file format used by NSS, used when applications omit the type parameter, is the older DBM file format, which forbids parallel access to the storage. The suggestion is to change the default file format to SQL, which allows parallel access to the storage.

Applications, or users using the NSS command line utilities, often provide the database storage location using a simple directory path parameter. Some might not be aware, or forget, that the parameter can be prefixed with a type modifier, either "dbm:" or "sql:".

As a result, when not providing this parameter, the file format used will be the fragile DBM file format. This is particuarly problematic, if a user attempts to modify the NSS storage using command line tools, while another process, such as a daemon, is running concurrently, which also accesses the same database in the DBM file format. This often results in corrupted database storage, which cannot be recovered.

By changing the default, all applications that currently use the DBM file format, will automatically be migrated to the SQL file format. NSS has the ability to discover if a storage location (a directory) contains the DBM file format. If configured to use the modern SQL format, NSS will automatically perform a one-time conversion from the DBM to the SQL format.

The same applies to the NSS command line utilities. If the NSS library default is changed to SQL, the NSS tools will also trigger the one-time conversion, or access the already converted files.

Benefit to Fedora

The benefit will be less data corruption, because the SQL file format explicitly allows parallel access to the storage files by multiple applications.

NSS has been supporting this new file format for many years, and several applications have already opted in to explicitly use the new file format, so it can be considered stable.

Scope

  • Proposal owners:

A small downstream patch needs to be applied to the NSS library package, which changes the library default.

  • Other developers:

It's up to developers of NSS applications, if they accept the new default and an automatic conversion, or if they prefer to continue to use the classic DBM storage format. Although not recommended, developers can easily do so, by adding a "dbm:" prefix to the storage parameter they provide to NSS at NSS library initialization time.

No help should be necessary. No mass rebuild necessary.

  • Policies and guidelines: N/A
  • Trademark approval: N/A

Upgrade/compatibility impact

Data will be migrated automatically.

How To Test

Each application or server softare that uses NSS should be tested for successful data migration. Ideally, an application/server configuration should be used, which already contains data from a previous Fedora version. It should be tested that after migration to the new Fedora version, the software continues to operate as expected. In particular, this means that the software still has access to the locally configure private keys, certificates, trust settings which the application/server has configured. (This only applies to applications that perform this kind of operation.)

User Experience

The result changes will be restricted to the file system level, and to applications that use the NSS library default.

Previously, the following filenames were used for NSS storage:

* cert8.db
* key3.db
* secmod.db

After the change, the following filenames will be used:

* cert9.db
* key4.db
* pkcs11.txt

Applications that migrate data from an older version will contain both sets of files in their storage. However, after the conversion, the old files will no longer be used, they can be considered a one-time backup.

Applications that are executed for the first time with the new library default already effective, will only get the new storage files.

Dependencies

No dependencies.

Contingency Plan

  • Contingency mechanism: (What to do? Who will do it?) Revert the shipped configuration, NSS maintainer.
  • Contingency deadline: beta freeze
  • Blocks release? No
  • Blocks product? No

Documentation

There are couple of upstream documents, which have been produced a couple of years ago, which might provide background on this change of technology.

* https://wiki.mozilla.org/NSS_Shared_DB
* https://wiki.mozilla.org/NSS_Shared_DB_And_LINUX

(Those documents partially focused on an additional property, which could potentially get achieved by migrating to the new file format: Having a common database for each system user, shared by all applications one user uses. However, this potential consolidation is more complicated to achieve. I suggest to EXCLUDE the consolidation from this change. The reason is the unresolved issue of merging databases from two sources to a single destination, when the sources used different database protection passwords. Migration will fail, unless application provides a user interface to prompt for the second database password.)

Release Notes

Yes, we should provide detailed release notes.