SecurityEngineering/NSS Startup and Shutdown in Gecko

This is an informational document outlining the modernization and simplification of NSS startup and shutdown in Gecko. It is organized in three parts: the current setup, the desired setup, and a roadmap for achieving the desired setup. If the current date is later than 1 November 2016, this document is likely out of date.

The Current Setup

Classes in PSM (and other parts of Gecko) implement a number of interfaces that require NSS functionality. Instantiating any of these classes causes the PSM component to be initialized. This initialization starts a few services needed by certificate verification. It also performs the initialization of NSS. This involves calling the overall NSS_Initialize function as well as some configuration options like only enabling specific cipher suites and loading the trust anchors for certificate verification.

The PSM component observes a number of events, including notification of profile changes, preference changes, and XPCOM shutdown. Upon receiving the "profile-before-change" notification, the PSM component releases all NSS resources held by PSM objects and calls NSS_Shutdown. Consequently, whenever a PSM object attempts to use an NSS resource or call an NSS function, it first must check if NSS has been shut down. This is coordinated by having such classes inherit from nsNSSShutDownObject. At each point NSS resources are used, these classes first acquire an nsNSSShutDownPreventionLock and check isAlreadyShutDown() (see nsNSSShutDown.h).

As a consequence of this design, when implementing new functionality that uses NSS, it is easy to do the wrong thing. For instance, code that merely calls NSS functions but do not hold NSS resources may forget to check and prevent NSS from shutting down during the use of that function. Another common mistake is to acquire the nsNSSShutDownPreventionLock but not actually check if NSS has been shut down. These have often led to shutdown crashes (see for example bug 1114741, bug 1046221, bug 1029173, and bug 911336).

The Desired Setup

NSS should be initialized exactly once and shut down exactly once. Code that uses it should only be able to run after NSS is guaranteed to be initialized. While such code is running, it should prevent NSS from being shut down out from under it. When NSS is about to be shut down, all NSS resources held by the platform should be released. Any NSS resource leaks as detected by NSS_Shutdown should be fatal in debug builds. Once NSS has been shut down (upon notification that the entire process is shutting down), all methods that would use NSS must first check for this and return an error.

Writing new code that correctly deals with these restrictions should be easy.

How to Get There

• Remove unused and/or unnecessary cruft from PSM/NSS initialization (bug 1215267)
  • nsPSMUITracker has already been removed: bug 1215690)
  • Profile change event handling: bug 1222179
  • (etc.)
• Fix all NSS shutdown leaks: bug 1230312
• Make NSS shutdown leaks fatal
• Handle the case where an object that needs to be tracked to free resources on NSS shutdown is created before NSS is initialized (this should and can be made to work - bug 1235634)
• Separate NSS-only initialization from PSM component initialization
• Ensure NSS is initialized before execution reaches any code that requires it
• Provide a better mechanism for preventing NSS from shutting down (and checking if it has already shut down)
  • Currently the only way to do this that (mostly) works is for a class to implement the nsNSSShutDownObject mechanism, acquire an nsNSSShutDownPreventionLock and check isAlreadyShutDown. It should be possible to perform the same steps without implementing nsNSSShutDownObject (indeed, this would be better, since that interface has more to do with releasing long-lived NSS resources at shutdown). Furthermore, this mechanism doesn't entirely work, because if an object that implements nsNSSShutDownObject is instantiated after NSS has been shut down, isAlreadyShutDown will actually return false.