To enhance security, mechanisms should be established and implemented to control processes and applications. These mechanisms could include process isolation, protection rings, and trusted computer base (TCB).
Process Isolation: Process isolation, executed by the operating system, maintains a high level of system trust by enforcing memory boundaries. Without process isolation, processes would overlap on each other’s memory space, compromising data or possibly making the system unstable. The operating system must also block unauthorized users from entering areas of the system to which they should not have access. These restrictions are done through the use of a virtual machine, which gives the user the impression they have full-access to the system, but in reality, processes are completely isolated. Further, some operating systems also use hardware isolation to increase system security. With hardware isolation, the processes are segmented both logically and physically.
Single-State and Multistate Systems: Single-state and multistate systems were developed to meet the requirements of handling sensitive government information with categories such as sensitive, secret, and top secret. These systems have influence on whether the sensitive information that is processed and retained on a system is managed by the system itself, or by an administrator.
- Single-state systems: also known as dedicated systems, are programmed to process a single category of information and are dedicated to one mode of operation. The system administrator is responsible for maintaining policy and procedures, while delegating which users have access and what level of access to the system.
- Multistate systems: allow multiple users to log in to the system and access various types of data correlated to the users’ levels of clearance. Multistate systems can run as a compartmentalized system and assign data on a need-to-know basis.
Rings of Protection: The operating system adheres to rings of protection to ascertain a user’s level of clearance. It guides the operating system with various levels at which to execute, code or restrict its access. Rings are organized into domains or layers with the most privileged domain located in the center and the least-privileged domain in the outermost ring.
- Layer 0 is the most trusted level. The operating system kernel resides at this level. Any process running at layer 0 is said to be operating in privileged mode.
- Layer 1 contains non-privileged portions of the operating system.
- Layer 2 is where I/O drivers, low-level operations, and utilities reside.
- Layer 3 is where applications and processes operate. This is the level at which individuals usually interact with the operating system. Applications operating here are said to be working in user mode.
Access rights decline as the ring number increases, making the most trusted processes placed in the center rings; and system components reside in the appropriate ring according to the principle of least privilege. This systematic arrangement assures that the processes have only the minimum privileges necessary to perform their functions.
Trusted Computer Base (TCB): The trusted computer base (TCB), defined in the U.S. Department of Defense standard known as “the Orange Book” (DoD Standard 5200.28), is the totality of protection mechanisms within a computer, including hardware, software, controls, and processes, that collaborate to achieve a trusted base to enforce an organization’s security policy. This is the trusted portion of an information system that can be followed and relied on to maintain security policy. The TCB handles confidentiality and integrity and monitors four basic functions:
- Input/output operations: this can present a security concern as operations from the outer rings might interface with rings of increased protection.
- Execution domain switching: another security concern because applications running in one domain often draw from applications or services in other domains.
- Memory protection: this must be monitored to verify confidentiality and integrity in storage.
- Process activation: the security risk here lies in the reading and processing of status information, and file access lists that are vulnerable to compromise of confidentiality in a multiprogramming environment.
The reference monitor is an important element of the TCB and is an abstract mechanism for validating access to objects by authorized subjects. The security kernel which processes user/application requests for system access, implements the reference monitor. Because it’s charged with maintaining control of authorized access, it must be safeguarded from alteration or any slight change and be tested for any anomalies.