Understanding Virtual Memory -Linux

Understanding Virtual Memory

by Norm Murray and Neil Horman

• Introduction
• Definitions
• The Life of a Page
• Tuning the VM
• Example Scenarios
• Further Reading
• About the Author

Introduction

One of the most important aspects of an operating system is the Virtual Memory Management system. Virtual Memory (VM) allows an operating system to perform many of its advanced functions, such as process isolation, file caching, and swapping. As such, it is imperative that an administrator understand the functions and tunable parameters of an operating system's Virtual Memory Manager so that optimal performance for a given workload may be achieved. After reading this article, the reader should have a rudimentary understanding of the data the Red Hat Enterprise Linux (RHEL3) VM controls and the algorithms it uses. Further, the reader should have a fairly good understanding of general Linux VM tuning techniques. It is important to note that Linux as an operating system has a proud legacy of overhaul. Items which no longer serve useful purposes or which have better implementations as technology advances are phased out. This implies that the tuning parameters described in this article may be out of date if you are using a newer or older kernel. Fear not however! With a well grounded understanding of the general mechanics of a VM, it is fairly easy to convert knowledge of VM tuning to another VM. The same general principles apply, and documentation for a given kernel (including its specific tunable parameters) can be found in the corresponding kernel source tree under the file Documentation/sysctl/vm.txt.

Definitions

To properly understand how a Virtual Memory Manager does its job, it helps to understand what components comprise a VM. While the low level view of a VM are overwhelming for most, a high level view is necessary to understand how a VM works and how it can be optimized for workloads.

What Comprises a VM

Figure 1. High Level Overview of VM Subsystem

The inner workings of the Linux virtual memory subsystem are quite complex, but it can be defined at a high level with the following components:

MMU

The Memory Management Unit (MMU) is the hardware base that makes a VM system possible. The MMU allows software to reference physical memory by aliased addresses, quite often more than one. It accomplishes this through the use of pages and page tables. The MMU uses a section of memory to translate virtual addresses into physical addresses via a series of table lookups.

Zoned Buddy Allocator

The Zoned Buddy Allocator is responsible for the management of page allocations to the entire system. This code manages lists of physically contiguous pages and maps them into the MMU page tables, so as to provide other kernel subsystems with valid physical address ranges when the kernel requests them (Physical to Virtual Address mapping is handled by a higher layer of the VM). The name Buddy Allocator is derived from the algorithm this subsystem uses to maintain it free page lists. All physical pages in RAM are cataloged by the Buddy Allocator and grouped into lists. Each list represents clusters of 2n pages, where n is incremented in each list. If no entries exist on the requested list, an entry from the next list up is broken into two separate clusters and is returned to the caller while the other is added to the next list down. When an allocation is returned to the buddy allocator, the reverse process happens. Note that the Buddy Allocator also manages memory zones, which define pools of memory which have different purposes. Currently there are three memory pools which the Buddy Allocator manages accesses for:

• DMA — This zone consists of the first 16 MB of RAM, from which legacy devices allocate to perform direct memory operations.

• NORMAL — This zone encompasses memory addresses from 16 MB to 1 GB and is used by the kernel for internal data structures as well as other system and user space allocations.

• HIGHMEM — This zone includes all memory above 1 GB and is used exclusively for system allocations (file system buffers, user space allocations, etc).

Slab Allocator

The Slab Allocator provides a more usable front end to the Buddy Allocator for those sections of the kernel which require memory in sizes that are more flexible than the standard 4 KB page. The Slab Allocator allows other kernel components to create caches of memory objects of a given size. The Slab Allocator is responsible for placing as many of the cache's objects on a page as possible and monitoring which objects are free and which are allocated. When allocations are requested and no more are available, the Slab Allocator requests more pages from the Buddy Allocator to satisfy the request. This allows kernel components to use memory in a much simpler way. This way components which make use of many small portions of memory are not required to individually implement memory management code so that too many pages are not wasted. The Slab Allocator may only allocate from the DMA and NORMAL zones.

Kernel Threads

The last component in the VM subsystem are the kernel threads: kscand, kswapd, kupdated, and bdflush. These tasks are responsible for the recovery and management of in use memory. All pages of memory have an associated state (for more information on the memory state machine, refer to the section called “The Life of a Page” section. In general, the active tasks in the kernel related to VM usage are responsible for attempting to move pages out of RAM. Periodically they examine RAM, trying to identify and free inactive memory so that it can be put to other uses in the system.

The Life of a Page

All of the memory managed by the VM is labeled by a state. These states help let the VM know what to do with a given page under various circumstances. Dependent on the current needs of the system, the VM may transfer pages from one state to the next, according to the state machine in Figure 2. “VM Page State Machine”. Using these states, the VM can determine what is being done with a page by the system at a given time and what actions the VM may take on the page. The states that have particular meanings are as follows:

1. FREE — All pages available for allocation begin in this state. This indicates to the VM that the page is not being used for any purpose and is available for allocation.

2. ACTIVE — Pages which have been allocated from the Buddy Allocator enter this state. It indicates to the VM that the page has been allocated and is actively in use by the kernel or a user process.

3. INACTIVE DIRTY — This state indicates that the page has fallen into disuse by the entity which allocated it and thus is a candidate for removal from main memory. The kscand task periodically sweeps through all the pages in memory, taking note of the amount of time the page has been in memory since it was last accessed. If kscand finds that a page has been accessed since it last visited the page, it increments the page's age counter; otherwise, it decrements that counter. If kscand finds a page with its age counter at zero, it moves the page to the inactive dirty state. Pages in the inactive dirty state are kept in a list of pages to be laundered.

4. INACTIVE LAUNDERED — This is an interim state in which those pages which have been selected for removal from main memory enter while their contents are being moved to disk. Only pages which were in the inactive dirty state can enter this state. When the disk I/O operation is complete, the page is moved to the inactive clean state, where it may be deallocated or overwritten for another purpose. If, during the disk operation, the page is accessed, the page is moved back into the active state.

5. INACTIVE CLEAN — Pages in this state have been laundered. This means that the contents of the page are in sync with the backed up data on disk. Thus, they may be deallocated by the VM or overwritten for other purposes.

Figure 2. VM Page State Machine

Tuning the VM

Now that the picture of the VM mechanism is sufficiently illustrated, how is it adjusted to fit certain workloads? There are two methods for changing tunable parameters in the Linux VM. The first is the sysctl interface. The sysctl interface is a programming oriented interface, which allows software programs to modify various tunable parameters directly. It is exported to system administrators via the sysctl utility, which allows an administrator to specify a value for any of the tunable VM parameters via the command line. For example:

sysctl -w vm.max map count=65535 

The sysctl utility also supports the use of a configuration file (/etc/sysctl.conf), in which all the desirable changes to a VM can be recorded for a system and restored after a restart of the operating system, making this access method suitable for long term changes to a system VM. The file is straightforward in its layout, using simple key-value pairs with comments for clarity. For example:

 #Adjust the min and max read-ahead for files vm.max-readahead=64 vm.min-readahead=32 #turn on memory over-commit  vm.overcommit_memory=2 #bump up the percentage of memory in use to activate bdflush vm.bdflush="40 500 0 0 500 3000 60 20 0"  

The second method of modifying VM tunable parameters is via the proc file system. This method exports every group of VM tunables as a virtual file, accessible via all the common Linux utilities used for modifying file contents. The VM tunables are available in the directory /proc/sys/vm/ and are most commonly read and modified using the cat and echo commands. For example, use the command cat /proc/sys/vm/kswapd to view the current value of the kswapd tunable. The output should be similar to:

 512 32 8  

Then, use the following command to modify the value of the tunable:

echo 511 31 7 > /proc/sys/vm/kswapd 

Use the cat /proc/sys/vm/kswapd command again to verify that the value was modified. The output should be:

 511 31 7  

The proc file system interface is a convenient method for making adjustments to the VM while attempting to isolate the peak performance of a system. For convenience, the following sections list the VM tunable parameters as the filenames they are exported to in the /proc/sys/vm/ directory. Unless otherwise noted, these tunables apply to the RHEL3 2.4.21-4 kernel.

bdflush

The bdflush file contains 9 parameters, of which 6 are tunable. These parameters affect the rate at which pages in the buffer cache (the subset of pagecache which stores files in memory) are freed and returned to disk. By adjusting the various values in this file, a system can be tuned to achieve better performance in environments where large amounts of file I/O are performed. Table 1. “bdflush Parameters” defines the parameters for bdflush in the order they appear in the file.

Parameter	Description
nfract	The percentage of dirty pages in the buffer cache required to activate the bdflush task
ndirty	The maximum number of dirty pages in the buffer cache to write to disk in each bdflush execution
reserved1	Reserved for future use
reserved2	Reserved for future
interval	The number of jiffies (10ms periods) to delay between bdflush iterations
age_buffer	The time for a normal buffer to age before it is considered for flushing back to disk
nfract_sync	The percentage of dirty pages in the buffer cache required to cause the tasks which are writing pages of memory to begin writing those pages to disk instead
nfract_stop_bdflush	The percentage of dirty pages in buffer cache required to allow bdflush to return to idle state
reserved3	Reserved for future use

Table 1. bdflush Parameters

Generally, systems that require more free memory for application allocation want to set the bdflush values higher (except for the age_buffer, which would be moved lower), so that file data is sent to disk more frequently and in greater volume, thus freeing up pages of RAM for application use. This, of course, comes at the expense of CPU cycles because the system processor spends more time moving data to disk and less time running applications. Conversely, systems which are required to perform large amounts of I/O would want to do the opposite to these values, allowing more RAM to be used to cache disk file so that file access is faster.

dcache_priority

This file controls the bias of the priority for caching directory contents. When the system is under stress, it selectively reduces the size of various file system caches in an effort to reclaim memory. By increasing this value, memory reclamation bias is shifted away from the dirent cache. By reducing this amount, the bias is shifted towards reclaiming dirent memory. This is not a particularly useful tuning parameter, but it can be helpful in maintaining the interactive response time on an otherwise heavily loaded system. If you experience intolerable delays in communicating with your system when it is busy performing other work, increasing this parameter may help.

hugetlb_pool

The hugetlb_pool file is responsible for recording the number of megabytes used for huge pages. Huge pages are just like regular pages in the VM, only they are an order of magnitude larger. Note also that huge pages are not swappable. Huge pages are both beneficial and detrimental to a system. They are helpful in that each huge page takes only one set of entries in the VM page tables, which allows for a higher degree of virtual address caching in the TLB (Translation Look-aside Buffer: A device which caches virtual address translations for faster lookups) and a requisite performance improvement. On the downside, they are very large and can be wasteful of memory resources for those applications which do not need large amounts of memory. Some applications, however, do require large amounts of memory and can make good use of huge pages if they are written to be aware of them. If a system is running applications which require large amounts of memory and is aware of this feature, then it is advantageous to increase this value to an amount satisfactory to that application or set of applications.

inactive_clean_percent

This control specifies the minimum percentage of pages in each page zone that must be in the clean or laundered state. If any zone drops below this threshold, and the system is under pressure for more memory, then that zone will begin having its inactive dirty pages laundered. Note that this control is only available on the 2.4.21-5EL kernels forward. Raising the value for the corresponding zone which is memory starved causes pages to be paged out more quickly, eliminating memory starvation at the expense of CPU clock cycles. Lowering this number allows more data to remain in RAM, increasing the system performance but at the risk of memory starvation.

kswapd

While this set of parameters previously defined how frequently and in what volume a system moved non-buffer cache pages to disk, in Red Hat Enterprise Linux 3, these controls are unused.

max_map_count

The max_map_count file allows for the restriction of the number of VMAs (Virtual Memory Areas) that a particular process can own. A Virtual Memory Area is a contiguous area of virtual address space. These areas are created during the life of the process when the program attempts to memory map a file, links to a shared memory segment, or allocates heap space. Tuning this value limits the amount of these VMAs that a process can own. Limiting the amount of VMAs a process can own can lead to problematic application behavior because the system will return out of memory errors when a process reaches its VMA limit but can free up lowmem for other kernel uses. If your system is running low on memory in the NORMAL zone, then lowering this value will help free up memory for kernel use.

max-readahead

The max-readahead tunable affects how early the Linux VFS (Virtual File System) fetches the next block of a file from memory. File readahead values are determined on a per file basis in the VFS and are adjusted based on the behavior of the application accessing the file. Anytime the current position being read in a file plus the current read ahead value results in the file pointer pointing to the next block in the file, that block is fetched from disk. By raising this value, the Linux kernel allows the readahead value to grow larger, resulting in more blocks being prefetched from disks which predictably access files in uniform linear fashion. This can result in performance improvements but can also result in excess (and often unnecessary) memory usage. Lowering this value has the opposite affect. By forcing readaheads to be less aggressive, memory may be conserved at a potential performance impact.

min-readahead

Like max-readahead, min-readahead places a floor on the readahead value. Raising this number forces a file's readahead value to be unconditionally higher, which can bring about performance improvements provided that all file access in the system is predictably linear from the start to the end of a file. This, of course, results in higher memory usage from the pagecache. Conversely, lowering this value, allows the kernel to conserve pagecache memory at a potential performance cost.

overcommit_memory

overcommit_memory is a value which sets the general kernel policy toward granting memory allocations. If the value is 0, then the kernel checks to determine if there is enough memory free to grant a memory request to a malloc call from an application. If there is enough memory, then the request is granted. Otherwise, it is denied and an error code is returned to the application. If the setting in this file is 1, the kernel allows all memory allocations, regardless of the current memory allocation state. If the value is set to 2, then the kernel grants allocations above the amount of physical RAM and swap in the system as defined by the overcommit_ratio value. Enabling this feature can be somewhat helpful in environments which allocate large amounts of memory expecting worst case scenarios but do not use it all.

overcommit_ratio

The overcommit_ratio tunable defines the amount by which the kernel overextends its memory resources in the event that overcommit_memory is set to the value of 2. The value in this file represents a percentage added to the amount of actual RAM in a system when considering whether to grant a particular memory request. For instance, if this value is set to 50, then the kernel would treat a system with 1 GB of RAM and 1 GB of swap as a system with 2.5 GB of allocatable memory when considering whether to grant a malloc request from an application. The general formula for this tunable is:

allocatable memory=(swap size + (RAM size * overcommit ratio)) 

Use these previous two parameters with caution. Enabling overcommit_memory can create significant performance gains at little cost but only if your applications are suited to its use. If your applications use all of the memory they allocate, memory overcommit can lead to short performance gains followed by long latencies as your applications are swapped out to disk frequently when they must compete for oversubscribed RAM. Also, ensure that you have at least enough swap space to cover the overallocation of RAM (meaning that your swap space should be at least big enough to handle the percentage if overcommit in addition to the regular 50 percent of RAM that is normally recommended).

pagecache

The pagecache file adjusts the amount of RAM which can be used by the page cache. The page cache holds various pieces of data, such as open files from disk, memory mapped files, and pages of executable programs. Modifying the values in this file dictates how much of memory is used for this purpose. Table 2. “pagecache Parameters” defines the parameters for pagecache in the order they appear in the file.

Parameter	Description
min	The minimum amount of memory to reserve for pagecache use.
borrow	The percentage of pagecache pages kswapd uses to balance the reclaiming of pagecache pages and process memory.
max	If more memory than this percentage is used by pagecache, kswapd only evicts pages from the pagecache. Once the amount of memory in pagecache is below this threshold, kswapd begins moving process pages to swap again.

Table 2. pagecache Parameters

Increasing these values allows more programs and cached files to stay in memory longer, thereby allowing applications to execute more quickly. On memory starved systems, however, this may lead to application delays as processes must wait for memory to become available. Moving these values downward swaps processes and other disk-backed data out more quickly, allowing for other processes to obtain memory more easily and increasing execution speed. For most workloads the automatic tuning is sufficient. However, if your workload suffers from excessive swapping and a large cache, you may want to reduce the values until the swapping problem goes away.

page-cluster

The kernel attempts to read multiple pages from disk on a page fault to avoid excessive seeks on the hard drive. This parameter defines the number of pages the kernel tries to read from memory during each page fault. The value is interpreted as 2^page-cluster pages for each page fault. A page fault is encountered every time a virtual memory address is accessed for which there is not yet a corresponding physical page assigned or for which the corresponding physical page has been swapped to disk. If the memory address has been requested in a valid way (for example, the application contains the address in its virtual memory map), then the kernel associates a page of RAM with the address or retrieves the page from disk and places it back in RAM. Then the kernel restarts the application from where it left off. By increasing the page-cluster value, pages subsequent to the requested page are also retrieved, meaning that if the workload of a particular system accesses data in RAM in a linear fashion, increasing this parameter can provide significant performance gains (much like the file readahead parameters described earlier). Of course if your workload accesses data discreetly in many separate areas of memory, then this can just as easily cause performance degradation.

Example Scenarios

Now that we have covered the details of kernel tuning, let us look at some example workloads and the various tuning parameters that may improve system performance.

File (IMAP, Web, etc.) Server

This workload is geared towards performing a large amount of I/O to and from the local disk, thus benefiting from an adjustment allowing more files to be maintained in RAM. This speeds up I/O by caching more files in RAM and eliminating the need to wait for disk I/O to complete. A simple change to sysctl.conf as follows usually benefits this workload:

 #increase the amount of RAM pagecache is allowed to use  #before we start moving it back to disk  vm.pagecache="10 40 100"  

General Compute Server With Many Active Users

This workload is a very general type of configuration. It involves many active users who likely run many processes, all of which may or may not be CPU intensive or I/O intensive or a combination thereof. As the default VM configuration attempts to find a balance between I/O and process memory usage, it may be best to leave most configuration settings alone in this case. However, this environment likely contains many small processes which, regardless of workload, consume memory resources, particularly lowmem. It may help, therefore, to tune the VM to conserve low memory resources when possible:

 #lower the pagecache max to keep from eating all memory up with cache  vm.pagecache=10 25 50  #lower max-readahead to reduce the amount of unneeded IO  vm.max-readahead=16  

Non interactive (Batch) Computing Server

A batch computing server is usually the exact opposite of a file server. Applications run without human interaction, and they commonly perform with little I/O. The number of processes running on controlled. Consequently this system should allow maximum throughput:

 #Reduce the amount of pagecache normally allowed vm.pagecache="1 10 100" #do not worry about conserving lowmem, not that many processes vm.max_map_count=128000 14 #crank up overcommit, processes can sleep as they are not interactive vm.overcommit=2  vm.overcommit_ratio=75  

Further Reading

1. Understanding the Linux Kernel by Daniel Bovet and Marco Cesati (O'Reilly & Associates)

2. Virtual Memory Behavior in Red Hat Enterprise Linux AS 2.1 by Bob Matthews and Norm Murray

3. Towards an O(1) VM by Rik Van Riel

4. The Linux Kernel Source Tree, versions 2.4.21-4EL & 2.4.21-5EL

Cool Unix Commands

Cool Commands

Peter Baer Galvin

There are so many commands in Solaris that it is difficult to separate the cool ones from the mundane. For example, there are commands to report how much time a program spends in each system call, and commands to dynamically show system activities, and most of these commands are included with Solaris 8 as well as Solaris 9. This month, I’m highlighting some of the commands that you might find particularly useful.

Systems administrators are tool users. Through experience, we have learned that the more tools we have, the better able we are to diagnose problems and implement solutions. The commands included in this column are gleaned from experience, friends, acquaintances, and from attendance at the SunNetwork 2002 conference in September. “The /procodile Hunter” talk by Solaris kernel developers Brian Cantrill and Mike Shapiro was especially enlightening and frightening because Cantrill wrote code to illustrate a point faster than Shapiro could explain the point they were trying to illustrate!

Useful Solaris Commands

truss -c (Solaris >= 8): This astounding option to truss provides a profile summary of the command being trussed:

$ truss -c grep asdf work.doc  syscall              seconds   calls  errors  _exit                    .00       1  read                     .01      24  open                     .00       8      4  close                    .00       5  brk                      .00      15  stat                     .00       1  fstat                    .00       4  execve                   .00       1  mmap                     .00      10  munmap                   .01       3  memcntl                  .00       2  llseek                   .00       1  open64                   .00       1                          ----     ---    ---  sys totals:              .02      76      4  usr time:                .00  elapsed:                 .05

It can also show profile data on a running process. In this case, the data shows what the process did between when truss was started and when truss execution was terminated with a control-c. It’s ideal for determining why a process is hung without having to wade through the pages of truss output.

truss -d and truss -D (Solaris >= 8): These truss options show the time associated with each system call being shown by truss and is excellent for finding performance problems in custom or commercial code. For example:

$ truss -d who  Base time stamp:  1035385727.3460  [ Wed Oct 23 11:08:47 EDT 2002 ]   0.0000 execve(“/usr/bin/who”, 0xFFBEFD5C, 0xFFBEFD64)  argc = 1   0.0032 stat(“/usr/bin/who”, 0xFFBEFA98)                = 0   0.0037 open(“/var/ld/ld.config”, O_RDONLY)             Err#2 ENOENT   0.0042 open(“/usr/local/lib/libc.so.1”, O_RDONLY)      Err#2 ENOENT   0.0047 open(“/usr/lib/libc.so.1”, O_RDONLY)            = 3   0.0051 fstat(3, 0xFFBEF42C)                            = 0  . . .

truss -D is even more useful, showing the time delta between system calls:

Dilbert> truss -D who   0.0000 execve(“/usr/bin/who”, 0xFFBEFD5C, 0xFFBEFD64)  argc = 1   0.0028 stat(“/usr/bin/who”, 0xFFBEFA98)                = 0   0.0005 open(“/var/ld/ld.config”, O_RDONLY)             Err#2 ENOENT   0.0006 open(“/usr/local/lib/libc.so.1”, O_RDONLY)      Err#2 ENOENT   0.0005 open(“/usr/lib/libc.so.1”, O_RDONLY)            = 3   0.0004 fstat(3, 0xFFBEF42C)                            = 0

In this example, the stat system call took a lot longer than the others.

truss -T: This is a great debugging help. It will stop a process at the execution of a specified system call. (“-U” does the same, but with user-level function calls.) A core could then be taken for further analysis, or any of the /proc tools could be used to determine many aspects of the status of the process.

truss -l (improved in Solaris 9): Shows the thread number of each call in a multi-threaded processes. Solaris 9 truss -l finally makes it possible to watch the execution of a multi-threaded application.

Truss is truly a powerful tool. It can be used on core files to analyze what caused the problem, for example. It can also show details on user-level library calls (either system libraries or programmer libraries) via the “-u” option.

pkg-get: This is a nice tool (http://www.bolthole.com/solaris) for automatically getting freeware packages. It is configured via /etc/pkg-get.conf. Once it’s up and running, execute pkg-get -a to get a list of available packages, and pkg-get -i to get and install a given package.

plimit (Solaris >= 8): This command displays and sets the per-process limits on a running process. This is handy if a long-running process is running up against a limit (for example, number of open files). Rather than using limit and restarting the command, plimit can modify the running process.

coreadm (Solaris >= 8): In the “old” days (before coreadm), core dumps were placed in the process’s working directory. Core files would also overwrite each other. All this and more has been addressed by coreadm, a tool to manage core file creation. With it, you can specify whether to save cores, where cores should be stored, how many versions should be retained, and more. Settings can be retained between reboots by coreadm modifying /etc/coreadm.conf.

pgrep (Solaris >= 8): pgrep searches through /proc for processes matching the given criteria, and returns their process-ids. A great option is “-n”, which returns the newest process that matches.

preap (Solaris >= 9): Reaps zombie processes. Any processes stuck in the “z” state (as shown by ps), can be removed from the system with this command.

pargs (Solaris >= 9): Shows the arguments and environment variables of a process.

nohup -p (Solaris >= 9): The nohup command can be used to start a process, so that if the shell that started the process closes (i.e., the process gets a “SIGHUP” signal), the process will keep running. This is useful for backgrounding a task that should continue running no matter what happens around it. But what happens if you start a process and later want to HUP-proof it? With Solaris 9, nohup -p takes a process-id and causes SIGHUP to be ignored.

prstat (Solaris >= 8): prstat is top and a lot more. Both commands provide a screen’s worth of process and other information and update it frequently, for a nice window on system performance. prstat has much better accuracy than top. It also has some nice options. “-a” shows process and user information concurrently (sorted by CPU hog, by default). “-c” causes it to act like vmstat (new reports printed below old ones). “-C” shows processes in a processor set. “-j” shows processes in a “project”. “-L” shows per-thread information as well as per-process. “-m” and “-v” show quite a bit of per-process performance detail (including pages, traps, lock wait, and CPU wait). The output data can also be sorted by resident-set (real memory) size, virtual memory size, execute time, and so on. prstat is very useful on systems without top, and should probably be used instead of top because of its accuracy (and some sites care that it is a supported program).

trapstat (Solaris >= 9): trapstat joins lockstat and kstat as the most inscrutable commands on Solaris. Each shows gory details about the innards of the running operating system. Each is indispensable in solving strange happenings on a Solaris system. Best of all, their output is good to send along with bug reports, but further study can reveal useful information for general use as well.

vmstat -p (Solaris >= 8): Until this option became available, it was almost impossible (see the “se toolkit”) to determine what kind of memory demand was causing a system to page. vmstat -p is key because it not only shows whether your system is under memory stress (via the “sr” column), it also shows whether that stress is from application code, application data, or I/O. “-p” can really help pinpoint the cause of any mysterious memory issues on Solaris.

pmap -x (Solaris >= 8, bugs fixed in Solaris >= 9): If the process with memory problems is known, and more details on its memory use are needed, check out pmap -x. The target process-id has its memory map fully explained, as in:

# pmap -x 1779  1779:   -ksh   Address  Kbytes     RSS    Anon  Locked Mode   Mapped File  00010000     192     192       -       - r-x--  ksh  00040000       8       8       8       - rwx--  ksh  00042000      32      32       8       - rwx--    [ heap ]  FF180000     680     664       -       - r-x--  libc.so.1  FF23A000      24      24       -       - rwx--  libc.so.1  FF240000       8       8       -       - rwx--  libc.so.1  FF280000     568     472       -       - r-x--  libnsl.so.1  FF31E000      32      32       -       - rwx--  libnsl.so.1  FF326000      32      24       -       - rwx--  libnsl.so.1  FF340000      16      16       -       - r-x--  libc_psr.so.1  FF350000      16      16       -       - r-x--  libmp.so.2  FF364000       8       8       -       - rwx--  libmp.so.2  FF380000      40      40       -       - r-x--  libsocket.so.1  FF39A000       8       8       -       - rwx--  libsocket.so.1  FF3A0000       8       8       -       - r-x--  libdl.so.1  FF3B0000       8       8       8       - rwx--    [ anon ]  FF3C0000     152     152       -       - r-x--  ld.so.1  FF3F6000       8       8       8       - rwx--  ld.so.1  FFBFE000       8       8       8       - rw---    [ stack ]  -------- ------- ------- ------- -------  total Kb    1848    1728      40       -

Here we see each chunk of memory, what it is being used for, how much space it is taking (virtual and real), and mode information.

df -h (Solaris >= 9): This command is popular on Linux, and just made its way into Solaris. df -h displays summary information about file systems in human-readable form:

$ df -h  Filesystem             size   used  avail capacity  Mounted on  /dev/dsk/c0t0d0s0      4.8G   1.7G   3.0G    37%    /  /proc                    0K     0K     0K     0%    /proc  mnttab                   0K     0K     0K     0%    /etc/mnttab  fd                       0K     0K     0K     0%    /dev/fd  swap                   848M    40K   848M     1%    /var/run  swap                   849M   1.0M   848M     1%    /tmp  /dev/dsk/c0t0d0s7       13G    78K    13G     1%    /export/home

Conclusion

Each administrator has a set of tools used daily, and another set of tools to help in a pinch. This column included a wide variety of commands and options that are lesser known, but can be very useful. Do you have favorite tools that have saved you in a bind? If so, please send them to me so I can expand my tool set as well. Alternately, send along any tools that you hate or that you feel are dangerous, which could also turn into a useful column!

HOWTO: Mirrored root disk on Solaris

http://www.brandonhutchinson.com/Mirroring_disks_with_DiskSuite.html 0. Partition the first disk # format c0t0d0 Use the partition tool (=> "p <enter>, p <enter>"!) to setup the slices. We assume the following slice setup afterwards: # Tag Flag Cylinders Size Blocks - ---------- ---- ------------- -------- -------------------- 0 root wm 0 - 812 400.15MB (813/0/0) 819504 1 swap wu 813 - 1333 256.43MB (521/0/0) 525168 2 backup wm 0 - 17659 8.49GB (17660/0/0) 17801280 3 unassigned wm 1334 - 1354 10.34MB (21/0/0) 21168 4 var wm 1355 - 8522 3.45GB (7168/0/0) 7225344 5 usr wm 8523 - 14764 3.00GB (6242/0/0) 6291936 6 unassigned wm 14765 - 16845 1.00GB (2081/0/0) 2097648 7 home wm 16846 - 17659 400.15MB (813/0/0) 819504 1. Copy the partition table of the first disk to its future mirror disk # prtvtoc /dev/rdsk/c0t0d0s2 fmthard -s - /dev/rdsk/c0t1d0s2 2. Create at least two state database replicas on each disk # metadb -a -f -c 2 c0t0d0s3 c0t1d0s3 Check the state of all replicas with metadb: # metadb Notes: A state database replica contains configuration and state information about the meta devices. Make sure that always at least 50% of the replicas are active! 3. Create the root slice mirror and its first submirror # metainit -f d10 1 1 c0t0d0s0 # metainit -f d20 1 1 c0t1d0s0 # metainit d30 -m d10 Run metaroot to prepare /etc/vfstab and /etc/system (do this only for the root slice!): # metaroot d30 4. Create the swap slice mirror and its first submirror # metainit -f d11 1 1 c0t0d0s1 # metainit -f d21 1 1 c0t1d0s1 # metainit d31 -m d11 5. Create the var slice mirror and its first submirror # metainit -f d14 1 1 c0t0d0s4 # metainit -f d24 1 1 c0t1d0s4 # metainit d34 -m d14 6. Create the usr slice mirror and its first submirror # metainit -f d15 1 1 c0t0d0s5 # metainit -f d25 1 1 c0t1d0s5 # metainit d35 -m d15 7. Create the unassigned slice mirror and its first submirror # metainit -f d16 1 1 c0t0d0s6 # metainit -f d26 1 1 c0t1d0s6 # metainit d36 -m d16 8. Create the home slice mirror and its first submirror # metainit -f d17 1 1 c0t0d0s7 # metainit -f d27 1 1 c0t1d0s7 # metainit d37 -m d17 9. Edit /etc/vfstab to mount all mirrors after boot, including mirrored swap /etc/vfstab before changes: fd - /dev/fd fd - no - /proc - /proc proc - no - /dev/dsk/c0t0d0s1 - - swap - no - /dev/md/dsk/d30 /dev/md/rdsk/d30 / ufs 1 no logging /dev/dsk/c0t0d0s5 /dev/rdsk/c0t0d0s5 /usr ufs 1 no ro,logging /dev/dsk/c0t0d0s4 /dev/rdsk/c0t0d0s4 /var ufs 1 no nosuid,logging /dev/dsk/c0t0d0s7 /dev/rdsk/c0t0d0s7 /home ufs 2 yes nosuid,logging /dev/dsk/c0t0d0s6 /dev/rdsk/c0t0d0s6 /opt ufs 2 yes nosuid,logging swap - /tmp tmpfs - yes - /etc/vfstab after changes: fd - /dev/fd fd - no - /proc - /proc proc - no - /dev/md/dsk/d31 - - swap - no - /dev/md/dsk/d30 /dev/md/rdsk/d30 / ufs 1 no logging /dev/md/dsk/d35 /dev/md/rdsk/d35 /usr ufs 1 no ro,logging /dev/md/dsk/d34 /dev/md/rdsk/d34 /var ufs 1 no nosuid,logging /dev/md/dsk/d37 /dev/md/rdsk/d37 /home ufs 2 yes nosuid,logging /dev/md/dsk/d36 /dev/md/rdsk/d36 /opt ufs 2 yes nosuid,logging swap - /tmp tmpfs - yes - Notes: The entry for the root device (/) has already been altered by the metaroot command we executed before. 10. Reboot the system # lockfs -fa && init 6 11. Attach the second submirrors to all mirrors # metattach d30 d20 # metattach d31 d21 # metattach d34 d24 # metattach d35 d25 # metattach d36 d26 # metattach d37 d27 Notes: This will finally cause the data from the boot disk to be synchronized with the mirror drive. You can use metastat to track the mirroring progress. 12. Change the crash dump device to the swap metadevice # dumpadm -d `swap -l tail -1 awk '{print $1}' 13. Make the mirror disk bootable # installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c0t1d0s0 Notes: This will install a boot block to the second disk. 14. Determine the physical device path of the mirror disk # ls -l /dev/dsk/c0t1d0s0 ... /dev/dsk/c0t1d0s0 -> ../../devices/pci@1f,4000/scsi@3/sd@1,0:a 15. Create a device alias for the mirror disk # eeprom "nvramrc=devalias mirror /pci@1f,4000/scsi@3/disk@1,0" # eeprom "use-nvramrc?=true" Add the mirror device alias to the Open Boot parameter boot-device to prepare the case of a problem with the primary boot device. # eeprom "boot-device=disk mirror cdrom net" You can also configure the device alias and boot-device list from the Open Boot Prompt (OBP a.k.a. ok prompt): ok nvalias mirror /pci@1f,4000/scsi@3/disk@1,0 ok use-nvramrc?=true ok boot-device=disk mirror cdrom net Notes: From the OBP, you can use boot mirror to boot from the mirror disk. On my test system, I had to replace sd@1,0:a with disk@1,0. Use devalias on the OBP prompt to determine the correct device path.

Active FTP vs. Passive FTP, a Definitive Explanation

Active FTP vs. Passive FTP, a Definitive Explanation

http://slacksite.com/other/ftp.html

Introduction

One of the most commonly seen questions when dealing with firewalls and
other Internet connectivity issues is the difference between active and
passive FTP and how best to support either or both of them. Hopefully
the following text will help to clear up some of the confusion over how
to support FTP in a firewalled environment.

This may not be the definitive explanation, as the title claims,
however, I've heard enough good feedback and seen this document linked
in enough places to know that quite a few people have found it to be
useful. I am always looking for ways to improve things though, and if
you find something that is not quite clear or needs more explanation,
please let me know! Recent additions to this document include the
examples of both active and passive command line FTP sessions. These
session examples should help make things a bit clearer. They also
provide a nice picture into what goes on behind the scenes during an FTP
session. Now, on to the information...

The Basics

FTP is a TCP based service exclusively. There is no UDP component to
FTP. FTP is an unusual service in that it utilizes two ports, a 'data'
port and a 'command' port (also known as the control port).
Traditionally these are port 21 for the command port and port 20 for the
data port. The confusion begins however, when we find that depending on
the mode, the data port is not always on port 20.

Active FTP

In active mode FTP the client connects from a random unprivileged port
(N > 1023) to the FTP server's command port, port 21. Then, the client
starts listening to port N+1 and sends the FTP command PORT N+1 to the
FTP server. The server will then connect back to the client's specified
data port from its local data port, which is port 20.

>From the server-side firewall's standpoint, to support active mode FTP
the following communication channels need to be opened:

FTP server's port 21 from anywhere (Client initiates
connection)

FTP server's port 21 to ports > 1023 (Server responds to
client's control port)

FTP server's port 20 to ports > 1023 (Server initiates
data connection to client's data port)

FTP server's port 20 from ports > 1023 (Client sends
ACKs to server's data port)

When drawn out, the connection appears as follows:

<outbind://235/images/ftp/activeftp.gif>

In step 1, the client's command port contacts the server's command port
and sends the command PORT 1027. The server then sends an ACK back to
the client's command port in step 2. In step 3 the server initiates a
connection on its local data port to the data port the client specified
earlier. Finally, the client sends an ACK back as shown in step 4.

The main problem with active mode FTP actually falls on the client side.
The FTP client doesn't make the actual connection to the data port of
the server--it simply tells the server what port it is listening on and
the server connects back to the specified port on the client. From the
client side firewall this appears to be an outside system initiating a
connection to an internal client--something that is usually blocked.

Active FTP Example

Below is an actual example of an active FTP session. The only things
that have been changed are the server names, IP addresses, and user
names. In this example an FTP session is initiated from
testbox1.slacksite.com (192.168.150.80), a linux box running the
standard FTP command line client, to testbox2.slacksite.com
(192.168.150.90), a linux box running ProFTPd 1.2.2RC2. The debugging
(-d) flag is used with the FTP client to show what is going on behind
the scenes. Everything in red is the debugging output which shows the
actual FTP commands being sent to the server and the responses generated
from those commands. Normal server output is shown in black, and user
input is in bold.

There are a few interesting things to consider about this dialog. Notice
that when the PORT command is issued, it specifies a port on the client
(192.168.150.80) system, rather than the server. We will see the
opposite behavior when we use passive FTP. While we are on the subject,
a quick note about the format of the PORT command. As you can see in the
example below it is formatted as a series of six numbers separated by
commas. The first four octets are the IP address while the last two
octets comprise the port that will be used for the data connection. To
find the actual port multiply the fifth octet by 256 and then add the
sixth octet to the total. Thus in the example below the port number is (
(14*256) + 178), or 3762. A quick check with netstat should confirm this
information.

testbox1: {/home/p-t/slacker/public_html} % ftp -d testbox2
Connected to testbox2.slacksite.com.
220 testbox2.slacksite.com FTP server ready.
Name (testbox2:slacker): slacker
---> USER slacker
331 Password required for slacker.
Password: TmpPass
---> PASS XXXX
230 User slacker logged in.
---> SYST
215 UNIX Type: L8
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
ftp: setsockopt (ignored): Permission denied
---> PORT 192,168,150,80,14,178
200 PORT command successful.
---> LIST
150 Opening ASCII mode data connection for file list.
drwx------ 3 slacker users 104 Jul 27 01:45 public_html
226 Transfer complete.
ftp> quit
---> QUIT
221 Goodbye.

Passive FTP

In order to resolve the issue of the server initiating the connection to
the client a different method for FTP connections was developed. This
was known as passive mode, or PASV, after the command used by the client
to tell the server it is in passive mode.

In passive mode FTP the client initiates both connections to the server,
solving the problem of firewalls filtering the incoming data port
connection to the client from the server. When opening an FTP
connection, the client opens two random unprivileged ports locally (N >
1023 and N+1). The first port contacts the server on port 21, but
instead of then issuing a PORT command and allowing the server to
connect back to its data port, the client will issue the PASV command.
The result of this is that the server then opens a random unprivileged
port (P > 1023) and sends the PORT P command back to the client. The
client then initiates the connection from port N+1 to port P on the
server to transfer data.

>From the server-side firewall's standpoint, to support passive mode FTP
the following communication channels need to be opened:

FTP server's port 21 from anywhere (Client initiates
connection)

FTP server's port 21 to ports > 1023 (Server responds to
client's control port)

FTP server's ports > 1023 from anywhere (Client
initiates data connection to random port specified by server)

FTP server's ports > 1023 to remote ports > 1023 (Server
sends ACKs (and data) to client's data port)

When drawn, a passive mode FTP connection looks like this:

<outbind://235/images/ftp/passiveftp.gif>

In step 1, the client contacts the server on the command port and issues
the PASV command. The server then replies in step 2 with PORT 2024,
telling the client which port it is listening to for the data
connection. In step 3 the client then initiates the data connection from
its data port to the specified server data port. Finally, the server
sends back an ACK in step 4 to the client's data port.

While passive mode FTP solves many of the problems from the client side,
it opens up a whole range of problems on the server side. The biggest
issue is the need to allow any remote connection to high numbered ports
on the server. Fortunately, many FTP daemons, including the popular
WU-FTPD allow the administrator to specify a range of ports which the
FTP server will use. See Appendix 1 <outbind://235/ftp-appendix1.html>
for more information.

The second issue involves supporting and troubleshooting clients which
do (or do not) support passive mode. As an example, the command line FTP
utility provided with Solaris does not support passive mode,
necessitating a third-party FTP client, such as ncftp.

With the massive popularity of the World Wide Web, many people prefer to
use their web browser as an FTP client. Most browsers only support
passive mode when accessing ftp:// URLs. This can either be good or bad
depending on what the servers and firewalls are configured to support.

Passive FTP Example

Below is an actual example of a passive FTP session. The only things
that have been changed are the server names, IP addresses, and user
names. In this example an FTP session is initiated from
testbox1.slacksite.com (192.168.150.80), a linux box running the
standard FTP command line client, to testbox2.slacksite.com
(192.168.150.90), a linux box running ProFTPd 1.2.2RC2. The debugging
(-d) flag is used with the FTP client to show what is going on behind
the scenes. Everything in red is the debugging output which shows the
actual FTP commands being sent to the server and the responses generated
from those commands. Normal server output is shown in black, and user
input is in bold.

Notice the difference in the PORT command in this example as opposed to
the active FTP example. Here, we see a port being opened on the server
(192.168.150.90) system, rather than the client. See the discussion
about the format of the PORT command above, in the Active FTP Example
section <outbind://235/actexample> .

testbox1: {/home/p-t/slacker/public_html} % ftp -d testbox2
Connected to testbox2.slacksite.com.
220 testbox2.slacksite.com FTP server ready.
Name (testbox2:slacker): slacker
---> USER slacker
331 Password required for slacker.
Password: TmpPass
---> PASS XXXX
230 User slacker logged in.
---> SYST
215 UNIX Type: L8
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> passive
Passive mode on.
ftp> ls
ftp: setsockopt (ignored): Permission denied
---> PASV
227 Entering Passive Mode (192,168,150,90,195,149).
---> LIST
150 Opening ASCII mode data connection for file list
drwx------ 3 slacker users 104 Jul 27 01:45 public_html
226 Transfer complete.
ftp> quit
---> QUIT
221 Goodbye.

Other Notes

A reader, Maarten Sjouw, pointed out that active FTP will not function
when used in conjunction with a client-side NAT (Network Address
Translation) device which is not smart enough to alter the IP address
info in FTP packets.

Summary

The following chart should help admins remember how each FTP mode works:

Active FTP :
command : client >1023 -> server 21
data : client >1023 <- server 20

Passive FTP :
command : client >1023 -> server 21
data : client >1023 -> server >1023

A quick summary of the pros and cons of active vs. passive FTP is also
in order:

Active FTP is beneficial to the FTP server admin, but detrimental to the
client side admin. The FTP server attempts to make connections to random
high ports on the client, which would almost certainly be blocked by a
firewall on the client side. Passive FTP is beneficial to the client,
but detrimental to the FTP server admin. The client will make both
connections to the server, but one of them will be to a random high
port, which would almost certainly be blocked by a firewall on the
server side.

Luckily, there is somewhat of a compromise. Since admins running FTP
servers will need to make their servers accessible to the greatest
number of clients, they will almost certainly need to support passive
FTP. The exposure of high level ports on the server can be minimized by
specifying a limited port range for the FTP server to use. Thus,
everything except for this range of ports can be firewalled on the
server side. While this doesn't eliminate all risk to the server, it
decreases it tremendously. See Appendix 1
<outbind://235/ftp-appendix1.html> for more information.

References

An excellent reference on how various internet protocols work and the
issues involved in firewalling them can be found in the O'Reilly and
Associates book, Building Internet Firewalls, 2nd Ed, by Brent Chapman
and Elizabeth Zwicky.

Finally, the definitive reference on FTP would be RFC 959, which sets
forth the official specifications of the FTP protocol. RFCs can be
downloaded from numerous locations, including
http://www.faqs.org/rfcs/rfc959.html.

Configuring TCP/IP on Solaris - Overview of the Booting Processes

The Boot Process

The following information is provided for your reference. It is a brief overview of the network booting processes to help you better visualize what is happening during configuration.

Note - The names of startup scripts might change from one release to another.

You start the operating system on a host.

The kernel runs /sbin/init, as part of the booting process.

/sbin/init runs the /etc/rcS.d/S30rootusr.sh. startup script.

The script runs a number of system startup tasks, including establishing the minimum host and network configurations for diskless and dataless operations. /etc/rcS.d/S30rootusr.sh also mounts the /usr file system.

If the local database files contain the required configuration information (host name and IP address), the script uses it.

If the information is not available in local host configuration files, /etc/rcS.d/S30rootusr.sh uses RARP to acquire the host's IP address.

If the local files contain domain name, host name, and default router address, the machine uses them. If the configuration information is not in local files, then the system uses the Bootparams protocol to acquire the host name, domain name, and default router address. Note that the required information must be available on a network configuration server that is located on the same network as the host. This is necessary because no internetwork communications exist at this point.

After /etc/rcS/S30rootusr.sh completes its tasks and several other boot procedures have executed, /etc/rc2.d/S69inet runs. This script executes startup tasks that must be completed before the name services (NIS, NIS+, or DNS) can start. These tasks include configuring the IP routing and setting the domain name.

At completion of the S69inet tasks, /etc/rc2.d/S71rpc runs. This script starts the NIS, NIS+, or DNS name service.

After /etc/rc2.d/S71 runs, /etc/rc2.d/S72inetsvc runs. This script starts up services that depend on the presence of the name services. S72inetsvc also starts the daemon inetd, which manages user services such as telnet.

Configuring TCP/IP on Solaris - Network Configuration Procedures

Introduction

Network software installation takes place along with the installation of the operating system software. At that time, certain IP configuration parameters must be stored in appropriate files so they can be read at boot time.

The procedure is simply a matter of creating or editing the network- configuration files. How configuration information is made available to a machine's kernel depends on whether these files are stored locally (local files mode) or acquired from the network configuration server (network client mode). Parameters supplied during network configuration are:

IP address of each network interface on every machine

Host names of each machine on the network. You can type the host name in a local file or a name service database.

NIS, NIS+, or DNS domain name in which the machine resides, if applicable

Default router addresses. You supply this only if you have a simple network topology with only one router attached to each network, or your routers don’t run routing protocols such as the Router Discovery Server Protocol (RDISC) or the Router Information Protocol (RIP). (See Chapter 5 for more information about these protocols.)

Subnet mask (required only for networks with subnets)

This section contains complete information on creating and editing local configuration files.

How to Configure a Host for Local Files Mode

Use this procedure for configuring TCP/IP on a machine that run in local files mode.

Become superuser and change to the /etc directory.

Type the host name of the machine in the file /etc/nodename. For example, if the name of the host is tenere, type tenere in the file.

Create a file named /etc/hostname.interface for each network interface. (The Solaris installation program automatically creates this file for the primary network interface.)

Type either the interface IP address or the interface name in each /etc/hostname.interface file. For example, create a file named hostname.ie1, and type either the IP address of the host's interface or the host's name.

Edit the /etc/inet/hosts file to add:

IP addresses that you have assigned to any additional network interfaces in the local machine, along with the corresponding host name for each interface. The Solaris installation program will already have created entries for the primary network interface and loopback address.

IP address or addresses of the file server, if the /usr file system is NFS mounted.

Type the host's fully qualified domain name in the /etc/defaultdomain file. For example, suppose host tenere was part of the domain deserts.worldwide.com. Therefore, you would type: deserts.worldwide.com in /etc/defaultdomain.

Type the router's name in /etc/defaultrouter.

Type the name of the default router and its IP addresses in /etc/inet/hosts. Additional routing options are available. You can apply these options to a local files mode configuration.

If your network is subnetted, type the network number and the netmask in the file /etc/inet/netmasks. If you have set up a NIS or NIS+ server, you can type netmask information in the appropriate database on the server as long as server and clients are on the same network.

Reboot each machine on the network.

Setting Up a Network Configuration Server

If you plan to configure certain hosts as network clients, you must configure at least one machine on your network as a network configuration server. (Refer to “Network Configuration Servers” on page 45 for an introduction.) Setting up a network configuration server involves:

Turning on the network configuration daemons:
- in.tftpd
- in.rarpd
- rpc.bootparamd

Editing and maintaining the network configuration files on the configuration server.

How to Set Up a Network Configuration Server

Become superuser and change to the root directory of the prospective network configuration server.

Turn on the in.tftpd daemon by creating the directory /tftpboot:

# mkdir /tftpboot

This configures the machine as a TFTP, bootparams, and RARP server.

Create a symbolic link to the directory.

# ln -s /tftpboot/. /tftpboot/tftpboot

Enable the tftp line in intetd.conf. Check that the /etc/inetd.conf entry reads:

tftp dgram udp wait root /usr/sbin/in.tftpd in.tftpd -s /tftpboot

This prevents inettftpd() from retrieving any file other than one located in /tftpboot.

Edit the hosts database, and add the host names and IP addresses for every client on the network.

Edit the ethers database, and create entries for every host on the network to run in network client mode.

Edit the bootparams database. Use the wildcard entry or create an entry for every host that run in network client mode.

Reboot the server.

Configuring Network Clients

Network clients receive their configuration information from network configuration servers. Therefore, before you configure a host as a network client you must ensure that at least one network configuration server is set up for the network.

How to Configure Hosts for Network Client Mode

Do the following on each host to be configured in network client mode:

Become superuser.

Check the directory for the existence of an /etc/nodename file. If one exists, delete it.

Eliminating /etc/nodename causes the system to use the hostconfig program to obtain the host name, domain name, and router addresses from the network configuration server.

Create the file /etc/hostname.interface, if it does not exist. Make sure that the file is empty. An empty /etc/hostname.interface file causes the system to acquire the IP address from the network configuration server.

Ensure that the /etc/inet/hosts file contains only the host name and IP address of the loopback network interface. The file should not contain the IP address and host name for the local machine (primary network interface). EXCEPTION: For a diskless client (a machine with an NFS-mounted root file system), type the name and IP address of the server that provides the client's root file system (usually, but not always, the network configuration server).

Check for the existence of an /etc/defaultdomain file. If one exists, delete it.

The hostconfig program sets the domain name automatically. If you want to override the domain name set by hostconfig, type the substitute domain name in the file /etc/defaultdomain.

Ensure that the search paths in the client's /etc/nsswitch.conf reflects the name service requirements for your network.

How to Specify a Router for the Network Client

If you have only one router on the network and you want the network configuration server to specify its name automatically, ensure that the network client does not have a /etc/defaultrouter file.

To override the name of the default router provided by the network configuration server:

Create /etc/defaultrouter on the network client.

Type the host name and IP address of the machine you have designated as the default router.

Add the host name and IP address of the designated default router to the network client's /etc/inet/hosts.

If you have multiple routers on the network, create /etc/defaultrouter on the network client, but leave it empty.

Creating /etc/defaultrouter and leaving it empty causes one of the two dynamic routing protocols to run: ICMP Router Discovery protocol (RDISC), or Routing Information Protocol (RIP). The system first runs the program in.rdisc, which looks for routers that are running the router discovery protocol. If it finds one such router, in.rdisc continues to run and keeps track of the routers that are running the RDISC protocol.

If the system discovers that routers are not responding to the RDISC protocol, it uses RIP and runs the daemon in.routed to keep track of them.

After Installing a Network Client

After you have finished editing the files on each network client machine, do the following on the network configuration server.

Add entries for the hosts in the ethers and hosts databases.

Add entries for the hosts to the bootparams database. To simplify matters, you can type a wild card in the bootparams database in place of individual entries for each host.

Reboot the server.

Configuring TCP/IP on Solaris - Network Databases and nsswitch.conf File

Introduction

The network databases are files that provide information needed to configure the network. The network databases are:

hosts

netmasks

ethers

bootparams

protocols

services

networks

As part of the configuration process, you edit the hosts database and the netmasks database, if your network is subnetted. Two network databases, bootparams and ethers, are used to configure machines as network clients. The remaining databases are used by the operating system and seldom require editing.

Although it is not a network database, the nsswitch.conf file needs to be configured along with the relevant network databases. nsswitch.conf specifies which name service to use for a particular machine: NIS, NIS+, DNS, or local files.

How Name Services Affect Network Databases

Your network database takes a form that depends on the type of name service you select for your network. For example, the hosts database contains, at minimum, the host name and IP address of the local machine and any network interfaces directly connected to the local machine. However, the hosts database could contain other IP addresses and host names, depending on the type of name service on your network.

The network databases are used as follows:

Networks that use local files for their name service rely on files in the /etc/inet and /etc directories

NIS+ uses databases called NIS+ tables

NIS uses databases called NIS maps

DNS uses records with host information

Note - DNS boot and data files do not correspond directly to the network databases.

Network Database	Local Files	NIS+ Tables	NIS Maps
hosts	/etc/inet/hosts	hosts.ord_dir	hosts.byaddr hosts.byname
netmasks	/etc/inet/netmasks	netmasks.ord_dir	netmasks.byaddr
ethers	/etc/ethers	ethers.ord_dir	ethers.byname ethers.byaddr
bootparams	/etc/bootparams	bootparams.ord_dir	bootparams
protocols	/etc/inet/protocols	protocols.ord_dir	protocols.byname protocols.bynumber
services	/etc/inet/services	services.ord_dir	services.byname
networks	/etc/inet/networks	networks.ord_dir	networks.byaddr networks.byname

nsswitch.conf File - Specifying Which Name Service to Use

The /etc/nsswitch.conf file defines the search order of the network databases. The Solaris installation program creates a default /etc/nsswitch.conf file for the local machine, based on the name service you indicate during the installation process. If you selected the 'None' option, indicating local files for name service, the resulting nsswitch.conf file resembles the following example:

nsswitch.conf for Networks Using Files for Name Service

# /etc/nsswitch.files: # # An example file that could be copied over to /etc/nsswitch.conf; # it does not use any naming service. # # "hosts:" and "services:" in this file are used only if the # /etc/netconfig file contains "switch.so" as a # nametoaddr library for "inet" transports. passwd: files group: files hosts: files networks: files protocols: files rpc: files ethers: files netmasks: files bootparams: files publickey: files # At present there isn't a 'files' backend for netgroup; the # system will figure it out pretty quickly, # and won't use netgroups at all. netgroup: files automount: files aliases: files services: files sendmailvars: files

The nsswitch.conf(4) man page describes the file in detail. Its basic syntax is:

database name-service-to-search

The database field can list one of many types of databases searched by the operating system. For example, it could indicate a database affecting users, such as passwd or aliases, or a network database. The parameter name-service-to-search can have the values files, nis, or nis+ for the network databases. (The hosts database can also have dns as a name service to search.) You can also list more than one name service, such as nis+ and files.

In the above example, the only search option indicated is files. Therefore, the local machine gets security and automounting information, in addition to network database information, from files located in its /etc and /etc/inet directories.

Changing nsswitch.conf

The /etc directory contains the nsswitch.conf file created by the Solaris installation program. It also contains template files for the following name services:

nsswitch.files

nsswitch.nis

nsswitch.nis+

If you want to change from one name service to another, you can copy the appropriate template to nsswitch.conf. You can also selectively edit the nsswitch.conf file, and change the default name service to search for individual databases.

For example, on a network running NIS, you might have to change the nsswitch.conf file on diskless clients. The search path for the bootparams and ethers databases must list files as the first option, and nis. The example below shows the correct search paths.

nsswitch.conf for a Diskless Client on a Network Running NIS

## /etc/nsswitch.conf:# . . passwd: files nis group: file nis # consult /etc "files" only if nis is down. hosts: nis [NOTFOUND=return] files networks: nis [NOTFOUND=return] files protocols: nis [NOTFOUND=return] files rpc: nis [NOTFOUND=return] files ethers: files [NOTFOUND=return] nis netmasks: nis [NOTFOUND=return] files bootparams: files [NOTFOUND=return] nis publickey: nis netgroup: nis automount: files nis aliases: files nis # for efficient getservbyname() avoid nis services: files nis sendmailvars: files

bootparams Database

The bootparams database contains information used by diskless clients and machines configured to boot in the network client mode. You need to edit it if your network will have network clients. The database is built from information entered into the /etc/bootparams file.

The bootparams(4) man page contains complete syntax for this database. Its basic syntax is:

    machine-name file-key-server-name:pathname

For each diskless or network client machine, the entry might contain the following information: the name of the client, a list of keys, the names of servers, and path names.

The first item of each entry is the name of the client machine. Next is a list of keys, names of servers, and path names, separated by tab characters. All items but the first are optional. The database can contain a wildcard entry that will be matched by all clients. Here is an example:

bootparams Database

myclient root=myserver : /nfsroot/myclient \ swap=myserver : /nfsswap//myclient \ dump=myserver : /nfsdump/myclient

In this example the term dump=: tells diskless hosts not to look for a dump file.

Wildcard Entry for bootparams
In most cases, you will want to use the wildcard entry when editing the bootparams database to support diskless clients. This entry is:

    * root=server:/path dump=:

The asterisk (*) wildcard indicates that this entry applies to all clients not specifically named within the bootparams database.

ethers Database

The ethers database is built from information entered into the /etc/ethers file. It associates host names to their Ethernet addresses. You need to create an ethers database only if you are running the RARP daemon; that is, if you are configuring network clients or diskless machines.

RARP uses the file to map Ethernet addresses to IP addresses. If you are running the RARP daemon in.rarpd, you need to set up the ethers file and maintain it on all hosts running the daemon to reflect changes to the network.

The ethers(4) man page contains complete syntax information for this database. Its basic format is:

Ethernet-address hostname #comment
Ethernet-address is the Ethernet address of the host.
hostname is the official name of the host.
#comment is any kind of note you want to append to an entry in the file.

The equipment manufacturer provides the Ethernet address. If a machine does not display the Ethernet address when you power up, see your hardware manuals for assistance.

When adding entries to the ethers database, make sure that host names correspond to the primary names in the hosts database, not to the nicknames, as shown in the following example:

Entries in the ethers Database

8:0:20:1:40:16 fayoum 8:0:20:1:40:15 nubian 8:0:20:1:40:7 sahara # This is a comment 8:0:20:1:40:14 tenere

Other Network Databases

The remaining network databases seldom need to be edited.

networks database Database

The networks database associates network names with network numbers, enabling some applications to use and display names rather than numbers. The networks database is based on information in the /etc/inet/networks file. It contains the names of all networks to which your network connects via routers.

The Solaris installation program sets up the initial networks database. The only time you need to update it is when you add a new network to your existing network topology.

The networks(4) man page contains full syntax information for /etc/inet/networks. Here is its basic format:

  network-name network-number nickname(s) # comment   network-name is the official name for the network.   network-number is the number assigned by the InterNIC.   nickname is any other name by which the network is known.   #comment is any kind of note you want to append to an entry in the file.

It is particularly important that you maintain the networks file. The netstat program uses the information in this database to produce status tables. The example shows a sample /etc/networks file:

/etc/networks File

#ident "@(#)networks 1.4 92/07/14 SMI" /* SVr4.0 1.1 */ # # The networks file associates Internet Protocol (IP) network numbers with network names. The format of this file is: # # network-name network-number nicnames . . . # The loopback network is used only for intra-machine communication #loopback 127 # Internet networks # arpanet 10 arpa # Historical ucb-ether 46 ucbether # # local networks eng 193.9.0 #engineering acc 193.9.1 #accounting prog 193.9.2 #programming

protocols Database

The protocols database lists the TCP/IP protocols installed on your system and their numbers; the Solaris installation program automatically creates it. It is rare when this file requires administrative handling.

The protocols database contains the names of the TCP/IP protocols installed on the system. Its syntax is completely described in the protocols(4) man page. The example below shows an example of the /etc/inet/protocols file:

/etc/inet/protocols File

# # Internet (IP) protocols # ip 0 IP # internet protocol, pseudo protocol number icmp 1 ICMP # internet control message protocol tcp 6 TCP # transmission control protocol udp 17 UDP # user datagram protocol

services Database

The services database lists the names of TCP and UDP services and their well known port numbers; it is used by programs that call network services. The Solaris installation automatically creates the services database; it generally requires no administrative handling.

The services(4) man page contains complete syntax information. The example below shows an excerpt from a typical /etc/inet/services file:

/etc/inet/services File

# # Network services # echo 7/udp echo 7/tcp discard 9/udp sink null discard 11/tcp daytime 13/udp daytime 13/tcp netstat 15/tcp ftp-data 20/tcp ftp 21/tcp telnet 23/tcp time 37/tcp timeserver time 37/udp timeserver name 42/udp nameserver whois 43/tcp nickname