Learning Solaris 10

Section 1 : General

1.1 : Do I need special hardware for running Zones ?
No, Zones is a software feature of Solaris 10 and as such is available on all supported Hardware compatible with Solaris 10.

1.2 : Which applications are supported to run on Zones ?
The ISV is responsible of supporting its software on Zones. Technically speaking, many applications will be able to run inside zones without any modification. Some special cases do exist, like those explained in Section 3.

1.3 What about license costs if I run my application in a Zone on a specific number of CPUs?
It is possible to give one Zone exclusive access to a number of CPUs, through the ressource pools functionality ( see section 5 ). Sun is pushing so that ISVs would base their licensing cost on the amount of CPUs assigned to the Zone, as opposed to the total number of CPUs.
This Oracle document officially recognizes a Zone as hardware partitioning technology, much like a Solaris Domain. If your zone is bound to a 3 CPUs ressource pool, Oracle only requires a 3 CPU license.

1.4 Can I run different Solaris releases in different zones?
No. There is only one Solaris Kernel running, in the Global Zone. All execution environments created in zones rely on that unique kernel.

1.5 Can I use zones to test patches?
It depends. If it is a Solaris patch, then no as it will be installed on all zones anyway. If it is a non-global zone patch ( like an Apache patch for instance ) then it would be possible to create a second zone on the same machine that would be identical to the first one and used for test purposes. See Question 3.3 for more information on patch management with zones.

1.6 : Would there be a reason to use zones even if I want to run only ONE workload on my Solaris server?
Absolutely! For security reasons, run your workload in one non-global zone. The security barriers built around it make sure that any security leak that would allow an intruder to hack the non-global zone will not allow him to take control of the Global zone ie the server. If you were careful enough to use some defensive technique in the Global zone, like for instance some intrusion detection tool(s), you will then be able to watch what the attacker is doing while he is not able to hide from you.

1.7 : Can I run Linux applications inside a Zone?
Sun announced that it will provide support to companies that want to run applications on Red Hat Advanced Server 3 inside a Zone. This is the output of project “Janus” which allows Linux apps to run inside Solaris WITHOUT any modification. Refer to the official Sun SCLA page. Other Unix-lile distributions might be supported in the future through a follow-up more ambitious project : BrandZ. See Question 1.8.

1.8 : What is a “Branded Zone” (BrandZ)?
From the BrandZ project page :

“BrandZ is a framework that extends the Solaris Zones infrastructure to create Branded Zones, which are zones that contain non-native operating environments. The term “non-native” is intentionally vague, as the infrastructure allows for the creation of a wide range of operating environments.
(…)
The lx brand enables Linux binary applications to run unmodified on Solaris, within zones running a complete Linux userspace. The combination of BrandZ and the lx brand will be productized as Solaris Containers for Linux Applications.”
This project is still a work-in-progress.

1.9 : Can I install Zones into a really minimal system?
Currently, Zone packages, “SUNWzoner” & “SUNWzoneu”, have got some heavy dependencies. In particular, Java JRE & the X subsystem have to be installed for the Zone packages to get installed. The bug is solved in Solaris Express 02/06, which means that there is hope for that change to be incorporated into the next update of Solaris 10.

=================================================================================

Section 2 : Creation – Configuration

2.1 : What are these four “add-inherit-pkg-dir” in my zone configuration and may I remove them?
Absolutely. These are there because by default, Solaris wants the non-global zone and the Global zone to share the text segments from the executables and shared libraries that are part of the 4 “add-inherit-pkg-dir” : /usr, /platform, /sbin, /lib. These 4 directories are loopback mounted from the Global zone into your non-global zone in read-only mode. The other advantages of this technique are the smaller disk footprint needed for the non-global zone and possibly the speed of the non-global zone installation, since less packages will need to be copied ( only those with the pkginfo(4) parameter SUNW_PKGTYPE set to root ).
If you remove them from the zone configuration, your zone will require approx. 2GB of disk space but you will have the maximum flexibility for additional software installation.

Note that creating a zone with the -b option will result in an empty configuration, without any “add-inherit-pkg-dir”. ( see question 6.9 )
global# zonecfg -z my-zone
my-zone: No such zone configured
Use ‘cretae’ to begin configuring a new zone.
zonecfg:my-zone> create -b
zonecfg:my-zone> info
zonepath:
autoboot: false
pool:
zonecfg:my-zone>

2.2 : Which kind of devices may I NOT add using the zonecfg “set devices” command?

• Devices that expose system data : /dev/kmem, /dev/lockstat, …
• Devices that expose network data : /dev/hme, /dev/ip, …

2.3 : How do I add a special netmask for a zone’s IP address?
When configuring the zone, you can use a prefix length next to the IP address to specify the nemask to use.
For instance, the following configuration would result in a configuration where a logical interface will be added for eri0, with the netmask 255.255.255.0 :

global# zonecfg -z my-zone
zonecfg:my-zone> add net
zonecfg:my-zone:net> set physical=eri0
zonecfg:my-zone:net> set address=10.2.3.4/24
zonecfg:my-zone:net> end
zonecfg:my-zone>

2.4 : How to hide a subdirectory of a directory that is loopback mounted from the Gloabl zone ?
Suppose that you want to have /usr in the non-global zone loopback mounted from the Global zone but that you don’t want your non-global zone to have access to /usr/local.

global# zonecfg -z my-zone
zonecfg:my-zone> add fs
zonecfg:my-zone:fs> set dir=/usr/local
zonecfg:my-zone:fs> set special=/empty
zonecfg:my-zone:fs> set type=lofs
zonecfg:my-zone:fs> add options ro
zonecfg:my-zone:fs> end
zonecfg:my-zone:fs>

2.5 : How do I add a filesystem to my non-global zone?

• Use a LOFS mount:

  global# newfs /dev/rdsk/c1t0d0s0
  global# mount /dev/dsk/c1t0d0s0 /mystuff
  global# zonecfg -z my-zone
  zonecfg:my-zone> add fs
  zonecfg:my-zone:fs> set dir=/usr/mystuff
  zonecfg:my-zone:fs> set special=/mystuff
  zonecfg:my-zone:fs> set type=lofs
  zonecfg:my-zone:fs> end

• Use a UFS mount:

  global# newfs /dev/rdsk/c1t0d0s0
  global# zonecfg -z my-zone
  zonecfg:my-zone> add fs
  zonecfg:my-zone:fs> set dir=/usr/mystuff
  zonecfg:my-zone:fs> set special=/dev/dsk/c1t0d0s0
  zonecfg:my-zone:fs> set raw=/dev/rdsk/c1t0d0s0
  zonecfg:my-zone:fs> set type=ufs
  zonecfg:my-zone:fs> end

• Export the device node and mount from the non-global zone:

  global# zonecfg -z my-zone
  zonecfg:my-zone> add device
  zonecfg:my-zone:device> set match=/dev/rdsk/c1t0d0s0
  zonecfg:my-zone:device> end
  zonecfg:my-zone> add device
  zonecfg:my-zone:device> set match=/dev/dsk/c1t0d0s0
  zonecfg:my-zone:device> end
  my-zone# newfs /dev/rdsk/c1t0d0s0
  my-zone# mount /dev/dsk/c1t0d0s0 /usr/mystuff

• Mount the FS directly from the Global zone when the non-global zone is running:

  global# mount /dev/dsk/c1t0d0s0 /export/zones/zone1/root/mnt

• Using lofiadm

2.6 : How many containers can one domain or computer have, both theoretically and realistically?
One single instance of Solaris has a theoretical limit of 8192 zones. The real-life number will of course depends on resource consumption, namely CPU, memory and network usage as well as storage needs.

2.7 : How do I configure the identity (hostname, timeserver, timezone,… ) of a non-global zone?
After the non-global zone has been installed, all kind of typical identity information needs to be provided. It can be done in two ways:

• Interactively : the Global zone administrator boots the zone for the first time. A “sysidtool” process ( the same that the one used for a standard installation ) is then launched inside the non-global zone. The administrator needs then to connect to the non-global zone console using the “zlogin -C my-zone” command. He will then be able to provide answers to all the common questions. The zone will then reboot.
• Non interactively : the Global zone administrator creates the file /etc/sysidcfg in the non-global zone directory tree. The file contains all the answers to the “sysidtool” command, pretty much like in the Jumpstart network installation procedure. Detailed reference for this file can be found on this docs.sun.com guide. For an example of this, refer to this zones lab

2.8 : How do I configure a default route for a non-global zone?
As there is only one TCP/IP stack, the complete network information must be configured from the Global zone. The Global zone administrator can add one (or more) default route for each subnet, the kernel will make sure that the right one is used for each non-global zone depending on its IP & netmask.

2.9 : Is it possible to clone a non-global zone?
Not yet but it will be eventually. A Sun engineer has started working on this. If you can live with an unsupported procedure, have a look at this lab.

2.10 : Where do zone installation default files come from?
After installation, the non-global zone /etc directory is populated with “empty” files. No users defined in /etc/passwd, no groups in /etc/group,… So these files can obviously not be copied from the Global zone at zone install time. Because the granularity of the installation is the package, looking in the package database gives good results.
#grep /etc/passwd /var/sadm/install/contents
/etc/passwd e passwd 0644 root sys 580 48298 1127200974 SUNWcsr
tells that the /etc/passwd file is part of the SUNWcsr package. Looking inside the directory /var/sadm/pkg/SUNWcsr/save, we find those files that will be saved in case of a patch modifying the package. And the passwd file will be located in /var/sadm/pkg/SUNWnfscr/save/pspool/SUNWcsr/reloc/etc
Note : modifying these files to get customized etc files after zone installation is not the way to go! Go for scripts.

2.11 : May I install a zone in a NFS-exported directory so that diskless clients may run them?
No. Not supported.

2.12 : Is it possible to configure/install non-global zones directly from a Jumpstart server?
No. However, there is a JETzones package in the latest JET (Jumpstart Enterprise Toolkit) software.

2.13 : Is there a graphical tool that can be used to configure/install zones ?
Yes but only if the zones will reside on a Sun Fire T1000 or T2000. The Consolidation Tool v1.0 is an unsupported tool that makes it possible to use Solaris Containers ( a.k.a. Zones + Resource Management ) without having too much expertise in the subject.

=================================================================================

Section 3 : Administration

3.1 : Why is snoop not working in a non-global zone?
Snoop works by talking to the “DLPI” interface that itself contacts the appropriate NIC drivers. To implement the network isolation feature of non-global zones, access to DLPI and the lower levels interfaces is disallowed.

3.2 : How do I block traffic between non-global zones?
From the G lobal zone, use the “route reject” command. IPFilter cannot be used.

3.3 : What is the patch story in non-global zones?
Each zone maintains its own package and patch database. Every package/patch could then theoretically be installed individually into one or more zones, global or not. However, a number of other restrictions apply.

Patches for Solaris 10 can be broken down into these categories:

1. Patches that can only be applied from the global zone, that apply to the global and all non-global zones.

   These patches set SUNW_PKG_ALLZONES=true in their pkginfo file. (See the pkginfo(4) man page for more information.) These patches typically deliver binaries and files that affect the running OS. Although they are only applicable in the global zone, they must take effect in all non-global zones as well.

2. Patches that can be applied from any running zone.

   These packages set SUNW_PKG_ALLZONES=false. (See the pkginfo(4) man page for more information.) These patches can be applied in the global zone for the global zone, or applied in a non-global zone for the (same) non-global zone. These are typically application patches, such as those for a web server.

3. Patches that can only be applied to the current zone.

   These patches set SUNW_PKG_THISZONE=true and SUNW_PKG_ALLZONES=false. They can only be applied to the current zone (global or non-global). For instance, running patchadd in the global will result in the patch being applied only to the global zone. If run in the non-global zone, then the patch just applies to that non-global zone.
   This variable mimics the bahavior of the ‘-G’ option to patchadd. (See the patchadd(1M) man page for more information.)

These variables cannot change from their FCS values, so a patch cannot set SUNW_PKG_ALLZONES=true if the installed version is false. All these variables default to “false” if not defined.

Thanks to Penny from Sun Micro for providing (most of) this answer.

3.4 : How are patches applied to non-global zones?
In Solaris 10 FCS, running patchadd in the global zone results in each non-running zone being booted twice, once to check dependencies and the second time to actually apply the patch. This behavior has changed in Solaris 10 Update 1 : non-running zones are now not actually booted at all.

Thanks to Penny from Sun Micro for providing this answer.

NB One must take into account that what happens if a non-global zone cannot be brought online for any reason during a patchadd operation, is more or less unclear.

3.5 : Can I change a non-global zone IP while it is running?
Yes. Just use the “ifconfig” command from the Global zone. Don’t forget to also update the non-global zone’s configuration if you want the change to be persistent.

3.6 : Can I add access to a filesystem to a non-global zone while it is running?
Yes, as long as it doesn’t require the exporting of a new disk device to the non-global zone. See Question 2.5 & Question 3.7

3.7 : Can I add access to a device to a non-global zone while it is running?
No. You need to update the non-global zone’s configuration and reboot the zone.

3.8 : How does auditing work with zones?
The way you would expect it : the Global zone administrator may look at all audit records. The non-global zone administrator is only able to look at its zone. Observability is probably an advantage of zones over “VmWare-like” virtualization techniques.

3.9 : Is it possible to move a non-global zone?
Not yet but it will be eventually. A Sun engineer has started working on this.

3.10 : How do I backup non-global zones?
The main question is whether you need to perform the backup from the Global zone or from the non-global zone.

• From Global zone
  1. You have some enterprise backup software like NetBackup or Legato Networker. Even “ufsdump” requires access to devices which might not be available from the non-global zone.
  2. You want to be able to restore your entire zone, not only the data it contains. Main example is Disaster recovery.
• From non-global zone
  1. You just want to be able to restore the data used in a non-global zone.
  2. You want/need to use the backup tool of the application running in the non-global zone.

3.11 : How do I backup a non-global zone’s configuration?

global# zonecfg -z my-zone export > /myzone.config

3.12 : How do I restore a non-global zone’s configuration?

global# zonecfg -z my-zone -f /myzone.config

=================================================================================

Section 4 : Integration with other Solaris features

4.1 : Zones & IPFilter ?
Configurable only from the Global zone. A restriction is that IPFilter cannot be used to restrict access between 2 non-global zones. The loopback interface is used for that purpose and IPFilter cannot be used to inspect packets goign through that interface. See question 3.2.

4.2 : Zones & ZFS ?
Starting with Solaris 10 06/06, Zfs is available as a standard supported feature. A siple HOWTO document describes few of the possibilities of integrating Solaris Zones & ZFS. Read the document or go to the sun.com article. In short, any ZFS filesystem can be assigned to a zone. That non-Global zone administrator is then responsible of that ZFS filesystem. His capabilities include creating other subfilesystems, changing options, setting quotas, taking snapshots, preparing backups,…
It is of course also possible to use a ZFS mount point for the “zonepath” configuration parameter.

4.3 : Zones & IPQoS ?
Controlling the bandwidth that a zone uses is possible. IPQoS feature is bundled in Solaris 10 and can be used to set bandwidth restrictions for all IP addresses used by the zone. This has to be configured from the Global zone.

4.4 : Zones & IPsec ?
Configurable only from the Global zone.

4.5 : Zones & IPMP ?
Configurable only from the Global zone.

4.6 : Zones & DTrace ?
It is not possible to use DTrace from inside a non-global zone, for security reason. What is possible is to inspect and debug applications running in a non-global zone from the Global zone. The DTrace global variable “zonename” is easily used in predicates.
Some companies are using this trick to debug their applications running on Solaris 8 or 9. They just install them on a Solaris 10 machine, inside a non-global zone and use DTrace to find the bugs/problems!
[Update Feb 23 2006] : Some DTrace observations will be possible when the “Configurable Privileges for Zones” project will be available.

4.7 : Zones & SunCluster ?
This is a large project that will be deployed in several phases. The first phase was released in Sun Cluster 3.1 8/05 Software.

4.8 : Zones & Solaris Volume Manager?
You cannot configure metadevices from a non-global zone. However, the Global zone administrator can export a metadevice to a non-global zone.

4.9 : Zones & Process Rights Management ?
All processes running in a non-global zone have limited privileges. All the privileges that would allow the non-global administrator to break the isolation concept have been removed from the inheritable privilege set of the zsched daemon, the one that starts all the others.
Another privilege, PRIV_PROC_ZONE, is required to be able to signal or control non-global zone processes from the Global zone.
[Update Feb 23 06] : a case has been opened in Sun Architecture Commitee by David Comay. “Configurable Privileges for Zones”. In short, it will be possible to configure a non-global zone through zonecfg so that another set of privileges is given to all Processes in this zone. It will be possible to add/remove privileges to/from a non-global Zone depending on whether you want to extend the possibilities/improve the security. Some privileges will be marked “not addable” while others will be “non removable”. This feature does not yet have a release date. Info :

4.10 : Zones & lofiadm?
Interesting if you want to restrict disk usage for non-global zones.

4.11 : Zones & coreadm ( core files management )?
From the admin guide : “The coreadm command is used to specify the name and location of core files produced by abnormally terminating processes. Core file paths that include the zonename of the zone in which the process executed can be produced by specifying the %z variable. The path name is relative to a zone’s root directory.”

4.12 : Zones & DHCP?
A non-global zone cannnot currently be configured as either DHCP server & client. There is some ongoing work to solve the problem but no target date yet.

4.13 : Zones & NTP server?
The NTP server can currently only be run from the Global zone since local zones are missing the required privileges to set it. There is currently nothing like a non-global Zone time. There is one system time, set by the Global Zone.

=================================================================================

Section 5 : Resource Management

5.1 : Can I prevent one non-global zone from consuming all the CPU time?
Yes! The standard Resource Management features have been extended to zones. The Fair Share Scheduler is a scheduling class controlling the proportion of CPU time that a certain entity may use. The administrator of the Global zone is in charge of setting the new ressource control zone.cpu-shares to assign a number of shares to each non-global zone ( the Global zone is assigned 1 share by default ). The ratio of a non-global zone’s shares to the total number of shares defines the minimum percentage of CPU time that all the processes running in that non-global zone is authorized to use. ‘Minimum’ is important because the Resource Management model of Solaris specifies that any portion of CPU time not requested by a certain entity ( the non-global zone in this case ) may be used by the other entities. So in short, the new model is an extension that allows CPU shares to be assigned to non-global zones in addition to projects.

5.2 : Can I prevent one application in a non-global zone from using all the CPU time?
Yes! The Resource Management model being hierarchical, the first thing to do is to assign a number of shares to each non-global zone . Within the non-global zone, the non-global zone administrator may now create projects in the standard way to differentiate between workloads running in the same non-global zone. Resource contention between these workloads can be resolved by assigning FSS shares to the projects.
An example : if zone_1 is assigned 25% of all the shares and project_1 is created into zone_1 and getting 40% of zone_1 CPU time, project_1 is ensured that it will be able to use a minimum of 10% of CPU time ( 40% of 25% ).

5.3 : Can I prevent a non-global zone from consuming all the memory?
Yes! And no… The reource capping feature of Solaris Resource Management allows one to set an upper bound to the amount of RAM used by a certain project. By creating projects inside a non-global zone and setting the rcap.max-rss project parameter, you can limit the amount of memory used by all the processes belonging to that project.
So it is not a zone-aware feature but it can be used within non-global zones through the use of projects. Note as well that contrarily to resource controls, resource capping limits are only enforced asynchronously by the rcapd daemon and not synchronously by the Kernel.

5.4 : Can I run a non-global zone processes on specific CPUs?
Yes! First create a processor set that contains some of your CPUs ( be careful, you still need some CPU time for the Kernel! ). Then create a resource pool and bind the processor set to it. Finally bind your non-global zone to the same resource pool.

• global# zonecfg -z myzone set pool=mypool

All the processes running in your non-global zone will then run on the CPUs you selected by creating the processor set.

5.5 Can I bind several non-global zones to the same resource pool?
Yes! By binding more than one non-global zone to the same pool, you restore the contention between processes running in the various non-global zones. You can control this contention by assigning FSS CPU shares to each non-global zone. Doing so grants you the right to use a certain proportion of time of the CPUs defined in the associated bound processor set.

5.6 Can I dynamically change the number of FSS shares assigned to non-global zones?
Yes! From the Global zone, being the Global zone administrator, use :

• global# prctl -i zonename -n zone.cpu-shares -r -v new_value

5.7 Is there a way to dynamically or permanently assign shares to the global zone ?
The prctl command can be used for that purpose.

• global# prctl -i global -n zone.cpu-shares -r -v new_value

There is currently no way to do that permanently. A workaround is to use a transient SMF service that starts before non-global zones are created and that sets the number of shares desired. This value would be kept in the SMF repository and would then be safely stored and made modifiable using svccfg. Menno Lageman from Sun provided a service manifest

=================================================================================

Section 6: files, commands & daemons

6.1 : The zoneadmd daemon
zoneadmd(1M) is a system daemon for creating the non-global zone virtual platform and managing state transition of the virtual platform. Each non-global zone’s virtual platform is managed by one different instance of zoneadmd.
The main functions of the daemon are :

• To implement a door server for clients to request zone state changes. Doors are used to let commands like zoneadm communicate with zoneadmd running in the non-global zones.
• To interface with zoneadm(1M) and zonecfg(1M), and zlogin(1M) to create, bring-up, and tear down the non-global zone virtual platform. This includes mounting the filesystems, creating devices in /dev, setting up network interfaces, configure zone-aware ressource management parameters & creating the zsched process.

6.2 : The zsched daemon
The kernel dummy process for a non-global zone. All processes in a non-global zone are descendants of zsched. This is important to understand how zones take advantage of the privileges feature : the inheritable privilege set of zsched determines the effective privilege set of all processes in the zone. By extracting the potentially dangerous privileges from this inheritable set, a security boundary is created around each zone.

6.3 The zcons driver
The zcons(7D) driver is channeling I/O between a non-global zone and the Global zone. There is one driver per non-global zone. In the non-global zone, /dev/console, /dev/sysmsg, … are all links to the zcons driver.

6.4 The zonecfg command
Used to configure a non-global zone in an interactive mode. Can also be used non interactively within scripts.

6.5 The zoneadm command
Used to let a non-global zone go from one state to another.

6.6 The zlogin command
Allows you to obtain a shell running on a non-global zone from a Global zone shell. The “-C” option provides exclusive access to the non-global zone console. Uses the zcons(7D) driver.

6.7 The /etc/zones/my-zone.xml file
This XML file contains the configuration of the non-global zone called ‘my-zone’. The file is created after having configured the zone using zonecfg. It gets modified each time the zonecfg command is used to modify one of the parameters.

6.8 The /etc/zones/index file
Contains a list of all the zones and their state. Is modified by the zoneadm command.

6.9 The /etc/zones/SUNWdefault.xml file
The zonecfg create command is going to prepair your zones with some default values. 4 “inherit-pkg-dir” parameters and “autoboot” set to false. This is specified in the SUNWdefault.xml file. You can modify this file if you want all your zones to have some common set of parameters. The XML file will be checked against its DTD file, which you find in “/usr/share/lib/xml/dtd/zonecfg.dtd.1“.

6.10 The /etc/zones/SUNWblank.xml file
The zonecfg create -b command is going to prepair your zones with “autoboot: false” as the only default parameter, as specified in the /etc/zones/SUNWblank.xml file. You may want to modify this file ( see previous question ).

=================================================================================

Section 7 : Third party software support

7.1 : Symantec/Veritas NetBackup
Supported with conditions.

================================================================