LDOM 3.0, T5 and M10

The release notes for LDOM 3.0 confirms what we already knew, logical domains will be supported not only on upcoming SPARC T5 systems but also on the new generation for Fujitsu SPARC Server, M10.

"To enable all the OracleVM Server for SPARC 3.0 features, you must run the required 

system firmware versions on the following platforms:

UltraSPARC T2 server Run at least version 7.4.2 of the system firmware.
UltraSPARC T2 Plus server Run at least version 7.4.2 of the system firmware.
SPARC T3 server Run at least version 8.2.1.b of the system firmware.
SPARC T4 server Run at least version 8.2.1.b of the system firmware.
SPARC T5 server Use any installed version of the system firmware.

This firmware is preinstalled on the SPARC T4 and SPARC T5 servers. The required firmware
for Fujitsu M10 systems is preinstalled on your system. For information about the required
Oracle Solaris OS version, see “Required and Recommended Oracle Solaris OS Versions” on
page 12."

The new SPARC M10 systems can now be seen on Fujitsus site (English translation).
From the Register:  Fujitsu launches 'Athena' Sparc64-X servers in Japan

Solaris 11 Express VBox images

If you want to try Solaris 11 Express without dedicating hardware or even spend time on an installation there are now VirtualBox images available for download from Oracle. The images have the VirtualBox guest additions pre-installed.

Instructions and download links are available at The Observatory

VirtualBox 4 Beta on Solaris

Several beta releases of VirtualBox have been released in the last weeks. I had problems with the first two but beta 3 and later works nicely for me on Solaris 11 Express.

New features in 4.0

• New settings/disk file layout for VM portability; see the manual for more information.
• Open Virtualization Format Archive (OVA) support; see the manual for more information.
• VMM: support more than 1.5/2 GB guest RAM on 32-bit hosts
• Language bindings: uniform Java bindings for both local (COM/XPCOM) and remote (SOAP) invocation APIs
• Chipset: added support for the Intel ICH9 chipset with 3 PCI buses, PCI express and Message Signaled Interrupts (MSI)
• Audio: Intel HD Audio is now available as guest hardware, for better support with modern guest operating systems (e.g. 64-bit Windows; bug #2785).
• GUI: redesigned user interface with guest window preview
• GUI: new display mode with downscaled guest display
• Resource control: added support for limiting a VM's CPU time and IO bandwidth.
• Storage: support asynchronous I/O for iSCSI, VMDK, VHD and Parallels images
• Storage: support for resizing VDI and VHD images
• Windows Additions: support for automatically updating the Guest Additions (requires installed Windows Guest Additions 4.0 or later)
• Guest Additions: support for copying files into the guest file system

Update: Whats seems to be the release images are now available:
http://download.virtualbox.org/virtualbox/4.0.0

Zone enhancements

Recently a couple of enhancement for zones have been integrated into the OpenSolaris source.

PSARC/2010/132 Delegated Administration for Zones
This will make it possible to delegate administration of local zones to user with not-superuser privileges. This will add three new privileges: solaris.zone.clonefrom for cloning zones, solaris.zone.login for zlogin and solaris.zone.manage for zone management.

PSARC 2010/144 lofi(7D) in non global zones
Will make it possible to use lofi, the loopback file driver inside a local zone.

VirtualBox and OpenOffice

Just a quick news flash that new versions of OpenOffice and VirtualBox are available for download. VirtualBox is a maintenance upgrade to version 3.1.4, the OpenOffice version is 3.2.

The VirtualBox changelog and download here.
Download OpenOffice here and read the release notes.

OpenSolaris Text based installer update

There are finally some characters on the screen for the Text based installer project, the project pages have been updated. There is now access information for the project gate, build instructions and links to a code review (which does not seem to work right now).

The text installer is a essential part for bringing OpenSolaris into the datacenter, especally for the SPARC server which seldom have any framebuffers. Today the only installation option for OpenSolaris on SPARC is the Automatic Installer (AI) which requires DHCP service, a installation server and either access to external package repositories or a internal mirror.

This project will make it much easier to deploy single instances of OpenSolaris on your SPARC. X86 will also benefit from this, especially when installing inside xVM domains which today requires setup of VNC to perform a graphical installation.

A prototype for X86 was created this summer, read more here. The option for a interactive installation on SPARC have been promised for the next OpenSolaris release.

Crossbow for the win

Here's a very good paper on the Crossbow network virtualization and resource control which is available in OpenSolaris since 2009.06: Crossbow Virtual Wire: Network in a Box.

This paper even won the best paper award for Usenix LISA 2009. The blog of one of the authors is available here where he writes about the award and the BOF at Usenix Lisa 09.

Well worth a read if you don't know what crossbow is and can do, and if you do it's still worth a read.

Deduplication with zones

One of the major strengths of zones in Solaris is that they are very lightweight, since they share the same kernel they have low CPU, I/O and memory overhead. In Solaris 10 the ability to create "sparse" zones is available, with this option the local zones created shares most of the binaries and libraries with the global zone. This does not only save space, it also saves memory since all zones share the same instances of common binaries and libraries. The downside of sparse zones is that they have a very strong relationship with the global zone and no modifications unique to any zone can be made to the shared filesystems.

In OpenSolaris and later updates of Solaris 10 the ability to clone a zone is available. A zone is installed on a ZFS filesystem of which a clone is created for every new zone. Only minor modifications are made to the cloned filesystem to give the zone it's unique identity. This works much like deduplication until you patch or upgrade the system, which will make all the clones contain their own copies of the new data even if it's common to other zone instances.

Sparse zones are not supported by the new packaging system in OpenSolaris and it might never be. But zones in OpenSolaris only installs a very basic set of packages, which makes a clean install of a zone very small to begin with, they can then be placed on a compressed filesystem, and in OpenSolaris 2010.03 this filesystem can also be deduplicated.

I've done a small test to see how much space will be used by every zone instance with both compression and deduplication. These are freshly installed zones with have been booted once so that everything have been initialized in the zones:

A single zone on a LZJB compressed and deduped ZFS filsystem:

NAME     SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
zdup01  9.94G   241M  9.70G     2%  1.01x  ONLINE  -

Two zones:

NAME     SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
zdup01  9.94G   253M  9.69G     2%  1.99x  ONLINE  -

Three zones:

NAME     SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
zdup01  9.94G   263M  9.68G     2%  2.96x  ONLINE  -

So every zone uses a little more than 10MB of disk space and with deduplication you also get the same benefits in memory footprint as a with a sparse zone since there is only one deduplicated instance of Solaris libraries and binaries for the zones. They are however not shared with the global zone, since it boots from a separate pool without compression and deduplication. Unlike zones on a cloned ZFS filesystem the deduplication will continue to work after upgrading the zones and for software added to the zone from for example the pkg repositories post install time.

It looks like I can continue continue to run 20 zones on my thirteen year old Ultra 2 workhorse even if I upgrade it to OpenSolaris one day.

Solaris 10 containers integrated

Solaris 10 have a technology known as brandz, which can be seen as a translation layer between the Solaris kernel and a local zone. This layer can be used to provide an execution environment inside the zone that mimics another release of Solaris or even another OS. This have been used to create the LX brand that runs Linux inside a container and it have also been used to crate Solaris 8 and 9 containers.

This have probably helped accelerate Solaris 10 adaptation and made it easier for customers to take advantage of new Solaris features and hardware even if their application environment could not be upgraded. A good example of this is that you can take a Solaris 8 application, put in in a branded zone and use dtrace, which was not available before Solaris 10 to debug the application.

Now that Solaris 10 containers have integrated plan is to provide a similar service for Solaris 10 so that installations and zones can be migrated into Solaris 10 containers on OpenSolaris. This is probably even more important now since all version of Solaris post 10 will feature a whole new packaging system. Upgrades from Solaris 10 to any later Solaris release is not possible and will probably never be in the way we have known upgrades in earlier Solaris releases.

It will be possible migrate both existing Solaris 10 zones(v2v) and whole installations(p2v) into containers on OpenSolaris.

You can take a look at the change here where there is also a link to the PSARC which contains much more information.

OSOL 2010.03, the story so far

There have been some time since the OpenSolaris 2009.06 release, and the developers have been busy. I've compiled a list of changes in ONNV that caught my attention along with a few new packages added to the repository.

A few of the major features in so far in my opinion:

• Tripple parity raidz, offers even better data protection than raidz2, enabling even wider stripes and/or more fault tolerance for zpools.
• xVM is synced with XEN 3.3, up from version 3.1, offering better compatibility and performance.
• Crossbow enhancements, bridging, anti-spoofing, Solaris packet capture
• ZFS User and group quotas

If you find anything you like, you can try it out in the latest development release which is available here.

ZFS/Storage
triple-parity RAID-Z (6854612)
ZFS user/group quotas & space accounting (PSARC/2009/204)
iSCSI initiator tunables (PSARC/2009/369)
zpool autoexpand property (PSARC/2008/353)
COMSTAR Infiniband SRP Target - PSARC/2009/111
ZFS logbias property (PSARC/2009/423)
Removing a slog doesn't work (6574286)
FCoE (Fibre Channel over Ethernet) Target (PSARC/2008/310
FCoE (Fibre Channel over Ethernet) Initiator (PSARC/2008/311)
Multiple disk sector size support (PSARC/2008/769)
zfs snapshot holds (PSARC/2009/297)
SATA Framework Port Multiplier Support (PSARC/2009/394)
zfs checksum ereport payload additions (PSARC/2009/497)
Solaris needs reparse point support (PSARC 2009/387)
ZFS support for Access Based Enumeration (PSARC/2009/246)
If 'zfs destroy' fails, it can leave a zvol device link missing (6438937)
zpool destruction/export should better handle stale zvol links (6573142)
zpool import with 8500 snapshots took 11hours (6761786)
zfs caching performance problem (6859997)
stat() performance on files on zfs should be improved (6775100)

Network
Solaris gldv3/wifi needs to support 802.11n (6814606)
RBridges: Routing Bridges (PSARC/2007/596)
Solaris Bridging (PSARC/2008/055)
Bridging Updates (PSARC/2009/344)
Clearview IP Tunneling (PSARC/2009/373)
Datalink Administration from Non-Global Zones (PSARC/2009/410)
Solaris Packet Capture (PSARC/2009/232)
Anti-spoofing Link Protection (PSARC/2009/436
flowadm(1m) remote_port flow attribute (PSARC/2009/488)

Other
Boomer: Next Generation Solaris Audio (PSARC/2008/318)
ls enhancements (PSARC/2009/228)
Upgrade NTP to Version 4 (PSARC/2009/244)
Solaris on Extended partition (PSARC/2006/379)
Disk IO PM Enhancement (PSARC/2009/310)
System Management Agent (SMA1.0) migration to Net-SNMP 5.4.1 (LSARC/2008/355)
Upgrade OpenSSL to 0.9.8k (6806386)
Need to synch with newer versions of Xen and associated tools (6849090)
LatencyTOP for OpenSolaris (PSARC/2009/339)
Wireless USB support (PSARC/2007/425)

New drivers:
Atmel AT76C50x USB IEEE 802.11b Wireless Device Driver (PSARC/2009/143)
Ralink RT2700/2800 IEEE802.11 a/b/g/n wireless network device (PSARC/2009/167)
RealTek RTL8187L USB 802.11b/g Wireless Driver - (PSARC/2008/754)
add a VIA Rhine Ethernet driver to Solaris (PSARC/2008/619)
audio1575 driver available
audiocmi driver (PSARC/2009/263)
bfe fast ethernet driver (PSARC/2009/242)
Driver for LSI MPT2.0 compliant SAS controller (PSARC/2008/443)
Atheros AR5416/5418/9280/9281/9285 wireless Driver (PSARC/2009/322)
audiovia97 (PSARC/2009/321)
Myricom 10 Gigabit Ethernet Driver (PSARC/2009/185)
Atheros/Attansic Ethernet Gigbit Ethernet Driver (PSARC/2009/405)
audiols driver (PSARC/2009/385)
audiosbp16x audio driver (PSARC/2009/384)
Marvell Yukon Gigabit Ethernet Driver (PSARC/2009/190)
audiosolo driver (PSARC/2009/487)

Some highlights among the over 200 packages added to the repository:
SUNWwireshark Wireshark - Network protocol analyzer
SUNWparted GNU Parted - Partition Editor
SUNWiperf tool for measuring maximum TCP and UDP bandwidth
SUNWiftop iftop - Display bandwidth usage on an interface
SUNWsnort snort - Network Intrusion Detector
SUNWdosbox DosBox - DOS Emulator
SUNWejabberd ejabberd - Jabber/XMPP instant messaging server
SUNWiozone iozone - a filesystem benchmark tool
SUNWrtorrent rtorrent - a BitTorrent client for ncurses
SUNWsynergy Synergy Mouse/Keyboard sharing
SUNWareca Areca backup utilities

Many Python python modules, languages and OpenJDK 7.0 have also been added.

Virtualization in 2009.06 (Part 2, VirtualBox)

To continue the "getting started" tutorials on virtualization on OSOL 2009.06 we will take a look at VirtualBox. VirtualBox might be a good Solution for remote desktop installations, fully virtualized machines with graphical console over the network. As with the previous posts this one will be focused on how to deploy it on your server. If you run virtualbox on your desktop it is probably easier to use the supplied GUI, which is on pair with the VMWare workstation GUI.

VirtualBox only supports full virtualization, so the performance will not be as good as a paravirtualized xVM domain or Solaris zone, but it works good for graphical access to virtual machines over network with lighter workloads.

Install VirtualBox

VirtualBox is not hosted in the standard OSOL repositories, it is in the extra repository (along with flash, JavaFX SDK and others). There is some minor hassle to get access to this repository, you will ned to register an account with sun then download and install certificates. You can register here and download the certificate, there are install instructions on the site, but here is the procedure:

$ pfexec mkdir -m 0755 -p /var/pkg/ssl
$ pfexec cp -i ~/Desktop/OpenSolaris_extras.key.pem /var/pkg/ssl
$ pfexec cp -i ~/Desktop/OpenSolaris_extras.certificate.pem /var/pkg/ssl
$ pfexec pkg set-authority \
-k /var/pkg/ssl/OpenSolaris_extras.key.pem \
-c /var/pkg/ssl/OpenSolaris_extras.certificate.pem \
-O https://pkg.sun.com/opensolaris/extra/ extra
Now install VirtualBox:

$ pfexec pkg install virtualbox virtualbox/kernel

Create virtual machines

Create and install a virtual machine:

$ cd /opt/VirtualBox
$ VBoxManage createvm --name "WinXP" --register
$ VBoxManage modifyvm "WinXP" --memory "512MB" \
  --acpi on --boot1 dvd --nic1 nat
$ VBoxManage createhd --filename "WinXP.vdi" \
  --size 5000 --remember
$ VBoxManage modifyvm "WinXP" --hda "WinXP.vdi"
$ VBoxManage controlvm "WinXP" dvdattach \
  /path_to_dvd/winxp.iso
$ VBoxHeadLess --startvm "WinXP"

Connect to the server with an RDP client and perform the installation. There are free RDP Clients available for most Operating systems:

Solaris/OpenSolaris/*BSD: rdekstop (pkg install SUNWrdesktop)
MacOS X: Remote Desktop Connection

After the installation is done, the guest can be controlled with VBoxManage. VBoxManage can control all aspects of the guest, snapshots, power, USB sound. Here are a few basic commands:

Start: VBoxHeadLess -s winxp

Poweroff: VboxManage controlvm WinXP poweroff

Reset: VboxManage controlvm WinXP reset

If you want to install the VirtualBox Guest additions, just inject the DVD:

$ VBoxManage controlvm "WinXP" dvdattach \
/opt/VirtualBox/additions/VBoxGuestAdditions.iso

Virtualization in 2009.06 (Part 1, xVM)

Install xVM

It seems that i have started a little tutorial trail for working with OSOL 2009.06. A few of my friends are about to install this release, so i thought i might as well make it a few blog entries, there are probably other people out there that want some help to get a quick start of OSOL 2009.06.

This will be the first entry about virtualization, first we get xVM running, later entries will describe some basic setup of Solaris zones and VirtualBox.

Install the xVM packages:

pfexec pkg install xvm-gui SUNWvdisk

Edit /rpool/boot/grub/menu.lst, copy your current entry and modify it to something similar to this:

title OpenSolaris 2009.06 xVM
findroot (pool_rpool,1,a)
bootfs rpool/ROOT/opensolaris
kernel$ /boot/$ISADIR/xen.gz
module$ /platform/i86xpv/kernel/$ISADIR/unix /platform/i86xpv/kernel/$ISADIR/unix -B $ZFS-BOOTFS,console=text
module$ /platform/i86pc/$ISADIR/boot_archive

Reboot into this grub entry and if everything works set this as your default boot entry:

$ bootadm list-menu
the location for the active GRUB menu is: /rpool/boot/grub/menu.lst
default 0
timeout 30
0 OpenSolaris 2009.06
1 OpenSolaris 2009.06 xVM

$ bootadm set-menu default=1

Enable the xVM services

If you want to be able to connect to VNC over the network to perform the installation, make xen listen on external addresses for VNC connections:

$ pfexec svccfg -s xvm/xend setprop config/vnc-listen = astring: \"0.0.0.0\"

Enable the xVM services and set a password for VNC connections:

$ pfexec svccfg -s xvm/xend setprop config/vncpasswd = astring \"yourpass\"
$ pfexec svcadm refresh xvm/xend
$ pfexec svcadm enable -r xvm/virtd
$ pfexec svcadm enable -r xvm/domains

( Ignore messages about multiple instances for dependencies )

Installing domains

Now we should be able to create our domU instances, First we create a zvol to be used as disk for the domU:

$ pfexec zfs create -o compression=on -V 10G zpool01/myzvol

Install a paravirtualized domain e.g. OSOL 2009.06:

$ pfexec virt-install --nographics -p -r 1024 -n osol0906 -f /dev/zvol/zpool01/myzvol -l /zpool01/dump/osol-0906-x86.iso

Connect to the console and answer the language questions:

$ pfexec xm console osol0906

Back on dom0 get the address, port and password for the VNC console for the OSOL installation, first get the domain id:

$ pfexec virsh domid osol0906

Get the address, port and password using the domain id:

$ pfexec /usr/lib/xen/bin/xenstore-read /local/domain//ipaddr/0
$ pfexec /usr/lib/xen/bin/xenstore-read /local/domain//guest/vnc/port
$ pfexec /usr/lib/xen/bin/xenstore-read /local/domain//guest/vnc/passwd

Connect with a VNC client to address and port, authenticate using the password.

Install an OS without support for paravirtualization, e.g. Windows:

$ pfexec virt-install -v --vnc -n windows -r 1024 -f /dev/zvol/dsk/zpool01/myzvol -c /zpool01/dump/windows.iso --os-type=windows

Connect to the xVM VNC console using the password provided earlier with svccfg/vncpasswd.

When installation is done domains can be listed with xm list and started with xm start.