LDoms static Direct I/O support

Logical Domain capable servers can soon assign devices on a slot granularity to LDOMS. Previously only whole PCI-busses could be assigned to domains. This will provide more freedom and better performance for virtualized environments using logical domains. A LDOM could with this enhancement have direct access to it's own NIC and HBA for better performance. It could also make smaller configurations less complex since no virtual disk servers and or virtual networks switches would be needed in an I/O domain. This features has integrated in b133 and will be part of the OpenSolaris 2010.03 release.

"This is an RFE filed to Static Direct I/O feature support in Solaris.
This project enables PCI express devices to be assigned directly to logical domains at the slot level. The assignments are "static" in the sense that any changes to the disposition of devices requires a reboot of the root domain to take effect."

LDoms static Direct I/O support (6848789)

Text based install with OSOL b131!

The Text based installer project have produced preliminary install imaged for both SPARC and X86. Good news but no time to write got to continue to watch the Sun/Oracle webcast

Download links are available on the here.

A first test installation in a OSOL xVM domU works good apart from that I seemed to have some problems with the terminal settings:

No more Sunshine

Sun is no more.

This feels a bit awkward but Oracle has completed the acquisition of Sun Microsystems (JAVA, the stock ticker previously known as SUNW). I hope not everyone follows Jonathan's advise and let go of the Sun culture, the products and support are far from the whole value of the company. I also hope that the majority of all the talented people still at Sun will get a chance to continue to produce great hardware and software inside Oracle.

Lets hope something good comes out of this, that Oracle can make more money on the products from Sun and that more resources are spent developing existing and future technologies and products that relate to Unix and Solaris. I also hope that Oracle understands that we need the openness that we have come to learn from Sun in recent years, we want to be part of the development process and have access to the sources and development releases.

We'll have to wait for another hour or so to see the Oracle webcast will shred some light on all this.

Best of luck to all Sun employees and to the new Oracle corporation, goodbye Sun and thanks for the fifteen years I have known you. The next Sunrise will not be purple but pure red.

Read the announcement here.

Darren in ZFS-crypto video interview

For those of you waiting for ZFS on-disk encryption, here is an video interview with Darren. He gives a overview of the projects and even get's into some preliminary thoughts about performance.

It still looks like the Integration Target is Q1CY10 which would place it in a build quite close to the OSOL 2010.03 release (which targets build 134). But it has been pushed forward several times before, also all the Oracle integration will probably not make it happen any sooner.

Don't let zpool(1M) scare you

While doing some reliability tests for my new storage node I came across some negative behavior from zpool(1M). It has a tendency to recommend that you destroy the whole pool and restore from backup even if it's possible that the pool can be saved.

Like when trying to import a raidz2 pool which has lost connectivity with one too many disks:

cannot import 'zpool01': I/O error
        Destroy and re-create the pool from
        a backup source.

Or even worse, when importing a second pool with the same name:

cannot import 'rpool': pool already exists
        Destroy and re-create the pool from
        a backup source.

Both situations where easily solved, the first by reconnecting a drive and the second I just had forgot to provide zpool import with a new name for the pool. Of course the first case could have been a fatal failure for the pool, but I think the error messages should be a bit less pessimistic, it can otherwise make new users do terrible things.

I've filed a bug but have not yet received a bugid for it.

Green light from EU for Oracle/Sun

Another non-technical post, but it affect most of the things i write about here. As it turns out, the European Commission had nothing to object to the deal after all, it has now been approved. Not much to say about this, it's all over the web by now. Read the whole press release here.

From the press release:
Competition Commissioner Neelie Kroes said: "I am now satisfied that competition and innovation will be preserved on all the markets concerned. Oracle's acquisition of Sun has the potential to revitalise important assets and create new and innovative products."

News on future Sun/Oracle strategy

Looks like we may get some more answers regarding the future of Oracle and Sun soon, Oracle hosts an event on January 27.

Oracle Corporation (NASDAQ: ORCL) announced today that it will host a live event for customers, partners, press and analysts on January 27, 9:00 AM – 2:00 PM PT, at its headquarters in Redwood Shores, California. Oracle CEO Larry Ellison, along with executives from Oracle and Sun, will outline the strategy for the combined companies, product roadmaps, and how customers will benefit from having all components - hardware, operating system, database, middleware, and applications - engineered to work together. The event will be broadcast globally.

I hope we at least will get some concrete answers on how they plan to handle Solaris and OpenSolaris.

Here is the full press release but the above is pretty much it without lots of disclaimers.

Last of the SXCE builds is out

Build 130 of Solaris Express Community Edition, SXCE have been released. This will be the last build of Solaris delivered in this form of distribution. All future OpenSolaris development will be in the form of the OpenSolaris 20xx.xx releases. As most of you know SXCE was the original distribution of the Solaris Nevada development branch, it uses the same packaging system and installation tools as Solaris 10. I've been using this distribution from the very beginning and am still using it since OpenSolaris does not include Xsun, only Xorg which lacks drivers for many framebuffers used in mature workstations. It looks like the currently running live upgrade of my SunBlade 1000 will be the last, should I be forced to switch my workstation every ten years if I want to keep up with the latest Solaris releases?

From the announcement on osol-discuss:
"There have been a number of questions regarding SXCE, so here is the
latest information.

* Nevada RE Build 130 will be the final SXCE build.
* Build 130 is scheduled for release on January 13th.
* The Build 130 images will be available for two weeks.
* After January 2010, all existing SXCE releases will be removed
from the SDLC (Sun Download Center) and no additional SXCE images
of any kind will be distributed.

* OpenSolaris 2010.03 is currently set for release on March 26th"

Since they will be removed from the download center it's probably a good time to download and archive your favorite builds soon.

If you are unfamiliar with the various Solaris distribution names, look here, the names are correct but some of the details are old.

Solaris Samba update

I looks like both OpenSolaris and the next update of Solaris 10 will be getting a updated version of Samba.
This is probably most interesting for Solaris 10 since OpenSolaris has it's own native implementation of CIFS in the kernel. This update is needed for comparability with Windows 7 which requires version 3.4 of Samba.

Some details from the PSARC:
"Samba is the only available CIFS volume and printing server in Solaris 10 and 9 (production releases of the Solaris).

Samba is the only available CIFS printing server in Solaris 9 and above.

Besides Apache, Samba is the most important component in Solaris in the case of the business deployment. It plays primary role in interoperability with the MS Windows on Solaris 10 and 9.

Samba.org community discontinued support of the Samba 3.0.x currently bundled with all releases of Solaris."

Update Samba to release 3.4 [PSARC/2009/682]
CR 6852659, Update samba to 3.3.5 or later

First hints on Solaris 10 update 9

I come across some hints on changes in the the next Solaris 10 update probably scheduled around April, [45]/10? It looks like the zpool will be lifted to version 19 which would mean the following new high level features:

17 Triple-parity RAID-Z
18 Snapshot user holds
19 Log device removal

This would mean there would be no zpool deduplication in the next Solars 10 release, but that is not much of a surprise given that it was just recently integrated into ON. Triple-parity raidz and log device removal will nevertheless be a good addition while keeping the risk low.

It also looks like a iSER (iSCSI Extensions for Remote DMA) backport are in the works for this release.

Two additional 10G NIC drivers have mentioned update 9 integration in their respective PSARC:
Emulex PCIe converged NIC driver [PSARC/2009/526]
qlge - QLogic PCIe converged NIC driver [PSARC/2009/525]

Update:
Samba update to 3.4 also looks like it's aiming for this release together with iSCSI boot and Firefox 3.5.

Update2: The name for this release and additional features of ZFS is mentioned in a later post here.

zpool split

PSARC/2009/511 zpool split have integrated into upcoming ON build 131. This feature will make it easier to split off mirrored disks as a new pool and use it somewhere else or lock it away. This operation have been a quite common operation when using volume managers and some people seem to have missed this in ZFS.

From the PSARC:

"OVERVIEW:

Some practices in data centers are built around the use of a volume manager's ability to clone data. An administrator will attach a set of disks to mirror an existing configuration, wait for the resilver to
complete, and then physically detach and remove those disks to a new location.

Currently in zfs, the only way to achieve this is by using zpool offline to disable a set of disks, zpool detach to permanently remove them after they've been offlined, move the disks over to a new host, zpool force-import of the moved disks, and then zpool detach the disks that were left behind.

This is cumbersome and prone to error, and even then the new pool cannot be imported on the same host as the original.

PROPOSED SOLUTION:

Introduce a "zpool split" command. This will allow an administrator to extract one disk from each mirrored top-level vdev and use them to create a new pool with an exact copy of the data. The new pool can then be imported on any machine that supports that pool's version.

Looking back a decade, part 3

This is the last part in the series, bare with me, i'm back to my ordinary posts after his. We are now looking at the time from 2008 to present.

The last year of the decade where very challenging for Sun, forcing more cutbacks and changes, the profit and stock price was no way near where it should be. Sadly, while Sun have had many good technical solutions and technologies they often have failed to profit from them. The crown jewel, Solaris also faced increased competition mainly from open source operating systems forcing it to evolve and change more rapidly.

A few years after Solaris 10 was released, Sun created a new distribution of OpenSolaris, this effort was named Project Indiana. The aim was to create a new packaging system and to make OpenSolaris more user friendly with a new graphical installer. The user environment was also to be updated to make it more similar to what other open source operating systems provided.

So after all the years of success in the nineties and early 2000 why did Solaris need to change except for opening the source? Solaris 10 was solid and combined an unique feature set with good performance, what problems initiated this new project? As I see it there have been three main areas that other operating systems have done better in than Solaris, all impacting new users acceptance of the platform:

Packaging
While SYSV packages have worked well for a long time (sort of) it lacks any function to install software from central repositories and to resolve dependencies between packages. This makes it harder for new users to get their software on the platform. It's not so much of a problem for large deployments where every package is controlled and well tested, but for developers, small companies and hobbyist it can be a bit of a challenge. The old system with patches was equally hard for new users, there where no built-in function in the patching system to find and apply new updates.

Software
Acceptance and integration of free software have been slow in Solaris, other operating systems have had a wider and more current base of packages available for installation in their repositories.

Hardware
Hardware support on the X86 platform have been limited, especially for home assembled systems that might use hardware that is not common in severs or finished systems from larger vendors.

On the bright side, all three points have been improved drastically. Project Indiana (what is now OpenSolaris 20xx.xx) addressed the first two points, the new packaging system IPS support central repositories and much more software is now available that can be directly installed with a simple command, including updates. It also comes with a brand new graphical (and soon text based) installer that makes the installation experience easier and prettier.

Hardware support for X86 platforms have also improved. Sun have developed more drivers and begun to support a wider range of x86 hardware, other vendors such as Intel and AMD have also contributed. There is support for laptops, WiFI chipsets, power management including suspend/resume and pretty all the latest CPUs from ADM and Intel. There is also support for the latest and greatest graphic cards from NVidia.

Solaris have several unique features not matched by any other operating system today, but the three above points have probably made it harder for new users to adapt the platform. Especially in recent years when users have became custom to easy software installation by a single click. In the good old days it could take days of compiling and patching to get any software installed ;)

As happened with many other of Suns inventions such as VFS, NFS, PAM, loadable kernel modules and RPC, the good ones have continued to spread from Solaris to other operating systems. MacOS X now has DTrace (almost got ZFS too) and FreeBSD has successfully ported ZFS.

Beginning with the integration of ZFS Sun became involved in the storage market, Solaris today contains a complete storage stack with ZFS, iSCSI, CIFS, NDMP, FCOE and NFS. This lead to (or probably been driven by) the amber road storage appliance in 2008. OpenSolaris was used as the base for this appliance system, leveraging DTrace and ZFS to the end user with a simple and powerful web user interface, by far the best graphical interface Sun has ever made. Amber road is based on ordinary X64 servers produced by Sun tied together with OpenSolaris and a custom built interface. This is a quite impressive product, and much thanks to that it uses both general propose hardware and most of the Software it's very competitively priced. I've thought we would see more of this, a similar for the MySQL databases for example.

The next challenge for Solaris will be the transformation from Solaris 10 to what today is OpenSolaris. This will not only be a new major release, it will also switch to the new packaging and installation system, both interactive and the succeeder to jumpstart named AI. OpenSolaris still needs some time to mature before it can be the source of the Solaris 11 or what marketing decides to name it. But it's getting closer, especially with the 2010.03 release, especially if it gets all its promised console based installation features.

On the hardware side Sun begun to partner with Intel from previously mainly working with AMD. The long awaited new Rock CPU was canceled in the last minute mid 2009, the launch was probably not long away since even Solaris 10 updates available to customers included some support for Rock. But the following bugid put an end to that: "6858457 Remove Solaris support for UltraSPARC-AT10 processor". Sun now have the Niagara line of CMT processors for throughput computing, but relies on Fujitsu for single thread performance, Rock should have made this gap smaller, delivering CMT with better performance per thread. The current M-class of servers are performing well, the SPARC64-VII processors from Fujitsu have seen several updates in the least years. They will probably continue be competitive for some time to come but SPARC customers need to to be assured with plans and commitment for future generations.

In the last year of the decade it was announced that Oracle intended to buy Sun, and while the deal is not finished, anything else is pretty much unthinkable by now. I think it wold have been better if Sun could have remained independent, but Sun needed increased profit, it's was becoming clear that this downward spiral was nothing Sun could get out of on it's own. The offer discussed right before the Oracle deal was that IBM where to buy Sun, this would most likely have been a disaster for most of Sun products. IBM would have piked the the best parts and then assimilated the rest. And there are several technologies and products from Sun that have a great value besides Solaris, SPARC, Unified Storage, MySQL and Java, for example VirtualBox, NetBeans and SunRay, to name a few.

Solaris and several other products have a good foundation to stand on, but the limbo caused by the Oracle deal is making Sun bleed, the acquisition should have been completed many month ago. OpenSolaris is off to a good start as the base of the next Solaris but more resources are needed in the project. Hopefully Oracle will keep it's promise and continue to invest in Solaris to a even greater extent than Sun has been able to do the last years. The same applies to the hardware, Oracle needs to deliver on it's promise to invest more in SPARC and show a new roadmap, otherwise the future of Solaris will only be on X64 in a few years. This might sadly be the case in the long run anyhow, it looks like we are on a road towards an single desktop and server architecture. If this is the case, please at least include OBP on these servers...

The next month will probably reveal more of the future for Sun and it's products, it will be interesting to see Oracles true intentions once the deal is finished and how Sun will be incorporated into big red.

Looking back a decade, part 2

I've found out that this takes much time and it's hard to sort of the important things out of everything you would want to cover and not going into any depth. But three parts where promised so here's what have had time to write. The events will be dated around 2004-2006.

In the end of January 2005 "s10_74L2" a was finished, the release build of Solaris 10, which still is the active and supported major version. It had several features that changed the way we work with Unix systems. It was also the larges update to Solaris since the move to SVR4 in 2.0. The main focus was on performance but there where also huge improvements in virtualization, fault management and observability.

Previous Solaris releases was sometimes bashed with names such as "slowlaris", especially when run on workstation class machines. Solaris had overhead that needed to be removed and other operating systems did some things faster. The slogan "if another OS is faster, it is a Solaris bug" was used during development and it payed of making Solaris 10 substantially faster than in predecessors. One thing developed to improve performance was a library for microbenchmarking of system calls, libmicro. With performance data on every system call the ones which where slow could be fixed and compared to other operating systems. Another area of improvement was the networking stack, much of this was delivered by two projects, FireEngine for better TCP performance and Yosemite (in update 2) which did the same for UDP.

Solaris 10 was also the first release to contain light weight virtualization in the form of Solaris zones for both security and workload separation. While the initial release had several flaws when upgrading, it was fixed and many more features added in subsequent update releases.

DTrace, Service Management Facility and Fault Manager Daemon was other important additions, especially DTrace which really was, and still is, groundbreaking for debugging and performance tuning. ZFS which was much needed solution for an own modern volume manager and filesystem for Solaris was often mentioned as a major feature long before Solaris 10 was released, for example in the "Ten moves ahead" campaign. But ZFS was not released until over a year later with the second update, Solaris 10 6/06.

All this was crucial for Solaris to survive the competition and Solaris 10 delivered, albeit with somewhat slow adaptation in the beginning due to the large amount of changes. Without all the work put into this release Solaris would probably have lost it's competitive edge and faded away. The improved performance was required for Solaris to continue to be a competitive OS, the improved security and virtualization where needed additions, but what made most impact was probably DTrace and ZFS which both broke new ground.

Something worth a notice is that Oracle did chose Solaris 10 as it's preferred open source 64-bit developer platform in November 2006. So even if the bounds between Sun and Oracle had been stronger, it did still exist.

2005 was not only the year Solaris 10 was released, it was also the year it became free and open, Solaris 10 was made available freely for download and the source was released. As discussed in the first part, this might have been a bit late, especially the free for use bit, the source was probably even harder to rush, but giving people a chance to test Solaris for free should have been given long before Solaris 10, at least on the X86 side.

On the hardware side the dual-core UltraSPARC IV "Jaguar" was in the heart of most SPARC servers shipped. Sun and Fujitsu formed a partnership to develop future SPARC64 systems together. This resulted in Sun canceling it's own UltraSPARC V processor "Millennium" and shifted its focus on developing Chip MultiThreading processors with technology acquired when buying Aftra Websystems. The first CMT processor was released in November 2005, the UltraSPARC T1 with 8 cores and 4 hardware threads per core. Sun also hinted on a much more advanced CMT processor that was under development, named Rock, or the UltraSPARC RK, which would include features such as transactional memory and higher thread performance than the T1. The Fujitsu partnership helped Sun a lot, both by ending the head-on competition with Fujitsu and by giving Sun the CPU they needed for single thread performance while developing it's own line of CMT processors.

In the X86 space Sun bought Kealia which was started by one of the Sun founders, Andy Bechtolsheim. Kealia focused on developing 64-bit AMD opteron servers and became the foundation for the next generation of X86 servers including the 4 unit, 48 disk storage server x4500 "Thumper".

To summarize, Sun spent lots of resources on making Solaris more competitive and making if free, forming a alliance which cut the R&D spending on current SPARC systems while developing the next generation of system based on CMT. This was also the time when the real commitment to X86 servers began, with Kealia.