Wednesday, December 30, 2009

Looking back a decade, part 1

It's now a new decade and a good time to take a look on the important things that's happened the last 10 years with focus on Sun and Solaris. I'll try reflect on the good things and mention some of the bad things that have forced Solaris and Sun into a series of transitions. I'll split this into three posts. I'll begin around the year 2000, probably in the 1998-2002 range.

The new millennia began with a massive hysteria about the Y2K bug and everybody was upgrading server, storing food and preparing for the final hour. Sun had spent the nineties transforming the business from workstations towards servers and fighting for open standards and network computing. Many Solaris sites where at the time running Solaris 2.6 and applying all the "Y2K" patches to their systems.

Solaris 7 was available, but since it was the first 64-bit release of Solaris, third party development was lagging. Veritas Volume manager (VxVM), which at the time was close to a mandatory part of all enterprise installations was delayed for a very long time. Sun was without a good volume manager for Solaris and UFS was yet to be enhanced for better performance and disk suite had not been kept up to date since Sun even had bundled VxVM. All this was probably enough to have a fresh look at a new storage solution for, i think an early internal name for this project was "pacific", but I might be wrong. Probably as an interim Solution the Solstice volume manager was integrated into the next release, Solaris 8 as the Solaris volume manager.

Apart for the lag in third part drivers the transition to 64-bit was made early in Solaris, and without any of the hassle we are seeing on other platforms today, ten years later. There was one distribution with could run either a 32 or 64 bit kernel and in the later binaries of different bitwidth could execute side by side.

Solaris 2.6 to Solaris 7 where dependable and competitive for the time, but compared to what we are used to today it was a bit slower and more of just an standard System V implementation than todays Solaris. It was also more "bare" with little additional software beside the core OS included. Several administrative tools that we take for granted today was not available, some of them was introduced with Solaris 8 such as prstat, pgrep, pkill and mdb. The Solaris development process was much more closed, no source, no blogs about new features, no public ARC cases and no external development builds to test.

On the hardware side the UltraSPARC II was powering the Ultra Enterprise series. While being the first high volume server series from Sun it was dependable and sold very good. The size ranged from the one socket Ultra Enterprise 1 to the 64 socket Ultra Enterprise 10000 (StarFire). With such large systems even in the late nineties it's no surprise that Solaris have had little problem scaling on todays multi core processors. I remember thinking if I could get my hands on a E3000 when it was time for retirement, over ten years later I now have one as beer table beside my armchair, not quite what I intended back then.

Sun had several very good years late nineties, they where the number one Unix vendor and for a short time I think they even one of the top storage resellers, much due to all internal drives. But when the decline began in 2000 sun was hit hard. The investments into research and development did however continue which bare fruit some years later and is still making a difference, but more on that in the next post.

Looking back, I sometimes wonder what would have happened if Sun made the decision to make Solaris freely available and open already at this time. It would probably have grown the community and general acceptance much faster since Solaris had more presence outside server environments then. It was more widely used at universities and still had some of the workstation business left. Then again, no other of the big vendor have released their source, it's not available for AIX, Irix, HPUX or Windows. The initial lack of commitment to the X86 platform probably also hurt Sun when it became de-facto standard for small servers, but as long as it did not provide any threat to SPARC, too little effort was put into the alternative hardware platform.

Sunday, December 27, 2009

Build 130 available

There is some latency here during the holiday but on christmas eve the latest development version of OpenSolaris 2010.03 was released, it is based on build 130. This new release has move to a new major version of Xorg (1.7.3) and thunderbird (3.0). Later version of python and ruby among other changes. There have been some visible feature enhancement to the installation system also, read more in the full announcement.

"The ability to install on to an iSCSI target[1]

A new "bootable" Automated Install (AI) CD and USB ISO that
does not require an AI server to be setup[2]

Distribution Constructor enhancements to create OVF 1.0 virtual
machine (VM) images[3]

Extended and logical partition support for both the Live CD and
AI installers[4]"


Since we are getting close to the release and focus now is on fixing bugs (and delivering the text based installer), please give this build a spin and if you find any new bugs please report them as soon as possible. Look here for a quick introduction on searching the bug database and reporting new bugs. Be sure read the announcement before installing/upgrading, it includes several known bugs.

Note: there seems to be some problems on graphical workstations with this version, more specifically Nvidia OpenGL, look at the announcement and bug 6912450. It's probably a good idea to disable compiz prior to upgrading to b130 since failure of OpenGL will make the whole desktop unusable if compiz is enabled.

I've also updated the software version table .

ISO/USB images for SPARC/X86 is as usual available at www.genunix.org.

Thursday, December 17, 2009

New and improved CPU support

OpenSolaris 2010.03 will support for two new generations of processors, one SPARC and one AMD X86. Fault management will also be enhanced for the upcoming Intel Nehalem EX.

Rainbow falls or perhaps "UltraSPARC T3"
Support for the next generation of CMT processors from Sun planned for 2010 was added to build 131. This new chip will have 16 cores with 8 (or even 16?) threads each. No public PSARC or bugid is available, but here is the putback:
PSARC 2009/177 Solaris support for Rainbow Falls platforms
Here is a presentation of the CPU from the hotchips conference:
Sun's Next Generation CMT Processor

AMD Magny-Cours processor
A 8-12 core CPU that should appear in 2010, the following changes was integrated into build 128:
6843035 Need support for Magny-Cours processors
6860401 FMA CPU Topology & Memory Topology needs to support Magny Cours (Multi chip Module)

Intel Nehalem EX
Intel next Xeon CPU with 8 cores, 2 threads per core. Is also supported in OpenSolaris 2009.6 but FMA was added in b127:
FMA for Intel Nehalem EX
There is also a whitepaper on Nehalem support in OpenSolaris, worth a read:
The Solaris OS and Intel Nehalem EX

Sun/Oracle merger news

There have been some news regarding the Sun/Oracle merger this week. It is finally looking like the deal will get approved. Oracle announced its plans and commitment for MySQL this monday and yesterday the New York post wrote that the merger got green light.

"Larry Ellison's Oracle won an all-clear from Europe's top antitrust official to proceed with its $7.4 billion purchase of Sun Microsystems after reaching a verbal agreement to protect a piece of software at the heart of a months-long dispute."

Read the whole article here. Lets hope this is true and that we'll see the end of this long story soon.

ZFS SNIA presentation

This is a couple of months old now but it covers many of the new ZFS features, much the same that was discussed at the kernel conference. Many of the feature is now integrated in OpenSolaris such as zpool recovery, deduplication, radidz3 and quotas. Some of the features mentioned that will be released at a later date includes BP rewrite and encryption.

ZFS the next word...

Thursday, December 10, 2009

OpenSolaris software table

I've compiled a version list of some software packages which are part of the installation or available in the default repository for all OpenSolaris releases. There are quite possible different versions available in other repositories, but these are the default versions shipped by Sun and integrated with the rest of the release. If multiple versions are available only the latest is listed. The latest development release is included which should be released in March next year and named 2010.03, this release is of course subject to change so the table will probably be updated at least when the final release is available.

Update: The table has been updated to reflect the current state as of build 132.























































































































































Release2008.052008.112009.062010.031
ON build86101b111b133
Xorg7.2.5/1.3.07.2.5/1.3.07.4/1.5.37.5/1.7.4
Gnome2.20.22.24.02.24.22.28.0
Zpool10131422
ZFS3334
Firefox2.0.0.143.0.43.1B33.5.7
Thunderbird2.0.0.122.0.0.172.0.0.213.0.1
Postgres8.2.68.3.48.3.78.4.2
MySQL5.0.455.0.675.1.305.1.37
Python2.4.42.52.6.12.6.4
Perl5.8.45.8.45.8.45.10.0
PHP5.2.45.2.45.2.95.2.12
Ruby1.8.6p1101.8.6p2871.8.7p721.8.7p174
GCC3.4.33.4.34.3.24.3.2
xVM (XEN)3.13.13.13.4
Java1.6.0_041.6.0_101.6.0_131.6.0_18
Nvidia drivers169.12177.8180.44190.53
Mesa6.5.27.0.47.2.57.4.4
Apache2.2.82.2.92.2.112.2.14
OpenSSL0.9.8a20.9.8a20.9.8a20.9.8l


1. Latest build in development branch, 2010.03 is scheduled for a release in March, target build is 134.
3. OpenSSL 0.9.8a has various security patches applied which differs between the releases.

Tuesday, December 8, 2009

Last open build before 2010.03 closed (b130)

There will not be any more major features added to the OpenSolaris builds before the next release, build 130 was closed today and now the gate will begin to freeze in preparation for the 2010.03 release. The next four builds will focus mainly on bug fixes and device drivers.

Here is the schedule for the remaining planned builds before release:
onnv_131        01/04/2010      01/11/2010 Bug fixes, escalations & device drivers only
** Winter Break (4 week build)
onnv_132 01/19/2010 01/25/2010* Bug fixes, escalations & device drivers only
onnv_133 02/01/2010 02/08/2010 Stopper fixes only
onnv_134 02/16/2010 02/22/2010* Stopper fixes only

heads-up: guidelines for builds 127-130
ON Build and WOS Delivery Schedule

Sunday, December 6, 2009

No more NIS+

It has been planed for years now NIS+ has been removed from the development branch of OpenSolaris, soon all that nistbladm(1) skill will be obsolete. Since it integrated in time for b129 the first Solaris without NIS+ will be OpenSolaris 2010.03. If you still use NIS+ you can continue to use it as long that you stick with Solaris 10, but it might be time to start planing the migration.

NIS+ End-of-Feature (EOF) Announcement FAQ
PSARC/2009/503 Removal of NIS+

Thursday, December 3, 2009

First OpenSolaris build with ZFS dedup available

A new build (128a) in the development branch of OpenSolaris was released today. This build has support for the long awaited ZFS deduplication. Now you can download it and try it for yourself. This build includes other interesting features also such as zfs send dedup and zpool recovery.

While am at it, there have been a few other software releases this week:

Netbeans 6.8 Release Candidate 1 have been released and is available here.

VirtualBox 3.1 made it into a final release and is available for download.

Wednesday, December 2, 2009

OpenSolaris Text based installer update

There are finally some characters on the screen for the Text based installer project, the project pages have been updated. There is now access information for the project gate, build instructions and links to a code review (which does not seem to work right now).

The text installer is a essential part for bringing OpenSolaris into the datacenter, especally for the SPARC server which seldom have any framebuffers. Today the only installation option for OpenSolaris on SPARC is the Automatic Installer (AI) which requires DHCP service, a installation server and either access to external package repositories or a internal mirror.

This project will make it much easier to deploy single instances of OpenSolaris on your SPARC. X86 will also benefit from this, especially when installing inside xVM domains which today requires setup of VNC to perform a graphical installation.

A prototype for X86 was created this summer, read more here. The option for a interactive installation on SPARC have been promised for the next OpenSolaris release.

Tuesday, December 1, 2009

Installing the new Oracle 11g on OpenSolaris

I've already mentioned that Oracle 11g release 2 is now available for Solaris 10 on the X64 platform. I tried to install this release on the latest OpenSolaris development build (b127), it did initially fail but i found a quick workaround. This is of course not suitable to anything else than for pure testing and fun since it's currently only certified for Solaris 10 (Update 6 and later).

The helpful error massage from the installer was "Error in invoking target 'all_no_orcl' of makefile", but the log file shows what have gone wrong: "INFO: ld: fatal: library -lcrypto: not found"

The problem is that the OpenSSL libraries have moved out of /usr/sfw/lib in OpenSolaris, they are now located under /lib. The Oracle installer supplied the linker with the old location "-L/usr/sfw/lib/amd64" thous it failed.

A quick and very dirty fix to this is to provide a link to the library at the old location:
# ln -s /lib/amd64/libcrypto.so /usr/sfw/lib/amd64
Please drop a comment if there is a way of redirecting Oracle instead, but this was good enough for my little test.

After this I got the RDBMS up and running, including the web based Oracle Enterprise Manager if that's your thing.

As with Solaris 10 Oracle works fine inside zones, but in OpenSolaris zones are by default only a bare minimum of packages, therefore the following must be installed in the zone before Oracle:
# pkg install SUNWsprot SUNWmfrun SUNWuiu8 SUNWunzip SUNWbtool