Open Source in Mac OS X Server

Apple has leveraged open source software to make rapid progress in developing a new server business after its efforts to enter server markets fell flat in the mid 90s.

Apple had initially intended to ship its desktop version of Mac OS X in 1999 and then in 2000, but a number of complications related to Rhapsody held things back:

•the need to strip out dependancies on third party commercial code, including Adobe’s Display PostScript
•the subsequent replacement of the entire display system with something new
•the demand to incorporate native support for existing classic Mac OS development
•the need to update the aging Unix foundation in NeXTSTEP, which hadn’t seen regular development since 1995

Darwin: 2000

Part of the work to move from Rhapsody to Mac OS X included efforts to make it possible to release the kernel and core OS as open source by removing proprietary entanglements.

Rhapsody got a final last gasp with the silent release of Mac OS X Server 1.2 in February 2000; in April, Apple released Darwin 1.0, the open source core of its new OS and a significant departure from Rhapsody.

While Mac OS X as a PC operating system was now dead, Darwin was offered in builds for both Power PC Macs and Intel PCs, and could theoretically be ported to other platforms. However, the point of Darwin was not to act as a rival to Linux, but rather to allow Mac developers better access to the internals of the new OS.

Apple also announced the open source release of its QuickTime Streaming Server as Darwin Streaming Server, the first open source, standards-based, free Internet streaming server. It was adapted specifically to run on FreeBSD, Linux, Solaris and Windows NT.

Criticism of Darwin

Opening Darwin was not simple. Apple’s development work on Mac OS X is managed using XBS, a complex and publicly undocumented internal build system; since nobody outside of Apple has access to the system, it was very difficult to build functional installs of Darwin from the source code as changes were introduced.

Outside developers complained that Apple wasn’t providing enough documentation to make sense of its development system, and that it was very difficult to make meaningful contributions to the Darwin project.

Further, Apple intentionally held up access to code changes prior to commercial releases, both in order to keep features secret but also to stabilize a set release prior to shipping. During that time, Apple would continue to work on the next version, leaving outside developers unplugged from what was happening.

Criticism of Apple’s open source efforts helps to highlight the difficulty of opening up commercial projects. Sun faced similar issues in the release of Solaris as OpenSolaris and StarOffice as OpenOffice.org; once a project opens, its can be challenging to deliver commercial releases in conjunction with the open source version.

Darwin is not GNU

In the case of Darwin, Apple was more interested in providing third party developers with access to internal code than in creating a community where developers would write its operating system for it.

Apple’s earlier experience with mkLinux, an open source Linux project the company hosted from 1996-1998, suggests that Apple did not have wild illusions that the “open source community” would rush in and complete its operating system.

Efforts to portray Darwin as a failure for not being more similar to GNU/Linux or other non-commercial open source projects neglect to consider that Linux and GNU do not have thousands of engineers on their payroll and millions of desktop customers to entertain.

The only way for Apple to dramatically embrace open source would be to spin Darwin off as a completely independent project, but that would neither benefit Apple nor outside developers. As long as Darwin provides access to the internals of the commercial Mac OS X, there will be justifiable friction and complaint.

Darwin Version Numbers

At the May 2000 WWDC, Apple shipped Mac OS X Developer Preview 4, announced that a Public Beta version would follow later in the year, and promised a commercial 1.0 release in January 2001. That initial 2001 version was later numbered 10.0, to follow the existing Mac OS version numbering scheme.

While Darwin was initially assigned its own version numbers--1.2.1 corresponded to the Public Beta, and 1.3.1 to the commercial Mac OS X 10.0 release--it was later changed to match Mac OS X releases.

At Mac OS X 10.1.1, Darwin jumped to version 5.1; the major five suggested the fifth version of NeXTSTEP while the minor one related to the minor release version of Mac OS X 10.1.1.

Today’s Darwin 8.8 coincides with Mac OS X 10.4.8: Darwin’s eight generations of NeXT technology, ten branded versions of Mac OS, the fourth major new release of Mac OS X, and eight minor revisions. The build number of Mac OS X releases also reflects the Darwin 8.8 major version: 10.4.8 is Build 8L127. (NeXTSTEP 8.8!)

Mac OS X Server 10.0, 10.1: 2001

While Apple internally retained the ability for Mac OS X to run on Intel hardware, the operating system was no longer based upon the OpenStep specification; Mac OS X 10.0 instead became a new lineage of NeXTSTEP.

Server versions of Mac OS X 10.0 and the free follow up 10.1 version mostly migrated the existing features of server applications included in the Rhapsody-based Server 1.0.

Apple focused mainly on getting Mac OS X ready for desktop users, as without a desktop product a Mac server wouldn't have much of an audience.

Quartz vs X Window

Part of that desktop focus was the move from a Copland Mac OS 8 look in Rhapsody to the entirely new Aqua interface. Aqua demonstrated the compositing and translucency features of Quartz, the new graphics rendering and compositing engine that replaced Display PostScript.

Given Apple’s expanding use of open source code, some questioned why the company developed its own graphics technology rather than using the open X Window specification used by Linux and most other Unix-like systems.

The answer helps to explain why Apple sometimes uses open source, and sometimes develops new technology independently: existing software is not always the best tool for the job.

Transition to a Modern OS

Mac OS X 10.0 and 10.1 faced some of the same transition issues as Windows 2000. Microsoft struggled to migrate its DOS based Windows products to the NT kernel, but was unable to do this by Windows 2000 (NT 5.0). The followup Windows XP (NT 5.1) finally ended the line of Windows versions based on DOS.

Similarly, prior to Mac OS X 10.2, the classic Mac OS 9 still offered some functionality that wasn't yet available in its Unix-based Mac OS X. In 2002, Apple could finally announce the burial of its classic Mac OS.

The difference was that Microsoft migrated its users from DOS to NT, which was actually more proprietary, closed, and unique; MS-DOS was not open, but at least its internals were widely understood. DOS was largely based upon CP/M, which Microsoft used as a foundation to improve upon. NT was written from scratch.

Apple moved in the opposite direction, from the completely secret Mac OS built from scratch to one based upon Unix, offering application compatibility and architectural similarity with other Unix systems.

The result has been that Microsoft has isolated itself from external advances made in both operating system and server technology, while Apple has greatly benefitted from its participation with NetBSD, FreeBSD, OpenBSD, GNU, and other open source projects.

Mac OS X Server 10.2 Jaguar: 2002

Apple’s pace in developing Mac OS X rapidly gained ground with Jaguar, the first version of Mac OS X ready for mainstream desktop users. For Jaguar Server 10.2, Apple introduced:

•JHFS+, a journaling version of the Mac file system
•Open Directory, an LDAP based directory server--resurrecting the Open Directory name from PowerTalk era
•an improved QuickTime Streaming Server 4 and QuickTime Broadcaster
•a standardized printing architecture build upon the Common Unix Printing System
•the Xserve, its first custom designed server hardware since the AIX-based Apple Network Servers
•Server Status remote management tools based on SNMP

Those features show how Apple both significantly benefitted from using open source and open standards, while also developing new code, portions of which were shared, including improvements to Streaming Server and HFS.

Apple developed a business behind selling Jaguar at retail, and it became its first operating system with significant retail sales. Those sales enabled Apple to invest more development efforts into improvements to the Mac OS X frameworks and build upon the foundation established.

Mac OS X Panther Server 10.3: 2003

The following year, Panther introduced the new Safari browser, a reworked Finder, Exposé, and Fast User Switching.

On the server side, Apple continued to update and incorporate more open source projects into the system. Using Samba 3, Panther Server could act as a Windows domain controller. Apple also bundled MySQL and PHP for web development and improved and updated other services, including its directory server and mail servers.

Mac OS X Tiger Server 10.4: 2005

Tiger Server continued to introduce a mix of open source software with an Apple designed user interface, along with custom developed applications, some of which were designed to accommodate or parallel Windows features.

For example, Apple added support for Access Control Lists. Support for ACLs offers a mechanism for more complex file permissions, compatible with those used in Windows.

Tiger also introduced Portable Home Directories, which allow desktop clients to log into a server, download a user profile, and then keep their user directory in sync with the server. This is similar to roaming profiles and synced shared folders in Windows.

A desktop user can log in and sync their local user profile with the server, keeping desktop items, other files, and some user preferences all in sync between multiple machines. The Xserve mini described the concept as something Apple should deliver to home users; it is currently only available in Tiger Server.

Other custom developed additions include Xgrid distributed processing and 64-bit application support. Apple also incorporated its WebObjects deployment server into Tiger Server, which had earlier been a $50,000 license.

Apple also developed the iChat instant messaging server based upon the open Jabber XMPP. This change also appeared in the iChat client: in addition to working with the proprietary AOL IM system, iChat can now be used with any Jabber system, including Google Talk.

Tiger Server also bundled Blojsom, a Java version of the Blossom weblog server. Apple designed custom skins for the weblog server, making it easy for companies running Mac OS X Server to set up simple blogs, but it left blogging as a poorly positioned service, since it was painstakingly difficult to make any significant changes to hosted blogs.

Later this year, Apple is releasing Mac OS X Leopard Server, which expands upon Tiger Server with the usual refinements, but also introduces entirely new workgroup and collaboration services, including the iCal Server already released as a free open source project. The next article takes a look.