Voyager Discussion

Contents 1 Discussion 2 Documentation 2.1 IBM Redbooks 2.2 Darwin Documentation 3 Technologies to study 3.1 Kernels 4 Design Discussion 4.1 Graphics Drivers 4.2 2D API 4.3 Input Handling and Event Queue 4.4 Window Manager 4.4.1 Ideas for a new Window Manager 4.4.2 Started development on Neptune 4.5 CMD 4.6 MM 4.7 Toolkits 4.8 SOM 4.9 Desktop (WPS) 4.9.1 Spirit and Purpose of the WPS 4.9.2 Security 4.10 REXX 4.11 Network Transparency 4.12 Backward Compatibility 5 Stuff 6 Questions 7 Implementation 8 Press coverage

Discussion

There is now a FAQ about the Voyager project here.

This document is already some kind of discussion, you can add your comments in here as well. For real discussion please consult the mailinglist voyager-dev@netlabs.org, or use the gmane.org web/news interface to it.

Documentation

IBM Redbooks

• SG24-4630-00 OS/2 Warp (PowerPC Edition) A First Look
• SG24-4639-00 OS/2 Warp (PowerPC Edition) Graphical Subsystem

We do have the first one digitaly, but not the second one. In case anyone finds that, let us know.

Darwin Documentation

There is plenty of documentation available about Darwin, a good start is Kernel Programming, which gives a nice overview about the design of the kernel.

• Filechange notification in Mac OSX "Tiger".

Technologies to study

• OpenGL, FreeGLUT, CPW (Application framework for windows, menus etc.). Here's a Readme text.
• glitz - OpenGL interface for Cairo, so one could run Cairo directly on OpenGL, see Xegl for a diagram.
• Glitz: Hardware Accelerated Image Compositing Using OpenGL and as PDF
• The State of Linux Graphics, comments and more comments
• Current gtk+ development- among other topics about cairo integration and a new theme system
• GTK+ Planning - a more recent plan of GTK+ development
• X on GL states very clearly, what has still to be done
• DirectFB with rootless XServer, hardware accelerated OpenGL, GTK+ backend, an experimental Cairo backend, Qt backend, DirectVNC, SDL backend
• Enlightenment and its support libs. Here's a Live CD.
  • Why Englightenment? So far I just knew it as a fancy desktop, what makes it special? Ktk 15:39, 30 October 2005 (CET)
  • Cinc: it's more than just a fancy desktop. It offers quite some libs e.g. HW accelerated canvas on top of OpenGL, libs for alphachannel blending of pics, antialiased text, imlib and quite some more.
• GNUstep.org
• OpenSolaris
• Ogre 3D The open source real time 3D rendering engine - for nice 3D effects on the Voyager desktop
• Microsoft Singularity - interesting paper by Microsoft about a concept study of a new OS
• Quartz and Quartz extreme, nice article giving an overview
• Klik: application bundles for KDE.
• More about application bundles: http://mozart.chat.net/~jeske/unsolicitedDave/app_wrappers_yet_again.html, http://mozart.chat.net/~jeske/unsolicitedDave/next_wrapper_description.html, http://mozart.chat.net/~jeske/unsolicitedDave/kde_encap_ideas.html
• Rox a quite nice desktop environment with features we also seek to implement
• Symphony OS - a desktop in XUL. We should integrate XUL as good as possible with Firefox
• eyeOS - web based OS
• Novell & Xgl - some stuff Novell did for Xgl
• X Developer Conference - some slides about the latest one, interesting stuff there!
• 40+ Suggestions for a Better Desktop
• The Tango Desktop Project - exists to help create a consistent graphical user interface experience for free and Open Source software.
• XGL v. AIGLX)
• VirtualGL
• The PulseAudio Server (PDF). A presentation about an audio server for *nix.
• XCB - X Compatibility Layer -> get rid of Xlib
• XAR - The XAR project aims to provide an easily extensible archive format.

Kernels

• http://haiku-os.org - OpenBeOS successor
• http://www.ertos.nicta.com.au/software/darbat/ - Darwin on L4
• http://osnews.com/story.php/15960/Introduction-to-MINIX-3 - Tannenbaum about Minix 3
• http://www.cs.vu.nl/~ast/reliable-os/ - Tannenbaum about Microkernels, gives a very nice overview about them

Design Discussion

Creating a binary compatible OS/2 clone may be tempting but is not strictly necessary. Important is the user experience. Which toolkit should be used for primary development?

Graphics Drivers

• X11
• OpenGL ES (an OpenGL only desktop without X!)
• DirectFB
• SciTech Snap Graphics

These graphics engines can be emulated vice-versa: You can run X11, OpenGL, and DirectFB programs on the same graphics driver. We should have a look at HW support when chosing the driver system. For example I've read OpenGL drivers are not that common (often they're not HW accelerated). A viable idea may be to just use OpenGL as the core API on top of something else. There're quite a lot OpenGL implementations running on different low level drivers or even Mesa.

2D API

• Cairo. Cairo has a PDF-like engine which seems to be inspired by Quartz of MacOS X.
• Cairo runs on almost everything, including OS/2 and X but also directly on OpenGL hardware with Glitz. So one could use it as a complete GPI replacement without the need of X one day.
• To make things easier 24Bit screens should be the primary target. Hell it's 2005 today...

It seems there are many possiblities here!

• General Graphics Interface
• Cairo (used by GTK+)
• Evas (used by Enlightenment)
• libart (used by Gnome)
• Anti-Grain Geometry
• Simple DirectMedia Layer
• Arthur (used by KDE)

We will have to use more than one. KDE programs use Arthur, GTK+ ones use Cairo, and so on ... These libraries are (mostly) used for display and printing. So we won't have a consistent graphical subsystem with this multitude.

All these libraries have several backends, e.g. X11, DirectFB, OpenGL, OS/2, ...

Even if we do have to use more than one we should focus on a library that is widly supported, at the moment my impression is that Cairo will win the race. Mozilla is using it for new versions, OOo will use it and GTK+ is migrating into it as well. Also projects like Glitz will dramatically speedup the rendering in the Cairo engine. See some comments of one of the GTK+ developers.

Input Handling and Event Queue

Between the 2D API and the window manager one layer is missing

• that cares for user input by mouse, keyboard, ...
• that knows the basics about windows and controls their behaviour (A window manager only modifies this behaviour!)
• a Session manager that manages the processes which want do to screen output
• a message architecture that connects everything and enables inter-client communication

Questions:

• Should we choose X for that? Quasi-standard, but old
• modify existing(?) systems
• port Presentation Manager -> access to source code needed
• code that ourselves?

• xynth might be an option

Window Manager

To manage the windows we need some sort of window manager. So far most implementations are done for X, we need to abstract that directly on Cairo.

• Window Manager Specification Project
• Inter-Client Communication Conventions Manual - Support this standard in order to let X applications run
• Waimea - WM which is using Cairo for all rendering, they write it's for X so one would have to check if this can be separated from X as well.

Ideas for a new Window Manager

What features should a new windows manager have? Discussion was moved to a separate page:

Ideas for a new Window Manager

Started development on Neptune

I started the development of the window manager. The project is called neptune, since this planet is one of those being observed by NASA's initial Voyager I mission. Currently I'm developing under XOrg, and have the physical screens simulated by several Cairo Surfaces (the whole window manager will be based on the multi screen design). Neptune has a to early state to publish anything. Currently I'm just experimenting with different architecture approaches, and am trying to figure out the best way to implement the multiscreen handling. Once everything is done, it should be easy to "port" it to run without the X server, since all the painting is done through cairo. So it should be a matter of having glitz using a framebuffer GL backend instaed of the glx.

CMD

• full capabilities of current CMD.EXE concerning
  • usage of system messages and error codes
  • nested io redirection
  • grouping commands with round brackets
  • &, && |, and || operators
  • correct wildcard matching for files with multiple extensions
  • PROMPT command
  • keep UPPERCASE environment variable names only ! (unlike 32-bit Command Interpreter from Jonathan de Boyne Pollard)
• proper implementation of EXTPROC command (fix errors with exptproc scripts not residing in current

directory)

• new general features
  • support extended command set of Win NT CMD.EXE
    • at least include string functions with extended SET command
  • extend IF clause with new conditions
    • os version match
    • free space on a drive
    • is drive removeable
    • is drive USB mass storage
    • more ?
    • easy syncing/unmounting for usb mass storage devices
  • support PATHEXT variable
    • start files with their associated program
  • automatic environment vars
    • DATE, DATE_SORTED, TIME, TIME_SORTED, WEEKDAY, DAY, MONTH, YEAR, YEARDAY
    • CMDLINE, ERRORLEVEL
    • OS, OSVER
    • BOOTDRIVE
  • support filename completion
• new internal or extended commands
  • INITVIOCMD determines a command being executed when initialized as a VIO session
  • INITPMCMD determines a command being executed when initialized as a PM session
  • SET allows to include equal signs in value of environment var
  • SET /P retrieves value of environment var from keyboard
  • SHIFT xx shifts parameter by positive number
  • ECHO /N does not append CRLF to output
  • CDD command changes directory and drive
  • TIMER command starts, stops or displays the value of a timer
  • SLEEP n waits for n secs
  • PAUSE n waits for n secs
  • implement START command with all features of DosStartSession, among others implement
    • /WAIT (synchronous start)
    • /NC (NoAutoClose)
    • /DOS[:settings.dat] (if DOS-Box still available)
    • /POS:x,y,x1,y1 (initial window position)
    • /ICON:icon.ico (system menu icon)
  • MOVE files with implicit copy/delete across partitions
  • exlicit UNLOCK command
  • implicit unlock with COPY, DEL, REN, MOVE (very usefull, but also dangerous ?)
  • macro support with ALIAS command
• new batch handling or batch commands
  •  %* specifies all parameters (like %1, %2 etc)
  • in for loops support %%a%f, %%a%p, %%a%d and more for specifying filename, path and driveletter of matching file
  • GOSUB - executes a subroutine
  • RETURN [xx] - exits a batchfile or subroutine [with errorlevel]
  • CANCEL (ends batch processing, but does not close window as EXIT does)
• Integration of any REXX replacement like with current CMD.EXE
  • call of REXX Scripts with CMD
  • usage of exit handlers
  • manipulation of environment vars within REXX (changes remain after end of script, unless setlocal/endlocal() is called), including BEGINLIBPATH and ENDLIBPATH
  • readonly access to the LIBPATH value as an environment variable

MM

• The concept of IO procedures should be retained at least for file IO and format converting.

Toolkits

• GTK+
• GnuStep (look at Apples programs to see what's possible. Yes I know Apple uses Cocoa not Gnustep)
• other?

SOM

Dropping SOM for something better? Creating a poor mans SOM kernel isn't too difficult (Cinc: there's already a working prototype for the basic functionality).

If SOM is dropped, no binary compatibility for old WPS enhancements is given. This may break a considerable amount of OS/2 apps. This may not be an issue if applications have to be recompiled anyway for a new plattform. Cinc: I don't see many OS/2-apps which are worth to be supported no matter how much work it needs. Binary compatibility is out of sight IMHO. Nobody really needs it (anyone really using OD nowadays?).

Cinc: Note for those with reading (well more with comprehension problems): I'm talking here about SOM and WPS programs, not OS/2 programs in general (maybe someone noticed the headline which says "SOM"). Anyone knows any widespread commercial WPS extension except ObjectDesktop??? I don't. I don't believe it's worth to support the almost not existant WPS integration of StarOffice5.xx. Geez, that thing is from the computer stoneage now. So before whining about missing WPS binary compatibility first try to find something needing it... The remaining critics are invited to add binary compatibility if they want it. Nobody will hinder them. Quite the opposite, the toolkit comes with every eCS so just join the effort.

Desktop (WPS)

• The WPS should be designed with REXX scriptability in mind (like WPS-wizard addon). Ruby may be another language worth to be supported.

Work is in progress. See the desktop page.

Note: Don't use the linked page for discussion. If you want to discuss use this section or the discussion page (hey that's what it was made for).

Spirit and Purpose of the WPS

The WPS only shows those parts of the computer system you have to deal with as a user.

Its objects are taken from the user's world of experience. The implementation details are hidden.

It is very intuitive to use. A Graphical GUI shows you, what is possible to do and what not. (Unlike a command line! The unix shell TAB key is a notable exception. But generally this can easier shown with graphics.)

The WPS is only for the user! Programs have DLLs and APIs of their own. (If not, then there is something missing on the system.)

Some WPS objects show or represent system details. That is, because daily administration tasks can also be regarded as "usage".

Security

Security should be a concern when creating the desktop application. GTK isn't secure by design so the desktop won't be either. Nevertheless we shouldn't open additional holes. The concept of the desktop allows arbitrary class and method replacement. So trojans can do whatever they want with it when they manage to get into the desktop app.

Ideas:

• Only allow signed/hashed classes to be loaded
• Maybe require that every author uses GnuPG to sign his/her class
• User has to allow loading any additional classes once
• "Save" classes will be listed in the users save class list
• The save class list is signed (GnuPG) to prevent altering

Is this sufficient?

REXX

• REXX should be implemented like AppleScript so controlling of applications from other apps is possible (commands not tied to one process).
• Regina may be used as an interpreter.
• any REXX replacement should implement the following
  • true CMD integration like under OS/2
    • access to environment vars of the calling interpreter. Cinc: ???
  • support of the full REXX C API or an equivalent concept (macro space control, app handlers, WPS object access API etc). Most of this is supported by Regina AFAIK.
  • complete reimplementation of REXXUTIL ! I seem to remember having seen such a project for Windows and Unix.
• It should be possible to write GUI based apps with REXX, a bit Dr.Dialog like

Network Transparency

It would be cool to execute applications remotely.

What we don't want is a distributed operating system like Amoeba or Inferno.

The network transport would be inside Cairo or between Cairo and OpenGL. (Above Cairo makes no sense, because Cairo can also manipulate memory-stored images; below OpenGL is too much data overhead.)

When a user runs an application remotely, he wants to access devices on the client and the server. Examples for devices are

• printers
• mice, keyboard
• audio
• screens
• hard disks
• USB attached devices
• DVD burners

We need some kind of abstraction layer. It would be nice, if everything can be controlled by WPS objects:

• drag application on computer -> application gets executed there
• double-click (or also drag'n'drop) keyboard object -> this keyboard works for my console session

Backward Compatibility

The question about backward compatibility is indeed a good one, it's risky to break with the current OS/2 API for religious reasons but one should have a look at that more from a technical point of view. The OS/2 API is not really modern anymore, programming directly on PM is much more work than using a modern toolkit like GTK+.

At the moment it's unfortunately unlikely that PM and WPS source get released by IBM.

Options:

• implement parts of PM and GPI on top of the new 2D API and try to integrate it with the toolkit choosen (same look & feel, clipboard support, D&D, etc. That's a lot of work, the question is if this is really necessary. We would have to do the same for supporting other toolkits like qt (which is a requirement).
• Implement only some crucial parts to ease source code migration. For example message queues, concept of object windows (often used to comunicate with background threads).
• Map often used PM-functions to counterparts of the chosen toolkit.
• Use a VM and run OS/2 or eCS inside the VM. There are very promising opensource VMs like
  • XEN
  • QEMU

XEN seems to be the most promising one at the moment because it will support dual core CPUs in the future so one can run the guest OS on a separate CPU core. So far one had to alter the guest OS in sourcecode but that changes at the moment because they integrate parts of QEMU sourcecode inside XEN to emulate those parts (memory allocation and such stuff).

Stuff

• With the advance of virtualization technologies like XEN, Vanderpool and multicore cpus it becomes more and more common to have a second, virtual computer running in a VM. It should be possible to map the virtual screen of the VM to a physically existing display. (The idea is to give the user the feeling that he actually has two different computers (with a shared set of input devices). It might even be possible to add another set of input devices, so that two users could work on one computer. (one or maybe even both using an emulated computer...)

Lexip: I think this should be no problem, since the WM will feature virtual desktops, which can be placed on any physical screen. Thus, if one places the virtual desktop holding the VM on a physical screen, you have what you want to. It will even be possible to split screens, and let several virtual desktops share one screen.

• What's bad about Unix. Nice reading
• Plan 9. The system the author of the reading above advocates to some degree. Not compatible enough, but interesting concepts.

Questions

• the Xegl diagram mentioned above only shows the path of the graphics; the path of e.g. mouse and keyboard input is missing; that's not as trivial as it sounds (it has been missing in Xegl, when that was stopped); will we use OS/2-like driver model or wait for x.org 7.0 (currently RC1!) and take out the input system from there? Other sources (hotplug, ...)?
• one good thing about OS/2 are the fast interactions when you click somewhere, like opening a folder or getting the focus on a window. Why is that like this? We should try to keep that stuff fast.
  • I think the problem of current user interfaces is that they have a lot data dependencies. This increases the response time as they often have to wait for the data to become available before they can start to draw. Data dependencies mostly stem from an overdose of flexibility (I'm talking windows xp now).
    • XP's start menu: it is based on the directory structure in the startmenu folder. So in order to draw the menu it has to read from the hard disk several times as it needs to find out the structure and load the icons of the programs (they are mostly cached after the first time).
    • Drive windows, where the window needs to gather/update the drive information before it can be drawn the first time.
    • Explorer windows that display extended file info (eg id3 tags of mp3s) need to parse the files first. (in xp a corrupt media file can block the explorer, before you even get the chance to touch it)

Short:

 Eyecandy and advanced/unnecessary features (even when switched off) waste cpu time.

• do we want the traditional window manager<->application scheme? With the Gnome/KDE interfaces? Then we are deep in X again ...
  • the question is if we can incooperate the WM functionality in GTK somehow. As far as I remember Fresco implemented GTK API as well so one might have a look at that.
  • IMO GTK runs on top of a WM. The WM just gives GTK a surface to work with but manages where on screen this surface will be painted. Even OS/2 has something like a WM managing the windows for us.

• As soon as we further expand the usage of WPS objects, the templates folder will grow and likewise the number of possible interactions and combinations of objects.
  • How can the objects be sorted or structured so that the template folder won't scare of casual users? Answer: Just use the means of the WPS to do so. Override the right methods and use your programming brain. It's all documented int the WPS Programming Reference.
  • How can we tell the users about interactions?
    • Example Drag'n'Drop: When the user starts dragging, display possible targets with arrows, colors, or other marks?

Or even let the objects come out of their hiding place, if they are not visible on the screen or in subfolders?

• • • Example Creating of structures: XWorkplace has "Create new" in the context menu with the entries "Data File", "Folder", "Url Folder", "Program Object" as standard. The idea is good, but can be better. Take a strategy game you don't know and build up a base very fast. Now let an OS/2 newbie do the same with OS/2 objects. It is much more complicated. There is no way to see what objects fit into each other and you have to know that there is a templates folder and there is a system setup folder with palettes and themes. You could sum up these sentences as the demand for "Feedback for the Users".

Implementation

You will find links to pages with some beef here later on. While the Voyager page is mainly for discussion and brainstorming the following pages go into detail and present the actual design decisions, roadmaps etc.

• Information about the desktop development.
• Some ideas about media handling in Voyager

Press coverage

We slowly start to get some press:

• Slashdot - Keeping the OS/2 Flame Alive
• And the original at Newsforge