RFC: GNOME 2.0 Multimedia strategy

Christian Schaller <Uraeus linuxrising org> · 18 May 2001 22:35:56 +0200

During GUADEC 2 there was some discussion about what we should do with multimedia support in regards to GNOME 2.0. The general concensus is that ESD has to go, and there was also a leaning towards replacing it with artsd in order to both get a better soundserver and to have increased compatibility with KDE. In this document I will try to outline why I think that will be a to limited approach. I would like to point out that I have personally been trying to help out with GStreamer for some time now, so I could be percieved as biased :) Why is this important to decide now: Due to the limited time until the GNOME 2.0 freeze we need to quickly finalize the strategy for Multimedia in GNOME 2.0 in order for audio and video applications developers to be able to port their applications in time. What should be done: The basis for my proposal is that we adopt GStreamer (http://www.gstreamer.net) as the official sound and video API for GNOME 2.0. This will give wide ranging advantages for GNOME. a) We will not only have sound but also video support. Gstreamer today supports AVI, Quicktime, Mpeg, FLI and VOB files and more are in the works. b) Greater flexibility in regards to sound output. All GNOME applications based upon GStreamer will then be able to output either directly to the native sound architecture or through sound servers such as ESD, artsd or gnostream. The idea here is to have a controlcenter applet which will let the user choose a default output type. Today many of the GNOME multimedia applets and applications either just support ESD or they have a large library of their own output plug-ins. This we can now solve in a much better way. Just porting these apps to artsd would just give us the same situation with a better soundserver. c) Using GStreamer will be a better long term solution for GNOME since it is created using glib for the backend and all the current sample applications are developed using the GNOME libraries. d) While artsd is really good as a soundserver, its design is not that ideal for videoserving. Using GStreamer we can output to artsd for KDE compatability (gstreamer already supports artsd) but also switch to a more video&audio based setup like gnostream (http://gnostream.sourceforge.net) when it becomes available witouth having to rewrite applications. This also solves the problem with audio/video sync when using a standalone soundserver as those of you who talked with Jim Gettys during GUADEC2 probably are aware of. e) We will have a full featured multimedia architecture making GNOME the natural development environment for MM applications. Remember GStreamer isn't a mediaplayer, it is a multimedia architecture which already supports encoding of videos and sound too. f) We will have a multimedia architecture with broad developer support and commercial support. RidgeRun has 1 fulltime and 2 people part-time on GStreamer. GStreamer has in addition to that we have around 12-13 developers helping out in their spare time or as partime on company time. Considering the number of commits to GNOME multimedia currently in CVS I think this will give us a incredible incredible increase in developer time on our multimedia support. g) Long time GNOME developers are already working on GStreamer integration and improvement. Arik Devens for instance is the maintainer of the gstmediaplay Mediaplayer application bundled with Gstreamer. Erdi Gergo is working with us to make his bonobo-media compoents work with GStreamer. Bastien Nocera (hadess) has created gnome-vfs input and output plugins and is creating a iTunes clone for GNOME based upon GStreamer. h) Having something as GStreamer as part of the GNOME development platform means that we could for instance use it in Nautilus to easily create a music view which supports all audio formats GStreamer supports instead of having to create new code for each format. We could also do video preview if that is of interest. i) A GStreamer Mozilla plugin is in the works which would be a great benefit since all our applications using Mozilla for html like Mozilla, Galeon and Nautilus would be able to easily support multimedia webpages out of the box. For a full rundown on what is supported and is worked upon in GStreamer I suggest taking a look at the current roadmap for the upcomming 0.2.0 release of GStreamer. http://www.linuxrising.com/files/gstreroadmap2.html Challenges for the GStreamer project: Currently GStreamer has a small core component written in assembler (cothreads). This part is currently only ported to Intel, Sparc, ARM, Alpha and PowerPC. The remaining asm cores can be writen blind (like succesfully done with SPARC) but testers to confirm success would be needed. In the long run these asm parts will probably be replaced by GNU Pth with IBM's N:M mods which is a pthread safe C-only implementation. Another problem is that while GStreamer very much want to be part of GNOME, they also want to keep their independence due to being also targeted at non-gnome systems, especially embeded ones. So moving GStreamer from SF CVS will probably not be of interest to the GStreamer developers. Maintaining a release library at ftp.gnome.org should however be no problem. A major feaure currently missing in GStreamer is MIDI. There is no special reason for this except that none of the developers have gotten around to implementing it yet. Another thing needed is a simple API layer between GStreamer and GNOME to enable simple stuff like `gnome_play_sound("bleep.wav")` for all those apps not in need of the full GStreamer API. This will be based upon the current gstmediaplay API (libgstplay). What is currently needed: In order to get this kickstarted and ready to run for the GNOME 2.0 release there need to be a general consensus from the GNOME hackers that this is the way forward (that would be true either way actually), so we can send out the message to applet and applications developers to start looking into porting their applications to the new API. This decision doesn't need to mean we don't bundle and use artsd with GNOME 2.0 and use that as the default GStreamer audio output, but it gives us flexibility for the future and users the option of using other technologies which might fit their needs better. Christian 