analyzing architectural evolution issues of multimedia ... · systems that are reusable,...

Multimedia Tools and Applications, 22, 31–51, 2004c© 2004 Kluwer Academic Publishers. Manufactured in The Netherlands.

Analyzing Architectural Evolution Issuesof Multimedia Frameworks

M. PINTO [email protected]. AMOR [email protected]. FUENTES [email protected]. TROYA [email protected] de Lenguajes y Ciencias de la Computacion, Universidad de Malaga,Campus de Teatinos, cp. 29071, Malaga, Spain

Abstract. The growing complexity in the development of Web-based services in general, and multimedia servicesin particular, makes necessary the application of sound development methods. New multimedia devices, codingalgorithms, network protocols, etc., are continually appearing but, unfortunately, current solutions for developingmultimedia applications do not accurately support architectural evolution issues for already deployed applications.Thus, the latest Software Engineering technologies should be applied to the development of open, reusable, andhigh-quality multimedia and Web-based software. In this paper, we apply component and framework technologies,two of the current trends in Software Engineering, to the development of multimedia services over the Web,presenting and comparing widespread solutions in use today.

Keywords: Component-Based Software Engineering, Framework Technology, Web-based Multimedia Services,Distributed Web-based Systems, Java Media Framework, MultiTEL, Software Engineering for MultimediaSystems

1. Introduction

The spread of the Internet and the World Wide Web has contributed to the development ofnew and complex services, many of them involving the interchange of multimedia informa-tion. Although these Multimedia Services (MSs) are not new, they are in constant evolutiondue to the advances in technology, which now make possible the implementation of syn-chronous real-time services that exchange high-quality audio and video. Unfortunately,current solutions for developing MSs neither provide powerful tools for deploying complexWeb-based multimedia systems, nor accurately support architectural evolution issues foralready deployed applications.

The rapid construction of reliable, portable, and efficient MSs faces the growing com-plexity of these systems, the heterogeneity of multimedia devices, and the diversity ofoperating systems and communication platforms. This argues for the development of newsystems that are reusable, extensible, and open. In this context, Component-Based SoftwareEngineering (CBSE) [2] and Framework Technology [4] are becoming essential in orderto reduce the cost and improve the quality of software. Consequently, our work focuses onhow to apply the aforementioned technologies to establish the development methods thatguide the design and implementation of multimedia applications.

32 PINTO ET AL.

Table 1. Multimedia development approaches.

API Digital Media Library (Silicon Graphics)

– Audio and video files and live stream creation

– Presentation of audio and video

– Encoding of audio and video

Components Windows Media Tools (Microsoft)

– Programs, plug-ins, and utilities for creating files and live stream content

Windows Media Services (Microsoft)

– Distribution of audio, video and other media

Windows Media Player (Microsoft)

– Presentation of video, audio and other multimedia data types

Frameworks Java Media Framework (Sun)

– Media files and live stream capture and presentation

– Media conversion

– Video and Audio conference using RTP

– Multimedia data management reference architecture

– Cross-platform technology

MultiTEL (Gisum, University of Malaga)

– Media Files and live stream capture and presentation

– Video and Audio conference and video on demand

– Multimedia services development reference architecture

– Cross-platform technology

– Plug-and-play mechanism of multimedia resources

Current platforms offer APIs to deal with the capture and management of multimediadevices (e.g., Silicon Graphics’ Digital Media Library [14]), and many of them are basedon components (e.g., Microsoft’s Windows Media Technology [10]), and frameworks tech-nologies (e.g., Sun’s Java Media Framework (JMF) [15]) or on both such as MultiTEL [6],a non-commercial compositional framework which was developed by the authors of thispaper. Table 1 shows a comparative study of these solutions.

In a component-based platform, components act as replacement units that can be com-posed to build extensible, reusable, and open systems. This technology allows the construc-tion of an application by plugging in commercial off-the-shelf components, developed atdifferent times, by different people, and with different uses in mind [2]. However, the flexi-bility provided by components should be complemented by a framework that imposes someunderlying architecture [9]. A framework is a reusable, “semi-complete” application thatcan be specialized to produce custom applications [4], and therefore framework technologycan be considered a better approach than isolated object-oriented or component-orientedparadigms.

The main goal of this paper is to present a comparative study, mainly between twoMS development frameworks: JMF and MultiTEL.1 They present similar characteristics

ANALYZING ARCHITECTURAL EVOLUTION ISSUES 33

(see Table 1), for instance both are developed in Java, define a reference architecture, andproduce MSs that can be run everywhere. Both aim to provide powerful abstractions andmechanisms to facilitate the deployment of MSs and to support architectural evolution ofderived services. The main criteria applied in this study is the extensibility and adaptabil-ity of each framework to the evolving requirements of MSs, such as the use of differentcommunication protocols, the negotiation of data formats or the management of partici-pants with different user profiles or that play different roles in the service. While makingthe comparison, we have found how some limitations and inconveniences of JMF (such asthe complexity of the framework architecture—which is poorly documented—making theextension of base components a dreadful task, and the lack of a full architecture for theconstruction of complete MSs) are easily managed using MultiTEL. In addition, we pro-pose some enhancements to JMF that try to mitigate some of these limitations. The resultingframework that extends JMF provides a reference architecture to develop open MSs over theWeb.

We illustrate our proposal with a case study of a tele-education service at the University ofMalaga (TeleUni). TeleUni is a videoconference service where multiple users (the students)interact and collaborate remotely with another user (the teacher). The teacher is also ableto show educational videos and save the lecture, so that the students can download it later.The latter functionality is designed with a video-on-demand service that is part of theTeleUni system. Finally, people (students or teachers) who are interested in attending thelecture must own the appropriate multimedia devices, e.g., a camera, a microphone, and aspeaker.

2. Technologies for multimedia services development

The main difference between MSs and other distributed services is the diversity of hardwaredevices (e.g., cameras, microphones, speakers, etc.) needing to be managed, which increasesthe complexity of multimedia programming. These devices capture/present media data indifferent formats, which makes format negotiation between the participants necessary. Inaddition, other requirements must be also taken into account, such as participant preferences,the capacity of the network, or the quality of service (QoS) required. Each participant in theservice may have different platform-specific devices that produce real-time data in differentformats. Users should be able to execute MSs in heterogeneous systems, and devices mustbe modeled by components that are managed locally by each participant, offering a commoninterface to the service.

Other important issues concerning multimedia programming are media synchronizationand the use of different communication protocols in different versions of the same service,like UDP and RTP (Real-Time Transport Protocol) [13]. RTP is a transport protocol designedto be used by applications with real-time requirements. It provides information about theformat and the time of the transferred data allowing the correct reconstruction of data in thetarget. Furthermore, it offers control information about the quality of current transmissionand the participants engaged in the session by the RTCP (Real-Time Transport ControlProtocol) protocol.

34 PINTO ET AL.

Figure 1. MultiTEL platform.

2.1. MultiTEL overview

MultiTEL [5] defines a compositional model implemented as a component platform,2 and anapplication framework for developing multimedia services for the Web, including the man-agement of network resources and multimedia devices. This framework models multimediadevices as components that implement standard interfaces. Currently, several multimedia,video on demand (VoD), and collaborative services developed using MultiTEL are beingused in the area of educational and CSCW services at the University of Malaga.

The MultiTEL platform is divided into two levels, as shown in figure 1. In the higherlevel are running MSs that are composed by the service, multimedia, and network sub-systems. In MultiTEL, an MS is made up of a set of components which model entities ofthe real world, and a set of connectors that implement the coordination protocols betweencomponents (the service connector of figure 1 coordinates the interactions between service,multimedia, and network subsystems). Component/connector communication is defined ascomponents throwing events and receiving input messages, while connectors catch eventsand send output messages. However, components and connectors are not wired together ina fixed manner, but are dynamically connected by using the services provided by the UserService Part (USP) object, located in the lower level of the MultiTEL architecture. Whena component sends an event, the USP determines the connectors that must manage thisevent by consulting some information about the MS software architecture (figure 1). Thus,a component does not know the reference of target connectors. From the users’ point ofview, the USP is simply an applet that is downloaded through a web navigator.

The use of MultiTEL middleware platform brings out additional benefits to derivedapplications, encouraging the handling of architectural evolution issues in the same wayas Design patterns (DPs) [8] do. Really, some portions of our framework illustrate the


Figure 2. MultiTEL resource manager.

beneficial applications of DPs, but in this proposal you can make use of them from a uniqueaccess point: the MulitTEL platform [12]. The core services offered by this platform arethe dynamic composition of unknown components, separation of interface definition fromcomponent implementation and the use of a communication mechanism more powerfulthan the message passing typical of object orientation.

Following the MultiTEL platform description of figure 1, another essential component isthe Resource Manager (RM). In MultiTEL, the devices are captured through the RM thathides the device platform dependencies by providing a unique identifier and a common in-terface for each device type. For example, producer devices implement the ProducerDeviceinterface and receiver devices implement the ReceiverDevice interface.

Figure 2 shows how an MS asks the resource manager for capturing devices, by only sup-plying the unique identifiers (“camera” and “speaker”) of devices and a data format. Hence,the MS does not need to know the actual implementation class; it only has to invoke the meth-ods of the common interface implemented by the device object. Only the local RM knows theimplementation class, which is stored in the ResourceInfo structure. Thus, the componentsthat model the devices join the service at runtime through a plug&play mechanism.

2.2. JMF overview

The JMF 1.0 API was introduced at the end of 1997 to enable Java programs which incorpo-rated time-based media. However, it was not until the JMF 2.0 API, published in November

36 PINTO ET AL.

Figure 3. Java media framework architecture.

1999, that this API provided support for capturing and storing media data (figure 3). Sinceit is a Java/Sun technology it also defines and uses some JavaBeans, the component modelfor the graphical components of Sun.

JMF defines two different ways of programming. The first one is the presentation oflocal or remote audio and/or video files, or real-time data captured from a camera or amicrophone. As you can see in figure 3, JMF offers plug-ins to do format conversions,multiplex or demultiplex data or synchronize several media. Second, to develop distributedMSs with real-time streams, JMF provides a set of components that implement the RTPprotocol (through the RTP API).

Like MultiTEL, JMF provides a Resource Manager (RM) implemented by the componentCaptureDeviceManager. As we can see in figure 4, the RM/JMF deals with the registrationof multimedia devices with the addDevice() method and provides device information tocapture them with methods getDevice() and getDeviceList(). However, it does not performa real capture as MultiTEL does.

Let’s consider the capture process tagged twice as (3, 4, 5) in figure 4 (this figure shows theassociations between JMF classes resultant from the implementation code of a multimediaservice that captures and presents data from a camera). At first we can notice that JMF doesnot provide an atomic method to create a device. That is, an application must invoke severalmethods that create a set of chained components, although only one of them really capturesthe device. In JMF, a device is modeled by a component that implements the captureDeviceinterface and extends the DataSource class. By invoking the getLocator() method over oneof the CaptureDeviceInfo structure’s elements returned by the RM/JMF, we obtain a medialocator. This locator is the entry to the point 3 of figure 4. The code in tags (4, 5, 6) uses theJMF final class Manager that is the access point for obtaining system dependent resourcessuch as DataSources, Processors or Players. All this code gives an idea about how deepmust be the user knowledge about multimedia programming, and in particular, about theJMF API classes, and only for capturing a device.

Regarding data presentation, JMF uses the Player component to reproduce the streamsof data received through the network. As showed in label (6) we must pass a DataSourcecomponent as parameter to the player creation method createPlayer(). The DataSource (edsvariable) is used to deliver the input media-stream to the Player, that is, the player reads theinput media from this DataSource. This component initiates the data capture as its start()method is invoked.


Figure 4. JMF device capture/presentation process.

3. MultiTEL versus JMF

We are going to evaluate the MultiTEL and JMF frameworks based not only on the com-plexity of building multimedia services with them, but also on how extensible, reusable,maintainable, and interoperable they are. Another important issue is the extra functional-ity that these frameworks additionally offer to the capture and management of multimediadevices. This is a very important issue because device management is only a tiny area inthe development of a complete multimedia service, and this is precisely the main differencebetween JMF and MultiTEL. The comparison is based on our own experience in buildingbasic videoconference and VoD services using these platforms.

38 PINTO ET AL.

Both JMF and MultiTEL are implemented in Java and define a set of classes for themanagement of multimedia devices independently of client platforms, and both provide areference architecture used in the development of multimedia services.

However, JMF only provides a framework to capture multimedia devices and transmitthe media data between the participants, while MultiTEL allows the definition of completeMSs, covering all the phases of the service, which includes not only device capture anddata transmission, but also the user connection, media negotiation, and service schedulingphases. In addition, MultiTEL offers a directory service with pre-developed componentsand applications that developers can use to build their own services.

Coming back to the TeleUni service referred in the introduction, using JMF facilities wecan only capture multimedia devices (e.g., camera, microphone, speaker, and display) forthe students and the teacher and manage the transmission of multimedia data. The rest ofthe functionality related to the development of a complete multimedia service must be im-plemented from scratch. The reason is that JMF does not provide a reference architecture toguide us in the implementation of, for instance, the participant connection phase, where thesystem should check whether a student is registered in a specific class, or the schedulephase, that establishes the type of interaction between the teacher and students. Regardingtypes of interaction, TeleUni states that when a student asks a question, all the participantsof the lecture will receive it, but only the teacher can answer it.

On the other hand, MultiTEL offers several implementation alternatives for each of thesephases, by simply replacing the corresponding component or connector. For instance, itdefines several access protocols, such as the manager access protocol in which a managerdecides whether a user can join the service, or the database access protocol, where thereis an access control database containing the users enrolled to a service. In TeleUni we mayapply the latter, using a database of registered students for access control.

However, JMF and MultiTEL also have some similarities. Both frameworks define a setof pseudo-standard interfaces that must be implemented by service components and bothmanage multimedia devices through a local resource manager. Nevertheless, JMF allowsthe capture of devices without the use of the resource manager, by knowing the deviceidentifier. But this solution is platform-dependent because device identifiers change fromone system to another. For instance, a camera identifier in Windows is named as vfw://0, inSolaris as sunvideo://0, and in a PCI-based Sun station as sunvideoplus://0. On the contrary,platform dependency is hidden in MultiTEL by the definition of a unique identifier for eachdevice (i.e., “camera”, microphone”, etc.) and by always capturing the devices through theresource manager. Table 2 sum up the most noteworthy similarities and differences betweenboth approaches.

3.1. Videoconference and video on demand MSs architecture

In the TeleUni service, a videoconference service is launched by the teacher and studentsto see and hear each other and a VoD service is used to transmit educational videos. Theservice architecture imposed by JMF is depicted in figure 5, where a RTP Session Managercomponent performs media transmission operations for each participant in the service. Thiscomponent is a local representation of the RTP session, it maintains information about the


Table 2. MultiTEL versus JMF.

1. Both provide a multimedia reference architecture, but JMF only provides the functionality for device captureand the data transmission phase, whereas MultiTEL make easier the definition of complete multimedia services(connection, negotiation, schedule, device capture, and transmission phases).

2. Both use a Local Resource Manager to capture multimedia devices, but JMF also allows the capture of devicesby providing the device identifier, avoiding the RM, but this solution is platform dependent.

3. MultiTEL provides an atomic method for capturing media devices, whereas the capture process of JMF ismore complex.

4. MultiTEL performs the capture of both producer and receiver devices before transmission begins, assuring thesuccess of end to end transmission, but JMF only performs the capture of the producer devices. The receiverdevices are captured by a different component, the JMF Player, that does it after the reception of data.

Figure 5. JMF architecture for multimedia services.

participants in the service, and controls the input and output data streams. Each participantreceives RTP data packets and RTCP control packets through this component.

The main steps to create an RTP session and initiate the transmission of RTP streamsare outlined in figures 5 and 6 depicts the main code referring to these steps. In the firststep tagged as (1), each participant joins the TeleUni service, which causes the capture ofhis/her multimedia producer devices and the creation of an RTP Session Manager instance.For each producer device (similarly to figure 4), you have to get a MediaLocator to indicatethe type and format of the data generated by the device, create a CaptureDevice using that

40 PINTO ET AL.

Figure 6. Main steps in the startup of an RTP session.

MediaLocator to get the data from the device, construct a Processor using the previouslycreated DataSource, set the output format of the Processor to an RTP-specific format andcreate the output DataSource from the Processor. In the second step, the RTP SessionMan-ager creates a stream called SendStream and connects it to the device output DataSource(labeled (2) in figure 5). In the reception site (step 3), the service must wait for an eventthat gives information about input data from a new participant. When this occurs the RTPSessionManager creates a stream called ReceiveStream and a Player component for datapresentation.

Coming back to the TeleUni service, when the teacher begins the lecture through thevideoconference, each instance of a student RTP Session Manager receives a notificationthat the teacher is transmitting RTP data and creates a ReceiveStream and a Player instanceto receive this data.


The software architecture for transporting multimedia data proposed by JMF is veryrestrictive in some aspects. Although, everybody agree that RTP is the appropriate protocolfor multimedia data transmission, it is not the only option to transfer multimedia data.Someone may like to use the IP multicast, the services provided by an ATM network, ora reservation protocol such as RSVP, among others. Regardless of that, JMF imposes theuse of RTP for videoconference or VoD services, therefore multimedia services that requirethe use of an alternative communication protocol cannot be programmed with JMF [3].In the current version [15], since JMF does not offer a common interface modeling theconnections between the participants of a service, the networking part of this architectureneither can be extended with a new communication protocol that may appear in the future,nor can designers choose among protocols that offer different QoS. Consequently, servicesthat require the use of, for instance, proprietary protocols or new ones, must wait until JMFprovides the components implementing these protocols or simply implement them fromscratch.

Now, we are going to describe devices capture and multimedia data transmission inMultiTEL. MultiTEL distinguishes between the device management modeled by the multi-media architecture and the connections management modeled by the network architecture.The main difference to JMF is that MultiTEL provides a frame architecture to any kind ofvideoconference or VoD service. As shown in figure 7, inside the multimedia architecture,the AllocCont connector initiates the multimedia devices (modeled by the MultimediaDe-vice component) capture by sending a request to the Resource Manager. Once the devicecomponents have been created, the service accesses them through the ContCom connectorwhich encapsulates the device behavior. Unlike JMF, where only the producer devices arecaptured and the data presentation is made through the Player component, in MultiTELboth the producer and the receiver devices are previously captured before the transmissionstarts. This approach is more extensible because it allows the RM to manage all types ofdevices in a uniform way. Thus, the addition of new devices, for example, a DVD recorder,can be made in a straightforward manner. In MultiTEL, consumer devices implement thesame interface, likewise producer ones, that is, there is no difference between writing in a

Figure 7. MultiTEL architecture for multimedia services.

42 PINTO ET AL.

DVD recorder or in a file. However in JMF, different types of receivers have to be treatedwith different components, decreasing drastically the reusability of service architecturesdue to the high dependency on the devices implementation components. For instance, JMFuses the Player component to write media data in a real-time device as a speaker, but usesanother kind of component, the DataSink, to write the same kind of data in a file or in aRTP session.

After the devices are captured, the components and connectors of the network architec-ture establish connections among them and start to control the data transmission (figure 7again). The NetworkSAP component stores useful information about service participants inthe HomeInfo structure. This information consists of the list of devices captured for partic-ipants, and the transport ports by device. The main contribution of MultiTEL architecture(which JMF does not have) is the definition of the VirtualConnection component. Thiscomponent is local to each participant and encapsulates all the connections established forthat user by using a particular communication protocol. Thus, in MultiTEL, changing thecommunication protocol for a service is achieved by simply providing a new implementa-tion for this component, that will be wired to the rest of components and connectors, whichremain unchanged, at runtime.

Our proposal to overcome JMF’s limitations regarding evolution of communication pro-tocols is the addition of a new component that models participants’ connection graph. Theresulting architecture is shown in figure 8. The new component NetworkComponent hasthe same functionality as the VirtualConnection component of MultiTEL, so the interfacedefinition is the same. Producer devices are modeled in the same way as previously, butthe data generated by each device is converted to a byte array with the getData() methodof the DataSource component. Regarding receiver devices, it is not enough to use a Player

Figure 8. Extension of JMF architecture to achieve communication protocol independence.


because this component does not allow a byte array as an input, so the solution goes throughthe implementation of a DataSource component, which receives the data from the networkthrough the NetworkComponent. Optionally, for changing the data format, one can usea Processor component that carries out any format conversion and sends the output datato a Player component through a new DataSource component. As it is shown, achievingcommunication protocol independence in JMF implies the use of so many components.

Moreover, as the NetworkComponent component is defined by a unique interface, JMFusers are able to use different communication protocols by only writing an implementationof this component using the desired protocol. Like MultiTEL, this extended architecture ofthe JMF framework can be applied to any kind of videoconference or VoD service that willbenefit from a more flexible and evolving architecture concerning communication protocols.

Another significant issue in multimedia programming is the data format negotiation. Itis not realistic to expect that all the participants who may join the service will transmit datain the same or compatible format. On the one hand, each participant in the service will havedevices from different manufacturers that may produce or consume distinct sets of dataformat. Thus, very frequently, data formats sent by various participants are incompatible.There are multiple criteria to approach format negotiation. First, we can take into accountuser profiles given in terms of the availability of plug-ins that can decode the received dataor the accessibility of encoders to change the data format to one accepted by decoders.For instance, a student of the TeleUni service may configure her/his environment to receivevideos in MPEG-4 format while other one only admits JPEG and MPEG-3 decodes. Thesame may occur in the transmission of real-time data. Another much more relevant criteriafor an efficient transmission of data is the quality of service (QoS) required by the service.For instance, data formats are characterized by resolution, CPU demand, and networkbandwidth. If users demand high-quality data, more network bandwidth is required, so theQoS (data format) should be adapted to the availability of the bandwidth. It is probable inTeleUni that night classes will offer a higher QoS (high resolution video, short responsetime, etc.), than those imparted in the morning during the rush hour.

In MultiTEL, the format negotiation is not a problem, because the software architecturepresented in figure 7 implements a negotiation protocol inside the AllocCont connector. Inaddition to the capture of devices task mentioned previously, this connector also negotiatesthe transmission format among participants. This negotiation can be as simple as capturingthe devices in the format indicated in the logical connection graph (LCG). Since the LCGis provided by the user that plays the role of service manager as a configuration parameter,there is no negotiation. However, the AllocCont connector can encapsulate a negotiationprotocol as complex as necessary, involving only the instance of every participant.

On the other hand, the JMF architecture does not address how the different participantsnegotiate data formats. Supposing that RTP is the communication protocol used in a service,every participant can know the transmission format of any other participant by inspecting thepayload type field of RTP data packets. In addition, a participant can change the data formatwhen some problems in the transmission are detected by analyzing the control informationof RTCP control packets. But how can participants negotiate the format? How do they knowwhether other participants will be able to reproduce the data they send? JMF says nothingin this regard, so the problem remains unresolved.

44 PINTO ET AL.

Our proposal to endow JMF with this capability is the use of the APP packet. The APPpacket is an RTCP control packet that allows applications to exchange their own messageswith a non-RTP format. Thus, before the transmission of RTP data packets, participantsshould use the APP packet to transport the messages of a format negotiation protocol. Theadvantage of this solution against other ones (such as establishing a different communicationpath between participants to carry out the negotiation) is the saving of network resources.

Sending an APP packet is performed by throwing an ApplicationEvent event with thereceivers having to listen for this event. The fields of this packet are the integer appSub-type, the Java string appString, and an array of bytes appData. The appSubtype can beused to set the unique identifier of a user protocol, for instance, format negotiation. TheappString contains the type of the message and the data contains message’s parameters.For instance, to start a format negotiation one of the participants sends the event Applica-tionEvent(100,“initiateFormatNegotiation”,null), where “100” is the unique identifier forall the messages involved in the format negotiation.

3.2. Reusing and extending developed services

Suppose that last year most students complained about the QoS of TeleUni; so the Universitydecides to modify the TeleUni service to add a more efficient data compression algorithm.In order to do this, we need to extend the TeleUni software architecture by adding theappropriate codecs. The extensibility and reusability of a multimedia architecture is a keyaspect in the development of web-based multimedia services. In this section, we presenthow MultiTEL and JMF software architectures can be extended to support new encodersor decoders, that may appear over time. Figure 9 shows how to model a device with andwithout a codec in both JMF and MultiTEL.

In MultiTEL, when a component (e.g., MultimediaDevice) sends an event to a connector(e.g., ContCom), the component does not know which connectors will receive the event.

Figure 9. Adding new devices to JMF and MultiTEL.


The USP object (see figure 2), contains architectural information about how componentsand connectors of a service are connected at runtime. The USP performs the dynamiccomposition of components and connectors by intercepting components’ output events andresending them to the appropriate connectors. We now address how to add a new codecto an existing service, where this codec is controlled by a connector named ContCodec.In terms of the MultiTEL model we need to put a connector and a component betweenthe MultimediaDevice component and ContCom connector. From the MultimediaDevicecomponent point of view, it is identical, but now its output events will be caught by theContCodec connector. This is feasible because this connector implements at least the sameinterface as the ContCom connector. Since the USP contains the list of components, con-nectors, and connection constraints, that is, information about the structure of the service,the designer only has to extend the USP class by adding the new component and connector,and setting the new connections among them according figure 9. We want to point out thatdue to the dynamic composition mechanism of MultiTEL it is not necessary to change theimplementation of any component or connector of the existing service.

On the other hand, making the same extension in JMF entails changing the service sourcecode. As depicted in figure 9, instead of obtaining the output data of the device from thefirst DataSource component, we have to insert a Processor component that makes thenew codification (e.g., MPEG-5). This Processor has associated a new, associated outputDataSource component that provides the output data in MPEG-5 format. Thus, before this,the extension client components had a reference to the first DataSource component (nowthis component is passed as an input parameter of the Processor component), but afterextension this reference must be replaced by the second DataSource component.

Another type of extension would be to convert the data codification, for instance fromJPEG to MPEG. Once more, in MultiTEL this change is easier than in JMF because inMultiTEL each component and connector is identified by its role in the service that isindependent of the implementation (components and connectors are created by the USPthat obtains the implementation class name knowing its role and then uses the reflexiveprogramming to create one instance). Modifications of a Codec component are again ac-complished by extending the USP class, but only the method that contains the line wherethe role Codec is associated with an implementation, putting the class name that imple-ments the new codec. However, in JMF the service developer must change the source codeof the Processor component to change the data to MPEG instead of JPEG, which is notdesirable.

3.3. Adding new multimedia devices

We now study the extensibility of multimedia services developed by both JMF and Multi-TEL, by analyzing the impact of adding a new multimedia device. Currently, both frame-works can make use of the latest desktop multimedia kits, such as cameras connected to avideo capture card or WebCam cameras connected to a USB port. Nevertheless, new mul-timedia devices are appearing every day, such as network cameras, the most recent digitalcamera technology. A network camera has an IP address for connection to the Internet, andincorporates a web server, so it does not need a computer to capture and transmit video

46 PINTO ET AL.

frames. This sort of camera offers an easy way to capture and transmit real-time videoframes through the Internet using a standard navigator [1], so the capture of video framesis very different from any other kind of existing camera.

Regarding the TeleUni service case study, suppose the University wants to take advantageof this new camera technology, so a network camera is installed in a conference room. Thefollowing week an important conference is going to take place at the University and somestudents who are enrolled in the TeleUni classes are interested in attending as part of theirdistance classes. However, the TeleUni service is not prepared to deal with this new kind ofcamera, so it is necessary to add this camera as a new device. First, we are going to addresshow this new device can be modeled both in MultiTEL and JMF and then we will discussdifferent applications of a network camera.

In both frameworks this new device has to be registered in the resource manager byproviding an implementation class that encapsulates the capture of video frames from thisnew camera. The camera captures video frames and stores the successive frames by rewritingthe same JPEG file. Thus, the network camera component must read this JPEG file throughthe network (through an HTTP connection as shown in figure 10).

In MultiTEL the addition of the new device implies the implementation of a Virtual-NetworkCamera component that implements the ProducerNode interface, where the read()method is in charge of reading the video frames from an HTTP connection (figure 10). Fromthe architectural viewpoint a network camera component is only a different implementationof a camera, so the unique identifier of this new component is “camera” too (it plays thecamera role inside the service architecture). Thus, the rest of the multimedia architecture(e.g., the ContCom connector) will continue to catch events from or send messages to acomponent that plays the “camera” role.

On the other hand, in JMF, we can appreciate that the process of implementing a newdevice is more complicated and tedious. First of all, this new device must be registered by adifferent identifier (e.g., http://host1/camera) so modifications in the client components ofthis new device have to be done. Figure 10 shows that it is necessary to develop two newclasses. The first one is the NetworkCamera that implements the CaptureDevice interface

Figure 10. New device in JMF and MultiTEL.


and extends the abstract class DataSource. In addition, this class contains a reference toa SourceStream object. This object will return the data after the invocation of the methodread(), so the developer needs to extend this class and redefine this method with the accesscode to the camera web server. The data retrieved from the camera web server is codedsimilarly in both frameworks. For each frame, an HTTP connection is opened and the frameis downloaded from the web server in JPEG format.

Once we have defined and implemented a component modeling this new device, how canwe incorporate and use it in a multimedia service? The easiest way to do it is to use thiscamera as if it were a local camera of a virtual computer. For instance, it can be used it inthis way by a user who does not have a camera at a computer, or by a conference attendeesituated in a conference room with a hidden computer that transmits speech, not only to thelocal audience, but to remote participants. In this case, the software architecture includingthis camera remains the same and the developer does not have to make any changes in theservice. The reason is that both cameras have the same interface and the source of the data,obtained locally or through the network, is transparent to the service.

However, these sorts of cameras offer far more possibilities. For instance, one can haveone of these cameras installed in the outside of a building, e.g., controlling a cash dispenser,or in an exam room keeping an eye on students, . . . .

To overcome the problem of simultaneous HTTP connections on the camera web server,we use a network camera server (figure 11) that includes only the multimedia and networkarchitectures of MultiTEL. This server or camera mirror captures video frames from thenetwork camera with the VirtualNetworkCamera component. Participants who want toreceive frames from this camera will join the server (in the same way as in a VoD service),and the VirtualConnection component of the network camera server will multicast the videodata among all the connected users.

To sum up, as fast as new technologies arise a multimedia framework can support theeasy integration of new devices inside legacy multimedia services. In this way, those devicesthat have emerged more recently, such as a network camera or a DVD, can be included aspart of pre-developed services.

Figure 11. Using network camera with a server.

48 PINTO ET AL.

4. The future of JMF

Sun is continuously providing new versions of JMF that solve some bugs of previousreleases. The latest release is JMF 2.1.1a. But unfortunately the complexity of multimediaservices development remains the same, because the JMF API does not change in this newrelease. We receive JMF users feedback through the mail list [email protected] and the amount of questions sent to this distribution list confirms our assertion aboutJMF’s complexity. Most users are having really serious problems in developing very simplemultimedia services, like audio capture or compressed audio presentation. This confirmsthe fact that although JMF was considered suitable for new multimedia programmers, itconfuses more than it clarifies multimedia issues. Thus, we think that the proposals in thispaper will be particularly useful for them.

On the other hand, instead of continuing to improve and update the API, Sun decidedto make the JMF source code available for developers, but only for research and non-commercial use. Surely it will be very hard to understand the internal behavior of JMFcomponents, mainly by code inspection, but finally researchers will have the opportunity tomodify and extend this framework by applying solutions like ours. Thus, we encourage themultimedia programming community to propose new extensions and modifications to JMF,as we have done, in order to obtain a standard, but more tractable framework for non-expertprogrammers.

5. Conclusions

We have shown that it is possible to successfully apply the component-based software engi-neering and framework technology in the development of Web-based multimedia services,which improves their design. Multimedia programming includes new issues, such as re-source reservation, format negotiation or real-time restrictions, that make the developmentof these services from scratch quite difficult. The compositional framework technology isa good approach to abstract all these aspects allowing the construction of open, reusable,and extensible services.

The main goal of this work has been the comparison of JMF and MultiTEL, two success-ful multimedia frameworks, mainly from the architectural evolution viewpoint. We havefocused on how easy to use, extensible, and adaptable both approaches are.

We have shown that:

• MultiTEL offers a broad service architecture; thus, the development of a complete multi-media service can be achieved with less effort than in JMF, because JMF only provides areference architecture for the capture, transmission, and presentation of multimedia data.Therefore the developers must implement the rest of the service from scratch

• the learning curve of JMF is higher than that of MultiTEL, due to the complex relationshipsbetween the components imposed by the JMF architecture

• videoconference and VoD services in JMF impose the use of the RTP communicationprotocol, whereas MultiTEL services are independent of communication protocols

• it is more difficult to extend a service in JMF that in MultiTEL


• to add new behaviour (e.g., incorporate a new multimedia device) in JMF it is morecomplicated than in MultiTEL

We have proposed solutions to some of the limitations of JMF. One of the extensionsproposed for JMF is a new component interface that models a set of connections establishedby any communication protocol (similar to the VirtualConnection component of MultiTEL).In addition, we have suggested enlarging JMF with a format negotiation phase whose goalis to find a data format according to users’ preferences and QoS constraints.

However, we are conscious that another significant criterion regarding comparing thetwo frameworks is efficiency. Although both are implemented in Java, which is a synonymfor inefficiency, JMF offers a more efficient implementation because it directly accessesdevice drivers, whereas MultiTEL implements the multimedia device components throughJava native methods.

As a result of this comparison, extensions which solve some of JMFs limitations canbe defined in order to obtain a more standard framework than MultiTEL. The resultingframework would provide a reference architecture to develop MSs over the Web using JMF.

After developing several VoD and videoconference services (such as TeleUni), we are nowextending MultiTEL to address various trends. First, we have been adding new componentsand connectors to support the sharing of GUIs, to be able to develop any kind of collaborativeapplication. For instance, we have built some collaborative games that take advantage ofthis feature, such as PictionaryTM and Tic Tac Toe. On the other hand, in order to promotethe use of MultiTEL in the international community we have specified a UML profile forMultiTEL [7], that will help to understand the model better. Since the separation of concernsis a good approach for making distributed software architectures more open to architecturalchanges, we are extending the component-connector model of MultiTEL to support theseparation of any kind of issue and not only computation and coordination [11].

Acknowledgments

We would like to thank the anonymous referees for they insightful comments and sugges-tions, that greatly helped us improving the quality and presentation of the paper. This workhas been partially funded by CICYT project TIC99-1083-C02-01, and the telecommunica-tion organization “Fundacion Retevision”.

Notes

1. The Web page of MultiTEL can be found at http://www.lcc.uma.es/∼lff/MultiTEL.2. A component platform defines a mechanism for locating and composing components at runtime, so that the con-

stituent components do not have direct references inside. Well known examples are EJB/Sun,.NET/Microsoftor CCM/OMG.

References

1. Axis Network Camera, http://www.es.axis.com/products/camera servers/index.html.2. W.A. Brown and C.W. Kurt, “The current state of CBSE,” IEEE Software, Sept./Oct. 1998.3. L. deCarmo, Core Java Media Framework API, Prentice Hall, 1999.

50 PINTO ET AL.

4. M.E. Fayad and D.C. Schmidt, “Object-oriented application frameworks,” Communications of the ACM,Vol. 40, No. 10, 1997.

5. L. Fuentes and J.M. Troya, “A component-oriented architecture to design multimedia services on a distributedplatform,” in Proc. of WorldWide Computing and its Applications. WWCA’97, held on March 1997 inTsukuba, Japan, Springer LNCS No. 1274, Aug. 1997.

6. L. Fuentes and J.M. Troya, “A Java framework for web-based multimedia and collaborative applications,”IEEE Internet Computing, Vol. 3, No. 2, 1999.

7. L. Fuentes, J.M. Troya, and A. Vallecillo, “Building a UML profile for MultiTEL,” next publication in theAnnals of Software Engineering in the special issue Object-Oriented Web-based Software Engineering, 2001.

8. E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design Patterns: Elements of Reusable Object-OrientedSoftware, Addison-Wesley, 1995.

9. D. Krieger, “The emergence of distributed component platforms,” IEEE Computer, March 1998.10. Microsoft Corporation, “An introduction to Windows Media Technology,” http://msdn.microsoft.com/

workshop/imedia/windowsmedia/IntroToWMT.asp.11. M. Pinto, M. Amor, L. Fuentes, and J.M. Troya, “Collaborative virtual environment development: An aspect-

oriented approach,” in Proceedings of Distributed Dynamic Multiservice Architecture (DDMA’01), 21stInternational Conference on Distributed Computing Systems Workshops, IEEE Computer Society, April2001, pp. 97–102.

12. M. Pinto et al., “Run-time coordination of components: Design patterns vs. Component & aspect basedplatforms,” Workshop ASoC (Advanced Separation of Concerns), asociado al ECCOP’01, Junio 2001.

13. H. Schulzrinne, “RTP. A transport protocol for real-time applications,” RFC 1889, Jan. 1996.14. Silicon Graphics, Digital Media Programming Guide, IRIS Insight Library.15. Sun Microsystems, Inc., Java Media Framework Guide, 2001. http://java.sun.com/products/java-media/jmf/

2.0/jmf20-fcs-guide/JMFTOC.html.

Monica Pinto is an Assistant Professor since 2000 at the University of Malaga (Spain), where she receivedthe M.Sc. degree in Computer Science in 1998. Her research interests deal with Component-Based SoftwareDevelopment, Aspect-Oriented Software Development, application frameworks, and multimedia applications.She has participated as a speaker in international conferences and is involved in several research projects on thesesubjects. Actually, she is concluding her Ph.D. thesis, which is focused on combining components and aspects.

Mercedes Amor received her M.Sc. degree in Computer Science from the University of Malaga (Spain) in 1998,where she is an Assistant Professor at the Department of Computer Science since 2000. Her Ph.D. thesis focus on


Software Agents modeling. Also, her research interests deal with componentware, frameworks, aspect orientation,collaborative virtual environments and multimedia applications. She has participated as a speaker in internationalconferences and currently she is involved in research projects on these subjects.

Lidia Fuentes received her M.Sc. degree in Computer Science from the University of Málaga (Spain) in 1992and the Doctor degree in 1998 from the same University. She is an Associate Professor at the Department ofComputer Science of the University of Malaga since 1993. Her research interests deal with Component-BasedSoftware Development, frameworks, aspect orientation, software agents and multimedia applications. In addition,she has participated in several international conferences as a speaker, reviewer, and chairman. Her most significantpublications can be found in international journals from ACM and IEEE, such as IEEE Internet Computing, IEEETransactions on Software Engineering, ACM Computing Surveys or Software Practice and Experience. She hasactively participated in several research projects and also she has done consulting activities in telecommunicationcompanies such as Telefonica and Retevision (Spain).

Jose M. Troya received the M.Sc. (1975) and Ph.D. (1980) degrees in physics from the Madrid ComplutenseUniversity. From 1980 to 1988, he was an Associate Professor at that university, and, in 1988, became a fullprofessor at the University of Malaga, where he has lead the Software Engineering Group. His research interestsinclude parallel programming, distributed systems, and software architectures. He is very involved in severalnational and international research projects, has written articles for the most relevant computer conferences andjournals, supervised numerous Ph.D. thesis, and organized several workshops and international conferences.

analyzing architectural evolution issues of multimedia ... · systems that are reusable,...

Documents