IRI has several components that need to be recorded and played back. In this project we are only concerned with the Audio and Video components. The new IRI uses JMF for capturing and playing Audio and Video.

Each user can speak/stop speaking at anytime, while IRI imposing an upper limit on the number of users who may be speaking at the same time, this limit is too large such that for all practical purposes no user is blocked from speaking whenever he wishes. If necessary some users are preempted to make room for other new users who like to speak, but this rarely happens. During the lifetime of an IRI session, each user may speak (create a new JMF audio capturing thread and thus causes the creating of a new JMF player) several times. The following are two possible schemes for recording and playing-back IRI audio.
 
 

Scheme 1:Preserve the individuality of each speaker.

We may identify the streams belonging to each speaker (from RTP/RTCP) and record the audio of each speaker in one file indexed by another file that records the timestamps of the start/stop. Let I.x and I.a be the index file and the audio file of user I, 1< = I <=N, where N is the number of distinct users who ever spoke during an IRI recording session.

To play the recorded audio, we may create at most N players: Player (I), 1<=I<=N. Player(I) plays the audio stored in file I.a and is driven by the events stored in file I.x. Let T0 and Tf be the timestamp of the first and the last entry in the index file and Tf . To reduce the number of concurrent players, each player is created just prior to time t0 and is deleted immediately after time Tf. The number of concurrent players could be further minimized by deleting/recreating a player during long periods of inactivates (silence).

This scheme preserves the individuality of each speaker and gives us the complete freedom to play and mix the audio signals of any subset of the speakers at the play-out time.

 Next figure shows an example of the files associated with two users: U1 and U2.

Scheme 2: Mix before you save.

In this scheme, we save the mixed audio output of all the JMF players as if the user has a cassette recorder next to the workstation without identifying any individual speaker in the mixed signal. In such scheme we have two files: the audio file and the index file contains the start/stop timestamps of the mixed audio signal in the audio file.

To play-out the recorded audio, a JMF player is created to play the content of the audio file and the index file is used to control the play-out periods (stop playing during the silence period in order to synchronize the audio with video play-out). If no synchronization is needed (e.g., the user does not play any other media like video or slide presentation), then the JMF player can play the audio file without using the index file which may speedup the play-out period of the recorded audio file.

Scheme 3: Combine both Schemes 1+2

We may combine both scheme 1 and 2 and save both the individual and the mixed audio of session users. In this case we can choose between very fine-grain play-out of each individual user (using scheme 1) and coarse-grain play-out (using scheme 2)  of the mixed signal with silence periods suppression.

In IRI there is a limited number N of Video windows that must be shared among all session participants. For example, one window my used for the teacher (or the presenter which can be any student), another for a site video (an overall video of a classroom taken by a wall-mounted camera) which can be shared among several sites and one or two windows to be shared among all other users for short periods (e.g. to ask or answer a question or to make a comment).
 

Scheme 1: Preserve the details of each video sender.

Similar to the audio, we can save the video of each sender source in a single file storing the video media and use an index file to record the timing events (start/stop) and media properties (e.g., frame rate).

To playback the recorded video, we may create up to N concurrent players driven by the index files. We can also play only the video of any particular individual (e.g., the teacher) or any subset of stored videos. It is very interesting to play the audio file of the associated individual, which may or may not make sense based on the context. For example, we may play the audio and video of the teacher, but we may hear the teacher answer a question asked by a student, but we don’t know what the question was since we did not hear the student who asked the question.

 Next figure shows an example of the files associated with two users: U1 and U2.

Scheme 2: N-files

Since we may only have at most N windows, we may use N files and save all video streams in only N files. In addition, we may have N index files that records the timing events, the media properties and the source identities of the senders. According to this scheme, a given sender may have his videos stored in one or any subset of the N files.
 

Project Teams