Windows NT Systems Programming: Spring 1999

[ Home | Syllabus | Course Notes | Assignments | Search]

Intro to MultiMedia

What is Multimedia?

Audio
- Discrete files (.wav, . midi)
- Continuous Streams (internet telephone)
Pictures
- BitMaps
- GIF
- JPEG
Video
- Discrete files
- Continuous streams (tele video)
Combined A/V (.avi)

Analog/Digitial Conversion Drivers

Literally thousands of different devices for capturing, displaying, playing, recording. compressing, sending, receiving media files/streams.
Many standards for formatting
Need to consolidate into an overall architecture for handling various media.
DIB (Device Independent Bitmap) frees programmer from the differences in display driver/ monitor technology.
RIFF file format is self documenting sound files
AVI file format is self document synchronized audio and video files.
Program negotiates with system to find appropriate drivers, conversion routines, formats, etc.

Looking at Audio

High, intemediate and low level audio handling.
We saw a high level access in the first windows programs shown (hello using win32 calls)
PlaySound ("hellowin.wav", NULL, SND_FILENAME | SND_ASYNC) ;
Media Control Interface (MCI)
High Level access works because the media files are self documenting as to format
this documentation is used to negotiate with system to find appropriate drivers.
A/D and D/A conversions
- Sound waves can be characterized by several parameters
  - Frequency
  - Amplitude
- Incoming Sound Waves are usually first converted into voltage levels by a microphone
- Outgoing Sound Waves are usually generated from voltage levels driving a speaker
- These voltage levels can be digitized by A/D and D/A converters.
- Accuracy of sound reproduction depends on voltage/sound wave fidelity and the A/D/A conversions
  - Frequency = sample rate
  - Amplitude = bits per sample
- Human ear: frequencies 200 - 20,000 cycles/second, more than 256 levels
- nyquist sample rate is twice the highest frequency: so CD quality is 44.1 KHz
- Telephone is approx up to 4000 Hz. = 8,000 samples per second
- 8 bits/sample is low, 16 is much better = 64kBPS to 128kBPS for talking audio quality
- Most common sampling technique is Pulse Code Modulation (PCM). (uncompressed format)
- Most Audio drivers support 8 and/or 16 bit PCM
- Can be more clever about sampling if you use differential methods (only send the difference between successive samples.) or information theoretic sampling techniques (entropy).
  - MuLaw (high quality 8 bit standard)
  - Adaptive Differential PCM (4 bit, 8bit?)

Low Level Audio APIs

Skip the multimedia file I/O functions (mmio)
Store and Retrieve raw audio data (NOT self documenting)
Data Structures:
- WAVE buffer (actual audio data in some format known to the audio device driver)
- WAVE Headers (describes individual WAVE buffer)
- FORMAT record (describes format of audio recording)

Wave Header

typedef struct { 
    LPSTR  lpData; 
    DWORD  dwBufferLength; 
    DWORD  dwBytesRecorded; 
    DWORD  dwUser; 
    DWORD  dwFlags; 
    DWORD  dwLoops; 
    struct wavehdr_tag * lpNext; 
    DWORD  reserved; 
} WAVEHDR;

lpData: pointer to Wave buffer
Length: of Wave buffer
Recorded: number of bytes recorded on input
User: for use for the user
Flags:

Flags supplying information about the buffer. The following values are defined:

WHDR_BEGINLOOP

This buffer is the first buffer in a loop. This flag is used only with output buffers.

WHDR_DONE

Set by the device driver to indicate that it is finished with the buffer and is returning it to the application.

WHDR_ENDLOOP

This buffer is the last buffer in a loop. This flag is used only with output buffers.

WHDR_INQUEUE

Set by Windows to indicate that the buffer is queued for playback.

WHDR_PREPARED

Set by Windows to indicate that the buffer has been prepared with the waveInPrepareHeader or waveOutPrepareHeader function.

Loops: number of times to play the loop
wavehdr_tag: reserved for multi-segment audio buffers

WAVEFORMATEX

One per recording

typedef struct { 
    WORD  wFormatTag; 
    WORD  nChannels; 
    DWORD nSamplesPerSec; 
    DWORD nAvgBytesPerSec; 
    WORD  nBlockAlign; 
    WORD  wBitsPerSample; 
    WORD  cbSize; 
} WAVEFORMATEX;

wFormatTag: Format of the recording (see mmreg.h for supported tags) - e.g. PCM
nChannels: 1 for mono, 2 for stereo
nSamplesPerSec and nAvgBytesPerSec: may be variable for compressed formats, fixed for PCM.
FOr PCM nAvgBytesPerSec = nSamplesPerSec * nBlockAlign
nBlockAlign: minimum atomic data unit in bytes.
For PCM, nBlockAlign = (nChannels*wBitsPerSample)/8
wBitsPerSample: number of bits per sample.
For PCM, either 8 or 16.
cbSize: size of extra format information appended to end of this structure and used by nonPCM format for parameters of the encoding.

Examples:

Telephone quality, mono, 8 bit PCM

WAVEFORMATEX PCMWaveFmtRecord;
	
	PCMWaveFmtRecord.wFormatTag = WAVE_FORMAT_PCM;
	PCMWaveFmtRecord.nChannels = 1;
	PCMWaveFmtRecord.nSamplesPerSec = 8000;
	PCMWaveFmtRecord.wBitsPerSample = 8;
	PCMWaveFmtRecord.nAvgBytesPerSec = PCMWaveFmtRecord.nSamplesPerSec
		* PCMWaveFmtRecord.wBitsPerSample / 8;
	PCMWaveFmtRecord.nBlockAlign = PCMWaveFmtRecord.nChannels *
		PCMWaveFmtRecord.wBitsPerSample / 8;

Audio API functions

First Examine a simple playback program.

bool playBack(char* buffer,int numrcv)
{
	//Handle for the WAVE device.
	HWAVEOUT hWaveOut;
	WAVEFORMATEX PCMWaveFmtRecord;
	int errorCode;
	
	PCMWaveFmtRecord.wFormatTag = WAVE_FORMAT_PCM;
	PCMWaveFmtRecord.nChannels = 1;
	PCMWaveFmtRecord.nSamplesPerSec = 8000;
	PCMWaveFmtRecord.nAvgBytesPerSec = 8000;
	PCMWaveFmtRecord.nBlockAlign = 1;
	PCMWaveFmtRecord.wBitsPerSample = 8;
	errorCode = waveOutOpen(NULL, WAVE_MAPPER,
							 &PCMWaveFmtRecord, 
							 0L, 0L, WAVE_FORMAT_QUERY);
	if (errorCode)
	{
		//MessageBox(" Device Cannot handle this format");
		return false;
	}
	
	waveOutOpen(&hWaveOut, WAVE_MAPPER,
		&PCMWaveFmtRecord,
		0L, 0L, 0L);
	
	//Prepare the header.
	WaveHeader.lpData = (LPSTR) buffer;
	WaveHeader.dwBufferLength = numrcv;
	WaveHeader.dwFlags = WaveHeader.reserved = 0;
	WaveHeader.dwLoops = 0;
	WaveHeader.lpNext = NULL;
	
	waveOutPrepareHeader(hWaveOut, &WaveHeader, sizeof(WaveHeader));
	
	//Write data to WAVE device.
	waveOutWrite(hWaveOut, &WaveHeader, sizeof(WaveHeader));
	
	//Loop until playback is finished.
	do {}
	while (!(WaveHeader.dwFlags & WHDR_DONE)); // inefficient busy wait!
	
	//Unprepare the wave output header.
	waveOutUnprepareHeader(hWaveOut, &WaveHeader, sizeof(WaveHeader));
	
	//Close the WAVE device.
	waveOutClose(hWaveOut);
	
	return true;	
}

F1: waveOutOpen is used both to query a device driver to see if it supports the requested format and to open the device driver.
In this call, we query the driver (hence the first parameter which is the handle returned is NULL), WAVE_MAPPER tells the system to find any driver which can support the format specified in the PCMWaveFmtRecord. Returns an error is no appropriate drivers found. Could retry with a different format.
F2: Before writing a buffer, you must first prepare the header.
F3: Wiat till device is done playing buffer.
F4: After device is done, clean up the buffer.

Call Backs

The above program was simplistic in that only one buffer was played back
And it is inefficient in the busy wait.
Here is a recording program which records 500 byte buffers at a time
Has the audio device call back when the buffer is ready
Stores the buffer in a file using a separate thread
Coordinates the call back and thread using an event.
Many of the function calls for input (recording) are analogous to the output (playback) ones.

Setup Function

HANDLE recordDone;		// event of audio input completion 
HWAVEIN hWaveIn;		// onput WAVE audio device handle 
char inbuf[BUFSIZE];	// WAVE input data buffer area 
HANDLE audioFileHandle;		// handle for audio file

WAVEHDR WaveHeader;	
// . . .

void CMessage::RecordMessage()
// Records Message to file
// for testing name is hardcoded in.
{
	
	DWORD threadid;
	HANDLE RecordHandle;
	
	audioFileHandle = CreateFile("TestRecord", GENERIC_WRITE,
		0,0, CREATE_ALWAYS, 0, 0);
	
	recordDone = CreateEvent(0, FALSE, FALSE, 0);
	ResetEvent(recordDone);
	RecordHandle = CreateThread(NULL, 0,
		(LPTHREAD_START_ROUTINE) Recorder, NULL, 0,
		&threadid);
	openWaveDev();
	msgApp.m_pMainWnd->SetWindowText("Start Recording"); 
	WaitForSingleObject(RecordHandle, INFINITE);
	waveInClose(hWaveIn);
	msgApp.m_pMainWnd->SetWindowText("Done Recording"); 
	CloseHandle(audioFileHandle);
    return;
}

F5: Used to synchronize callback and thread

openWaveDev

void openWaveDev()
{
	MMRESULT Rc;
	WAVEFORMATEX PCMWaveFmtRecord;
	
	PCMWaveFmtRecord.wFormatTag = WAVE_FORMAT_PCM;
	PCMWaveFmtRecord.nChannels = 1;
	PCMWaveFmtRecord.nSamplesPerSec = 8000;
	PCMWaveFmtRecord.wBitsPerSample = 8;
	PCMWaveFmtRecord.nAvgBytesPerSec = PCMWaveFmtRecord.nSamplesPerSec
		* PCMWaveFmtRecord.wBitsPerSample / 8;
	PCMWaveFmtRecord.nBlockAlign = PCMWaveFmtRecord.nChannels *
		PCMWaveFmtRecord.wBitsPerSample / 8;
	
	Rc = waveInOpen(&hWaveIn, WAVE_MAPPER, &PCMWaveFmtRecord,
		(DWORD) recordCallBack, 0L,
		CALLBACK_FUNCTION);
	if (Rc)
	{ // must not be an audio recording device
		return;
	}
	WaveHeader.dwBufferLength = BUFSIZE;
	WaveHeader.lpData = (LPSTR) (inbuf);
	WaveHeader.dwFlags = WaveHeader.reserved = 0;
	WaveHeader.dwLoops = 0;
	WaveHeader.lpNext = 0;
	Rc = waveInPrepareHeader(hWaveIn, &WaveHeader,
		sizeof(WAVEHDR));
	
	Rc = waveInAddBuffer(hWaveIn, &WaveHeader,
			     sizeof(WAVEHDR));
		
	ResetEvent(recordDone);
	
	if (Rc = waveInStart(hWaveIn))
	{
		waveInClose(hWaveIn);
	}
}

F6: opens audio device for recording, specifies "recordCallBack" as callback procedure
NOTE: dispense with query since we assume that it can handle 8 bit PCM (of course, there may be no audio device on the machine at all)
F7: Sends buffer to the recording device to be filled.
F8: Starts the input device

CallBack Function

void CALLBACK recordCallBack(HWAVEIN hWaveIn, UINT uMsg,
						 DWORD p1, DWORD p2, DWORD p3)
{
	if (uMsg != WIM_DATA)
		return;
	SetEvent(recordDone);
}

F9: either open/close or data ready
NOTE: do not call any wave functions in the callback for fear of going into a loop

Thread Functions

UINT Recorder(void *p)
{
	int i; // number of BUFSIZ buffers to record before reaching MAXBUF
	BOOL success;
	DWORD numWrite;
	
	for(i = 0; i < MAXBUF/BUFSIZE; i++) // just record a little bit
	{
		WaitForSingleObject(recordDone, INFINITE); // wait till device has called back
		if (!(WaveHeader.dwFlags & WHDR_DONE))  // just make sure really done
		{
			continue;
		}
		
		if(success = WriteFile(audioFileHandle, WaveHeader.lpData,
			WaveHeader.dwBytesRecorded, &numWrite, 0))
		{
		}
		if (waveInUnprepareHeader(hWaveIn, &WaveHeader,
			sizeof(WAVEHDR)))
		{
		}
		WaveHeader.dwFlags = 0;
		
		if (waveInPrepareHeader(hWaveIn, &WaveHeader,
			sizeof(WAVEHDR)))
		{
		}
		if (waveInAddBuffer(hWaveIn, &WaveHeader,
			sizeof(WAVEHDR)))
		{
		}
		
	}
	waveInReset(hWaveIn); 
	waveInClose(hWaveIn);
	return 0;
}

F10: For testing record a fixed number of buffers
F11: Wait for callback to signal
F12: clean up previous buffer
F13: Stops device

AudioDemo Program

Main Window is a dialog box.

// main.h

#include <afxwin.h>
#include "Message.h"


// Define an application object
class CMsgApp : public CWinApp
{
public:
	virtual BOOL InitInstance(); // everybody overrides this one
	CMessage msg;
};

Main.cpp

#include "main.h"
#include "CMsgDlg.h"
#include "resource.h"
// Create an instance of the application

CMsgApp msgApp;  

// Init the application and the main window
BOOL CMsgApp::InitInstance()
{
	CMsgDlg dlg;
	m_pMainWnd = &dlg;
	dlg.DoModal();
	
	// Since the dialog has been closed, return FALSE so that we exit the
	//  application, rather than start the application's message pump.
	return FALSE;

}

(source code here)