motorola canopy voip whitepaper v5-8-5-04 1.7kern.com › motorola ›...

Motorola Canopy

Voice over IP over Canopy September 27, 2004

Motorola Voice over IP over Canopy

��

� ��......................................................................................................................... 2

�� ....................................................................................................................... 4

�� ................................................................................................................... 6 CIRCUIT SWITCHED VS. PACKET SWITCHED TELEPHONY ........................................................................... 6

Circuit Switched: .................................................................................................................................... 6 Packet-Switched Telephony:................................................................................................................... 6

CODER/DECODER (CODEC) ........................................................................................................................ 6 INTEGRATION OF VOIP WITH THE PUBLIC SWITCHED TELEPHONE NETWORK ............................................ 7 THE ANATOMY OF A VOIP CALL................................................................................................................ 7

Encoding and Packetization ................................................................................................................... 7 Transport ................................................................................................................................................ 8 Dejittering and Decoding ....................................................................................................................... 8 Mouth-to-ear delay and overall distortion ............................................................................................. 8

STANDARDS FOR MEASURING CALL QUALITY ........................................................................................... 9 CONSIDERATIONS FOR NETWORK CAPACITY PLANNING ............................................................................ 9 ERLANG TABLES ...................................................................................................................................... 10

Erlang to VoIP Bandwidth Calculation................................................................................................ 11

�� ..................................................................... 12 CANOPY FRAME PACKETS ........................................................................................................................ 12

Control slots ......................................................................................................................................... 12 Downlink and Uplink Acknowledgement slots...................................................................................... 12

HIGH PRIORITY / QUALITY OF SERVICE (QOS) ......................................................................................... 12

�� ...................................................................................... 13 ADVANCED CALL QUALITY MEASUREMENTS ............................................................................................ 13 TESTS VOIP-ENABLED NETWORK EQUIPMENT.......................................................................................... 13 EMULATES COMPLEX NETWORKS IN TEST LAB.......................................................................................... 13 OPTIMIZES NETWORK DESIGN ................................................................................................................... 13

�� !�� .................. 14 CANOPY CONFIGURATION ........................................................................................................................ 14 VONAGE CONFIGURATION........................................................................................................................ 14 TEST SCENARIOS ...................................................................................................................................... 14 TEST RESULTS .......................................................................................................................................... 15 FINDINGS .................................................................................................................................................. 15 RECOMMENDATIONS ................................................................................................................................ 15

�� ......................................... 16 CANOPY CONFIGURATION ........................................................................................................................ 16 IXIA CHARIOT CONFIGURATION .............................................................................................................. 16 TEST SCENARIOS ...................................................................................................................................... 17 TEST RESULTS .......................................................................................................................................... 17 FINDINGS .................................................................................................................................................. 17

Summary of Results .............................................................................................................................. 17 Quality of Service ................................................................................................................................. 18 Theoretical Modeling based on Actual Results .................................................................................... 19

RECOMMENDATIONS ................................................................................................................................ 21


��!�� .......................................................................................................................................... 23 PHASE II TEST RESULTS ........................................................................................................................... 23

Canopy set at 50% downlink / 50% uplink – G.711a ........................................................................... 23 Canopy set at 50% downlink / 50% uplink – G.711u ........................................................................... 23 Canopy set at 50% downlink / 50% uplink – G.726 ............................................................................. 24 Canopy set at 50% downlink / 50% uplink – G.729 ............................................................................. 24 Canopy set at 50% downlink / 50% uplink – G.723.1-ACELP............................................................. 24 Canopy set at 50% downlink / 50% uplink – G.723.1-MPMLQ........................................................... 25 Canopy set at 75% downlink / 25% uplink – G.711a ........................................................................... 25 Canopy set at 75% downlink / 25% uplink – G.711u ........................................................................... 26 Canopy set at 75% downlink / 25% uplink – G.726 ............................................................................. 26 Canopy set at 75% downlink / 25% uplink – G.729 ............................................................................. 27 Canopy set at 75% downlink / 25% uplink – G.723.1-ACELP and G.723.1-MPMLQ......................... 27

TEST RESULTS WITH CANOPY QOS .......................................................................................................... 28 High Priority = 1 Slot........................................................................................................................... 28 High Priority = 2 Slots ......................................................................................................................... 28 High Priority = 3 Slots ......................................................................................................................... 29

�� ............................................................................................................ 31

��......................................................................................................................................... 32


4

�"�#$�%&��$''�()�� The Motorola Canopy solution is an excellent wireless broadband data transporter between a central site access point (AP) and many end-point subscriber modules (SM). The solution is being well received world-wide by telecom providers, wireless internet service providers (WISPs), and enterprises where site-to-site wireless data connectivity is preferred over traditional wired solutions. It has also been observed that the Canopy product can be an effective transport medium for real-time Voice over Internet Protocol (VoIP) technology. VoIP over Canopy is a reality for early adopters because of the low latency, high bandwidth, reliability, and scalability of Canopy. Based on this information, Motorola has been receiving an increased interest in using Canopy as more than a general data mover for such applications as real-time VoIP solutions. Therefore, Motorola has retained an external third party consulting and testing organization, West Monroe Partners, to validate the potential for VoIP over Canopy as well as to conduct testing to benchmark performance and scalability. To begin, the team focused on modeling and testing VoIP calls to confirm and benchmark the usability and performance over Canopy. A lab was built to simulate a test environment where a number of elements were varied to determine optimal configuration and performance limitations for potential Motorola customers of the Canopy product. The testing was conducted in two phases. The goal of the initial phase of VoIP testing over Canopy was to develop a theoretical model using the Vonage broadband product to simulate an actual VoIP phone call and document the results and performance. Vonage broadband phone service was used to test an actual VoIP call going through Canopy out through the Internet. The call was monitored to gather statistics on bandwidth usage so a theoretical model could be created to determine how many VoIP calls a Canopy AP can support. The Canopy modules were setup in a point to multipoint network configuration with one Access Point (AP) and two Subscriber Modules (SM). During the second phase the objective was to test scalability with more than just one VoIP call and determine where the theoretical breaking points of the Canopy network exist. To enable multiple VoIP call testing without needing the interaction of actual people to test calls and judge call quality, IXIA’s Chariot product was used to perform traffic pattern analysis and load testing of multiple calls across a Canopy network. Canopy modules were setup in a point to multipoint network configuration with one Access Point (AP) and two Subscriber Modules (SM) in Phase II. Several settings and configurations were adjusted during the second phase to determine optimal design for performance and scalability. The types of settings that were adjusted during the second phase of testing to determine the Canopy network limitations and optimal configuration were:

Bandwidth percent allocation between downlink and uplink connections Number of calls per SM Number of SMs per AP Compression Algorithm (Codec) Quality of Service – High Priority (on or off) Voice Traffic with and without data traffic

For the purposes of this paper Canopy 5.2 GHz radios were used and all SMs were set at a distance of within 2 miles of the AP. This testing was based on typical RF configurations without interference.


5

��

� ��

��

��

�� !��

�� "��"�� #��

$�� %�� Taking into consideration the anticipated traffic patterns of the end users (predominately data or voice or an equal mixture of both) the customer can configure the network optimally to maximize their return on investment of equipment and capital expense, while still providing a high quality of service and cost effective alternative solution to toll dialing to the end user. Based on our findings, it has been calculated that in an all voice traffic network using the G.711 codec (common VoIP codec), Canopy can support 28 - 33 simultaneous voice calls per AP (see Phase 2 Findings). The call volume can be distributed over a number of SMs, not to exceed 5 calls per individual SM. If a wireless ISP is looking to support both voice and data traffic on the same connection to the end user, and half of the bandwidth was allocated to data and half to voice, then it is expected that approximately 14 to 16 simultaneous calls could be transmitted per AP. This would allow the voice calls to be transmitted at a high quality level while still allocating sufficient bandwidth for data transmission.

&� � �� %��

��$ ��!$�� '(�%�� #��)*(+

� �� *�#�� ,-��.*�

�$�� /-��/*�� #��$ ��*�#��

0�,-��.*�� Depending on a Canopy customer’s business plan for rate of oversubscription and acceptable level of blocks calls, the number of calls stated above will be adjusted up or down depending on the expected traffic volume. To properly plan a Canopy network configuration with VoIP, Canopy customers are strongly advised to use Erlang tables in creating a business plan to estimate the load on their network to be able to determine a specific oversubscription rate (acceptable amount of blocked calls). Please see the “Voice Over IP Basics” section of this paper for more information on Erlang tables.


6

��%#��&�(��%#�� Voice over IP (VoIP) permits the movement of voice traffic over Internet Protocol (IP)-based network. IP is a standard for data transmission based on packet-switching technology. Voice is broken into a series of packets at the transmitting end. The components are then reassembled and decoded at the receiving device. Voice communications is both real time and mission-critical. Any delay can make a call prohibitive and lead to an undesired poor quality of service. Packet loss can be caused by router congestion that may lead to a loss of portions of words or sentences. Traffic can multiply as the number of routers is increased in the network leading to longer delays. Network jitter, where packets don't arrive in sequence, can lead to unavoidable delays and poor quality of service.

Circuit Switched vs. Packet Switched Telephony

Circuit Switched: Nearly all voice traffic is circuit switched and transmitted over a Public Switched Telephone Network (PSTN). The speed with which voice is transmitted is an aggregate rate of 64 kbps. A direct connection between two connection points provides a permanent 64 kbps link for the duration of the call. This link cannot be used for any other purpose during this time. PSTN provides low latency (delay) and is bidirectional, allowing for a two-way or full-duplex conversation to take place. The main shortfalls of circuit switching are provided by the inflexibility and inefficiency inherited in the network by requiring a dedicated connection each time.

Packet-Switched Telephony: In a packet-switched network, data is broken down into packets, each with a destination address. When the packets are transmitted through the network, the addresses are read at each router, or network switch, for forward routing. At the destination, the packets are reassembled and re-sequenced. Depending on congestion levels in the network, packets may take different routes on their way to the destination. Packet switching provides a virtual circuit connection and is generally half-duplex. The main difference from the circuit-switched network is that there is no dedicated connection. This is a connectionless network, which allows network resources to be used very efficiently as bandwidth can be shared between applications.

Coder/Decoder (Codec) A voice coder is the device that converts an analog voice signal into a digital signal. The digital signal is also compressed to reduce bandwidth requirements. Using a hybrid coding technique with complex algorithms, the voice waveform is sampled and the speech parameters are extracted. Thus, in any predefined time period, the waveform is assembled by a synthesis technique to closely assemble the original waveform. The best way to reduce latency is to change the voice coding method; however, the trade-off is voice quality vs. bandwidth required. While there is a delay in the voice compression methods used, there is little further delay with decompression regardless of the algorithm used.


7

Table 1

Compression Algorithms

Algorithm Description and Rates

G.711u Pulse code modulation (PCM) specifies the initial analog-to-digital conversion of speech. Speech is transmitted at 64 kbps – which is considered to be toll quality. ITU standard for H.323-compliant codecs and most frequently used in the USA.

G.711a Same as above, however it uses the A-law for companding, which is the most frequently used standard in Europe.

G.726 A waveform coder that uses Adaptive Differential Pulse Code Modulation (ADPCM) at 32 kbps. ADPCM is a variation of PCM, which only sends the difference between two adjacent samples, producing a lower bit rate.

G.729 High-performing codec; offers compression with high quality. Algorithm runs at 8.4 kbps with 10-ms delay and a compression ratio of 8-to-1.

G.723.1-MPMLQ

ITU algorithm that offers voice transmission with quality at a rate of 6.3 kbps with 30-ms delay. Uses the multi-pulse maximum likelihood quantization (MPMLQ) impression algorithm.

G.723.1- ACELP

ITU algorithm that offers voice transmission with quality at a rate of 5.3 kbps with 30-ms delay. Uses the conjugate structure algebraic code excited linear predictive compression (ACELP) algorithm.

Source: Gartner & IXIA

Integration of VoIP with the Public Switched Telephone Network Despite some similarities, there are fundamental differences in the way signaling takes place in a PSTN and in packet networks based on IP. Signaling is essential to ensure call-related control information necessary to establish, bill and terminate connections. To allow full convergence and seamless integration of functionality between PSTN and IP networks, signaling has to be developed to avoid degradation of services. The signaling used in a PSTN is carried over a different physical network known as Signaling System 7 (SS7). SS7 messages are exchanged in the form of data packets similar to the way IP networks transmit data, on a dedicated overlay network used exclusively for signaling. By using this separate network for "out-of-band" signaling, this ensures that lines are clear and are free prior to setting up calls. IP networks do not use the same type of “out-of-band” signaling. Instead, they use the protocols such as H.323 or SIPs that are not compatible with the PSTN. A new signaling architecture has been required to integrate both networks seamlessly; however, this does not yet exist. A variety of protocols have been proposed to meet PSTN-IP integration, and these include H.323, SIP IPS7 and Megaco to migrate interoperability and signaling issues. Nevertheless, these standards are being improved to protect against packet loss and to allow for the transmission of toll-quality voice services. Until these are resolved, it is likely that Internet telephony traffic will be carried over dedicated IP networks.

The Anatomy of a VoIP Call

Encoding and Packetization In the first stage, the digitized voice signal (for example, in G.711 format) is encoded and packetized. This operation can be performed either in the user terminal or in a gateway (for example, between a PSTN and a packet-based network). In the latter case, it is assumed that the circuit-switched transport of the voice signal from the user terminal to the gateway introduces only


8

a negligible amount of delay and distortion. The packetization delay is defined as the time needed to collect all voice samples that end up in one packet. A new voice packet is produced at the end of every interval of duration equal to the packetization delay. The choice of the packetization delay involves a trade-off between effective bit rate and delay, because the payload size also scales linearly with the packetization delay

Transport In the second stage, packet transport, the resulting flow of voice packets is transported over a packet-based network consisting of several (access and/or core) nodes. The network delay, i.e., the delay incurred by transporting a voice packet over the network, can be split into two parts: a deterministic part referred to as the minimal network delay; and a stochastic part referred to as the total queuing delay. The minimal network delay consists mainly of the propagation delay (5 µs/km), the sum of all serialization delays, and the route look-up delay. It is assumed that route updates are so infrequent that the probability of one occurring during a phone call is negligible. Hence, the minimal network delay is constant. The total queuing delay is the sum of the queuing delays in all the nodes that are crossed. The queuing delay in one network node is due to the competition of several flows for the available resources in the queue of that node. The total queuing delay is responsible for the jitter introduced in the voice flow. That is, a flow of voice packets that entered the network with constant inter-arrival times does not leave the network in the same way, because some voice packets are delayed more than others. The jitter (also referred to as packet delay variation) is defined as the delay difference between the fastest packet and the slowest packet (though one still considered as on time). Note that during transport over the network, a fraction of the packets may get lost, either due to overflowing queues or due to the erroneous transport over unreliable links.

Dejittering and Decoding Dejittering is absolutely necessary. Since the decoder needs the packets at a constant rate, the jittered packet flow is dejittered and decoded in the third stage. Dejittering a voice flow consists of retaining the fast packets in the dejittering buffer to allow the slow ones to catch up. The fast packets are the ones that do not have to queue in any of the nodes. As voice can tolerate some packet loss, it is not mandatory that the dejittering mechanism wait for the slowest packet. A (small) fraction of the packets may be considered as arriving too late. This fraction of overdue packets results in what is called a dejittering loss. If the dejittering buffer were to know if the first packet that arrived was a slow or a fast one, it could compensate precisely for its queuing delay. Since this knowledge is generally not available, the dejittering mechanism can either assume the worst case (i.e., assume that the first packet to arrive was the fastest possible) or try to gradually learn how the delay of the first packets relates to the delay of consecutive packets.

Mouth-to-ear delay and overall distortion The individual contributions of the above stages (Encoding and Packetization, Transport, Dejittering and Decoding) combine to give the one-way mouth-to-ear delay and the overall packet loss. The mouth-to-ear delay of a packetized phone call (in one direction) is made up of four components:

1. The packetization delay. 2. The DSP delay, which includes the delay due to encoding, decoding, look-ahead, and

echo control. The lower limit of this delay is the sum of all look-aheads: even if technology continues to evolve to culminate in DSPs with dazzling processing power, the look-aheads remain unaffected.

3. The total minimal network delay, which is the delay of the fastest possible packet. The total propagation delay is the lower limit of the minimal network delay.


9

4. The dejittering delay, which is the delay introduced by the adaptive dejittering mechanism on the fastest possible packet to compensate for the total queuing delay encountered by the slowest packet.

Overall distortion stems from encoding the voice signal and packet loss in the network or in the dejittering buffer.

Standards for Measuring Call Quality Call quality measurement has traditionally been subjective: picking up a telephone and listening to the quality of the voice. The leading subjective measurement of voice quality is the MOS (mean opinion score) as described in the ITU (International Telecommunications Union) recommendation. In voice communications, particularly Internet telephony, the mean opinion score (MOS) provides a numerical measure of the quality of human speech at the destination end of the circuit. The scheme uses subjective tests (opinionated scores) that are mathematically averaged to obtain a quantitative indicator of the system performance.

Compressor/decompressor (codec) systems and digital signal processing (DSP) are commonly used in voice communications because they conserve bandwidth. But they also degrade voice fidelity. The best codecs provide the most bandwidth conservation while producing the least degradation of the signal. Bandwidth can be measured using laboratory instruments, but voice quality requires human interpretation.

To determine MOS, a number of listeners rate the quality of test sentences read aloud over the communications circuit by male and female speakers. A listener gives each sentence a rating as follows: (1) bad; (2) poor; (3) fair; (4) good; (5) excellent. The MOS is the arithmetic mean of all the individual scores, and can range from 1 (worst) to 5 (best).

Table 2

Mean Opinion Score (lower limit) User Satisfaction

4.34 Very satisfied 4.03 Satisfied 3.60 Some users dissatisfied 3.10 Many users dissatisfied 2.58 Nearly all users dissatisfied

The E-model is a complex formula; the output of an E-model calculation is a single score, called an “R factor,” derived from delays and equipment impairment factors. Once an R factor is obtained, it can be mapped to an estimated MOS. R factor values range from 100 (excellent) down to 0 (poor). An estimated MOS can be directly calculated from the E model’s R factor.

Considerations for Network Capacity Planning When a Canopy provider (WISP) is building a business plan they must take into account, oversubscription rate, number of acceptable blocked calls, and quality of service to customers. All of these factors are important and must be considered before providing or advertising VoIP service. The WISP will have to make a choice on whether or not they will provide VoIP service through call manager equipment (i.e. Cisco, Nortel, etc.) or if customers will acquire service on


10

their own (i.e. Vonage, AT&T CallVantage, etc.) and leverage Canopy for broadband access where the third-party CPE (customer premise equipment) device does the VoIP encapsulation. As in any other bandwidth based network there is going to come a point when the network is at capacity. When this happens, there are a few options on how to handle it depending on the choice of service. In the case of VoIP service being provided through a call manager, the WISP has a couple of options. The WISP can either continue to allow calls to be added to the network and allocate less and less bandwidth per call or block the last call that pushes the network over its capacity. If the number of calls on the network is allowed to increase without limit, this will degrade the call quality of all calls on the network because less bandwidth will be available per call. This is not the recommended approach because this will cause an overall low quality of service resulting in jittering phone connections to many customers. However, if the last call is just blocked and given a “network is busy” signal, then this user can just try redialing in a few seconds when some capacity will most likely have become available and all other calls will not be affected. Managing the bandwidth in this fashion will allow a higher level of service to be provided and managed across the network. If the WISP is going to allow its customers to purchase a Vonage type service on their own and utilize Canopy for broadband bandwidth, this is a little more difficult to manage. In this scenario, the WISP will not be able to differentiate between regular data packets and voice packets that are encapsulated as data. They will only have the option to manage overall bandwidth consumption. A WISP should take this into consideration when forming their business plan in this scenario. Since the WISP will have less control over voice call quality on an individual call level, they may want to be more conservative in the service guarantees that they state for their customers around voice quality. To assist WISPs in this type of capacity planning it is recommended that they take into consideration Erlang tables which are discussed in the next section.

Erlang Tables An Erlang is a unit of telecommunications traffic measurement. Strictly speaking, an Erlang represents the continuous use of one voice path. In practice, it is used to describe the total traffic volume of one hour. For example, if a group of users made 30 calls in one hour, and each call had an average call duration of 5 minutes, then the number of Erlangs this represents is worked out as follows:

Minutes of traffic in the hour = number of calls x duration = 30 x 5 = 150 Hours of traffic in the hour = 150 / 60 Hours of traffic in the hour = 2.5 Traffic figure = 2.5 Erlangs

Erlang traffic measurements are made in order to help telecommunications network designers understand traffic patterns within their voice networks. This is essential if they are to successfully design their network topology. Erlang traffic measurements or estimates can be used to work out how many lines are required between a telephone system and a central office, or in the case of Canopy, given a level of available bandwidth, determine the acceptable amount of blocked calls between a SM and an AP. Several traffic models exist which share their name with the Erlang unit of traffic. They are formulas which can be used to estimate the number of lines required in a network.


11

The main Erlang traffic models are listed below:

Erlang B This is the most commonly used traffic model and is used to work out how many lines are required if the traffic figure (in Erlangs) during the busiest hour is known. The model assumes that all blocked calls are immediately cleared.

Extended Erlang B This model is similar to Erlang B, but takes into account that a percentage of calls are immediately represented to the system if they encounter blocking (a busy signal). The retry percentage can be specified.

Erlang C This model assumes that all blocked calls stay in the system until they can be handled. This model can be applied to the design of call center staffing arrangements where, if calls cannot be immediately answered, they enter a queue.

Erlang to VoIP Bandwidth Calculation As explained above, the concepts of Erlang tables can be applied in a number of different ways to a voice telecommunications network. In the context of a Canopy network, the users will be applying these concepts to VoIP applications. There are a number of calculators that are available to assist Canopy users in developing an appropriate business plan for their network. Below is an example calculation of Erlangs capacity for a VoIP network using a Erlang to VoIP Bandwidth Calculator7: Given That:

G.711 Codec (64 kbps) is being used with a 20 ms packet duration The Acceptable amount of blocked calls is 1 out of 100 (.01) Available bandwidth is 3.1 Mbps (3250.59 kbps)

Then the network can handle a level of 28.1 Erlangs. This means that in its busiest hour the network can handle 28.1 hours worth of total traffic volume. This volume can be distributed among a number of different users, which should be taken into consideration in developing an appropriate business plan.


12

��*)��('��%��(��%+

Canopy Frame Packets There are 33 data slots in a basic canopy frame and each slot has a 64 byte payload. However, this number will vary based on the number of control slots selected. Each frame has three control slots by default and can be increased at the cost of data slots. Every two control slots allocated decreases available data slots by one. There are also three downlink and three uplink acknowledgement slots.

Control slots The control slots are contention slots for the SMs to request to transmit data to the AP. Frame duration is 2.5ms (400 frames/second). The SMs can only transmit data when they have been granted a control slot by the AP.

Downlink and Uplink Acknowledgement slots There are three downlink and three uplink acknowledgement slots in a canopy frame. The purpose of the acknowledgement slots is to confirm a successful receipt of an upload or download to the AP. The amount of acknowledgement slots can be altered in the AP configuration, but is not suggested unless turning on QoS.

High Priority / Quality of Service (QoS) The purpose of the high priority channel is to handle traffic with a small tolerance for latency, such as voice. If the Type of Service (TOS) bit is set for high priority the AP will prioritize this traffic in the queue and hold back any data that is not designated as such. The high priority designation is a static allocation, meaning that when a number of slots are reserved for high priority they can only be used for this purpose. If no high priority traffic is being passed the designated high priority bandwidth will remain idle and unavailable for other traffic.


13

�� ,�(%��%+��-�(� To determine values such as MOS and R-factor, it is not feasible to have human listeners to make these subjective judgments at all times. For the purposes of these tests and this paper, IXIA’s Chariot software product was used to determine these values and compile the data necessary. Chariot has the capability to provide a tremendous amount of data in a testing environment. The following is an example of the types of information that can be gathered. For the purposes of this whitepaper, the focus was to use the advanced call quality measurements to determine how VoIP traffic performs on the Canopy network.

Advanced call quality measurements Predicts call quality by calculating a MOS based on the industry standard E-model specified in the ITU recommendation G.107. Improving on the base standard, the VoIP Test Module takes into account additional network factors, such as jitter and consecutive lost datagrams, which can severely impact overall call quality

Tests VoIP-enabled network equipment Examines the effectiveness and performance of VoIP-enabled network equipment. The VoIP Test Module enables the user to verify that prioritization techniques work as planned with a mixture of traffic and measure the performance impact of other network elements, such as VPNs, on delay-sensitive VoIP traffic. Enables the user to test the limits of the network by generating up to 10,000 VoIP sessions. By identifying the point where call quality begins to suffer, the VoIP Test Module empowers the user to make informed decisions about the implementation and expansion of VoIP in the network.

Emulates complex networks in test lab Allows the user to emulate complex networks with a mixture of both VoIP and non-VoIP traffic by using Chariot and its VoIP Test Module. By using Chariot in the lab environment, the user can stress test network equipment, test network changes before deployment or replicate end-user environments and reported problems. Chariot evaluates the effectiveness of QoS. The user can ensure that voice traffic is receiving necessary resources at the proper time without starving other business-critical applications.

Optimizes network design Supplies on-demand testing for tuning network to minimize delay, jitter and lost data.


14

�,�� ,��(��%#��'�.��%+�$�%+��+��&�(��*) In the initial phase of VoIP testing over Canopy, a Vonage broadband phone was used to test an actual VoIP call going through Canopy out through the Internet. The call was monitored to gather statistics on bandwidth usage so a theoretical model could be created to determine how many VoIP calls a Canopy AP can support.

Canopy Configuration In this phase, the Canopy modules were setup in a point to multipoint network configuration with one Access Point (AP) and two Subscriber Modules (SM). All Canopy modules were 5.2 GHz with uplink and downlink percentages at 50%. During this test, Quality of Service (QoS) was not set as it would not have any affect on one VoIP call with no data traffic in the throughput of the call. The AP was connected to the Internet via a DMZ and router. We used a Vonage VT1000 (made by Motorola) to conduct actual VoIP calls via the Internet. While conducting the VoIP calls, Compuware’s Application Vantage tool was used to monitor the calls and gather call statistics (throughput, frames, etc.).

Hub

Internet

CompuwareTool

AP RouterDMZSM

Vonage/Motorola

Analog PhoneSM

Vonage Configuration The Vonage VT1000 can be used with three different codecs:

G.711 (listed at 90 kbps by Vonage) G.726 (listed at 50 kbps by Vonage) G.729 (listed at 30 kbps by Vonage)

All three codecs were used during the tests to determine theoretical call capacity results for the AP depending on the codec being tested.

Test Scenarios Three calls were completed from the Vonage VT1000 for each codec. Each call was recorded and monitored with the Compuware’s Application Vantage tool.


15

Test Results The results below were gathered using Compuware’s Application Vantage tool. The “Theoretical maximum VoIP calls per AP” is based on Motorola’s Canopy documentation that the maximum uplink throughput using more than one SM is 3.1 Mbps (3250.59 kbps). Since a VoIP call is full-duplex and approximately the same amount of throughput will go upstream as well as downstream, the amount of VoIP calls Canopy can support is limited by the lower throughput limitation of the AP. When the AP is configured with two or more SMs on the network, the Canopy AP is documented as having up to 3.1 Mbps (3250.59 kbps) upstream, so 3.1 Mbps (3250.59 kbps) was used as the theoretical maximum.

Table 3

Vonage call (kbps)

Duration (seconds)

Ave. Bandwidth

(kbps)

AP Max* (kbps)

Theoretical max VoIP calls per AP

Frame Size

(bytes)

Frames per

second G.711 (90) 94.63 162 3250.59 20.1 214 96 G.726 (50) 105.04 99 3250.59 32.8 134 93 G.729 (30) 76.96 58 3250.59 56 74 99

* Upstream maximum of AP is approximately 3.1 Mbps (3250.59 kbps)

Findings The Access Point should be able to handle anywhere from 20 to 56 calls depending on the codec used and if all the traffic is VoIP (no data). The Access Point can handle more VoIP calls when using a codec that uses lower compression.

The Canopy system was fairly easy to setup and assemble and integrates well with the Vonage product. There were no special configurations needed to setup the Vonage VoIP product to work successfully on the Canopy network.

Recommendations The following recommendations are based on the output and statistics determined during the above testing:

When configuring a Canopy network that is primarily for VoIP traffic, the AP should be set at 50% for the ratio of uplink to downlink traffic. Since the uplink throughput is the constraint for VoIP traffic on a Canopy network, the user needs to have the uplink throughput as close to the 3.1 Mbps (3250.59 kbps) maximum.

In a primarily data network, it is likely that the downlink to uplink percentage will be set at approximately 75% to accommodate for most subscribers doing more download-type traffic. If VoIP is added to a saturated data network, customers could experience choppy and jittery VoIP calls. A WISP will need to carefully plan and test before adding VoIP as the throughput to an SM has many variables (distance between AP and SM, uplink/downlink percentage, number of SMs on the network, etc.).


16

�,�� #��%�%�)��%+�$�%+�� ,�(%�� In the second phase, the objective was to test scalability with more than just one VoIP call. To enable multiple VoIP call testing without needing the interaction of actual people to test calls and judge the call quality, IXIA’s Chariot product was used to perform traffic pattern analysis and load testing of multiple calls across a Canopy network.

Canopy Configuration In this phase, Canopy modules were setup in a point to multipoint network configuration with one Access Point (AP) and two Subscriber Modules (SM). Tests were performed using both one and two SMs, but two SMs were always registered with the AP. All Canopy modules were 5.2 GHz with the following variables:

Uplink / Downlink percentage o 50% / 50% o 75% / 25%

Quality of Service (QoS) o Tests were run with QoS on and off o High priority percentage was set at 25%. o Acknowledgement and Control slots:

� High priority uplink acknowledgement slots, high priority downlink acknowledgement slots, and high priority control slots were set at 3.

� Total uplink acknowledgement slots, total downlink acknowledgement slots, and total control slots were set at 6.

Switch

AP Switch

SM

IXIA Endpoint

Hub

IXIA EndPoint

SM

IXIA EndPoint

IXIA Chariot Monitor

IXIA Chariot Configuration The Chariot tool consists of two programs: the IXIA Chariot Console and the IXIA Endpoints. The console station is used to setup and initiate all tests. The tests can consist of VoIP traffic, multiple types of data traffic, or both VoIP and data traffic. The Endpoints are used to accept and receive the data defined in the test scenario. The tool allows VoIP or data traffic to be generated and sent to different Endpoints to simulate real-world traffic over a network. In this scenario, an Endpoint was put on the end of each SM and the AP with the Chariot console connected via a hub at an SM to be able to initiate and monitor all tests.


17

Test Scenarios The testing environment was a combination of one or two SMs and one AP. Additionally, there are a number of variables that can influence the overall performance of the voice/data network. The variables that were adjusted during testing included:

Bandwidth percent allocation between downlink and uplink connections o The downlink and uplink percentage allocation was adjusted to determine

optimal performance and limitations of the equipment. The two scenarios that were tested were 50% downlink and 50% uplink as well as 75% downlink and 25% Uplink.

Number of calls per SM o The call volume was adjusted from one call up to as many as 25 calls per

SM. The purpose of adjusting this variable was to determine the theoretical maximum capacity of each SM and AP.

Number of SMs per AP o This was adjusted to test the limitations and performance of the AP under

different conditions. Compression Algorithm (Codec)

o As a result of altering the compression, the codec affects the possible call volume and call quality.

Quality of Service – High Priority (on or off) o The high priority bit in the Canopy product is simply a Type of Service (TOS)

bit that the user can select to either have on or off. In an all data world the high priority bit designation is not as critical given that a slight delay on the transmission of data traffic is not nearly as meaningful to network performance as a delay on a voice packet. A delay to a voice packet can result in an audible flaw in quality as jitter and lost information. The high priority bit was applied in different test scenarios to demonstrate the impact it would have on transmitting voice traffic more cleanly and managing data traffic accordingly.

Voice traffic with and without data traffic o To properly test the impact of having a high priority bit designated this

scenario was tested in conjunction with data traffic passing at the same time.

Test Results For a summary of the test results from Phase II, please see the “Findings” section below. For detailed test results broken down codec and variable scenario please see Appendix A.

Findings

Summary of Results After collectively looking at the testing results for the different codecs with different variables it is more apparent which performed the best with the most robust call quality. The summary below is looking at the tests with two SMs. The detailed information on the testing results with one SM is included in Appendix A., however, for the purposes of analysis and theoretical modeling the results from two SMs were used as they gave more real world results. One of the other main distinctions to note in the data is the difference in performance of 50/50 downlink/uplink percentage versus 75/25. The uplink connection is the limiting factor in the call quality and bandwidth throughput because the SMs have less bandwidth capacity on the uplink. Taking this into consideration along with the amount of percentage that is allocated for downlink/uplink traffic,


18

some conclusions can be drawn about which scenarios would work best for each network configuration. For the most robust codecs five calls per SM appeared to be the “breaking point” after which point the call quality dropped off considerably. Given that the Canopy product is currently used primarily as a “data mover,” one would want to take into consideration the performance of the Canopy under these different configurations. Noting that the downlink/uplink allocation is a static setting, the user wants to make sure that they are not leaving more bandwidth than necessary in an idle position. When determining how to allocate the data it is worth while to take a look at the differences in performance below. In a 50/50 configuration one can clearly push more calls through with a higher rate of quality (higher MOS). If the anticipation is that the network will have predominately voice traffic, the user will want to get as close to the 50/50 configuration as possible to raise the amount of possible VoIP calls that Canopy can support. In a 75/25 configuration where users will still be doing some download of “data” traffic, the user can still expect acceptable call quality with a significant volume of calls being pushed through the Canopy network. Actual Results with two SMs

Table 4

Codec Downlink/Uplink Percentage

Number of Calls from Each SM

Total # of Calls MOS

G.711a 50/50 5 10 4.36 G.711a 75/25 4 8 4.13 G.711u 50/50 5 10 4.37 G.711u 75/25 4 8 4.32 G.726 50/50 5 10 4.16 G.726 75/25 5 10 4.02 G.729 50/50 5 10 3.93 G.729 75/25 5 10 4.01

G.723.1 ACELP 50/50 8 16 3.60 G.723.1 MPMLQ 50/50 8 16 3.80

Quality of Service The high priority setting on the Canopy product is strictly a bit designation indicating that the traffic with that bit set has high priority and goes before any other data. Several different configurations were tested to determine the robustness of the QoS setting. One, two and three high priority slots were allocated during testing. The results showed that with the 75/25 downlink/uplink configuration, the best performance was with two high priority slots set with two SMs. The high priority slots are set in the Canopy configuration table by indicating which percentage of available slots the user would like to set as high priority. To achieve two slots, the high priority percentage was set to 25% (to achieve two slots, a customer may need to try multiple percentage configurations as the slots allocated will be dependent on the specific Canopy setup). Tests in this configuration yielded a very good MOS value of 4.33. As part of the QoS testing, different size data packets were sent with the voice traffic to determine if there would be any negative impact on the quality of the voice call. These tests were successful with varying data size files being passed. Starting with a file size of 128 kb and gradually moving up to a file size of 1.2 Mb added to the network during voice testing, calls were still transmitted with a MOS value of approximately 4.3. This demonstrates that the Canopy QoS


19

performs as desired as the latency and delay sensitive voice traffic is given priority and is able to perform with acceptable quality no matter how much data is being passed at the same time.

Theoretical Modeling based on Actual Results After looking at the actual results from testing and knowing the maximum throughput of the AP, it is possible to extrapolate what the system performance would be like under expanded conditions. The capacity of the AP for an uplink connection with multiple SMs is 3.1 Mbps (3250.59 kbps) of data. The theoretical bandwidth compression of each codec is also known and listed in the table below. The actual bandwidth compression can be calculated by taking the rate at which the data was sent for a particular call and multiplying that by the frame size.. The rate is calculated by taking the total amount of data sent over one call test and dividing that by the call duration. For example, the G.711 codec sent 7,500 packets of data during a 150 second call. The rate is 50 packets/sec. The rate for G.726 and G.729 was identical to G.711, 50 packets/sec. G.723.1 MPMLQ and ACELP had a rate of 33 packets/sec. It is also known that a Canopy frame is 64 bytes. From the testing done with the Ixia Chariot software, the frame size used for all of the codecs is also known. The frame size for each codec, including overhead, is listed in the table below. By knowing the total frame size and the rate of transmission, the actual compression rate can be calculated. By continuing the example of the G.711 codec, 50 packets/sec (rate of transmission) times 238 bytes (frame size) equals 11,900 Bps. This now should be converted from Bytes/sec to Bits/sec by multiplying the value by 8, which equals 95 kbps. When the size of the frame packet of the codec is larger than 64 bytes, 4 bytes for overhead must be added for routing purposes to each packet before the packet is fragmented into 64 byte packets. Using this information as a benchmark for a typical VoIP application, the efficiency of data transfer on the network can be calculated. By taking into account all of this information a more accurate theoretical model can be constructed. Given all of this information, the theoretical total number of simultaneous calls that the AP can support can be calculated and is listed in the below table. Looking at the G.711 codec as an example, the flow of the calculation would be as follows:

Given that there is approximately 3.1 Mbps (3250.59 kbps) of total bandwidth available and the actual data compression rate is 95 kbps, the theoretical calculation would simply be 3250.59 Kbps / 95kbps = 34 simultaneous calls.

Since the total frame size is 238 bytes and the Canopy equipment transmits in 64 byte packets, 4 Canopy frames (256 bytes) would be required to handle the total bandwidth from the G.711 codec. To account for overhead, 4 bytes must be added to each fragmented frame. In this case 4 frames are required, so 16 bytes total must be added to the 238 byte frame. This produces an average transmission efficiency rate of 99% overall (254 bytes/256 bytes). In a transmission where each 64 byte Canopy packet was being completely utilized (the application produced packets in 64 bytes frames), the network would be operating under perfect conditions.

The theoretical 34 calls can now be used in conjunction with the efficiency rate to calculate the adjusted theoretical number of calls the network can handle. This results in 34 calls times 99% efficiency, which is equal to 33 calls total.

This same calculation can be made for each codec using the corresponding numbers with each different compression algorithm.


20

Table 5

Codec Listed Compression

Actual Compression Throughput

Total Frame Size

(bytes)

Theoretical Simultaneous Calls an AP Can Support

Transmission Efficiency

Rate

Theoretical Simultaneous Calls an AP Can Support

Factoring Efficiency

Rate G.711a

& G.711u

64 kbps 95 kbps 238 34 99% 33

G.726 32 kbps 55 kbps 138 59 78% 46

G.729 8.4 kbps 39 kbps 98 83 83% 68

G.723.1 -

MPMLQ 6.3 kbps 26 kbps 98 125 83% 103

G.723.1 -

ACELP 5.3 kbps 27 kbps 102 120 86% 103

*Upstream maximum of AP is approximately 3.1 Mbps (3250.59 kbps)

The test scenarios that were conducted with one and two SMs demonstrated a clear understanding of the capabilities of individual SMs and the call volume limitations. These results are documented by codec in Appendix A. These tests demonstrated that by utilizing different codecs the SM will have a varying breaking point of when call quality becomes unacceptable. For one SM, this point was approximately five calls depending on the codec. The AP also has a breaking point at the number of simultaneous calls it can support with acceptable call quality with the uplink being the limitation. The initial thought would be that the AP has a total bandwidth limitation and the theoretical maximum should be approximately the same no matter whether the calls are coming from one SM or multiple SMs. However, it is known that the performance of the AP does scale with an increased number of SMs. An example of this is shown by looking at the performance of the G.711 codec when comparing one SM to two SMs. With the G.711 codec and one SM, the total number of calls that the AP could handle at an acceptable quality level was five calls. Similarly looking at the performance with two SMs it is shown that the AP can now handle a total of ten calls between the two SMs. These results show that the AP does scale as SMs are added and suggests that as more SMs are added, the AP will be able to support the theoretical numbers listed in the table above. It is important to note that these results are in a voice only environment. As the amount of data transferred on the upstream increases, the amount of bandwidth allocated to voice traffic will decrease. The way to guarantee that the same bandwidth is consistently allocated to voice traffic is through the high priority classification under the quality of service option. Taking the actual data from the test results and comparing it against the theoretical calculations of what an AP can handle, the maximum number of calls to AP can be estimated and evaluated per codec. Knowing that the G.711a, G.711u, G.726 and G.729 codecs have produced the most reliable results during testing, those codecs are evaluated below. The test results (low MOS scores) from the G.723.1-ACELP and G.723.1-MPMLQ codecs suggest that these codecs do not


21

transfer well over a VoIP network due to the low data compression (kbps). It is recommended to not use these codecs for any VoIP applications independent of the network being used.

G.711a & G.711u By looking at the column “Theoretical Simultaneous Calls an AP Can Support Factoring Efficiency Rate” from the above table one can see that when using the G.711a and G.711u codecs the AP can theoretically handle approximately 33 calls at 95 kbps per call. This call volume can be generated using as few as 7 SMs or as many as 33 SMs depending on the call volume per SM. Since it has been shown that the AP’s robustness scales with increased traffic it is expected that it should be able to actually handle the theoretical value of 33 calls. Given the high MOS score value with five calls from each SM (4.37), it is expected that the AP will still be able maintain a quality level that is considered satisfactory as it scales to 33 calls.

G.726 By looking at the column “Theoretical Simultaneous Calls an AP Can Support Factoring Efficiency Rate” from the above table one can see that when using the G.726 codec the AP can theoretically handle approximately 46 calls at 55 kbps per call. This call volume can be generated using as few as 10 SMs or as many as 46 SMs, depending on the call volume per SM. Although theoretically the G.726 codec appears like it can handle a large volume of simultaneous calls, by looking at the MOS score at a low call volume it is the expectation that as the network scales any additional degradation in the MOS score would make the call quality unacceptable. Using two SMs and five calls from each SM, the MOS score was 4.16, which is considered a good value. However, when the network scales to as many as 46 simultaneous calls per AP it is expected that the MOS score will continue to degrade to an unacceptable level and may not be able to reach the theoretical maximum.

G.729 By looking at the column “Theoretical Simultaneous Calls an AP Can Support Factoring Efficiency Rate” from the above table one can see that when using the G.729 codec the AP can theoretically handle approximately 68 calls at 39 kbps per call. This call volume can be generated using as few as 13 SMs or as many as 68 SMs depending on the call volume per SM. Similarly to the G.726 codec, theoretically the G.729 codec appears like it can handle a large call volume of simultaneous calls. However, looking at the MOS score at a low call volume it is the expectation that as the network scales any additional degradation in the MOS score would make the call quality unacceptable. Using two SMs and five calls from each SM, the MOS score was 3.93, which is already on the borderline of when call quality begins to falter. In conclusion, even though 68 calls is the theoretical maximum, the call quality will lower to an unacceptable level and will most likely not get very close to 68 calls with an acceptable call quality.

Recommendations Based on the testing done in Phase II the following Conclusions & Recommendations are being made:

The G.711a & G.711u codecs performed the best in terms of highest MOS value with a high number of successful simultaneous calls completed. It is recommended to use these codecs for all VoIP applications. Although other codecs may allow more


22

calls to pass through, it is at the cost of degraded call quality.

As the number of SMs is increased the performance of the AP also scales, which will aid in the Canopy network reaching the theoretical call maximum.

The number of calls an AP can handle will vary based on the codec. The codecs

with higher bandwidth compression, such as G.729, will allow more calls to be completed per AP as they take up less bandwidth per call. However, with these codecs the call quality is considerably less and as the call volume increases the MOS value will degrade and lead to unacceptable call quality as the theoretical maximum is approached.

The codecs with the lower bandwidth compression, such as G.711, allowed fewer

calls to pass through at a much higher quality level. It would be expected that degradation in MOS value here would not have as much of an impact because it is starting at a much higher value.

Bandwidth allocation through Canopy is static and should be taken into consideration

when determining how much bandwidth is needed for a customer’s network.

o For a network that will be running primarily voice traffic, a WISP would want to consider setting the downlink/uplink percentage at 50/50. This will give sufficient bandwidth to the uplink on the AP and will allow the SM to support more calls at a higher call quality rate.

o In a customer environment where “non-voice data” is still the primary type of data being transferred, it may be that the WISP will require a significant amount of bandwidth for downstream data transfer. The downstream bandwidth allocation requirement could force the WISP to put the downlink/uplink closer to a 75%/25% configuration. If the expectation is that a customer will not be doing more than 1 - 2 calls simultaneously from one SM, the 75%/25% configuration can transmit the calls with very good quality (MOS value greater than 4 and low packet loss).

A high performance setting for Canopy QoS with the downlink/uplink set at 75%/25%

is two high priority slots active. This will allow sufficient bandwidth for 1 or 2 calls to be completed with good call quality, which will be typical in most Canopy applications in a residential setting.

QoS was shown to successfully filter data traffic independent of the size of the data

file. While increasing the data file size incrementally up to 1.2 Mb the voice call quality still passed at a MOS value well above the acceptable level of 4.


23

**�.%"�

Phase II Test Results The following tables depict the statistical results for each test using the IXIA Chariot software. Within each of the results below, the test setup, explanation and all variables are defined.

Canopy set at 50% downlink / 50% uplink – G.711a Multiple tests were conducted with VoIP traffic only for the G.711a (64 kbps) codec over one and two SMs. From the test results below, five VoIP calls were run successfully over one SM, but the addition of a sixth call begins to show call degradation. As the upstream VoIP traffic begins to get a large one-way delay and amount of lost packets, the MOS score lowers which indicates call quality would become jittery and would not be acceptable. Tests were then conducted using traffic from two SMs. As SMs are added, the AP is able to scale better as 5 calls from each SM were successfully completed (10 total). As a sixth call was added to each SM, a sharp increase in lost bytes and one-way delay resulted which contributed to unacceptable MOS scores. In a two SM environment, 10 – 11 calls can be supported successfully, but any additional calls will cause degradation in performance.

Table 6

Codec Downlink /

Uplink Percentage

Number of SMs

Number of calls from each SM

One way delay

avg (ms) Bytes Sent MOS

avg.

R-Value Avg.

Lost Bytes

G.711a 50/50 1 5 20 12,000,000 4.37 91.32 0

G.711a 50/50 1 6 56 13,942,400 3.62 71.21 221,594

G.711a 50/50 2 5 32 11,999,840 4.36 90.85 160

G.711a 50/50 2 6 93 13,932,160 2.87 51.10 467,680

Canopy set at 50% downlink / 50% uplink – G.711u The tests below were conducted for the G.711u codec (64 kbps). This codec is very similar to G.711a and received similar results. From the data below, five VoIP calls were run successfully over one SM, but the addition of a sixth call begins to show call degradation. When the tests are run over two SMs, very similar results to the G.711a codec are produced. Five calls from each SM were successful, but the addition of a sixth (11th and 12th calls) increased lost bytes and one-way delay which contributed to unacceptable MOS scores.

Table 7

Codec Downlink /

Uplink Percentage

Number of SMs


One way delay


avg.

R-Value Avg.

Lost Bytes

G.711u 50/50 1 5 19 11,979,040 4.34 90.42 20,960

G.711u 50/50 1 10 66 18,892,320 2.69 45.82 4,843,840

G.711u 50/50 2 5 13 11,999,840 4.37 91.35 160

G.711u 50/50 2 6 100 13,928,960 2.86 51.24 456,160


24

Canopy set at 50% downlink / 50% uplink – G.726 Multiple tests were conducted with VoIP traffic only for the G.726 codec (32 kbps) over one and two SMs. From the test results below, six VoIP calls were run successfully over one SM, but the addition of a seventh call begins to show call degradation. As the upstream VoIP traffic begins to get a large one-way delay and amount of lost packets, the MOS score lowers which indicates call quality would become jittery and choppy and would not be acceptable. Tests were then conducted using traffic from two SMs. As SMs are added, the AP is able to scale better and is able to successfully complete 5 calls from each SM (10 total). As additional calls were added to each SM a sharp increase in lost bytes and one-way delay resulted which contributed to unacceptable MOS scores. In a two SM environment, 12 – 14 calls can be supported successfully, but any additional calls will cause degradation in performance.

Table 8

Codec Downlink /

Uplink Percentage

Number of SMs

Number of calls

from each SM

One way delay


avg.

R-Value Avg.

Lost Bytes

G.726 50/50 1 6 23 7,199,920 4.18 84.44 80 G.726 50/50 1 7 68 8,397,120 2.60 45.75 1,390,320 G.726 50/50 2 5 22 11,998,320 4.16 83.96 1,680 G.726 50/50 2 8 66 16,963,960 2.52 40.57 2,236,000

Canopy set at 50% downlink / 50% uplink – G.729 Multiple tests were conducted with VoIP traffic only for the G.729 (8 kbps) codec over one and two SMs. From the test results below, six VoIP calls were run successfully over one SM, but the addition of a seventh call begins to show call degradation. As the upstream VoIP traffic begins to get a large one-way delay and a large amount of lost packets, the MOS score lowers which indicates call quality would become jittery and would not be acceptable. Tests were then conducted using traffic from two SMs. As SMs are added, the AP is able to scale better as five calls from each SM (10 total) were completed at the acceptable limit. As additional calls were added to each SM, a sharp increase in lost bytes and one-way delay resulted in unacceptable MOS scores. In a two SM environment, 10 calls can be supported successfully, but any additional calls will cause degradation in performance.

Table 9

Codec Downlink /

Uplink Percentage

Number of SMs


One way delay


avg. R-Value

Avg. Lost

Bytes

G.729 50/50 1 6 21 1,800,000 4.02 79.87 20

G.729 50/50 1 7 57 2,096,860 3.41 67.18 98,260

G.729 50/50 2 5 21 2,991,500 3.93 77.72 8,500 G.729 50/50 2 8 54 4,799,760 2.90 56.66 486,640

Canopy set at 50% downlink / 50% uplink – G.723.1-ACELP Multiple tests were conducted with VoIP traffic only for the G.723.1-ACELP (5.3 kbps) codec over one and two SMs. From the test results below, this codec received a low MOS score even though there was no lost data and a low one-way delay. Test results showed that anywhere from one to five calls had approximately the same results. Also, performing a test with a low amount of calls between two SMs resulted in a low MOS score.


25

The test results suggest that this codec does not transfer well over a VoIP network due to the low kbps. It is recommended to not use this codec for any VoIP applications independent of the network being used.

Table 10

Codec Downlink /

Uplink Percentage

Number of SMs


One way delay


avg. R-Value

Avg. Lost

Bytes

G.7231 - ACELP 50/50 1 1 - 5 18 990,000 3.63 70.65 0

G.7231 - ACELP 50/50 1 10 45 1,977,540 3.24 62.61 38,000

G.7231 - ACELP 50/50 2 8 30 3,168,000 3.60 70.09 320

G.7231 - ACELP 50/50 2 10 56 3,959,780 3.19 61.56 76,180

Canopy set at 50% downlink / 50% uplink – G.723.1-MPMLQ Multiple tests were conducted with VoIP traffic only for the G.723.1-ACELP (6.3 kbps) codec over one and two SMs. From the test results below, this codec received very similar scores as the ACELP codec as it displayed a low MOS score even though there was no lost data and a low one-way delay. Test results showed that anywhere from one to five calls had approximately the same results. Adding a low amount of calls between two SMs resulted in a similarly low MOS score. The test results suggest that this codec does not transfer well over a VoIP network due to the low kbps. It is recommended to not use this codec for any VoIP applications independent of the network being used.

Table 11

Codec Downlink /

Uplink Percentage

Number of SMs

Number of calls

from each SM

One way delay


avg.

R-Value Avg.

Lost Bytes

G.7231 - MPMLQ 50/50 1 1 - 5 19 1,176,000 3.80 74.63 0

G.7231 - MPMLQ 50/50 1 10 58 2,346,672 3.44 67.06 35,184

G.7231 - MPMLQ 50/50 2 8 24 3,763,200 3.80 74.39 312

G.7231 - MPMLQ 50/50 2 10 91 4,646,688 2.46 42.11 1,198,272

Canopy set at 75% downlink / 25% uplink – G.711a Multiple tests were conducted with VoIP traffic only for the G.711a (64 kbps) codec over one and two SMs. From the test results below, four VoIP calls were run successfully over one SM, but the addition of a fifth call begins to show call degradation. As the upstream VoIP traffic begins to get a large one-way delay and a large amount of lost packets, the MOS score lowers which indicates call quality would become jittery and would not be acceptable. Tests were then conducted using traffic from two SMs. As SMs are added, the AP is able to scale better as four calls from each SM (8 total) were completed successfully. As a fifth call was added to each SM, a sharp increase in lost bytes and one-way delay contributed to unacceptable


26

MOS scores. In a two SM environment, 8 – 9 calls can be supported successfully, but any additional calls will cause degradation in performance.

Table 12

Codec Downlink /

Uplink Percentage

Number of SMs

Number of calls

from each SM

One way delay avg

(ms) Bytes Sent MOS

avg.

R-Value Avg.

Lost Bytes

G.711a 75/25 1 4 22 9,512,000 4.22 87.07 16,960

G.711a 75/25 1 5 68 11,906,720 3.54 69.87 218,880

G.711a 75/25 2 4 24 19,047,040 4.13 85.02 52,320

G.711a 75/25 2 5 71 16,842,240 3.41 67.24 349,280

Canopy set at 75% downlink / 25% uplink – G.711u Multiple tests were conducted with VoIP traffic only for the G.711u (64 kbps) codec over one and two SMs. From the test results below, four VoIP calls were run successfully over one SM, but the addition of a fifth call begins to show call degradation. As the upstream VoIP traffic begins to get a large one-way delay and a large amount of lost packets, the MOS score lowers which indicates call quality would become jittery and choppy and would not be acceptable. Tests were then conducted using traffic from two SMs. As SMs are added, the AP is able to scale better as four calls from each SM (8 total) we completed successfully. As a fifth call to each SM was added, a sharp increase in lost bytes and one-way delay contributed to unacceptable MOS scores. In a two SM environment, 8 – 9 calls can be supported successfully, but any additional calls will cause degradation in performance.

Table 13

Codec Downlink /

Uplink Percentage

Number of SMs


One way delay


avg. R-Value

Avg. Lost

Bytes

G.711u 75/25 1 4 23 9,600,000 4.33 90.13 4,960

G.711u 75/25 1 5 69 12,000,000 3.78 76.11 121,600

G.711u 75/25 2 4 28 19,200,000 4.32 89.67 11,840

G.711u 75/25 2 5 70 23,999,360 3.78 76.29 240,960

Canopy set at 75% downlink / 25% uplink – G.726 Multiple tests were conducted with VoIP traffic only for the G.726 (32 kbps) codec over two SMs. From the test results below, 5 calls from each SM (10 total) were completed successfully. As a sixth call was added to each SM, a sharp increase in lost bytes and one-way delay contributed to unacceptable MOS scores. In a two SM environment, 10 – 11 calls can be supported successfully, but any additional calls will cause degradation in performance.

Table 14

Codec Downlink /

Uplink Percentage

Number of SMs


One way delay


avg. R-Value

Avg. Lost

Bytes

G.726 75/25 2 5 40 11,875,520 4.02 80.70 22,720

G.726 75/25 2 6 75 14,399,360 2.83 50.64 948,720


27

Canopy set at 75% downlink / 25% uplink – G.729 Multiple tests were conducted with VoIP traffic only for the G.729 (8 kbps) codec over two SMs. From the test results below, 5 calls from each SM (10 total) were completed successfully. As a sixth call to each SM was added, a sharp increase in lost bytes and one-way delay contributed to unacceptable MOS scores. In a two SM environment, 10 – 11 calls can be supported successfully, but any additional calls will cause degradation in performance.

Table 15

Codec Downlink /

Uplink Percentage

Number of SMs


One way delay


avg. R-Value

Avg. Lost

Bytes

G.729 75/25 2 5 24 2,961,440 4.01 79.73 1,040

G.729 75/25 2 6 61 3,546,880 3.57 70.44 96,880

Canopy set at 75% downlink / 25% uplink – G.723.1-ACELP and G.723.1-MPMLQ Due to the poor performance of these codecs during the tests where Canopy was set at 50% downlink and 50% uplink, no tests were run in the 75% / 25% configuration. Since the uplink is the known constraint, lowering it would only lower the performance of the codec. The test results suggest that this codec does not transfer well over a VoIP network due to the low kbps. It is recommended to not use these codecs for any VoIP applications independent of the network being used.

Test Scenarios with Canopy Quality of Service (QoS) To test the robustness of the quality of service functionality a data file was passed through at the same time as the VoIP. The purpose was to see how effectively the AP could regulate the traffic and if it was at all affected by the amount of data that was being simultaneously passed with the VoIP traffic. The size of the data files were incrementally increased starting at 128 Kb up to 1.2 Mb. As shown in the table below, the quality of the call did not fluctuate noticeably with increased amount of data being passed. This is shown by noting the MOS values being at 4.36 when 128 kbps file was passed and 4.29 when a 1.2 Mbps file was passed. These results demonstrate that regardless of the amount of data, the AP will provide a level of guaranteed bandwidth for VoIP traffic.

Table 16

Codec Downlink/

Uplink Percentage

Testing Config.

(# of SMs)

Number of Calls from

SM

Data File Bandwidth

Size Bytes Sent MOS

Avg. R-Value

Avg. Lost

Bytes

G.711u 75/25 1 1 128 Kbps 8,800,000 4.36 90.95 480

G.711u 75/25 1 1 1.2 Mbps 62,400,000 4.29 88.89 2,400

Now that it has been shown that the high priority bit will serve the function of regulating the data traffic while voice traffic is present, the limitations needed to be tested. In the following test scenarios the number of high priority slots allocated for voice traffic was adjusted from 1 slot up to 3 slots. This was done to demonstrate whether there is a positive, negative or indifferent impact on overall performance and throughput as a result of adding more slots for high priority. Another variable that was adjusted was the bandwidth allocation on the downlink and uplink from 75/25 to 50/50. The number of “Acknowledgement” slots for downlink and uplink were not varied.


28

The different codecs were tested with one or two SMs as well as different call volumes to determine the breaking point in quality.

Test Results with Canopy QoS Testing on QoS was done using both the 75/25 and 50/50 downlink/uplink settings. In regards to QoS, the most important setting is how many high priority slots are allocated. This is a function of the downlink/uplink percentage and the high priority percentage that is set in the configuration of the AP. Different percentage configurations will yield a varied number of high priority slots depending on your Canopy environment and configurations. When applying these settings it is encouraged to adjust them accordingly such that the desired number of high priority slots is produced. Some example scenarios and their results from the testing are shown below.

High Priority = 1 Slot This test was done with 1 high priority slot set. To achieve one high priority slot, the downlink/uplink percentage was set to 75/25, the high priority percentage was set to 25%, and the G.711u codec was tested. The tests were run with one and two SMs. With one SM two calls were successfully completed, which is shown in the table below with a MOS score of 4.34. With two SMs, one call from each SM failed to complete at an acceptable quality level, which is shown in the table below with a MOS score of 2.69.

Downlink Data Percentage 75% UpLink Data Percentage 25%

Downlink Slots 21 Uplink Slots 7

High Priority Slots 1

Total Num UAck Slots 6 UAcks Reserved High 3

Num DAck Slots 6 DAcks Reserved High 3

Num Ctl Slots 6 Num Ctl Slots Reserved High 3

Table 17

Codec Downlink/

Uplink Percentage

Testing Config.

(# of SMs)

Number of Calls from SM

Data File Bandwidth

Size Bytes Sent MOS

Avg. R-Value

Avg. Lost Bytes

G.711u 75/25 1 2 22 4,800,000 4.34 90.43 2,080

G.711u 75/25 1 3 231 7,195,680 2.69 45.81 1,766,880

G.711u 75/25 2 1 355 4,800,000 2.69 45.81 132,800

High Priority = 2 Slots This test was done with two high priority slots set. To achieve two high priority slots, the downlink/uplink percentage was set to 75/25, the high priority percentage was set to 25% and the G.711u codec was tested. The tests were run with one and two SMs. With one SM two calls were successfully completed, which is shown in the below table with a MOS score of 4.30. With two SMs, two calls total (one from each SM) were completed at an acceptable quality level, which is shown in the table below with a MOS score of 4.33. Once the call volume was increased to two calls per SM, the high priority channel failed to handle it at an acceptable quality level.


29







Table 18

Codec Downlink/

Uplink Percentage

Testing Config.

(# of SMs)


One Way Delay Avg.

(ms) Bytes Sent MOS

Avg. R-Value

Avg. Lost Bytes

G.711u 75/25 1 2 30 4,800,000 4.30 89.32 2,880

G.711u 75/25 1 3 128 7,200,000 2.73 49.60 529,440

G.711u 75/25 2 1 21 4,800,000 4.33 90.14 2,400

G.711u 75/25 2 2 68 9,600,000 2.73 47.41 282,240 The G.711 codec has shown to be the most consistent and highest in quality in terms of overall performance. The above tests showed that the 75/25 bandwidth allocation did not allow a large amount of calls through the network while using the G.711 codec and some instances didn’t allow even one call through. It was determined that further testing on the other codecs would not produce more favorable results, so the testing was limited to just this codec.

High Priority = 3 Slots This test was with 3 high priority slots set. To achieve three high priority slots, the downlink/uplink percentage was set to 50/50, the high priority percentage was set to 25% and the G.711, G.726 and G.729 codecs were tested. The tests were run with one and two SMs. By allocating more bandwidth by increasing the uplink percentage, an extra high priority slot was made available. This allowed more calls to be successfully completed utilizing the high priority channels. However, keep in mind that these slot allocations are static and one data slot needs to be traded for every additional slot that is allocated for high priority VoIP traffic. To make sure that there is not too much idle bandwidth in the network, it is in the best interest of the customer to closely calculate how much bandwidth is necessary for high priority.








30

G.711 G.711a and G.711u performed almost identically, so the results of just G.711u have been listed below as a representation of both tests. With one SM, three calls were successfully completed with an average MOS score of 4.29. When another SM was added the total number of successful calls increased to four, (2 calls from each SM), with a MOS score of 4.29. Table 19

Codec Downlink/

Uplink Percentage

Testing Config.

(# of SMs)


One Way Delay Avg.

(ms) Bytes Sent MOS

Avg. R-Value

Avg. Lost Bytes

G.711 50/50 1 3 31 7,200,000 4.29 89.06 6,560 G.711 50/50 1 4 108 9,599,680 2.68 45.76 967,040

G.711 50/50 2 2 25 9,600,000 4.29 89.05 8,320

G.711 50/50 2 3 124 14,369,120 3.08 59.59 455,040

G.726 With one SM, three calls were successfully completed with an average MOS score of 4.12. When an additional SM was added, the total number of successful calls increased to six (3 calls from each SM) with a MOS score of 4.07.

Table 20

Codec Downlink/

Uplink Percentage

Testing Config.

(# of SMs)


One Way Delay Avg.

(ms) Bytes Sent MOS

Avg. R-Value

Avg. Lost Bytes

G.726 50/50 1 3 25 3,600,000 4.12 82.82 2,640

G.726 50/50 1 4 91 4,800,000 2.84 55.82 142,560

G.726 50/50 2 3 25 7,200,000 4.07 81.77 8,880

G.726 50/50 2 4 93 9,600,000 2.87 56.44 290,640

G.729 With one SM, four calls were successfully completed with an average MOS score of 4.02. When another SM was added the total number of successful calls increased to eight (4 calls from each SM) with an average MOS score of 4.02.

Table 21

Codec Downlink/

Uplink Percentage

Testing Config.

(# of SMs)


One Way Delay Avg.

(ms) Bytes Sent MOS

Avg. R-Value

Avg. Lost Bytes

G.729 50/50 1 4 22 1,200,000 4.02 79.90 80

G.729 50/50 1 5 77 1,500,000 3.39 66.52 63,900

G.729 50/50 2 4 22 2,400,000 4.02 79.82 640

G.729 50/50 2 5 78 2,999,920 3.38 66.21 128,460


31

��()�� #(�)'��

Acronym Meaning

AP Access Point Codec Compressor/Decompressor CPE Customer Premise Equipment DSP Digital Signal Processing ITU International Telecommunications Union kbps 1000 bits per second Kbps 1024 bits per second Mbps 1024 Kbps = 1,048,576 bits MOS Mean Opinion Score - To determine MOS, a number of listeners rate the

quality of test sentences read aloud over the communications circuit by male and female speakers. A listener gives each sentence a rating as follows: (1) bad; (2) poor; (3) fair; (4) good; (5) excellent. The MOS is the arithmetic mean of all the individual scores, and can range from 1 (worst) to 5 (best)

PSTN Public Switched Telephone Network QoS Quality of Service R-Value/Factor The E-model is a complex formula; the output of an E-model calculation is

a single score, called an “R factor,” derived from delays and equipment impairment factors. R factor values range from 100 (excellent) down to 0 (poor).

SM Subscriber Module TOS Type of Service VoIP Voice Over IP VPN Virtual Private Network WISP Wireless Internet Service Provider


32

��(�#�� 1. “Canopy Network Advanced Technical Training,” Canopy Advanced Technical Training

Course, Motorola. 2. D. De Vleeschauwer and J.Janseen, “Voice performance over packet based networks,”

Alcatel Networks Technology White Paper, 2002. 3. J. Walker, “A handbook for Successful VoIP Deployment: Network Testing, QoS and More,”

NetIQ Corporation, 2002. 4. K. Adams, K. Bhalla, “An Introduction to Internet Telephony (or Voice over IP),” Operational

Management Report, Gartner Group, November 25, 2003. 5. “VoIP Test Module for Chariot,” NetIQ Corporation, 2002. 6. Westbay Engineers Limited “What is an Erlang,” http://erlang.com/whatis.html. 7. Westbay Engineers Limited “Erlang to VoIP Bandwidth Calculator,”

http://erlang.com/calculator/eipb/.

Disclaimer: This whitepaper merely provides a starting point for planning and sizing hardware requirements for customers to deploy VoIP over Canopy. Because these tests were run in constrained environments, such as an isolated lab, they do not necessarily translate directly to deployable scenarios. Therefore, it is important to understand that while this whitepaper is meant to help customers prepare for a VoIP over Canopy roll out and capacity-planning effort, any data generated contained in this whitepaper is only meant for general sizing, benchmarking, or deployment recommendations. Results may not be representative and may vary. Accordingly, neither Motorola nor West Monroe Partners can guarantee actual results in a real world deployment. In addition to these benchmarking results and recommendations, customers should also consider, but not limit, evaluation to point-to-point mileage, line of sight, network capacity, and expected peak call volume time (Erlang tables) when planning a VoIP over Canopy deployment.

motorola canopy voip whitepaper v5-8-5-04 1.7kern.com › motorola ›...

Documents