ict-619555 rescue d4.4 version 1 - cordiserable aims to close the performance validation circle...
TRANSCRIPT
ICT-619555 RESCUE
D4.4 Version 1.0
Final report on practical assessment of the RESCUE architecture
Contractual Date of Delivery to the CEC: 30 September 2016
Actual Date of Delivery to the CEC: 31 October 2016
Editor Sebastian Sosnik
Author(s) Christian Schneider, Oleksii Skoblikov, Mario Lorenz,Nils Hollmach, Martin Kaske, Marek Natkaniec,Marek Sikora, Katarzyna Kosek-Szott, Szymon Szott,Jacek Wszołek, Janusz Gozdecki, Krzysztof Łoziak,Łukasz Prasnal, Sebastian Sosnik, Łukasz Trzeciakowski,Olayinka Adigun, Nikos Pavlatos, Hicham Khalife, JawadSeddar, Valtteri Tervo, Anton Paatelma, Jiguang He,Markku Jokinen
Participants AGH, TUIL, FQS, UBITECH, TCS, UOULU
Work package WP4 - Validation, Integration and Field Trials
Estimated person months 74
Security PU
Nature R
Version 1.0
Total number of pages 87
Abstract: This deliverable summarizes the practical assessment of the links-on-the-fly concept. To do so,a software and hardware integration based on GNU Radio and SDR devices has been performed. Inten-sive verification and validation within three different testing facilities provided a stable framework for thesubsequent assessment. The evaluation strategy comprises experiments under controllable and reproducibletest conditions considering the OTAinVEE approach followed by field trials in an indoor testbed emulatingthe public safety use case, as well as outdoor tests emulating the V2V scenario. Finally, the outcomes ofthe practical evaluations are compared and analyzed jointly. These results are also analyzed in the light ofprevious outcomes from other work packages within the RESCUE project.
Keyword list: validation, verification, SDR, channel sounding, channel modelling, OTAinVEE, demonstra-tion, V2V, VANET, public safety operations, lossy links, multi-hop communication, wireless networks
Disclaimer: -
RESCUE D4.4, v1.0
Executive Summary
D4.4 is the final report of WP4. It presents the software and hardware integration into the selected SDR platformas well as the functional validation and first results within the OTAinVEE test facility. Moreover, two real fieldexperimental trials have been planned and conducted for an indoor and outdoor scenario respectively. The deliv-erable aims to close the performance validation circle envisaged within the RESCUE project: increasing degree ofrealistic assumptions during the specific validation stages starting from WP1 over to WP2 and WP3 and finally toWP4. However, real field experiments are usually limited by the number of device deployments, capabilities of theselected software and hardware platform as well as limits in the available experimental time-frame. Therefore, thereported experiments cover the basic scenarios TS0 and TS1, consisting of two and three nodes, respectively.
The deliverable consists of four main parts. The first part gives a detailed technical background of the software andhardware framework used to incorporate the RESCUE architecture. For this reason, and in order to obtain stable,reproducible and trustable results, the open-source GNU Radio framework and USRPs from Ettus/National Instru-ments have been selected as a compromise between flexibility, performance, and costs. Standard building blocksfrom GNU Radio have been customized to provide a basic physical and MAC layout. Contributions from WP2and WP3 in terms of implemented software blocks were integrated, including the RESCUE coding algorithms(from WP2) and the network protocols (from WP3). Besides iterative bug fixing and code optimization, the chal-lenging tasks during the testbed verification were the frame synchronization, SNR estimation, and calibration ofthe testbeds between different WP4 partners.
The second part summarizes the verification and experiments based on the OTAinVEE (over-the-air in a virtualelectromagnetic environment) concept. Within RESCUE, this specific validation stage turned out to be very valu-able since the controllable and reproducible test conditions allowed for deep software and hardware verificationfollowed by intensive performance validation before the real field experiments were conducted.
The third part of the deliverable focuses on the real field experiments. Initially, trials were planned only for thepublic safety scenario with a focus on an indoor deployment. For the V2V use case, performance studies basedon the OTAinVEE framework had been programmed. However, following the recommendation from the RES-CUE reviewers suggesting to study the RESCUE architecture not only under synthesized traffic and propagationconditions but also to conduct real field experiments, the consortium deployed additional efforts for this purpose.Consequently, for both deployments the technical configuration and experimental test plan are detailed in this doc-ument. Whereby for the V2V setup one of the challenging parts was to integrate an efficient hence remote accessto each mobile node in such a way that the individual measurements could be configured and started while thevehicles were moving. Numerous experiments have been conducted in particular for TS0 PHY and MAC as wellas for TS1. However, other scenarios or more intense trials have been limited by the enormous time consumptionof each single test to provide enough datasets fulfilling statistical means.
The fourth part of D4.4 provides final assessments and conclusions. It aims to bridge the research and validationmethods used within the RESCUE project and their subsequent performance results. The considered stages ofperformance validation balance between realism and simplification while moving from theory to practice. Resultsfrom three different WPs: WP2, WP3 as well as WP4 are discussed and related among each other. From thevalidation method perspective it can be concluded that this project was able to research the capabilities of thelinks-on-the-fly concept with a very ambitious approach bridging the ineluctable gap between theory and practice.It was found that the gains of this new architecture are impacted by the implementation of more practical-orientedvalidation methods of increasing complexity. A detailed analysis identifying the potential impact of the validationmethodologies on the given results, e.g., the small number of nodes during the experiments or constraints by thesoftware and hardware implementation and integration of the test platform, are left for future study.
Page 2 (87)
RESCUE D4.4, v1.0
Authors
Partner Name E-mail / Phone
Technische Universtitat Christian Schneider [email protected] (TUIL) Mario Lorenz [email protected]
Oleksii Skoblikov [email protected] Hollmach [email protected] Kaske [email protected]
AGH University of Science Marek Natkaniec [email protected] Technology (AGH) Marek Sikora [email protected]
Katarzyna Kosek-Szott [email protected] Szott [email protected] Gozdecki [email protected] Wszołek [email protected] Łoziak [email protected]Łukasz Prasnal [email protected]
FQS Poland (FQS) Sebastian Sosnik [email protected]Łukasz Trzeciakowski [email protected]
UbiTech Limited Olayinka Adigun [email protected] Pavlatos [email protected]
Thales Communications Hicham Khalife [email protected] SAS (TCS) Jawad Seddar [email protected]
University of Oulu Valtteri Tervo [email protected](UOULU) Anton Paatelma [email protected]
Jiguang He [email protected] Jokinen [email protected]
Page 3 (87)
RESCUE D4.4, v1.0
Table of Contents
Executive Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
List of Acronyms and Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2. Integration of RESCUE Architecture into SDR Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2 Integration Roadmap and Work Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.3 RESCUE Transceiver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3.1 General Technical Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.3.2 Console Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.3.3 Console Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4 Major Issues and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.4.1 Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.4.1.1 “Blind” Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.4.1.2 Preamble Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4.2 Differential Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.4.3 CSMA/CA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.4.3.1 Carrier Sensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.4.3.2 MAC to CS delay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.4.3.3 Collision Avoidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.4.3.4 CSMA/CA Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.4.4 SNR Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.4.4.1 GNU Radio MPSK SNR estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.4.4.2 BER to SNR Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3. Verification within the OTAinVEE Testing Facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.2 Overview of OTAinVEE Testbed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.3 USRP Hardware Qualification and Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.4 Selected Scenarios and Test Cases for Performance Evaluation and Validation . . . . . . . . . . . . . . . . . . . . . . . . 303.5 Test Setup and Calibration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.5.1 AWGN .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.5.2 Rayleigh-Fading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.5.3 V2V-WIM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.6 Key Performance Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.7 OTAinVEE Validation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.7.1 TS0-PHY Tests (AWGN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.7.1.1 Test Description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.7.1.2 Effects of LO Offset Tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.7.1.3 Blind Synchronizer vs Preamble Correlator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.7.1.4 Further Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.7.1.5 Test Results - AWGN with Static Doppler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.7.1.6 Choice of Synchronizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.7.2 TS0-MAC Tests (AWGN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.7.2.1 Test Description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.7.2.2 Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Page 4 (87)
RESCUE D4.4, v1.0
3.7.3 TS1-MAC Tests (AWGN & Fading) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.7.3.1 Test Description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.7.3.2 Test Results - AWGN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.7.3.3 Test Results - Rayleigh Fading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.7.3.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4. Validation in Real Field Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.1 Description of RESCUE Experimental Validation Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454.2 Validation in Indoor Safety Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.2.1 Testbed Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454.2.2 TS0 PHY Experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484.2.3 TS0 MAC Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534.2.4 TS1 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.3 Validation in Outdoor V2V Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644.3.1 Requirements on V2V Testbed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 644.3.2 Description of V2V Testbed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654.3.3 V2V Test Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 694.3.4 Post-processing Experimental Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704.3.5 TS1 MAC Dynamic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5. Final Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
5.1 Comparison of Scenarios and Methodologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765.2 Performance Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
6. Summary & Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 826.2 Conclusion and Synopsis of the RESCUE Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
7. References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Page 5 (87)
RESCUE D4.4, v1.0
List of Acronyms and Abbreviations
16QAM Sixteen-Phase Quadrature Amplitude Modulation
ACK Acknowledgement
AGC Automatic Gain Control
ARQ Automatic Repeat Request
AWGN Additive White Gaussian Noise
BER Bit Error Rate
BLER Block Error Rate
BS Base Station
CDF Cumulative Distribution Function
CIR Channel-Impulse-Response
CMD Correlation-Matrix-Distance
CRC Cyclic Redundancy Check
CSMA/CA Carrier Sense Multiple Access/Collision Avoidance
DE-QPSK Differentially Encoded QPSK
DIFS Distributed Inter-Frame Space
DMC Dense Multipath Components
DoA Direction-of-Arrival
DoD Direction-of-Departure
DoW Description of Work
DuT Device under Test
FEC Forward Error Correction
FER Frame Error Rate
GBSCM Geometry based Stochastic Channel Model
GLSF Generalized-Local-Scattering-Function
GPS Global Positioning System
GPSDO GPS Disciplined Oscillator
HER Header Error Rate
ITS Intelligent Transportation Systems
Page 6 (87)
RESCUE D4.4, v1.0
JD Joint Decoding
KPI Key Performance Indicator
LLR Log-Likelihood Ratio
LF Lossy Forwarding
LoS Line-of-Sight
LOTF Links-on-the-fly
LSP Large-Scale-Parameter
MAC Medium-Access-Control
MaxSDR maximum-signal-to-remainder-ratio
MIMO Multiple-Input Multiple-Output
NLoS Non-Line-of-Sight
NTP Network Time Protocol
NUC Next Unit of Computing
OCXO Oven-Controlled Crystal Oscillator
OSI Open Systems Interconnection
OTA Over-The-Air
OTAinVEE Over-The-Air in Virtual Electromagnetic Environment
PAS Power-Angular-Spectrum
pdf probability density function
PDP Power-Delay-Profile
PDU Protocol Data Unit
PHY Physical Layer
PLL Phase Locked Loop
PMT Polymorphic Types
PPS Pulses Per Second
PRBS Pseudo Random Bit Stream
QPSK Quadrature Phase-Shift Keying
RCS Radar-Cross-Section
RRC Root-Raised Cosine
Page 7 (87)
RESCUE D4.4, v1.0
RF Radio Frequency
RX Receiver
SC Specular Propagation Paths
SCME Spatial Channel Model Extended
SDF Selective Decode and Forward
SDR Software Defined Radio
SIFS Short Inter-Frame Space
SIMO Single-Input Multiple-Output
SNR Signal-to-Noise Ratio
SPUCA Stacked Polarimetric Uniform Circular Array
SSH Secure SHell
ToA Time-of-Arrival
TX Transmitter
TS Toy Scenario
UHD USRP Hardware Driver
UPS Un-interruptable Power Supply
USRP Universal Software Radio Peripheral
VANET Vehicular Ad-hoc Networks
V2I Vehicle-To-Infrastructure
V2V Vehicle-To-Vehicle
WIM WINNER channel Model
Page 8 (87)
RESCUE D4.4, v1.0
1. Introduction
The RESCUE project – “Links-on-the-fly Technology for Robust, Efficient, and Smart Communication in Un-predictable Environments” – aims at designing a communication network that is robust of sudden topologicalchanges. In a wireless operator infrastructure, such changes are often caused by an accidental base station col-lapse, which affects the network coverage and link qualities. Deliverable D1.1 [1] defines two examples of suchcases: public safety operations that take place in areas where the communication infrastructure is partially inop-erable due to a disaster such as an earthquake, or vehicle-to-vehicle (V2V) communication, where cars and othervehicles share, for example, safety-critical information about the road and traffic conditions with each other.
The main technical concept of RESCUE is Lossy Forwarding, which is an innovative Distributed Turbo Codingsolution. Unlike in the conventional Decode-and-Forward relaying, with lossy forwarding a relay forwards amessage regardless whether errors have been detected after decoding. At the destination a joint decoding (JD)technique is applied, which exploits the high correlation of the messages received via different network paths.
In addition to developing the links-on-the-fly concept, one key objective of RESCUE is to validate the proposedsolutions via various means: information theoretic analyses, model-based numerical simulations, over-the-air(OTA) testing, and – for a proof-of-concept – field trials and demonstrations employing programmable software-defined-radio (SDR) platform devices. Indeed, in order to bridge the gap between information theory and concretenetworking protocols, the RESCUE project employs a methodology spanning from theoretical validation throughsimulations reaching real radio prototyping and experimentation. This ”from theory to practise” approach thatwas applied in the RESCUE project is summarized in Figure 1.1. In WP2 and WP3 the performance evaluationwas done mainly by software simulations with different modelling assumptions (w.r.t. the propagation channel)and using different software tools (i.e. MATLAB, ns3, GNU Radio). The simulation results provided realisticperformance figures to quantify the gains/losses of the novel technology and serve as reference for the practicalexperiments. To complement the previous steps, this deliverable summarizes WP4 – “Validation, Integration andField Trials” - by focusing on the RESCUE software-hardware integration and its practical verification in over-the-air tests.
WP1
• Information-theoretic analysis and evaluation
• Public safety as well as V2V key technologies and protocols
• Evaluation based on simple channel models as AWGN
WP2 & WP3
• Functional analysis and design of physical and protocol layer
• Consider specific requirements for both use cases
• Evaluation based on more complex models: Rayleigh, Rician and Nakagami fading partially combined with different path loss models as well as ETIS ITS G5 tapped delay line model
WP 4
• Evaluation as validation and verification based on practical experiments
• SDR testbed and Over-The-Air (OTA) facility
• Evaluation based on OTA with extended WINNER channel model and real field tests and demonstration
Incre
asing p
rop
agation
chan
ne
l influ
en
ce In
creasin
g reality an
d valid
ity
Figure 1.1: Strategy for RESCUE concept validation and evaluation
For our validation purposes, the RESCUE software was implemented within the GNU Radio framework andintegrated with the SDR devices called USRP (Universal Software Radio Peripheral). One key advantage ofusing GNU Radio platform, resides in the fact that the same source code is used for simulations and for over-the-air transmission with USRPs. Consequently, the same GNU Radio RESCUE software that was implemented,documented and tested in WP2/WP3, was then reused for radio experiments in WP4. The connection of RESCUE
Page 9 (87)
RESCUE D4.4, v1.0
software with the SDR hardware resulted in the creation of the RESCUE Transceiver and permitted the set up ofa multi node test platform.
The main goal of this document is to evaluate the performance of the RESCUE protocol suite first in a radiocontrolled environments and then investigate the usability of this concept in real-world conditions. To achieve thegoals, three types of testbed validation were done in the RESCUE project:
1. Over the air in virtual Electromagnetic Environment (OTAinVEE) facility tests targeting mainly on the vali-dation of the RESCUE Transceiver in realistic and more importantly reproducible and controllable wirelessenvironments.
2. Field trials for V2V use cases. Real field tests are very challenging. Therefore the V2V field trials wereperformed based on a small vehicle testbed to validate in a realistic manner RESCUE main contributions.
3. Indoor validation of the public safety use case. For practical reasons, the public safety use case was emulatedin indoor facilities, where the communication channels between the RESCUE Transceivers suffer fromattenuations caused by heavy walls or floors instead of large distances.
This document compiles a wide set of experimental results investigating a wide set of RESCUE features. Theseresults are highly credible since they are obtained with over the air radio experiments using the programmableand fully controllable GNU Radio framework. Thus, this document can be seen as the final outcome of theRESCUE project allowing to draw conclusions on the links-on-the-fly concept and giving a future perspective forthis technology.
The deliverable is structured as follows: Section 2 describes the hardware and software components of the RES-CUE Transceiver. Sections 3, 4 contain the description and results of the experiments done in the OTAinVEEfacility, V2V field trials, indoor validation testbed, while section 5 summarizes the achieved results. Section 6gives the overall conclusion of the RESCUE technology and its practical verification.
Page 10 (87)
RESCUE D4.4, v1.0
2. Integration of RESCUE Architecture into SDR Platform
2.1 Background
The activities for the integration of the RESCUE architecture onto the software defined radio platform encom-passes inputs from various WPs of the RESCUE project. Figure 2.1 shows the interrelation of various WPs withthe GNU Radio implementation.
GR RESCUE
WP2
• Coding algorithm
• RESCUE PHY implementation
WP3
• Frame format
• MAC operation
• ARQ, sliding window
• CSMA/CA
• Cross-layer design WP4 • PHY and MAC integration
• Tuning for OTA tests
• SDR based experiments
Figure 2.1: Work package contributions to the GNU Radio implementation of RESCUE (GR RESCUE)
In deliverable D2.1.1 [2] the design of the RESCUE coding algorithm was presented and tested with MATLABsimulations. Deliverable D2.3 [3] described the implementation and verification of this coding algorithm using theGNU Radio software framework. Additionally, the RESCUE PHY layer concept was proposed in D2.3 to enableboth:
• simulation of the RESCUE implementation using GNU Radio channel models and
• over-the-air execution of RESCUE on USRP hardware.
This provided a platform for the hardware implementation of the RESCUE concepts designed and developedwithin WP2, WP3, and WP4, which permitted to create a full radio proof of concept.
Within WP3, D3.1 [4] and D3.2 [5] presented the initial design of the Links-On-The-Fly (LOTF) architectureconsidering a cross-layer approach for all layers of the OSI reference model, with functional changes made to thelowest two layers (PHY and data link) in order to maintain compatibility with existing networks and leverage thegains from using PHY coding with lossy links. The network layer utilized RESCUE’s modified routing protocolsin such a way that would permit LOTF functionality. The transport and upper layers remained unchanged. Thedetailed design aspects of the different MAC, networking and ARQ protocols and rate adaptation with simulation-based performance evaluation and results were shown in these deliverables to quantify the gain of the PHY LOTFconcept at upper layers.
D3.3 [6] presented the practical understanding of the LOTF architecture and the prototype implementation ofthe message transfer protocols of WP3 within GNU Radio along with the validation of these protocols through
Page 11 (87)
RESCUE D4.4, v1.0
GNU Radio simulations. The simulator used in D3.3 extended the simulator used in D2.3 by adding data linkand network layer protocols on top of the PHY implementation described and validated within D2.3. The GNURadio implementation assured a higher accuracy of the PHY layer and coding algorithms compared to the ns3-simulation-based approach and the results shown in D3.3 represented an important step from theoretical analysisof the LOTF concept towards experimental verification with real devices tested under realistic conditions.
This deliverable (D4.4) now provides the details of the final integration activities gathering inputs from previouslycompleted deliverables of WP1-4 and provides a demonstration of the LOTF functionality.
2.2 Integration Roadmap and Work Flow
The software development for the RESCUE proof of concept was a cross-WP task, which can be divided into fourmain stages summarized in Table 2.1.
WP Task Date Specified in Described inWP3 RESCUE Data Link Aug – Dec 2014 D3.1 D3.3WP2 RESCUE PHY and Coding algorithms Jan – Oct 2015 D2.1.1 D2.3WP3 PHY and Data Link Layer integration Nov 2015 – Mar 2016 D2.3, D3.1, D3.2 D3.3WP4 Hardware integration Aug 2015 – Aug 2016 D2.3, D3.3, D4.1 D4.4
Table 2.1: Main software development tasks
The RESCUE implementation in GNU Radio started from the data link layer. This was a good choice since itprepared the framework for the development of the PHY layer and the coding algorithms, which are an essentialfeature of the RESCUE concept. These coding algorithms were validated in simulation using various channelmodels and the results were described in D2.3. The PHY and data link layer were thereafter integrated and testedin simulations. The results of this integration and validation (through simulation) were reported in D3.3.
In parallel to the PHY and data link integration, the first over-the-air tests were executed using USRPs. Thesepractical tests revealed some issues, mainly in the PHY layer, which were not detected during the RESCUE PHYsimulations. Tuning the PHY layer and making the GNU Radio RESCUE implementation work over-the-air wasthe subject of the last task (hardware integration) which is described in this document.
For each separate software development task, an existing project deliverable was used as the specification. Theoverall project plan given in the “Description of Work” (DOW) project document was followed and the softwareparts required for the final integration were delivered on time. The project schedule is illustrated in Figure 2.2.Filled bars indicate the software development phase, while empty bars the “software testing” phase, e.g., runningsimulations and experiments, validating results, fixing defects, writing the documentation and deliverables.
The project deliverables and milestones related to the RESCUE implementation and integration defined in theDOW document are listed in Table 2.2.
Page 12 (87)
RESCUE D4.4, v1.0
WPX
RESCUE PHY (WP2)
RESCUE MAC (WP3)
PHY/MAC Integration (WP4)
1 2 3 4 5 6 7 8 9 10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
D
D3.3
Aug 2014
D
D2.3
D
D4.4
M
MS6
M
MS4
M
MS5
Nov 2015 Apr 2016 Oct 2016
Figure 2.2: Software development timeline of RESCUE implementation in GNU Radio
Deliverable/Milestone Date DescriptionD2.3 Nov 2015 Feasibility Assessment in Model Based EnvironmentsD3.3 Apr 2016 Report on implementation of the WP3 architectureD4.4 Sep 2016 Final report on practical assessment of the RESCUE architectureMS4 Nov 2015 Sync point on Implementation and validationMS5 Jan 2016 Integration of adopted solution on SDR (WP4)MS6 Mar 2016 Final implementation of protocols and modules (WP2, WP3)
Table 2.2: Deliverables and milestones
2.3 RESCUE Transceiver
The transceiver is an important element to demonstrate the RESCUE concept and is described in this subsection.The integrated RESCUE Transceiver is a standalone GNU Radio application that can be executed on a singlenode, e.g., a PC-USRP set. Such a node can communicate with other RESCUE nodes by sending/receiving DATAand ACK frames using the RESCUE format and by acting as the transmitter (frame source), receiver (framedestination), or relay (frame forwarder). Running the RESCUE transceiver on several PC-USRP sets allows forpractical tests of the RESCUE concept in various network configurations and topologies.
The RESCUE Transceiver is a GNU Radio flow represented graphically in Figure 2.3. This transceiver flowconnects and integrates the three main software parts developed in the RESCUE project: the RESCUE data linklayer, the RESCUE PHY layer and the RESCUE coding algorithms, which were implemented as GNU Radioblocks and are briefly described in Table 2.3. A more precise description and validation of these blocks can befound in the project deliverables D2.3 and D3.3.
RESCUE MAC, RESCUE PHY, and the coding algorithms are independent GNU Radio blocks. They can be usedwithout each other, hence both layers and the coding algorithms were separately ”unit tested” before integration.The RESCUE MAC block has two ports from app and to app, which allow for the connection and the exchangeof messages (protocol data units) with higher layers.
Page 13 (87)
RESCUE D4.4, v1.0
Figure 2.3: RESCUE transceiver flow: connects the main RESCUE blocks with USRP Sink and Source
Block Name Described in Functionality ResponsibilityRESCUE MAC Complex D3.3 MAC operation:
- medium access- basic acknowledgements- frame handlingroutingend-to-end ARQ
Handle PHY layer PDU received onfrom radio port and pass to PDU tohigher layers on to app portHandle PDU from higher layers re-ceived on from app port and pass tothe PHY layer on to radio port
RESCUE Encoder D2.3 Coding algorithms Encode RESCUE PDU comingfrom the MAC layer and pass it tothe PHY layer
RESCUE Decoder D2.3 Coding algorithms Decode RESCUE PDU comingfrom the PHY layer and pass it tothe MAC layer
RESCUE PHY PSK Soft D2.3 Modulation
Demodulation
Convert asynchronous messages(PDUs) to complex modulated sig-nal at baseband for USRP SinkConvert the radio signal comingfrom USRP Source into RESCUEPHY PDUs (packets)
UHD USRP Source GNU Radio Radio Signal Reception Produce a stream of baseband sam-ples by sampling RF on a selectedUSRP antenna
UHD USRP Sink GNU Radio Radio Signal Transmission Transmit the stream of basebandsamples on the USRP antenna
Table 2.3: Building blocks of the RESCUE Transceiver GNU Radio flow
From a technical, software engineering point of view, the integrated RESCUE flow is based on the exchange ofasynchronous messages (PDUs) between GNU Radio blocks. The frame formats of RESCUE ACK and DATAframes were described in D3.2. In GNU Radio, the message passing interface heavily relies on Polymorphic Types(PMTs). To be compatible with GNU Radio message passing, each RESCUE frame sent by a block (publishedon the message output port) is previously converted to the PMT format. On the other hand, each incoming PMT
Page 14 (87)
RESCUE D4.4, v1.0
message (a message received on the message input port) is first converted back to the RESCUE frame format andthen processed by the block.
The standard GNU Radio blocks UHD USRP Sink/Source enable the execution of the RESCUE blocks in over-the-air radio experiments by connecting the RESCUE PHY input/output with the USRP Sink/Source in the integratedRESCUE flow. This way, the USRP Sink/Source blocks complete the hardware integration by connecting theRESCUE software implementation with the SDR device (USRP).
2.3.1 General Technical Features
The RESCUE Transceiver enables the evaluation of the RESCUE concept on USRPs in practical radio experi-ments. The technical features of these experiments were initially described in deliverable D4.1 [7]. Table 2.4 liststhe general features of the RESCUE software and hardware, which were used in RESCUE experiments.
Basic parametersDuplexing Method Half-DuplexChannel Access Method Packet Based
CSMA/CAUSRPDevice USRP N 210Daughterboard SBX 400-4400 MHz Rx/Tx (40 MHz)Number of Antennas 2, one for TX and one for RXCenter Frequency 2.53 GHz (2.48 GHz – 2.60 GHz)Bandwidth as small as possible, single carrier operationSample Rate 1 million samples per secondSamples per Symbol 2Roll off Filter Root raised cosine (RRC), alpha 0.35Clock Source External precision reference (GPS)OthersModulation DE-QPSKHeader size 35 bytesHeader Code Rate 1/6Payload Size Variable. By default 1000 bytes.Payload Code Rate 1/2
Table 2.4: Technical features
Page 15 (87)
RESCUE D4.4, v1.0
2.3.2 Console Applications
The RESCUE Transceiver was used as a base for creating console applications for the execution of specific exper-iments and tests. The console applications are listed and briefly described in Table 2.5
Console App Description PurposeRESCUE PHY TX
RESCUE PHY RX
Sends DATA frames with the given pay-load length and time-interval. It is the TXflow of RESCUE Transceiver without theRESCUE MAC block.Receives and decodes RESCUE framesand then computes the decoder error rate.It is the RX flow of RESCUE Transceiverwithout the RESCUE MAC block.
Test a single link:– compute FER/HER– determine link quality– find RESCUE area of operationCompare single and joint-decoderTune RESCUE PHY:– test synchronization– adjust PHY layer parameters
RESCUE MAC TX
RESCUE MAC RX
MAC TX is RESCUE Transceiver with anadditional frame generator. It sends DATAframes with the given payload length andcomputes various performance indicators.MAC RX is the RESCUE Transceiver. Itreceives and handles RESCUE frames asdescribed in the MAC Operation diagramin D3.2. It can act as both Destination orRelay node in a given scenario (topology)
Evaluate RESCUE on multipleSDRs (USRPs):- test various topologies and toyscenarios- compare baselines- obtain numerical results- compute KPIs: throughput, de-lay (round-trip time), retransmis-sion count, FER
Table 2.5: Console applications used in RESCUE experiments
2.3.3 Console Parameters
The RESCUE console applications offer several console parameters and boolean options for an easy setup andcustomization of the RESCUE experiments. These parameters listed in Table 2.6 can be divided into the followinggroups:
• USRP settings: RF settings and other technical details of the SDR
• “SNR” settings: There are three parameters that enable controlling the SNR of a link in OTA radio ex-periments: the amplitude in RESCUE PHY block, the TX gain in USRP Sink, and the RX Gain in USRPSource. Using them, the SNR can be changed relatively, e.g., increasing TX gain by 1 dB should increasethe SNR of a link by 1dB, increasing the amplitude by a factor of 2 should change the SNR by around 6 dBor 20× log10 2.
• RESCUE PHY experiment settings: These parameters control the execution of the experiment or the be-haviour of the PHY blocks. Boolean options enable/disable the essential RESCUE features (e.g., joint-decoding), which sets the operation mode to a specific baseline and allows to run experiments for thisselected baseline.
• RESCUE MAC experiment settings: These parameters control the execution of the experiment or the be-haviour of the MAC blocks. Boolean options enable/disable the essential RESCUE features (e.g., joint-decoder, lossy-forwarding), which sets the operation mode to a specific baseline and allows to run experi-ments for this selected baseline.
• Other technical settings: These pure technical parameters are required to configure and execute the experi-ment, e.g., the IP address of the USRP, the logging level, the maximal experiment time.
RESCUE PHY console applications do not include the RESCUE MAC experiment settings. On the other hand,MAC console applications include all the listed options.
Page 16 (87)
RESCUE D4.4, v1.0
Parameter Description DefaultValue
Values used in RESCUEexperiments
USRP Settings–samp-rate Sample rate 200K 1M–freq Frequency 2.53 GHz 2.48-2.60 GHz
2.53GHz was often used–lo-offset Local oscillator offset 2M 2M or 0–external-clock Use external clock No Yes
“SNR” settings–rx-gain Gain of USRP Source in dB 0 0-32
20 is a good choice–tx-gain Gain of USRP Sink in dB 0 0-32–ampl Amplitude 0.4 0.4-0.7
RESCUE PHY Experiment Settings–payload-size Payload size in bytes 1000 1000–strobe-interval Strobe interval in milliseconds 500 100-1000–frame-count Frame count 50 50-2000–duplicates-count How many duplicates of the same
frame are sent.0 0 in MAC tests
1,2,3,5 in PHY test–joint-decoder Enable joint decoder Off On and Off–correlate-preamble Use correlate preamble in RESCUE
PHYOff On and Off
RESCUE MAC Experiment Settings–mac-addr Local address for RESCUE MAC 1 1,2,3, ...–dst-addr Destination address 3 3–difs=DIFS DIFS 0.1 0.1–arq-timeout ARQ timeout 1.0 0.2-1.0–max-arq-attempts Max ARQ attempts 4 0-4–tx-window-size TX window size 1 1–rx-window-size RX window size 16 16–no-joint-decoder Do not use joint decoder Off On and Off–no-lossy-forwarding
Do not forward erroneous frames Off On and Off
Table 2.6: RESCUE PHY and MAC RX/TX application options
2.4 Major Issues and Problems
This sub-section discusses the issues and problems encountered during the various stages of integrating differentcomponents. Efforts and solutions to address these problems are also discussed.
2.4.1 Synchronization
At the beginning of the OTA radio experiments, it became obvious that synchronization is the most difficult partof the hardware integration. Despite the fact that RESCUE PHY was validated in simulations with GNU Ra-dio channel models, the synchronization parts required additional tuning and parameter adjustment when runningRESCUE TX/RX applications on USRPs. All the other blocks of the RESCUE Transceiver (modulation/demodu-lation, differential encoding/decoding, mapping/demappping, access code correlation, packetizing/depacketizing,coding algorithms, ARQ, MAC operation, etc.) proved to be “hardware independent”. After successful tuning andfixing the synchronization blocks, the other blocks worked as expected in radio experiments without any additionalactions or fixes.
The RESCUE design and implementation followed the packet-based channel access in the form of carrier sensemultiple access with collision avoidance (CSMA/CA) and the TDD Half-Duplex duplexing method. In this ap-proach, frames are received with interruptions thereby requiring reliable synchronization in the RESCUE PHYlayer. Two different types of “synchronizers” were implemented and tested in RESCUE experiments:
Page 17 (87)
RESCUE D4.4, v1.0
• ”blind” synchronization, which is the “default” GNU Radio solution and
• synchronization based on preamble correlation.
Both types of synchronizers are available in the RESCUE Transceiver and in the console applications. The blindsynchronizer is the default one. To enable the synchronizer with “preamble correlation”, the option –preamble-correlation should be specified.
2.4.1.1 “Blind” Synchronization
The goal of synchronization is to estimate the timing, frequency, and phase offset of the received signal. Ad-ditionally, automatic gain control (AGC) must be done. While AGC is not really “synchronization”, it followssimilar principles of an adaptive loop. The default synchronization flow is shown in Figure 2.4 and consists offour blocks:
• AGC – Automatic Gain Control
• FLL Band Edge – Frequency Lock Loop
• Polyphase Clock Sync – Time offset estimation
• Constellation Receiver. The Costas Loop is an integral part of this block.
Figure 2.4: RESCUE PHY RX flow with blind synchronization. A long preamble is required for propersynchronization.
The above blocks are so called “blind blocks”. They do not use any known “training sequence” to facilitatethe synchronization process as the synchronization parameters are estimated by “statistical analysis” of unknownreceived data. In over-the-air experiments it was observed that a long preamble is needed to make the blindsynchronizer work properly. Table 2.7 shows the results of an example experiment with two cable-connectedUSRPs. In this experiment, frames were sent on a single link under good conditions (high SNR), so that theexpected frame error rate was close to zero. Other main parameters set in this experiment were as follows:sample-rate of 400k, frequency of 2.53 GHz, payload size of 1000 bytes, air-time-per-frame of 90 ms, and intervalbetween frames of 300 ms.
The goal of these experiments was to find the proper value for preamble length for the “blind synchronizer” in theRESCUE PHY implementation with DE-QPSK. A preamble length of 256 bytes was chosen. This translates to2048 samples; in RESCUE PHY QSPK modulation (2 samples-per-symbol, 2 bits-per-symbol) 1 byte is equal to8 samples.
Page 18 (87)
RESCUE D4.4, v1.0
Preamble length Measure Time Frame Count Header Error Rate Frame Error Ratebytes minutes
8 10 1976 0.169 0.16932 10 1965 0.064 0.07364 15 2975 0.054 0.175
128 15 2960 0.001 0.012256 10 1934 0.001 0.001
Table 2.7: Preamble length tests.
More precise over-the-air tests of the “blind synchronization” led to the following conclusions:
• FLL Band-Edge block is not needed if external time source (reference) is used,
• a few parameters should be adjusted/corrected for better synchronization as shown in Table 2.8.
Parameter[Block]
RESCUE Value GNU Radio Value
Preamble length 256 bytes 2 bytesSamples per symbol 2 4Loop Bandwidth[Polyphase Clock Sync]
6.28/8000 6.28/100
Loop Bandwidth[Constellation Receiver]
6.28/1000 6.28/100
Maximum Rate Deviation[Polyphase Clock Sync]
0.085 1.5
Table 2.8: Parameters used in RESCUE PHY blocks of the blind synchronizer vs GNU Radio default values.
2.4.1.2 Preamble Correlation
In the “preamble correlation” method, synchronization is done using a known training signal sent in the preamble.To implement this in RESCUE, the blind GNU Radio flows of RESCUE PHY were extended by inserting twoblocks: Add Preamble block in TX flow and Packet Detector block in RX flow as shown in Figures 2.5 and 2.6.
Figure 2.5: Extension of TX Flow: Add Preamble block inserts a predefined preamble.
On the transmitter side, the Add Preamble block creates a pseudonoise preamble as a training signal with a lengthof 100 samples and inserts it before the beginning of the frame. On the receiver side, the Packet Detector block
Page 19 (87)
RESCUE D4.4, v1.0
Figure 2.6: Insertion in RX Flow: Packet Detector block has a matched filter for the known preamble.
has a matched filter for this known preamble. When a packet is received, the “centre of mass” in cross-correlationbetween received sequence and stored preamble is calculated. If the preamble was correlated, a time tag is insertedto the point where first symbol of preamble begins. The Polyphase clock sync block uses the inserted time tag andimmediately gets well timed samples for symbol decision. As a result of this, the preamble length (100 samples) ismuch shorter in comparison to the blind synchronizer (2048 samples). This part of the code was written using theGNU Radio “Correlate and Sync” example as a basis. The Packet detector blocks the data stream until a packetis detected and then forwards some predefined number of samples to the next block. The number of forwardedsamples depend on the frame type: ACK or DATA and payload size, and this allows to save some processingpower.
2.4.2 Differential Encoding
Most simulations in D2.3 and D3.3 were done using coherent (non-differential) modulation: QPSK and 16QAM.However, in the current GNU Radio version coherent modulation works properly for high SNR values only,which is not the RESCUE area of operation. In the low-SNR regime, it is not possible to run the coherent, singlecarrier modulations in over-the-air experiments using standard GNU Radio blocks because of phase-ambiguity.To overcome the phase-ambiguity problem, differentially encoded modulations are used in GNU Radio. From theGNU Radio tutorial:
Notice in our discussion above that nothing we did had any knowledge of the transmitted symbol-to-constellation mapping, which means we might have an ambiguity of 90 degrees in the constellation.Luckily, we avoided this problem by transmitting differential symbols.
Therefore, all radio experiments described in this document were done using differentially encoded QPSK (DE-QPSK).
The phase ambiguity problem was described in more detail in D2.3, which also includes simulation results forDE-QPSK. They serve as a reference for the results of the practical experiments presented in this document.
2.4.3 CSMA/CA
The CSMA/CA implementation in RESCUE was described in D3.3 in Chapter 3.7.2. However, it was validatedby GNU Radio simulation only. In those simulations collisions were avoided in a very simple way using the CS-MA/CA Emulator described in Chapter 3.8.2 of D3.3. Therefore, the real validation of the CSMA/CA algorithmand the carrier sensing method was done in over-the-air tests and is described below.
2.4.3.1 Carrier Sensing
The GNU Radio flow shown in Figure 2.7 demonstrates how carrier sensing was initially implemented and tested.The test flow contains two separate sub-flows:
• standard RESCUE transmit flow – in which frames are assembled and transmitted by the TX Antenna. This
Page 20 (87)
RESCUE D4.4, v1.0
flow ends with UHD USRP Sink block,
• “carrier-sense” flow – in which signal power is measured at the RX Antenna; this flow starts with the UHDUSRP Source block.
Since a USRP has two antennas, one for the receiver (RX) and one for the transmitter (TX), the carrier sense flowcan be executed on one USRP. Example results for a ”non-continuous” packet transmission are illustrated in theleft-bottom corner of Figure 2.7.
Figure 2.7: Carrier sensing using “signal power threshold” method: GNU Radio test flow with exampleresults. Signal power time-chart for 3 received frames is shown in the bottom-left corner.
2.4.3.2 MAC to CS delay
A critical point in the RESCUE GNU Radio implementation of CSMA/CA was timing. In particular, it is the“MAC-to-CS” delay, which is the time between the following events:
• MAC block sends the frame on the TX side. This is the moment when the frame is published on the outputport of RESCUE MAC block.
• Probe block detects its signal on the RX side. This is the moment when the RX signal power becomesgreater than the threshold values.
From Figure 2.7 it can be deduced that the overall MAC-to-CS delay is the sum of delays in GNU Radio TXflow: Encoder time, RESCUE PHY time, USRP Sink time, and the carrier sense delay at the RX side in theUSRP Source and Probe blocks. Not all of these delays could be measured in radio experiments. However, themeasured delays are given in Table 2.9. The main parameters of the experiment were as follows: sample-rate of1M, frequency of 2.53 GHz, payload size of 1000 bytes, air-time-per-frame of 20 ms, interval between frames of200 ms.
Page 21 (87)
RESCUE D4.4, v1.0
GNU Radio Block DelayEncoder 3msRESCUE PHY 1msUSRP Sink + USRP Source + Carrier Sense 6ms ± 2msTotal 10ms
Table 2.9: MAC-to-CS Delay for a frame with payload size of 1000 bytes estimated in practical experiments
It was found that the delay of USRP Sink and Source blocks is large: 6 ms. Thus, the USRP blocks have a largeimpact on overall MAC-to-CS delay. The delay of the USRP Sink and Source blocks is caused by the delaybetween two separate hardware devices: USRP and PC, which is a general limitation of software defined radiosystems.
The overall MAC-to-CS delay for a RESCUE frame with a payload size of 1000 bytes, measured in the abovedescribed experiments was approximately 10 ms. This value is very important for CSMA/CA parametrization.
2.4.3.3 Collision Avoidance
CSMA/CA does not guarantee that all collisions can be avoided. In fact in CSMA/CA collisions may still happenin rare cases. The MAC transmission is detected with a delay and the MAC-to-CS delay was described in theprevious subsection. This delay may cause two or more nodes to start sending frames at the same time, whichwill result in a collision. However, there is a couple of parameters to minimize collisions such as DIFS, SIFS, orrandom slot time.
The values for those parameters should be selected by taking into account the MAC-to-CS delay value, which was15 ms. The final parameter values are listed in Table 2.10. They were determined using the trial and error methodwhere a set of practical experiments were executed by varying parameter values in order to minimize the numberof collisions.
Parameter ValueDIFS 100msSIFS 10msslot time 10msContention Window 16
Table 2.10: CSMA/CA parameters used in USRP experiments
2.4.3.4 CSMA/CA Integration
CSMA/CA is a cross-layer mechanism connecting the PHY and MAC layers. Carrier sensing is done at the PHYlayer and then used in the MAC layer as part of its CSMA/CA algorithm. In the RESCUE project, carrier sensingon USRP was done in two ways:
• by measuring (probing) the received signal power and comparing it to a given threshold,
• by correlating the preamble or the access code of the RESCUE frame and checking the correlator state.
Each solution has its pros an cons, however the “correlator” solution was finally chosen for the CSMA/CA algo-rithm and implemented in the RESCUE Transceiver application used in practical tests. This turned out to be moresuitable, reliable, and robust for the RESCUE experiments than the method based on signal power threshold. Thesignal power threshold is difficult to determine for low SNRs and moreover it depends on USRP settings, i.e.,rx-gain.
Page 22 (87)
RESCUE D4.4, v1.0
2.4.4 SNR Estimation
SNR estimation is needed for the setup of practical experiments and for the analysis of experiment results. Twotypes of SNR estimators were used in RESCUE:
• Generic SNR estimators for PSK signals implemented in GNU Radio,
• BER to SNR converter.
2.4.4.1 GNU Radio MPSK SNR estimators
The standard MPSK SNR estimator from GNU Radio calculates the SNR from a sequence of IQ-samples using3 different algorithms: simple, m2m4 and svr. An example evaluation of the GNU Radio MPSK SNR estimatorsin over-the-air radio experiments is presented in Figure 2.10. The experiment was done in the indoor testbedon 2 USRPs separated by a wall. Because of this obstacle, higher values of TX-Gain (10-25 dB) were set on theUSRP (in USRP Sink block) to achieve the SNR range approximately from 0 to 15 dB. The other main experimentsetting were as follows: sample-rate of 1M, frequency of 2.53GHz, rx-gain of 18 dB, amplitude of 0.7. It has tobe noted that the chart shows the average SNR value. Each point was calculated as the average of 6000 SNRmeasurements; 3 separate tests lasting 30 seconds were performed and during each test 2000 SNR measurementswere obtained.
It was found in over-the-air tests that the SNRs returned by different algorithms may differ by several dBs. Thismeans that the estimation is not very precise and that the estimated SNR value should be used with caution. Thesimple algorithm worked well for SNR greater than 10 dB, but overestimated the SNR for values less than 10 dB.In the RESCUE area of operation, for DE-QPSK it is the SNR interval [3dB, 10dB], the m2m4 and svr algorithmsgive best results. Moreover the results of these algorithms (m2m4 and svr) agree with the BER to SNR converter,which is described and validated in the next chapter.
10 12 14 16 18 20 22 24 260
5
10
15
TX Gain set on USRP [dB]
Est
imat
edSN
R[d
B]
simplem2m4
svrBER SNR Estimator
Figure 2.8: Comparison of SNR estimation algorithms
2.4.4.2 BER to SNR Conversion
Assume SNR is known. It can be expressed as Eb/N0 (energy per bit to noise power spectral density ratio). Thenthe theoretical (expected) BER can be calculated for that SNR. The formula for DE-QPSK is:
BER(Eb/N0) = er f c(√
Eb/N0)(1− 12 er f c(
√Eb/N0)) [8]
The BER SNR estimator inverses this logic in order to estimate SNR. Having the BER value measured during
Page 23 (87)
RESCUE D4.4, v1.0
experiment, it uses a binary search algorithm (also called the bisection method) to find the Eb/N0 for which theformula returns a given BER. Finally Eb/N0 is converted to SNR for the given modulation (QPSK).
0 2 4 6 8 100
2
4
6
8
10
12
SNR set in OTAinVEE [dB]
Est
imat
edSN
R[d
B]
HeaderPayload
0 2 4 6 8 100
0.2
0.4
0.6
0.8
1
SNR set in OTAinVEE [dB]
Est
imat
ion
Err
or[d
B]
HeaderPayload
Figure 2.9: Accuracy of BER SNR Estimator in OTAinVEE experiments
The BER SNR estimation method was tested and positively validated in cable-connected experiments within theOTAinVEE Testing Facility, see Figure 2.9. The results are taken from a single link (TS0) experiment where 360frames with payload of 1000 bytes were sent in 6 repetitions. The other experiment parameters were as follows:sample-rate of 1M, frequency of 2.49 GHz, rx-gain of 20 dB, amplitude of 0.4, interval between frames of 300ms.
Since the BER to SNR conversion proved to be a reliable method for SNR estimation in cable-connected exper-iments, it was decided to rely on this method also in OTA radio experiments. The BER for each received frameand the average BER per each link are written to the log files by both RX console applications: RESCUE PHYRX and RESCUE MAC RX.
An example evaluation of the BER SNR estimator in over-the-air radio experiments is presented in Figure 2.10.The experiment was done on 2 USRPs in the indoor testbed with the following main settings: sample-rate of1M, frequency of 2.53GHz, rx-gain of 18 dB, amplitude of 0.7, interval between frames of 200 ms. In the resultchart it can be observed that the estimated SNR grows linearly with the TX-Gain set on the USRP (in USRP Sinkblock)
8 10 12 14 16 180
2
4
6
8
10
TX Gain set on USRP [dB]
Est
imat
edSN
R[d
B]
y(x)0.87 · x−6.09
Figure 2.10: Evaluation of BER SNR Estimator in OTA experiments
Page 24 (87)
RESCUE D4.4, v1.0
3. Verification within the OTAinVEE Testing Facility
3.1 Motivation
In the development cycle, depicted in figure 3.1, once basic functionality has been established, the solution-under-test must prove itself under increasingly more realistic conditions, making these tests increasingly difficult.
Simulations
Lab Tests
(conducted,
OTA
Field Trials
- Easy
- Automated
- Little Effort
- Theoretical
- Planable
- Reproducible
- Moderate Effort
- Reasonable Results
Hardware-in-the-
Loop)
- Expensive
- Not Planable
- Not Reproducible
- True Result
Figure 3.1: Development of a radio system
The earlier tests in RESCUE, consisting mainly of simulations, the challenges faced were mainly those of anidealized, theoretical system with explicit or implicit assumptions about impairments (or the lack thereof). Forexample, when analyzing the performance of channel coding, not much regard is paid to the modulation - it isabstracted away into a set of Log-Likelihood Ratio (LLR) values for the received, encoded bits. If, for example,coherent modulation is assumed, then this assumption implies perfect carrier recovery, both in frequency and inphase. A similar assumption is that of perfect symbol- and frame timing synchronization.
Within WP4, the RESCUE radio SDR implementation as implemented in the previous work packages was com-prehensively tested, running on real hardware (Ettus Research USRP), with the radio channel still being syntheticby means of the TU Ilmenau / Fraunhofer IIS FORTE OTAinVEE test facility.
Transmitter- and Receiver-Hardware is typically assumed to be flawless and would need to operate flawlessly forthe results obtained to match earlier simulations. Unfortunately, all practically existing hardware has flaws, eitherdue to operating on the limits of available technology, or due to specific hardware design decisions, balancingless-than-perfect hardware behaviour in one parameter against that of a different parameter, or against economicalpressure (Time-To-Market, Unit Cost).
For example, one such parameter is the frequency accuracy and stability. A nominal value may be reached byusing high-quality oscillators (for example, an Oven-controlled crystal oscillator). This is not only costly, butultimately unnecessary given that in the RESCUE mobile scenario, the received frequencies will be shifted e.g.
Page 25 (87)
RESCUE D4.4, v1.0
by Doppler shift anyway. The RESCUE system must therefore continue to work under such impairments.
The main issue however is the absolute real-time requirement. In a simulation, the simulated time can be scaledin any way as required. An arbitrarily complex computation can be made to appear instantly in simulated time,but this is rarely possible in real time. A corollary of this is however also that simulations can not be speed upby simulating faster-than-real-time, which does have significant effect on the time requirements for the tests usingour testbed.
In this chapter, the OTAinVEE facility, its application to the RESCUE project, the performed tests with theirscenarios and the attained results will be described in detail.
3.2 Overview of OTAinVEE Testbed
The Fraunhofer IIS / TU Ilmenau FORTE (Facility for Over-The-Air Research and Testing) comprises of anadvanced laboratory environment for both satellite and terrestrial radio communications. The terrestrial part is afacility whose primary purpose is to test MIMO radio systems using a black-box hardware-in-the-loop approach.To do that, the radio environment of the Device under Test (DuT) is re-created using Wave Field Synthesis (WFS).WFS is the method of recreating all relevant impinging electromagnetic waves, be it from the line-of-sight pathor from reflections, in a spatially exact manner [9], [10]. If so required, in addition, the same can be done forinterferers or from reflections of the interferers. Ultimately, this recreates a pre-defined virtual electromagneticscenario in a completely controllable and reproducible form.
Figure 3.2: Emulation of real-world scenario inside a laboratory environment
In practice, this is realized by a set of illumination antennas inside an anechoic chamber, which are driven byindividual signals such that proper plane waves are created using Huygens’ Principle [11]. The principal schematicfor such a test setup is depicted in 3.3.
The central piece of the setup is a distributed 12x In / 32x Out Channel emulator capable of full connectivity, i.e.applying an individual time-variant impulse response to each input/output pair. The bandwidth of this system,inline with the requirements of modern cellular radio systems, is up to 80 MHz. The system is distributed over6 dual-channel A/D converters, 16 processing units (FDSP) and 16 dual-channel signal generators. A photo isshown in figure 3.4. The Spirent GSS8000 shown in the lower right corner emulates GNSS signal sources andwas not used within this project.
In order to test RESCUE, it was planned to emulate up to 7 nodes using this setup, one of which via WFS, theremaining 6 via cable connections, as shown in figure 3.5.
Significant time to run RESCUE experiments on this setup was booked (and used) in Q2/2016. Unfortunately,several major issues with the RESCUE software implementation remained to be found at that time. It soon becameapparent that we would need more time to conduct those experiments, extending to the full Q3/2016. Unfortu-nately, the OTA setup was already booked out for other projects. However, we were able to secure equipment toset up a limited “Mini-OTA” - Setup, consisting of 4 input channels and 4 output channels and their respectiveprocessing units.
Page 26 (87)
RESCUE D4.4, v1.0
Figure 3.3: Schematic of a traditional OTA setup
This test setup, shown in figure 3.6, was made available exclusively for RESCUE use and enabled tests up andincluding RESCUE TS1, consisting of 3 nodes. The 4th channel was required for noise/interference input.
The main limitations of this test setup as applicable to the RESCUE demonstrator implementation are relatedto the imperfections of the RESCUE Transceiver. The RESCUE Transceiver is particularly constrained by theperformance of the (joint) Turbo Decoder and the analog (GNU Radio) processing blocks.
This resulted in a maximum sample rate of 1 MSps, and a modulation symbol rate of 500 kSps. This is only atiny fraction of the bandwidth available from the Test setup, and also only a tiny fraction of the bandwidth as wasoriginally planned and designed within RESCUE. The impact here is twofold:
1. The data rate is severely limited. With this low data rate, the amount of data that must be tested to attainstatistical significance for the result takes a very long time. Many tests (for different scenarios had to be run,so the test time was limited to approx. 30 minutes per test, yielding statistical significance for Frame ErrorRate of only about 10−2. Nevertheless, the full run for each test ran over multiple days each.
2. The channel emulator, prepared for wideband channels, only had a limited frequency resolution for the taps(in frequency domain). Therefore, it was not possible to run a full “scaled-down” channel to “simulate” thefull mobile channel. Instead, with only a limited number of taps available, a much simplified channel modelhad to be used during the tests.
While in general, the measurements should still support a general performance evaluation of RESCUE, theseshortcomings will negatively influence the results, limiting comparability to the simulations done as part of WP2and WP3. The Vehicle-To-Vehicle (V2V) experiments will suffer from similar issues.
3.3 USRP Hardware Qualification and Calibration
It was decided to use the USRP series software defined radios made by Ettus Research to implement the RESCUETransceiver. These devices are reasonably affordable and well established in the research community for SDRdemonstration work (Google Scholar shows ¿ 8000 hits [12]). With the various project partners, several of thesedevices were available from previous projects, others have been procured as part of this project.
Ettus USRP series devices are offered with different baseband capabilities in terms of available sample rate, avail-
Page 27 (87)
RESCUE D4.4, v1.0
Figure 3.4: Photo of the OTA rack. Visible are the S1000 signal generators, FDSP processing units andR4000 AD converters (only 4 were installed at the time)
Node_1
Node_2
Node_3
Node_4
Node_5
Node_6
Node_7
FDSP
FDSP
FDSP
FDSP
30 dBTX
30 dB
30 dB
30 dB
30 dB
30 dB
TX
TX
TX
TX
TX
30 dBTX
30 dB
30 dB
30 dB
30 dB
30 dB
FDSP
8x
30 dB
R4000_1
R4000_2
R4000_3
R4000_4
S1000_1
S1000_2
S1000_3
S1000_4
S1000_11
Figure 3.5: Schematic of the OTA setup, as planned for RESCUE tests
able FPGA processing features and hardware interface to the host computer. Noteworthy are the USRP1, an USBbased device, and the USRP2, which featured Gigabit Ethernet as interface to its host. The successor of the USRP2are the USRP N2xx series, and later the USRP X series. Recently, Ettus Research was acquired by National Instru-ments, and some of these devices are sold under the USRP RIO brand. For RESCUE purposes, all these devices,from their specifications, would be equally suitable. Since the host CPU is the main performance limiting factor,the sample rate is capped at 1 MSps, which would be even attainable by the USRP1 (or its successor, the USRPBxxx series).
Lab setups early in the course of WP4 have shown that the available USRP2 and USRP N2xx as well as theUSRP X310 were suitable, and could be used interchangeably due to the unified software interface Ettus Researchprovides (“UHD”). However it became apparent that the quality of the reference frequency oscillators variedunexpectedly, either due to the quality or due to the aging of the crystals used. At the test frequency of 2490 MHz,the N2xx devices varied by around 2–3 kHz (an accuracy of roughly 1 · 10−6, worthy of a crystal oscillator).Actually, in a practical mobile system, this would be of no concern since the Doppler shift causes a frequencyuncertainty in about the same order of magnitude. The USRP2 units we had resulted in a significantly higherfrequency error of 20 kHz to 30 kHz (or an error of 1 ·10−5). This is a significant fraction of the carrier bandwidthand way above what could be expected due to Doppler shift.
Page 28 (87)
RESCUE D4.4, v1.0
Figure 3.6: ”Mini-OTA” setup for RESCUE tests
It was thus judged that the frequency reference should be made exact (with regards to Doppler shift) by providingexternal clock synchronization either to a common reference, or at least to a high quality reference, which wasthus utilized in all tests reported herein.
In order to form a complete radio, these devices need to be augmented by a “daughter board” as RF frontend.Several such RF frontends are available from Ettus, with different properties and frequency. The XCVR2450daughterboards, for example, are based on a WIFI chipset and provide mainly coverage of the 2.4 GHz ISM band,but is limited to Half-Duplex operation. The SBX daughterboard is a more general coverage frontend, working inthe frequency range between 400 MHz through 4000 GHz. Most project partners opted for SBX boards. DuringWP4, the available units were checked and calibrated against each other.
Figure 3.7: SBX receive spectra. Same signal, fed into an SBX revision 2 (blue), and SBX revision 3 (red,green). The green signal is amplified 20 dB to verify the spectrum shape
In figure 3.7, the results of one of these comparisons is shown. Depicted is the spectrum of a pure carrier signal,generated by a Rohde & Schwarz SMU200, as received with a SBX Revision 2 (blue), compared to a SBXRevision 3 (red), and a SBX Revision 3, with an additional 20 dB amplifier for reference purposes. It can be clearlyseen that the noise properties differ between SBX revisions. The rev3 variant shows better overall sensitivity (lowernoise floor), but at the center the typical phase noise spectrum of a Phase Locked Loop (PLL) is clearly visible,when in the revision 2, it is hidden by wideband white noise.
While probably not significant for the tests, the carrier signal was pure when viewed on a Rohde&Schwarz FSQ-8Spectrum analyzer, proving that the phase noise sidebands indeed were caused by the USRP.
In order to guarantee uniform reproducibility of the experiment results between the experiment, it has been decidedthat for all experiments, USRP N210, and SBX Revision 3 daughterboards should be used. Although it limits thenumber of available devices and precludes tests with more than 3 nodes.
A final note on the usage of USRPs for RESCUE: It became apparent during the experiments that the shielding of
Page 29 (87)
RESCUE D4.4, v1.0
the units is not perfect, leaving the units somewhat susceptible to mutual RF interference when units are placedclosely together. During the RESCUE OTA experiments, this issue was worked around by placing the units in adistance of approx. 2 m, and keeping the received signal level (and the received noise) at comparably high level.
3.4 Selected Scenarios and Test Cases for Performance Evaluation and Validation
In line with the scenarios as defined throughout the RESCUE project, within the Over-The-Air (OTA) verificationphase, for purposes of the Over-The-Air in Virtual Electromagnetic Environment (OTAinVEE) tests, we used aTS0 (single point-to-point) link, and a TS1 link (single relay case).
The tests were structured in two classes, PHY tests and MAC tests. A significant amount of time was spent with thePHY tests in order to get the modulator/demodulator/decoder working with acceptable performance, reproducibleat all project partners.
Many issues have been found (and subsequently mitigated) at the various stages of tests. Since a review of theimpact of those tests often found the need to redo prior tests, together with the throughput limitation discussed insection 3.2, the experiments were ultimately limited by the available time where both test equipment and devicesunder test were available.
The test conditions were chosen to subsequently exercise the various areas of the software, in particular to assessusable Signal-to-Noise Ratio (SNR) range, tunable parameters and implementation alternatives. The tests startwith the simple point-to-point (TS0) link in Additive White Gaussian Noise (AWGN). The channel was thenfurther impaired with a frequency offset (static Doppler shift) in order to test the frequency synchronization.
Physical Layer (PHY) functionality was deemed satisfactory in august and was subsequently frozen.
MAC tests were then run for TS0 to establish baseline performance, and subsequently we moved on to the TS1scenario.
The TS1 scenario was set up with the 3 relay locations A, B and C as in [3] with Source at position (0,0),Destination at (1,0) and Relay at (0.25,0.7), (0.75,0.7) and (0.5;0.866) respectively, as shown in figures 3.8through 3.10.
S
R
D
Figure 3.8: Position A, Relay at(0.25,0.7)
S
R
D
Figure 3.9: Position B, Relay at(0.75,0.7)
S
R
D
Figure 3.10: Position C, Relayat (0.5,0.866)
In many cases, multiple tests have been performed, testing the various relaying strategies enabling or disablingvarious parts of the RESCUE system. The following nomenclature has been used for those tests:
Baseline is a traditional selective Decode & Forward Relay system (only correctly decoded packages are for-warded). The joint decoder is used neither at the relay nor at the destination.
RESCUE-SDF-CRC is the selective Decode & Forward system, but with the joint decoder enabled at the desti-nation.
RESCUE-JD is an extension of RESCUE-SDF-CRC (only error-free packets are forwarded), where the joint
Page 30 (87)
RESCUE D4.4, v1.0
decoder is also enabled at the Relay, enabling improved decoding of the frame before relaying
RESCUE-LF The RESCUE Lossy Forwarding strategy, where packets containing errors are forwarded, providedthe BER is below a given threshold. At the relay, no joint decoder is being used. The BER threshold forlossy forwarding was set at 0.05.
RESCUE-LF-JD Same as RESCUE-LF, but with the Joint Decoder also active at the relay.
RESCUE-LF2-JD Additional test performed. Same as RESCUE-LF-JD, but with BER threshold for lossy for-warding set to 0.2
Initially, a static AWGN channel was used, with the Source/Destination SNR as base reference, and the Source/Re-lay and Relay/Destination SNR computed accordingly to the geometric channel model. A path loss exponent of3.52 was used.1
Since static Doppler compensation capability was proven during the PHY experiments, for lack of time no staticDoppler offset tests were conducted for the TS1-MAC-AWGN case.
Since RESCUE performance is expected to improve in a fading channel, a simple Rayleigh fading channel wasconstructed. As no channel estimation and equalization was implemented in the RESCUE software stack, a singletap Rayleigh fading channel was selected. While not quite as complex as the ITS G5 channel or even the channelmodel developed within RESCUE, this channel provides fading while still being relatively easy to demodulate.
To make the block fading assumption (which was inherent in the WP1 analysis) hold, the Doppler spread hadto be limited to 0.6 Hz. Compared to what is to be expected in practical systems, this value is extremely small,but without more effort in implementing channel estimation and equalization, this restriction was necessary. As aresult, V2V tests are expected to yield significantly worse results.
3.5 Test Setup and Calibration
In order to verify the correct operation of the OTAinVEE setup, the OTAinVEE setup was verified and calibratedagainst a Rohde & Schwarz SMU200A signal generator and a Rohde & Schwarz FSQ-8 Spectrum analyzer /Vector Signal Analyzer.
As with all USRP based experiments conducted within the RESCUE project, a carrier frequency in the 2.5 GHzband was used.
The main verification parameters were SNR and applied (static) Doppler shift, in the AWGN channel. TheSMU200A allows the generation of a Pseudo Random Bit Stream (PRBS) sequence modulated continuous QPSKsignal. Symbol rate and Root-Raised Cosine (RRC) excess bandwidth were set to the RESCUE specification(Table 2.4.
The FSQ-8 then allowed the analysis of this modulated signal (again, modulation parameters were set according toRESCUE specifications). The FSQ-8, in addition to displaying the frequency spectrum and measuring the powerlevels, can measure SNR/MER. Displayed accuracy was around 1 dB.
These measurements were used to calibrate/verify USRP output powers, and OTAinVEE attenuations (Path Loss)as well as the power of the added noise (AWGN).
In order to mitigate previously observed interference due to insufficient shielding of the USRPs, where the USRPwould receive the transmitted signal of other USRPs directly instead of only via the emulated channel, it wasdecided to run a rather high receive signal level (roughly -40 dBm) and inject the white noise (in the requiredratio), instead of going for the low-power approach and using the device-inherent white noise floor as reference.The high power signal+noise receive signal would always be significantly stronger than (and thus dominate over)the stray RF signals.
1Path loss exponents used in RESCUE range from 3 in [4] to 4 in [2]. For D4.4, we have chosen 3.52 as this is the value also has been usedin [2] and represents the middle ground.
Page 31 (87)
RESCUE D4.4, v1.0
Sanity checks were conducted from time to time during the several weeks of experiment time, to ensure theperformance of the OTA system continued to be as expected. No anomalies were encountered.
A selection of channel models was considered for testing, which are outlined in the following. The models wereAWGN-with-static-Doppler,Rayleigh-Fading-Single-Tap and V2V-WIM-Single-Tap. Please note that due to insuf-ficient time the V2V-WIM model could not be tested but it is still mentioned here for the sake of completeness.
3.5.1 AWGN
The AWGN channel was realized using the FORTE OTA equipment in conducted mode, adding the signal on oneport, and white Gaussian noise generated using a Rohde & Schwarz SMU200 on a second port. This setup allowscreation of a well-defined SNR above any thermal (or other) noise generated by the OTA setup or the receptiondevice by setting the input levels of the signal sources versus the noise source. When required, static Doppler wasalso applied by a tap rotating with j2π fdtu in the complex plane, as applied by the channel emulator hardware andwhere fd is the Doppler frequency and tu the update time between channel realizations at the channel emulator.
3.5.2 Rayleigh-Fading
Since theoretical analysis [1], [13] suggested a clearer difference between Lossy and Non-Lossy Forwarding in afading channel, a fading channel was provided. Since the RESCUE demonstrator as implemented during WP2 andWP3 did not include channel estimation and a channel equalizer, and moreover, due to the limited sample rate, thebandwidth of the demonstrator system could still considered more a narrowband system, only a simplified channelwas considered. This simplified channel consisted of a Tap-Delay-Line Model with a single Rayleigh Fading tap.Rayleigh fading was generated using Jake’s (Sum-Of-Sinusoids) model [14].
16 Sines were used, and the Doppler spread limited to 0.6 Hz. The latter restriction is rather severe, but was chosenso that the block fading assumption held.
This tap-delay line model was applied using real-time convolution using OTA equipment as the channel emula-tor.
3.5.3 V2V-WIM
The V2V-WIM channel model is an extension of the popular WINNER channel model (WIM) that was partiallydeveloped in WP4 of the RESCUE project. Its goal is to provide a spatial consistent geometry based stochasticchannel model that is suitable for V2V applications. Due to lack of time in the RESCUE project it was not possibleto conduct comprehensive OTAinVEE-tests with the V2V-WIM model, however, its potential application will bedescribed in the following.
The basic concept of V2V-WIM is outlined in [15]. Please note that the integration of mobile scatterers as describedin [15] is not yet implemented. In essence the model uses the traditional WIM to obtain channel realisations atdistinct anchor nodes, the radio channel at a specific location of a transmitter-receiver-pair is then determinedusing spatial interpolation with the aforementioned anchor nodes.
The implementation requires the user to specify a certain trajectory of transmitter and receiver and a set of controlparameters for the simulation. The code itself is designed to work on a single-location basis, i.e. an out loopis required to iterate over all points of a given trajectory. The following Matlab-code illustrates how to setup atrajectory. The spatial sampling of the trajectory has to be less than half of a wavelength to guarantee Doppler-conform sampling (the largest possible sampling depends on speed and direction of transmitter/receiver and angle-of-arrival/-departure of each ray, however, half-wavelength sampling guarantees correct Doppler treatment).
d e l t a t =0.25; % time sampling in secondss =0:2.99792458 e8 /2 .53 e+9/2:10; % lambda/2 spac ing at 2 .53GHzSrc . Pos=[ s /2 ; repmat ( 0 , [ 1 numel ( s ) ] ) ] ; % Src along x , ha l f−speed o f DstSrc . Ve loc i ty = [ [ 0 ; 0 ] d i f f ( Src . Pos , 1 , 2 ) / d e l t a t ] ;Src . Height = 1 . 5 ;Src . Pos = [ Src . Pos ; repmat ( Src . Height , 1 , s i z e ( Src . Pos , 2 ) ) ] ;
Page 32 (87)
RESCUE D4.4, v1.0
Src . Rot = [ 0 ; 0 ; 0 ] ;Src . d e l t a t=d e l t a t ;Dst . Pos=[ s+350; repmat ( 50 , [ 1 numel ( s ) ] ) ] ; % 350m f r on t o f Src , 50m sh i f t e d
in yDst . Ve loc i ty = [ [ 0 ; 0 ] d i f f ( Dst . Pos , 1 , 2 ) / d e l t a t ] ;Dst . Height = 1 . 5 ;Dst . Pos = [ Dst . Pos ; repmat (Dst . Height , 1 , s i z e (Dst . Pos , 2 ) ) ] ;Dst . Rot = [ 0 ; 0 ; 0 ] ;Dst . d e l t a t=d e l t a t ;
The control structure wimpar is the same as in WIM with some extensions. Those extensions are mainly thesampling time and the propagation scenario (LoS/NLoS which can be arbitrarily changed within a trajectory).Some other extensions were necessary to control certain modifications to the WIM thus making it possible to usethe WIM code within V2V-WIM as it could be used traditionally. All the parameters preceded by V2V are specificto the V2V-WIM implementation.
%% Model and s imu la t i on s e t t i n g swimpar=wimparset ;wimpar . NumTimeSamples=1;wimpar . PathLossModelUsed = 'yes ' ;wimpar . ShadowingModelUsed = 'yes ' ;wimpar . IntraClusterDsUsed = 'no' ;wimpar . CenterFrequency=2.53 e+9;wimpar . V2VScenario = 16 ;wimpar . V2VPropagCondition=0; %0 = NLoS , 1=LoSwimpar . V2VMapArrayHeight=1.5 ;wimpar . V2VGenerateBulkOnly=f a l s e ; % do or do not generate channel matrixwimpar .V2VMap = V2VMap(100) ;wimpar . NumClustersV2V = 20 ;% d e l t a t has to be the same f o r a l l nodeswimpar . Delta T=Src . d e l t a t ;wimpar . OffsetTimeSamples=1;
After the trajectory and the wimpar is initialized the V2V-WIM implementation can be called for each element ofthe trajectory.
%% Antenna Arrays o f V2V nodesArrays=examp l e syn t e t i c a r r ay s ;Tx . Array = [ { 1 } ] ;Rx . Array = [ 2 ] ;% main loop over t r a j e c t o r yf o r i = 1 : s i z e ( Src . Pos , 2)
curTx=Src ; curRx=Dst ; curTx . Array = [ { 1 } ] ; curRx . Array = [ 2 ] ;curTx . Pos = Src . Pos ( : , i ) ; curTx . Ve loc i ty=Src . Ve loc i ty ( : , i ) ;curRx . Pos = Dst . Pos ( : , i ) ; curRx . Ve loc i ty=Dst . Ve loc i ty ( : , i ) ;curwimpar=wimpar ; curwimpar . V2VPropagCondition=0;i f ( i >1)
dopp l e r phase s prev=V2V chan (1) . dopp l e r phase s ( i −1 ,1) ;e l s e
dopp l e r phase s prev=s t r u c t ( ' subpath phases ' , 0 , 'Phi LOS' , 0 ) ;end[H, de lays , bulk , intpar , dopp l e r phase s ]= V2VWim ( curTx , curRx , Arrays , V
, curwimpar , dopp l e r phase s prev ) ;V2V chan .H{ i}=H{1} ;V2V chan . de lays ( i , : )=de lays ;V2V chan . bulkpars ( i , 1 )=bulk ;V2V chan . dopp l e r phase s ( i , 1 )=dopp le r phase s ;
end
Page 33 (87)
RESCUE D4.4, v1.0
0 10 20 30 40 50 60 70 80 90−160
−150
−140
−130
time [s]
mag
nitu
de[d
B]
Figure 3.11: Evolution of magnitude of first tap/cluster over simulation time
After the loop finishes V2V chan.H contains the channel coefficients for each tap and for each transmitter-receiver-antenna-pair. In the examples presented above only a single antenna is used on each side.
>> s i z e (V2V chan (1 ) .H)ans =
1 338>> H=[V2V chan (1 ) .H{ : } ] ;>> s i z e (H)ans =
1 338 24>> f i g u r e ; p l o t ( [ 0 : s i z e (H, 2 ) −1]*wimpar . Delta T ,10* l og10 ( abs (H( 1 , : , 1 ) ) . ˆ 2 ) )
Figure 3.11 illustrates the evolution of the magnitude of the first tap along the trajectory. The coefficients of thefirst tap can finally be used in the same manner as the coefficient of the single-tap Rayleigh fading channel.
3.6 Key Performance Indicators
From a discussion of the Key Performance Indicator (KPI)s to use for the test, it emerged that it is difficult tomeasure the individual success of the various RESCUE partner’s interests with a single KPI. This is due to thefact that the different layers work together to compensate failures. In the PHY layer tests, the natural performancemetric is Frame Error rate, but also considering Header Error Rate. The correct reception of the header is anabsolute requirement for RESCUE decoding to work, since the header provides frame identification and Interleaverselection data. Therefore, the header part of the frame is protected with its own stronger Forward Error Correction(FEC) and Cyclic Redundancy Check (CRC) and therefore allows its own error count.
These Error Rates are being measured at the Destination node. It is measured by counting the incorrectly receivedheaders (after detecting a frame), and incorrectly received frames. This measurement mode is not entirely accuratesince it will ignore all frame errors prior to the first successfully decoded frame.
In order to have statistical significance of the Frame Error Rate (FER) results to a level of at least 10−2 - a valuethat can be seen e.g. in , a minimum of 400 frames were tested for each data point.
For the MAC tests, it was decided to use Throughput as the primary KPI, since it is a composite performance metricinvolving all the layers, not only the RESCUE joint decoding, but also the RESCUE MAC protocol. Unfortunately,this joint metric makes it difficult to discern between the performance of RESCUE MAC and the RESCUE coding.This is due to the fact that both methods complement each other in getting error free data to the destination. Evenif the RESCUE coding fails, the automatic retransmission will make sure that the effective Frame Error Rate is at(or near) zero.
Beside the Throughput, two additional KPI’s are computed in the MAC experiments: average Delay and average
Page 34 (87)
RESCUE D4.4, v1.0
Retransmission Count. The Delay is the round-trip time (RTT) of a frame from MAC perspective; It is the timebetween the moment when the RESCUE MAC block sends a DATA frame and the moment when it receives itsACK frame. The Retransmission Count is the number of ARQ attempts needed for a successful delivery of aframe.
All three KPI’s, Throughput, Delay, Retransmission Count, are measured at the source node, taking into accountthe transmit frames, and the acknowledgement from the destination. Therefore they also depend upon the ACKframes from the destination being properly received at the source.
Again, to reach statistical significance, it was attempted to transmit 500 frames.
3.7 OTAinVEE Validation Results
3.7.1 TS0-PHY Tests (AWGN)
3.7.1.1 Test Description
These tests are to establish the basic performance of the PHY layer of the software. In order to demonstrateany RESCUE functionality, the basic modulation/demodulation and coding/decoding software must work andperform close to theory. In particular, it should not exhibit any noticeable implementation loss over a theoreticalideal system, otherwise any purported “RESCUE gains” could well be hidden, or strongly distorted.
The simple TS0 PHY scenario consists of a transmitter and a receiver, with an AWGN channel between them.
As an initial matter, the task is to measure AWGN performance of the implemented system, in particular to verifythe operation of modulator/demodulator and the synchronizer. The initially available implementation for thesynchronization was based on a traditional approach using Costas Loops for carrier synchronization. This is a so-called blind estimator, i.e. the carrier is estimated without knowledge of the transmitted signal bits. Due to severeissues attributed to this “blind” synchronizer at the beginning of WP4, an alternative approach was proposed andimplemented, based on correlation of a known preamble.
These initial tests are thus being run for both synchronizers each, and the results will be used to determine whichsynchronizer is to be used in the future trials.
3.7.1.2 Effects of LO Offset Tuning
0 1 2 3 4 5 610−3
10−2
10−1
100
SNR [dB]
Err
orR
ate
HERFER
HER LO onFER LO on
0 1 2 3 4 5 610−3
10−2
10−1
100
SNR [dB]
Err
orR
ate
HERFER
HER LO onFER LO on
Figure 3.12: Performance of “blind” (left) vs. “correlator” synchronizer (right), with 10 MHz LO Offset offor on
A further issue was whether LO Offset should or should not be used. LO Offset is a feature of some USRP
Page 35 (87)
RESCUE D4.4, v1.0
frontends to shift the carrier out of the passband, so that the received signal is not affected by LO leakage/DCCarriers. In the various tests of the various teams in WP4 there were reports that this may negatively affectperformance.
Figure 3.12 shows a comparison of both synchronizers with and without the LO offset feature enabled. Shown arethe Header Error Rate (HER) and the FER. These differ since the header uses a stronger FEC code.
Both synchronizers work slightly better with the LO offset enabled. It was therefore concluded that the featureshould be enabled.
3.7.1.3 Blind Synchronizer vs Preamble Correlator
In Figure 3.13 we compare the performance of the standard (blind) synchronizer vs. the correlator synchronizer.
0 1 2 3 4 5 610−3
10−2
10−1
100
SNR [dB]
Err
orR
ate
HERFER
HER CORFER COR
0 1 2 3 4 5 610−3
10−2
10−1
100
SNR [dB]
Err
orR
ate
HERFER
HER CORFER COR
Figure 3.13: ”blind” vs. “correlator” synchronizer header- and frame error rate; standard strobe interval(left), strobe interval 1 s (right)
In both cases, the “blind” synchronizer performs slightly better in the low SNR Header Error Rate tests, whereasfor the frame error rates (in the 3. . . 5 dB region), the correlation synchronizer has a slight edge. This is particularlytrue in the “strobe interval 1s” case. This case simulates 1 second inactivity on the channel, so the estimators arebeing fed noise. This noise drives off the estimator.
At the beginning of WP4, with the (GNU Radio standard) parametrizations for the various synchronization loops,this effect was so significant that no meaningful low-SNR communication was possible. This was the main reasonto develop the Correlator synchronizer. It turned out however that careful tuning and parametrization of the “blind”synchronizer allowed almost-as-good performance.
3.7.1.4 Further Results
Two more facts can be established from the same test:
As shown in figure 3.14, the SNR as estimated by the test script using the payload bit errors matches closely(within about 0.5 dB) to the SNR set up by the channel emulator, in a range between 0 and 10 dB. This is withinthe level of accuracy achieved when calibrating the setup. For SNRs higher than 10 dB, the transmission waserror-free within the frame length, so no better SNR estimates could be made. The SNR estimator based on theBit Error rate, as part of the RESCUE software stack, is therefore found to work accurately.
Furthermore, figure 3.15 shows the principal working range of the joint decoder for RESCUE. The blue curve
Page 36 (87)
RESCUE D4.4, v1.0
0 2 4 6 8 10 12 140
2
4
6
8
10
12
SNR [dB]
est.
SNR[d
B]
Figure 3.14: Estimated SNR vs. Channel Emulator setting (true value)
−2 0 2 4 6 810−3
10−2
10−1
100
SNR [dB]
Err
orR
ate
HERFER
FER(J)
Figure 3.15: Joint decoder FER(J), compared to standard HER and FER
shows the gain of using a joint decoder, when both frames are transmitted twice, in the same SNR.
A very important interim RESCUE result, this test therefore shows that the joint decoder is working and providesa gain in a practical implementation, although in a simple AWGN case.
It can be seen that only within the rather narrow SNR range of 2 dB. . . 5 dB, Joint Decoder RESCUE coding gaincan be expected, at least as shown by our limited tests.
3.7.1.5 Test Results - AWGN with Static Doppler
100 Hz and 500 Hz static frequency offset (Doppler shift) were added to the signal. Otherwise the same conditionsas in TS0-AWGN-PHY were tested. The results are shown in 3.16 and 3.17, respectively. Comparing Frame- andHeader Error rate, again for header error rate the standard synchronizer appears to have a slight edge, whereas forthe total frame error rate, the correlator synchronizer has an advantage.
Page 37 (87)
RESCUE D4.4, v1.0
2 3 4 5 6 7 810−3
10−2
10−1
100
SNR [dB]
FER
BLIND 0 HzCOR 0Hz
BLIND 100 HzCOR 100Hz
BLIND 500 HzCOR 500 Hz
Figure 3.16: Frame Error Rate under Static Doppler, blind and correlator synchronizer
0 2 4 6 810−3
10−2
10−1
100
SNR[dB]
HE
R
BLIND 0 HzCOR 0Hz
BLIND 100 HzCOR 100Hz
BLIND 500 HzCOR 500 Hz
Figure 3.17: Header Error Rate under Static Doppler, blind and correlator synchronizer
3.7.1.6 Choice of Synchronizer
Since Doppler performance is mainly a feature of the synchronizer, raw (uncoded) Bit error Rate was also consid-ered. The data is shown in figure 3.18.
Unfortunately, during this test it became increasingly apparent that the implementation of the correlator synchro-nizer suffers from a software bug. During the test runs, intermittent hangs of the software were noticed. Initially,this was assumed to be due to GNU Radio issues. However, analysis of the results so far have shown that noneof those software hangs ever happened while testing the standard blind synchronizer. Since the problem onlyappeared intermittently and could not be reproduced on purpose, this software bug could not be found in thetime available. It was thus decided to use the blind synchronizer exclusively for all future tests. As establishedin the previous tests, while the Correlator synchronizer seems to have a slight edge, this edge is minimal, so nosignificantly different results are expected.
Page 38 (87)
RESCUE D4.4, v1.0
1 2 3 4 5 6 710−5
10−4
10−3
10−2
10−1
100
SNR [dB]
BE
R
BLIND 0 HzCOR 0Hz
BLIND 100 HzCOR 100Hz
BLIND 500 HzCOR 500 Hz
Figure 3.18: Bit Error Rate under Static Doppler (0, 100, 500Hz ), blind and correlator synchronizer
3.7.2 TS0-MAC Tests (AWGN)
3.7.2.1 Test Description
In the following tests, the Medium-Access-Control (MAC) layer functionality was enabled and tested, to ascertainfunctionality and performance characteristics of the MAC layer.
The hardware setup is similar to the TS0-PHY tests, except since the MAC layer features interaction between thetwo nodes (for Automatic Repeat Request (ARQ)), the return channel has to be enabled. Parameters of the returnchannel were set to the same value as the forward channel, ensuring channel symmetry.
The tests were conducted in a pure AWGN channel as well as in an AWGN channel with 100 Hz of static Doppler,with an intent to verify the findings of the previous chapter where it was found that static Doppler has negligibleinfluence on the results.
3.7.2.2 Test Results
The results, given in figures 3.19 and 3.20 confirm this assumption as true. Neither the individual frame errorrates (figure 3.19) nor the total throughput (figure 3.20) show more than marginal differences between Dopplerand non-Doppler cases.
The plots shown are for Baseline (no ARQ, no Joint Decoding), enabled ARQ, and enabled ARQ and JointDecoding at the Destination; with and without static Doppler, respectively.
It may therefore be concluded that the system under test can cope with static Doppler of 100 Hz; PHY tests hintthat also 500 Hz are OK for this particular implementation. Due to time limitations, this has not been verifiedfor higher Doppler frequencies, as this was not deemed necessary with regard to the planned V2V experiments.A practical system might require better capabilities, either because it would work at higher frequencies (e.g. the5.9 GHz band) or due to the lack of precision frequency references as utilized in these experiments.
Figure 3.20 further shows that Joint Decoding, together with ARQ, provides significant gain in the Low-SNRregion. JD alone will not be sufficient – without ARQ, there is nothing to jointly decode with.
Page 39 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 7 810−3
10−2
10−1
100
SNR [dB]
FER
BaselineBaseline/DOP
ARQARQ/DOPARQ/JD
ARQ/JD/DOP
Figure 3.19: FER, TS0 MAC AWGN, with/without static Doppler (100 Hz) for Baseline, ARQ and JointDecoder
2 3 4 5 6 7 8 90
5
10
15
20
25
30
35
SNR [dB]
Thr
ough
put[
kBit/s]
BaselineBaseline/DOP
ARQARQ/DOPARQ/JD
ARQ/JD/DOP
Figure 3.20: TS0-MAC-AWGN Throughput, without/with 100 Hz static Doppler, for Baseline, ARQ, JointDecoder
Page 40 (87)
RESCUE D4.4, v1.0
3.7.3 TS1-MAC Tests (AWGN & Fading)
3.7.3.1 Test Description
Moving on to tests of Toy Scenario 1, the test setup was extended. It now consists of 3 nodes (USRP + supportingPC’s). Using the Mini-OTA-Setup, the 6 resulting directed channels were emulated, the parameters (path loss,noise levels, Doppler, fading) being individually set as required by the experiments. Again, the PC’s system timewere synchronized using Network Time Protocol (NTP), providing timestamps with millisecond accuracy on thethree nodes. This allowed detailed log file analysis to determine the exact time sequence. No static Doppler wasset for the AWGN channel.
3.7.3.2 Test Results - AWGN
The throughput results of the TS1-MAC-AWGN tests are shown in figure 3.21. The scenarios under test and theirnaming have been described in section 3.4.
The main result can be immediately seen: Any joint decoding at the destination improves the throughput signif-icantly, compared to the baseline without joint decoding (shown in red). The other test cases behaved virtuallyidentical. As expected, Position A achieved the best throughput, since the relay, being positioned closer to thesource, will almost always decode correctly and retransmit most accurate data, so that Lossy Forwarding is notnecessary.
3.7.3.3 Test Results - Rayleigh Fading
The result of the test using a Rayleigh-Fading channel is shown in figure 3.22.
Again, the baseline throughput (no joint decoding at the Destination) performs worst, in all positions. However,the performance is better than in the AWGN case. This can be explained by the X axis now denoting averageSNR, implying that some frames exhibit better-than-average SNR. This case - gain by occasional constructivesuperposition of waves - allows communication when this would not be possible before.
Again, in Position A, no significant difference between the various relay schemes are visible once relaying isperformed, for again, the average SNR at the relay would be significantly better, providing mostly error-freerelaying, no matter which relaying strategy was employed.
In positions B and C however, small differences between the strategies can be seen. These differences are verysmall and may not itself be statistically significant, but do show a trend. It was assumed that by choosing the relaylocation for the A and B positions a bit more extreme, the differences would be more clearly visible. This wasverified by experiment, with the results shown in Figure 3.23; the ”more extreme” relay locations are depicted aspositions A′ and B′. Unfortunately, no significant differences were seen.
3.7.3.4 Conclusions
The RESCUE-SDF-CRC forwarding strategy (green line) performs only marginally better than the baseline strat-egy. So do both the RESCUE-LF (yellow) and RESCUE-LF2 (magenta) strategies. On the top end of the perfor-mance are the -JD strategies.
The observed differences in performance are very small. In order to claim statistical significance significantlymore and longer measurements need to be done, a feat which was not possible during WP4 (in total, about 3months of OTA Test time were used to achieve the conclusions presented here). The data appears to support theclaim that the joint decoding - both at the Destination and at the Relay, has a more significant impact on throughputperformance than the Lossy Forwarding.
Page 41 (87)
RESCUE D4.4, v1.0
2 3 4 5 6 70
10
20
30
SNR [dB]
Thr
ough
put[
kBit/s]
BaselineRESCUE-SDF-CRC
RESCUE-JDRESCUE-LF
RESCUE-LF-JDRESCUE-LF2-JD
2 3 4 5 6 70
10
20
30
SNR [dB]
Thr
ough
put[
kBit/s]
BaselineRESCUE-SDF-CRC
RESCUE-JDRESCUE-LF
RESCUE-LF-JDRESCUE-LF2-JD
2 3 4 5 6 70
10
20
30
SNR [dB]
Thr
ough
put[
kBit/s]
BaselineRESCUE-SDF-CRC
RESCUE-JDRESCUE-LF
RESCUE-LF-JDRESCUE-LF2-JD
Figure 3.21: TS1-MAC-AWGN Throughput, Position A(top), B(middle), C(bottom)
Even though that no gain from Lossy Forwarding could be shown in these tests, it does not mean that this gaindoes not exist; merely that it was not visible in these particular experiments.
Page 42 (87)
RESCUE D4.4, v1.0
1 2 3 4 5 6 7 80
5
10
15
20
25
AVG SNR [dB]
Thr
ough
put[
kBit/s]
BaselineRESCUE-SDF-CRC
RESCUE-JDRESCUE-LF
RESCUE-LF2RESCUE-LF-JD
RESCUE-LF2-JD
1 2 3 4 5 6 7 80
5
10
15
20
25
AVG SNR [dB]
Thr
ough
put[
kBit/s]
BaselineRESCUE-SDF-CRC
RESCUE-JDRESCUE-LF
RESCUE-LF2RESCUE-LF-JD
RESCUE-LF2-JD
1 2 3 4 5 6 7 80
5
10
15
20
25
AVG SNR [dB]
Thr
ough
put[
kBit/s]
BaselineRESCUE-SDF-CRC
RESCUE-JDRESCUE-LF
RESCUE-LF2RESCUE-LF-JD
RESCUE-LF2-JD
Figure 3.22: TS1-MAC-1TAP Throughput, Position A(top), B(middle), C(bottom)
Page 43 (87)
RESCUE D4.4, v1.0
1 2 3 4 5 6 7 80
5
10
15
20
25
SNR [dB]
Thr
ough
put[
kBit/s]
BaselineRESCUE-SDF-CRC
RESCUE-JDRESCUE-LF
RESCUE-LF2RESCUE-LF-JD
RESCUE-LF2-JD
1 2 3 4 5 6 7 80
5
10
15
20
25
SNR [dB]
Thr
ough
put[
kBit/s]
BaselineRESCUE-SDF-CRC
RESCUE-JDRESCUE-LF
RESCUE-LF2RESCUE-LF-JD
RESCUE-LF2-JD
Figure 3.23: TS1-MAC-1TAP Throughput, Position A′(top), B′(bottom)
Page 44 (87)
RESCUE D4.4, v1.0
4. Validation in Real Field Experiments
4.1 Description of RESCUE Experimental Validation Strategy
To evaluate the RESCUE technology in the real world a set of real-field experiments was planned and conductedwithin the WP4.
It was initially planned to investigate RESCUE performance in Vehicle-to-Vehicle (V2V) communication scenariothrough OTAinVEE emulation only by applying an appropriate channel model. However, it was later requestedby the project reviewers and the European Commission officers to also conduct real-field V2V experiments.
Following this request, there are two general communication scenarios in which the RESCUE functionality isevaluated: public safety indoor and V2V outdoor scenarios [7]. While in the first case the RESCUE nodes remainstationary during the whole time of the experiment, in the later case, they also considered mobile with the averagespeed of up to 20 km/h.
Similarly to the OTAinVEE verification, due to the time constraints, it was decided to investigate RESCUE per-formance in Toy Scenario 0 and Toy Scenario 1 only. For the indoor safety application the TS0 was studied bothwith PHY-only and MAC(+PHY) RESCUE implementations, while in the TS1 just the MAC(+PHY) RESCUEprotocol stack examined. In the V2V validation campaign we avoided spending time on reproducing outdoors thesame full set of experiments as in the public safety case. Instead we concentrated on the nodes’ mobility featurein the TS1.
It was initially aimed to validate the full range of RESCUE strategies. While in the indoor scenarios this target wassuccessfully hit, in the V2V scenario we had to significantly reduce the number of experiments because of somedelays in the delivery of the stable version of RESCUE software, some unexpected difficulties with the RESCUEV2V testbed, and much higher effort needed to conduct the V2V experiments.
The KPIs are generally kept the same as in the OTAinVEE verification (see section 3.6).
The next sections describe the experimental testbeds for the indoor safety and V2V scenarios, the ways the vali-dation runs are setup and the results thereof accompanied by some interim conclusions.
4.2 Validation in Indoor Safety Scenarios
In this section we describe the indoor testbed for the safety scenario, as well as the configuration and results forthree series of experiments TS0 PHY, TS0 MAC and TS1 MAC.
4.2.1 Testbed Description
The indoor safety scenario was deployed in the offices of the Department of Telecommunications at the AGHUniversity of Science and Technology in Krakow, Poland. The building where the testbed is located is a typicaloffice building with a skeleton made of steel profiles and walls made of two layers of gypsum plasterboards. Theindoor testbed consists of five stations distributed within the building so as to resemble a typical office environ-ment. Three stations are placed on the ground floor while the remaining two are located on the first floor. Thespatial structure of the test facility is presented in Figure 4.1.
The network topology presents us with a multitude of source-destination (TS0) links as well as many possiblesource-relay-destination (TS1) configurations. Tests were performed on a large subset of the possible pairs ortriples. The results which presented herein are for the most interesting cases. For example, the 10–12 link wasalways of good quality and therefore did not provide meaningful results.
Each of the deployed stations consists of an NI-USRP N210 device equipped with a SBX v.3 daughterboard and aDell Inspirion laptop with an Intel i7 processor. All of the SBX daughterboards were calibrated using both internalprocedures provided by the UHD framework and external calibration procedures performed by TUIL. In order tooptimize the GNU Radio performance the VOLK profiling routines were performed for each station.
Page 45 (87)
RESCUE D4.4, v1.0
10.33.0.14Rx
10.33.0.12Tx
10.33.0.10Rx
10.33.0.13Rx
opposite side of building
10.33.0.11Rx
opposite side of building
First Floor Corridor
10.33.0.14Rx
10.33.0.11Rx
Roof GPS Rx for synchronisation
Office Room
Office Room
Office Room
Office Room
Office Room
Office Room
Office Room
Office Room
Office Room
Office Room
Ground Floor Corridor
10.33.0.10Rx
10.33.0.12Tx
10.33.0.13Rx
Server Room
MeetingRoom
Office Room
Office Room
Office Room
Office Room
Office Room
Office Room
Optical Lab
Figure 4.1: Office testbed for validating indoor safety scenarios. In each single test, one station was chosenas the source (Tx), while all other stations were listening (Rx), as either destination or relay.
The detailed station parameters are listed in Table 4.1. Each of the stations was controlled by the Kubuntu 14.04operating system and ran GNU Radio 3.7.10 with patches provided by the RESCUE consortium. Each station wasrunning up-to-date RESCUE software.
Four of the five stations were synchronized using an external frequency source. The indoor testbed used NanosyncII GPSDO with external rooftop antennas in order to provide a 10 MHz reference frequency to the test stations.The Nanosync II GPSDO provides long term frequency stability even in case of temporary loss of GPS signalthanks to a built-in algorithm that compensates external temperature changes and aging of the internal OCXO. Thereference signal is distributed among stations using RG58 cables. The structure of the reference signal distributionnetwork is presented in Figure 4.2.
In order to enable testing the influence of the reference frequency’s stability on the RESCUE implementation, oneof the safety test bed stations (10.33.0.13) was excluded from the external clock synchronization network. For thisstation a frequency correction value is provided at each stage of the experiment setup.
Each of the test stations was equipped with two gigabit Ethernet network interfaces, one of which was used todirectly connect the USRP device while the other was used for controlling the stations over a private fixed network.In order to facilitate experiment planning and software configuration, the same IP address of each USRP device
Page 46 (87)
RESCUE D4.4, v1.0
Table 4.1: Station parameters
Parameter name Parameter valueSDR device type USRP N210Daughterboard type SBX v.3Frequency range [MHz] 400–4400Tx gain range [dB] 0–31.5Tx gain range [dB] 0–31.5Bandwidth [MHz] 40Number of CPUs 6CPU clock freq [GHz] 1.8Memory size [GB] 8Network interface number 2Network interface type Gigabit EthernetSDR external clock Yes
Rooftop
GPS antenna
Nanosync II
GPS receiver10.33.0.12
10.33.0.1110.33.0.10
10.33.0.14
Rx/Tx
REF IN
Rx/TxREF IN
REF IN
REF IN
Rx/Tx
Rx/Tx
10 MHz Out
Figure 4.2: Diagram of the external clock distribution network.
was configured for each of the test stations. For each station the USRP device was located at address 192.168.0.2.The internal fixed network connectivity of the test stations was based on IPv4 with an address pool of 10.33.0.0/24.The networking parameters of each of the test stations are presented in Figure 4.1 and summarized in Table 4.2.
Page 47 (87)
RESCUE D4.4, v1.0
Table 4.2: Test stations addressing and locations
Station No. IP Address Physical Location10 10.33.0.10 Room 017A, ground floor11 10.33.0.11 Room 107, 1st floor12 10.33.0.12 Room 017B, ground floor13 10.33.0.13 Room 013, ground floor14 10.33.0.14 Room 118, 1st floor
4.2.2 TS0 PHY Experiments
In the first set of experiments, we evaluated the operation of the RESCUE PHY layer in a TS0 scenario (i.e., withpoint-to-point links). The following KPIs were selected for this set of experiments:
• header error rate (HER),
• frame error rate (FER) without joint decoder (i.e., with a single decoder): FER(s),
• frame error rate (FER) with joint decoder: FER(j).
One Tx host and four Rx hosts were used simultaneously during the experiments in a typical office environment(Fig. 4.1). Each of the Rx hosts operated with a fixed rx-gain adjusted in order to achieve a full FER scalefor each of the Rx stations. The changes of SNR were enforced by increasing the tx-gain parameter value insubsequent experiments. Both local (for station 10.33.0.13) and external frequency sources (all other stations)were used during the experiments. For 10.33.0.13, a frequency correction value was added. All experimentswere conducted using the rescue phy tx.py and rescue phy rx.py scripts with the following commonoptions:
• Transmitter parameters: --correlate-preamble --samp-rate 1M -t 300--strobe-interval 100 --ampl 0.4
• Receiver parameters: --correlate-preamble -j --samp-rate 1M -t 300
During each experiment, the value of tx-gain was varied from 0 to 20 with a step of 0.5 dB.
The experiments were performed for all link pairs in the testbed. Below, we present the most interesting casesfor the following link pairs: (10, 11), (11, 12), (10, 13), and (11, 14). In each case the duplicate count was set tovarious non-negative values to observe its effect on the results.
For the (10, 11) link, Figure 4.3a presents the frame error rate for varying quality in the case of no duplicates.This test confirms that the header protection is sufficient. Obviously, there is no gain from the joint decoder inthis case of no retransmissions. Increasing the duplicate count to one (Figure 4.3b), we observe that the HERand FER(s) values are similar. For FER(j) there is improved performance in the range of about 3–5 dB where theframe error rate is at 10−1 instead of 100. Further increasing the number of retransmissions to three (Figure 4.3c)we observe that the HER has changed, most likely due to changes in channel conditions and the inefficiency ofthe SNR measurements. Nonetheless, the joint decoder performs even better than before with an average FER of10−2 in the 3–5 dB range.
The results of the TS0 PHY experiment for the (11, 12) link are shown in Figure 4.3. Comparing HER on thethree different subfigures for the three duplicate count settings, it can be seen that in the first case (Figure 4.4a),the results are different. Also, in this case there was no gain from the joint decoder because of the duplicate countsetting. For a duplicate count of one (Figure 4.4b) we see that the header protection is sufficient and that the jointdecoder gain (for a FER of 10−1) is about 0.5 dB. Increasing the duplicate count to two increases this gain to about2 dB (Figure 4.4c).
Similar conclusions are also applicable to the results for the (10, 13) link: the HER is different for zero duplicates
Page 48 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
FER(s)FER(j)HER
(a) Duplicate count of 0
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
FER(s)FER(j)HER
(b) Duplicate count of 1
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
FER(s)FER(j)HER
(c) Duplicate count of 3
Figure 4.3: PHY performance in TS0 on the (10, 11) link.
(Figure 4.5a) while the joint decoder gain varies from 0.5 dB (Figure 4.5b) to 2 dB (Figure 4.5c). In some cases,for SNR values above 7dB, we noticed a problem with HER (Figure 4.5a) or FER(j) (Figure 4.5b) -– probably dueto some experimental error. Unexpected FER values were also visible for high SNR values during measurementson the (11, 14) link (Figure 4.6), with the exception of the duplicates count set to two. In the latter case, the jointdecoding gain is best visible and equal to about 1.5 dB.
To summarize, from the TS0 PHY tests we learned the following:
• the varying quality of the radio channel made the experiments difficult to reproduce – the HER values shouldbe similar for each link regardless of duplicate count settings,
• the experiments allowed us to assess the performance of the RESCUE coding algorithms in a scenariowithout higher-layer overhead and the observed gain was usually about 2 dB when three frame copies werejointly decoded (duplicate count of two).
Page 49 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
FER(s)FER(j)HER
(a) Duplicate count of 0
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
FER(s)FER(j)HER
(b) Duplicate count of 1
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
FER(s)FER(j)HER
(c) Duplicate count of 2
Figure 4.4: PHY performance in TS0 on the (11, 12) link.
Page 50 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
FER(s)FER(j)HER
(a) Duplicate count of 0
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
FER(s)FER(j)HER
(b) Duplicate count of 1
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
FER(s)FER(j)HER
(c) Duplicate count of 3
Figure 4.5: PHY performance in TS0 on the (10, 13) link.
Page 51 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
FER(s)FER(j)HER
(a) Duplicate count of 0
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
FER(s)FER(j)HER
(b) Duplicate count of 1
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
FER(s)FER(j)HER
(c) Duplicate count of 2
Figure 4.6: PHY performance in TS0 on the (11, 14) link.
Page 52 (87)
RESCUE D4.4, v1.0
4.2.3 TS0 MAC Experiments
Having validated the operation of the PHY layer, we next added the MAC layer in the TS0 scenario and validatedthe operation of the RESCUE MAC under point-to-point links. Note that without a relay, all relaying strategiesare equivalent and we are testing only the performance of joint decoding at the destination. The following KPIswere selected for this set of experiments:
• throughput [kbit/s],
• frame error rate (FER),
• number of retransmissions,
• delay [s].
Pairs of one Tx and four Rx hosts were used during these experiments, as deployed in a typical office environment(Fig. 4.1). During all experiments the --rx-gain parameter for the receiving station was set to a fixed value.The changes of SNR were again enforced by increasing the --tx-gain parameter value in subsequent exper-iments. The --rx-gain parameter of the transmitter station was set to the maximum permissible value (31.5dB) and the --tx-gain parameter of the receiving station was also set to the maximum permitted value of (31.5dB) in order to guarantee error free transmission of ACK frames. Both local (for station 10.33.0.13) and externalfrequency sources (all other stations) were used during the experiments. For 10.33.0.13, a frequency correctionvalue was added.
All experiments were conducted using the rescue mac tx.py and rescue mac rx.py scripts with the fol-lowing common options:
• Transmitter parameters: --samp-rate 1M -c 500 --strobe-interval 200 --ampl 0.4--trace -d --rx-gain 31.5 -l 1000
• Receiver parameters: --samp-rate 1M -t 300 -d --trace --ampl 0.4 --tx-gain31.5 -l 1000 -t 500
During each experiment, the value of --tx-gain was varied from 0 to 20 with a step of 0.5 dB. This allowedto vary the quality of the source-destination link. In all subsequent figures, the X axis presents the average SNRof the evaluated link during the experiment. Furthermore, to provide a clearer presentation of the results, curvefitting was applied.
The MAC performance of the (10, 12) link in TS0 is shown in Figure 4.7. For this link we were able to obtainthe most measurement points within the studied SNR range. In terms of throughput, RESCUE outperforms thebaseline within the range of about 1–2 dB of average link SNR. The achieved throughput values are in general lowdue to the limited computational power of the devices. It can be observed on all figures (Fig. 4.7, 4.8, 4.9, 4.10,4.11) that, because of computational power limitation, throughput cannot cross the maximum value of 33 [kbit/s].However, for low SNR values the relative throughput gain is quite high. The results show, that applying jointdecoding can bring a non-zero throughput, while the baseline’s result is close to zero. The throughput performancegain provided by RESCUE (joint decoding) is mirrored in the other metrics: delay (50% improvement at 4 dB)and FER (over 1 dB gain at 10−1).
However, the most important metric is the number of required retransmissions. Figure 4.7c shows that RESCUEenables significant reduction of the required retransmissions, up to 50%. Joint decoding of lossy frames pro-vides the destination-side decoder with additional information, therefore increases the probability that the framedecoding process finished successfully (i.e., the outage probability).
Measurements were also performed for three other links:
• the (10, 11) link (Figure 4.9),
• the (11, 12) link (Figure 4.10),
Page 53 (87)
RESCUE D4.4, v1.0
• the (11, 14) link (Figure 4.11).
However, in these cases it was difficult to obtain measurements in the RESCUE “area of operation” because theaverage SNR usually turned out to be either too low (< 3 dB) or too high (> 10 dB).
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNR [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE
(a) Throughput
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNR [dB]D
elay
[s]
BaselineRESCUE
(b) Delay
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNR [dB]
Ret
rans
mis
sion
s
BaselineRESCUE
(c) Retransmissions
0 1 2 3 4 5 6 7 8 9 10
10−2
10−1
100
SNR [dB]
FER
BaselineRESCUE
(d) FER
Figure 4.7: MAC performance in TS0 on the (10, 12) link.
Similar performance was observed on the same link in the reverse direction (Figure 4.8. The RESCUE gain inthroughput, compared to link (10, 12), is slightly smaller (1–1.5 dB), but the delay reduction is still around 50%(Fig. 4.8b). The required number of retransmissions, which is the most important KPI, is also on a similar level –the reduction is around 40%.
The radio links (10, 11) and (12, 11) are much longer then the previously described link (10, 12) and also ondifferent floors (i.e., they cross a ceiling). Therefore, radio conditions in these channels are more demanding,which resulted in difficulties in performing the measurements. That is why we were not able to obtain as manymeasurement points as in the case of the (10, 12) link. However the most important observation from the resultanalysis is that the RESCUE gain in terms of reduction of the number of required retransmissions is around 60%in both (10, 11) and (12, 11) links. This is a 20% bigger gain compared to the (10, 12) link. This means that inpoor radio conditions the RESCUE concept brings higher gains, which is in line with theory.
Page 54 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNR [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE
(a) Throughput
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNR [dB]
Del
ay[s
]
BaselineRESCUE
(b) Delay
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNR [dB]
Ret
rans
mis
sion
s
BaselineRESCUE
(c) Retransmissions
0 1 2 3 4 5 6 7 8 9 10
10−2
10−1
100
SNR [dB]
FER
BaselineRESCUE
(d) FER
Figure 4.8: MAC performance in TS0 on the (12, 10) link.
Page 55 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNR [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE
(a) Throughput
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNR [dB]
Del
ay[s
]
BaselineRESCUE
(b) Delay
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNR [dB]
Ret
rans
mis
sion
s
BaselineRESCUE
(c) Retransmissions
0 1 2 3 4 5 6 7 8 9 10
10−2
10−1
100
SNR [dB]
FER
BaselineRESCUE
(d) FER
Figure 4.9: MAC performance in TS0 on the (10, 11) link.
Page 56 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNR [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE
(a) Throughput
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNR [dB]
Del
ay[s
]
BaselineRESCUE
(b) Delay
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNR [dB]
Ret
rans
mis
sion
s
BaselineRESCUE
(c) Retransmissions
0 1 2 3 4 5 6 7 8 9 10
10−2
10−1
100
SNR [dB]
FER
BaselineRESCUE
(d) FER
Figure 4.10: MAC performance in TS0 on the (11, 12) link.
Page 57 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNR [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE
(a) Throughput
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNR [dB]
Del
ay[s
][s]
BaselineRESCUE
(b) Delay
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNR [dB]
Ret
rans
mis
sion
s
BaselineRESCUE
(c) Retransmissions
0 1 2 3 4 5 6 7 8 9 10
10−2
10−1
100
SNR [dB]
FER
BaselineRESCUE
(d) FER
Figure 4.11: MAC performance in TS0 on the (11, 14) link.
Page 58 (87)
RESCUE D4.4, v1.0
4.2.4 TS1 Experiments
Having established the performance of the operation of RESCUE on point-to-point wireless links, we proceedto verify its operation in a scenario with a single relay (TS1). Based on initial tests in the indoor scenario, weconcluded that the node deployment (Figure 4.1) allows us to test a source–relay–destination connection usingnodes 10, 12, and 14. The placement of this trio necessitated TS1 experiments with the relay in position A or B.Suitable configurations for a relay in position C (equidistant from source and destination) could not be obtainedin the testbed.
For relay position A, the source was node 12, the destination – node 14, and node 10 acted as the relay (placedclose to the source). The TS1 configuration has three degrees of freedom in the qualities of the three links (SD,SR, RD). Since we are considering relay position A, the SR link has high SNR and is considered lossless. Next,we fixed the quality of the RD link to exhibit various levels of average SNR (from 0 to 10 dB). Then, we conductedexperiments for all forwarding strategies by varying the average SNR on the SD link. This allowed us to obtaina complete performance profile of the RESCUE forwarding strategies for the indoor scenario. The throughputresults of the experiments are presented in Figure 4.12. From the comparison of the individual subfigures, weconclude that by improving the value of SNRRD we diminish the gains that can be provided by RESCUE. Forexample, for SNRRD = 10 dB, the TS1 topology always has two perfect links (SR and RD) which constitute alossless end-to-end path. Therefore, all strategies exhibit the same performance (Figure 4.12f). Evidently, in thiscase, for low SNRSD values all traffic is sent through the relay because the obtained throughput is approximatelyhalf of the throughput for high SNRSD values (the direct link case). The gain from RESCUE is most visible whenthe RD link quality is in the RESCUE “area of operation” (SNRRD ≤ 6 dB). In these cases, the best performancefor low SNRSD is provided by the RESCUE-JD followed by RESCUE-LF (cf. Figure 4.12c). A complete set ofMAC KPIs for a selected RD link quality is given in Figure 4.13. The throughput performance of the variousrelaying strategies corresponds to the results of the other metrics: delay, retransmissions, and FER.
In order to assess the TS1 performance with the relay in position B, we switched the roles of the source anddestination, i.e., node 14 was the former, node 12 – the latter, and node 10 remained the relay. In this topology, theRD link is considered perfect, the SR link quality is fixed to selected average SNR values (between 0 and 10 dB),and the SD link quality is varied in each experiment. The throughput results of the experiments are presented inFigure 4.14. Similarly to the previous case with the relay in position A, for high SNRSR we always have a losslessend-to-end path and all strategies perform similarly (Figure 4.14f). The effects of using the RESCUE strategies forlower SNRSR values are less pronounced in this case in comparison to position A. Still the gains are important asthey allow communication in SNR regimes where the baseline fails (cf. the throughput values for SNRSD ∈ [1,5]in Figure 4.14c). In the extreme case, i.e., for SNRSR = 0 dB (Figure 4.14a), RESCUE-LF was the only strategywhich provided any results in the experiment. A complete set of MAC KPIs for a selected RD link quality isprovided in Figure 4.15. As before, the throughput performance of the various relaying strategies corresponds tothe results of the other metrics: delay, retransmissions, and FER.
Page 59 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(a) SNRRD = 0 dB
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(b) SNRRD = 2 dB
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(c) SNRRD = 4 dB
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(d) SNRRD = 6 dB
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(e) SNRRD = 8 dB
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(f) SNRRD = 10 dB
Figure 4.12: Comparison of throughput performance in TS1 with relay in position A: the (12-10-14) linkfor SNRRD ∈ {0,2,4,6,8,10} dB.
Page 60 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(a) Throughput
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNRSD [dB]
Del
ay[s
]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(b) Delay
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNRSD [dB]
Ret
rans
mis
sion
s
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(c) Retransmissions
0 1 2 3 4 5 6 7 8 9 10
10−2
10−1
100
SNRSD [dB]
Fram
eer
rorr
ate
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(d) FER
Figure 4.13: MAC performance in TS1 with relay in position A: the (12-10-14) link for SNRRD = 4 dB.
Page 61 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
RESCUE-LF
(a) SNRSR = 0 dB
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(b) SNRSR = 2 dB
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(c) SNRSR = 4 dB
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(d) SNRSR = 6 dB
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(e) SNRSR = 8 dB
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(f) SNRSR = 10 dB
Figure 4.14: Comparison of throughput performance in TS1 with relay in position B: the (14-10-12) linkfor SNRSR ∈ {0,2,4,6,8,10} dB.
Page 62 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 7 8 9 100
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(a) Throughput
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNRSD [dB]
Del
ay[s
]
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(b) Delay
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
SNRSD [dB]
Ret
rans
mis
sion
s
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(c) Retransmissions
0 1 2 3 4 5 6 7 8 9 10
10−2
10−1
100
SNRSD [dB]
Fram
eer
rorr
ate
BaselineRESCUE-JDRESCUE-LF
RESCUE-SDF-CRC
(d) FER
Figure 4.15: MAC performance in TS1 with relay in position B: the (14-10-12) link for SNRRD = 4 dB.
Page 63 (87)
RESCUE D4.4, v1.0
4.3 Validation in Outdoor V2V Scenarios
The set of outdoor V2V experiments is aimed at investigating the RESCUE gain in the most dynamic conditions(compared to the channels emulated in OTAinVEE at TU Ilmenau and measured in indoor safety scenarios atAGH). However this also imposes certain requirements upon the experimental setup in order to guarantee reliabil-ity and repeatability of the obtained results.
4.3.1 Requirements on V2V Testbed
Conducted OTAinVEE as well as static indoor evaluation testbeds enjoyed their luxury of having dedicated spaceavailable over long period of time (months) to setup and conduct RESCUE measurements. In contrast, the V2Vtestbed had mobile platforms (vehicles) available for some limited time only. Furthermore it had also to meetsome additional constraints.
Limited availability of the vehicles put a requirement of denser (than in OTAinVEE and indoor safety testbeds)integration of the hardware and software for V2V case. Each of the three testbeds (for Source, Relay and Destina-tion) had to be compact and portable and therefore quickly installable into a car. To meet this constraint it wasdecided to integrate all the testbed components within one board, which would only require to attach antennas,power supply and management tools to start/stop a measurement.
This would also mean that the devices which run RESCUE software had to be small, but still quite computationallypowerful; would have IBM PC compatible architecture (since RESCUE software was developed for regular PCand could not be ported to a traditionally more compact platform, such as e.g. embedded system); and wouldprovide convenient interface to operate the RESCUE nodes.
Since the objective was to evaluate RESCUE technology in a mobile scenario, where communicating nodes couldbe quickly re-positioned to a given location outdoors, or even be mobile while performing a test run, the V2Vtestbed should have been designed so that it is able to operate without a stationary power supply. As the alternativesources of power supply the standard car battery or an external battery were considered. Still it was unacceptablethat the testbed would for the whole time be powered from the car battery, since this would drain the batteryquickly. As a compromise a hybrid power supply chain was designed and implemented, which allows theV2V setup be powered from the central power supply of 220-230V during the stationary operation and run from abattery while being re-positioned or performing driving tests. Furthermore, switching over from one power supplyto another would not cause a blackout and a test run would not be affected.
Similarly to the problem of power supply, the problem with external reference clock had to be solved. In OTAin-VEE as well as in indoor testbeds, the 10 MHz reference was distributed from a single source over cables. InV2V this is impossible for obvious reasons. Therefore, each of the RESCUE nodes had to have an independentreference frequency source, which should lock USRP to a desired central frequency. As discussed in the section3.3 above, RESCUE nodes in the target mode (signal bandwidth, central frequency) can tolerate up to 2-3 kHzoffset. In the V2V experiments this offset would comprise inaccuracy in USRP central frequency and Dopplerspread resulted from the movement of communicating RESCUE nodes and possibly some of the reflectors in theenvironment. Maximum Doppler frequency (for the nodes moving towards each other) can be calculated usingthe following equation:
∆ f =νr−νs
cf0
where νr, νs - speed of the receiver and source nodes, m/s, c≈ 3×108m/s - velocity of electromagnetic waves inthe air, f0 - central frequency, Hz.
Assuming for each node the practical maximum speed of 240 km/h (applies to Germany only) and central fre-quency of 2.53 GHz, the maximum change in frequency caused by the Doppler effect should not exceed 1200 Hz.This means that the maximum inaccuracy in the central frequency should not exceed 1 kHz.
Another requirement arose from the nature of wireless channel on which the RESCUE functionality has to beevaluated in a V2V test. In contrast to OTAinVEE, where the channel parameters are fully controllable (up tothe noise figure instances); and indoor stationary scenario, where channel parameters change because of chang-ing environment (e.g. moving people, opened window or door etc.), but should generally remain reproducible
Page 64 (87)
RESCUE D4.4, v1.0
between longer measurement runs, in V2V because of additional mobility of the communicating nodes, the chan-nel conditions may vary severely from one frame to another. It is nevertheless desired to be able to match sometemporary improvements or degradations of the channel within a test run with changes in the environment. Thisshould help to explain behavior of the communication system during the later analysis and post-processing of themeasured data. To achieve this some additional meta data about the environment should be collected. In our V2Vtestbed such meta data comes in the form of instantaneous GPS-coordinates of every communicating node, and360°photos taken regularly by each of the RESCUE nodes.
To produce reliable KPI values, the RESCUE software requires that the communication nodes are started in agiven order: first the Destination node, then - for all the Toy Scenarios except TS0 - the Relay(s), and finally theSource node. This imposed a requirement of coordinated operation of multiple RESCUE nodes, preferably froma single centralized entity. Apparently, this should have also be true for moving cars.
Finally, the RESCUE evaluation within WP4 had to be carried out by two partner institutions: AGH and TUIlmenau. While remaining time and budget did not permit traveling from one institution to another, a strongrequirement was to minimize the manpower needed to conduct V2V real-field evaluations.
4.3.2 Description of V2V Testbed
The V2V testbed designed and built in TU Ilmenau meets all of the listed requirements. Its structural diagram isshown in figure 4.16. There are two nodes sketched in the diagram: a RESCUE node and the node, which wouldperform the centralized management of the evaluation runs.
Figure 4.16: V2V node schematics
All the RESCUE nodes - the Source, Relay and Destination - have the same structure, the only part which isdifferent is the RESCUE script, which is running on the node and the input parameters thereof.
Let us take a preciser look at a RESCUE node structure, its individual hardware components, and how they areintegrated.
During the V2V measurement campaign at TU Ilmenau 3 cars kindly provided by Fraunhofer IIS were used:
Page 65 (87)
RESCUE D4.4, v1.0
Volkswagen Sharan for the Source node, Range Rover for the Relay node and Volkswagen Golf for the Destinationnode (see figure 4.17). Further more, two out of these cars (Range Rover and VW Sharan) had already got DC/ACconverters installed.
Figure 4.17: V2V mobile platforms
Most of the components within the RESCUE testbed require reliable 230V AC power supply. Therefore, designof the hybrid power supply chain got precise attention while creating the RESCUE V2V testbed.
The hybrid power supply chain consists of a WAECO Priority power switcher with two inputs and one output.One of the inputs (Input 1) should be directly plugged into a 220-230V outlet whenever it is available. In such acase the priority power switch automatically switches to this input, even if it was receiving current from the otherinput (Input 2) before. If Input 1 is unplugged, then the WAECO switch switches over to the Input 2, which isconnected to the DC/AC converter.
The DC/AC converter transforms the 12V DC into 230V AC. The source of the 12V DC is either the standard carbattery or an external battery. In two out of three measurement cars (in VW Sharan and Range Rover) the DC/ACconverters were already installed. Which made it much easier to integrate the RESCUE testbeds into the cars. InVW Golf an external battery was used as shown in the figure 4.18.
Figure 4.18: V2V RESCUE testbed installed into a car
Page 66 (87)
RESCUE D4.4, v1.0
The output of the WAECO priority switch is connected to uninterruptable power source (UPS). It was noticedwhen changing from one input to another, that the priority switch would interrupt the power supply for up to 1 s.This is unacceptable, since it would cause a power cycle of the USRP and NUC. The UPS has its own battery andserves as the buffer while switching from one power source to another.
A power strip, connected to the UPS output provides a stable and reliable source of 230V AC for all the compo-nents, used in the Testbed (Intel NUC, USRP, GPSDO and fish-eye camera)
The RESCUE testbed designed to split the power chain from the rest of the devices. The power chain is thereforelocated on one (bottom) side of the board, while the rest of the components are mounted on the other (top) side ofthe board.
In the core of every RESCUE node there is a USRP N210 + SBX v.3. It is the same type and model of USRPdevices as used in the OTAinVEE (actually taken over for V2V experiments) as well as in indoor public safetyevaluation campaigns (see Table 4.1). This guarantees exactly the same performance of the SDR platform indifferent test scenarios and enables comparability of the evaluation results between all the 3 practical cases.
To run the RESCUE software, which incorporates some computationally intensive signal processing blocks, weused Intel NUC 5i5RYH platforms with Core i5 quad-core CPUs and IBM PC compatible architecture. Thisenabled us running RESCUE software at 1 MSamp/s rate, while reducing more than twice the size of the process-ing unit compared to the size of regular laptops, used by our colleagues from AGH. The RESCUE logs and themeta-data (GPS coordinates, photos) are stored on a fast internal SSD (Samsung SSD 850 EVO 500GB).
Since the RESCUE testbeds are to be located inside the cars, additional attenuation is introduced due to the carshell when using standard VERT2400 antennas. To overcome this, we decided to use different antennas, whichare mounted outside the car. An obvious choice is magnetic-mounted antennas (MMA), which are easy to mountat the roof of a car. In the upper hemisphere these antennas have a wide beam, which shape is close to omni-directional. In our V2V setup these antennas are directly attached to the RF ports of USRP, without additionalpower amplifiers. The output power of the SBX v.3 daughterboards reaches 100 mW, which is enough (based onthe preliminary outdoor experiments conducted at TU Ilmenau) to establish a 200 m communication range in afree-space Line-of-Sight (LOS) scenario.
To distribute the external 10 MHz reference clock we used GPS. For this purpose the Miller G3RUH GPS Disci-plined Oscillator (GPSDO) were utilized. These consist of GPSDO block and external power supply, which feedsthe GPSDO with 12V DC. As the power source for the GPSDOs in one setup we used dedicated external battery,while in the other setups we used AC/DC converters. The first solution guarantees uninterruptible power supplyfor hours, but needs to be charged at least once per day, while the latter solution requires a stable 230V AC source.Since we have developed a stable hybrid power supply chain anyway, the latter solution turned out to be morereliable and easy to maintain, since there is no need to charge another small battery.
The GPSDO module provides the following outputs: 10 MHz reference clock, 1 PPS signal and a RS-232 DataInterface. The GPS disciplined 10 MHz reference frequency allows to synchronize the Local Oscillators of theRESCUE nodes (the relative frequency offset does not exceed 50 Hz). 1 PPS is not required in our setup.
The effect of LO frequency drift was already discussed in the context of OTAinVEE verification (see section 3.3),however for outdoor scenario the frequency error turned out to be even higher (∆ f ≈ 6 kHz) for exactly thesame set of USRP N210 devices. Figure 4.19 compares the frequency of the sinusoid received at the Destinationwithout (a) and with (b) external 10 MHz reference frequency attached to both nodes. The reference sinusoid( fs = 200 kHz) was up converted to f0 = 2.53GHz and transmitted from the Source node over the air.
To receive a reliable GPS signal an external passive GPS antenna was used. This antenna was mounted at the roofof a car and ensured a stable lock of at least 4 satellites at each time.
The Data Interface feeds the GPS positions and time date in the NMEA format. This data arrives into a NUC viaUSB interface and is then stored alongside with the RESCUE logs.
MOBOTIX Q24M fish-eye camera provided pictures of the outdoor environment with the frequency of at least 1Hz. These pictures were also stored for each test run separately.
Page 67 (87)
RESCUE D4.4, v1.0
(a) (b)
Figure 4.19: Frequency error correction with the GPSDO external reference. fT X = 2.5302GHz, fRX without(a) and with (b) external reference attached to both nodes.
Since NUC, USRP and fish-eye MOBOTIX camera all use Ethernet protocol for communication, it was requiredto create an on-board network within every RESCUE testbed. This is accomplished by using the TP-LINK TL-SG108 1-Gigabit Ethernet switch. This also allows connecting to the NUC via cable over VNC or SSH protocole.g. from an external Laptop to start RESCUE script, copy measurement data and perform other maintenancetasks.
Conducting dynamic (with moving RESCUE nodes) V2V evaluation runs is not possible by a single person -we inevitably need at least one person per vehicle to drive it. It is only possible to reduce the number of peoplerequired to operate RESCUE testbeds. A straightforward approach would be to have one more person in eachvehicle, who would start RESCUE software and would communicate to each other to start the software in therequired order. This approach would require at least 4 persons to run TS0, 6 people to run TS1, and even more forthe other Toy Scenarios.
This number can be reduced if starting the software is automated. Then we would need only 3 persons for TS0,4 persons for TS1 (3 drivers + 1 operator) and so on. This can be achieved by accessing NUCs remotely to startRESCUE software. Despite the experiments were planned to be conducted in TU Ilmenau campus, which isalready almost completely covered by Wi-Fi, we decided to use 3G network to provide more stable wireless link(handover between 3G base stations is generally more reliable than handover between Wi-Fi access points), andincrease flexibility of our testbed and next time run the V2V evaluation e.g. in the city center if necessary. Thisapproach has also its drawbacks: the latency and bandwidth of the wireless links (used for remote access) in caseof 3G is significantly reduced compared to Wi-Fi (it takes sometimes up to 2 seconds to propagate the commandto a RESCUE node over 3G network). We used 3G USB dongles Huawei E170 and O2 accounts to implement theremote access.
Because of the reduced bandwidth, the only reasonable option to provide remote access to the RESCUE nodesis SSH protocol. This solution also allows us operating NUC without local monitor, keyboard and mouse in socalled ”headless” mode, which further reduces the size of RESCUE testbed.
To be able to achieve statistical significance, similar to OTAinVEE verification, some tests, depending on thelink quality, can take much time. During this time the nodes should continue moving, which would require thedrivers to stay concentrated on the road for some tens of minutes, or even a couple of hours. It is then helpfulto inform the drivers, as soon as a measurement has started or finished or whether a software run is taking placeat the moment. Although usage of handheld transceivers would probably be the most straightforward solution,we decided to automate communication between the remote operator and the vehicle drivers and in the same timeavoid ”out-of-range” problem when using simple handheld devices. Instead some assisting software, which runs inthe background on every NUC, was used. In essence this software monitors by the means of the operating system,if RESCUE software is started or some other actions are undertaken, and plays out a voice announcement aboutthe action via Audio Output of the NUC. Being connected to the vehicle audio system or to external speakers, thislooks similar to a standard navigation system. A free copy of such a software, called VoiceAssistant, was providedby the IT Workshop BUGOFF (Kharkiv, Ukraine). Using this approach the V2V experiments were automated,and the amount of human resources required to conduct RESCUE outdoor evaluation was significantly reduced.
After all the described components are integrated, the front side of the RESCUE testbed looks as shown in the
Page 68 (87)
RESCUE D4.4, v1.0
figure 4.20.
Figure 4.20: V2V RESCUE testbed layout
The software configuration of the RESCUE V2V testbed is very much similar to those used for OTAinVEEverification and indoor safety validation campaigns:
• xUbuntu 14.04.5 LTS;
• UHD v. 3.10;
• GNU Radio v. 3.7.10;
• latest version of RESCUE software;
• SSH-server and client applications;
• VoiceAssistant software.
For the V2V evaluation campaign 3 almost identical (except for power source for the GPSDO) RESCUE testbedswere assembled. These were installed in the back compartment of the measurement vehicles, the magneticmounted (2.5 GHz and GPS) antennas as well as the fish-eye cameras were installed on the rooftop (see fig-ure 4.21) and attached to the respective RF and data ports of the RESCUE testbeds.
4.3.3 V2V Test Plan
A real-world V2V experiment is significantly more complex than conducting emulation of RESCUE system oper-ation with V2V channel model in OTAinVEE. This experiment requires much more effort for its preparation, andmore resources to conduct it.
Therefore, within the given time, it was only possible to carry out a very limited number of V2V test runs. Due to
Page 69 (87)
RESCUE D4.4, v1.0
(a) (b)
Figure 4.21: RECUE node for V2V experiments
various complications with RESCUE software and its integration with USRP hardware, described in the previoussections, which had to be mitigated within WP4, the V2V trials have effectively been conducted only within thelast two weeks of the project and delayed the submission of the project by one month.
Therefore, the results offer neither statistical significance, nor full variety of different RESCUE cases. Theseresults should be treated as a showcase or validation of the RESCUE implementation in a real-world V2V com-munication scenario.
To draw any conclusion about possible gains from the RESCUE technology in V2V scenario, further tests mustbe conducted.
In the V2V evaluation campaign, the SNR is set through distance (amplitude, TX and RX gains are constant).Every run contains 300 frames.
4.3.4 Post-processing Experimental Data
In contrast to the previous verification (OTAinVEE) and validation (Indoor Safety) test cases, the radio channelconditions in the V2V are expected to severely change from one frame to another. The main reasons for this aredynamic environment (moving RESCUE nodes, and other objects around), rich multi-path channel due to multiplereflections etc. Therefore, in V2V validation it is important to analyze not only overall statistics for a specific run,but also look deeper at what is happening at the level of individual frames. To achieve this, TU Ilmenau developedadditional post-processing script, which jointly analyzes the logs produced by the RESCUE Source, Relay andDestination nodes, and plots the desired KPI.
For every KPI the post-processing script creates four subplots: one for each of S-D, S-R, R-D links plus onesubplot for the overall performance of the system - combining 2 different paths according to the specified baseline
Page 70 (87)
RESCUE D4.4, v1.0
Figure 4.22: Time-State diagramm for an example TS1 RESCUE-JD run
or RESCUE strategy. In case of TS0 there are only two subplots, which contain identical information, the subplotsfor S-R, R-D links remain blank.
One additional feature called ”Time-State Diagram”, which is implemented in the post-processing script allows tolook even deeper into the sub-frame level. An example of such a diagram for TS1 validation run is shown in thefigure 4.22. The diagram is organized (on the Y-axis) in three lines, which correspond to the three RESCUE nodes,the X-axis represents time. On the diagram one can see numerous markers, which represent different events, whichoccur in each of the communicating nodes. For example, the first marker in the line called ”Source” representsthe event when the Source node transmit the first frame. After that simultaneously the Relay and the Destinationreceive and decode the frame header and after some time the frame payload. We can notice that the Destinationnode required more time for this, since it additionally checks whether there were copies of the same frame receivedpreviously. After the payload of the first frame is successfully decoded at the Destination, the Destination sendsan ACK frame, which is simultaneously received at the Source and at the Relay. After some time the Relay willforward this ACK to the Source node once again (Relay always forwards all the ACKs it receives since it cannotknow if this was successfully received by the Source before).
The time-state diagram provides some important functions: it allows to verify that the PHY+MAC functionality isimplemented correctly and corresponds to the desired behavior; it potentially (although not always) gives a simpletool to identify possible frame collisions; it gives some insight into which operation takes approximately whatamount of time.
The post-processing script shows its power especially when analyzing results of V2V experiments. It will be usedin the next section to represent the results of RESCUE validation in the dynamic V2V TS1 case.
4.3.5 TS1 MAC Dynamic
Assuming limited time available for the V2V experiments, it was for us most interesting to take a look andcompare TS1 results for RESCUE-LF against the baseline (see definition in section 3.4). In both cases there wereno retransmissions allowed to have a clearer look at the performance of the compared strategies.
The route and constellation of the RESCUE nodes are picked in such a way to emulate a crossing scenario withcars approaching it from different sides. In this experiment the Relay remained stationary, while Source and Des-tinations were moving in circles around the campus usually having non-line-of-sight (NLOS) condition betweenthem (see Fig. 4.23).
The Source, Relay and Destination nodes’ software was started with the following parameters.
Page 71 (87)
RESCUE D4.4, v1.0
Figure 4.23: V2V RESCUE TS1. Route for the nodes in the dynamic test.
For the Baseline:
Source:
./rescue_mac_tx.py -d --trace --mac-addr=1 --external-clock--samp-rate 1000000 -l 1000 -c 300 --ampl 0.7 --rx2-antenna--rx-gain 30 --tx-gain 30 --freq 2530000000 --usrp-addr=addr=192.168.10.41--strobe-interval 200 --max-arq-attempts 0
Relay:
./rescue_mac_rx.py -d --no-lossy-forwarding --no-joint-decoder--trace --mac-addr=2 --external-clock \\ --samp-rate 1000000 -l 1000-t 600 --ampl 0.7 --rx2-antenna --rx-gain 30 --tx-gain 30--freq 2530000000 --usrp-addr=addr=192.168.10.32
Destination:
./rescue_mac_rx.py -d --no-joint-decoder --trace --mac-addr=3
Page 72 (87)
RESCUE D4.4, v1.0
--external-clock --samp-rate 1000000 -l 1000 -t 600 --ampl 0.7--rx2-antenna --rx-gain 30 --tx-gain 30 --freq 2530000000--usrp-addr=addr=192.168.10.42
50 100 150 200 250 3000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8HER = 0.03FER = 0.087
Frame seq. #
Del
ay,[
s]
Decoded framesLost frames
50 100 150 200 250 3000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8HER = 0.159FER = 0.186
Frame seq. #D
elay
,[s]
Decoded framesLost frames
50 100 150 200 250 3000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8HER = 0.157FER = 0.31
Frame seq. #
Del
ay,[
s]
Decoded framesLost frames
50 100 150 200 250 3000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8HER = 0.107FER = 0.133
Frame seq. #
Del
ay,[
s]
Decoded framesLost frames
Figure 4.24: TS1-MAC-V2V dynamic test (Baseline) Delay, S-R (top left), R-D (top right), S-D (bottom left),overall (bottom right)
For the RESCUE-LF:
Source:
./rescue_mac_tx.py -d --trace --mac-addr=1 --external-clock--samp-rate 1000000 -l 1000 -c 300 --ampl 0.7 --rx2-antenna--rx-gain 30 --tx-gain 30 --freq 2530000000 --usrp-addr=addr=192.168.10.41--strobe-interval 200 --max-arq-attempts 0
Relay:
./rescue_mac_rx.py -d --trace --mac-addr=2 --external-clock--samp-rate 1000000 -l 1000 -t 600 --ampl 0.7 --rx2-antenna--rx-gain 30 --tx-gain 30 --freq 2530000000 --usrp-addr=addr=192.168.10.32
Destination:
Page 73 (87)
RESCUE D4.4, v1.0
./rescue_mac_rx.py -d --trace --mac-addr=3 --external-clock--samp-rate 1000000 -l 1000 -t 600 --ampl 0.7 --rx2-antenna--rx-gain 30 --tx-gain 30 --freq 2530000000 --usrp-addr=addr=192.168.10.42
It is important to mention that in the V2V case we had exactly the same forward and backward link qualities.I.e. ACK frames would be transmitted over the same lossy channel without a guarantee of error-free arrival at theSource or at the Relay (as it was done in the indoor safety scenario).
The results for one of the dynamic runs of TS1 with baseline strategy are plotted in the figure 4.24. A similar plotfor the RESCUE-LF strategy is shown in the figure 4.25. These plots show the instantaneous delays in processingindividual frames being transmitted over each of the S-D, S-R, R-D links as well as the average HER and FERof these transmissions. The Y-coordinate of the red markers reflects the average delay over the given link. In thebottom right plot the overall one-way system delay is plotted together with the effective average HER and FERover the transmission of 300 frames.
50 100 150 200 250 3000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8HER = 0.01FER = 0.06
Frame seq. #
Del
ay,[
s]
Decoded framesLost frames
50 100 150 200 250 3000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8HER = 0.528FER = 0.615
Frame seq. #
Del
ay,[
s]
Decoded framesLost frames
50 100 150 200 250 3000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8HER = 0.443FER = 0.543
Frame seq. #
Del
ay,[
s]
Decoded framesLost frames
50 100 150 200 250 3000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8HER = 0.293FER = 0.350
Frame seq. #
Del
ay,[
s]
Decoded framesLost frames
Figure 4.25: TS1-MAC-V2V dynamic test (RESCUE-LF) Delay, S-R (top left), R-D (top right), S-D (bottomleft), overall (bottom right)
The methodological problem of evaluating results of real-world dynamic V2V experiments and comparing therebyperformance of different RESCUE strategies is that the dynamic V2V tests are generally non-reproducible. Sincethe environmental conditions and dynamic constellation of the RESCUE nodes is unique, it is generally unfair tocompare RESCUE strategies on some small set of transmitted frames (like 300 frames in each of our runs).
To somehow mitigate this problem a new metric called normalized payload error rate (δ = FER−HERHER ) is introduced
here. The nominator contains the fraction of frames which were marked as corrupted because of the errors in their
Page 74 (87)
RESCUE D4.4, v1.0
payloads (but not in headers). Dividing this by HER shows, how far from the ”RESCUE region of operation” weare. If δ ≈ 0, then the SNR is too low - in almost all the cases whenever a frame was lost, it was lost completely(neither payload nor header could be decoded). If δ −→ ∞ then no communication with baseline system ispossible (all the headers are decoded, but all the payloads are corrupted). This is the region in which RESCUEshould provide the highest gain.
For our dynamic V2V TS1 case it is easy to see that compared to the baseline where δB = 0.24, RESCUE LF pro-vides slightly better results (δR = 0.19). However, similar to the previous verification (OTAinVEE) and validation(indoor safety) studies, it is difficult to claim the impact of lossy forwarding upon the overall results.
Page 75 (87)
RESCUE D4.4, v1.0
5. Final Assessment
In this chapter, we summarize all the performance results obtained in the RESCUE project during the variousstages of validation. We begin by comparing the scenarios and methodologies used in the course of the project.Then, we perform a side-by-side comparison of the relevant results and we finish with generalized conclusionsfrom this analysis.
5.1 Comparison of Scenarios and Methodologies
The scenarios and methodologies used for validating the RESCUE implementation ranged across layers of theOSI model, network topologies, as well as the software and hardware tools used (Figure 1.1). These criteria andmethodologies are summarized in Table 5.1, which also includes the reference to the corresponding RESCUEdeliverable. First, after the implementation of the RESCUE coding algorithms in C++, they were validated inpoint-to-point links (TS0). Then, the coding algorithms were integrated into GNU Radio and evaluated by sim-ulations: first individually, then with the full PHY layer implementation of RESCUE. These evaluations werereported in D2.3. In parallel, work was done in WP3 regarding the higher layers. The networking stack, alongwith a PHY layer abstraction, was implemented in ns-3 and evaluated in D3.2. In the next step, the RESCUEnetwork stack was implemented in GNU Radio and integrated with the PHY layer implementation. GNU Radiosimulations for this case were reported in D3.3. Finally, in WP4 we moved from GNU Radio simulations toover-the-air radio environments with the OTAinVEE, indoor public safety, and V2V real field scenarios.
In order to precisely characterize and understand obtained results during the whole project, we compare the overthe air performance to simulations. The objective here is to identify and generalize behaviour (clearly absolutevalues can differ) for assessment and conclusions.
In terms of evaluated topologies, we focused on TS0 (for basic validation of the implementation) and TS1 (forvalidating lossy forwarding with a relay node). However, the flexibility of the network simulators allowed us totest multiple topologies. In ns-3 we were able to test: three toy scenarios (TS1-TS3), the routing toy scenario(RTS), multihop topologies (representative of either a public safety scenario or the mobile V2V case), and linetopologies (for end-to-end ARQ performance measurements). Furthermore, in the GNU Radio simulations it waspossible to evaluate a subset of these topologies. The over-the-air radio environments reported in this deliverablehad limitations which enabled only the validation of TS0 and TS1.
In all cases the main input parameter was the (average) link SNR between nodes. In the simulations furtherparameters were used as input (such as the number of nodes). In terms of KPI, for PHY layer evaluation the FERwas the main metric, for higher layers – the throughput.
5.2 Performance Assessment
The side-by-side comparison of results can be done for three scenarios: TS0 PHY, TS0 MAC, and TS1. Thiscomparison is between previously obtained results and results from the over-the-air radio experiments reported inthis deliverable. Table 5.2 provides an overview of the origin of the results. For each scenario under comparison,we selected a single representative result from each of the previous sections.
For the TS0 PHY case, we have three sets of results (Figure 5.1): from the GNU Radio simulations in D2.3, fromthe OTAinVEE tests, and from the indoor scenario. The first two used an AWGN channel, while the last one – afading channel. We compare the FER for the header as well as for the whole data frames in the case of a singledecoded frame (FER(s) – the baseline) and for joint decoding (FER(j)). The first observation is that the HERperformance is different for the GNU Radio simulations (Figure 5.1a). This is because during those experimentsthe coding rate of the header was 1/4. Since then, it become necessary for over-the-air tests to improve theheader’s coding rate to 1/6. Otherwise, we observe that the simulation and experiment results agree quite well.For example, in terms of performance at FER= 10−3, the indoor scenario tests confirmed the simulation results:in Figure 5.1c we observe that FER= 10−3 is achieved at 5.5 dB for the baseline and at 3 dB for joint decoding.Thus, we conclude that RESCUE joint decoding provides a gain of over 2 dB. However, the gain observed in theOTAinVEE tests was closer to 1 dB.
Page 76 (87)
RESCUE D4.4, v1.0
Table 5.1: Comparison of evaluation scenarios and methodologies
Type Dx.x Topologies Input parameters KPI
Codingalgorithms (C++)
D2.3 TS0 Link SNR FER
Codingalgorithms (GNURadio)
D2.3 TS0 Link SNR FER
PHY Layer (GNURadio)
D2.3 TS0, TS1 Link SNR FER
Network stack(ns-3)
D3.2 TS1, TS2, TS3,RTS, V2V, PS, Line
Link SNR, no. of nodes, nodedistance/speed, transmit power
Throughput, FER,delay, jitter
Network stack(GNU Radio)
D3.3 TS1, TS2, TS3,RTS, Line
Link SNR, no. of nodes Throughput, FER
OTAinVEE D4.4 TS0, TS1 Link SNR, channel type Throughput, FERIndoor D4.4 TS0, TS1 Link SNR (node distance, TX/RX
gain)Throughput, FER,delay, retransmissions
V2V D4.4 TS0, TS1 Link SNR (node distance) Throughput, FER,delay, retransmissions
Table 5.2: Comparison of evaluation results
Scenario Previous results OTAinVEE Indoor
TS0 PHY D2.3 Section 3.7.1 Section 4.2.2TS0 MAC Performed for D3.3 but not reported Section 3.7.2 Section 4.2.3TS1 D3.3 Section 3.7.3 Section 4.2.4
Adding the MAC layer allowed us to assess the RESCUE performance in terms of throughput. For the TS0 case,we compare the GNU Radio simulations (conducted for D3.3, though not reported therein), the OTAinVEE, andthe indoor scenario test (Figure 5.2). As before, the first two used an AWGN channel, while the last one – a fadingchannel. Also, the simulation and experiment results again agree quite well. Figures 5.2a and 5.2b present resultsfor AWGN channel models – they exhibit a steep “turbo-cliff” for the baseline which means that below a givenSNR no communication is possible. For the indoor testbed (Figure 5.2c), this behaviour disappears: there is nosteep cliff for the baseline. In this case, the single-decoder baseline moves closer to the joint decoding strategy.Here, the RESCUE “area of operation” was about 2 dB (from 3 to 5 dB) during the simulations and OTAinVEEtests. However, for the indoor case the throughput gain seems to more consistent over a wider SNR range.
For TS1, we compare the case when the relay is in position B as this allows the lossy forwarding (RESCUE-LF) strategy to differ from the other proposed ones (in position A the relay receives lossless frames). In thiscase we have four sets of results (Figure 5.3): from GNU Radio simulations, from OTAinVEE with two channeltypes (AWGN and fading), and for the indoor scenario (for SNRSR = 4 dB). The GNU Radio simulations used anAWGN channel and the results are similar to OTAinVEE for AWGN (with the previously described “turbo-cliff”experienced by the baseline). The only exception is the RESCUE-LF strategy obtaining slightly higher throughputin the range of 1–2 dB of SNR. The fading channel is a more accurate model of a real channel. Therefore, theOTAinVEE (fading channel) results are similar to the indoor scenario. However, the coding gain is smaller thanin the case of AWGN channels. Nonetheless, the indoor scenario results show that the baseline provides (almostor exactly) zero throughput for low quality links, whereas RESCUE can maintain connectivity. Also, in the GNURadio simulations a coherent QPSK modulation was used. In the other cases DE-QPSK was used. Therefore,there is a 3 dB shift in the SNR ranges of RESCUE’s applicability. Comparing the three RESCUE forwardingstrategies, we observe that, with the exception of the GNU Radio simulations, the throughput performance for lowSNR values is ordered as follows: RESCUE-JD, RESCUE-SDF-CRC, and RESCUE-LF.
Page 77 (87)
RESCUE D4.4, v1.0
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
HERFER(s)FER(j)
(a) GNU Radio simulations (AWGN channel)
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
HERFER(s)FER(j)
(b) OTAinVEE (AWGN channel)
0 1 2 3 4 5 6 710−3
10−2
10−1
100
SNR [dB]
FER
HERFER(s)FER(j)
(c) Indoor scenario (fading channel)
Figure 5.1: Comparison of TS0 PHY performance.
Page 78 (87)
RESCUE D4.4, v1.0
2 3 4 5 6 7 80
10
20
30
SNR [dB]
Thr
ough
put[
kb/s
]
RESCUEBaseline
(a) GNU Radio simulations (AWGN channel)
2 3 4 5 6 7 80
10
20
30
SNR [dB]
Thr
ough
put[
kb/s
]
RESCUEBaseline
(b) OTAinVEE (AWGN channel)
2 3 4 5 6 7 80
10
20
30
SNR [dB]
Thr
ough
put[
kb/s
]
RESCUEBaseline
(c) Indoor scenario (fading channel)
Figure 5.2: Comparison of TS0 MAC performance.
Page 79 (87)
RESCUE D4.4, v1.0
−2 −1 0 1 2 3 4 50
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
RESCUE-LFRESCUE-SDF-CRC
RESCUE-JDBaseline
(a) GNU Radio simulations (AWGN channel)
1 2 3 4 5 6 7 80
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
RESCUE-LFRESCUE-SDF-CRC
RESCUE-JDBaseline
(b) OTAinVEE (AWGN channel)
1 2 3 4 5 6 7 80
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
RESCUE-LFRESCUE-SDF-CRC
RESCUE-JDBaseline
(c) OTAinVEE (fading channel)
1 2 3 4 5 6 7 80
10
20
30
SNRSD [dB]
Thr
ough
put[
kbit/
s]
RESCUE-LFRESCUE-SDF-CRC
RESCUE-JDBaseline
(d) Indoor scenario (fading channel)
Figure 5.3: Comparison of TS1 performance (relay in position B). For all cases, SNRSR=SNRSD, except (d)where SNRSR = 4 dB. Also, in (d) the data points were omitted to make the fitted curves morevisible.
Page 80 (87)
RESCUE D4.4, v1.0
5.3 Conclusions
The result of the implementation work in the RESCUE project is a suite of algorithms and protocols operating atthe physical, data link, and network layers. Together, they enable the paradigm of lossy forwarding. The variouselements of this suite were tested using a variety of methodologies of increasing complexity. This allowed us tobalance between realism and simplification as well as costs while moving from theory to practice without losingreliability of the results. In terms of the tools used, we conclude that:
• the ns-3 simulator allowed to verify the RESCUE network stack under the largest set of scenarios; however,it used only an abstraction of the underlying coding algorithms,
• GNU Radio allowed running both simulations as well as over-the-air tests with USRP devices and devel-oping in this tool eased the transition from simulated to emulated and real radio links; however, its higherlayer implementations were not as good as in ns-3.
In terms of measurable impacts related to validation, we have achieved the following:
• Tested the functionality of the integrated modules of WP2 and WP3 against simulation based results – theresults agree with each other (taking into account the limitations of each method).
• Validated at least one modulation and coding scheme related to the PHY layer, one MAC, one ARQ and onerouting protocol in the SDR platform – the separate validation of each protocol in GNU Radio was reportedin previous deliverables (D2.3 and D3.3).
• Integrated the designed physical layer, MAC, and routing protocols over the SDR platform in order to testreal message transfers over RESCUE scenarios initially defined in WP1 and adapted to realistic experi-mentation environments in WP4 – due to the time-consuming nature of the GNU Radio experiments onlyselected scenarios could be validated. This precluded analyses of multi-hop networks which would allow usto validate the routing protocols. Nonetheless, the agreement of the ns-3 and GNU Radio-based simulationresults confirm that the routing and ARQ design was appropriate.
In summary, we were able to demonstrate the effectiveness of links-on-the fly in the presence of lossy links via anentirely rethought PHY/MAC/NET cross-layer approach. The presented results confirm that for a link with lowaverage SNR, the RESCUE approach maintains connectivity while the baseline throughput is (almost or exactly)equal to zero. We also noted that this gain was diminished as more realistic conditions were considered. Webelieve however, with multiple available relays (scenarios TS2 and TS3 with two or more relays) the gain mayslightly increase.
Page 81 (87)
RESCUE D4.4, v1.0
6. Summary & Conclusion
6.1 Summary
D4.4 presents the overall integration of the RESCUE architecture onto the SDR platform, the verification of theRESCUE architecture through the use of the OTAinVEE facility, the validation using experiments in real fields forboth indoor public safety and outdoor V2V scenarios and the final assessment of the RESCUE concept with testcases using realistic indicators. The results of this deliverable reveals the outcomes of the research from variouswork packages within RESCUE project. The integration, verification, validation and assessment of the RESCUEarchitecture has been carried out on simple network models regarded to as toy scenarios and the vehicle-to-vehicletestbed. In order to provide a holistic preview to the work in this deliverable, a brief recap of achievements ofvarious WP are provided below, along with their contribution towards the final assessment in D4.4.
WP1 mainly conducted the theoretical performance limit analysis on different kinds of cooperative networks,which exploit the lossy forwarding relaying strategy. In the one-way relay network, the achievable rate region wasdetermined by source coding with a helper, approximated by Slepian-Wolf theorem. The exact and approximatedoutage probabilities were calculated based on the aforementioned achievable rate regions. It was established thatlossy forwarding outperforms the conventional decode-and-forward relaying strategy with respect to outage prob-ability. One interesting finding was that with lossy forwarding concept the optimal relay position shifted to exactlythe midpoint between the source and destination. For the single-source multiple-relay and single-destination net-work without direct link between the source and destination, our work focused on the chief executive officer(CEO) problem, where all the source-to-relay links are lossy. We first reduced the binary CEO problem to a bi-nary multi-terminal source coding problem. Then, we derived the tighter outer bound on the rate distortion regionfor the binary multi-terminal source coding problem based on the converse proof of the bound. Furthermore, alower bound on the Hamming distortion for the CEO problem was obtained by minimizing the distortion functionsubject to the inequalities between the derived outer bound and the channel capacities. Finally, an extension of thebinary CEO problem to an arbitrary number of terminals was investigated. For the single-source multiple-relayand single-destination network with direct link between the source and destination, the challenges were sourcecoding with multiple helpers. It was extremely challenging to solve this problem. However, upper bound of theoutage probability was achieved by reducing and relaxing the rate constraints based on Slepian-Wolf theorem.Closed form expression for the outage probability was derived for the high signal to noise ratio regime when thenumber of relays is less than five. For the multiple access relay channel (MARC), three different cases, which are:(i) perfect source-to-relay links with orthogonal transmission, (ii) imperfect source-to-relay links with orthogonaltransmission and (iii) imperfect source-to-relay links with non-orthogonal transmission were investigated. Theoutage probabilities were calculated based on source coding with a helper for the first and second cases while theoutage probability for the last case was obtained based on multiple access channel with a helper. It was shown thatthe outage performance of the non-orthogonal MARC approached its orthogonal counterpart with perfect source-to-relay links while the throughput of the non-orthogonal MARC network could be significantly enhanced.
In WP2, the design of the practical error correction coding/decoding that can efficiently utilize the network corre-lation at the physical layer (PHY) was one of the major research items in the RESCUE project. The key idea wasto extend the conventional distributed turbo coding approach to be resilient against the errors occurring during theintermediate hops under specified conditions. These conditions are either the existence of the direct component,i.e., one of the signal copies needs to be error free before the last hop, or the number of erroneous copies needsto be sufficiently large. The research work carried out in the RESCUE project focused on simple network modelsreferred to as toy scenarios (TSs) defined in D1.2.1 [13]. It was already known from literature that cooperativerelaying showed benefits in low SNR region. The cost of additional spatial degrees of freedom is the increasedoverhead which starts to dominate when the quality of the direct channel between the source and the destinationincreases. Therefore, we focused on investigating the low SNR region thereby implying the reliability aspect ofthe communication. The work reported in D2.1.1 provided the initial results and descriptions of the coding/decod-ing and source-correlation estimation algorithms considered in the RESCUE project. The state-of-the-art codeswere reported to perform very close to the theoretical limits. Therefore, the remaining challenge of the designof coding/decoding algorithms were related to practical aspects, such as finding the optimal coding parametersand efficient correlation estimation algorithms. Several novel correlation estimation algorithms were reported inD2.1.1 and D2.3 provided detailed descriptions for the design of coding parameters. Deliverable D2.3 also de-scribed the implementation of RESCUE PHY in the software defined radio (SDR) platform. This work was veryuseful in order to find out the practical problems related to lossy forwarding (LF) concept. One of the major prob-
Page 82 (87)
RESCUE D4.4, v1.0
lems observed at this stage was the header error control, since an error-free header was needed to properly processthe received data frame. Hence, the error protection of the header was increased for later experiments. Anotherimportant observation at this stage was that the experiments in low SNR region introduced problems in synchro-nization which had to be fixed for the later experiments. The development of the low SNR receiver processingas well as finding the alternative solutions for the header error problem were the necessary research directionsin order to increase the usefulness of joint decoding (JD). Within WP2, academic contributions have also beenmade by investigating optimal power allocation strategies for different toy scenarios assuming lossy forwardingand these were reported in D2.2.1 [16] and D2.2.2 [17]. Due to the limited project resources, these algorithmswere not implemented to the experimental setup considered in this deliverable. However, it was observed that ifwe can cooperatively balance the resources between the nodes, the performance improvement might be signifi-cant. The final report of coding/decoding algorithms considered in RESCUE was provided in D2.1.2 [18]. It wasobserved that in TS1 with geometric path loss assumption, it was often better to retransmit from the source ratherthan forwarding the lossy frames from the relay. The conclusion drawn in D2.1.2 is that LF is beneficial in thecases where the retransmission is not possible, or in cases where it is preferable to avoid using the same nodefor retransmission. As a conclusion on RESCUE coding/decoding algorithms, it was stated that the code is verysimple but remarkably strong performing close to the theoretical limits. It was also mentioned that this is a newcode design and up to the authors’ best knowledge, there is no existing work with this extend.
Within WP3, message transfer design has been an important component in demonstrating the concept of lossyforwarding in a working prototype. From the onset, the decision was made to go for a distributed and ad-hocnetworking approach (typical for both public safety and vehicle-to-vehicle communications). Among the crucialinitial choices were selecting a packet-based approach, adopting a single-channel scheme, and allowing for dis-tributed control. These choices allowed the move from theoretical, stochastic performance metrics (such as frameerror rate), to more real-world, end-user metrics (such as throughput). Incorporating the message transfer designwith the lossy forwarding concept allows us to see what services can be supported under what kind of environmentsettings. The design of the data link layer protocols, tightly coupled with the design of protocols at the physicaland networking layers has been a non-trivial task, which was assisted through a step-wise approach to the designprocess. We began with a theoretical design in D3.1, followed by an implementation and verification in a networksimulator in D3.2. Next, the solutions were implemented and validated separately through simulations on an SDRplatform as reported in D3.3. Finally, they have been integrated and tested over-the-air as reported in this deliver-able. From this approach, we conclude first that simulations can aid with the design process by helping establishoptimal parameters and identifying bugs in the code, and second that the over-the-air tests demonstrate the morepractical applicability of the protocols under evaluation. The main result of the performed experiments led to theidentification of the RESCUE ”area of operation”, i.e., the radio and topology conditions, where conventionalcommunications practically ceases to exist while the RESCUE approach provides connectivity. Additionally, wealso discovered that compounded errors arising from using consecutive lossy links are difficult to handle. There-fore, joint decoding at relays was recommended whenever possible. This could also be augmented with the partialARQ protocol described in D3.2. Consequently, adopting the lossy forwarding concept to multi-hop multi-pathtopologies incurs a large overhead, not necessarily outweighed by the costs. In this aspect, the focus on the ”toyscenarios” was a good decision, as all multi-hop topologies can be broken into such basic scenarios.
Moving on to the implementation process, it was concluded that while GNU Radio is a good tool for developingand prototyping software-defined radio, it was found lacking a reliable network stack. Conversely, the ns-3 net-work simulator provided a full network stack for modification, but lacked the required coding algorithms neededby RESCUE as well as a more realistic radio channel. From a cross-layer design perspective, it was difficult tofine-tune the various performance parameters involved as there was no one-size-fits-all set of parameter config-urations, especially when taking multi-hop topologies into account. This difficulty was also compounded by thetime-consuming nature of the SDR experiments. Finally, from the message transfer perspective, we had all therequired building blocks within a prototype device which supports the lossy forwarding functionality.
Now in D4.4, above contributions from WPs have been integrated and tested over-the-air as reported in this deliv-erable. This deliverable begins with details of integration activities of the RESCUE architecture, the fundamentalsof the transceiver used in RESCUE and a discussion of major issues, challenges and the solutions offered in theintegration activities. The verification activities within the OTAinVEE facility is also described with key perfor-mance indicators, and results from the toy scenarios and defined scenarios are presented. The validation resultsfrom the two chosen deployment scenarios; Indoor Safety and V2V conducted at AGH and TUIL respectivelyare also presented with a final assessment discussion. The OTAinVEE was verified and calibrated to ensure itsaccuracy for the various verification and validation exercise with sanity checks conducted at intervals to ensure
Page 83 (87)
RESCUE D4.4, v1.0
consistent performance. Various KPI such as FER, throughput, delay and average retransmission count havebeen selected to cover both the PHY and MAC layers. All experiments have been conducted within a range ofSNR with the lower SNR range of 3-6dB regarded as RESCUE’s “area of operation”. In the verification of theRESCUE concept within the OTAinVEE environment, it was deduced that RESCUE lossy forwarding schemesand RESCUE selective-decode-forward with CRC performed marginally better than baseline strategies. The jointdecoding schemes however demonstrated comparatively better performances within the limit of the experimentperformed. The validation of the RESCUE architecture in the real field was carried out for both the indoor publicsafety scenario and the V2V. The indoor scenario reported three series of experiments namely: TS0 PHY, TS0MAC, and TS1.
For TS0 PHY, an additional KPI; Header Error Rate (HER) was used in the performance evaluation and theFER was further distinguished as FER(s) and FER(j) to represent without joint decoding and joint decodingrespectively. For the various duplicate counts and the links, it was deduced that generally, over the range of SNRunder test the joint decoding scheme has a better performance compared with the scheme with no joint decodingor single decoder. The HER values of the links differ because of the varying quality of the radio channels inthe experiments which were difficult to reproduce. For the TS0 MAC, building on the validated operation ofthe TSO PHY layer scenario and using the KPIs (throughput, FER, number of retransmission and delay) on thevarious point-to-point links, our experiments showed that the RESCUE scheme exhibited comparatively betterperformance to the baseline over a large segment of the lower range of SNR in the experiments. While the resultsshow that the performance of the different point-to-point links differs from one another, the results are consistentwithin SNR of RESCUE’s “area of operations”.
In the TS1 Experiments, the verification was done in a scenario with one single relay. Four schemes namely:Baseline, RESCUE-JD, RESCUE-LF and RESCUE SDF-CRC were evaluated. The results showed that improv-ing the value of SNR between the relay and destination reduces the gains that can be provided by RESCUE. Thegain from RESCUE is most visible when the relay to destination link quality is within the RESCUE “area ofoperation”. RESCUE’s outdoor V2V experiments examined the RESCUE architecture in a more dynamic andreal-life environment with unpredictable wireless propagation conditions. The obtained results showed that RES-CUE concept of lossy forwarding is mainly beneficial when the joint decoding is applied. This is similar to thetrend of results displayed in the OTAinVEE and Indoor safety scenarios.
Lastly, our work has investigated, experimented and verified the concepts of the links-on-the fly using lossy com-munication links via an entirely rethought PHY/MAC/NET cross-layer approach. The results presented con-firmed that for links with low-to-average SNR, the RESCUE approach maintains connectivity when the baselinethroughput is (almost or exactly) equal to zero. Our results also confirmed the importance of the joint decodingfunctionality to realize the gain of spatial diversity.
6.2 Conclusion and Synopsis of the RESCUE Architecture
RESCUE is one of the pioneering projects in Europe, with the ambition of making a radical shift in mobilenetworking from the lossless to lossy communication link design. The key innovation of RESCUE lies in theincreasing demand of providing reliable and efficient wireless services in the public safety areas, where wirelessenvironments are highly dynamic and largely unpredictable. The major technical challenge arises from the factof lacking network coordinators in the unpredictable environments, which renders accurate link budget allocationvery expensive or in most cases impossible. In such cases, wireless devices can form large-scale ad-hoc (or mesh)mobile networks, and transfer the messages through multi-hop and multipath routing.
The scientific vision of RESCUE lies in the hypothesis of having many misinformed intermediate nodes in thelarge-scale ad-hoc (or mesh) networks, such that none of the paths between a source-destination pair can delivera clean copy of the message. In other words, a destination may receive several copies of the message from itsneighbours; however none of the neighbours has a clean copy. This interesting networking behaviour can be mod-elled into a multiple access channel with correlated sources, of which the capacity upper bound as well as theoutage behaviour were analytically studied in WP1. Moreover, extensive theoretical investigations were carriedout in WP1, with the emphasis on capacity theorems and outage behaviours of four fundamental networking mod-els (toy scenarios): a) three-terminal relay channel, b) CEO model, c) CEO model with direct source-destinationlink, and d) a more complicated model with multiple destinations. As indicated in the WP1 conclusion, the RES-CUE concept of lossy forwarding well outperforms the conventional decode-and-forward concept, as the latter
Page 84 (87)
RESCUE D4.4, v1.0
has its performance limited by the source-relay link capacity. Moreover, the RESCUE lossy-forwarding conceptdemonstrated increasing and impressive gains with the increase of parallel paths.
The positive results generated in WP1 rather encourage WP2 to carry out intensive research towards distributedcoding and modulation techniques for the RESCUE lossy forwarding concept. The research focus was mainly onthe toy scenarios defined in the deliverable D1.2.1. Error probability (BER or FER) is one of the key measures,which was utilised for the appraisal of the communications quality. The interesting phenomena appeared inthe three-terminal relay channel. As expected, the RESCUE lossy-forwarding concept outperforms the fullydecode-and-forward protocol, as the latter does not enjoy the spatial diversity gain. On the other hand, the lossy-forwarding protocol did not perform better than the selective decode-and-forward protocol. This is neither asurprise nor a negative result. Instead, this result nicely supports the hypothesis of the RESCUE concept. Thereason is simple. From the cooperative relaying perspective, both protocols enjoy the full spatial diversity gain.However, the selective decode-and-forward protocol takes advantage of having a clean copy of the message fromthe source, and thus enjoys better performance. From the lossy-networking point of view, the three-terminalrelay channel is an over-simplified model, which cannot accurately describe the RESCUE operating scenarios. Toclarify this issue, we shall remind that RESCUE assumes the existence of many misinformed neighbours aroundthe destination, and the destination can hardly receive a clean copy of the message via established paths. However,in the three-terminal channels, the two neighbours connected to the destination carry 50% or more clean copiesof the message. Such a figure clearly conflicts with the assumption of the RESCUE concept. In order to furthervalidate our insight, WP2 also compared both protocols (i.e., selective decode-forward and lossy-forward) in thecase of having two successive relays, where relays can be easily misinformed. Despite still a simple model, itdiffers from the three-terminal relay channel in terms of the largely reduced ratio of clean copies. Therefore,this model is much closer to the hypothesis of RESCUE. Not surprisingly, it was shown that the RESCUE lossy-forwarding protocol outperforms the selective decode-and-forward protocol by 8 dB or more in this model; pleasesee the deliverable D2.1.2.
In addition, WP2 also discovered two interesting phenomena, which are perhaps worth mention. The first phe-nomenon appeared in the comparison between the theoretical BERs/BLERs obtained using the error rate modelpresented in D2.1.2, and the simulation results (see Figures 6.6 – 6.7 in D2.1.2). When the relay did not have er-rors, the simulation results well coincide with the theoretical results. However, when the relay was misinformed,the theoretical results serve as a lower bound of the simulation results. The lower bound is tight either at lowSNRs or high SNRs, but rather loose at moderate SNRs (i.e., -4 – 0 dB). This phenomenon occurs mainly due tothe use of approximate error model in theoretical analysis, which yields relatively optimistic error prediction atthe moderate SNRs. Due to the same reason, the theoretical results did not lead to a consistent conclusion withthe simulation results in terms of the best strategy of the relay placement. This encourages us to find a better errormodel in the future theoretical research.
The second phenomenon is observed from the simulation results illustrated in the deliverable [D2.1.2, Figure2.10]. It was shown that better FER was achieved when relays had fewer errors, and there was a considerably largegap between the lossless case and lossy cases. It is worth noting that this result neither implies the inefficiencyof the joint decoder nor the inefficiency of the lossy design. Instead, it encourages more links (relays) to beinvolved in the joint decoding so as to close the FER gap. By this means, we trade off the spectral efficiency forthe link reliability. In order to maintain the spectral efficiency in the presence of many relays, the opportunisticlossy-forward protocol was investigated in WP2, and it showed promising performance.
In order to validate the physical-layer algorithms developed in WP2, WP3 investigated new MAC protocols forthe RESCUE concept, and WP4 paid intensive efforts on the implementation of the lossy-forwarding algorithmsand protocols developed in WP2 and WP3. Concerning the limited resource and time scale, the major focus ofWP4 was decided to be on the over-the-air test of the three-terminal relay channel. This seemingly simple testcase actually carries lots of practical challenges of implementation, which include the selection of appropriatemodulation schemes, coding schemes, channel gain/SNR configurations, timing synchronisation between threeUSRPs, frequency offset estimation and correction, channel estimation and equalisation, accurate SNR estimation,and many other issues. These are not individual tasks that can be optimised separately. Instead, these are concretetasks, which must be carefully combined and jointly optimised. Moreover, it often took very long time to collect aset of reliable data for further analysis; and even a very minor mistake could trigger a rerun of the trials. The goodnews is; all the planned tasks were completed on time, and the test-bed results generated in WP4 nicely coincidedwith the simulation results obtained in WP2 and WP3.
Page 85 (87)
RESCUE D4.4, v1.0
In addition, with the advices received from the project reviewers, the RESCUE consortium discovered additionalchallenge of implementing the RESCUE concept, i.e., how to handle the PHY header in the lossy networkingdesign. RESCUE’s current approach is to employ a very strong code to protect the PHY header. By this means,the RESCUE team can focus their research on the main tasks, i.e., the physical layer protocols as promised in theDoW. However, the RESCUE consortium recognised it as a temporary solution, and found the lossy MAC designa very interesting and challenging research topic for the academic community as well as industry.
In summary, RESCUE project was a first (successful) research attempt targeting the lossy forwarding concept.Our work reached the real field test of an integrated PHY and MAC proof of concept and highlighted the lossyforwarding gains in many ”simple” scenarios. The obtained results show consistency between simulations, GNURadio emulations, OTAinVEE and over the air tests what makes us confident with the pertinence of the followedmethodology as well as the accuracy of the attained results.
We think that many things are still to be explored in this domain. We hope that the end of RESCUE will be thebeginning of many related research and more applied initiatives.
Page 86 (87)
RESCUE D4.4, v1.0
7. References
[1] ICT-619555 RESCUE Deliverable. D1.1 – System Scenarios and Technical Requirements. Technical report,ICT-619555 RESCUE Project Consortium, May 2014.
[2] ICT-619555 RESCUE Deliverable. D2.1.1 – Interim Report of Detailed Coding/Decoding Algorithms andCorrelation Estimation Techniques. Technical report, ICT-619555 RESCUE Project Consortium, October2014.
[3] ICT-619555 RESCUE Deliverable. D2.3 – Feasibility Assessment in Model Based Environments. Technicalreport, ICT-619555 RESCUE Project Consortium, November 2015.
[4] ICT-619555 RESCUE Deliverable. D3.1 – MAC and Routing Architecture and Interfaces Specification.Technical report, ICT-619555 RESCUE Project Consortium, November 2014.
[5] ICT-619555 RESCUE Deliverable. D3.2 – Report on revised WP3 architecture including simulation results.Technical report, ICT-619555 RESCUE Project Consortium, December 2015.
[6] ICT-619555 RESCUE Deliverable. D3.3 – Report on implementation of the WP3 architecture. Technicalreport, ICT-619555 RESCUE Project Consortium, April 2016.
[7] ICT-619555 RESCUE Deliverable. D4.1 – Report on Considered Scenarios for Performance Validation andField Trials. Technical report, ICT-619555 RESCUE Project Consortium, April 2014.
[8] M. K Simon. On the bit-error probability of differentially encoded QPSK and offset QPSK in the presenceof carrier synchronization. IEEE Trans. Commun., pages 806–812, May 2006.
[9] Tommi Laitinen, Pekka Kyosti, Jukka-Pekka Nuutinen, and Pertti Vainikainen. On the number of OTAantenna elements for plane-wave synthesis in a MIMO-OTA test system involving a circular antenna array.In Proceedings of the Fourth European Conference on Antennas and Propagation, pages 1–5. IEEE, 2010.
[10] Jukka-Pekka Nuutinen, Pekka Kyosti, Yu Gao, and Michael D. Foegelle. On the MIMO OTA test system.In Communications and Networking in China (CHINACOM), 2010 5th International ICST Conference on,pages 1–5. IEEE, 2010.
[11] A. J. Berkhout. A Holographic Approach to Acoustic Control. J. Audio Eng. Soc, 36(12):977–995, 1988.
[12] Google Scholar.
[13] ICT-619555 RESCUE Deliverable. D1.2.1 – Assessment on Feasibility, Achievability, and Limits. Technicalreport, ICT-619555 RESCUE Project Consortium, April 2015.
[14] William C. Jakes. Microwave mobile communications. Wiley, New York, 1974.
[15] ICT-619555 RESCUE Deliverable. D4.3 – Report on Channel Analysis and Modelling. Technical report,ICT-619555 RESCUE Project Consortium, August 2015.
[16] ICT-619555 RESCUE Deliverable. D2.2.1 – Interim Report of Detailed Optimisation Algorithm and Perfor-mance Comparisons. Technical report, ICT-619555 RESCUE Project Consortium, April 2015.
[17] ICT-619555 RESCUE Deliverable. D2.2.2 – Final Report of Detailed Optimisation Algorithm and Perfor-mance Comparisons. Technical report, ICT-619555 RESCUE Project Consortium, July 2016.
[18] ICT-619555 RESCUE Deliverable. D2.1.2 – Final Report of The Coding/Decoding Algorithms and Perfor-mance Comparison. Technical report, ICT-619555 RESCUE Project Consortium, August 2016.
Page 87 (87)