1

Optimizing TCP Forwarder Performance

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 8, NO. 2, APRIL 2000資工碩一 M9129018陳宏仁

2

Outline

Introduction TCP Forwarding Connection Splicing Connection Splicing In SCOUT Conclusion

3

Introduction

4

Introduction

TCP forwarder A network node that establishes and forwards data

between a pair of TCP connection TCP forwarding

Indirect TCP communication via a proxy Connection splicing

Improve TCP forwarding performance

TCP forwarder

TCP connection TCP connection

5

TCP Forwarding

6

TCP Forwarding

Proxy Mediate the communication Interpose between two connection Control the flow of data between the

communicating parties

Proxy has two mode Control mode Forwarding mode

Control Mode Forwarding Mode Control Mode

Processing control function

Move data between connection

Back to control mode

7

TCP Forwarding (cont.)

Proxy can be classified into four categories First

In control mode only during connection setup After connection setup, switch to forwarding mode

for the duration of connection Second

Authenticate the user or request Check user ID, password , and destination of the

Telnet request

FTP Proxy

Telnet Proxy

8

TCP Forwarding (cont.)

Third Remains in control mode for all data transferred in

one direction (HTTP proxy) Switch to forwarding mode for data transferred in

the other (HTTP server) Fourth

Remains in control mode and continuously monitors data passed in both directions

HTTP Proxy

Proxy

9

Firewall

Data from one network pass through the proxy which forwards them to the other network

If the desired security guarantees are not violated

10

Mobile Computing

Filtering data Reduce or remove too big data

When mobile host is connected to wired network Only relay data in forward mode

Allow a mobile host to change its point of attachment to network Mobile host can terminate TCP connections Move to new location with a new IP address Establish a new set of TCP connections to proxy

11

Connection Splicing

12

Connection Splicing

The basic idea of connection splicing To detect when a proxy makes a transition from control

mode to forwarding mode Splice two TCP connections together into a single

forwarding path through the system

Unoptimized TCP forwarder Optimized TCP forwarderWith spliced connection

13

Forwarding

Primary task on FWD processing step Change the header of incoming TCP segment to account

for the differences in the two original TCP connections

Source Port Destination Port

Sequence Number

Acknowledge Number

Data

OffsetReserved

URG

ACK

PSH

RST

SYN

FIN

Window

Checksum Urgent Pointer

Options Padding

Data

14

Forwarding (cont.)

From connection A to connection B Output.DstPort = RemotePortB

Output.SrcPort = LocalPortB

Output.SeqNum = Input.SeqNum + SeqNumOffsetA->B

Output.Ack = Input.Ack – SeqNumOffsetB->A

Output.Cksum = Input.Cksum + CksumPatchA->B

Connection A Connection B

TCP forwarder

15

Splicing

TCP buffers contain acknowledged data

Forwarder can’t let TCP acknowledge new data Give it more data to deliver reliably Impractical to wait until two connections go idle before

completing the splice

16

Splicing (cont.)

Two way to handle newly arriving segment during transition period Delay the activation of spliced connection until

after buffers have drained TCP acknowledge segments After transition is complete, buffered segments are

processed by FWD Allow FWD to begin forwarding data concurrently

with draining the buffers All newly arriving segments are delivered to both the

original TCP protocol and to FWD

17

Unsplicing

When the forwarding proxy switches from forwarding mode to control mode, connections must be unspliced

Difficult to decide when proxy should switch back to control mode Proxy has to find control information by looking at

out-of-order segments

18

Unsplicing (cont.)

Dealing with acknowledgements makes it difficult to unsplice a connection No acknowledged segment

Reconstruct TCP connections Acknowledged segment

Wait for all of segments be acknowledged Continuously monitor segment stream until copy all

unacknowledged segments

19

Flow Control

During unoptimized operation Flow control is handled by two independent TCP

protocols on forwarder, and TCP protocol on the end hosts

During optimized operation Flow control is handled by the end host only

TCP forwarder can restrict window size to avoid unnecessary retransmissions

20

Additional Optimizations

Connection splicing optimization can be applied not only at TCP level, but also to unfragmented IP datagram

Forwarder can process IP datagrams similarly to an IP router, with additional TCP segment header manipulation

21

Connection Splicing in SCOUT

22

Connection Splicing In SCOUT

SCOUT is a configurable OS explicitly designed to support data flow Video streams through an MPGE player A pair of TCP connections through a firewall

23

2-Path

As going from one path to another often will require a context switch

Like firewall structure

24

1-Path

Similar to 2-path configuration, except two network devices are connected by a single path

25

FWD

Optimized version of 1-path Splice into a single connection & forwarder is

reduced to updating TCP header Support reassembly of IP packets

26

IP/FWD

Further Optimized version of FWD Network level packets are modified directly and

forwarded Don’t support reassembly of IP packets

27

IP Router

Modify network packets directly in the same way as IP/FWD

Not update TCP header

28

In Linux Configuration

TIS firewall Offer full filter functionality, but use a null filter

Filtering IP router Filtering on IP addresses, protocol & port number Like IP/FWD case in SCOUT

IP router Basic in-kernel Linux IP forwarding with no

filtering

29

Test Setup

200MHz PentiumPro workstation 256KB cache, 128MB RAM Digital Fast EtherWORKS PCI 10/100 32-bit PCI

10/100 MB/s adapters Linux version 2.0.30

30

Processing Overhead

Back-to-back latency & network interface latency

31

Processing Overhead (cont.)

Summarizes the processing of a single packet in firewalls and routers for both SCOUT & Linux

32

Aggregate Throughput

Measure aggregate throughput of one, two, and three concurrent TCP connections over 2-path & IP/FWD

Packet is 1460 bytes in 100Mbit Ethernet

Mbyte/S

33

Cost of Unsplicing

First Fix up TCP header during spliced operation FWD keeps track of SN, ACK number, window of

spliced TCP connection Second

Determine when to unsplice Third

Require to initiate two TCP state machine Last

Impact on end-to-end throughput

34

Conclusion

35

Conclusion

Connection splicing is a good idea, but it doesn’t tell us how to implement

36

THE END

37

38

Cost Of Splicing

TCP sequence number trace showing the effects of the SCOUT implementation of splicing

39

Connection Splicing

An optimization technique that improves TCP forwarding performance

Basic idea of connection splicing To detect when a proxy makes a transition

from control mode to forwarding mode And then splice the two TCP connections

together into a single forwarding path through the system

40

Optimizing two TCP connectionsinto a single spliced connection (1)

Unoptimized TCP forwarder Require TCP segments to traverse TCP twice,

with each instance of TCP maintaining the full state of the connection

41

Optimizing two TCP connectionsinto a single spliced connection (2)

Optimized TCP forwarder (with spliced connection) Replace the proxy and two TCP processing steps with a

single FWD processing step FWD maintains just enough state to forward TCP

segment successfully from one network to another

42

Flow Path of TCP Forwarding

TCP forwarding starts in the unoptimized configuration

When proxy shifts from control to forwarding mode Makes a transition to optimized configuration

When TCP forwarding back to control mode Revert back to the unoptimized

configuration

43

Three Cases To Consider

Optimized TCP forwarder in the steady state

Unoptimized TCP forwarder becomes optimized TCP forwarder

Optimized TCP forwarder back to unoptimized TCP forwarder

44

Forwarding (1)

The primary task of FWD processing step Change the header of incoming TCP

segment to account for the difference in the two original TCP connections

If TCP connection establishment was interleaved One connection knew what port and

sequence numbers were used by other connection

Additional optimization are possible

45

Forwarding (2)

46

Forwarding (3)

When forward connection A to connection B

Port Number TCP forwarder operate as a classical proxy

Source and destination port numbers of segments arriving on A have to be changed to the port numbers of connection B

TCP forwarder is a transparent proxy Proxy uses the same port numbers

Output.DstPort = RemotePortB

Output.SrcPort = LocalPortB

47

Forwarding (4)

Sequence Number TCP initializes SN randomly for each

independent connection The SN for an outgoing segment is

computed by adding a fixed offset to the SN in the incoming segment

Output.SeqNum = Input.SeqNum + SeqNumOffsetA->B

48

Forwarding (5) ?????????

Acknowledge Number ACK number acknowledges SN forwarded

in the other direction ACK number in a outgoing segment is

computed by subtracting from the SN in the incoming segment, the SN offset for segments flowing in the other direction

Output.Ack = Input.Ack – SeqNumOffsetB-

>A

In my opinion Output.Ack = Input.Ack + SeqNumOffsetA->B

49

Forwarding (6)

Checksum Modifying the other fields require

adjusting the TCP checksum Output.Cksum = Input.Cksum +

CksumPatchA->B

50

Forwarding (7)

In the unspliced case Segments sent to proxy are put to Incoming

TCP stack Check if they can reach their destination Data are buffered in outgoing TCP stack until

they are acknowledged by the destination In the spliced case

No longer traverse the two TCP protocol stack Not acknowledge proxy, nor resend data to

destination

51

Splicing (1)

The real problem is transitioning from the unspliced state to the spliced state

52

Splicing (2)

Acknowledged data must be reliably delivered to their destination

During the time the data are being drained, however, new segments may arrive Forwarder can’t let TCP acknowledge

new data Impractical to wait until two connection

go idle before completing the splice

53

Splicing (3)

Two ways to handle newly arriving segments during this transition period Delay activation of spliced connection

until after the buffers have drained This solution may drop data if FWD buffers

overflow while TCP buffer are being drained Allow FWD to begin forwarding data

concurrently with draining the buffers All newly arriving segments are delivered to

both the original TCP protocol and to FWD Cause data to be delivered out-of-order

54

Splicing (4)

Before packet processing can be altered Computering SN offset & checksum

patches for FWD SN offset can be calculated as soon as all

acknowledged data have been drained from forwarder buffer

Checksum patch can be calculated as soon as the other offset known

55

Unspliced (1)

56

Flow Control

57

Additional Optimizations

58

Other Issues

59

Connection Splicing In SCOUT

60

61

2-Path

62

1-Path

63

Proxy For Unoptimized Forwarding

Detect a transition to forwarding mode Stops processing incoming segments &

allows segments to accumulate in the path’s input queue

Unlinks two TCP stages & proxy stage from the path & replaces them with a

64

FWD

65

IP / FWD

66

IP Router

Modifies network packets directly in the same way as IP/FWD

Not support reassembly of IP packets

67

Compare With Linux

TIS Firewall Trusted Information System TIS firewall toolkit offers full filter

functionality Use a null filter

Filtering IP Router The in-kernel Linux IP forwarding has

support for filtering on IP address, protocol number & port number

Closest thing in Linux to IP/FWD case in SCOUT

68

Compare With Linux

IP router Basic in-kernel Linux IP forwarding with

no filtering

69

Test Environment

200 MHz PentiumPro workstation 256 KB cache 128 MB ram Digital Fast EtherWORKS PCI 10/100 32-

bit PCI 10/100 Mb/s adapter Linux 2.0.30

70

Test Environment