Optimized Sensory-motor Couplings plus Strategy Extensions for the TORCS Car Racing Challenge

COBOSTAR Cognitive BodySpaces for Torcs based Adaptive

Racing

Outline:• TORCS Competition Setup

• COBOSTAR Design

• Parameter Optimization

• Strategy Modifications

TORCS Competition Setup

TORCS Competition Setup:

Only local information /

Main Ideano complete track

Information available

TORCS Competition Setup:Available Sensors

• Angle Sensor: current angle between the car direction and the track axis.

• Speed Sensors: speed in both axes.• 19 Range Sensors: free track space in front of

the car.• 36 opponent Sensors: notice opponents around

the car.• Additional Sensors: current engine speed,

current gear, rotation and speed for all wheels, damage of the car, etc.

TORCS Competition Setup:

Car Control•Gas and Break Pedals•Gear shifting•Steering

COBOSTAR Design

COBOSTAR Design:1 .On Track Strategy

• Distance and direction of the longest range sensor

2 .Off Track Strategy• Angle to track axis• Relative position to track center

COBOSTAR Design:On Track Strategy

Using:

Calculate: Target steering

angle:

Target speed:

Finally:

COBOSTAR Design: on Track Strategy

:Gas or Break

How to Break :

How to Steer :

COBOSTAR Design: off Track Strategy

what changes?

• Distance sensors to track borders are unavailable.

• Steering becomes more difficult.

• Wheel slippage is much stronger.

Using:

Calculate:Target steering angle:Target speed:

COBOSTAR Design: off Track Strategy

Stuck behavior:

COBOSTAR Design: off Track StrategyAnti Slip Regulation:

If nothing else works,Switch to reverse gear and stay in this mode until angle to track axis is halved or until stuck again

Parameter Optimization

Parameter Optimization

CMA – Covariance Matrix AdaptationSmart(er) search for the best solution. Takes in

account the dependencies between the parameters

Evolution Process:• All parameters were optimized on various TORCS

tracks.• All sets of parameters were compared on all tracks.• The most general parameter set was chosen.

Optimization Algorithm:

Parameter Optimization – On RoadFitness Function: 1( / distance raced + 1)

Parameter Optimization – On Road

• The most general set, wins only in four of the tracks (not that general).

• Second set with most wins, is only rated 6th – different behavioral strategies suite for different track types.

• Best strategy for a track isn’t always the strategy that was optimized on that track – local optimum.

• Differences between worst and best performance on each track – hard to get a general strategy

Interesting findings

Parameter Optimization – On Road

Blast from the past:

some strategies control this formula with P1 and some with P2.

Parameter Optimization – Off Road

• Not the same Fitness function. If the car controller is good, then the car will not reach “off-road”

Why not the same as on-road?

• Crash Strategy: every 300 meters causes the car to go off the road.

• Now the same fitness function can be used.

Solution

Strategy Modifications

Strategy Modifications – Gear ShiftingShift up:

Shift down:

Each time the engine reaches 9500 RPM

Each time the engine drops bellow:

3300, 6200, 7000, 7300, 7700

For gears: 2, 3, 4, 5, 6 respectively

Strategy Modifications – Large TrackThe problem:

Solution:

The target angle is interpolated between the maximal distance sensor and its neighbors, causing the to drive on slightly wavy trajectory.

Measuring the track width at the beginning of the race. If it exceeded a hand-set threshold, some steering factors were set by hand instead of the evolved ones.

Strategy Modifications – 2nd LapSwitching Strategies:

Analyze the general properties of the track and switch to more suitable strategy.

Crash Point:Remembering crash points from the first lap, in

the next lap the car would go into “passive mode”.

QUESTIONS?