atca based controller - cern · the european xfel x-ray laser project x-ray free-electron laser...
TRANSCRIPT
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
ATCA Based Controller
Wojciech ZabołotnyKrzysztof Poźniak
Institute of Electronic SystemsWarsaw University of Technology
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Controller definition
TASKS:• Execution of control algorithm basing on Control Data provided by Low
Level Applications and on results of measurement of the Input signals.• Providing of processed measurement data to the Low Level Applications
layer• Monitoring and Exception Handling for safety requirements
Simple & low latency automat stabilizing the amplitude & phase
of electrical field during the single pulse
CONTROLLERLOW & HIGH
LEVEL APLICATIONTS
e x e c u t i o na r r a n g e m e n t
RF SYSTEM
C o n t r o l a l g o r i t h m s
DOOCS
managing
fieldcontrol
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Requirements for the controller• Basic technical requirements
– Multiple input channels
– Low latency (as low latency as possible, however it depends on IF and technological limits)
• Modular, parametrized and reconfigurable structure– Modularity - It should be possible to distribute the design between a few AMC
boards, and even between a few ATCA carrier boards (however it will impose some additional latency)
– Parametrized – the number of memory blocks, of DSPs used, of input channels serviced may be changed without significant redesign
– Reconfigurable – the general structure of the controller will be stable, even if some blocks are moved from FPGA to DSP.
• Design methodology related requirements– Controller is supposed to be a complex system, to assure high maintainability e.g.
to avoid human errors the automatic implementation methods (DSP on FPGA, DOOCS integration) must be developed
– Full testability is needed (in simulation, with real hardware, with hardware-software cosimulation)
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Reliability requirements of the XFEL controller
• Required: Continuous operation, one maintenance day per month• ATCA provided functions for increased availability:
– Redundant power supply
– Full mesh topology – no global bus for boards' interconnection, which could be blocked by a damaged board
• Controller design features contributing to increased availability– Algorithm able to operate even when some analog inputs are failed
• Use of redundancy in the analog input signals
• Possibility to work with limited number of signals
• Use of feed-forward alone as the „last resort solution”
– Possible redundant implementation of the controller
– Exception handling (e.g. Cavity gradient monitoring)
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Functional requirements for the controller
• Channel calibration• Vector sum• Error calculation• Feed forward• Generation of output signals with klystron linearization• Provision of measured data for:
– Low Level Applications layer for updating of Control Data for the Controller
– For monitoring (DOOCS) and diagnostics (Exception Handling)
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Multi – channel ADC
MULTI – CAVITY MODULEVectorModulator
Multi–channel Down-Converter
Control Data & Parameters
probes
Klystron
DAC
RF
SY
STE
M
Outline of the LLRF control structure
Tunercontrol
Beamdetector
C O N T R O L
DATA Preprocessing
SYSTEM MODEL IDENTIFICATIONLOW LEVEL
APLICATIONTS
Exception Detection and HandlingController Core
reflectedforward
forward
C T
R L
COMMUNICATION
IQ
Data Acquisition Memory
I/O Data
HIGH LEVEL APLICATIONTS D O O C S
MONITORING
Control Data& Parameters
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Vector sum control – results from MTS testsAdaptive Feed-Forward (gain=0) Feed-Forward + Feedback (gain=100)
0 500 1000 1500
0
5
10
15
20Cavity output envelope
MV
0 500 1000 1500
-5
0
5
10
Klystron output
mA
0 500 1000 1500-0.6
-0.4
-0.2
0
0.2
0.4
time [10-6
s]
rad
Phase of cavity and klystron
0 500 1000 1500
-50
0
50
100
150
200
250
time [10-6
s]
Detuning estimation
Hz
IQAbs
cavity
klystron
IQAbs
filling
filling
flattop
flattophalfbandwidth = 220 Hz
600 800 1000 120017.99
18
18.01
18.02
18.03
Amplitud of vector sumzoomed flattop range
MV
0 200 400 600 800 1000 12000
20
40
60
80
100
Klystron forward power
KW
600 800 1000 1200-2
-1
0
1
2x 10
-3
time [10-6
s]
Phase of vector sumzoomed flattop range
rad
rms ~ 4*10-4
rad
rms ~ 2*10-4
0 200 400 600 800 1000 12000
0.2
0.4
0.6
0.8
1
time [10-6 s]
Normalized Klystron characteristics
gain []
phase [rad]
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Vector sum control – with beam, without FF compensationCAVITY FIELD AMPLITUDE [MV] CAVITY FIELD PHASE [rad]
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Multiboard implementation of the controller
• Single board implementation impossible:– Main problem – multiple A/D converters (96 channels needed)
– reasonable limit: 8 or 12 channels/board
• It is necessary to transmit the converted signals between the boards
• If possible, it will be preferable to split the algorithm between the boards
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
out
Functional Concept of Controller
LOW LEVEL APPLICATIONS
Coefficients Set-Point Gain Feed-ForwardSimulation
Error CORRECTOR FeedbackI/Q
DETECTORVectorSum
– +
CALIBRATORDACADC +
C O N T R O L L E R
Data Acquisition Memory for LLA, HLA, monitoring, diagnostics
EXCEPTION HANDLING
AMCs ATCA carrier board
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
out
Functional Concept of Controller
LOW LEVEL APPLICATIONS
Coefficients Set-Point Gain Feed-ForwardSimulation
Error CORRECTOR FeedbackI/Q
DETECTORVectorSum
– +
CALIBRATORDACADC +
C O N T R O L L E R
Data Acquisition Memory for LLA, HLA, monitoring, diagnostics
EXCEPTION HANDLING
AMCs ATCA carrier board
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
out
Functional Concept of Controller
LOW LEVEL APPLICATIONS
Coefficients Set-Point Gain Feed-ForwardSimulation
Error CORRECTOR FeedbackI/Q
DETECTORVectorSum
– +
CALIBRATORDACADC +
C O N T R O L L E R
Data Acquisition Memory for LLA, HLA, monitoring, diagnostics
EXCEPTION HANDLING
AMCs ATCA carrier board
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Structure of the ATCA LLRF controler
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
ATCA communication capabilities used in the distributed controller implementation
• Low latency LVDS links provided by ATCA are essential for fast exchange of information between the distributed components of the controller
• Ethernet interface supports communication with the DOOCS system
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Scalability of the XFEL controller
• The controller algorithm may be distributed between the FPGA chips, DSP processor(s) and general purpose CPU
• The number of ATCA carrier boards used to implement the controller may vary (depending mainly on the amount of inputs)
• The ATCA architecture allows us to use multiple AMC modules, multiple exchangeable AMC modules may be used– DSP modules
– FPGA modules
• Fast communication interfaces allowing data exchange between the AMC modules may be used to efficiently implement distributed algorithms
• Number of installed modules may be suited to the needs of the algorithm
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Debugging of the XFEL controller in the ATCA architecture
• Debugging and continuous monitoring of integrity of procesing is essential for such distributed system
• Efficient debugging of the firmware requires fast communication links to transfer the debugging and control data
• The ATCA architecture provides the following features usable for that purpose:– Ethernet link to each carrier board
– PCI Express link to each AMC board
– JTAG connection to each AMC on the carrier
– LVDS interfaces to AMC boards (may be used for custom debugging interfaces)
• IPMI Interface („last resort solution” - low speed)
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Development activity proposal
Development team● ISE Employees
● Wojciech Zabolotny, PhD● Krzysztof Pozniak, PhD● Tomasz Czarski, MSc● Maciej Linczuk, PhD
● IPJ Employees● Jaroslaw Szewinski, MSc● Grzegorz Plebański, MSc● Krzysztof Sajna, Msc
● DESY Employees/ISE PhD stud.● Waldemar Koprek, MSc● Piotr Pucyk, MSc
● DCMS PhD Students● Wojciech Jałmużna, MSc
Tasks realization● Controller
● Controller Core● Exception Detection & Handling● Control Data & Parameters● Data Acquisition Memory
● Integration with „communication” package● Simulation
● Integration with LLA simulation
Schedule (heavily depends on hardware availability) – 24 m.
● Development on the existing SIMCON hardware (due to portable design) – 12 m.
● Tests in simulation – 15 m. (in the background)
● Porting to target platform – 12 m.
XFELThe EuropeanX-Ray Laser Project X-Ray Free-Electron Laser
Wojciech M. Zabołotny, Krzysztof T. Poźniak, Intitute of Electronic SystemsATCA LLRF meeting, 3.12.2007
Conclusions
• The proposed implementation of the Controller has been successfully verified in the FLASH experiment– however further work is needed on: exception handling, monitoring and
diagnostics, optimal distribution of algorithms (with LLA) between available resources, integration with the communication
• The Controller design is very tightly associated with the design of Low Level Applications and Communication
• Features of the ATCA technology useful for implementation of the LLRF controller– High reliability architecture
– Scalability – both on the crate level and on the carrier board level
– Low latency connections between carrier boards and between AMC boards
– Big amount of backplane connectors available for the user and full mesh connections in the ZONE2 area for easy acquisition and concentration of multiple analog signals