1
1.0 Introduction: Yeast Fermentation
Modelling, Simulation and Computing Laboratory (mscLab)School of Engineering and Information Technology, Universiti Malaysia Sabah, Malaysia
SOMChE201125th Symposium of Malaysian Chemical Engineers 2011
Kuantan, Malaysia, 28 November -1 December 2011
• Saccharomyces cerevisiae (aka baker’s yeast) - the most studied strain
• As the benchmark
• What we want? ---MORE YEAST! Enhance the supply of raw material Sustainable environment and production Future potential
Sugar + oxygen → yeast + CO2 + alcohol
2
1.0 Introduction: Fed batch operation
Modelling, Simulation and Computing Laboratory (mscLab)School of Engineering and Information Technology, Universiti Malaysia Sabah, Malaysia
SOMChE201125th Symposium of Malaysian Chemical Engineers 2011
Kuantan, Malaysia, 28 November -1 December 2011
Yeast (biomass)
Fermentor
Product (after some time)
Substrate (Food)
(1) Dynamic growth behavior (2) Quality (3) Resource saving
WHY Fed-batch?
AIM: Maximize yeastMinimize alcohol
3
2.0 Objective
• To introduce and apply the learning algorithm (Q-Learning) infermentation system to determine the optimal substrate feeding profile
• To use the Q-Learning optimized profile as predetermined profile forfermentation system
Motivations:• Unsupervised learning ability• Reduction of human input• Applicable to highly dynamic system• Multi-objectives algorithm
Modelling, Simulation and Computing Laboratory (mscLab)School of Engineering and Information Technology, Universiti Malaysia Sabah, Malaysia
SOMChE201125th Symposium of Malaysian Chemical Engineers 2011
Kuantan, Malaysia, 28 November -1 December 2011
4
3.0 Methodology
Modelling, Simulation and Computing Laboratory (mscLab)School of Engineering and Information Technology, Universiti Malaysia Sabah, Malaysia
SOMChE201125th Symposium of Malaysian Chemical Engineers 2011
Kuantan, Malaysia, 28 November -1 December 2011
Observation and Learning Module• QL can be described as the following equation:
Wheres = state s’ = next statea = action a’ = next actionQ = current Q-valueQ’ = Expected Q-value in next statet = time α = learning rateR = reward γ = discount factor
• Q-learning (QL) is an unsupervised learning algorithm
• Works by learning state-action function that gives expected reward of taking an action in a specific state with a fixed policy.
5
3.0 Methodology
In this research, QL is used to:
•Observe and learn the effect ofinput feed flow rate on thevarious product concentration
•Maximum reward is chosen forthe feed which gives maximumyeast and less alcohol
•Determine the optimal feedprofile by total maximum reward.
Modelling, Simulation and Computing Laboratory (mscLab)School of Engineering and Information Technology, Universiti Malaysia Sabah, Malaysia
SOMChE201125th Symposium of Malaysian Chemical Engineers 2011
Kuantan, Malaysia, 28 November -1 December 2011
6
4.0 Simulation
Modelling, Simulation and Computing Laboratory (mscLab)School of Engineering and Information Technology, Universiti Malaysia Sabah, Malaysia
SOMChE201125th Symposium of Malaysian Chemical Engineers 2011
Kuantan, Malaysia, 28 November -1 December 2011
• Complex process dynamics
• Substrate and product inhibition
DYNAMIC
KINETIC
Material Balances
Product (after some time)
Feed inS0
7
5.0 Results and Discussion
Modelling, Simulation and Computing Laboratory (mscLab)School of Engineering and Information Technology, Universiti Malaysia Sabah, Malaysia
SOMChE201125th Symposium of Malaysian Chemical Engineers 2011
Kuantan, Malaysia, 28 November -1 December 2011
0 1 2 3 4 5 6 7 8 9 100
0.5
1
1.5
2
2.5
3
3.5
4
Time (h)
Con
cent
ratio
n of
Glu
cose
(g/L
), E
than
ol (g
/L)
glucoseethanol
0 1 2 3 4 5 6 7 8 9 1010
15
20
25
30
35
40
45
50
Time (h)
Con
cent
ratio
n of
Yea
st (g
/L)
yeast
0 1 2 3 4 5 6 7 8 9 100
0.5
1
1.5
2
2.5
3
3.5
4
Time (h)C
once
ntra
tion
of G
luco
se (g
/L),
Eth
anol
(g/L
)
glucoseethanol
0 1 2 3 4 5 6 7 8 9 1010
15
20
25
30
35
40
45
50
Time (h)
Con
cent
ratio
n of
Yea
st (g
/L)
yeast
Q-learning (QL) optimized feeding performances
Nominal exponential feeding (EF) performances: F = 500e0.05t
Comparison of substrate (glucose), undesirable products (ethanol) and biomass(yeast) concentrations for exponential feeding and Q-learning feeding strategy.
8
5.0 Results and Discussion
Modelling, Simulation and Computing Laboratory (mscLab)School of Engineering and Information Technology, Universiti Malaysia Sabah, Malaysia
SOMChE201125th Symposium of Malaysian Chemical Engineers 2011
Kuantan, Malaysia, 28 November -1 December 2011
0 1 2 3 4 5 6 7 8 9 100
500
1000
1500
2000
2500
3000
Time (h)
Feed
Flo
w R
ate
(L/h
)
Feed flow rate (EF)Feed flow rate (QL)
QL manage to optimize feed flow rate to the maximum before inhibitions andoverflow metabolism which triggers ethanol production happens.
9
5.0 Results and Discussion
Modelling, Simulation and Computing Laboratory (mscLab)School of Engineering and Information Technology, Universiti Malaysia Sabah, Malaysia
SOMChE201125th Symposium of Malaysian Chemical Engineers 2011
Kuantan, Malaysia, 28 November -1 December 2011
0 1 2 3 4 5 6 7 8 9 100
0.5
1
1.5
2
2.5
3
3.5
4
Time (h)
Con
cent
ratio
n of
Glu
cose
(g/L
), E
than
ol (g
/L)
glucose (EF)glucose (QL)
0 1 2 3 4 5 6 7 8 9 100
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Time (h)
Con
cent
ratio
n of
Glu
cose
(g/L
), E
than
ol (g
/L)
ethanol (EF)ethanol (QL)
0 1 2 3 4 5 6 7 8 9 100
5
10
15
20
25
30
35
40
45
50
Time (h)
Con
cent
ratio
n of
Glu
cose
(g/L
), E
than
ol (g
/L)
yeast (EF)yeast (QL)
QL-suggested feeding profile is able to achieve multi-objective purposes: reduce substrate usage, maximize yeast production and minimize ethanol.
10
6.0 Conclusion
• Based on the performances, Q-learning is able to:
suggest satisfying optimal profile as predetermined feeding strategy for yeast fermentation based on its objectives;
act in dynamic-complex system;
suggest an alternative algorithm for biosystem
Modelling, Simulation and Computing Laboratory (mscLab)School of Engineering and Information Technology, Universiti Malaysia Sabah, Malaysia
SOMChE201125th Symposium of Malaysian Chemical Engineers 2011
Kuantan, Malaysia, 28 November -1 December 2011