quantum-inspired bidirectional associative memory … · quantum-inspired bidirectional associative...
TRANSCRIPT
Quantum-Inspired Bidirectional Associative Memory
for Human–Robot Communication
Naoki Masuyama* and Chu Kiong Loo†
Faculty of Computer Science and Information Technology,
University of Malaya, 50603 Kuala Lumpur, Malaysia*[email protected]
Naoyuki Kubota
Department of Systems Design,Tokyo Metropolitan University,
6-6 Asahigaoka, Hino, Tokyo, 191-0065, Japan
Received 11 August 2013
Accepted 23 January 2014Published 30 May 2014
The emerging research area of a quantum-inspired computing has been applied to various ¯eld
such as computational intelligence, and showed its superior abilities. However, most existing
researches are focused on theoretical simulations, and have not been implemented in systemsunder practical environment. For human–robot communication, associative memory becomes
essential for multi-modal communication. However, it always su®ers from low memory capacity
and recall reliability. In this paper, we propose a quantum-inspired bidirectional associative
memory with fuzzy inference. We show that fuzzy inference satis¯es basic postulates of quantummechanics, but also learning algorithm for weight matrix in associative memory. In addition, we
construct a communication system with robot partner using proposed model. This is the ¯rst
successful attempt to overcome conventional problems in associative memory model with arobot application.
Keywords: Associative memory; quantum computing; fuzzy inference; human–robot interaction.
1. Introduction
Recently, the aging society is one of the inevitable problems in some countries.
Especially, the number of elderly people living alone is increasing and they are
lacking a chance of communication with other people compared with elderly people
who live with family.1 Thus, these people would increase the probability of the
cognitive decline and a high risk of dementia. In order to improve the situation of less
communication, some elderly care robot has been developed.2–7
International Journal of Humanoid Robotics
Vol. 11, No. 2 (2014) 1450006 (22 pages)
°c World Scienti¯c Publishing Company
DOI: 10.1142/S0219843614500066
1450006-1
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
The issue of social communication has been discussed in sociology, develop-
mental psychology, relevance theory, and embodied cognitive science.8–14 Cogni-
tive psychology has tried to construct a computer that can think.8 In the \society
of mind" theory proposed by Minsky, intelligence is explained as a combination of
multiple simpler elements. He argues that although each agent is itself intelligent,
it is not enough to simply explain what each separate agent does. Rather, it is a
group of agents that can accomplish things.11 Similarly, the theory o®ers insight
into the debate about human communication.12 According to this theory, human
thinking is not simply transmitted, but is an event that is shared between two
people. Such a shared environment is called a mutual cognitive environment.15
Through the communication with humans, the robot can understand personal
preferences, interests and intentions. And by performing mutual cognitive envi-
ronment sharing with human, the robot can provide conversation topics by itself.
In this situation, it is preferable that the selected topics are related to the shared
cognitive environment. Conventionally, topics provided by the robot are selected
based on previous conversations with the human or human behaviors. In our
previous work, we proposed a multi-modal communication system and its
computational intelligence.16 The basic idea of the multi-modal communication
and cognitive environment are similar to this paper. But previous paper is not
considered relevance or continuity of topics. On the other hand, the system in this
paper considered it. Thus, this paper can handle relationships between di®erent
information. If robot provides topics without any relevance or continuity, sharing
a cognitive environment is di±cult. However, if the principles of associative
memory in a shared cognitive environment are applied, the robot can provide more
relevent topics for conversation. As a result, communication between the robot
and human will be more smooth and active.
Various types of associative memory have been proposed. In the early 1980's,
Hop¯eld proposed an auto-associative memory model to store and recall information.
This model, however, su®ers from a lack of memory capacity and noise tolerance. In
the late 1980s, Kosko extended the Hop¯eld model and introduced bidirectional
associative memory (BAM).17 However, the original Kosko BAM also su®ers from
low storage capacity and poor recall reliability. To improve the performance, various
types of implementations have been introduced e.g., added dummy neurons to each
layer,18 and added hidden layer.19 A General model for BAM (GBAM) is de¯ned as a
weight matrix in each layer, and developed an algorithm for learning the asymptotic
stability conditions.20 Thanks to this structure, memory capacity and noise tolerance
are quite improved from Kosko BAM. As another approach, there is a model that
apply quantum mechanics for Hop¯eld model.21–23 This model demonstrates that
quantum information processing in neural structures results in an exponential in-
crease in storage capacity and can explain the extensive memorization and inferen-
cing capabilities of humans. However, it is limited to auto-associative memory.
N. Masuyama, C. K. Loo & N. Kubota
1450006-2
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
The existing models su®er from limited memory capacity and noise tolerance
problems. In this paper, we develop a theory for quantum-inspired bidirectional
associative memory (QBAM). The QBAM result from neural associative memories
if the elements w of the weight matrix W are taken to be fuzzy variables. In this
paper, we show that fuzzy inference satis¯es not only basic postulates of quantum
mechanics, but also learning algorithm for weight matrix in associative memory.
Instead of Hebbian learning, it is assumed that the incremental changes of the
weights are performed with the use of fuzzy inference. It is the ¯rst attempt to
provide an e®ective solution to conventional problems in associative memory.
Moreover, it is one of the ¯rst quantum associative memory models that can be
applied to human–robot Interaction in a real-life environment. In Sec. 2, ¯rst of
all, we show the structure of QBAM. Next, we show similarity between fuzzy
inference and quantum mechanics in QBAM, and proof fuzzy inference satis¯es
two basic postulates of quantum mechanics. In Sec. 3, we present simulation
experiment about memory capacity and noise tolerance of QBAM. In Sec. 4, we
explain computational intelligence technologies and the overall architecture of the
robot partner system using the proposed method. Section 5 presents the experi-
mental results of the potential communication between the robot partner and the
human through object, gesture and voice recognition.
2. Quantum Mechanics for Associative Memory
Superposition and unitarity are the key features of quantum mechanics. Superpo-
sition can be explained as \multiple states", which exist simultaneously in the
quantum system. The evolution of a closed quantum system is described by a unitary
transformation. If we apply fuzzy inference with same width triangular membership
functions in weight matrix, existence of superposition and unitary operation are
satis¯ed. In QBAM, unitarity is satis¯ed by rotations between spaces that are
spanned by the eigenvectors of weight matrices.24 Mathematical proofs are presented
from Secs. 2.5–2.8.
2.1. Structure of quantum-inspired BAM
. X-layer to Y-layer
Quantum-Inspired BAM for Human–Robot Communication
1450006-3
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
. Y-layer to X-layer
Let fX ðkÞ;X ðkÞ; . . . ;X ðkÞg and fY ðkÞ;Y ðkÞ; . . . ;Y ðkÞg, for k ¼ 1; 2; . . . ; be the
bipolar pattern to be stored. k denotes the number of pattern pairs, M and N denote
the number of neurons in X-layer and Y-layer, respectively, W Tij and Wij denote the
weight matrix in X-layer and Y-layer, respectively.
The weight matrix W Tij and Wij are as follows:
. X-layer to Y-layer
W Tij ¼ 1
K
XNj¼1
XMi¼1
vTj ui; ð3Þ
. Y-layer to X-layer
Wij ¼1
K
XMi¼1
XNj¼1
uTi vj ; ð4Þ
where ui and vj are calculated by Gram-Schmidt orthogonalization that according to
a1 ¼ A1=jjA1jjðp ¼ 1Þ, bp ¼ Ap �Pk�1
i¼p�1ðai;AiÞai and ap ¼ bp=jjbpjjð2 � p � kÞ,where A denotes the vector of performing orthogonalization, a and b denote
orthonormalized vector and orthogonalized vector, respectively.
The de¯nition of the weight matrix update is as Eqs. (5) and (6).
. X-layer to Y-layer
WTðtþ1Þij ¼ W
TðtÞij þ
�F ; If y kðstoredÞ
XMi¼1
WTðtÞij x k � 0 and W
TðtÞij � 0
F ; If y kðstoredÞ
XMi¼1
WTðtÞij x k � 0 and W
TðtÞij < 0:
8>>>><>>>>:
WTðtÞij ; Otherwise
8>>>>>><>>>>>>:
ð5Þ
. Y-layer to X-layer
Wðtþ1Þij ¼ W
ðtÞij þ
�F ; If x kðstoredÞ
XNj¼1
WðtÞij y k � 0 and W
ðtÞij � 0
F ; If x kðstoredÞ
XNj¼1
WðtÞij y k � 0 and W
ðtÞij < 0:
8>>>><>>>>:
WðtÞij ; Otherwise
8>>>>>>><>>>>>>>:
ð6Þ
N. Masuyama, C. K. Loo & N. Kubota
1450006-4
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
In Eqs. (5) and (6), t denotes time steps, exponential T denotes transpose. xðkÞðstoredÞ
and yðkÞðstoredÞ denote kth stored pattern, and x ðkÞ and y ðkÞ denote inner state of
X-layer and Y-layer, respectively. M and N denote the number of neurons in
X-layer and Y-layer, respectively. Fð0 < F � 1Þ denotes variation amount in po-
sition for the center of Fuzzy sets. The learning algorithm is referred from a
GBAM.20 This model tried to ¯nd an asymptotic stability condition of learning
algorithm for improving its abilities. The fundamental update algorithm is based
on Hebbian learning. Thus, we considered that it can be applied to the proposed
model.
2.2. Eigenstructure analysis of quantum-inspired BAM
Lemma 1. If the fundamental memory vectors of the associate memory are
chosen to be orthogonal, then they are collinear to the eigenvectors of matrices W T
and W .
Proof. The eigenvectors x and y of matrices W and W T satisfy the following:
The fundamental memory vectors �x and �y are taken to be orthogonal to each
other, i.e., �x Tj �x i ¼ �ði � jÞ and �yT
i �yj ¼ �ðj � iÞ, where � is Kronecker's function. The
weight matrix W and W T are given by W ¼ PMi¼1
PNj¼1 �y
Tj �x i and
W T ¼ PNj¼1
PMi¼1 �x
Ti �yj . Thus, the following holds:
. X-layer to Y-layer
W T �xk ¼1
K
XMi¼1
XNj¼1
�x Ti �yj
( )�xk ¼
1
K
XMi¼1
XNj¼1
�yið�x Ti �xkÞ ¼
1
K�yk
) W T �xk ¼1
K�yk : ð8Þ
. Y-layer to X-layer
W �yk ¼1
K
XNj¼1
XMi¼1
�yTj �x i
( )�yk ¼
1
K
XNj¼1
XMi¼1
�x jð�yTj �ykÞ ¼1
K�xk
) W �yk ¼1
K�xk : ð9Þ
Lemma 2. If the memory vectors of an associate memory are chosen randomly and
the number of neurons N and M are large, then there is high probability for them to be
orthogonal.
Quantum-Inspired BAM for Human–Robot Communication
1450006-5
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
Proof. The normalized internal product of memory vectors xi and xk , yj and yk are
considered
where, h denotes the hth fundamental memory, M denotes the number of neurons in
X-layer, N denotes the number of neurons in Y-layer. For largeM and N , and x ih, x
ik ,
y jh, y
jk randomly chosen from f�1; 1g. It holds EðPjiÞ ¼ 0, EðQijÞ ¼ 0 and
Therefore, EðPjiÞ ¼ 0, EðPji � �PjiÞ2 ¼ ð1=NMÞ2, and EðQijÞ ¼ 0, EðQij � �QijÞ2 ¼ð1=MN Þ2. The Central Limit Theorem (CLT) is applied here. This state:
(i) Consider fPkg a sequence of mutually independent random variables fPkg,which follow a common distribution. It is assumed that Pk has mean �P and
variance �2P , and let P ¼ P11 þ P12 þ � � � þ PNM ¼ PN
j¼1
PMi¼1 Pji . Then, as NM
approaches in¯nity, the probability distribution of the sum random variable P
approaches Gaussian distribution
ðP � NM�PÞffiffiffiffiffiffiffiffiffiNM
p�P
\NMð0; 1Þ: ð12Þ
(ii) Consider fQkg a sequence of mutually independent random variables fQkg, whichfollow a common distribution. It is assumed that Qk has mean �Q and variance
�2Q, and let Q ¼ Q11 þQ12 þ � � � þQMN ¼ PM
i¼1
PNj¼1 Qij . Then as MN approa-
ches in¯nity the probability distribution of the sum random variable Q approa-
ches Gaussian distribution
ðQ �MN�QÞffiffiffiffiffiffiffiffiffiMN
p�Q
\MN ð0; 1Þ: ð13Þ
According to CLT, the probability distribution of the sum variablePN
j¼1PMi¼1 y
jhx
ik ¼ NM
PNj¼1
PMi¼1 Pji and
PMi¼1
PNj¼1 x
ihy
jk ¼ MN
PMi¼1
PNj¼1 Qij . Each of
them follows a Gaussian distribution of center:
N. Masuyama, C. K. Loo & N. Kubota
1450006-6
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
Therefore, for large number of neurons M and N , i.e., for high dimensional spaces
1=NM ! 0, 1=MN ! 0 and the vectors xi and xk , yj and yk will be practically
orthogonal.
Thus, taking into account the orthogonality of the fundamental memories xk ,
yk and Lemma 1, it can be deduced that memory patterns in high dimensional
spaces practically coincide with the eigenvectors of the weight matrices W T and W .
2.3. Hebbian learning through fuzzy inference
. X-layer to Y-layer
In Fig. 1(a), A1;A2; . . . ;Am�1;Am and A�1;A�2; . . . ;A�mþ1;A�m are the fuzzy
subsets in which the universe of discourse of the variable wTij . The sets Ai and A�i are
selected to have the same spread and to satisfy the strong fuzzy partition equalityPmi¼1 �AmðxÞ ¼ 1 and
Pmi¼1 �A�mðxÞ ¼ 1, respectively.
. Y-layer to X-layer
In Fig. 1(b), B1;B2; . . . ;Bn�1;Bn and B�1;B�2; . . . ;B�nþ1;B�n are the fuzzy subsets
in which the universe of discourse of the variable wij . The sets Bj and B�j are selected
to have the same spread and to satisfy the strong fuzzy partition equalityPn
j¼1
�BnðyÞ ¼ 1 andPn
j¼1 �B�nðyÞ ¼ 1, respectively.
The fuzzi¯er is selected to be a triangular one. The main t-norm is used for
derivation of the fuzzy relational matrices Rim and Rd
m (X-layer), Rin and Rd
n (Y-
layer), where exponential i and d denotes \increase" and \decrease", respectively.
(a)
(b)
Fig. 1. The fuzzy set in (a) X-layer and (b) Y-layer.
Quantum-Inspired BAM for Human–Robot Communication
1450006-7
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
These matrices have the following properties:
. X-layer to Y-layer
ðaÞ W Tij � 0 :
Am ¼ Rim � Am�1; Am�1 ¼ Rd
m � Am; ð15aÞðbÞ W T
ij < 0 :
A�m ¼ Ri�m � A�mþ1; A�mþ1 ¼ Rd
�m � A�m: ð15bÞ. Y-layer to X-layer
ðaÞ Wij � 0 :
Bn ¼ Rin � Bn�1; Bn�1 ¼ Rd
n � Bn; ð16aÞðbÞ Wij < 0 :
B�n ¼ Ri�n � B�nþ1; B�nþ1 ¼ Rd
�n � B�n: ð16bÞ
The max-min inference is used, while the defuzzi¯er is a center of average one. The
learning algorithm is inspired from Hebbian learning and in the case of binary
memory vectors xk and yk can be stated as follows:
. X-layer to Y-layer
IF ykXmi¼1
W Tij x
ki � 0 and W T
ij < 0; THEN increase W Tij ; ð17aÞ
IF ykXmi¼1
W Tij x
ki � 0 and W T
ij � 0; THEN decrease W Tij : ð17bÞ
. Y-layer to X-layer
IF xkXnj¼1
Wijykj � 0 and Wij < 0; THEN increase Wij ; ð18aÞ
IF xkXnj¼1
Wijykj � 0 and Wij � 0; THEN decrease Wij : ð18bÞ
The weight matrix update with above fuzzy learning algorithm results by fuzzy
weight matrices. The latter can be decomposed into a superposition of associative
memories.
The whole weight matrices of associative memory W T and W equals a weighted
averaging of the individual weight matrices �W T and �W , i.e.,
. X-layer to Y-layer
W T ¼X2M
i¼1
�i �WTi ; ð19Þ
N. Masuyama, C. K. Loo & N. Kubota
1450006-8
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
. Y-layer to X-layer
W ¼X2N
j¼1
�j �Wj ; ð20Þ
where the non-negative weights �i and �j indicate the contribution of each local
associative memory �WTi and �W j . M and N denote the number of neurons in X-layer
and Y-layer, respectively. We will show a brief explanation of fuzzy value and each
variables in weight matrix in Sec. 2.7. In addition, Rigatos presents more details with
numeric example.22
2.4. Fundamental of quantum mechanics
In quantum mechanics, the state of an isolated quantum system Q is represented
by a vector j ðtÞi in a Hilbert space. This vector satis¯es Schr€odinger's di®usion
equation.24
i}d
dtj ðtÞi ¼ H ðtÞ; ð21Þ
where H denotes Hamiltonian operator that gives the total energy of a particle
(potential plus kinetic energy) H ¼ ðp2=2mÞ þ VðxÞ. The probability to ¯nd the
particle between x and x þ dx at the time instant t, the wave function ðx;tÞ can
be analyzed in a set of orthonormal eigenfunctions in a Hilbert space: ðx;tÞ ¼P1m¼1 cm m. Here, the coe±cients cm is an indication of the probability to describe
the particle's position x at time t by the eigenfunction m.
From Eq. (21), the average position of the particle is found to be
hxi ¼X1m¼1
jjcmjj2am; ð22Þ
where jjcmjj2 denotes the probability that the particle's position be described by
the eigenfunction m. The particle position x is the associated eigenvalue am. The
eigenvalue am is chosen with probability P / jjcmjj2.In the same way, the probability to ¯nd the particle between y and y þ dy at the
time instant t is given by PðyÞdy ¼ j ðy;tÞj2, then derived the following:
hyi ¼X1n¼1
jjcnjj2bn; ð23Þ
where jjcnjj2 denotes the probability that the particle's position be described by the
eigenfunction n.
2.5. Similarity of quantum mechanics and fuzzy inference
Assume that the fuzzy variables x and y belong to a universe of discourse, that
is quantized to an in¯nite number of fuzzy sets Ai;A�iði ¼ 1; 2; . . . ;1Þ and
Bj ;B�jðj ¼ 1; 2; . . . ;1Þ, e.g., the axis of real number R is partitioned to an in¯nite
Quantum-Inspired BAM for Human–Robot Communication
1450006-9
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
number of fuzzy sets with the same space and width. The fuzzy sets have the
following properties:
(a) They satisfy
. X-layer to Y-layer
X1m¼1
�AmðxÞ ¼ 1 ðW Tij � 0Þ; ð24aÞ
X1m¼1
�A�mðxÞ ¼ 1 ðW Tij < 0Þ; ð24bÞ
. Y-layer to X-layer
X1n¼1
�BnðyÞ ¼ 1 ðWij � 0Þ; ð25aÞ
X1n¼1
�B�nðyÞ ¼ 1 ðWij < 0Þ: ð25bÞ
(b) Each fuzzy set Am and Bn are described by its center am, bn and its width F .
(c) The average value of variable x and y will be given by
. X-layer to Y-layer
hxi ¼X1m¼1
�AmðxÞam ðW Tij � 0Þ; ð26aÞ
hxi ¼X1m¼1
�A�mðxÞa�m ðW Tij < 0Þ; ð26bÞ
. Y-layer to X-layer
hyi ¼X1n¼1
�BnðyÞbn ðWij � 0Þ; ð27aÞ
hyi ¼X1n¼1
�B�nðyÞbn ðWij < 0Þ: ð27bÞ
2.6. Fuzzy inference is performed through unitary operators
Theorem 1. The increase and decrease fuzzy operators that were described in the
rule-base are unitary.
Proof.
. X-layer to Y-layer
The fuzzy relational matrices RiðW Tij � 0Þ, R�iðW T
ij < 0Þ used by the increase
and decrease fuzzy operators satisfy the following fuzzy relational equations,
N. Masuyama, C. K. Loo & N. Kubota
1450006-10
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
respectively:
(1) Increase mode
ðaÞ W Tij � 0 :
A2 ¼ Ri1 � A1; A3 ¼ Ri
2 � A2; . . . ;Am ¼ Rim�1 � Am�1; ð28aÞ
ðbÞ W Tij < 0 :
A�2 ¼ Ri�1 � A�1; A�3 ¼ Ri
�2 � A�2; . . . ;A�m ¼ Ri�mþ1 � A�mþ1: ð28bÞ
(2) Decrease mode
ðaÞ W Tij � 0 :
A1 ¼ Rd1 � A2; A2 ¼ Rd
2 � A3; . . . ;Am�1 ¼ Rdm�1 � Am; ð29aÞ
ðbÞ W Tij < 0 :
A�1 ¼ Rd�1 � A�2; A�2 ¼ Rd
�2 � A�3; . . . ;A�mþ1 ¼ Rd�mþ1 � A�m: ð29bÞ
In both cases, \�" denotes the max-min composition. Substituting Am�1 ¼Rd
m�1 � Am in Am ¼ Rim�1 � Am�1 one gets Am ¼ Ri
m�1 � ðRdm�1 � AmÞ, in the same
way, A�m ¼ Ri�mþ1 � ðRd
�mþ1 � A�mÞ, and using the associativity of the max-min
composition yields Am ¼ ðRim�1 � Rd
m�1Þ � Am, A�m ¼ ðRi�mþ1 � Rd
�mþ1Þ � A�m,
respectively, i.e.,
Setting Am ¼ Rim�1 � Am�1 in Am�1 ¼ Rd
m�1 � Am, A�m ¼ Ri�mþ1 � A�mþ1 in
A�mþ1 ¼ Rd�mþ1 � A�m, and using the associativity of the max-min composition
yields Am�1 ¼ ðRdm�1 � Ri
m�1Þ � Am�1, A�mþ1 ¼ ðRd�mþ1 � Ri
�mþ1Þ � A�mþ1, i.e.,
Furthermore, due to the generation of the matrices Rim�1 and Rd
m�1, Ri�mþ1 and
Rd�mþ1 using Mandanis inference system,25 it holds the following relation:
From (31a) and (32a), (31b) and (32b) are deduced following:
Here, (33a) and (33b) are satis¯ed unitary.
Quantum-Inspired BAM for Human–Robot Communication
1450006-11
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
. Y-layer to X-layer
In the same way as X-layer to Y-layer, the fuzzy relational matrices RjðWij � 0Þ,R�jðWij < 0Þ used by the increase and decrease fuzzy operators satisfy fuzzy rela-
tional equations. Therefore, same as (30a) and (30b) to (32a) and (32b), it will be
derived following relationships that are similar with (33a) and (33b):
Therefore, the increase and decrease operators are unitary.
2.7. Existence of superposition in the weight matrix
. X-layer to Y-layer
Assume that weight element wkm of matrix W T , i.e., the element of the kth row
and themth column ofW T . Due to strong fuzzy partition, this weight belongs to two
adjacent fuzzy sets Ai and Aiþ1. The corresponding centers of the fuzzy sets are a ikm
and a iþ1km , and because of the strong fuzzy partition, the associatedmemberships will be
�km ¼ �Aiand 1� �km ¼ �Aiþ1
. Therefore, wkm is described by the sets f�km; aAi
kmg andf1� �km; a
Aiþ1
km g. Taking the possible combinations of the memberships for each
weight, the matrices that have as elements the memberships �km, 1� �km are gener-
ated. Taking the possible combinations of the projections of each weight to the centers
of the adjacent fuzzy sets, the matrices that have as elements, the centers aAi
km and
aAiþ1
km are generated. Using the above, the decomposition of the weight matrixW T into
the set of superimposing matrices �WTi ði ¼ 1; 2; . . . ; 2M Þ.
. Y-layer to X-layer
Assume that weight element wkn of matrixW , i.e., the element of the kth row and the
nth column of W . Due to strong fuzzy partition, this weight belongs to two adjacent
fuzzy sets Bj and Bjþ1. The corresponding centers of the fuzzy sets are bjkn and bjþ1kn ,
and because of the strong fuzzy partition, the associated memberships will be �kn ¼�Bj
and 1� �kn ¼ �Bjþ1. Therefore, wkn is described by the sets f�kn ; bBj
kng and
f1� �kn; �Bjþ1
kn g. Here, same as X-layer to Y-layer, the decomposition of the weight
matrix W into the set of superimposing matrices �Wj ðj ¼ 1; 2; . . . ; 2N Þ.
2.8. Evolution of eigenvector spaces via unitary rotations
It will be shown that the transition between the vector spaces that are associated
with matrices �WTi s and �W js are described by unitary rotations, respectively. These
are stated in the following theorem:
Theorem 2. The rotations between the spaces that are spanned by the eigenvectors
of the weight matrices �WTi and �W j are unitary operators.
N. Masuyama, C. K. Loo & N. Kubota
1450006-12
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
Proof.
. X-layer to Y-layer
Let xi, yi, zi and xj , yj , zj be the unit vectors of the bases which span the spaces
associated with the matrices �WTi and �W
Tj , respectively. Then a memory vector p
can be described in both spaces as p ¼ ðpxi ; pyi ; pzi ÞT and p ¼ ðpxj ; pyj ; pzj ÞT . Tran-sition from the reference system �W
Ti ! fxi; yi; zig to the reference system �W
Tj !
fxj ; yj ; zjg is expressed by the rotation matrix R, i.e., p �WTi¼ R � p�w T
j. Taking the
components of vectors p �WTi
and p �WTj, one gets p �W
Ti¼ pxi xi þ pyi yi þ pzi zi and
p �WTj¼ pxj xj þ pyj yj þ pzj zj . Furthermore it is true that
p �WTi¼ R � p �W
Tj)
pxipyipzi
0@
1A ¼
xixj xiyj xizjyixj yiyj yizjzixj ziyj zizj
0@
1A pxj
pyjpzj
0@
1A: ð35Þ
Similarly, one can obtain the transformation from p �WTi
to p �WTj, i.e., p �W
Ti¼
Q � p �WTj. Since dot products are commutative, one obtains Q ¼ R�1 ¼ RT . There-
fore, the transition from the reference system �WTi to the reference system �W
Tj is
described by unitary operators
QR ¼ RTR ¼ R�1R ¼ I : ð36Þ. Y-layer to X-layer
Let xj , yj , zj and xi, yi, zi be the unit vectors of the bases which span the spaces
associated with the matrices �W j and �W i, respectively. Then a memory vector q can
be described in both spaces as q ¼ ðqxj ; qyj ; qzj ÞT and q ¼ ðqxi ; qyi ; qzi ÞT . Transitionfrom the reference system �W j ! fxj ; yj ; zjg to the reference system �W i ! fxi; yi; zigis expressed by the rotation matrix Q, i.e., q �W j
¼ Q � q�wi. Taking the components of
vectors q �W jand q �W i
, one gets q �W j¼ qxj xj þ qyj yj þ qzj zj and q �W i
¼ qxi xi þ qyi yi þ qzi zi.
Furthermore, it is true that
q �W j¼ Q � q �W i
)qxjqyjqzj
0@
1A ¼
xjxi xjyi xjziyjxi yjyi yjzizjxi zjyi zjzi
0@
1A qxi
qyiqzi
0@
1A: ð37Þ
Similarly, one can obtain the transformation from q �W ito q �W j
, i.e., q �W i¼ R � q �W j
.
Since dot products are commutative, one obtains R ¼ Q�1 ¼ QT . Therefore, the
transition from the reference system �W i to the reference system �W j that is described
by unitary operators
RQ ¼ QTQ ¼ Q�1Q ¼ I : ð38Þ
3. Simulation Experiment
In this section, we compare Kosko BAM (BAM),16,17 GBAM20 and QBAM in terms
of memory capacity and noise tolerance. GBAM is de¯ned as a weight matrix in each
layer, and developed an algorithm for learning the asymptotic stability conditions.
Quantum-Inspired BAM for Human–Robot Communication
1450006-13
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
Fig. 2. One part of pattern pair sets (alphabet, number, image).
(a)
(b)
Fig. 3. The result of memory capacity (the number of maximum stored patterns: K ¼ 256). (a) scope:
K ¼ 1 to 30. (b) K ¼ 1 to 256.
N. Masuyama, C. K. Loo & N. Kubota
1450006-14
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
We prepare the number of stored pattern pair sets K ¼ 256 (alphabet, number,
image and random pattern). We considered that if the correct recall rate for k
pattern pairs was over 90%, k pattern pairs could be stored in memory. Figure 2
shows one part of pattern pair sets. These patterns represent 25 (5 by 5) bipolar
patterns as neuron for X-layer and Y-layer, respectively.
3.1. Memory capacity
Memory capacity is an important element of associative memory performance.
Figure 3(a) is the scope of 1 to 30 in Fig. 3(b). In Fig. 3(a) with GBAM, the recall
rate will be lower than 90% around K ¼ 20, and BAM can be stored quite less
pattern pair sets. On the other hand, in Fig. 3b, the result of QBAM shows over 90%
recall rate with any point of number of pairs K. In addition, the number of stored
patterns will be increased, and the recall rate will be decreased in BAM and GBAM.
(a)
(b)
Fig. 4. The result of noise tolerance. (a) K ¼ 20. (b) K ¼ 75.
Quantum-Inspired BAM for Human–Robot Communication
1450006-15
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
In contrast, the recall rate of QBAM keeps over 90% with any points, and even if the
number of stored patterns is increased, the recall rate can be held. Therefore, we can
regard that QBAM has outstanding memory capacity than BAM and GBAM.
3.2. Noise tolerance
Noise tolerance is another signi¯cant function in associative memory. We measure
the noise tolerance by adding the noises on input data randomly in X-layer. We set
two types of conditions (K ¼ 20, 75). According to Fig. 3(a), BAM cannot memorize
correctly in K ¼ 20, GBAM can be stored in K ¼ 20. QBAM can be stored in both
conditions. In Fig. 4(a), the recall rate of BAM and GBAM are decreased at the point
of 50% noise rate. Considering recall rate, GBAM can be stored correctly with 50%
noise. In contrast, QBAM shows quite high recall rate even if input data contains
high noise rate. From the result in Fig. 3(b), only QBAM can be stored as pattern
pair sets with K ¼ 75. Same as Fig. 4(a), QBAM in Fig. 4(b) shows high recall rate
although input data has high noise rate. Thus, it can be considered that QBAM has a
superior performance in terms of noise tolerance.
From the results of simulation experiments in terms of memory capacity and noise
tolerance, it can be regarded that QBAM has a superior performance compared to
conventional methods.
4. A Robot System Architecture and Computational Intelligence
for Cognitive Development
4.1. A robot system for associative memory
We developed a human–robot communication system with a robot partner based on
associative memory. Figure 5 shows the structure of the system. It is composed of a
robot partner, Microsoft Kinect (movement, voice and gesture recognition technol-
ogy), a microphone and a server PC running Julius software. Julius is an open-source
Fig. 5. The structure of robot system.
N. Masuyama, C. K. Loo & N. Kubota
1450006-16
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
continuous speech recognition engine with a large vocabulary.26 Kinect extracts
RGB and distance data. Using these data as input, the server PC can detect an
object's color and shape using a k-means algorithm and a steady-state genetic al-
gorithm (SSGA), or recognize gestures using spiking neural networks (SNNs).9 The
microphone collects human voice signals for Julius. Julius connects with the server
PC by TCP/IP. The server PC also calculates the relationship between the object,
gesture and words using QBAM. Then, based on this relationship, the server PC
sends the words or behavior order to the robot partner using TCP/IP. Figure 6
shows a sample of robot behaviors.
4.2. Computational intelligence for a robot partner
Figure 7 shows example of object and gesture recognition in our system. For object
recognition, we focused on colored objects and shape recognition using SSGA based
on template matching via image processing. Furthermore, we applied a k-means
algorithm for the clustering of candidate templates in order to ¯nd several objects
simultaneously. For gesture recognition, we used OpenNI to extract a human hand
position. SNNs were applied in order to memorize the spatio-temporal patterns of
gesture, and to classify gestures; furthermore, we applied a self-organizing map
(SOM) using pulse outputs from the SNNs. SNNs are pulse-coded neural networks
Fig. 6. The example of robot partner behaviors (lower bye-bye, upper bye-bye and up & down).
(a) (b)
Fig. 7. Example of computational intelligence for recognition. (a) object. (b) gesture.
Quantum-Inspired BAM for Human–Robot Communication
1450006-17
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
which memorize spatial and temporal contexts. We used a simple spike response
model to reduce the computational cost. As mentioned above, we applied Julius
software for voice recognition.26 Julius works in real time, and recognition accuracy
has been shown to be over 90% in a 20,000-word reading test. We utilized color,
shape, gesture and word information as input data for associative memory.
5. Experimental Results
This section presents the experimental results of the communication between robot
partner and human using proposed model through object recognition, gesture rec-
ognition and voice recognition. We de¯ned ¯ve types of relationship for associative
memory between object, gesture and word in Table 1. If we input information to the
system, corresponding information with Table 1 will be recalled. There are three
modalities (object, gesture and word) in each ID as Table 1. Thus, input and recalled
signal show same ID. Here, each input and output are composed by bipolar pattern.
Figure 8(a) shows the input sequentially as object, gesture and word. Figures 8(b)–
8(d) show sequentially output of relationship that is associated by BAM, GBAM and
(a) (b)
(c) (d)
Fig. 8. The input ID and the result of output ID as relationships. (a) Input ID. (b) BAM. (c) GBAM.
(d) QBAM.
Table 1. Relationship between object, gesture and word.
ID Relationship
Object Gesture Word
0 No object No gesture No word1 Red circle Lower bye-bye Red circle
2 Green triangle Upper bye-bye Green triangle
3 Blue rectangle Up & down Blue rectangle
4 Blue rectangle ��� ���
N. Masuyama, C. K. Loo & N. Kubota
1450006-18
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
QBAM, respectively. According to Table 1, the output waveform should show the
same form as the input one except for input ID 4.
For ID 4, if there is an input as unknown (not stored) signal, the system regards
that unknown signal as no input. This is because of our de¯nition. In Fig. 8(b), due to
the quite low memory capacity, BAM cannot recall the correct output. The result of
GBAM is slightly improved from BAM, because it has a better memory capacity
than BAM. However, recall process has still some failures in GBAM. On the other
hand, in Fig. 8(d), output of QBAM show the same waveform with input one except
for input ID 4, thanks to the large memory capacity and the high recall reliability.
Figures 9–11 show the recall rate between input and output in GBAM and QBAM.
Here, in each ¯gures, the axis labels on the right-hand side represent input information,
the axis labels on the lower side represent recalled information. This relationship
follows Table 1. In Fig. 9(a), most of the recall rates do not exceed 90%. In contrast, in
Fig. 9(b), for each input, the two outputs in accordance with Table 1 indicate the
perfect recall rate. Similarly, Figs. 10 and 11 have the same tendency in results.
From the results of experiment with robot system, we regard that QBAM is
e®ective method for the associative communication system with robot partner.
(a) (b)
Fig. 9. The result of recall rate (input: object, recall: gesture and word). (a) GBAM. (b) QBAM.
(a) (b)
Fig. 10. The result of recall rate (input: gesture, recall: word and object). (a) GBAM. (b) QBAM.
Quantum-Inspired BAM for Human–Robot Communication
1450006-19
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
6. Conclusion
This paper has proposed aQBAM. It discussed the capability of the proposedmodel for
human–robot interaction to process color, shape, gesture and word information. We
have showed the similarity between fuzzy inference and quantum mechanics, and
proposed method satis¯ed basic postulates of quantum mechanics through mathe-
matical proofs. The simulation results showed that the proposed model has superior
abilities in terms of memory capacity and noise tolerance. In addition, the experiment
with a robot system showed the e®ectiveness of the proposed model for human–robot
interaction. From these results, we argue that the proposedmodel is the ¯rst successful
attempt to overcome several problems found in practical robot applications. In addi-
tion, if we de¯ne more relationships and di®erent information of object, gesture or
word, it will be much e®ective for communication between robot partner and humans.
As future works, we will add more relationships and develop the quantum-in-
spired multidirectional associative memory. It has more e®ectiveness for communi-
cation because it can be associated from one thing to many things at the same time.
Furthermore, we will apply complex value to fuzzy inference to represent oscillation.
The oscillation is one of the important factors in quantum mechanics. It can be
expected to improve memory capacity and noise tolerance.
Acknowledgments
The authors would like to acknowledge a scholarship provided by the University of
Malaya (Fellowship Scheme). This research is supported in part by HIR grant UM.
C/625/1/HIR/MOHE/FCSIT/10 from the University of Malaya.
References
1. D. Callahan, Setting Limits: Medical Goals in an Aging Society (Georgetown UniversityPress, 1995).
2. M. R. Banks, L. M. Willoughby and W. A. Banks, Animal-assisted therapy and lonelinessin nursing homes: Use of robotic versus living dogs, Am. Med. Dir. Assoc. 9(3) (2008)972–980.
(a) (b)
Fig. 11. The result of recall rate (input: word, recall: object and gesture). (a) GBAM. (b) QBAM.
N. Masuyama, C. K. Loo & N. Kubota
1450006-20
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
3. M. Kanoh, S. Kato and H. Itoh, Facial expressions using emotional space in sensitivitycommunication robot \ifbot", IEEE Int. Conf. Robots and Systems (Sendai, Japan,2004), pp. 1586–1591.
4. E. Oztop et al., Human–humanoid interaction: Is a humanoid robot perceived as ahuman? Int. J. Human. Robot. 2(4) (2005) 537–559.
5. M. Heerink, B. Kr€ose, B. Wielinga and V. Evers, Enjoyment intention to use and actualuse of a conversational robot by elderly people, 3rd ACM/IEEE Int. Conf. Human–RobotInteraction, (Amsterdam, The Netherlands, 2008), pp. 113–120.
6. J. Cassell, Embodied conversational agents: Representation and intelligence in userinterface, AI Mag. 22(3) (2001) 67–83.
7. Y. Toda and N. Kubota, Computational intelligence for human-friendly robot partnersbased on multi-modal communication, IEEE Global Conf. Consumer Electronics,(Tokyo, Japan, 2012), pp. 314–318.
8. R. Pfeifer and C. Scheier, Understanding Intelligence, (MIT Press, Cambridge, 1999).9. M. W. Eysenck, Psychology, An Integrated Approach, (Harlow, Essex, Longman, UK,
1998).10. R. L. Gregory, The Mind, (Oxford University Press, London, UK, 1998).11. M. Minsky, The Society of Mind, (Shimon and Schuster, New York, 1986).12. D. Sperber and D. Wilson, Relevance ��� Communication and Cognition, (Oxford,
Blackwell, UK, 1995).13. M. Asada et al., Cognitive developmental robotics: A survey, Auton. Mental. Dev. 1(1)
(2009) 12–34.14. U. Neisser, Cognitive Psychology, (Appleton, New York, 1967).15. A. Yorita and N. Kubota, Cognitive development in partner robots for information
support to elderly people, Auton. Mental. Dev. 3(1) (2011) 64–73.16. N. Masuyama et al., Computational intelligence for human interactive communication of
robot partners, 12th Paci¯c Rim Int. Conf. Trends in Arti¯cial Intelligence (PRICAI),(Sarawak, Malaysia, 2012), pp. 771–776.
17. B. Kosko, Fuzzy associative memories, IEEE Trans. Syst. Man Cybern. 21 (1976) 85–95.18. Y. F. Wang, J. B. Cruz and J. H. Mulligan, Guaranteed recall of all training pairs for
bidirectional associative memory, IEEE Trans. Neural Netw. 2(6) (1991) 559–567.19. H. Kang, Multilayer associative neural network (MANNs): Storage capacity versus per-
fect recall, IEEE Trans. Neural Netw. 5 (1994) 812–822.20. H. Shi, Y. Zhao and X. Zhuang, A general model for bidirectional associative memories,
IEEE Trans. Cybern. 28(4) (1998) 511–519.21. G. G. Rigatos and S. G. Tzafestas, Parallelization of a fuzzy control algorithm using
quantum computation, IEEE Trans. Fuzzy Syst. 157(13) (2006) 1797–1813.22. G. G. Rigatos and S. G. Tzafestas, Quantum learning for neural associative memories,
IEEE Trans. Fuzzy Syst. 10(4) (2002) 451–460.23. G. G. Rigatos and S. G. Tzafestas, Stochastic processes and neuronal modelling: Quan-
tum harmonic oscillator dynamics in neural structures, Neural Process. Lett. 32 (2010)167–199.
24. D. J. Gri±ths, Introduction to Quantum Mechanics, (University Science Books,New Jersey, Prentice Hall, 1995).
25. B. Kosko, Neural networks and fuzzy systems: A dynamical systems approach to machineintelligence, Neural Process. Lett. 32 (2010) 167–199.
26. A. Lee and T. Kawahara, Recent development of open-source speech recognition enginejulius, Asia-Paci¯c Signal and Information Processing Association Annual Summit andConf., (Sapporo, Japan, 2009).
Quantum-Inspired BAM for Human–Robot Communication
1450006-21
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.
Naoki Masuyama graduated from Nihon University, Chiba,
Japan in 2010. He received the M.E. degree from Tokyo Metro-
politan University, Tokyo, Japan in 2012. He is currently working
toward the Ph.D. degree at University of Malaya, Kuala Lumpur,
Malaysia. His research interest is computational intelligence for
human–robot interaction.
Chu Kiong Loo obtained his Ph.D. from University Sains
Malaysia, B.Eng. (First class Hons in Mechanical Engineering)
from University Malaya.
Formerly he was a design Engineer in various industrial ¯rms
in di®erent capacities as well as he is the founder of Advanced
Robotics Lab in University of Malaya.
He has been involved in the application research of Peruss
Quantum Associative Model and Pribrams Holonomic Brain
Model in humanoid vision projects. Currently, he is the Professor of Computer
Science and Information Technology, University of Malaya, Malaysia. He has com-
pleted many funded projects by Ministry of Science in Malaysia and High Impact
Research Grant from Ministry of Higher Education, Malaysia. Loos research expe-
rience includes brain inspired quantum neural network, constructivism inspired
neural network, synergetic neural networks and humanoid research.
Naoyuki Kubota graduated from Osaka Kyoiku University in
1992, received the M.E. degree from Hokkaido University in 1994,
and received the D.E. degree from Nagoya University, Japan in
1997. He joined Osaka Institute of Technology, in 1997. He joined
the Department of Human and Arti¯cial Intelligence Systems,
Fukui University as an associate professor in 2000.
He joined the Department of Mechanical Engineering, Tokyo
Metropolitan University in 2004. He was an associate professor
from 2005 to 2012, and has been a professor since 2012 at the Department of System
Design in Tokyo Metropolitan University, Japan.
N. Masuyama, C. K. Loo & N. Kubota
1450006-22
Int.
J. H
uman
. Rob
ot. D
ownl
oade
d fr
om w
ww
.wor
ldsc
ient
ific
.com
by D
r. C
hu K
iong
Loo
on
06/1
0/14
. For
per
sona
l use
onl
y.