new creating space-time affordances via an autonomous sensor …oka... · 2020. 7. 23. · creating...

Creating Space-Time Affordances via anAutonomous Sensor Network

Norihiro MaruyamaThe University of Tokyo

3-8-1 Komaba

Tokyo, Japan

Email: [email protected]

Mizuki OkaCenter for Knowledge Structuring

The University of Tokyo

7-3-1 Hongo, Tokyo Japan


Takashi IkegamiThe University of Tokyo

3-8-1 Komaba

Tokyo, Japan


Abstract—An autonomous sensor network (ASN), which iscomposed of many sensor units, is proposed and studied as anew challenge for artificial life in open-environment experiments.An aim of this study is to investigate how an artificial lifesystem behaves in a real, open, ever-changing environment withshort- and long-term environmental changes, including humanbehaviors. Compared to robot experiments in an open space, wesay that ASN will not move around in a space but can be testedfor a longer period of time with strongly autonomous behavior,without ignoring the messiness of the environment,.

The sensor network is unique because we employ artificialchemistry to enable it to control the sensitivity of each sensorunit (i.e., sensors are not simply reacting to environmentalchanges but are mutually affecting each other). Depending onthe environmental pattern and interaction conditions betweensensors, the system switches between a resonating state anda resting state. The resting state is the baseline activity ofthe network for a normal environmental condition, and theresonating state emerges when the environmental parameter (i.e.,the light condition) changes. We tested the nature of this behaviorby an emulator, showing the transition point in its samplingperiodicity from a resting state to a resonating state.

To ”experience” this switching behavior between a resting andresonating state, we organized a sound installation by translatingthe sensor states into sound patterns with unidirectional ultra-sonic speakers. Such art installation can be a good opportunityto test long-term adaptive behaviors of artificial life in the realworld.

I. INTRODUCTION

Research on artificial life involves attempts to produce

lifelike phenomena through simulations using computer mod-

els, robotics, and biochemistry. In this paper, we propose a

new approach to artificial life experimentation in an open

environment, with an autonomous sensor network (ASN). It

takes the form of an experimental sound art installation, aims

to explore behavior over a longer term in a messy environment.

Without having attractive affordance, any art system is doomed

to fail to attract people. Whereas people will often contemplate

natural scenes such as an oceanfront or a great valley for an

indefinite length of time, they rarely spend more than three

minutes in front of an art object, especially if it is demanding.

People quickly get tired if an object contains too many explicit

messages to digest or if it is just a simple reacting system.

Potentially, space designers or artists could attract people

and cause them to stay for relatively longer periods of time

with their innovative pieces, yet their innovations or methods

are generally not capable of being systematically prepared or

reproduced. The question is how to create a new space-time

affordance.

The idea of affordance was originally proposed by J. J. Gib-

son in the realm of ecological psychology and later imported

by D. Norman into the field of human-computer interaction.

“Affordance”has many definitions, depending on the field in

which the term is used. In this paper, we refer to affordance

as environmental information that is unconsciously acquired

and that is not attributed to a simple physical stimulus in the

environment. This definition gave us the idea of looking for a

systematic way to make an artificial object by creating a new

affordance.

Our main study principle for creating a new affordance is

autonomy in a system or environment, a concept that has been

fostered in the field of artificial life. Autonomy in a system

creates a pleasant distance between object and observer but

also arouses emotions which, like those we have for our pets,

can give rise to long-lasting relationships.

The system obtains sensor information from the environ-

ment, maintains a basic (artificial) metabolism, and changes

its behavior after a certain amount of information processing

has occurred (e.g., light and humidity sensor data). At the

sound installation, the sensor dynamics are then translated

into a soundscape by two automatically controlled parametric

speakers. By creating the new affordance in the form of a

soundscape, we are also proposing a new idea of how an

artificial system can resonate with a particular environmental

pattern, for example, dry weather with bright sunshine. We

believe such resonating behavior is a characteristic of an

autonomous system.

II. BACKGROUND

Artificial life has been intensively studied for the past 20

years, focusing on evolution, autonomy, and self-reproduction.

However, we have not yet succeeded in producing lifelike

phenomena anywhere. One of the reasons may be that artificial

life has been researched or designed in an environment that is

too ”clean”(i.e., inside computers). We need to bring artificial

life into the real world and test behaviors over longer durations

to produce lifelike phenomena.

67978-1-4673-5863-7/13/$31.00 c©2013 IEEE

Many robot experiments provided typical examples of arti-

ficial life in the real world. The first artificial life robot was

built by Grey Walter around 1950 [1]. Then, Rodney Brook

built several insect-type robots, including garbage collectors (

now known as robot vacuum cleaners) in the 1980s [2]. In the

1990s, Sony’s pet dog robot AIBO [3] was built, and in 2000s,

various humanoid robots such as Honda’s ASIMO [4] and

iCub [5] were designed. Walter’s robot tortoises, Elmer and

Elsie, showed playful dancing behaviors combined with simple

reflective actions, and the descendant robots were designed to

show more ”intelligent” behaviors by replacing vacuum tubes

with sophisticated IC chips. These days, so many automated

machines surround us, however, that Walter’s original idea of

emulating lifelike behavior seems was not developed enough.Therefore, we still need to understand and incorporate

autonomy into artificial systems. For the system to exhibit

autonomous behavior, it must incorporate an internal dynamic

that memorizes the system’s past experiences and learns envi-

ronmental affordances. We have to consider the environmental

complexity more seriously to study affordances in the environ-

ment. Affordance is all about the space-time pattern embedded

in the environment; a system must assimilate its behavior to the

environment to use the affordance. In other words, messiness

in the environment creates mind (or mindful behaviors) even

within artificial life (private conversation with Andy Clark).In this work, we are particularly interested in how a system

can perceive and generate space-time affordance in the real

world. Having the same philosophy in mind, Ikegami has

developed an autonomous artificial system in an open envi-

ronment, that is, public space, called the Mind Time Machine

(MTM) [6], [7]. MTM continuously and autonomously gener-

ates images using 15 video cameras and their feedback images

in real time. The type of information that is processed and

memorized, which is controlled by layered neural networks,

determines the system’s autonomous behavior. Similar to the

work that has been implemented in MTM, we are interested in

how the system decides when and where to sense information,

as well as how the sensor dynamics change to adapt to the

open environment over a long period of time.The unique idea with ASN is that, instead of using the visual

input images, we take various sensor information, obtained

from the real world and packed in a digital buffer, as chemical

species that diffuse through wireless connections. Multiple

sensor nodes are connected through a wireless network, and

the reaction proceeds based on sensor information taken from

an open environment. We will also organize an experimental

sound installation with this system to explore behavior over a

longer term in a messy environment.

III. AUTONOMOUS SENSOR NETWORK (ASN)

A. Outlook

We propose an ASN that is spatially distributed in the

real world. One node has two sensors, light and humidity,

that sense the corresponding environment information with an

adaptive sensing periodicity (or cycle). The sensor information

obtained by each node will be sent to other nodes (we set

(a) overview of sensor unit

(b) implemented sensor unit

Fig. 1. Overview of a sensor unit mechanism in the autonomous sensornetwork. The two buffers accumulate sensor information (light and humidity)from the sensors and other data sent from other nodes connected wirelesslyin the space. The reaction-diffusion module uses values in these buffers tochanges the length of sensing cycle. The below picture shows an implementedsensor unit.

TABLE ICOMPARISON BETWEEN CHEMICAL AND SENSOR NETWORKS

chemical network sensor network

chemical species sensor typechemical sensors digital sensors

chemical reaction tank digital sensor bufferdiffusion packet switching

the number of nodes at two) via wireless connections. In

other words, each sensor is attached to a buffer of each node

that accumulates sensor information from its own sensor. Two

kinds of buffers (one associated with light, and the other with

humidity) are associated with each node.

This system is unique in that we use a metaphor of spatially

extended chemical reaction schema. A modified Gray-Scott

reaction-diffusion model is used as a design for this sensor

network [8] [9]. This model is a translation of a spatially

extended chemical reaction into an active sensing and wireless

network system. The comparison is summarized in Table 1.

Sensory data in each unit are put onto the buffer, and an

assumed reaction will take place in that buffer. Suppose that

sensor values are A and B received by the corresponding

sensor. For example, we use the reaction A + B2 → C to

change the sampling periodicity (cycle) of the sensor, whose

68 2013 IEEE Symposium on Artificial Life (ALife)

reaction speed is proportional to [A][B]2. The cycle length is

defined as how often a sensor receives the sensory value from

the environment. Namely, the cycle length will be increased or

decreased, proportional to the reaction rate. It should be noted

that the sensor values will not be affected by the reaction but,

only the cycle length will be updated. After computing the

reaction rates, those sensor values will be sent to the other

wirelessly connected sensor nodes. As a result, the sensing

data will be circulating in the network through a wireless

connection.Fig. 1 illustrates the rough circuit of Arduino, XBee, and the

main processor that implements the virtual chemical network

in one unit, which has four inputs from and four outputs to

other sensor nodes connected by a wireless connection. They

are controlled by two XBees and two Arduinos. 1) The XBee

is used to receive sensor data from other nodes and to send

the data to the main processor. 2) The Arduino is used to

send sensor data obtained from the sensor itself to the main

processor when requested. 3) All of the data is eventually

received at the input controller, a module implemented in the

main processor. The main thread in the system emulates the

chemical reaction thread. The buffers store sensor values in

this thread, and the sensor data sent from the input controller

are saved in these buffers. If a buffer becomes full, the input

data are not received and are lost. The reaction thread is

periodically executed by using the sensor data saved in the

buffers for every 1,000 msec. As a result, the sensor sampling

cycle will be updated.

B. Detailed Schema

i) Storing Sensor ValuesEach sensor updates the sensory state every 1000 msec

cycle, depending on the filling rate of the corresponding buffer.

We define a filling rate of a sensor type j in a buffer as Fj

using a sensory state sji at the buffer site i(= 0, 1, 2, . . . , 255),as follows:

Fj =

∑N−1i=0 sji

NVmax(1)

Each sensor state sj , (j = 1, , , 256) , ranging from 0 to

Vmax, is stored in the buffer of the size 256. If the buffer is

fully occupied, Fj takes the maximal size 1. If the buffer is

empty, Fj takes 0.Two kinds of sensors (u and v) are used here: the light

sensor and the humidity sensor, respectively. We refer to the

filling rates for these sensors as Fu and Fv , respectively.ii) Updating Sensing CyclesIn addition to this, we define Tu and Tv as the sensing cycle

for each sensor; namely, each sensor samples every Tu or Tv

second, and the cycle length will also be updated as,

kuΔTu = −u · v2 + k1 · u (2)

kvΔTv = u · v2 − k0 · v (3)

where ΔTu and ΔTv are the changes of the cycle lengths.

The parameters k0 and k1 are given arbitrary values (in

our experiments, k0 and k1 are set to 0.045 and 0.025,

respectively). In both equations, the first term represents a

reaction, and the second terms represent in-flow in case of

u (light) and out-flow in case of v (humidity).

This formula can be related to the artificial chemical reac-

tion, that is, if u and v are created through a reaction u·v2. We

adopted the formula to update the sensory sampling rate; for

example, the sensory sampling cycle will be decreased when

chemicals are produced too fast.

Additionally, every buffer discard the most oldest packet

every updating cycle.

iii) Switching Sensor Values with Other Nodes

The sensor node sends packets to other nodes every Td

period, where we assume each node has two other nodes to

communicate with. Some packets containing sensor values will

be lost during transmission, which is an interesting difference

between the sensor network and the chemical network.

IV. EXPERIMENTAL SETUP

The sensor system was tested both in the university labo-

ratory room (February, 2012) and the public bookstore over

a two-week period (February 13th through 26th, 2012, from

10am to 10pm). We put the sensor network in those two

locations by distributing sensor units, equipped with both light

and humidity sensors, and maintaining a diary of daily sensor

data. The room had a sunlight from windows, and the system

was exposed to completely different conditions depending on

whether it was day or night.

The laboratory room and the bookstore have different light

conditions and traffic conditions (i.e., the number of people

coming in and out). i) The university laboratory room stays

quiet with 4 to 5 people at the most coming through per day.

It has wide windows on a wall, so that the room is bright at

the window side. ii) The bookstore is a bit more quiet, but

many more people come through. It is also much darker than

the laboratory room and shows little difference in night and

day.

V. RESULTS AND ANALYSIS

By setting the sensor network in two spaces and letting it

run for more than a week, we noticed that at least two dynamic

modes emerged in this network. We will further analyze the

modes by setting up a simulator of the network.

A. Resonating and Resting State

The sensor data in our system changed significantly, de-

pending on the environmental condition. Both sensor values

usually decreased in the evenings and then activated during the

daytime. In Fig. 2, we compare the sensor data obtained over

24 hours between the two spaces: one in a bookstore, which

is a larger space remote from windows and many visitors all

day long, and the other one is a laboratory room, which is a

relatively smaller space with wide windows and a few people

coming through per day.

A more dynamic pattern was observed in the laboratory

room, and relatively smaller changes were observed in the

2013 IEEE Symposium on Artificial Life (ALife) 69

Fig. 2. The sensor values for both light and humidity and the sensing cycle values are depicted as a function of time (in hours). The upper-left figureshows the sensor patterns at the bookstore, while the bottom-left figure shows those in the laboratory. Similarly, the upper-right figure shows the sensing cyclepatterns at the bookstore, and the bottom-right figure shows those in the laboratory.

Fig. 3. Time-series of the sensor values obtained for each node during a one-week period. The four figures at the left show the input sensor patterns of thefour nodes at the bookstore. The four figures at the right show those in the laboratory. A week periodicity with a rectangle form is found in the bookstoreand a week periodicity with an irregular form is found in the laboratory room.

bookstore. To underline this difference, we plotted the sam-

pling cycles of the sensors in the right figure of Fig. 2.

Comparing the two locations, we see that the length of

sensing cycles often diverges in the laboratory setting with

a number of outstanding large peaks; however, no such peaks

are found in the bookstore. The peaks are caused by the excess

light coming from the window in the laboratory room. These

spontaneous temporal oscillations with the large amplitude are

termed the resonating state of a network. On the other hand, a

relatively stable but noisy oscillation is termed the resting stateof the network. From the morning to the evening, a network

changes from the resting state to the resonating state and then

comes back to the resting state in the evening.

This resonating state is a characteristic of the present sensor

network caused by periodic/chaotic switching between the

filling up and exhausting of sensor data states. We made a

simple simulator to emulate the behavior of the actual sensor

network with actual/virtual sensing data.

B. A Long-Term Observation

Fig.3 shows temporal behaviors of four sensor units, and

the corresponding sensing cycle is shown in Fig.4 in the two

locations. First, we notice that even if the sensing value shows

a perfect periodicity of 24 hours, the corresponding sensing

cycle doesn’t always show such periodicity in a bookstore. In

the lab room, we have very different oscillating patterns from

day to day in sensing values as well as the sensing cycle. Due

to the large windows in the lab, this difference is probably

caused by the weather conditions, but it is also affected by

the human behavior (how many people are in the lab in that


Fig. 4. The sensing cycle shows a periodic motion over a week. Just as in Fig. 3, the left four figures show the result at the bookstore, and the right one isin the laboratory.

Fig. 5. Phase diagram of the state of the light sensor (left) and that of thehumidity sensor (right), which is characterized by the number of peaks per24 hours, computed with the simulator under the recorded sensor data. Thetotal number of peaks at each point is shown in grading scales. The brightercolor has the more number of peaks. For each diagram, the horizontal axisis the proportional coefficient and the vertical axis is the amount of diffusionper one cycle.

day).It is also interesting that the humidity can also show a

periodicity of 24 hours in a bookstore, since the humidity

should be equal irrespective of the weather condition. This

periodicity in the humidity sensor can be caused by the

coupling with the light sensor of the same unit, but it also

can be caused by the other sensor units wirelessly connected

and distributed in the bookstore. Because some of the nodes

are put in the deepest corner of the bookstore, no environ-

mental changes were expected for the nodes. Therefore, the

spontaneous oscillation of those nodes must be affected by the

connected sensor nodes.As we will study in the next subsection, even if the

environment has no temporal changes, the sensor values and

the cycle do oscillate. This is proved by a simulator run within

a single computer.

C. Emulating the Experiments with a Simulator

A simulation of this sensor network is done by equating

one cycle in a computer to 1 msec of the real experiment. For

each one simulation cycle, we execute reaction and diffusion

if it satisfies the condition, and we update the buffer states and

sampling cycles every one second. At the same time, sensory

data will be taken in from the recorded inputs can be applied

(or we use constant inputs for testing). Corresponding to the

real-world simulation, we recorded the internal states of the

system per every 1,000 msec.

We organized a phase diagram of the network by changing

the proportionality coefficients ku and kv in equations (2) (3)

and the amount of diffusion per one cycle.

The number of local peaks of the sensing cycle is depicted

in Fig. 5. In the diagram, we put light sensors on the left

and humidity sensors on the right. The numbers of peaks are

bursting in the upper-middle region of the diagram of the light

sensors.

By changing either the space location or the parameter sets,

we found that the resonating state of the network emerged.

Using the simulator with constant inputs from the environ-

ment, we examined how the network state changes by the

different inputs. In Fig.6 and Fig.7, we tested how the sensing

cycle changes by increasing the input values. This bifurcation

diagram was organized by superimposing the local peaks of the

dynamics for each input value. When the points are scattered,

the oscillation at that input value shows aperiodic motion. Note

that when the input values are lower than a critical value,

the sensing cycle (of its local peak values) simply shows a

fixed-point behavior. Above the critical point, it bifurcates into

other attractors. The attractor behaves differently for humidity

and light sensors as shown in Fig. 6. A major attractor is a

period 3 or a weakly chaotic oscillation. A similar behavior

is obtained in Fig.7, where the peaks of humidity states were

superimposed by fixing the humidity input and changing the

light inputs. We see a broadened period 2 oscillation that

becomes chaotic then to a fixed point and again becomes a

chaotic oscillation above a critical point.


Fig. 6. Bifurcation diagram of the sampling cycles under the constantenvironment. The horizontal axis shows the input sensor value (same as lightsensor and humidity sensor), and the local peaks of the time-series of thesampling cycle (top is the light sensor, and bottom is the humidity sensor)are superimposed in the vertical axis.

Fig. 7. Bifurcation diagram of the sampling cycles under the constanthumidity and the light values. The horizontal axis shows the light input valueand peaks of time-series of the sampling cycle superimposed in the verticalaxis.

VI. DISCUSSIONS

When we brought the system to a bookstore, we wanted to

create a new digital media art using the ASN. In particular,

we aimed to synthesize a new soundscape with this network.

When the amount of input sensor data in the buffer is ac-

cumulated over a threshold (5,000 for light sensor data, and

80 for humidity sensor data), the sound is generated from the

data point. A sound is a direct transformation of the amplified

sensor data, so it sounds like white noise. The point of making

this sound installation is to create a space-time affordance.

Specifically, unidirectional ultrasonic speakers are controlled

to move their faces periodically, and sound is emitted from

them via 32bit/48K Hz. Thanks to the nature of the speakers,

people experience the sound images as if they are whispered in

one ear. The main reason for using this type of speaker was to

Fig. 8. Installation of the ASN system, as a sound installation in a bookstorein Tokyo. The picture is an outlook of the installation. By walking aroundthe store, visitors experience different soundscapes generated by the twoautomatically controlled parametric speakers.

have people listening to the soundscape without detecting the

sound sources. People who come to this installation experience

a sense of ”here and now” by virtue of the undiffused nature

of the sound from the speakers. By walking around the

store, visitors experience soundscapes from a flow of abstract

information. Fig. 8 illustrates the display at the store.Creating new kinds of affordances with artificial life is

presented as a form of sound art installation; the system

provides a new type of affordance (to people who visit the

space) using a soundscape. So-called ambient music is usually

composed by a person and makes the same set of sounds

whenever it is played. By contrast, our system constantly

generates different sounds in a different space and time. Thus,

the system adapts itself to the place and time in which it

is installed. Sensitivity to place and time patterns is a new

feature of the artificial soundscape. We believe that auditory

rather than visual perception changes the way in which we

perceive space.“Surround-ness,”a unique nature of“ space-

ness,”has more to do with auditory than visual perception.

Therefore, this installation exemplifies how we can design a

space affordance with sounds.

VII. CONCLUSION

The significance of this work is twofold. First, artificial life

is shown as an autonomous chemical network that is translated

into a digital sensor network system. Second, this work shows

how an artificial life system behaves in an open environment

as opposed to a closed, simulated environment for relatively

longer periods of time.A most interesting finding here is that the network spon-

taneously generates a resonating state (and a resting state)

to a particular set of parameters or to the space and time

context, without having predefined conditions or functions. In

this study, we used only two kinds of sensors to organize

a chemical reaction, but, as with our human body sensors,

many sensors can be installed and integrated to give other

resonating states. We expect this to become a new method of

experimentation with artificial life in an open environment.

Acknowledgement This research work was supported by

the Grant-in-Aid for Scientific Research ”Theoretical Studies


of Diversity in Space Time Perception by Using a VR System”

(#24650117).

REFERENCES

[1] W. Walter, “An imitation of life,” Scientific American, vol. 182, no. 5, pp.42–45, 1950.

[2] R. Brooks, “A robot that walks; emergent behaviors from a carefullyevolved network,” Neural computation, vol. 1, no. 2, pp. 253–262, 1989.

[3] “Aibo,” http://www.sony.jp/products/Consumer/aibo/index.html.[4] Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki, and

K. Fujimura, “The intelligent asimo: System overview and integration,” inIntelligent Robots and Systems, 2002. IEEE/RSJ International Conferenceon, vol. 3. IEEE, 2002, pp. 2478–2483.

[5] G. Metta, G. Sandini, D. Vernon, L. Natale, and F. Nori, “The icubhumanoid robot: an open platform for research in embodied cognition,” inProceedings of the 8th workshop on performance metrics for intelligentsystems. ACM, 2008, pp. 50–56.

[6] T. Ikegami, “Studying a self-sustainable system by making a mind timemachine,” in Workshop on Self-Sustaining Systems. ACM, 2010, pp.1–8.

[7] ——, “A design for living technology: Experiments with the mind timemachine,” Artificial Life, vol. in press, pp. –, 2012.

[8] P. Gray and S. K. Scott, “Autocatalytic reactions in the isothermal,continuous stirred tank reactor; oscillations and instabilities in the systema+ 2b → 3b, b → c,” Chem. Engrg. Sci, vol. 39, pp. 1087–1097, 1984.

[9] J. E. Pearson, “Complex patterns in a simple system,” Science, NewSeries, vol. 261, pp. 189–192, 1993.


new creating space-time affordances via an autonomous sensor …oka... · 2020. 7. 23. · creating...

Documents