applying high-definition “hypermirror” to distance...

1

Applying High-Definition “HyperMirror” to Distance Learning Utilizing “KIZUNA”

By Akiko NAKAZAWA1), Taiichiro OKUBAYASHI

1), Hideki MORI1), Takanori MAESAKO

1), Osamu MORIKAWA2),

Masahiro NAKAO3), Naoya TOMII

3), Tomonori KURODA3), Tetsuo SATO

3), Teruo KAWASUGI4), and Gousei HASHIMOTO

4)

1) Graduate School of Human Sciences, Osaka University, Suita, Japan 2) National Institute of Advanced Industrial Science and Technology, Tsukuba, Japan

3) Japan Aerospace Exploration Agency, Tsukuba, Japan 4) Space Engineering Development Co. Ltd., Japan

This paper provides information about distance learning with a high-definition HyperMirror system utilizing the “KIZUNA” satellite and discusses applications of “KIZUNA” to education. HyperMirror is a special type of videoconferencing system. In HyperMirror, everyone appears in the same video image – in other words, everyone appears as if they are in the same room. HyperMirror has been used successfully for various distance learning projects, but it is necessary to use high-definition video in order to improve the video image quality of HyperMirror and allow a wider range of activities. “KIZUNA” allows the transmission and reception of high-definition video. As a result, we carried out a distance learning project with high-definition HyperMirror utilizing “KIZUNA.” The project was held from January 21st through 23rd, 2009 between elementary schools in Osaka and Oita in Japan. It consisted of 3 types of activities: exchange of information about their schools and towns, joint classes on space science and space development taught by specialists, and performances of traditional cultural activities with each other. The students enjoyed the distance learning activities and the project ended successfully. Several benefits to using high-definition video with HyperMirror were observed and “KIZUNA” overcame several of the common barriers to successful videoconferencing in schools. However, there are also limitations to the widespread implementation of the “KIZUNA” system so it is important to consider how to best take advantage of the unique benefits of “KIZUNA” when applying “KIZUNA” to education.

Key Words: ”KIZUNA”, HyperMirror, WINDS, High-definition, Distance Learning

1. Introduction Previous research on distance learning using high-definition video has shown benefits in using high-definition video instead of standard-definition video. For example, high-definition video allows students to read text on blackboards easier 1) and provides more realistic distance learning 2) than standard-definition video allows.

HyperMirror is a special type of videoconferencing system. Traditional videoconferencing systems have several weak points including unnatural eye contact and the disconnected feeling of being in two separate communication spaces. In order to solve these problems, HyperMirror was invented 3). In HyperMirror, everyone appears in the same video image – in other words, everyone appears as if they are in the same room. Participants see their mirror image in HyperMirror because this makes it easier to coordinate their movements with their video image on screen. Additionally, participants can interact with objects and materials, both local and remote. These characteristics of HyperMirror allow local and remote participants to do collaborative activities. For example, Fig. 1 shows an activity using HyperMirror in which Japanese and American students “virtually” shook hands. Various distance learning projects using HyperMirror have been carried out and it has become clear that HyperMirror provides richer distance

learning experiences because HyperMirror allows physical communication and promotes deeper understanding by allowing participants to “share” and “interact” with objects in both local and remote sites 4), 5).

In spite of the benefits, HyperMirror has brought to distance learning, there are still a number of unresolved issues. One such issue is video quality. It became necessary to improve the video image quality of HyperMirror because it can be difficult for participants to see pictures or letters on objects shown using HyperMirror. Switching to a high-definition video system can solve this problem.

However, one of the challenges of using high-definition video for HyperMirror is the need for a stable broadband

Fig. 1. Example of activity using HyperMirror: Japanese student and

U.S. student “virtually” shook hands in HyperMirror using a

standard-definition video system.

2

Internet connection (preferably at least 4Mbps). Although most Japanese schools have high-speed networks (many fiber-optic), network security problems are also common with firewall and NAT (Network Address Translation) systems blocking videoconferencing systems.

One solution to these problems is to utilize another network to connect videoconferencing systems. “KIZUNA” (WINDS), a satellite developed by JAXA and NICT, allows a stable wideband Internet connection with enough bandwidth to support the transmission and reception of high-definition video 6), 7).

Therefore, this paper explores the application of high-definition HyperMirror and “KIZUNA” to education and student evaluations of distance learning using the system. 2. Method 2.1. Details of Distance Learning 2.1.1 Network Environment As described in the Introduction, “KIZUNA” provides a stable wideband Internet connection that can support the use of a high-definition videoconferencing system. We set up portable satellite antennas at elementary schools in Osaka and Oita and connected the two schools via “KIZUNA” (Fig. 2). 2.1.2. Learning Environment Fig. 2 shows the distance learning environment and system structure for the project. The HyperMirror distance learning environment usually has a blue screen at a single site in order to create the HypeMirror video feed using chroma-key effects. This results in a HyperMirror video image that places the participants with the blue screen in the non-blue screen site. However, the distance learning described in this paper has blue screens installed at both sites in order to use pictures and movies from a computer as the common background because

teachers at both schools wanted to use pictures and movies in their presentations. As a result, the HyperMirror video image was composed with video images from Osaka and Oita overlayed on output video from a computer (Fig. 3). Documents for the distance learning activity (e.g. photos of the schools, movies, and astronomical charts) were loaded on the computer and displayed full screen to serve as a background. This allowed participants to interact with documents on their background as well as with remote participants in HyperMirror. Characteristics of the distance learning environment described in this paper: 1) All video images are high-definition video (video from

both schools and images from a computer). 2) Blue screens were set up at both sites and the HyperMirror

video was created by combining video images from Osaka and Oita with output from a computer.

Fig. 3. An example of composite HyperMirror image

Fig. 2. Simplified framework of distance learning environment and system structure

3

3) Video displayed on the monitor for participants in HyperMirror was also displayed on the monitor used by participants in the classrooms.

4) Participants in HyperMirror saw their mirror image on their local monitors with the mirror image of remote participants composited with them.

5) Signs that students held were prepared with mirrored text (which would appear as a normal text since all of the video participants saw in local monitors was shown as mirror images).

5) Documents on the computer were not displayed as mirror images (since documents were displayed as a separate video layer behind the mirror image HyperMirror video feeds).

2.1.3. Date, Participants, and Contents Our distance learning project was held from January 21st to January 23rd, 2009. Students of elementary schools in Osaka and Oita participated in the distance learning project. The school in Osaka is located in an urban area and has a total of about 600 students. By contrast, the school in Oita is located

Fig. 4. An example of a quiz activity

Fig. 5. Students used a pointer to highlight presentation materials

Fig. 7. Specialist lecturing using many image/text-rich slides

Fig. 8. Students studied constellations using astronomical charts

Fig. 6. Joint balloon rocket launching

Table 1. Participant and distance learning activities details

contents

school in Osaka 45 students (6 students from 3rd grade, 9 students from 4th grade, 11 students from 5th grade, 9 students from 6th grade)

school in Oita 23 students (4th grade)

total 68 students

school in Osaka107 students(1st lecture: 35 students, 2nd lecture: 35 students, 3rd lecture: 37 students; all students were 4th graders） *students attendedonly one of three lectures in the classroom connected with the remote site and watched the other 2 lectures in other classrooms becausethe classroom connected with the remote site could not fit all of the students at one time

school in Oita 91 students（18 students from 3rd grade, 22 students from 4th grade, 30 students from 5th grade, 21 students from 6th grade）

total 198 students

contents

school in Osaka 11 students (6th grade)

school in Oita 72 students （22 students from 4th grade, 31 students from 5th grade, 19 students from 6th grade）

total 83 students

participants

day 1

1. Life in space: International Space Station and Japanese Experiment Module "KIBO": lecture, simulated haircut in space, and Q&A

2. Overview about rockets and rocket-launching (history, how rockets work, what they are used for, etc.) : lecture, launching balloon rockets, and Q&A

1. Greetings and handshaking by City Mayors and Principals of both schools2. Introduction of each school by students3. Introduction of students' local towns　 : sights, festivals and dances, famous people, dialects, local products

day 2

day 31. Performance of traditional cultural activity from each school town: "Futon-Daiko" in Osaka and "Kodomo-Kagura" in Oita2. Joint Dancing; "Nanchu-Soran dancing"

participants

3. Constellations in winter, dawn of the universe, life cycle of stars, and planets in the solar system and comparison of their environments with earth: lecture,pointing out constellations/stars in the winter sky, and Q&A

contents

participants

4

in a rural area and there are only about 120 students in total. The number of participants in the distance learning activities is shown in Table 1. Distance learning activities consisted of three parts: exchange of information about their schools and towns, joint classes on space science and space development taught by specialists, and performances of traditional cultural activities with each other (Table 1.). 2.2. Methods of Analysis In order to evaluate distance learning with HyperMirror using “KIZUNA”, section 3 of this paper starts with a description of the characteristics of the distance learning activities. Section 4 discusses the questionnaire given to students and its results. Finally, section 5 discusses distance learning and educational applications of “KIZUNA” based on the results of the questionnaire. 3. Characteristics of Distance Learning Activities 3.1. Day 1

Day 1 focused on activities such as student presentations about their schools and towns. There were also "quiz" activities that took advantage of the HyperMirror environment. One side had students holding cards with answer choices and

the other side answered the questions with a student putting his/her hand on the shoulder of the remote student holding the card with the correct answer (Fig. 4).

Students also used a pointer to more easily interact with their presentation documents displayed on the background. Because of the use of high-definition video, participants could

Fig. 9. Joint dancing in HyperMirror

Fig. 10. Increased # of participants in HyperMirror environment

QuestionMean of Total Participants

(S.D. in Parentheses)

Osaka 4.44 (0.99)Oita 5.00 (0.00)Osaka 4.33 (1.15)Oita 5.00 (0.00)Osaka 3.96 (1.15)Oita 4.78 (0.67)Osaka 3.89 (0.98)Oita 4.91 (0.29)Osaka 2.98 (1.49)Oita 4.57 (0.73)



3.99 (1.26) n=8215. Did you feel as if you and the remote participants were dancing together?

4.24 (0.95) n=68

4.36 (1.11) n=1977. Do you want to participate in similar distance learning activities again in the future?

4. Was it easy to watch the video image of remote participants?

3.52 (1.48) n=67

Day 2

4.42 (1.00) n=1976. Did you enjoy the distance learning activities today?

5. Did you notice the delay in the video and audio?

Mean by Each School(S.D. in Parentheses)

1. Did you enjoy the distance learning activities today?

2. Do you want to participate in similar distance learning activities again in the future?

3. Did you feel as if you and the remote participants were interacting in the same room?

Day 1

4.63 (0.84) n=68

4.56 (0.98) n=68

4.24 (1.08) n=68

4.15 (1.16) n=184

3.34 (1.39) n=191

Day 3

8. Did you feel as if you and the remote participants were interacting in the same room?

9. Could you clearly see the photographs of stars and astronomical charts?

10. Did you notice the delay in the video and audio?

3.72 (1.29) n=197

14. Did you notice the delay in the video and audio? 3.37 (1.47) n=83

4.65 (0.71) n=83

12. Do you want to participate in similar distance learning activities again in the future? 4.69 (0.70) n=83

13. Could you clearly see the remote participants in the video image? 4.48 (0.83) n=82

11. Did you enjoy the distance learning activities today?

Table 2. Mean of rating scores of the questionnaire

5

see details of what presenters were highlighting using the pointer or presenters’ fingers (Fig. 5). 3.2. Day 2 On Day 2, two specialists of space science and space development lectured to a virtual joint class composed of students from both schools. The lecture also included some interactive activities such as a simulated haircut in space and joint balloon rocket launches (Fig. 6). During the lecture, specialists used a lot of text and images in their presentation documents displayed on the background, but participants could clearly see them. The specialists also highlighted material during their lessons by pointing to presentation materials displayed on the background (Fig. 7).

Because of the use of high-definition video, not only were texts and images very clear, but also astronomical charts. Students were able to study constellations and planets observed in winter with an astronomical chart displayed on the background and answer questions by pointing (Fig. 8). 3.3. Day 3 On Day 3, students introduced and performed traditional cultural activities for each other and involved students from both schools in interactive activities. Fig. 9 shows students from Osaka and Oita involved in a joint dancing activity in HyperMirror. During this activity, the time delay from using satellite communications was more noticeable. Another benefit of using high-definition video was allowing more participants to interact in the HyperMirror environment than previous standard definition video HyperMirror systems could accommodate (Fig. 10). 4. Questionnaire for Students and Results In order to measure students’ feelings about distance learning, we gave a questionnaire to the participating students. This section describes the questionnaire and its results. 4.1. Materials and Procedure Students participating in the distance learning project answered the questionnaire after each distance learning activity.

Students gave answers to the questions shown in Table 2 using a 5-point scale where 1 indicated strongly disagree and 5 indicated strongly agree. 4.2. Results Table 2 shows the mean values for answers to each question in the questionnaire. Most mean values for total participants were high except for questions 5, 8, 10, and 14. The mean for total participants for question 10 was especially lower than the same question on Day 1 (question 3). Additionally, the mean for the school in Osaka for question 14 was high and the mean for the school in Osaka for question 15 was low. 5. Discussion 5.1. Effects of High-definition Video on HyperMirror As described in Section 3, there were a number of benefits

from using high-definition video for HyperMirror. Especially, presentation materials and the facial expressions of participants in HyperMirror became clearer than was capable with standard-definition HyperMirror. In addition, the results of questions 4, 9, and 13 showed that participants noticed the improvements in the clarity of HyperMirror video.

The improved, sharper video image of HyperMirror also allowed teachers and students to use new materials such as astronomical charts in HyperMirror. Materials with fine details such as astronomical charts require high-resolution display in order to be effectively used in distance learning. HyperMirror using standard definition video has not allowed high enough resolution to use these kind of supplementary materials.

High-definition video also allows for more participants to fit on the screen and participate in activities in HyperMirror. Because the HyperMirror image size is fixed (i.e. you can't move or zoom the camera used for HyperMirror), there is a limit to the number of participants that can fit in a standard-definition HyperMirror image. When using high-definition video, the HyperMirror camera still remains fixed, however, the number of participants can be larger because the viewing angle of high-definition video is wider than standard-definition video. 5.2. Evaluation of Distance Learning According to the results of the questionnaire, students enjoyed the distance learning activities over the course of the 3 days of the project and they want to participate in distance learning activities again. These show that the three-day distance learning project was effective for students and successful. Questions 3 and 8 indicated that students felt more of a sense of being in the same room with remote students on day 1 than on day 2. This can be attributed to differences in the contents, activities, and types of classes held on Day1 and Day 2. The mean values for questions 5, 10, and 14 show that most participants were not bothered by the time delay caused by satellite communication, but some participants in Osaka were especially bothered by time delay on Day 3. This is most likely due to the activities carried out on Day 3 which accentuated the time delay: joint performances of traditional cultural activities and dancing. Music for the joint dancing was played from Osaka and students from Osaka led the Oita students in a joint taiko drum activity. The students in Oita received the audio and video with a slight delay from the satellite communication, but since they were only trying to follow the actions in the video image, the effects of the delay were not very noticeable for them. In Osaka, students danced to the music, but found that the dance movements of Oita students were not in sync with their own movements which (were in sync with the music). This is because the video of Oita students' actions was influenced by two delays from satellite communication. Oita students were moving in sync with the video from Osaka students, creating one delay and Osaka students viewing the video of Oita students' actions introducing another delay. Students from both schools were able to continue dancing in sync with the music they heard,

6

but Osaka students' responses to question 15 on day 3 showed a lesser sense of the feeling of dancing together so the double delay can be attributed to starting to negatively impact the sense of being and interacting in a joint space simultaneously that the HyperMirror system normally provides. This distance learning project received very positive evaluations from the participating students. However, it is important to consider the type of activities used to prevent the confusion that can result from trying to do timing-sensitive activities time delay inherent satellite communication based systems. 5.3. Applying “KIZUNA” to Education “KIZUNA” played an important role in the implementation of distance learning using high-definition video for this project. As described in Section 1, a stable broadband Internet connection is required for the transmission and reception of high-definition video. “KIZUNA” not only allowed the transmission and reception high-definition video, but also connected urban and rural schools over a great distance.

One point to consider when applying “KIZUNA” to distance learning is time delay. In the distance learning project described in this paper, some students were bothered by time delay during certain activities that were timing-sensitive. It is estimated that students can adjust to the time delay after a short period of time. However, it is preferable to choose activities that are less timing-sensitive and take advantage of the benefits of “KIZUNA.”

As stated in Section 1, for many schools in Japan, it is difficult to use videoconferencing systems. “KIZUNA” overcame these difficulties and allowed the schools to participate in a wide variety of activities facilitated by the high-definition video environment. “KIZUNA” has could be effective as a special tool for communication in the field of education. However, there are several limitations and barriers to schools using “KIZUNA” freely: set-up of special equipment to use “KIZUNA”, cost, and limited access.

Since opportunities to use “KIZUNA” are limited at the present time, it is important to consider whether activities or contents take advantage of the unique benefits of “KIZUNA” and decide how to utilize them when applying “KIZUNA” to education, especially distance learning. 6. Conclusion and Future Prospects This paper aims to provide information about distance learning with a high-definition HyperMirror system utilizing the “KIZUNA” satellite and discuss applications of “KIZUNA” to education. A distance learning project between two schools in Japan using high-definition HyperMirror and “KIZUNA” was successfully carried out between January 21st and 23rd, 2009. A number of benefits from using high-definition video with HyperMirror and the “KIZUNA”

satellite, but several limitations were also realized. The increased clarity of the high-definition video image allowed the use of a wider range of supplementary materials during videoconferences and contributed to an overall better videoconferencing experience. The “KIZUNA” satellite connected both urban and rural schools separated by a great distance and allowed unrestricted high-speed data transfer to support the high-definition HyperMirror videoconferences. The specialized equipment, set-up, and cost limit widespread use of “KIZUNA” currently, but it “KIZUNA” may be useful as a special tool in the education field for projects and activities that can take advantage of its unique benefits. In the future, we want to continue testing and evaluating distance learning using high-definition HyperMirror and “KIZUNA” in different configurations (for example multipoint connections utilizing “KIZUNA”). Acknowledgments We extend special thanks to the teachers and students of participating elementary schools in Osaka and Oita as well as the many other people who helped to carry out this distance learning project.

References 1) Shimizu, Y. and Shiroma, S.: Evaluation of HDTV Closed Circuit

System for Distance Education, The Journal of the Institute of Television Engineers of Japan., 44,12 (1990), pp.1717-1722.

2) Nishihara, I., Nakano, S., Nakashima, M., Aoki, K., and Kuroda, T.: BS-7-10 The study of high-presence remote education system with HDTV (High Definition Tele Vision), Proceedings of the Society Conference of IEICE , 2006_Communication (2), (2006), pp. "S-41"-"S-42".

3) Morikawa, O. and Maesako, T.: HyperMirror: Toward Pleasant-to-use Video Mediated Communication System, Proceedings of the 1998 ACM conference on CSCW, (1998), pp. 149-158.

4) Imai, A., Yamashiro, S., Matsukawa, H., Yamada, M., Maesako, T., Shibao, M., Okuji, K., and Ihara, K.: Pilot Study of HyperMirror System via the Internet in Elementary Schools along International Exchange between Japan and South Korea, Transactions of Japanese Society for Information and Systems in Education, 19,4 (2002), pp. 261-266.

5) Matsukawa, H., Imai, A., Shigeta, K., Okano, K., Kagehira, Y., Maesako, T., Utsumi, S., and Seki, Y.: A Case of Distance Learning Using Hyper Mirror System Via Satellite Phone, Japan journal of educational technology, 28,Suppl. (2005), pp.257-260.

6) Sato, T., Nakamura, Y. and Araki, T.: 3-1 Overview of WINDS Satellite, Review of the National Institute of Information and Communications Technology, 53, 4 (2007), pp.9-16

7) Kadowaki, N. and Suzuki, R.: 2 Overiew of the Wideband Internet Engineering Test and Demonstration Satellite Project, Review of the National Institute of Information and Communications Technology, 53, 4 (2007), pp.3-8