visitor evaluations communications report
TRANSCRIPT
Candidate Number: KYVH7
Evaluation Report ARCLG190 Museum Communication
Leventis Gallery Visitor Evaluations INSTITUTE OF ARCHAEOLOGY AT UCL
1 | E v a l u a t i o n R e p o r t K Y V H 7
Table of Contents
EXECUTIVE SUMMARY …………………………………………………………………………………………………………2
CONTEXT FOR THE EVALUATION PROJECT AND ITS TARGET AUDIENCE ……………………………….3
PURPOSE OF THE EVALUATION PROJECT & EVALUATION QUESTION …………………………………..4
METHOD(S) USED AND WHY ……………………………………………………………………………………………….5
PRESENTATION AND DISCUSSION OF KEY FINDINGS ……………………………………………………………7
CONCLUSIONS & RECOMMENATIONS ……………………………………………………………………………….13
EFFECTIVENESS OF THE METHOD AND THE TOOL ………………………………………………………………14
REFERENCES ………………………………………………………………………………………………………………………15
APPENDIX I: EVALUATION BRIEF…………………………………………………………………………………………16
APPENDIX II: INTERVIEW QUESTIONS …………………………………………………………………………………18
APPENDIX III: INTERVIEW RESULTS …………………………………………………………………………………….22
2 | E v a l u a t i o n R e p o r t K Y V H 7
EXECUTIVE SUMMARY
The purpose of this report is to detail the 33 visitor interviews conducted between 11 and
17 December 2015 by MA Museum Studies students in the Institute of Archaeology’s (IoA)
Leventis Gallery at University College London (UCL). Along with the non-‐visitor interviews
and visitor observations also conducted by the same researchers, these interviews
constitute the front-‐end evaluation for an exhibit on tools and the human mind to be
installed in the Leventis Gallery in May 2016. From the data collected, the installation group
understands the visitor profile of the current exhibition, their motivations for visiting, and
their opinions on current and potential use of space and content of exhibitions.
TOP FINDINGS:
The collected data shows of the 33 interviews conducted:
• All visitors were connected to UCL: student or staff (28 as students).
• 30 visit the gallery at least once per week.
• None cited viewing the exhibition as their primary reason for entering the gallery.
• 16 reported it is unclear We Need to Talk: Connecting Through Technology is
separate from the permanent display and 4 reported it was not immediately clear.
• 30 reported an interest in seeing either stone or modern tools in the future
exhibition; 25 expressed an interest in both.
• Anthropology, Archaeology, Medicine/Psychology, and Technology were themes
visitors were interested in regarding tools; Medicine/Psychology being most popular
(10).
• 10 expressed specific interest in interactive elements; only 3 were not interested.
3 | E v a l u a t i o n R e p o r t K Y V H 7
CONTEXT FOR THE EVALUATION PROJECT AND ITS TARGET AUDIENCE
CONTEXT
The Leventis Gallery is located on the ground floor of UCL’s IoA en route to the UCL lecture hall G6, an office, and IoA toilets. Over half of the space is comprised of a permanent archaeological exhibition on the Levant and public seating/study areas. Two cases and additional sections of the gallery are dedicated for the use of the Exhibition Project, an optional module for MA Museum Studies students, which annually develops an exhibit. Currently, We Need to Talk: Connecting Through Technology fills this section of the gallery. The subject for this next temporary exhibit will be tools and the human mind. A front-‐end evaluation study is required for the successful development of this future exhibition.
TARGET AUDIENCE
• Visitors in the Leventis Gallery
4 | E v a l u a t i o n R e p o r t K Y V H 7
PURPOSE OF THE EVALUATION PROJECT & EVALUATION QUESTIONS
PURPOSE
Thirteen UCL MA Museum Studies (2015-‐2016) students conducted visitor interviews with the purpose of:
• Gathering data from visitors to the current exhibits in the Leventis Gallery.
• Conducting a front-‐end evaluation to be used in the development of the upcoming
exhibit on tools and the human mind by combining visitor interview data with the
simultaneously conducted non-‐visitor interview and visitor observation data to
achieve this.
EVALUATION QUESTIONS
The evaluation team sought to answer the following questions1:
• Who uses the foyer space on the ground floor of the Institute?
• Why do people use the area?
• Do people realise that there are two different exhibitions in this area?
• What do people think We Need to Talk: Connecting Through Technology exhibition is
about?
• What do visitors want to see in a future exhibit about tools and the human mind?2
The questionnaire3 did this by dividing the questions into three main parts:
• Part 1 – Purpose of visit and use of space
• Part 2 – Interests relating to the future exhibition
• Part 3 – Visitor demographics
1 Appendix I: Evaluation Brief 2 This question was added to the original Evaluation Brief by the evaluation team during review. 3 Appendix II: Interview Questions
5 | E v a l u a t i o n R e p o r t K Y V H 7
METHOD(S) USED AND WHY
METHOD
The applied research method used was interviews with both open-‐ended and structured
components. The study used qualitative methods to allow for individuality and originality in
the responses and identify interesting exceptions to the majority (Diamond 1999:23). This
evaluator has produced qualitative and quantitative data, from which the team can analyse
the content statistically and extract patterns and repeated themes from the responses
(Diamond 1999: 89 & 151).
EVALUATORS
The evaluation team was comprised of 13 MA Museum Studies Students who are familiar
with the gallery space and the IoA as well as the content of the exhibitions and therefore,
did not need much preparation time before the study commenced (Diamond 1999: 18).
QUESTIONS
The interview questions were approved by the entire evaluation team and tested with a
pilot test prior to the conducted research (Diamond, Luke Uttal 2009: 80). The interview
employed a mixture of open and closed questions. The closed questions largely related to
individual demographics and visiting history (largely YES/NO questions) (Diamond 1999: 89);
the open questions provided the interviewee with more room to express their opinions to
openly and creatively inform the development of the future exhibition (Diamond 1999: 88).
6 | E v a l u a t i o n R e p o r t K Y V H 7
RECRUITMENT
Individual evaluators chose their own time and date for research and determined their own
method of recruitment; due to low visitor numbers during their time slot, this evaluator’s
team chose to invite every visitor who entered the space to participate.
ETHICS
All evaluators introduced themselves as IoA MA Museum Studies students and kept the
identities of the participants anonymous by coding interviews by interview number. The
evaluators explained the purpose of the study prior to the interview and thanked all
interviewees for their participation at the conclusion (Diamond 1999: 33, 83-‐90).
7 | E v a l u a t i o n R e p o r t K Y V H 7
PRESENTATION AND DISCUSSION OF KEY FINDINGS
VISITOR DEMOGRAPHICS
21 of the visitors were alone and nearly two-‐thirds were female. All visitors interviewed had a connection to UCL, the overwhelming majority (28/33) as students. Additionally, a significant portion of those interviewed were individuals under the age of 26.
Figure 1. Responses to Questions 13: “Age Profile” and 16: “Occupation.” Authors own. Appendix III: Interview Results.
Figure 2. Responses to Question 14: “Group Size.” Authors own. Appendix III: Interview Results.
0
2
4
6
8
10
12
14
16
18-‐21 22-‐25 26-‐34 35-‐44 45-‐54 55+
Staff
Students
0
5
10
15
20
25
Individual Pairs Trios
Group Size
Group Size
8 | E v a l u a t i o n R e p o r t K Y V H 7
Figure 3. Responses to Question 18: “Gender.” Authors own. Appendix III: Interview Results.
MOTIVATIONS
Though 30 of the 33 are considered frequent visitors (1+ day per week), not one interviewee cited the exhibition itself as their primary motivation for being in the gallery. The overwhelming majority (28/33) were present for either student or work related activities. The remaining 5 interviewees indicated they were using the gallery as a public IoA space.
Figure 4. Responses to Question 4: “Are you a frequent visitor?” Authors own. Appendix III: Interview Results.
Gender
Male
Female
Are you a frequent visitor?
Yes
No
9 | E v a l u a t i o n R e p o r t K Y V H 7
Student Business Work Leisure Toilet
22
6
2
3
Figure 5. Responses to Question 1: “What is the purpose of your visit here today?” Authors own. Appendix III: Interview Results.
CLARITY OF THE SPACE
Figure 6. Responses to Question 7: “Is it clear that there are two exhibitions in this space?” Authors own. Appendix III: Interview Results.
19 of 33 interviewed responded it was either not clear or not immediately clear the gallery held two exhibitions. Nearly half of those (9/19) had looked at the exhibits that same day.
To differentiate the two exhibitions:
• 10 indicated “better,” “larger,” “eye-‐catching” or “more signage” • 5 said physically “dividing” or “separating” the two exhibitions • 2 responded visitors could discover themselves if given “more information” or
“interacting with the exhibitions” • 2 were unsure
Better Signage Physical Separation Visitor Interaction Not Sure or N/A
10
5
2
2
Figure 7. Responses to Question 7.2: “If no, how do you think you could make that clearer?” Authors own. Appendix III: Interview Results.
Is it clear there are two exhibiXons in this space?
Yes
No
Not immediately
10 | E v a l u a t i o n R e p o r t K Y V H 7
OPINIONS ON THE WE NEED TO TALK: CONNECTING THROUGH TECHNOLOGY EXHIBIT
Figure 8. Responses to Question 6: “Did you have the chance to see the We Need to Talk: Connecting Through Technology exhibition today?” Authors own. Appendix III: Interview Results.
Of the 21 individuals who had previously seen the exhibit (16 on the day and 5 on previous days), 11 responded positively when asked their opinions of We Need to Talk.
Positive responses include:
• “interesting,” “enjoyed it,” “glad it’s there” • positive comments on the contrast to the permanent exhibition or amount of
objects covered
Negative responses include:
• “unchallenging,” “not very imaginative,” “not a clear theme,” “Eurocentric bias”
Positive Responses Negative Responses Neutral Responses No Responses
11
3
3
4
Figure 9. Responses to Question 6.1: “What did you make of it?” Authors own. Appendix III: Interview Results.
Visitors who have seen 'We Need to Talk' today
Yes
No
Not Today
11 | E v a l u a t i o n R e p o r t K Y V H 7
INTEREST IN A FUTURE EXHIBIT ON TOOLS AND THE HUMAN MIND
30 of 33 interviewees indicated an interest in either stone or modern tools; 25 expressed an interest in both. Only 2 stated they were “not really” or “not interested” in the topic. 3 responded with noncommittal “sure,” “might be” and “depends,” and 1 was confused over the definition of “tool;” these responses were not included.
Figure 10. Responses to Questions 9: “Are you interested in seeing stone tools and learning how they might have
contributed to the evolution of the human mind?” and Question 10: “Are you interested in seeing more modern tools?” Authors own. Appendix III: Interview Results.
Figure 11. Responses to Questions 9: “Are you interested in seeing stone tools and learning how they might have
contributed to the evolution of the human mind?” and Question 10: “Are you interested in seeing more modern tools?” Authors own. Appendix III: Interview Results.
0
5
10
15
20
25
30
Stone Tools Modern Tools
Yes
No
Interest in Both Stone and Modern Tools
Yes
No
Not Interested
12 | E v a l u a t i o n R e p o r t K Y V H 7
In response to Question 8: “What subjects relating to the theme of tools and the human mind would you be interested in?”:
• 25 interviewees identified interest specific themes • The field of medicine/psychology was named over other fields (10/25)4 • 2 indicated a general inerest in the topic • 1 was not interested • 1 stated the topic was “overdone” • 3 were confused by the question • 1 did not answer
Figure 12. Responses to Question 8: “What subjects relating to the theme of tools and the human mind would you be interested in?” Authors own. Appendix III: Interview Results.
From the 23 interviewees who responded to Question 11: “What else would you like to be able see / do in the exhibition?” two recurring interests developed:
• Handling Objects (7) • Interactives and Technology (10)
3 responded they were specifically not interested in more interactive elements.
4 This mirrors the significant portion of the interviewees who were psychology students waiting for a class in G6. This is discussed further in “Effectiveness of the Method and the Tool,” pg. 14.
0 2 4 6 8 10 12
Anthropology
Archaeology
Medicine/Psychology
Technology
Interest in Specific Themes
Interest in Specific Themes
13 | E v a l u a t i o n R e p o r t K Y V H 7
CONCLUSIONS & RECOMMENDATIONS
CONCLUSIONS
Based on the presented findings, this evaluator concludes:
• Visitors to the gallery are connected to UCL, overwhelmingly as students of the university.
• The purpose of these visits are not to view the exhibits themselves. • The two exhibits are not clearly distinguished from one another, even to visitors
who looked at the exhibitions that very day. • There is considerable interest in seeing both stone and modern tools as well as
interactive elements and handling opportunities in the developing exhibition on tools and the human mind.
• Interest in exhibit content correlates to subjects the student studies.5
RECOMMENDATIONS
Given this small sample, comprising only a third of the entire research, and taking into account the abilities of a small development team with time and resource limitations, this evaluator recommends the following for the development of the future exhibition on tools and the human mind:
• Utilize eye-‐catching design techniques that invite waiting or passing individuals to interact with the exhibit and the iPads.
• Create a visual marker to distinguish the two exhibitions for visitors (i.e. banner or large introductory sign to the temporary exhibit).
• Develop an exhibit that clearly defines the term “tool” and covers the progression or evolution of these tools from stone to modern.
• Connect the content of the developing exhibition to the class subjects taught in G6 (if available).
It should be noted these recommendations are preliminary and have not been compared to the remaining components of this front-‐end study: non-‐visitor interviews and visitor observations. The results from all three components should be compiled and interpreted to obtain more valid recommendations for the development team.
5 Discussed further in “Effectiveness of the Method and the Tool,” pg. 14.
14 | E v a l u a t i o n R e p o r t K Y V H 7
EFFECTIVENESS OF THE METHOD AND THE TOOL
COMMUNICATION
Each of the 13 evaluators were allowed to prompt the interview questions spontaneously, presenting each question to the interviewee in a different way and therefore, the data may not be consistent and difficult to standardize (Diamond 1999: 88). The evaluator also carries the possibility of imparting his or her own bias on the exhibit due to the extensive knowledge the evaluators have on the gallery (Diamond, Luke & Uttal 2009: 69).
Both the evaluation team and interviewees were a mix of native and non-‐native English speakers, which could result in miscommunication in an English interview; additionally, the origin of even the native English speakers (USA, UK, Australia) could cause confusion amongst themselves if colloquial terms were employed during the interview.
The vocabulary and language of the questions themselves can alter the data if an interviewee interprets the definition of a word differently than the evaluator. This occurred specifically with regards to the term “tool” appearing in Questions 8, 9, and 106. Three interviewees expressed confusion around the definition of the term “tool” and two specifically requested an explanation, the way in which the evaluator shaped the interviewees understanding of the term “tool” in turn determines his or her response to the question.
In interpreting the data evaluators also have their own language they use to code the responses as this evaluator has done with Question 87. This data may differ with each evaluator’s unique coding system. Similarly, the lack of clarity in the interview responses causes difficulty for an evaluator to interpret. One interviewee said “sure” in response to both questions regarding interests in seeing stone or modern tools, both closed-‐ended “YES/NO” questions. “Sure” does not fall squarely within “YES” or “No,” therefore this evaluator did not include this response.
INTERVIEW TIMES
A greater number of females were interviewed than males. This could be a result of the time and day of the interviews, a more even gender distribution would be expected with a longer-‐running evaluation resulting in a larger number of interviews.
The data compiled shows a high interest in seeing psychology and medically related content in the exhibit on tools and the human mind. This directly correlates to the high number of psychology students interviewed while waiting for a lecture in G6. This group may not be representative of the larger visitor population and also may not be returning to see the future exhibit if they no longer have a class in G6. 6 Appendix II: Interview Questions 7 See Figure 12.
15 | E v a l u a t i o n R e p o r t K Y V H 7
REFERENCES
Diamond, J. 1999. Practical Evaluation Guide: Tools for Museums and Other Informal Education Settings. Walnut Creek, CA: AltaMira Press.
Diamond, J., Luke, J.J. & Uttal, D.H. 2009. Practical evaluation guide: tool for museums and other informal educational settings (2nd ed.). Lanham, Md.: AltaMira Press.
16 | E v a l u a t i o n R e p o r t K Y V H 7
APPENDIX I: EVALUATION BRIEF
IOA Foyer exhibition space
Evaluation Brief
Evaluation aim The aim of this evaluation study is to collect some preliminary data about awareness and use of the exhibition space at the foyer on the ground floor of the Institute both from visitors and non-visitors. Findings will be used to inform the development of a new exhibition on Tools and the Human Mind. Evaluation questions To meet its aim, the evaluation sets out to answer the following questions, using either observations or interviews: Observations with visitors:
· Who uses the foyer space on the ground floor of the Institute?
· How is the spaces used?
· Are there different patterns of use at different times and dates?
· How long do people stay in the area?
· Are there any distinct modes of engagement at the exhibition space? Interviews with visitors:
· Who uses the foyer space on the ground floor of the Institute?
· Why do people use the area?
· Do people realise that there are two different exhibitions in this area?
· What do people think We Need to Talk: Connecting Through Technology exhibition is about?
Interviews with visitors:
· Who uses the foyer space on the ground floor of the Institute?
· Why do people use the area?
· Do people realise that there are two different exhibitions in this area?
· What do people think We Need to Talk: Connecting Through Technology exhibition is about?
Interviews with non-visitors:
· Do people in the local area (including passers-by, people who live and/or work in the area, UCL students and staff) know of the IOA?
o If so, do they know where it is located?
o How did they find out about it?
17 | E v a l u a t i o n R e p o r t K Y V H 7
· Do people in the local area (including passers-by, people who live and/or work in the area, UCL students and staff) know that the IOA is part of UCL?
· Have they ever been there? o If yes, when/why/what for? o If no, why not?
o What do they think they might find there (activities people carry out in the building)?
· Specifically, are people in the local area aware of the exhibition space on the ground floor of IOA?
O If yes, have they ever been?
o What did they do/see there? · {Explain what this space is about} Would they like to visit the exhibitions? What type (or theme) of exhibition they would like to see? And how it relates to their interests?
We Need to Talk: Connecting Through Technology concept Communication is an important part of our lives and technology plays a big role in how we interact with each other. Changes in technology have brought people together, spreading ideas around the world or in the same room. New technologies have often been marketed as an easy way to communicate with people. But these changes have also been met with worry that technology can isolate or control an individual or group. We Need to Talk: Connecting Through Technology looks at how communication technologies have influenced individuals and societies across different periods of time, and addresses the eagerness and anxieties that ensue. Evaluation methods You will be divided in three group and you will either: 1) observe visitors at the exhibition space, or 2) interview visitors at the exhibition space, or 3) interview non-visitors near the IOA and around UCL. Each student will conduct a minimum of 2-3 observation or interviews at busy periods. Framework for implementation Data collection will take place between 11 and 16 December 2015.
18 | E v a l u a t i o n R e p o r t K Y V H 7
APPENDIX II: INTERVIEW QUESTIONS
Visitor interview
Hello. My name is ..... and I am a student here at the Institute of Archaeology. The Institute is developing a new exhibition {interviewer: point to the temporary exhibition area and the interactive ‘totem’}. We would like to find out what our visitors think about that and I was wondering if you might help us. It will take about 10 minutes and we can have a sit {Interviewer: point at sitting area}.
PART 1
1. What is the purpose of your visit here today? {prompt: do you have a class, did you come to see the exhibition etc}
2. Have you been here before? YES / NO
3. When was the last time you visited?
4. Are you a frequent visitor? If so, why? E.g. (staff, students)?
5. Have you had a chance to see the exhibitions in the space?
5.1. If yes, when and what did you look at? (qualify, have they seen parts, whole, etc.?)
19 | E v a l u a t i o n R e p o r t K Y V H 7
6. {i f no t her e to s e e the exhib i t ion} Did you have the chance to see the We need to Talk: Connec t ing Through Techno logy exhibition today ?
6.1 What did you make of it?
7. Is it clear that there are two exhibitions in this space? YES / NO
7.1. If yes, what do you think they are about? {Interv i ewer : on ly ask i f v i s i to r s have no t a l r eady cover ed th i s}
7.2. If no, how do you think you could make that clearer?
20 | E v a l u a t i o n R e p o r t K Y V H 7
PART 2
A group of Museum Studies students will develop a new exhibition which will replace the one over there {point at two display cases} as well as the content in the interactive display {point at totem}. The exhibition will be about tools and the human mind.
8. What subjects relating to the theme of tools and the human mind would you be interested in?
9. Are you interested in seeing stone tools and learning how they might have contributed to the evolution of the human mind? YES/NO
10. Are you interested in seeing more modern tools? YES/NO
11. What else would you like to be able to see/do in the exhibition? (specific objects, other time periods interactives, lectures, online resources, etc.)
12. Is there anything else you would like to add?
21 | E v a l u a t i o n R e p o r t K Y V H 7
PART 3
Finally, may I ask you a few questions about you?
13. Age profile for themselves {Interviewer: show visitors the age groups below and ask them to point out which group they belong to}
Your age
q under 18 (what is your age?) _________
q 18-21 q 22-25 q 26-34 q 35-44 q 45-54
q 55-59 q 60-64 q 65-74 q 75+ q prefer not to say
14. Group size: 1 2 3 4 5 >5
15. Who with? Colleagues Friends (peers)
Family Family and Friends
16. Occupation: Student Staff Other
17. Where do you normally live?
! London ! Home Counties (which?) _________________________________________________ ! Rest of the UK ! Overseas (which country?)
{Interviewer: thank the visitor and note visitor’s gender}
Male / Female
22 | E v a l u a t i o n R e p o r t K Y V H 7
APPENDIX III: INTERVIEW RESULTS
9LVLWRU�4XHVWLR
QQDLUH�1XP
EHU(YDOXDWRU
'DWH
7LPH
4�����$
JH3URILOH
4����*
URXS�VL]H4����:
KR�:LWK
4���
2FFXSDWLRQ
4�����/RFDWLRQ
�.HUU\�-RFHO\Q
����'HFHP
EHU�����DP
������
��6WXGHQW
/RQGRQ
�.HUU\�-RFHO\Q
�����'HFHP
EHU�����DP
������
��6WDII
/RQGRQ
�:LOO
�������'HFHP
EHU�����DP
������
�6WXGHQW
6XUUH\
�:LOO
�����'HFHP
EHU�����DP
������
�6WXGHQW
�5HVHDUFK�3
+'�
6XVVH[
�:LOO
�����'HFHP
EHU�����DP
������
�6WXGHQW
(VVH[
�:LOO
�����'HFHP
EHU������3
0�����
��
VWXGHQW/RQGRQ��IURP
86$�
�0DU\�.
LWW\���'
HFHPHEHU
������30
������
�6WXGHQW
/RQGRQ
�0DU\�.
LWW\���'
HFHPHEHU
������30
������
�6WXGHQW
/RQGRQ
�0DU\�.
LWW\���'
HFHPHEHU
������30
������
�6WXGHQW
2YHUVHDV�
1HWKHUODQGV
��0DU\�.
LWW\���'
HFHPHEHU
������30
������
�6WXGHQW
5HVW�RI�8
.
��0DU\�.
LWW\���'
HFHPHEHU
������30
������
��6WDII
/RQGRQ
��(PPD�/\GLD
����'HFHP
EHU�����
������
��VWDII
/RQGRQ
��/\GLD�(
PPD
����'HFHP
EHU�����
������
)ULHQGV��SHHUV�VWXGHQW
/RQGRQ
��(PPD�/\GLD
����'HFHP
EHU�����
������
IULHQGV��SHHUV�VWXGHQW
+RP
H�&RXQW\
�6XUUH\�
��(PPD�/\GLD
����'HFHP
EHU�����
������
��VWXGHQW
2YHUVHDV��8
6$�
��%HWKDQ\�4
LQJ4LQ
����'HFHP
EHU�����3
0�����
���
6WXGHQW
/RQGRQ
��%HWKDQ\�4
LQJ4LQ
����'HFHP
EHU�����3
0�����
���
6WXGHQW
/RQGRQ
��%HWKDQ\�4
LQJ4LQ
����'HFHP
EHU�����3
0�����
���
6WXGHQW
/RQGRQ
��%HWKDQ\�4
LQJ4LQ
����'HFHP
EHU�����3
0�����
���
6WXGHQW
/RQGRQ
��$OOLVRQ
����'HFHP
EHU�����
������
��6WXGHQW
/RQGRQ
��$OOLVRQ
����'HFHP
EHU�����
������
��6WXGHQW
+HUWIURGVKLUH
��$OOLVRQ
�����'HFHP
EHU�����
������
��6WXGHQW
/RQGRQ
,QWHUYLHZ�'HWDLOV
3DUW����'HP
RJUDSKLFV
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
23 | E v a l u a t i o n R e p o r t K Y V H 7
9LVLWRU�4XHVWLR
QQDLUH�1XP
EHU(YDOXDWRU
'DWH
7LPH
4�����$
JH3URILOH
4����*
URXS�VL]H4����:
KR�:LWK
4���
2FFXSDWLRQ
4�����/RFDWLRQ
��$OOLVRQ
����'HFHP
EHU�����
������
��6WXGHQW
/RQGRQ
��$QQFKHQ
����'HFHP
EHU�����
������
��6WDII
/RQGRQ
��$QQFKHQ
����'HFHP
EHU������
������
��6WDII
+HDUWIRUGVKLUH
��D6WDFH\�,VDEHO
�����'HFHP
EHU�����
������
)ULHQGV��SHHUV�6WXGHQW
/RQGRQ
��E6WDFH\�,VDEHO
�����'HFHP
EHU�����
������
)ULHQGV��SHHUV�6WXGHQW
/RQGRQ
��6WDFH\�,VDEHO
�����'HFHP
HEU�����
������
��6WXGHQW
/RQGRQ
��6WDFH\�,VDEHO
�����'HFHP
HEU�����
������
��6WXGHQW
/RQGRQ
��6WDFH\�,VDEHO
����'HFHP
EHU�����
������
��6WXGHQW
/RQGRQ
��D6WDFH\�,VDEHO
����'HFHP
EHU�����
������
)ULHQGV��SHHUV�6WXGHQW
/RQGRQ
��E6WDFH\�,VDEHO
����'HFHP
EHU�����
������
)ULHQGV��SHHUV�6WXGHQW
/RQGRQ
��F6WDFH\�,VDEHO
����'HFHP
EHU�����
������
)ULHQGV��SHHUV�6WXGHQW
/RQGRQ
,QWHUYLHZ�'HWDLOV
3DUW����'HP
RJUDSKLFV
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
24 | E v a l u a t i o n R e p o r t K Y V H 7
4����*
HQGHU4���3
XUSRVH�RI�YLVLW4���9
LVLWHGEHIRUH
4���/DVW�9
LVLW4����9
LVLWIUHTXHQF\
4���+
DYH�\RXVHHQ"
4�����:
KHQ�DQG�ZKDW�GLG�\RX�ORRN�DW
)HPDOH
9LVLW�WR�EDWKURRP
��VWXGHQW<HV
$�IHZ
�GD\V�DJRIUHTXHQW�YLVLWRU
<HV���JODQFHG
EXW�QHYHU�FORVHO\ORRNHG
,Q�SDVVLQJ��QHYHU�UHDOO\�VWRSSHG�WR�H[DPLQH
FORVHO\��0RVWO\�ORRNLQJ�DW�SRWWHU\
0DOH
/HFWXUHU<HV
DERXW�����ZHHNV
DJR<HV��,P
�DOHFWXUHU
<HV
%HFDXVH�,P
�KHUH�DOO�WKH�WLPH�,�GRQW�UHDOO\
ORRN��*HQHUDO�UHFROOHFWLRQ�LV�WKDW�,�HQMR\�WKH
H[KLELWLRQV��/LNH�WKH�IDFW�YLVLWRUV�FRPH�IURP
RXWVLGH�WR�OHDUQ��*RRG�WR�KDYH�LW�E\�WKH
OHFWXUH�KDOO�EHFDXVH�WKH�VSDFH�LV�UHQWHG�RXW�PDQ\�SHRSOH�FDQ�VHH�LW�
)HPDOH
/HFWXUH<HV
$�ZHHN�DJR
)UHTXHQW�YLVLWRU�VWXGHQW�
<HV
%RWK�H[KLELWLRQV
)HPDOH
6WXG\�DW�,2
$<HV
DERXW�D�\HDU�DJ,QIUHTXHQW��QRWEDVHG�LQ�/RQGRQ
1R��QRW�\HW
�
)HPDOH
/HFWXUH<HV
/DVW�ZHHN
IUHTXHQW�YLVLWRU�VWXGHQW�
\HV*ODFLQJ�ORRN��QRW�D�JRRG�ORRN
)HPDOH
/HFWXUH\HV
ODVW�ZHHN
IUHTXHQW�YLVLWRU�VWXGHQW�
+DG�D�FKDQFH
WR��EXW�KDYH�QRW�
)HPDOH
&ODVV
<HV
���GD\V�DJR�IUHTXHQW�YLVLWRU
�VWXGHQW�<HV
�MXVW�WKH�SRWV�RQ�WKH�OHIW�
)HPDOH
�JR�WR�WKH�OLEUDU\�<HV
7KXUVGD\IUHTXHQW�YLVLWRU
�VWXGHQW�<HV
�WKH�FHUDPLFV�DQG�WKH�P
DOO�PDGQHVV�ERDUG
JDPH�
0DOH
&RXUVH
<HV
7KXUVGD\IUHTXHQW�YLVLWRU
�VWXGHQW�<HV
�QRWKLQJ�VSHFLILF��MXVW�ZDONHG�SDVW�
)HPDOH
�NLOOLQJ�WLPH�
<HV
�ODVW�0RQGD\�
IUHTXHQW�YLVLWRU�VWXGHQW�
<HV
�ELWV�RI�LW��WKH�SRWWHU\�VWXII�
0DOH
:RUN
<HV
)ULGD\IUHTXHQW�YLVLWRU
�VWDII�<HV
�WKH�VWXGHQW�H[KLELWLRQV��EHFDXVH�WKH�SRWVDUH�VWDWLF�
0DOH
:RUNV�LQ�EXLOGLQJ
<HV
/DVW�ZHHN
IUHTXHQW�YLVLWRU\HV
'LGQW�UHDOO\�ORRN
)HPDOH
6WXGHQW�VLWWLQJ�RQ�FRXFKHVUHOD[LQJ�EHIRUH�DQ�H[DP
\HVIULGD\
�����'HFHP
EHU�)UHTXHQW�YLVLWRU�
VWXGHQW\HV
/RRNHG�DW�SRWWHU\�DQG�WKH���V�ERDUG�JDPH�
)HPDOH
VWXGHQW\HV
)ULGD\�����'
HFHPEHU�
)UHTXHQW�YLVLWRUIRU�WXWRULDOV
\HV/RRNHG�DW�DOO�RI�LW
)HPDOH
3HDFH�DQG�TXLHW�EHIRUH�FODVV
\HVWKDW�P
RUQLQJIUHTXHQW�YLVLWRU
\HV/RRNHG�DW�LW�D�IHZ
�ZHHNV�DJR��DW�P
DOO�ERDUGJDP
H�
0DOH
:RUNV�SDUW�WLP
H�ZLWK�VWXGHQWV
<HV
��)UHTXHQW�YLVLWRU
<HV
+DV�VHHQ�ERWK��HQMR\V�GLVSOD\�YHU\�P
XFK
3DUW����&XUUHQW�([KLELWLRQ
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
25 | E v a l u a t i o n R e p o r t K Y V H 7
4����*
HQGHU4���3
XUSRVH�RI�YLVLW4���9
LVLWHGEHIRUH
4���/DVW�9
LVLW4����9
LVLWIUHTXHQF\
4���+
DYH�\RXVHHQ"
4�����:
KHQ�DQG�ZKDW�GLG�\RX�ORRN�DW
0DOH
6WXG\LQJ
<HV
��HYHU\�GD\
<HV
,PSOLHG�WKDW�KH�P
RVWO\�KDG�ORRNHG�DWDUFKDHRORJ\�GLVSOD\��VDLG�WKDWV�WKH�VWXII�KH
ZRUNV�Z
LWK
)HPDOH
7RLOHW<HV
��
UHJXODUO\��ZDLWLQJ
EHIRUH�OHFWXUHV�ELRORJ\��LQ�*
�\HV
ORRNHG�DW�EULHIO\�EHIRUH�OHFWXUHV
0DOH
ZRUNV�KHUH��3
K'�VWXGHQW�
\HV��
UHJXODUO\�ZRUNLQJ�KHUH
\HVVHHV�WKHP
�GDLO\
)HPDOH
6WXGHQW�YLVLWLQJ�WKH�/LEUDU\
<HV
��:HHNO\
<HV
)UHTXHQWO\��FHUWDLQ�SDUWV�WKDW�FDWFK�\RXU�H\H�L�H���'
�WRSRJUDSK\�PRGHO
)HPDOH
6WXGHQW�OHFWXUH
<HV
��:HHNO\
2FFDVVLRQDOO\
/HWV�7DON��7HFKQRORJ\��&RP
PXQLFDWLRQ
)HPDOH
6WXGHQW�OHFWXUH
<HV
��:HHNO\
1R
��
)HPDOH
6WXGHQW�OHFWXUH
<HV
��:HHNO\
<HV���EULHIO\VFDQ
:KHUHYHU�,�DP
�VWDQGLQJ�DW�WKH�PRP
HQW
0DOH
'RLQJ�P
DLQWHQDQFH�ZRUN�LQ
WKH�IR\HU��8&/�0
DLQWHQDQFHVWDII����Z
DV�MXVW�WDNLQJ�D�TXLFNEUHDN
<HV
<HDUV�DJR
1R
1R
��
0DOH
7DNLQJ�KLJK�UHV�SDQRUDPLF
SKRWRJUDSKV�RI�WKH�/HYDQWH[KLELWLRQ
<HV
$�IHZ
�GD\V�DJR1R
<HV
/RRNHG�DW�WKH�/HYDQW�H[KLELWLRQ��/RRNHG�DW�LWRYHUDOO��LQWHUHVWHG�LQ�FXUDWRULDO�SURFHVV��WRGD\�Z
LOO�EH�D�EHWWHU�FKDQFH�WR�ORRN�DWLQGLYLGXDO�REMHFWV��'
LG�QRW�ORRN�DW�:H�1
HHG7R�7DON���QRW�UHDOO\�KLV�LQWHUHVW�
)HPDOH
:DLWLQJ�IRU�D�SV\FKRORJ\
OHFWXUH<HV
7KXUVGD\���WK'HF
<HV�UHJXODUO\�
OHFWXUHV�LQ�*�
HYHU\�ZHHN
<HV
*HQHUDO�RYHUYLHZ
�RI�WKH�REMHFWV�LQ�WKH�:H
1HHG�WR�7DON�H[KLELWLRQ��HVSHFLDOO\�WKH�ROG
1RNLD�P
RELOH�SKRQH
)HPDOH
:DLWLQJ�IRU�D�SV\FKRORJ\
OHFWXUH<HV
7KXUVGD\���WK'HF
<HV�UHJXODUO\�
OHFWXUHV�LQ�*�
HYHU\�ZHHN
<HV
*HQHUDO�RYHUYLHZ
�RI�WKH�REMHFWV�LQ�WKH�:H
1HHG�WR�7DON�H[KLELWLRQ��HVSHFLDOO\�WKH�ROG
1RNLD�P
RELOH�SKRQH
)HPDOH
6WXGHQW�VWXG\LQJ�LQ�WKH�,R$
�*RLQJ�WR�WKH�GRZ
QVWDLUVWRLOHWV�
<HV
7KLV�PRULQLQJ
���WK�'HF�
<HV�UHJXDOUO\�
ZDLWLQJ�EHIRUHOHFWXUHV
<HV
%UL[WRQ�SRXQG��SULQWLQJ�SUHVV��EDQQHG�ERRNVDQG�OLWWOH�FRP
SXWHU�WKLQJ\�LQ�:H�1
HHG�WR7DON
0DOH
:DLWLQJ�IRU�D�SV\FKRORJ\
OHFWXUH<HV
7KXUVGD\���WK'HF
<HV�UHJXODUO\�
OHFWXUHV�LQ�*�
HYHU\�ZHHN
<HV
%URZ
VLQJ�WKH�GLVSOD\V�EHIRUH�FODVV�,QWHUHVWHG�LQ�WKH�ERDUG�JDP
H�DQG�ORRNHG�DWWKH�REMHFW�ODEHO�IRU�WKDW��+
DG�D�JR�RQ�RQH�RIWKH�L3
DGV�
3DUW����&XUUHQW�([KLELWLRQ
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
26 | E v a l u a t i o n R e p o r t K Y V H 7
4����*
HQGHU4���3
XUSRVH�RI�YLVLW4���9
LVLWHGEHIRUH
4���/DVW�9
LVLW4����9
LVLWIUHTXHQF\
4���+
DYH�\RXVHHQ"
4�����:
KHQ�DQG�ZKDW�GLG�\RX�ORRN�DW
)HPDOH
:DLWLQJ�IRU�D�SV\FKRORJ\
OHFWXUH<HV
7XHVGD\���WK'HF
<HV�UHJXODUO\�
OHFWXUHV�LQ�*�
HYHU\�ZHHN
7KLQN�VR$QFLHQW�YDVHV�DQG�MHZ
HOOU\
0DOH
:DLWLQJ�IRU�D�SV\FKRORJ\
OHFWXUH<HV
7KXUVGD\���WK'HF
<HV�UHJXODUO\�
OHFWXUHV�LQ�*�
HYHU\�ZHHN
*ODQFHG�DW�LW
1RWKLQJ�VSHFLILF
)HPDOH
:DLWLQJ�IRU�D�SV\FKRORJ\
OHFWXUH<HV
7KXUVGD\���WK'HF
<HV�UHJXODUO\�
OHFWXUHV�LQ�*�
HYHU\�ZHHN
1R
��
0DOH
:DLWLQJ�IRU�D�SV\FKRORJ\
OHFWXUH<HV
7KXUVGD\���WK'HF
<HV�UHJXODUO\�
OHFWXUHV�LQ�*�
HYHU\�ZHHN
*ODQFHG�DW�LW
1RWKLQJ�VSHFLILF
3DUW����&XUUHQW�([KLELWLRQ
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
27 | E v a l u a t i o n R e p o r t K Y V H 7
4����+
DYH�\RXVHHQ�Z
H�QHHGWR�WDON"
4������:
KDW�GLG�\RX�PDNH�RI�LW
4���LV�LW�FOHDU�WKHUH
DUH�WZR�H[LKLELWLRQV"
T�����LI�\HV�ZKDW�DERXW
4�����LI�QR�Z
KDW�FRXOG�EH�FOHDUHU
\HV��LYH�VHHQ�LWEXW�GLGQW�NQRZ
�LWZDV�FDOOHG�WKDW�
/LNH�WKH�IDFW�WKHUH�DUH�D�ORW�RIGLIIHUHQW�NLQGV�RI�REMHFWV�IURP
DQFLHQW�SRWWHU\�WR�PRGHUQ�SLHFHV�
<HV��QRZ
�WKDW�\RXVD\�LW�EXW�QRWKLQJ
UHDOO\�GLVWLQJXLVKLQJ,�Z
RXOG�VD\�DUFKDHRORJLFDO�PDWHULDO�DQG
FRPPXQLFDWLRQ�WHFKRORJ\
,�NQRZ�WKHUH�DUH�VLJQV�EXW�WKH\�GRQW
UHDOO\�VHHP�WR�VWDQG�RXW��EXW�Z
KHQ�LORRN�DW�LW�LW�VHHP
V�UHDOO\�ZHOO�GRQH�IRU
IRU�D�VWXGHQW�H[KLELWLRQ��,W�MXVW�IHHOVOLNH�D�KDOOZ
D\�
1R
��<HV
7KH�ILUVW�RQH�VHHPV�WR�EH�P
DLQWDLQLQJ�WKHLQWHUHVWV�RI�JUDQG�VFKRODUV�WKDW��Z
HQW"�WR�WKH�LQVLWXWH��0
DWHULDOV�IURP�WKH�P
LGGOHHDVWHUQ�DUHD���LQWHUHVWV�RI�WKH�VFKRODUV�DWWKH�WLP
H�RI�WKH�LQVWLWXWH��EHLQJ�IRXQGHG"��7KH�VHFRQG�FRYHUV�Z
KDWV�QHZ��Z
KDWVQRZ
�DQG�KRZ�WR�GLVSOD\�WKLV�
��
<HV��GLGQW
UHDOLVH�WKH�QDPH
(PMR\HG�WKH�FRQWUDVW�EHWZ
HHQROG�DQG�QHZ
��OLNHG�WKH�GLIIHUHQWHOHP
HQWV�DQG�IRFXV
1RW�DW�ILUVW��UHDOLVHG
LW�ZDV�WZ
R�VHSHUDWHVSDFHV�ODWHU�RQ�IURP
ILUVW�YLVLW�
1RW�FOHDU�DW�ILUVW��LQWHUDFWLQJ�Z
LWK�WKHH[KLELWLRQV�P
DNHV�LW�FOHDUHU
1R
��<HV
2OG�DQG�QHZ
�DUFKDHRORJ\���
QR��
\HV)DLUO\�REYLRXV�IURP
�VLJQDJH��WKRXJK�WKHELW�E\�WKH�GRRU�LVQW��LQWHUDFWLYH�WRWHP
��
QR�$IWHU�H[SODLQJ�FRQFHSW��,QWHUHVWLQ
QR�
0RUH�VLJQDJH�WR�P
DNH�WKH�GLIIHUHQFHFOHDUHU
QR��
<HV
�SRWV�IRU�WKH�RQH��DQG�WDONLQJ�IRU�WKHRWKHU�
�
\HV�LW�Z
DV�DOULJKW��QRW�FOHDU�
��VHSHUDWH�URRP
V�ZRXOG�EH�FOHDUHU�
<HV
�FRQWUDVWLQJ�WR�WKH�UHVW�RI�WKHJDOOHU\��LQWHUHVWLQJ�WR�WKLQN�DERXWWHFK�LQ�D�Z
D\�VLPLODU�WR�DUFK�
\HV�WKH�ILUVW�LV�FODVVLFDO�DUFK��WKH�RWKHU�LV
FRQWHPSRUDU\�WHFKQRORJ\�
�
<HV
�LQWHUHVWLQJ��QLFH�WR�EH�DEOH�WRUHDG�GLIIHUHQW�WKLQJV�Z
KLOH�,�ZDLW�
\HV�
�
<HV
�UHDOO\�JRRG�\HV
�WKH�ILUVW�KDV�OLWWOH�LQIR��WKH�VHFRQG�LV�RQFRP
PXQLFDWLQJ�EXW�LW�GRHVQW�WDNH�DQ\
QDUUDWLYH�IURP�WKH�SHUQDP
HQW�H[KLELW��
1R
��1R
��6LJQDJH�EHWZ
HHQ�WKH�WZR�GLVSOD\V
\HV/LNHG�LW��P
RGHUQ�HOHPHQWV�Z
HUHDQ�LQWHUHVWLQJ�LQFOXVLRQ�
QR��
/DUJHU�VLJQDJH�EHWZHHQ�H[KLELWLRQV
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
28 | E v a l u a t i o n R e p o r t K Y V H 7
4����+
DYH�\RXVHHQ�Z
H�QHHGWR�WDON"
4������:
KDW�GLG�\RX�PDNH�RI�LW
4���LV�LW�FOHDU�WKHUH
DUH�WZR�H[LKLELWLRQV"
T�����LI�\HV�ZKDW�DERXW
4�����LI�QR�Z
KDW�FRXOG�EH�FOHDUHU
\HVHQMR\HG�LW
QRW�DW�ILUVW��
%HWWHU�VLJQDJH
\HVQR
��EHWWHU�VLJQDJH
QRW�WRGD\��
6RUW�RI��DEOH�WR�ILJXUHLW�RXW�EHFDXVH�RI
GLIIHUHQW�WLPH�SHULRGV
����
\HV��
<HV
����
QRW�WRGD\��
1R
��
*UDSKLF��H\H�FDWFKLQJ�VLJQDJH��P
DNHLW�FOHDUHU�IRU�SHRSOH�QRW�IDP
LOLDU�ZLWK
DUFKDHRORJ\QRW�WRGD\�
GRHVQW�VHH�DVRIWHQ
JODG�LWV�WKHUH��OLNHV�KDYLQJVRP
HWKLQJ�FKDQJLQJ�UHJXODUO\�EXWGRHVQW�ORRN�LQ�GHWDLO
QR��
VLJQV�HDUOLHU�RQ�LQ�VSDFH�LQGLFDWLQJ�WZR
VHSDUDWH�H[KLELWV
1RW�7RGD\
'LGQ¶W�VHH�KRZ
�LW�ZDV�UHODWHG�WR
$UFKDHRORJ\��Z
KLFK�LV�D�JRRGWKLQJ
<HV
/HWV�7DON���&RP
PXQLFDWLRQ�QRZ
�DQGKDYH�KDG��P
RQH\�DQG�H[FKDQJH�VHFWLRQLVQW�FOHDU��$
UFKDHRORJ\���-XVW�SRWWHU\IURP
�WKH�1HDU�(
DVW� 1RW�FOHDU���FDVHV�LQ
/HWV�7DON�DUH���H[KLELW��7RWHP�VHHP
VXQUHODWHG
��
<HV
$ERXW�7HFKQQRORG\�DQG&RP
PXQLFDWLRQ
1R
��6LJQ�SRVWLQJ���VSOLW�RQH�VLGH�IURP
�WKHRWKHU��P
D\EH�D�ELJ�EDQQHU
1R
��1R
��%DQQHU�WR�GLVWLQJXLVK�WKH�WZ
R1RWLFHG�LW��QHYHU
UHDOO\�ORRNHG�WRRPXFK
1RWKLQJ�LQ�SDUWLFXODU
1R
��0RUH�LQIRUP
DWLRQ�RQ�HDFK�H[KLELW
1R
��1R
��1RW�VXUH
1R
��<HV
����
<HV
/LNHG�ORRNLQJ�DW�WKH�ROGWHFKQRORJ\�RQ�GLVSOD\
1R
��
.HHS�H[KLELWLRQV�RQ�RSSRVLWH�VLGHV�RIWKH�JDOOHU\��0
DNH�WKH�RXWVLGH�RI�WKHFDVHV�P
RUH�GLIIHUHQW
<HV
/LNHG�ORRNLQJ�DW�WKH�ROGWHFKQRORJ\�RQ�GLVSOD\
1R
��
.HHS�H[KLELWLRQV�RQ�RSSRVLWH�VLGHV�RIWKH�JDOOHU\��0
DNH�WKH�RXWVLGH�RI�WKHFDVHV�P
RUH�GLIIHUHQW
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
29 | E v a l u a t i o n R e p o r t K Y V H 7
4����+
DYH�\RXVHHQ�Z
H�QHHGWR�WDON"
4������:
KDW�GLG�\RX�PDNH�RI�LW
4���LV�LW�FOHDU�WKHUH
DUH�WZR�H[LKLELWLRQV"
T�����LI�\HV�ZKDW�DERXW
4�����LI�QR�Z
KDW�FRXOG�EH�FOHDUHU
<HV
6XUSULVLQJO\�XQFKDOOHQJLQJ��$
OO�WKHFRP
PXQLFDWLRQ�P
HWKRGVGLVFXVVHG�UHODWH�WR�WKH�SROLWLFDODQG�HFRQRP
LF�HOLWH��1RW�YHU\
LPDJLQDWLYH��'
RHVQW�VKRZFRP
PXQLFDWLRQ�LQ�D�QHZ
�ZD\��1
RWVROG�RQ�WKH�LGHD�RI�WKH�ERDUG
JDPH�DV�P
HWKRG�RIFRP
PXQLFDWLRQ��(
XURFHQWULF�ELDVWR�H[KLELWLRQ�
<HV
&RP
PXQLFDWLRQ�DQG�ROG�SRWV
��
<HV
1RW�D�FOHDU�WKHP
H1R
��
0DNH�D�GLYLGH�JDS�EHWZ
HHQ�WKH�WZR
H[KLELWLRQV��0D\EH�P
RYH�RQH�LQWRDQRWKHU�URRP
���WKH�VSDFH�EHWZHHQ�WKH
GRRUV�WR�WKH�JDOOHU\�DQG�WKH�OHFWXUHKDOO�
<HV
1RW�KHU�DUHD�RI�LQWHUHVW��P
RUHLQWHUHVWHG�LQ�WKH�DQFLHQW�Z
RUOG<HV
'LIIHUHQW�WLP
H�SHULRGV��DQFLHQW�WLPHV�YV�
PRGHUQ�WLP
HV��
1R
��1R
��%HWWHU�VLJQDJH
1R
��1R
��1$
1R
��<HV
3RWV�DQG�UHFHQW�KLVWRU\
��
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
30 | E v a l u a t i o n R e p o r t K Y V H 7
4����VXEMHFWV�UHODWLQJ�WR�WKH�WKHP
H�\RXG�EH�LQWHUHVWHG�LQ4����LQWHUHVWHG�LQ
VHHLQJ�VWRQH�WRROV4�����,QWHUHVWHG�LQ�P
RUHPRGHUQ�WRROV"
,�WKLQN�LI�\RX�ZHQW�IURP
�DFLHQW�WR�PRGHUQ�LWV�QLFH�WR�KDYH
WKDW�NLQG�RI�EUHDGWK�\HV
\HVGDG�XVHG�WR�EH�D�QHXURORJLVW��VFDQQLQJ�P
RWRU�DFWLYLWLHVLQWHUHVWLQJ�WR�VKRZ
�SDUWLFXODU�WDVNV�DQG�WRROV�DQG�ZKDW
SDUWV�RI�EUDLQ�GR�WKLV��$QDORJXHV�WRROV��P
DSSLQJ�VDPH
DUHDV�LQ�SDVW"\HV��UHODWHG�WRVFDQQLQJ�LGHD
\HV
<HV��DOO�DUFKDHRORJ\�LQ�JHQHUDO
<HV��OHDUQW�D�VLP
LODUWRSLF�ODVW�\HDU�UHOHYDQW�WR
XQGHUJUDGH�GHJUHH
<HV��3
HRSOH�GRQW�XQGHUVWDQGFRQFHSWV�RI�P
RGHUQDUFKDHRORJ\��H�J�
GHYHORSPHQW�RI�D[H�IURP
VWRQH�WR�PHWDO�Z
RXOG�EH�JRRGWR�JHW�YLVLWRUV�WR�WKLQN
0RVW�WKHP
HV��QR�H[DPSOHV�JLYHQ���VWXGLHG�VLP
LODU�WRSLFV\HV
\HV
<HV��Z
ULWLQJ�WRROV��PXUDOV�WR�LSDGV
\HV\HV
)HHOV�WRSLF�KDV�EHHQ�RYHU�GRQH��KDV�VHHQ�VLPLODU
H[KLELWLRQV�EHIRUHQR
\HV��VKRXOG�EH�PRUH���VW
FHQWXU\�FHQWULF
:ULWLQJ�LQVWUXP
HQWV�DQG�WHFKQRORJ\\HV
\HV
$[HV�DQG�EURQ]H�DJH�IOLQWV
\HV\HV
�ZRXOGQW�NQRZ
�ZKDW�WR�H[SHFW�Z
LWK�WKDW�WLWOH��TXLWH�EURDG�\HV
\HV�LQWHUDFWLRQ�EHWZ
HHQ�JOREDO�H[DPSOHV�RI�WRROV�DQG�WKH
KXPDQ�P
LQG�\HV
\HV���6
SHFLDOL]DWLRQ��D�SRLQW�LQ�WLPH�RI�P
DVVLYH�SUROLIHUDWLRQDQG�KRZ
�GRHV�WKDW�FRQQHFW�ZLWK�EURDGHU�GHYHORSP
HQWV�:DQW�WR�VHH�D�ELJJHU�SLFWXUH�UDWKHU�WKDQ�D
W\SRORJLFDO�UHJLRQDO�FKURQRORJLFDO�GLYHUVLW\�\HV
\HV
WHFKQRORJ\<HV
\HV
&RP
SOH[LWLHV�RI�WRRO�PDNLQJ�SURFHVV��HYROXWLRQ�RI�WRROV
\HV\HV
+RZ
�SHRSOH�XVH�WRROV�DQG�LQWHUDFW��HYROXWLRQ\HV
\HVYHU\�ILUVW�WRRO��KRZ
�LW�ZDV�FRQVWUXFWHG��Z
KHUH�LW�FDPH�IURP
�HYROXWLRQ�IURP
�WKDW�DUHD�FRPSDUHG�WR�RWKHU�DUHDV��KXP
DQGHYHORSP
HQW�LQ�GLIIHUHQW�SODFHV�
\HV����DW�ZKDW�SRLQW
GLG�ZH�HYROYH�WR�XVHWRROV"
\HV����HOHFWULF�VFUHZGULYHUV
VNXOOV�IURP�PHGLFDO�SURFHGXUHV��VXUJHU\�WRROV
\HVQR��WKH�ROGHU�WKH�EHWWHU
3DUW����)XWXUH�([KLELWLRQ
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
31 | E v a l u a t i o n R e p o r t K Y V H 7
4����VXEMHFWV�UHODWLQJ�WR�WKH�WKHP
H�\RXG�EH�LQWHUHVWHG�LQ4����LQWHUHVWHG�LQ
VHHLQJ�VWRQH�WRROV4�����,QWHUHVWHG�LQ�P
RUHPRGHUQ�WRROV"
DQ\�NLQG�RI�HDUO\�VWRQH�WRROV��HVSHFLDOO\�SDOHROLWKLF\HV��YHU\�LQWHUHVWHGLQ�RULJLQV�RI�WRRO�XVH
QR��H[FHSW�LQ�WKH�FRQWH[W�RIKRZ
�ZH�GHYHORS�IURP
�HDUO\�WRPRGHUQ��RWKHUZ
LVH�QRW�UHDOO\LQWHUHVWLQJ
HOHFWURFRQYXOVLYH�WKHUDS\��SV\FKRORJ\��ROG�PHGLFDO�WRROV�
KXPDQ�P
LQG�DQG�KRVSLWDO�HTXLSPHQW
D�OLWWOH�EXW�PRUH
LQWHUHVWHG�LQ�PRGHUQ
\HV
SDUW�RI�WKH�EUDLQ�DVVRFLDWHG�ZLWK�WRROV��LP
DJHV�RI�WKH�EUDLQZLWK�UHOHYDQW�DUHDV�KLJKOLJKWHG��FRJQLWLYH�DUHDV
\HV��EXW�QRWDUFKDHRORJLFDO�SRLQW
RI�YLHZ\HV��FRJQLWLRQ�DQG�WRROV�PXVLFDO�LQVWUXP
HQWV
/RQJ�WLPH�SURJUHVVLRQ�RI�WRROV�WHFKQRORJ\
<HV��EXW�QRW�WKHHQWLUH�H[KLELW
<HV��3
URJUHVVLRQ��QRW�VSOLW�����
(YROXWLRQ�RI�WRROV� �LQWHUHVWLQJ��SURJUHVVLRQ�RI�WHFKQRORJ\
WKURXJK�WKH�DJHV<HV
<HV��3
URJUHVVLRQ��GHILQLWHO\
1R�LGHD��,P
�QRW�LQWHUHVWHG1R
'HSHQGV�RQ�WKH�WRROV��1
RQHFRP
H�WR�PLQG
0HGLFFDO�ORERWRP
\�WRROV�EHFDXVH�,�DP�D�SV\FKRORJ\
VWXGHQW<HV
<HV��,I�LW�Z
DV�H[SODLQHG�ZKDW
WKH\�ZHUH�DQG�Z
K\�WKH\UHKHUH
1RW�VXUH
6XUH
6XUH
,QWHUHVWHG�LQ�KRZ�SHRSOH�VROYH�FRQFHSWXDO�SUREOHP
V�E\XVLQJ�UHDGLO\�DYDLODEOH�UHVRXUFHV�DQG�YLVXDOLVDWLRQSUDFWLFHV��L�H��P
LQG�DV�D�WRRO��,QWHUHVWHG�LQ�&RP
SXWHU6FLHQFHV�DV�D�ODUJH�VFDOH�WRRO��>7KLV�DQVZ
HU�UHODWHV�WRLQWHUYLHZ
HHV�SHUVRQDO�EDFNJURXQG��8&/�&
RPSXWHU
6FLHQFHV�VWDII�@�
1RW�UHDOO\
1RW�UHDOO\��0
RUH�LQWHUHVWHG�LQDEVWUDFW�DQG�FRFHSWXDODSSURDFKHV�WR�WRROV
(J\SWLDQ�P
HGLFDO�WRROV���WKH�LPSOHP
HQW�XVHG�IRU�UHPRYLQJ
WKH�EUDLQ<HV��YHU\�FRRO
<HV
3V\FKRORJLFDO�P
HGLFDO�WUHDWPHQWV��GHVFULSWLRQV�RI�WKH�P
LQG<HV
<HV��P
D\EH�PHGLFDO�WRROV
IURP�8&/V�FROOHFWLRQ
1RW�WKDW�P
DQ\��'RHVQW�XQGHUVWDQG�WKH�WKHP
H�VR�ZRXOG�OLNH
LW�H[SODLQHG�DQG�LW�DOVR�QHHGV�WR�EH�PRUH�GHILQHG��&
RXOGPL[�XS�Z
KDW�WKH�H[KLELWLRQ�WDONV�DERXW��EXW�GRHVQW�WKLQN�LWVKRXOG�WDNH�RQ�WRR�P
XFK�VR�SUREDEO\�EHWWHU�WR�IRFXV�RQ��PDLQ�VXEMHFW�
<HV��$
OWKRXJKGRHVQW�DJUHH�Z
LWKWKH�OLQNDJH�EHWZ
HHQVWRQH�WRROV�DQGOHDUQLQJ���VXFK�DVWKH�LGHD�WKDW�D
KDPPHU�FKDQJHV
QHXURVFLHQFH�
<HV��$
OWKRXJK�WKLV�VKRXOG�EHDERXW�WKH�HYROXWLRQ�RI�WRROVQRW�MXVW�ROG�DQG�QHZ
��6KRXOG
IRFXV�RQ�SURJUHVVLRQD�DQGDGYDQFHP
HQW�
3DUW����)XWXUH�([KLELWLRQ
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
32 | E v a l u a t i o n R e p o r t K Y V H 7
4����VXEMHFWV�UHODWLQJ�WR�WKH�WKHP
H�\RXG�EH�LQWHUHVWHG�LQ4����LQWHUHVWHG�LQ
VHHLQJ�VWRQH�WRROV4�����,QWHUHVWHG�LQ�P
RUHPRGHUQ�WRROV"
(YROXWLRQ�LQ�SV\FKRORJ\��'
HYHORSPHQW�RI�WRROV�RYHU�WLP
H�$UFKDHRORJ\
0LJKW�EH
�4XHVWLRQHG�Z
KDW�ZH�P
HDQWE\�WRROV��0
DFKLQHV
7RROV�XVHG�IRU�LQYDVLYH�SURFHGXUHV�RQ�WKH�EUDLQ<HV
1R��3
UHIHU�WR�VHH�UHFHQWKLVWRU\�DQG�DQFLHQW�WRROV�QRW
PRGHUQ
3V\FKRORJ\
<HV
<HV
1$
<HV
<HV
3V\FKRORJ\
<HV
<HV
3DUW����)XWXUH�([KLELWLRQ
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
33 | E v a l u a t i o n R e p o r t K Y V H 7
4�����:
KDW�HOVH�WR�VHH�GR4�����$
Q\WKLQJ�HOVH�WRDGG�
,I�\RX�KDYH�VRPH�RI�WKH�HDUO\�WHFKQRORJ\�,�JXHVV�LW�Z
RXOGEH�QLFH�WR�KDYH�D�Z
RUNVKRS�RU�OHFWXUH�DERXW�KRZ�LWV
PDGH�DQG�KRZ
�LWV�XVHGQR
ZDQW�WR�VHH�KRZ
�ZHUH�GLIIHUHQW�IURP
�SULPDWHV�RU�HYHQ
RWKHU�DQLPDOV��+
RZ�KXP
DQV�DUH�GLVWLQFW�DQG�ZKDW�LW
PHDQV�LQ�WHUP
V�RI�LQWHOOLJHQFH��8VH�RI�WRROV�DQG�KRZ
�WKDWVSUHDGV���FRP
PXQLFDWLRQ��ILGHOLW\�RI�WUDQVP
LVVLRQ�ZKDW�LW
UHTXLUHG�WR�SURGXFH�[��1HHG�VHSHUDWLRQ�RI�VSDFH��EUDLQ
GHYHORSPHQW��KDQG�H\H�FRRUGLQDWLRQ��WUDQLQJ�DQGOHDUQLQJ��P
XVFOH�PHP
RU\��
1RW�D�WHFKQRORJ\�SHUVRQ��)RXQG�LSDGV�LQWHUHVWLQJ�WR�JR
RQ�ZKLOH�Z
DLWLQJ�IRU�OHFWXUHV
7KH�VSDFH�KDV�D�UDQJH�RIREMHFWV�DQG�WKHP
HV��4XLWH
KDSS\�ZLWK�WKH�DUHD�DQGVHW�XS
1RWKLQJ�LQ�SDUWLFXODU��VHHQ�VLP
LODU�WKHPHV�FRYHUHG�LQ
H[KLELWLRQV�EHIRUH��$�JRRG�WRSLF�WR�FRYHU�LQ�JHQHUDO��ORWV
WR�WDON�DERXW��IOXLG�WKHPH�
&XUUHQW�IHDWXUHV�JUHDW�FRQVLGHULQJ�UHVRXUFHV
$OZD\V�IHOW�WKH�DUHD�Z
DV�DELW�GDUN��EHWWHU�OLJKWLQJPLJKW�P
DNH�LW�PRUH
DWWUDFWLYH�DQG�DFFHVVLEOH
1RWKLQJ�LQ�SDUWLFXODU
)RFXV�RQ�PRGHUQ�VLGH�RI
WKHPHV��P
RUH�LQIRUPDWLRQ
DQG�H\H�FDWFKLQJ�GLVSOD\V
,QWHUDFWLYHV�DUH�FRRO��GLGQW�IHHO�WKH\�ZHUH�GLVWLQFW
HQRXJK�WKRXJK��H�J��VLPLODU�VHW�XS�DQG�VW\OH�WR�RQH
DQRWKHU��DQG�QR�LPPHGLDWHO\�REYLRXV�
)HHOV�WKH�H[KLELWLRQ�VSDFHLV�DZ
NZDUG��WRR�ORQJ�DQG
WKLV�PDNHV�LW�D�IDLUO\
XQHQJDJLQJ�DUHD��SOXV�LWVE\�D�OHFWXUH�KDOO�Z
KLFKPHDQV�SHRSOHV�GRQW
DOZD\V�H[SORUH�WKH�VSDFH
�,�GRQW�NQRZ�
QR
�VWRQH�DJH�VWXII�QR
�QR�,�ZRXOGQW�NQRZ
�QR
�QR��FDQW�WKLQN�RI�DQ\WKLQJQR
WKLQJV�WR�KDQGOH��ZKLFK�LV�GLIILFXOW�LQ�WKLV�VHWWLQJ��3
URGXFW�ZKDW�WKHVH�WRROV�Z
HUH�XVHG�IRU´
³VXIILFHQWFRQWH[WXDO�EDFNJURXQGLQIR��7KH�DUFKDHRORJ\�LVYHU\�H[FOXVLYH´
KDQGOLQJ�REMHFWVQR
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
34 | E v a l u a t i o n R e p o r t K Y V H 7
4�����:
KDW�HOVH�WR�VHH�GR4�����$
Q\WKLQJ�HOVH�WRDGG�
WH[WLOHV��LQWHUDFWLYH�HOHPHQWV
QR
$�JUDQG�VFRSH�RI�WRRO�HYROXWLRQ��GLYHUVH�DJHV
QR
KDQGOLQJ�DUHD��VHH�KRZ�WRROV�Z
HUH�XVHG
$�ODWH�QLJKW�RSHQLQJ�Z
RXOGEH�JRRG��GUDZ
V�SHRSOH�LQ�PDNH�LW�VRFLDO�
ZRXOG�OLNH�D�OHFWXUH�SDQHO��GRHV�QRW�OLNH�LQWHUDFWLYHV
�JLPPLFN\��QR�RQH�XVHV�WKHP
�QR
����
YLGHRV�VKRZLQJ�KRZ
�WRROV�DUH�XVHG�LQ�FRQWH[W��
LQWHJUDWH�PXOWLSOH�DUHDV�SHUVSHFWLYHV��OLNH�FRP
ELQLQJDUFKDHRORJ\�DQG�VFLHQFH�RI�WKH�EUDLQ
QR
1�$
1�$
1�$
1�$
1�$
,�GLGQW�QRWLFH�WKHUH�ZHUH
L3DGV�LQ�WKH�URRP
�IRU����ZHHNV��0
DNH�WKDW�FOHDUHU
1�$
1�$
7RROV�UHODWLQJ�WR�DUFKLWHFWXUH�DQG�FRQVWUXFWLRQ��UHODWHV�WRLQWHUYLHZ
HHV�EDFNJURXQG�1R
0RGHOV�DV�D�Z
D\�RI�FRPPXQLFDWLQJ��0
DWKHPDWLFV�DV�D
WRRO��+RZ
�6FULSW�Z
ULWLQJ�DQG�UH�ZULWLQJ�FDQ�KHOS�\RX�VROYH
SUREOHPV��UHIHUHG�WR�DQ�H[KLELWLRQ�RI�WKH�%
HDWOHV�O\ULFV�DWWKH�%
ULWLVK�0XVHXP
����
'HVFULSWLRQV�RI�P
HGLFDO�HTXLSPHQW��OLNHV�GHWDLO���,W
GRHVQW�QHHG�WR�EH�PRUH�LQWHUDFWLYH�
/LNHV�WKH�FROOHFWLRQ�DQG�WKHEOHQG�RI�ROG�DQG�QHZ
'HVFULSWLRQV�RI�P
HGLFDO�HTXLSPHQW��OLNHV�GHWDLO���,W
GRHVQW�QHHG�WR�EH�PRUH�LQWHUDFWLYH�
1R
:RXOG�OLNH�WR�VHH�D�WKHP
DWLF�DSSURDFK��'LIIFXOW�WRSLF�VR
ZRXOG�QHHG�D�FOHDU�GYHORSP
HQW�RI�WKH�FRQFHSW��9LGHRV�RI
WRROV�RI�WKH�PLQG�EHLQJ�XVHG��$
EOH�WR�KROG�DQG�HQJDJHZLWK�WRROV��XVH�WRROV��,QWHUDFWLYHV�VKRXOG�XVH�P
LQG�WRROVDQG�QRW�OHDUQ�P
RUH�DERXW�VRPHWKLQJ�OLNH�VP
RNH�VLJQDOV�1R
,QWHUDFWLYHV���IRRWDJH�RI�WKH�XVH�RI�WRROV��+DQGOLQJ
UHSOLFDV�RI�WRROV�1R
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
35 | E v a l u a t i o n R e p o r t K Y V H 7
4�����:
KDW�HOVH�WR�VHH�GR4�����$
Q\WKLQJ�HOVH�WRDGG�
,QWHUDFWLYHV�DQG�LQYROYH�LQ�VRPH�Z
D\�H[SHULHQFHV�ZLWK
WRROV��HJ��WKURXJK�VHHLQJ�YLGHRV�RI�SHRSOH�XVLQJ�WKHP�
,W�LV�FRRO�WR�KDYHH[KLELWLRQV�LQ�WKLV�VSDFH�
,QWHUDFWLYHV�DQG�KDQGOLQJ�LWHPV
1R
,QWHUDFWLYHV�DQG�KDQGOLQJ�LWHPV
1R
,QWHUDFWLYHV1R
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V
36 | E v a l u a t i o n R e p o r t K Y V H 7
'DWH
7LPH
,QWHUYLHZHU�V�
��RI�5HMHFWLRQV
��RI�,QWHUYLHZV
������������DP
���DP.HUU\�-RFHO\Q
���
������������DP
���SP:LOO�$
OOLVRQ�
�
������������SP
��SP0DU\�.
LWW\�
�
�����������SP
��SP%HWKDQ\�4
LQJ�4LQ
��
�����������SP
��SP/\GLD�(
PPD
��
�����������SP
��SP$QQFKHQ
��
������������DP
���SP6WDFH\�,]]\
��
7RWDO��
��
7RWDO�QXPEHU�RI�SHRSOH�LQWHUYLHZ
HG�ZDV���DV�Z
H�KDG�WZR�JURXSV
0XVHXP
�&RP
PXQLFDWLRQ�9LVLWRU�,QWHUYLHZ
V