Visualizing the Problem Domain for Spreadsheet Users: A Mental Model Perspective

Bennett Kankuzi, Jorma Sajaniemi

School of Computing, Joensuu CampusUniversity of Eastern Finland, Finland

Outline

•Introduction•Description of Proposed Tool•Evaluation of the Tool (Methodology, Results

and Discussion)•Conclusion

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 2

Introduction

•A mental model can be defined as “a mental image of the world around us that we carry in our heads depicting only selected concepts and relationships that represent real systems” (Doyle & Ford, 1998)

– a mental model for a spreadsheet, therefore, does not carry all possible information, but just those aspects that the user finds appropriate for the current task

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 3

Introduction (cont’d)

•Important to understand spreadsheet authors’ mentalmodels when doing different spreadsheet processactivities

– to understand why the spreadsheet process is soerror-prone

– to develop the right tools and techniques forspreadsheet activities

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 4

Introduction (cont’d)

•Spreadsheet authors have at least three mental models:real-world, domain and spreadsheet models (Kankuzi &Sajaniemi, 2013)

– the real-world model that comprises general knowledgeof the world around us e.g. “motor vehicle”

– the domain model that represents knowledge of theproblem domain and the functionality of the spreadsheetin problem domain or application terms e.g. “fixedassets”

– the spreadsheet model that codes the expressions anddata relationships in the spreadsheet e.g. “cell B1”

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 5

Introduction (cont’d)

•Research Question:– Is it possible to develop an easy to use spreadsheet

understanding and debugging tool that relievesusers from spreadsheet details and lets them utilizemore of their mental model of the applicationdomain and hence improving the mapping betweenthe domain/real-world mental models and thespreadsheet mental model?

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 6

The Tool

•Translates traditional spreadsheet formulas into problem domain narratives and highlights referenced cells

– domain terms formed from labels (headers) through spatial layout information of each cell referenced to in the formula

•Implemented as an MS Excel add-on

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 7

Related Tools and Techniques

•In spreadsheets, symbolic names and formula translation have been used with the hope to clarify the mapping between various levels of abstraction

– use of named ranges such as in MS Excel and Google Spreadsheets– some spreadsheet visualization tools also do formula translations e.g.

Spreadsheet Professional – model-driven spreadsheet development approaches such as

ClassSheet models (Engels & Erwig, 2005) also translate formulas to more humanized higher level object oriented style formulas

•All these tools and techniques anecdotally assume that symbolic names and formula translation are useful to spreadsheet authors, but their usability has not been empirically evaluated nor psychologically justified

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 8

Evaluation of Tool - Overview

•Adapted Nielsen’s usability attributes of learnability, efficiency andsatisfaction (Nielsen, 1994) in the evaluation tasks

– evaluation involved 12 volunteering accountants (one woman andeleven men) who are frequent users of spreadsheets

– none of the participants had participated earlier in similar studies– first author visited each participant at their place of work

•Also investigated on effect of tool on the mental models ofspreadsheet authors

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 9

Evaluation of Tool - Learnability

•Methodology– short demo followed by two tasks

•Results– highlighting task (mark a narrated area on spreadsheet): mean 85%

correct (min 60%, max 100%)– translation task (convert narration into spreadsheet terms): mean 83%

correct (min 60%, max 100%)•Discussion

– good enough to proceed to the other evaluation tasks

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 10

Evaluation of Tool - Efficiency

•Methodology– within-subjects design experiment where the task was to locate errors

in a spreadsheet without the tool and with the tool– two roughly equivalent spreadsheets sourced from EUSES

spreadsheet corpus (Fisher & Rothermel, 2005) seeded with similarerrors adapted from Raffensperger(2005) and Duggirala(2012)

– some error types for seeded errors• Formula accidentally overwritten with constants (Error Type C)• Formula missing some range (Error Type D)• A wrong problem domain formula (Error Type G)

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 11

Evaluation of Tool - Efficiency (cont’d)

•Results

•Discussion– tool generally helps authors to catch more errors in spreadsheets (p =

0.021) although different aspects of the tool may be more helpful forsome error types than others

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 12

Evaluation of Tool - Satisfaction

•Methodology– participants were requested to write down their opinion of the two

scenarios in terms of how they find it easier to locate errors as well aswell as any suggested improvements to the tool

•Results– eleven out of the twelve participants found the tool helpful in locating

errors– one participant said that he found the tool confusing as he is used to

the “normal Excel”•Discussion

– generally, participants found the tool useful

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 13

Evaluation of Tool - Effect on Mental Models of Users

•Methodology– nine of the twelve participants wrote down explanations for

each of the located errors in the assigned tasks– explanations were analyzed and classified using an inter-rater

reliability verified adaptation of Good’s program summaryanalysis technique in which each object/noun is classified asspreadsheet specific or domain specific or real-world specific(Kankuzi & Sajaniemi, 2013)

•e.g. “column D” is classified as spreadsheet specific; ``totalliabilities’’ is classified as domain specific; and “money” isclassified as real-world specific

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 14

Evaluation of Tool – Effect on Mental Models of Users (cont’d)

•Results

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 15

p = 0.0114

N.S.

p = 0.0001

Evaluation of Tool - Effect on Mental Models of Users (cont’d)

•Discussion– participants used mostly spreadsheet terms when describing an error

in the without tool case while with the tool, the spreadsheet model isless prominent whereas the share of the domain model increases

– tool, therefore, improves the mapping between the spreadsheet anddomain models which makes understanding and debuggingspreadsheets more efficient (located more errors with the tool)

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 16

Conclusion

•Reported on a domain terms visualization tool developed to aid inspreadsheet comprehension and debugging

– tool was found to be learnable– tool helped the participants to locate more errors in spreadsheets– participants also found the tool useful in an error locating task– tool makes the spreadsheet model to decrease while at the same time

increasing the domain model– hence we put forward that the tool improves the mapping between

the spreadsheet and domain models which improves performance inunderstanding and debugging a spreadsheet

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 17

Thank you for your attention!

www.uef.fi

References

1. B. Kankuzi and J. Sajaniemi, “An Empirical Study of Spreadsheet Authors’ Mental Models in Explaining and Debugging Tasks,” in 2013 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 2013, pp. 15–18.

2. J. Nielsen, Usability Engineering. Boston: AP Professional, 19943. M. Wertheimer, A Source Book of Gestalt Psychology. London: Routledge & Kegan

Paul, 1938.4. M. Fisher and G. Rothermel, “The EUSES spreadsheet corpus: a shared resource

for supporting experimentation with spreadsheet dependability mechanisms,” in Proceedings of the First Workshop on End-User Software Engineering, ser. WEUSE I. New York, NY, USA: ACM, 2005, pp. 1–5.

5. J. Sajaniemi, “Modeling spreadsheet audit: A rigorous approach to automatic visualization,” Journal of Visual Languages & Computing, vol. 11, no. 1, pp. 49–82, 2000.

6. J. S. Davis, “Tools for spreadsheet auditing,” International Journal of Human-Computer Studies, vol. 45, no. 4, pp. 429–442, 1996.

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 19

References (cont’d)

5. R. Desimone and J. Duncan, “Neural Mechanisms of Selective Visual Attention,” Annual Review of Neurosciences, vol. 18, pp. 193–222, 1995.

6. P. Duggirala, Excel Auditing Functions [Spreadsheet Risk Management – Part 3 of 4], 2012, accessed December 2012. [Online]. Available: http://chandoo.org/wp/2012/01/18/excel-auditing-functions/

7. G. Engels and M. Erwig, “ClassSheets: automatic generation of spreadsheet applications from object-oriented specifications,” in Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering. ACM, 2005, pp. 124–133.

8. J. F. Raffensperger, The Art of the Spreadsheet, 2008, accessed December 2012. [Online]. Available: http://john.raffensperger.org/john/ArtOfTheSpreadsheet/

9. J. K. Doyle and D. N. Ford, “Mental Models Concepts for System Dynamics Research,” System Dynamics Review, vol. 14, no. 1, pp. 3–29, 1998.

1.8.2014Kankuzi, Sajaniemi VL/HCC 2014 20