Category Archives: Strategy

Delightfully Lost: A New Kind of Wayfinding at Kew

By Natasha Waterson and Mike Saunders

This paper was originally presented and published as part of the proceedings of Museums and the Web 2012.

1.   Introduction

The Royal Botanic Gardens, Kew was founded in 1759 and is both a botanic garden open to the public and a scientific research organisation. Since 2003, it has also been a UNESCO World Heritage Site, in recognition of its historic landscapes and architectural gems, including the largest surviving Victorian glasshouse in the world.

Just like a museum or gallery, Kew has collections—in this case dried specimens, plant artefacts, and archive documents. But it also has a living collection—its plant collection. This is the largest and most comprehensive living plant collection in the world, containing representatives of more than one in eight of all flowering plant species. Many of these species are endangered in their natural habitats, and Kew is increasingly involved in their conservation through projects such as the Millennium Seed Bank Partnership.

Through this living collection, which is publically displayed in the botanic gardens at Kew and Wakehurst Place, we aim to inspire visitors and transform their understanding of plant diversity, conservation, and our scientists’ research.

At Kew, this means helping people explore over 300 acres of outdoor space, three warm and wet glasshouses, and collections that may be above or below ground at any particular point in time! In the past, this has been a challenging, even prohibitive, environment in which to use digital technology. But mobile technologies, especially increasingly powerful visitor-owned devices, have opened up new opportunities at Kew for both wayfinding and interpretation.

To evaluate these opportunities, Kew has conducted a number of mobile trials over recent years, ranging from rented, GPS-enabled multimedia guides such as the “Kew Ranger” to experiments with visitors’ own devices, including the “Moore on your mobile” audio-guide (, and a dedicated research project called “Stories at Kew” in collaboration with the BBC as part of the European Union’s “Participate” project.


Figure 1: Stories@Kew research project
Credit: BBC

Building on the need that had been demonstrated by these trials, Kew decided, in 2010, to develop a “permanent” mobile app. But even having conducted a range of trials, there were still some key outstanding questions that had not been answered: What were our visitors’ motivations and information needs? How did they move around Kew? And, given that this was to be distributed solely on visitor-owned devices, which groups had significant smart-phone ownership?

2.   Understanding motivations, needs, and behaviors

Having consolidated existing data and feedback, it was clear that visitors’ motivations were complex and dependent on a wide range of factors, many of which were independent of demographic measures such as their age or where they lived. Therefore, their behaviors would not be most usefully grouped using a typical demographic segmentation. Instead, influencing factors might include whom they visited with, why they visited, how long they had for their visit, what special exhibitions were running, what time of the year it was, and—of course—whether it was raining or sunny.

To explore this further, Kew decided to conduct a motivation-led analysis of visitors to the gardens, with the aim of correlating their expectations, motivations, needs, and behaviors. This piece of work was commissioned from a specialist firm in the field of cultural visitor attractions, Morris Hargreaves McIntyre (MHM), which had developed motivational segmentations for other attractions. In particular, their brief was to look at:

  • How visitors navigated Kew, and where they encountered problems;
  • How visitors used entrances and wayfinding materials, both when they first arrived and during their visit;
  • The perceived needs that Kew was not fulfilling in terms of wayfinding and orientation;
  • How Kew could better help visitors navigate and discover what’s on offer.

Visitors were recruited in the gardens, using screening questions to ensure that a representative sample was achieved. Research was conducted at carefully chosen sampling points around the garden. These included all of the entrance gates, major attractions and facilities, and significant route intersections around the gardens.


Figure 2: Sampling points of visitor-tracking observations across the gardens

The research included over 1,500 visitor-tracking observations, 350 mini-interviews, 200 detailed exit interviews, and 85 “fulfillment maps,” which tracked an entire visit.

The output of the research confirmed that although a demographic breakdown could show us who was visiting the gardens, it often did not explain why visitors came or what they needed. For example, the same person might visit as a parent on one occasion and with a friend the next time. Their motivations and needs would be completely different during these two visits.

As part of the research, a motivational segmentation was developed using the responses to key questions. This divided visitors into ten groups, of which three groups accounted for 64 percent of all visitors (see also figure 3). These three groups are shown in the following table.

Motivational segment Description Needs
Social Spacers (25%) Repeat social visitors, over 35 years of age. Use facilities with family and friends. Likely to be members. Good knowledge of gardens. Ease of access, parking, comfort, orientation, good facilities, fun / engaging tours and activities.
Sensualists (20%) Retired or older audience. Use signage throughout site. Visiting to experience the beauty of nature. Unobtrusive signage, succinct interpretation, peace and quiet.
Leisure Families (19%) Socially motivated families of mixed ages. Come more than once a year with family. Little or no knowledge of subjects covered. Child-friendly facilities and activities, different levels of service to meet diverse age needs, fun activities.

Table 1: Key motivational segments at Kew Gardens

Each segment has different reasons for coming to Kew and different expectations of their visit, each of which MHM suggested were characterized by one of four fundamental types of motivation: social, intellectual, emotional, or spiritual.

As is evident from the following chart, the three largest segments, and indeed the majority of the other segments fit, into either the social or emotional motivation groups.


Figure 3: Motivational audience segments at Kew Gardens

The high proportion of people visiting Kew for social or emotional reasons had some significant implications. In particular, an assumption that many people approached their visit to Kew with an intellectual motivation (i.e., expecting to learn) appeared to be very wide of the mark.

However, there was evidence that people learnt more during their visit than they expected to—when asked at the end of their visit whether they had improved their knowledge, people were 10 percent more likely to respond that they had, compared to their expectation at the beginning of the visit.

Also of critical importance was evidence that a large majority of visitors neither see the need to plan nor actually do any planning at the beginning of their visit. Only 33 percent of visitors did any planning at an entrance to Kew, whilst 67 percent walked straight into the gardens without consulting any wayfinding or literature.

3.   Delightfully lost: the implications at Kew

The research provided several useful principles for development of a Kew app. First, it highlighted strong reasons to select particular target audiences based on a combination of their likelihood to own a smart phone, and their propensity to look for further information during their visit.

Secondly, it offered a number of design principles for the app experience:

  • Orientation should not be prescriptive: Navigation prompts are generally required “just in time” rather than at the beginning of visits;
  • Whilst motivations for visiting are predominantly to relax and enjoy Kew’s landscape with friends and family, knowledge acquisition does happen when an interest is found;
  • There are often unexpected outcomes of individuals’ visits, and this “serendipity” is viewed positively by visitors.

Together, these three design principles provided a strong steer for the way in which a mobile app could enhance people’s enjoyment of Kew, and in particular tap into their propensity to explore Kew without planning. The reasons that visitors do not want to plan initially seemed surprising, but are strongly supported by the emotional motivation of visitors. Not only do they not see the need to plan, but because their visit motivation is to escape and unwind from their structured lives, they actively reject the notion of being required to plan at Kew.

Because of the motivations of visitors to reject an imposed structure to their visit, and to focus on the joy of wandering through the landscape with their companions, the metaphor of “losing themselves in Kew” or becoming “delightfully lost” in the gardens was introduced.

On further examination, development of this idea appeared to have value not only as shorthand for a visitors’ motivations, but also as an organizing principle for the development of the app. By reconsidering the app as a tool to help people lose themselves rather than to organize a structured visit, the design brief for the app focused on moments when the visitor needs something, rather than the app itself dictating the structure of the visit.

As it was further developed, this design principle appeared to work on a number of levels:

  • It delivered the flexibility required to address the majority of visitor needs articulated in the research;
  • It created a scalable model of interaction for visitors—the app could be equally successful if it was used once or twenty times during a visit, depending on the visitor’s needs;
  • It clearly echoed the majority of visitor motivations;
  • It did not require people to spend large amounts of time looking at their phones to follow a route. Instead, they could enjoy the landscape, and their phone could become a set of tools to help them whenever and wherever they wanted to know more.

And in practice, it led to the following key app propositions being developed.

See today

An accessible set of images that capture the essence of what’s at Kew today. These include user-generated content and could feature plants, seasonal highlights, or festivals.


Figure 4: See Today recommendations in the Kew Gardens app


Although prescriptive navigation (for example, following a route) was not popular, the ability to make decisions at key intersections is important if a decision to go somewhere specific has been made. But the map needed to be customizable so that the visitor could decide what it shows.


Figure 5: Map interfaces in the Kew Gardens app

Dig deeper

A set of tools to encourage people, wherever and whenever they find something of interest, to look for more that Kew has to offer.

Although all three sections of the app respond to the design ideas outlined above, the Dig Deeper section is the most interesting in illustrating the “delightfully lost” principle. It contains three individual tools that each contribute to the principle in different ways.

Firstly, the quick response (QR) code reader is a powerful trigger tool that allows the visitor to choose when and where they want to activate a greater depth of information. Around the gardens, people can scan QR codes on plant labels to activate detailed information about that species, or on interpretation panels, triggering images and video. As an initial experiment, there are over thirty codes in the Rock Garden, over eighty on trees across the site, and a selection in the Princess of Wales Conservatory, meaning that during a typical visit there will be many opportunities to use one. The aim in the future is to distribute the codes so that when someone discovers a point of interest, there is a QR code available to scan. The key is that it is individuals’ point of interest that drives the interaction.


Figure 6: QR codes on garden plant labels, and the in-app screens they activate

Secondly, the augmented reality (AR) tool aims to surface information close to you as you move around the site. It is initially focused on Kew’s trees, since these cover the whole site and are of particular interest to many visitors. Using the phone’s camera, GPS, and compass, information about trees close by is superimposed onto the screen, surfacing content and information about trees that you can see around you.


Figure 7: Augmented reality in the Dig Deeper section of the Kew Gardens app

Finally, the app includes a collection tool, which curators (experts, historians, celebrities, and even visitors) can use to present groups of items to visitors. The theme for each collection can be led by geography, season, person, or even narrative.

Together, these tools offer myriad ways to engage with everything that Kew has to offer—all of them initiated and controlled by the user, and in response to the user’s personal and contextual needs. They support visitors losing themselves physically within the landscape and virtually within Kew’s massive collections.

App marketing and promotion

Because of the research evidence that most visitors do not make decisions about their visit at the entry gate, we decided to promote the app throughout the gardens as well as at the gates. We therefore designed a range of prompts at locations which have Wi-Fi coverage and are typically “pause” spots, such as cafes. Promotion of the app was also helped significantly at launch by being showcased in the “new and noteworthy” category by the App Store.

4.   Has it worked, and what’s next?

How has the Kew Gardens app been received by visitors, and has it achieved the aims for which we set out? To answer this question, we can look at the results of a summative evaluation study, our app logs, and comments on iTunes.

Summative evaluation

In autumn 2011, Kew commissioned a summative evaluation study to gain a qualitative insight into how visitors used the app at Kew—information that would not readily drop out of logs.

Qualitative research aims

We wanted to discover how visitors used the app, and whether it delivered on its “delightfully lost” premise. Our particular research aims were to find out:

  1. In what mode visitors used the app: Did they use it as a guide, or as a means to enhance sociable, meandering exploration?
  2. Whether the app facilitated unexpected discovery, and if it did whether it capitalised on these moments in any way; e.g. by prompting discussion or exploration of deeper app content. If it didn’t, why not?
  3. Whether the app helped visitors include things in their visit that they wouldn’t otherwise have done (e.g., Kew’s hidden gems), and what happened next if it did. If it didn’t, why not?
  4. Whether there are were any other changes to the app that users thought would make it more satisfying or useful to use.
  5. Whether use of the app has any effect on the users’ likelihood to pay a return visit to the gardens.

Research methods

The research was conducted over a school holiday week, on weekdays, and at weekends. Study subjects were recruited from our main target audience groups for the app, namely:

  • Leisure and learning families
  • Sightseers.

We also included some “Social Spacers”—an audience segment made up of older repeat visitors. Both first time and repeat visitors, and members and non-members, were included in the sample. All were iPhone users who had downloaded the app and had used it in the gardens.

We ran two focus groups, one for each of the two primary audience segments, each consisting of six to eight  participants. We also conducted six accompanied visits with a follow-up interview spread across all the indentified target-audience groups.

Research findings

Technology in natural surroundings

As an introduction to the research, we asked participants what they felt about both Kew and about their iPhone—with interesting results.

In association with Kew, common words were:

  • Serene, beautiful, relaxing, tranquil, yearning
  • Discovery, trees, learning
  • Personal, family history, nostalgic, children.

In response to the phrase, “Oh iPhone…” participants’ reactions really brought home the tension between technology and nature in a landscape such as Kew.

In many respects, participants felt their iPhone was an extension of themselves:

  • “I can’t live without you”
  • “My right arm”
  • “You keep me connected”
  • “Someone close.”

They were addicted, but not without misgivings:

  • “I never wanted you, but…”
  • “Addictive, like a drug”
  • “Would be nice to be without you”
  • “Stay at home when I’m on holiday”
  • “You go to Kew to get away from the technology. It’s a last bastion.”

So while there was ambivalence about using technology in such beautiful natural surroundings, in Kew’s case our participants were so hooked on their iPhones that given an app, they would certainly use it.

User experience for beginners and experts

Not all of our target audience groups were app experts, and our study showed that familiarity with the platform had a big impact on user experience.

Younger sightseers were more iPhone expert and typically gave the app a thorough workout to see just what it could do. They were more likely to try out and appreciate the AR and Flickr functionality.

Family visitors experimented more gradually and only used a small part of the app’s capabilities. However, they were delighted with the information in the app and used it as a way to expand their experience in the gardens and provide a learning opportunity for their children.

There was also a relationship between visitors’ existing knowledge of the gardens and their expertise with the platform. This had quite major impacts on user experience, as described in Figure 8 below.


Figure 8: The interplay between knowledge of Kew and familiarity with the iPhone, and its affect on user experience

The implications of this are that we need to find a balance between providing bells and whistles for confident users, while at the same time providing an easy-to-use core functionality for the less confident. It seems we are currently doing much better at the former, and have some work to do on the latter.

The behavioural implications of technical glitches

App users were generally very positive about the idea and intention of the Kew app. However, a significant number of our study participants experienced technical difficulties associated with the download routine for app updates—an essential process to give users the latest recommendations in a seasonally changing landscape. This seemed a particular problem if visitors quickly moved away from gate arrival points (where Wi-Fi was provided) and onto a 3G signal. We suspect this was compounded by a signal black spot adjacent to the popular Palm House. There was also a bug associated with the “locate me” button on the Google-powered map, which resulted in the blue dot disappearing for a few minutes if the app had been brought from background to foreground.

Visitors were very discouraged by these technical glitches, even when associated with network conditions. They expected and wanted the app to work all the time. However, they assumed that these troubles were temporary and would be put right by Kew—possibly a reflection of their trust in the Kew brand. Such technical difficulties made visitors spend less time on the app and give up earlier than they otherwise might have done.

How did visitors use the app?

We had set out to create an app that worked with visitors’ existing behaviours: an app that wouldn’t take away from the sense of being wonderfully lost in nature, but would allow visitors to discover more, make serendipitous discoveries, and then capitalise on them. Did we achieve these aims?

Overall, visitors identified four clear roles for the app:

  • Planning advisor
  • Tempter
  • Expert companion
  • Navigator.

Tempter and expert companion both fit with our original aims. Planning advisor is a bonus, but navigator runs slightly counter to our previous research. Could it be that given an app, visitors have certain expectations of it that override its context and place?

How did the app perform in each of these visitor-identified roles?

The app did particularly well in offering ideas for pre-planning and encouraging visitors away from their habitual routes. The “See today” photos and “Off the beaten track” map listings were popular.

This was especially the case for mothers with young children. Typical quotes included:

  • “You can navigate and play and choose walks.”
  • “I can show them the badger sett which I rediscovered through the app.”
  • “It’s a bravery thing for me. I stick to my path, my daughter heads for Climbers and Creepers. It’s bravery to go off the beaten track.”
  • “I love the source of information there. Tap on the icon, where you are.”
  • “Look at what’s on today. Do this bit for a couple of hours. Don’t do it all in one go.”
  • “It gives you information on the plants. You’re not crowding in with 200 other people.”
  • “You can start at a different gate. Come in at a different gate and rediscover a grove. It’s beautiful.”
  • “You’ve got something in your hand ready for next time. It encourages you to do stuff, to prepare your visit and go to new places.”

In the role of tempter, augmented reality was a powerful and enjoyable tool, but it was not yet accurate to an acceptable standard. This made correct tree identification difficult. We had known this might be an issue prior to release, but had very much wanted to experiment with the potential of AR, so had written text to provide identification tips. However, this was clearly not enough, and suggests that although AR is certainly something worth pursuing at Kew, we should do so in ways that better tolerate the technology’s current limitations. Whether we can do this using Layar’s SDK or require bespoke development remains to be seen.

In the role of “expert companion, the app successfully deepened and enriched the experience of everyone who used it. Visitors loved to learn and understand more about plants, history, buildings, and, in particular, trees. In this regard, they wanted even more than we had provided—they wanted an explanation for anything that caught their eye! They felt the app wasn’t quite there yet on this front, but thought that this was the intent and enjoyed what was there.

This feedback clearly has implications for us—there’s a lot to catch the eye at Kew; how can we produce enough content to provide that added value and satisfy every visitor? Is there a role for the amateur expert and user-generated content? How much do visitors really want the expert voice of Kew’s horticulturalists and scientists? It’s also worth noting that this finding is counter to our original research: given an iPhone app that tempts them, visitors become information hungry.

The only role that the app did not perform well in was as a navigator. This was due to several issues, some of which were technical and others to do with the size of the device. In particular, the “blue dot” bug caused difficulties. Users also wanted compass functionality (which we have now implemented). Not many spotted the “customise” map option, which had been designed to avoid a clutter of markers. They also wanted an overview of the garden. In the future, we would like to find out more about this request, as the zoomed-out version of the map aims to provide this to some extent. It is clearly not sufficient in the eyes of visitors, however. We also suspect there is a form-factor issue at work here, which may not be resolvable on a hand-held device.

Finally, despite the map, visitors still got lost (in a bad way!) when moving from A to B. We don’t know exactly why this was, although clearly the blue-dot bug had some effect. One hypothesis is that visitors need to see readily identifiable landmarks such as street names or buildings. Another is that visitors expect to be able to plot directions on a mobile map, as they can on native mapping applications. We had excluded this functionality from the app on the grounds of project aims and budget. In any case, we would have difficulty in providing either option at Kew—landmarks are often only visible when in close proximity, and although there are paths, we encourage visitors to walk across the grass and explore.

Does the app work socially?

The app worked socially for singles and couples and for mothers with young children, but the idea of social meandering with the app was not found, in practise. People preferred to use the app to decide where to go, put it down, and then consult it again when they wanted to know something. However, we are perfectly happy with this mode of use.

Does the app trigger repeat visits?

Our study found that the app is unlikely to trigger repeat visits in its own right, but it is highly likely to exert a positive force on the decision to visit; people are able to plan beforehand, see what’s on, and find a new route to try.

App logs

While it’s tricky to assess the qualitative aspects of app usage behaviour from logs, we can see if they corroborate our summative research.

Of the map icons, Kew’s star attractions remained consistently at the top of our logs; namely the Palm House, Treetop Walkway, and Princess of Wales Conservatory. However, there were also some unexpected hits, which suggest that visitors are using the app to go off their normal routes and find new things. For example, Kew’s ice house—a small, tucked-away, underground brick building that isn’t even marked on Kew’s paper map—has never been out of the top ten. Neither has our “badger sett”—sadly not an actual badger sett, but an underground play area for children that is at the far end of the garden, which is quite difficult to find. As both are family attractions, this seems to back up our summative evaluation findings. It also confirms that mothers with small children are a key app-user group for us.


Figure 9: Kew’s ice house—an unexpected and hidden hit with users of the Kew Gardens app

Kew’s formal plantings, such as our Cherry Walk and Azalea Garden, were also unexpectedly popular, out-of-the-way attractions. These are likely to appeal to gardening enthusiasts, which is interesting in the light of our original motivations study that suggested that there was a small group of intellectually motivated older visitors who had high smart-phone ownership—our “social spacers” segment.

In the future, we will also use our logs to determine which QR codes are most scanned, and the relative importance of QR and AR in getting visitors to detailed information.

Customer ratings on iTunes

Customer ratings and comments on iTunes broadly concur with our summative evaluation study and logs. Visitors are definitely not tolerant of bugs, download problems, or other technical glitches, wherever they stem from! However, they loved the AR and the fact the app was regularly updated with seasonal content.

Is “delightfully lost” a successful design principle for Kew?

Our evaluation to date shows that “delightfully lost” is a successful design principle for Kew, as demonstrated by the popularity of the app’s augmented reality functionality, and reports that mother’s were successfully using the map to try new paths through the gardens, even though it was not prescriptive on routes.

In many respects, though, we feel that the app so far doesn’t take the possibilities of this idea far enough. Our original vision was hampered by difficulties in procurement—in our experience, the UK government’s e-tendering portal actively discourages digital agencies from tendering for work. This leaves us with further work to do, but with an evaluation study that can now guide future work.

What’s next?

Our number one priority is to improve the app experience for tentative iPhone users. This means it’s critical that we iron out app bugs and improve download routines for app updates. Feedback on the map suggests we also need to make this easier to use, probably replicating iPhone native mapping functionality as much as we can.

For our iPhone-confident users, we need to provide fun, “wow” functionality. AR is worth further experimentation, possibly with a more playful feel.

5.   Is any of this applicable elsewhere?

Knowledge of our visitors’ motivations has certainly helped Kew design a better mobile app, but what of our learning could be applied elsewhere? To answer this question, it’s helpful to look again at visitors’ wants and needs. Are there any patterns across museums, galleries, and outdoor attractions such as zoos and heritage sites?

The first challenge here is that the visitor studies’ literature describes motivational segments in a variety of ways. However, while the words vary (Moussouri, pers. comm., 2012;, Falk, 2010; Christidou, 2010; MHM, 2011; Packer and Ballantyne, 2002), some fairly consistent and common categories seem to apply. Adapted from Falk (2010):

  • Emotional and spiritual – “food for the soul”
  • Experience – “been there, done that”
  • Intellectual – professional or hobby interest
  • Facilitator – socially motivated, helping the visit of someone else in the group (e.g. child, friend on a visit)
  • Exploration – curiosity-driven, open to new ideas, want something to grab their interest.

Falk (2010) proposes that visitors can be characterized by one or a combination of these motivations. He goes on to say that knowing the prevalence of motivational segments can helpfully suggest the types of experience museums and galleries need to provide in order to best satisfy their visitors’ needs. This has been challenged on methodological grounds (Dawson and Jenson, 2011) but in our experience remains a useful model.

So what do we know about visitors’ motivations at museums, galleries, and other visitor attractions in general? Packer and Ballantyne (2002) provide three case studies in comparison: a museum covering the natural environment and cultural history, an art gallery, and an aquarium. Significant differences were found between the motivations at the three sites. In particular, visitors to the aquarium rated social interaction and “restoration” (i.e., emotional and spiritual) goals more highly, and learning and “discovery” (i.e., exploration) goals less highly than museum and art gallery visitors.

This appears consistent with Kew’s motivations study and also with a similar study carried out by MHM across Tate’s sites (Kohler, pers. comm., 2012). At Tate, intellectual motivations are prevalent amongst a much greater proportion of visitors than they are at Kew. Moussouri (2012, pers. comm.) also suggests that motivations appear to cluster according to attraction type.

How does this compare with what museums are doing on mobile platforms? Tallon’s (2011) and the American Association of Museums’ (2011) surveys of mobile technology in museums show that the predominant form of mobile content in museums is still the audio tour, that the most common objective is to provide supplementary information, and that education departments have the most influence over content. However, as the above studies show, and as Goldman (2011) succinctly notes:

Information-seeking is one of only many potential uses of an individual’s phone (compared to social utility, accessibility, status, etc.), and is not by any means the most common use (Wei and Lo, 2006). Thus visitors’ perception of their phones does not immediately indicate the phone’s usefulness as an interpretative device. Whether visitors are likely to use their phones for interpretation depends on their goals for their museum visit.

Kew’s experience undoubtedly supports this position that visitors’ motivations will vary and cluster according to attraction type, and that it is useful to design mobile experiences with these in mind. Information provision is not necessarily the most beneficial aim, and while play is another route, there are plenty of other motivations that we can tap into.

As for the question of whether “delightfully lost” might apply as an experience principle at other attractions, the evidence so far suggests it might be useful in outdoor parks, aquaria, and zoos, where planning and information-seeking are not the top priorities for visitors. However, this is said with the proviso that relevant comparative studies are relatively few to date. In the future, this may become easier—in the UK, at least—as the UK’s national museums standardize their exit surveys and include questions that will segment audiences based on motivations (Kohler, pers. comm., 2012).

6.   Acknowledgements

We would like to thank our two research agencies, Morris Hargreaves McIntyre and Susie Fisher Group, for conducting two extremely useful studies, our visitors for taking part in them, and our design and development agency Make it Clear and Make it Digital for responding to our findings. Particular thanks also to Theano Moussouri and Sabine Kohler for giving such helpful insights into their research.

7.   References

American Association of Museums. (2011). 2011 Mobile Technology Survey. Last updated November 3, 2011. Consulted February 9, 2012. Available at:

Christidou, D. (2010). “Re-Introducing Visitors: Thoughts and Discussion on John Falk’s Notion of Visitors’ Identity-Related Visit Motivations.” Papers from the Institute of Archaeology, 20, 111–122. Available at:

Moussouri, T. (2012). Emails and telephone conversation regarding visitor motivations research. Personal communication. January 2012.

Kohler, S. (2012). Telephone conversation regarding visitor motivations research at Tate. Personal communication. February 2012.

Dawson, E., and E. Jensen. (2011). “Towards a ‘contextual turn’ in visitor studies: Evaluating visitor segmentation and identity-related motivations.” Visitor Studies, 14(2), 127–140. Available at:

Falk, J. (2010). “Calling All Spiritual Pilgrims: Identity in the Museum Experience.” Museum, January/February 2008. Available at:

Goldman, K.H. (2011). “Understanding Adoption of Mobile Technology in Museums.” Last updated August 5, 2011. Consulted February 9, 2012. Available at:

MHM (Morris Hargreaves McIntyre). (2010). Wayfinding, orientation and arrival modes at Kew Gardens. Unpublished report.

Packer, J., and R. Ballantyne. (2002). “Motivational Factors and the Visitor Experience: A Comparison of Three Sites.” Curator: The Museum Journal, 45(3), 183–198. Available at:

Tallon, L. (2011). Museums and Mobile Survey 2011. Last updated January 2011. Consulted February 9, 2012. Available at:

Mobile Product Development Principles

By Nancy Proctor

  • Mobile products should be accessible and used to enable access to experiences and resources for people of all abilities.
  • Mobile projects should expand and create new opportunities for engagement, not seek to reproduce existing ones on mobile devices.
  • Mobile should be understood as social media and projects should leverage its ability to create conversations, communities, and collaborations both alone and in combination with other platforms.
  • Wherever possible, a mobile website should be at the core of every mobile application project to enable multi-platform accessibility.
  • Digital content should be conceived for cross-platform use and re-use according to mobile content standards with quality metadata.
  • Avoid writing new and/or dedicated code, or using proprietary or dedicated systems.
  • Make code, tools, best practices and other learnings from mobile projects freely available to others to reuse.
  • For quality and consistency of user experience, mobile initiatives should use standard interfaces.
  • Embed metrics and analytic tools in every mobile product, and include audience research and product evaluation in every mobile project to inform iterative development and ensure quality.
  • Every mobile project or product must include a commercial or other plan for its sustainability and maintenance

Mobile Experience Design: What’s Your Roll-Out Strategy?

By Koven Smith

Rapid advancements in smartphone technology of the last few years have changed the nature of mobile experiences in museums utterly. Where tour-based audio guides were once the only type of mobile experience available to museum visitors, we are currently witnessing an explosion in the types of experiences from which visitors might choose. Augmented reality games, crowd-sourced content creation, or even experiences not designed to occur inside the museum at all are just a few of the new ways that museums are beginning to explore to enhance either a physical or virtual visit.

These new opportunities mean that museums must now take a more nuanced approach to how their mobile experiences are introduced to the public. In the past, the expense of providing mobile experiences to visitors (typically via audio) meant that those experiences needed to appeal to the broadest possible audience in order to make them cost-effective. This broad appeal was reflected in the brute-force marketing strategies employed by museums to encourage uptake: handing out mobile guides to visitors, advertising the guides with large signs at the entrance, and often providing mobile guides as a premium benefit of membership.

However, development of mobile applications and mobile web sites as replacement for dedicated devices as the primary means of delivering mobile experiences — and the concomitant reduction in production costs — means that mobile experiences in museums no longer need to be designed for a museum’s entire audience in order to be cost-effective. Many of these new types of mobile experiences are often aimed at a particular niche audience, whether scholars, gamers, children, or social butterflies. Each of these niche audiences requires its own type of solicitation, both via the design of the mobile application itself as well as the marketing campaign used to introduce it. Museums must therefore design the strategy by which a mobile experience is “rolled out” to the public as carefully as it designs the mobile experience itself. The goal of a successful mobile roll-out strategy should not be to reach more users, but rather to reach more of the right users.

Reaching the right users involves reflecting the needs of a given target group in the design of the application, but also in the ways the target group is approached to participate. Mobile applications designed for a small subset of a museum’s public shouldn’t be marketed to every single person who walks in the door, any more than an application designed for use by the “average” visitor should be marketed exclusively to the gaming community. A museum’s roll-out and marketing strategy should act as a signal to visitors indicating what type of experience they should expect; visitors should then be able to better self-select the kinds of experiences that are right for them. What follows is a discussion of three typical roll-out strategies for mobile experiences in museums, with a discussion of how the target audience, application design, and marketing strategy affect one another. These strategies should serve as solid starting points for any museum contemplating how to introduce its visitors to a new mobile experience.

Scenario 1: Broad Appeal

Target Audience: In a “broad appeal” scenario, the museum is marketing its mobile experience to every single visitor who enters the building. A mass-market roll-out scenario is designed to reach the largest number of potential users, typically from a wide array of demographic backgrounds. In this scenario, the visit to the museum drives use of the mobile application; the average user has probably not arrived at the museum already aware that a mobile experience is available to him or her, necessitating a wider-reaching information/marketing campaign. The typical result of this kind of campaign is a relatively passive type of engagement from a large number of users.

Design: If a mobile application is to be marketed to the masses in this way, it must be truly usable by those masses. The application should be highly fault-tolerant and forgiving of mistakes on the part of the visitor, not dependent on the user’s familiarity with other technologies or services in order to participate in the experience (“sign in with your Twitter account” would be a poor way to kick off the experience, for example), and not contingent on the user’s familiarity with specialized language or jargon. Because the user engagement in this type of scenario is likely to be low, the threshold to content consumption should also be low.

Strategy: Because this type of experience is designed for most (if not all) visitors to the museum, the roll-out of the experience should reflect this more “populist” nature. It should be impossible for any visitor to exit the building without knowing that there is a mobile experience available to him or her. The most straightforward way to encourage adoption is simply to rent or loan the visitor a device with the application pre-loaded on it. Prominent signage and other kinds of promotional materials (e.g., bookmarks reminding visitors to rent a tour, or offering a discount) at the entrances and in dwell spaces will also help to saturate the physical space with the awareness that the mobile experience is available.

In instances where giving a device to the visitor may not be possible, the museum must make the application available for download to users’ own devices. Making an application available in this way is not as straightforward as it might seem. First, signage must be available — at every location that the visitor might use the application — directing the visitor how to download the application to his or her personal device. Many users may not download the application at the front door, so it is important to have additional signs prompting download throughout the building, in locations where content is available. The nature of this signage should also reflect the nature of the application design. If the application is primarily aimed at researchers, scholars, or students, for instance, a broad appeal campaign may not be appropriate: the museum cannot create a broadly targeted information campaign for an application that will be difficult for all but a small minority of its visitors to use. A large sign saying “Download our mobile app!” is a not-so-subtle message to the user that the app will be easy to use, and will not require much from the visitor.

Scenario 2: Stealth

Target Audience: A “stealth” roll-out means that the museum has decided to market its mobile experience to a niche group without an overt information campaign. The expectation is that this group will be a subset of the museum’s total visitor profile, but that this smaller group will be far more actively engaged with the mobile experience than the “average” visitor. With a stealth campaign, users of the mobile application should already be aware of the application before a visit is made, if the application itself did not in fact prompt a visit.

Design: A “stealth” campaign is an appropriate means of marketing when discovery, exploration, and mystery are primary components of the application design. While the application shouldn’t necessarily be difficult to use, the act of figuring out how the application works should be a key part of its appeal. Because a stealth campaign is targeted at a smaller audience, the application should have an appeal tailored to the audience being targeted. An application designed to be a broad introduction to the museum’s collection, for instance, wouldn’t necessarily benefit from being rolled out in this manner.

Strategy: In a stealth campaign, the marketing of the mobile application is itself part of the total experience. The primary goal is a highly engaged user community, so the roll-out campaign should be designed to promote a high level of curiosity at the outset. There are a number of strategies a museum might employ in pursuit of this goal. A straightforward strategy might be to identify potential “influencers” in the museum’s community, and give those influencers a personal introduction to the application, with the expectation that these users will provoke others to use the application as well.

Another possibility would be to take a cue from alternate reality campaigns, and attempt to promote a sense of mystery around the application. In this scenario, the museum might use signage, but in a more oblique way than in a broad appeal campaign. The museum might embed “clues” that would prompt a visitor to download the application to his or her phone or to take additional actions. Clues could even be embedded in materials designed to be consumed outside of the museum, such as print materials or the museum’s web site. An effective stealth campaign should guarantee user interest and engagement long before the application itself is actually downloaded.

Scenario 3: Third-Party

The recent explosion in the number of museums making collections content available via APIs (“application programming interface”) has created a third viable scenario for museums: an application designed by a third party, completely outside the museum’s purview and control. Strategizing for both design and marketing of a mobile experience developed in this way is challenging, but not impossible. A museum in fact has the ability to influence both the design and introduction of a mobile experience, even when developed largely without that museum’s input. The target audience of this type of experience is variable, depending on the museum’s goals.

Design: Again, in this scenario the museum is looking for means by which it can influence the development of a mobile experience more than overtly control that development. Here, a museum might look at multiple types of content to make publicly available. A museum making its information available via an API might create a separate “mobile-ready” API that prioritizes the types of information the museum would like to see in a mobile device. Delivering data in this way helps to ensure that the mobile application developed by an outside developer will still focus on the kind of information that is important to the museum. A museum might also publish a list of locations within the building, with particular content attached to each, or a database of artists within the collection, or a geotagged list of artist birth/work locations. Museums might make any number of content types available that would encourage developers to create applications that travel outside of the normal “tour” format.

Strategy: A third-party mobile experience represents a unique challenge for museums from a roll-out and marketing standpoint. Because the museum may not know that an application is being developed until it is already publicly available, it is difficult to schedule its roll-out into a broader marketing strategy or schedule. In this scenario, what the museum should be looking to do is to provide incentives to potential mobile developers to work within the museum’s ideal framework. There are a number of ways a museum might do this. It might simply indicate a willingness to promote an application in its galleries or on its website if the application is developed according to certain standards. It might also be willing to provide physical infrastructure for certain types of applications (AR or gaming applications, for example). While doing this, it is critical that the museum keep in mind how to create brand differentiation between its own “official” applications and those developed by the community, inserting appropriate language into “terms of service” agreements and the like.


A diversity of mobile experience types demands a parallel diversity of marketing approaches. It is clear that museums need to begin making far more deliberate choices about how their mobile experiences are rolled out to the public. Making the right choice will help to ensure that the right visitors are paired with the right types of experiences. Whether the museum wishes to reach its visitors outside the building, via a game-style approach, or inside with a mobile tour, the roll-out strategy should be carefully considered at each and every stage of development.

So Many Devices, So Many Options: An Introduction to Cross-Platform Thinking

By Allegra Burnette

Traditionally we are trained to think that form follows function: first you decide the content and purpose of what you are designing, building or creating, and then you shape the form around that. But in many ways, mobile projects are the reverse — the function of your app is often led by the form factor of the device itself. A tablet, for example, offers a different kind of experience than a smartphone, and that difference can shape the app you develop.

Smartphones are inherently portable, enabling us to slip a vast array of content, activities, and resources into our pockets or bags and take them wherever we go, pulling them out when we need to look up something or find out where we are. They are therefore ideally suited for things like audio tours in the museum, finding out the location of a museum and the next event or film showing, and looking up an art term while standing in front of a work of art. Going beyond the one-way communication of a traditional audio tour, mobile phones enable two-way communication between the museum and its visitors, as well as visitors with other visitors.

Tablets, while also portable, are (at least currently) typically much less about a literal on-the-move experience — even Apple’s ads when the iPad came out last year showed someone with the device propped against his knees, leisurely perusing an app or browsing the Web. People use tablets while traveling on planes, trains, and in automobiles. They use them to read magazines, play games, and watch a movie. A tablet can be viewed by several people at once more easily than a phone can. But at the same time, the tablet format creates a more intimate personal space between user and device than a laptop or desktop computer does: we are interacting directly with the screen rather than through a separate keyboard or mouse, and we are often holding it rather than facing it.

At The Museum of Modern Art we prioritized the iPhone as our first platform for mobile development because iPhone users were the largest mobile audience of our website. (While Android was second largest at the time we set our priorities, the iPad overtook the Android audience shortly after it first came out in the spring of 2010, and remains the second largest at the time of writing.) In order to make the best use of our resources and to streamline development and ongoing maintenance, we developed an app that was a hybrid of a native app and a mobile site. Creating this hybrid app rather than just a mobile site gave us access to broader distribution through the Apple App Store. This also gave us the opportunity to create features like MoMA Snaps, a branded postcard activity, which would not have been possible through a browser version alone. But at the same time, it allowed us to develop a structural base that could be adapted for an Android app and the mobile version of our website,

The MoMA iPhone app that we developed was meant as both an in-museum and an offsite experience. Like many museums currently developing mobile apps, we wanted to include our audio tour content. But we were not intending to replace or supplement the current in-museum audio devices with iPods loaded with the app (due to the quantity needed, as well as distribution, security, and maintenance issues). Instead, our intention was to offer the app for people who wanted to access the content through their own devices when in the museum. We included the entire calendar of events and exhibitions and access to all of the online collection with the intention that people would also use the app to plan a visit or learn about works of art beyond the walls of the museum.

While the iPhone app (and the later Android and mobile versions) was a more general view of MoMA and its collection, the iPad app we developed for the Abstract Expressionist New York (AB EX NY) exhibition was an exploration in creating an experience specifically for a tablet device around a single exhibition. We chose not to do a tablet version of the mobile phone app immediately because we felt the form factor necessitated a different approach to the content. This initial tablet project gave us a chance to explore focused ideas on how we could present works in our collection, which in turn might later inform broader, tablet-based projects that we may develop in the future.

Several ideas we explored in the tablet format would have been less effective or not possible in a phone-based app including. For example, a split screen layout, which allows textual information to appear adjacent to a work of art. It is very difficult to combine text and art onscreen in a meaningful way on a phone — your focus is either on one or the other (which is in part why video and audio are particularly effective on a smartphone).

The home screen of the iPad app was a scrollable view of the works that were featured in the program, with the images shown loosely in scale with each other. This selection, combined with the “gallery” browse views, creates a different experience than the typical, more list-based phone app.

While these are certainly not all of the different layout considerations between a tablet and a phone app, they do hint at the larger issue at hand: How do we create compelling experiences for the different device form factors with the limited resources available to museums and in the rapidly changing face of technology? The sand is shifting so much right now that there is not currently a clear answer, but being strategic and thoughtful about how you approach the various platforms (tablet versus phone) and formats (app versus Web), while staying true to the content and your own capabilities (or those of a trusted consultant), is at least a start.

While the AB EX NY iPad app was intended to promote the exhibition, its related publication, and MoMA’s collection, we very specifically intended it to be an experience that took place outside of the exhibition, whether that meant people used it before or after a visit, or even if they never came to the museum at all. We even used images inspired by the Apple campaign of someone using the app in a non-museum space to reinforce that idea. Even though the app includes the content from the audio tour, it really didn’t occur to us that people might try to use it within the museum as a mobile app, until we read this in a review:

You may find the experience of lugging an iPad around the exhibit distracting; I certainly did at times, for no other reason than all the attention it attracted. But if you think about this as a piece of software, free to be downloaded onto any iPad anywhere with an Internet connection, then it dawns on you: a kid in Idaho, two time zones and two thousand from the MoMA, can experience this content as easily as a youngster from the Bronx.

This was a valuable lesson: no matter how we design these apps and no matter how carefully we tailor them to a particular platform, the known unknown is how and where people are going to use their mobile devices. Anecdotally, we have noticed that in the museum, people are using both phones and tablets, with phones making more of an appearance in the galleries and tablets used more in the interstitial spaces. But if we offer the same program on both phones and tablets, would visitors switch between devices based on where they are, or are they more device-consistent within the space of the museum? At this point, only more observation and testing will reveal the answer to that.

If we look at the number of mobile phone apps versus tablet apps in the Apple App Store and Android Market, we see far more of the former than the latter. While this is in part due to the fact that tablets are newer to the market and comprise a smaller share of the mobile device landscape in general, it may also be due to the different experience of using a tablet and the different requirements, including interface design, needed to develop those experiences. And while “function follows form” may be in large part the way we’ve started developing mobile museum apps, as tablets start to come out in varying sizes and phones screens get larger, the differences between a phone versus a tablet experience is likely to become blurred. Add to that the various ways that people use mobile devices, and there are more overlaps between smartphones and tablets. Careful planning and evolving development solutions should help clear a path through the morass as we move past our mobile beginnings to a multiplatform future.

Playful Apps

By Jane Burton

There are hundreds of thousands of apps for smartphone consumers to choose from, and most of them are games. Games make up 70 to 80 percent of all apps downloaded. The latest reports say that 26 million people spend at least 25 minutes every day playing games on their phones [Flurry Analytics, Feb 2011]. The incredible amount of innovation in smartphone mobile gaming is showing us how to create content that people really want to spend time with. The question museums and galleries need to answer is whether there is room in this marketplace for “serious games,” games that offer more than just pure entertainment.

The potential to bring significant ideas to life within the framework of game-play is something that has been brilliantly expressed in the work of Jane McGonigal, author of Reality is Broken, and by UK television’s Channel 4 Education team, who have put gaming at the heart of their content offering to young audiences. But little of the research and innovation around “serious games” has focused on the rapidly growing area of apps.

One of the most successful games produced by a museum is Launchball, from The Science Museum, London, which was first developed in 2007 to be played on kiosks in gallery and online. Designed for children ages of 8 to 14, the game requires you to guide a ball through a series of fiendish challenges, using fans, magnets and Tesla Coils to help you as you learn basic scientific principles along the way. The online version proved so popular, gaining 5.3 million players, that in 2009 the Science Museum re-launched it as a paid-for app. So far the app has been downloaded 7,842 times, enough to pay for its development and return a modest profit, says the museum. This is an achievement, given that the same game can still be played for free online. Nonetheless, the disparity in the figures is a striking reminder of the reach of the web compared to any given app store, and of the power of free content.

Another great “playful” offering, though not strictly a game, is the MEanderthal app from the Smithsonian’s National Museum of Natural History, which transforms your photo into the face of an early human. You upload a photo of your face, then choose which human species you’d like to become as you morph back in time. There is a serious point behind the fun: “We think it’s really important for people to make emotional connections to our ancestors,” commented Briana Pobiner, a paleoanthropologist at the Smithsonian. “It’s an important way to break down that barrier between things we think are so different or so ‘other.'”

At Tate, we’re interested in finding out if app gaming mechanisms can be applied to an art context. We have produced one game so far, and have two more in development. Launched in 2010, Tate Trumps is a digital card game you play with the art on display at Tate Modern. Visitors can download the game for free to an iPod Touch or iPhone, roam the galleries, choose seven high-scoring artworks, and then play a fast-paced and strategic game of Trumps. There are three different modes (Battle, Mood, or Collector) and you can play on your own or with your friends or family. In Battle mode, you need to ask yourself the question, “If this artwork came to life, how good would it be in a fight?” In Mood mode, you’re looking for artworks you think are menacing, exhilarating or absurd. Or, if you wish you had a gallery of your own, try Collector mode, and find pictures that are famous, recently produced or practical to house.

Tate Trumps is unashamedly light-hearted, but at its core promotes the acts of discovery and looking — key to any art experience — whilst encouraging people to form their own opinions. Being a multi-player game, it also acknowledges the fact that gallery-going, for many, is a social activity, shared amongst friends.
Tate Trumps was deliberately designed to be played only at Tate Modern in order to encourage a direct encounter with artworks. But of course app stores are global marketplaces, and we hadn’t reckoned on the frustration that not being able to play would engender in the majority of people who wouldn’t be coming to the gallery in the near future.

For our next game, currently in development, the brief was to come up with something that could be played anywhere, without visiting the museum, but with bonus content for those who can make it to Tate Modern. Shake your phone and the “Magic Tate Ball” will curate a piece of artwork that relates specifically to that unique moment in time. Pass it round the pub or check it on the train to find out which artworks fit the DNA of your daily life. In auto-mode, the application will use the iPhone’s sensors (microphone and GPS), along with other feeds like weather and time, to deduce the most appropriate artwork for the given criteria. In manual mode, the user can ask the Magic Tate Ball to generate ideas on themes: Inspire me; Shock me; Give me a Talking Point.

The third game we are developing pushes further into pure gaming territory. The challenge we’ve set ourselves is to take a simple, addictive form of gameplay along the lines of Doodle Jump and bring art into the mix, imparting meaningful information without getting in the way of the action.

The jury is still out on how successful games like these will be in terms of introducing new audiences to Tate’s Collection, and we will be evaluating them later this year. But in the meantime, here are a few things we’ve learnt along the way:

Know Your Audience.
It’s easy to assume that mobile gamers are teenage kids. Wrong. Forget the acne generation, unless you’re talking about console platforms like Xbox or the PS3. The typical gamer downloading games through app stores (and really, we’re still talking about iTunes, though Android is beginning to build a market share) is female, between 18 and 49, and well educated. A recent report published by Flurry, a San Francisco-based smartphone analytics firm, said: “Studying the U.S. mobile social gamer, we note that she earns over 50 percent more than the average American, is more than twice as likely to have earned a college bachelor’s degree, and is more likely to be white or Asian.” The number of men playing isn’t far behind, though: 47 percent of app-based gamers are male, compared to 53 percent female. In fact, the profile for these gamers is strikingly similar to the profile of many museum visitors, which suggests that app gamers may very well be open to cultural content delivered in this form.

However, this demographic will widen as Smartphones become more affordable and therefore more common over the next two years. If you want to use mobile games to reach a teen audience, start planning, but maybe not developing just yet, and look beyond the iPhone platform.

Make it Free.
With hundreds of thousands of apps available for mobile consumers to choose from, it’s a tough market, and most publishers are moving towards free apps. Some are supported by sponsorship, or possibly, if you’ve got a really hot property, by “freemium,” whereby you get the basic app free, but people pay for the fancier version. Anyone who has played “Angry Birds” will recognize this model. But only the most optimistic developer from the cultural sector would imagine they are going to make much money from a game.

Think about Discoverability.
Submitting your game into an app store is a bit like dropping thousands of dollars down a well. There’s an initial splash, which dwindles to a ripple, then silence. The iTunes app store is a busy place, and it’s hard to get noticed amongst the crowd. Getting noticed is known in the digital world as “discovery,” and there are myriad social networks and recommendation sites springing up that aim to make app discovery easier for consumers. Some of the sites are listed here. But the sands are ever shifting, and there are no sure-fire solutions. Being early to market in one of the less crowded app stores looking to rival iTunes is beginning to look like a smart move.

Jane Burton, head of content and creative director, Tate, London.

Understanding Adoption of Mobile Technology within Museums

By Kate Haley Goldman

Each week brings across my desk a fresh set of mobile market studies indicating how the proliferation of smartphones continues at a dramatic pace, web access is more and more mobile, phones have changed teen culture, phones have changed the culture for the rest of us, and smartphone domination is in our near future. Having established near ubiquity of phones in general, most studies foresee a continued acceleration of the adoption of the smart phone. The adoption acceleration has extended to museums, and in recent years is starting to become less talk and more actual products on the floor. Multiple recent studies have tackled the institutional perspective of mobile adoption and barriers to implementation. Among the most relevant for the museum field are the 2011 studies sponsored by AAM and Pocket Proof/Learning Times. This chapter will interpret the findings of these studies and frame questions for future studies.

The study by Fusion Analytics for AAM (supported by Guide by Cell) focused on the state of current and future institutional incorporation of mobile capabilities. The sample was somewhat larger than the study by Pocket Proof, focused primarily on the United States, and had a geographically wide and institutionally broad distribution. One key finding was that under half of American institutions within the study currently offer mobile interpretation. For those institutions that did not yet have mobile, the primary reasons were lack of budget, staff time and other resources. Most non-mobile institutions did not express a concern that they did not want their visitors using mobile devices, but they were concerned about visitor interest. Over one-third of museums without mobile products listed lack of visitor demand as why they did not offer such products. By comparison, the study from Pocket-Proof and Learning Times asked museum professionals about their attitudes towards the creation, implementation, and maintenance of mobile applications, segmenting the results by the differing challenges for those institutions who already have a mobile interpretation product compared to those who are planning to, but do not yet have such as product.

Both studies are admirable in their effort to provide baseline data and context for the proliferation of museum mobile projects. Looking at their data, it would seem that the vast majority of museums are currently working on such projects. And while that generalization may in fact be true, unfortunately we can’t be sure from the data presented within these studies.

Generalizability is a tricky proposition with research and evaluation studies. Fundamentally, research is judged on its reliability and validity. Reliability is defined as the consistency or stability of a measure from one test to the next. An accurate oven thermometer is reliable, measuring 350 degrees in the same fashion each time. Validity is the overall term used to describe whether a measure accurately measures what it is supposed to measure. An accurate oven thermometer might be consistent in temperature measurement but does not measure whether the food is hot enough. Oven temperature is not a valid measure of food temperature. It is possible to have reliable but not valid results: results that are repeatable in multiple testings but still do not measure the appropriate underlying construct. The oven might be consistent in the temperature measurement, but it still does not represent a valid measure for the temperature of the food.

Reliability is influenced by the design of the questionnaire, but most profoundly by the sampling within a project. The Pocket Proof study uses a nonprobability sampling: respondents were recruited via list-serves, social media and other outreach. While the sample sizes are large; there is no certainty that the population sampled is representative of the overall population of museums in the United States; indeed they are likely to be museums with strong connections to social media, and by extension, use of technology within the museum. This may bias the results in terms of the numbers who have mobile projects or are considering them, or bias in other ways, such as size of institution, type of mobile project attempted, etc.

The AAM study had a well-defined sampling frame: AAM member institutions and individual members. While there is likelihood of some amount of nonrandom measurement error, the issue of most concern is the response rate. Response rates are notoriously difficult on web-based surveys, even for surveys such as this one with incentives offered and a dedicated (but busy) membership. At what point can a response rate be deemed reliable? In classic survey response methodology, a response rate of 60%, though preferably 80%, is seen as acceptable for analysis and generalization purposes (Dillman, 2000). This is an extremely high bar to reach for a web survey of a general membership. Yet, with the 14% response rate of the AAM study, one could repeat the exact same study next year and have an entirely different 14% of the population respond with entirely different answers. Whether the implementation of mobile projects had gone up, down or remained constant, it would be impossible to say reliably. For the Pocket Proof study, we cannot calculate a response rate, as we don’t know how many individuals saw the survey request. We can not say with certainty what percentage of museums are currently conducting a mobile project, planning a mobile project, whether mobile projects are more common in certain types or sizes of institutions. While the concept that mobile interpretation is more prevalent in large art museums has significant face validity, due to the combination sampling strategy and response rate we have no consistent, reliable numbers.

While the results within these studies may not be generalizable, they still provide some valuable insights. One of the most useful aspects of both studies is the documentation of barriers faced by museums in implementing mobile projects. The percentages aren’t necessarily relevant here, but the rank order is. These lists provide institutions a list of most prevalent obstacles.

Setting aside the idea that some set of museums underrepresented within the surveys might face significantly different obstacles towards implementing their mobile project, the barriers faced by those that are already engaged in such projects, and by those contemplating such projects, is illuminating. As the Pocket Proof/Learning Times study notes, for institutions already using mobile interpretation, encouraging visitors to use the mobile interpretation was the largest challenge. Yet for others — vendors and researchers, as well as those considering projects — attracting new visitors via mobile was a primary goal. This disconnect represents a great opportunity for future research. Despite the numbers of institutions exploring mobile and the availability of phones, usage rates remain below 10% for permanent galleries (Proctor, 2010) For those contemplating employing a mobile interpretation strategy in the Pocket Proof study, the lack of visitor interest was at the bottom of the list of obstacles, whereas for those currently offering mobile interpretation, this issue was more of a concern. That many visitors simply find the concept of using their phone unappealing in this context (Haley Goldman, 2007) is a finding that must be further explored.

Mobile Research Opportunities
The groundwork laid by these studies promotes reflection on what sort of studies are still needed. Both of the studies here had the foresight to ask their participants what type of research is still needed within the industry. Building on those concepts from a researcher’s perspective, I propose three avenues of future potential research.

1. Visitor-based research: I know from personal experience that evaluation and research on visitor use of mobiles in museums is extraordinarily difficult. The small sample size makes it difficult to get a full picture of usage, the interpretation strategy creates difficulty in finding those who are using their mobiles; the list of challenges goes on and on. The research on mobiles in museums is overwhelming institutionally-oriented, as is the motivation for many of the museum implementation projects. Some institutions wish to avoid the overheads of providing audio tour devices to visitors, others wish to try to engage their public in more technologically current ways. The research described here focuses on the museum, their project readiness, their motivations and their obstacles. The visitors have a significantly different perspective, including their own motivations and barriers to use. The gap in visitor use can be examined through a number of lenses: how individuals adopt uses of technology, how visitors perceive their phones, and visitor motivations and desired outcomes for their museum visit.

Information-seeking is one of only many potential uses of an individual’s phone (compared to social utility, accessibility, status, etc.), and is not by any means the most common use (Wei and Lo, 2006). Thus visitors’ perception of their phones does not immediately indicate the phone’s usefulness as an interpretative device. Whether visitors are likely to use their phones for interpretation depends on their goals for their museum visit.

Similarly, while learning is a key element in many visitors’ articulations of why they choose to visit a particular museum on a particular day (Kelly, 2007), it is not the only motivation for a visit. Visitors come for destination visits (been there, done that), to spend meaningful time with family or guests, and for many other reasons. (Falk, Moussouri & Coulson, 1998). If the use of their phone for interpretation does not immediately further their goals for the visit, visitors will not make use of the opportunity. In-depth qualitative research exploring the relationship between visitor motivations, expectations of the mobile product, and resultant museum experience would help developers create better visitor personas. These personas, based on how visitors have adopted the technology, as well as their motivations and social groupings, would create better products.

2. Case Studies
As tempting (and useful) as it is to cast a wide net to look at mobile adoption across the field, analysis at a micro-level, both institutionally and from a visitor perspective, would be extremely illuminating. The AAM study notes case studies (and visitor research) at the top of the list for future studies. The generation of case studies would be an excellent complement to the analysis of barriers that each project might face. The AAM study notes that there are widely divergent goals for these projects, from increasing visitor engagement to marketing to bringing collections to a broader audience. Separate case studies by project goal would allow institutions to focus on the strategies most appropriate for their goals. The call for case studies is not new, and yet the number of viable case studies compared to the estimated number of projects occurring is very small. Pulling together a case study is quite difficult when deeply embedded within a project, and even more difficult to do in a way that is comparable with case studies from other institutions. A single collaborative project or effort to research and generate case studies from multiple institutions would provide the most comparison points for other institutions.

3. Stratified and/or Longitudinal Field-Wide Studies
For future studies designed to look at the landscape of mobiles in museums from an institutional perspective, perhaps rather than casting a field-wide net, it would be more profitable to cast a series of smaller, more fine-grained nets over time. Given that the museum field is so vast (the AAM study had thousands of responses, and yet that only results in a 14% response rate), future field-wide approaches should invest in stratified or longitudinal studies. A study of a stratified sample of museums would allow closer examination of factors that may influence development of mobile products, such as number of visitors or content focus of the institution. Longitudinal studies would also allow a more reasonable sample size, but could chart the change in mobile developments over time. Do institutions face the same barriers they did a year ago? For those just contemplating a mobile product, were they able to create one or are they still in contemplation?

In conclusion, there is much to be done, both on the product development portion of the mobiles in the museum field, and in the research components. These studies provide any important first step. The next steps should perhaps be narrower, but deeper and much more deeply tied to the visitor.

Kate Haley Goldman, director of learning research and evaluation, National Center for Interactive Learning, Boulder, Colo.


  • Dillman, D. A. 2000. Mail and Internet surveys: The tailored design method. 2nd ed. New York, NY: John Wiley & Sons.
  • Falk, J., Moussouri, T. and Coulson, D., 1998. `The Effect of Visitors’ Agendas on Museum Learning’, Curator, vol. 41, no 2, 106-120.
  • Wei, R. & V.H. Lo. (2006). “Staying connected while on the move: Cell phone use and social connectedness”. New Media and Society. Vol. 8(1):53-72.


Mobile Content Strategies for Content Sharing and Long-Term Sustainability

By Robert Stein

Sustainable Mobile Content
The 2010 Horizon Report for Museums highlights “mobiles” as one of two technology trends on the near-term horizon, noting that “Mobile technology has developed at a staggering pace over the last few years, and today affords many more opportunities for museums…” (Johnson, 2010) The recent explosion of mobile technology as an important way for museums to distribute content is undeniable. Dozens of new tools and companies have emerged in the past 24 months to address the needs of museums that are planning, producing and launching new mobile experiences. A recent Pew Internet survey indicates that 40% of American adults had access to the Internet from a mobile phone in 2010 (Smith, 2010), and studies from Gartner suggest that by 2013 mobile phones will overtake PCs as the most common method for accessing the Internet worldwide. (Gartner, 2010)

With four billion mobile phone subscribers worldwide, it’s clear that mobile devices and content will be an important means of access for museum visitors today and in the future. More recent anecdotal evidence suggests that these trends have continued to accelerate, and that an increasing number of museums are contemplating how they might deliver content via mobile devices. The Museums and Mobile Survey 2011 indicates that over half of large museums (annual attendance of more than 50,000) already have mobile experiences, and almost 70% of all museums say that their institution will “definitely” have in-house mobile content development within the next five years. (Tallon, 2011)

For museums, however, the relationship between museum content and technical change has always been challenging, given the dramatically different time-scales of the two disciplines. A museum’s primary “natural resource” is the content it produces in support of the concepts, collections, and programs that are the source of its mission. Museum collections evolve slowly over many decades, and the concepts and programming created to support the mission of museums are adapted continually, being more an evolutionary optimization of a consistent set of principled goals. This means that most museum content will remain relevant for many years after its creation.

Technology, on the other hand is defined by change. The well-known Moore’s Law states that the density of transistors on integrated circuits will double every 18 months. Applied to the rate of change in technology hardware, Moore’s law has proven to be an accurate predictor of technical innovation since the 1960s. In the last few years however, it seems that software innovation is out-pacing even this dramatic prediction. A recent New York Times article highlights the fact that innovations in software architectures and algorithms have recently trumped even the staggering pace of Moore’s Law for hardware innovation. (Lohr, 2011)

This defines a critical issue that is integral to understanding the relationship between museums and technology. How can museums flexibly adapt to the rapid changes of technical innovation while leveraging a body of content and collections that change at a comparatively glacial pace? Can museums create and maintain building blocks of content infrastructure that will last longer than any particular iteration of technology platforms?

The creation of open-software tools and standards for mobile tours and experiences that can be shared and referenced by museums and vendors would offer an effective way to answer many of these questions, and would provide a mechanism to ensure that content created today could be easily re-purposed and adapted to future generations of mobile platforms. Building consensus among museums and vendors for a description of mobile content, and building tools to aid in the adoption of this platform, are necessary steps to achieve the goals of content sustainability and cross-collection sharing that museums desire. A successful solution of this kind would provide a way to integrate and interoperate between a number of content-creation systems and mobile interfaces, allowing both vendor-provided and custom-developed application software to use the same set of content.

The key element in achieving such compatibility among mobile platforms is the existence of a specification—a common language—describing mobile content and the experiences they provide. In the summer of 2009, the Indianapolis Museum of Art (IMA) proposed a simple draft specification called TourML (pronounced tûrmoil) (Stein, 2009), which offers a working, but preliminary, example of what such a common language might look like.

In order to solicit a high level of input from the community, Robert Stein (IMA) and Nancy Proctor (Smithsonian Institution) organized several free community workshops, inviting museum staff members, academics and mobile vendors to join a preliminary effort to formulate just such a standard. In all, nearly 100 members of the museum community have played a significant role in these workshops, resulting in multiple subsequent revisions to the TourML specification. Notes from all meetings are available from the Museum Mobile Wiki, (Stein and Proctor, 2010) and the resulting TourML specification is available under an open-source license from the project’s Google Code Website. (Moad and Stein, 2009) A more complete discussion of the TourML specification was documented in a recent paper. (Stein and Proctor, 2011)

Figure 1. TourML used as an interchange format

Putting TourML to Work
Well-defined content specifications like TourML are particularly well suited to function as an interchange format or middleware between authoring tools and presentation tools. Figure 1 shows a proposed use for TourML as an intermediate layer in the publishing workflow for mobile tours. In this scheme, tour authors create content in museum-specific content management systems with support for TourML (i.e., the open-source content management system, Drupal). The content management system can then re-write that content as a TourML document. Both the document and all media assets needed for the tour can be bundled together in a single, platform-neutral package. Web applications or device native apps can be easily created to read the TourML document and access these media assets. Since the TourML document is platform agnostic, the same document can be used for apps on several different kinds of devices.

As a practical example of how TourML might be used, the Indianapolis Museum of Art released an open-source mobile tool called TAP in 2009. (Moad and Stein, 2009) TAP provides mobile tour authoring tools based on a Drupal CMS and publishes the tours as mobile apps for the web and iPod Touch/iPhone (Figure 1). As it builds mobile tours, TAP automatically encodes content elements using the TourML standard, so that whole tours can be easily exported from TAP to other platforms or future authoring systems. The goal is for at least 80% of a tour to be able to move directly across platforms, thanks to the TourML schema, minimizing the amount of human intervention required to customize the tour for its new environment.

Figure 2. Sample interfaces for TAP including mobile web and iPod versions

Since its release, TAP has been used by the IMA to provide five exhibition-related mobile tours and one outdoor mobile tour highlighting the museum’s Art and Nature Park. Figure 2 shows a few example screens from selected mobile tours. Since its release, TAP has been successfully downloaded and used by several other museums including the Museum of Fine Arts, Boston (Fleming and Kochis, 2011), and the Balboa Park Online Collaborative. (Sully, 2011)
The Role of Museums and Commercial Partners
The prevalence of software vendors offering mobile products, and the lack of technical expertise on the part of many museums, leads to an important and nuanced relationship between the museum and vendor communities concerning the preservation of museum mobile content. The nature of commercial competition often makes it impractical for vendors to lead the charge for portability and preservation. In addition, without a consensus in the museum community regarding content definitions, tools that ensure content portability and preservation are all but impossible. Previous successes in defining content specifications and standards, such as those supporting collection metadata (LIDO, CDWA Lite, and Dublin Core), have been led by the content producers. In short, the onus of consensus and collaboration around content standards falls squarely on the shoulders of the museum community. But the importance of a healthy collaboration with a community of reliable vendors cannot be overstated. It is prudent to recognize the fact that museums cannot hope to keep pace with technical change without the assistance of commercial vendors who specialize in particular areas. Doing so allows museums to take advantage of the targeted capacities of the most advanced mobile products, while maintaining control of the creation and preservation of its content—a core priority of the museum. Partnership with commercial vendors to encourage adoption of standards, and ensuring that these standards help to enhance the vendor’s business, is the only sure way to secure the viability of such an effort.

Through a ratification of the TourML specification or another similar effort, museums can spearhead the adoption of this specification by the vendor community. In explorations of the feasibility of this effort, many commercial vendors have been very receptive to TourML as a potential specification for mobile content, and have been actively engaged in the mobile workshops that have been held. Some vendors have already integrated early support for the draft TourML specification into their products. Ideally, a healthy relationship and collaboration between museums and the vendor community in this process will result in a viable and sustainable specification that can truly produce the benefits we seek.


  • Fleming, Jenna, Kochis, Jesse, Getchell, Phil. (2011) “Launching the MFA Multimedia Guide: Lessons Learned.” Museums and the Web 2011, Philadelphia, Pa. April 2011.
  • Johnson, Laurence F., Levine, Alan, Smith, Rachel S. and Witchy, Holly. (2010) “Horizon Report: Museum Edition.” Austin, TX: The New Media Consortium, 2010.
  • Lohr, Steve. “Software Progress Beats Moore’s Law” – Technology – Bits Blog – 7 May 2011. Web. 10 Mar. 2011.
  • Stein, Robert J., Proctor, Nancy. (2011) “TourML: An Emerging Specification for Museum Mobile Experiences.” Museums and the Web 2011, Philadelphia, Pa., April 2011.
  • Sully, Perian. “IPod/iPhone Mobile Tours for Everyone | Balboa Park Online Collaborative.” Balboa Park Online Collaborative | Museum Innovation Through Collaboration. 9 Mar., 2011. Web. 14 Mar. 2011. .