Zimm – AR in the Library (Part 1)

I spend a lot of time thinking about what AR is good for, and mostly this means not just choosing content, but places to augment, and audiences to inhabit that augmented place. Some places feel impossible to play in and others feel like giant piles of potential. Today I want to say a bit about the latter.

The creators of Zimm –  (Left to right) Chris Holden, Vanessa Svihla, and Yang Liu. Not present Cindy Pierard.

Libraries should be great places for AR games for a wide variety of purposes, and are an especially excellent place to take advantage of the new AR capabilities in ARIS and other platforms. Advances in the context awareness of mobile devices (AR vision, bluetooth beacons) make indoor games suddenly easier to pull off and more fun to play. Last year, I started working with a new colleague, Vanessa Svihla, and together with one of her students, we spent the spring putting together a small game in our own university library. We learned a lot that I hope might help others.

On this blog, I’d like to share a few thoughts about

• How libraries could be places for playing are games and for constructive, creative work between departments, students, staff, and faculty. A new place for AR and model for collaboration that can open doors.
• New AR features in ARIS and implications for design of AR games, especially in indoor spaces, and some design considerations for their use.
• How a diverse group of people can come together around game design and place, and some suggestions for doing this kind of work in a way that is approachable and hopefully sustainable.
• The pervasive nature of place. How we can use game design and play to engage with the places we live and work.

I’ve split this conversation into a few parts. If you’re coming to AR from far away, or just Pokemon Go, these articles might help you see the depth of purpose I and others have found in this work. There’s also a lot more on this very blog.

If you’d like to know more about Zimm itself, here’s a bit of basic info. If you’re in the area and would like to give the game a go, get in touch.

Intro Plaque from Zimm

Here, in Part 1 – Why Libraries, I lay out the basic reasons why I think libraries and AR could be like chocolate and peanut butter.

Part 1 – Why libraries?

Libraries are great places for so many reasons. They will likely play new and important roles in education and communities in the near future, and if you haven’t looked in a while, they already are. Precisely because “a place to find dead trees and ink” seems anachronistic now, libraries are confronting the future generally in more direct and creative ways on the whole than school. They are also very interesting to begin with and some of the only remaining public spaces in America, places to go and be and meet without needing to have a shopping agenda. Libraries provide diverse and deep resources, not just books and quiet. They are jumping off points for many journeys, from wizarding worlds, to job applications, to organizing community action. And while libraries are not playgrounds, places for yelling and jumping generally, they may be excellent places in which to structure other sorts of play. Each book is a world between covers, waiting to be discovered and shared. And even as the dead trees dwindle a bit, this basic ethos seems to pervade much of what a library might offer. I’ve felt it in every conversation I’ve had with a librarian over the last decade. They’re excited for the future and working hard to provide open doors that beckon in new ways.

A third of a controversial mural in Zimmerman Library at UNM. It’s racist content set the stage for our design

This setting is an excellent place for AR especially. The chief strength of AR is to help people explore worlds hidden in plain sight. It could easily serve their roles as centers of discovery.

And as we focus on libraries, let’s keep these goals lofty. Too often, we tend to get a bit too sidetracked by solving logistical problems for patrons in the most basic sense. I’ve seen a host of AR projects set in libraries whose sole objective was to simplify the finding of a correct call number in the stacks. Sure, the sorting system (whether Dewey or LoC) is a barrier to outsiders finding what they need, but the real barriers to participation in a library space are largely psychic.

This mirrors a lot of problems with learning where we focus on the mechanics of knowledge uptake in a very general way, not realizing that most problems really stem from a lack of care and familiarity, feeling like the learning places are where you can become something great.

So as we look at libraries for AR and other games, let’s see our design challenge as a need to realize the potential for exploration there. If the imagination is there, navigating call numbers might be more of a quest to complete, or a puzzle mechanic, not just a simple navigation UI concern. We should begin to think in terms of long lost treasure maps as much as an efficient system for locating a desired title.

We should remember that there’s more to finding your way than being able to locate a spot in the stacks.

Libraries are partly specific and partly generic

One of the problems faced by place-based AR projects is that the design feels too parochial, hard to pass on to others in new places. Libraries offer some help. Libraries today are large, varied, mostly indoor spaces. Stacks are just the beginning. And unlike a lot of the unique settings for AR (say a specific neighborhood in Albuquerque), libraries exhibit both the universal and local. No other library is quite like yours, but there are a lot of similarities across most, both in what you’ll find and how the space is organized. AR design work done in a particular library has potential to be easier to localize to other places and settings.

This exact mural isn’t in your library, but incidental art is commonplace. Our use of it in Zimm is easy to pass on.

Libraries also have potential as a sort of neutral ground, a place not controlled entirely by a single interest, use, or age group. Such neutrality can be hard to find in schools, but it is badly needed now, as we realize our divisions among job descriptions, disciplines, and function prevent us from addressing the educational needs of this century. We are bound within our roles, timetables, and departments, and libraries may give us some of the space we need to collaborate in more meaningful ways to supercharge learning beyond the ordinary.

As usual, what should be a simple blog post about a cool idea to make games in the library turns—for me—into a polemic on structural issues in education. So rather than going too far down that road, I’ll just list a couple reasons this stuff matters.

1. Agency for all. Staff have power here along with faculty. And students can easily be welcomed as co-owners of action and planning in a way that is just not possible in other areas. This neutrality is hugely important for empowering all stakeholders, a goal often missing from educational design and reform efforts, whose inclusion should be important as a basic humanistic principle of this work we call education.
2. Scale. It has been maybe the chief difficulty of educational innovation. How to spread ideas beyond their original contexts? Usually, and this is not a good approach, just convenient in a world of mass media, we put the idea in a box and send it off, excluding of the needs and expression of interest of multiple stakeholders in the new spaces. All stakeholders interests and needs need to be addressed, not just what those at the top demand of those below them. The internet affords new forms of growth in partnership—grassroots, non-hierarchical—that we hope to use as better routes to scale.
3. Sustainability. If an important project depends on the drive of a single person and their incentives, what happens when that person takes another job or decides it is not worth the trouble to keep pushing that stone uphill. Including others meaningfully is the first step in creating something that is more like an aspect of the ecosystem instead of just a pet project.

Not every single experiment needs to tackle these beasts head on, but work done without a consideration of how it leads into their toothy jaws and back out again is doomed from the start. If you’re new to this topic of conversation, I’d recommend reading Seymour Papert’s The Children’s Machine. It was written in 1993, a decade after his (and others’) groundbreaking Logo software was introduced to school, alongside computer hardware, with the idea that they would revolutionize education in schools. They didn’t and Papert has some cogent and timeless thoughts about why not.

Zimm – Parts 2-?

So that’s why libraries, structurally, might be good places to make AR games. Next, in Part 2, I’ll mention a few ways that the affordances of AR match the constraints of the space and where some apparent possibilities for design lie. We’ll look at some of the features of the game we made at Zimmermann (it’s called Zimm) and what we learned about the realities of making games in libraries. It’s also a good chance to look at some of the newer AR mechanics available in ARIS.

In Part 3, I’ll go through some of the practical logistics of this work, how a group of people can come together around the idea of making a game in a library, learn to work together, set appropriate expectations, and use this work as a way to grow closer together and understand your common cause as well as the burgeoning art of AR design. Usually, we spend a lot of time with AR talking about mechanics of the software, but not the groupings of people who will make these games. The latter is hugely important if this work is going to go anywhere in the end and remain a vernacular rather than a license we buy into.

If I get to Part 4, there’s one other issue that came up with Zimm that feels central to me when it comes to making sure we do the best work we can with AR. Place. Place is always important and more than location. Even when you don’t plan for it, it intrudes. Instead of seeing it as an unwanted variance, we can listen to it to do educational work that is more vital and relevant to the lives of those who participate in our experiments.

I’d love it if these articles, or just their titles, got someone else excited about making games in libraries, and not just to find call numbers. Once we have a few people experimenting in the area, the real interesting conversations can begin.

 

Showing Off Seesign

There’s a new project here at Local Games Lab ABQ: Seesign. It is Celestina Martinez’ tool to visually identify signs (as in sign language, not road signs). She used Nomen to make Seesign, a new platform from Field Day Lab, my ARIS co-conspirators.

Martinez’ goal with Seesign is to replace the very user-unfriendly dictionaries sign language students need to use to look up unfamiliar signs with something obvious and easy-to-use. The usual approach to looking up signs is to search by the English word you think might correspond to what you saw. The affordances of dictionaries are just not very useful for organizing visual as opposed to textual information. It is not hard to see that this is terribly inefficient for learners who need to look up signs they don’t recognize.All you have to do is mark the features you do notice: hand shapes, movement type, etc.

Seesign’s Main Screen

You do this by tapping the relevant choices on Seesign’s main screen (above). The possible matches to your choices are numbered and linked at the top right of the screen, and are narrowed in scope with each additional feature you mark. At any point, you can tap on this number to see those potential matches (below; in the actual app these are animated GIFs).

Likely Matches to a Choice on Main Screen

Nomen makes it quite simple for novices without much technical background to produce tools for this and similar identification tasks. More on that later, but for now, note that Ms. Martinez does not have a technical background and put this app together, with only minor assistance from me, part-time over this last year. She began work on Seesign last fall as part of an assignment in my course, Things That Make Us Smart, and has continued this semester as an independent study. She also submitted it to the 2016 UNM App Contest, but did not win any of the big prizes (robbed IMO).

Why Seesign Is Cool

Ms. Martinez describes it a bit better than I do, but basically, her app was born of the frustration of learning sign language using common tools. The usual approach to looking up signs is to search by the English word you think might correspond to what you saw. It is not hard to see that this is terribly inefficient. It works fine if you are trying to say something you know in English, but not at all for identifying signs you see. The affordances of dictionaries are just not very useful for organizing visual as opposed to textual information. Reverse-look-up visual dictionaries do exist, but they are not common and they are expensive and very limited in terms of

• The words they provide – very few
• How signs are represented – static images, very basic
• The mechanics of looking up – tons of pages to rifle through.

Seesign improves on all of these greatly with its obvious, streamlined interface for marking features and looking at the matching results. Not only is the user able to mark as little or as much of what they recognize—maybe you notice the type of movement, but not what hand shapes were used—but Nomen is also flexible in how it organizes and displays matches: showing both “likely” matches, signs that match all marked features, and “possible” matches, signs that match at least one marked feature. And since it is a web-app, unlike my long-time favorite ARIS, Seesign is accessible on almost every internet connected device with a screen: phones, tablets, laptops, desktops.

So far, Martinez has developed two iterations of her basic prototype (one each semester). She currently only has 10 words catalogued, but has worked extensively at

1. Developing a useful categorization scheme that classifies words in a way that brings out the visually distinct features of signs and fitting them into a small number of readily apparent choices, and
2. Producing a visual format for illustration and feedback on the users’  marked choices: animated GIFs.

I’ve been really impressed with Ms. Martinez’ work this last year, both in her general ability to start with an idea and really dig into developing it into something tangible (not just the app but the “soft” work too: finding a model to work with, entering the UNM app contest, etc.), but also the acuity with which she has developed (through thoughtful iteration) the experience of her app, especially through those two areas above. She also displayed attention to detail. Nomen is strictly-speaking authorable without writing code, but it is still essentially a working prototype of the basic functionality Field Day hopes to turn into something far more polished in the next year. In particular the formatting guidelines are strict and unforgiving. Ms. Martinez’ build compiled on her very first upload, a feat I doubt I will ever manage.

This work is worth looking at for several reasons, some of which go far beyond her specific innovations:

• It is a good idea: the advantage and use of this tool to aid sign language learners is clear.
• It begins to describe a model for extending learning outside the classroom based on interests developed within.
• It is an example of how accessible tools can open new areas of development by subject matter enthusiasts instead of technologists.
• It is an example of how a local games lab can encourage and support development in small but meaningful ways.
• It is an opportunity to discuss how events like app contests might focus innovation and enhance collaboration, and having seen this contest play out, what might be learned from what apps are being tried, why, and how.
• It is the first documented use of Nomen (there’s another one I’ve been working on I hope to post about soon), a new accessible and open tool that could see myriad uses by others across many fields of inquiry and contexts of interaction.

In follow-up posts, I’d like to discuss these more general ideas. If you beat me to it or have other resources to point to, then by all means…