Our objective is learning, so to determine if learning is happening we use assessments. This section explores what we deem as acceptable measures of learning and what evidence we've collected so far.

On assessment - A challenge

Our goal is to enable people to prepare for and survive a pandemic. Essentially we want behavior change. Perhaps the truest measure of whether people learn to survive a pandemic is observing them during one. However, it's impractical for us to wait until a pandemic. We therefore defined what near term measures we could assess that would form reliable prerequisites to survival during a pandemic.

We also want to prove that people will survive, but we want to show that our theoretical approach is sound. We've designed a unique cognitive-approach where we use models to teach emergency preparedness. It could be argued that a more behaviral (stimulus-response) approach would be more effective. Behavioral methods have been used successfully in fire safety training through the use of songs and repitition as in Stop-Drop-And-Roll. So then we looked at mental model research in education to use their assessment design as a best practice.

We faced another challenge in that our game is varied and that basic components build upon each other. Our design is to layer these building to form a framework of understanding. Therefore we felt we needed the entire finished game to reliably assess learning.


Again, we had to define a near term measure before we build the entire game.
Thus we came up with the following assessment measures:

Long Term Goal	Near Term measure
Learners know how to behave during an actual pandemic	Learners are able to recall information in a post-test. Knowledge is a basic pre-requisite to behavior change. If learners remember what they learned they are more like to transfer it to an actual situation. Learners are able to transfer learning other related emergencies.
All 100 minigames together form an integrated framework of understanding.	If 1 mini-game causes learning, it is probable that 100 may do so as well.
The game is highly sticky, people transfer lessons to real life.	Are players likely to send information from the game to friends. Does information from the game come up in conversations with friends and family a week after the game.
The game is highly motivating. The short, toy-like games encourage game play; more learning.	If people want to play the game it is likely they will increase exposure to learning elements. A measure would be playtest interviews and observation of play.

These smaller near term assessments become what we call "formative assessments" where we gain information during the development process of whether people are learning. We will note here that usability and game play assessments are performed during the design process and are separate from the "learning assessments described here. To see an example fo a usability test, go to the Playtests page.

Example Assessment for the Travel Game

The travel game is designed to:
- Give people a gut-level understanding of the wild-fire spread of viruses
- Build an awareness for the way public transportation affects spread

Method:
We will use interviews and playtest observation to measure learning. Players will be given the travel game to play for 20 minutes. At the beginning of the test, they will be background questions and then be given a scenario. After they play they will be given another scenario.


Pre-interview Question:
- How many people are in your home?
- How would rate your knowledge about bird flu on scale of 1 to 10?
- Do you play video games
- What is your age and gender?

Phase 1: Pre-Game Interview
I’m going to give you a scenario and I’m going to ask you to tell me what you think might happen next. Here is the scenario
When you go home tonight there will be a news report about 400 people falling ill in Hong Kong, Russia and London due to a deadly virus. The virus is believed to be highly contagious. It’s still very early, but experts have studied the virus and have qualified it as a pandemic, meaning it could spread to every nation on the planet. Just at the close of that report you hear some breaking news, one case has just appeared in Seattle.

Questions:
Think for a minute and tell me what you imagine will happen. What sort of activities do you imagine will happen next?
What would you personally do?



Phase 2: Post-Game Interview
I’m going to give you another scenario. This scenario just happened in the United States. A strain of bird flu was found in an animal last week. Scientists are unsure whether this is the lethal strain killing people in Indonesia right now. Tests are being run over the next couple of days. Health officials have been notified. The nation awaits an answer.

Questions:
Think for a minute and tell me what you imagine will happen. What sort of activities do you imagine will happen next?
What would you personally do?