Archive for the 'User Assessment' Category

ExhibitFiles at the Visitor Studies Association conference

Tuesday, July 19th, 2011 by Wendy Pollock

Coming to Chicago next week for the Visitor Studies Association conference? Get an update about evaluation of ExhibitFiles during a session on Tuesday, July 26, 9:00-10:15 a.m., when Carey Tisdal and I will be talking about Knowledge, Identity, and Networks in the Informal Learning Community. There are many other sessions on the program which should also be useful to exhibit practitioners. Find out more here.

If you contributed to the ExhibitFiles evaluation by responding to a survey or agreeing to be interviewed about your perspective, we truly appreciate your help. You can find the report of the remedial evaluation on the InformalScience website. Carey will also post results of the summative evaluation that is now underway.

How did you get here – and will you come back?

Monday, August 9th, 2010 by Wendy Pollock

Word of mouth: That’s how most members found out about ExhibitFiles.  Others stumbled upon it while searching the web. That’s by design: the site uses a number of strategies to make it likely profiles and posts will show up high on search results.

Once people have joined, why do they come back? For inspiration and help with a new project are two big reasons. But email from the site is number one – a reminder to come back and see what’s new.

That’s why we’ll be relaunching the newsletter soon. We’re looking forward to seeing members come back often, and contributing more.

These findings are based on a study carried out earlier this year by Carey Tisdal of Tisdal Consulting, St. Louis, Missouri.

Who belongs to ExhibitFiles?

Monday, August 2nd, 2010 by Wendy Pollock

Who belongs to ExhibitFiles? About 1,600 people from around the world have joined the site since its April 2007 opening – including our newest members, from the Chicago area and Kuala Lumpur, who joined in the last week.

Primary work responsibility of ExhibitFiles members, February 2010When we asked earlier this year, most ExhibitFiles members said they were experienced (54.7%) or senior (25.0%) professionals. Another 14.7% considered themselves “entry level,” and some (4.7%) were students. (Fewer than one percent called themselves “retired.”) Members are involved with exhibitions in a wide variety of ways. Most listed as their primary responsibility exhibition development (33.5%) or exhibition management (13.0%), but we also have members who work primarily in new media, visitor studies, and fabrication and maintenance. (See right.)

Science centers and museums are the type of organization or context in which the largest single group of members work.Primary organization or work context of ExhibitFiles members, February 2010 That’s not surprising, because the site was started by ASTC with funding from the National Science Foundation. But from the beginning, the site has been open to everyone in the museum exhibition community, including independent firms and consultants, the next largest group (17.2%). (See left.)

These findings are based on a study carried out earlier this year by Carey Tisdal of Tisdal Consulting, St. Louis, Missouri. Methods included a February 2010 survey of ExhibitFiles members (N=286, a response rate of 22.2%); in-depth interviews with users with high, medium, and low levels of participation (N=15); and analysis of the ExhibitFiles member database. We’ll be reporting more findings in future posts – and letting you know more about the changes we’re making in response.

New and improved

Friday, July 30th, 2010 by Wendy Pollock

We’ve listened to you. With your valuable feedback, we’ve been working on a new and improved version of ExhibitFiles, which opens today. Here are some of the features you may notice:

  • Bits – Those short posts we added a few months ago are now visible on the home page.
  • Profiles – We’ve added a tabbed section and place for you to pull in info from other social networking sites like LinkedIn.
  • Members page - We’ve added a sorting feature that highlights top contributors. Users without full names are no longer prioritized.
  • Search – Improvements include more targeted search and more result relevance.
  • Other improvements – We’ve improved browser compatibility and boosted speed.

Over the next few weeks, we’ll be sharing some of the results of the recent evaluation that helped us plan these and other changes still ahead. For now, many thanks to James Kassemi, Jim Spadaccini, and other members of the Ideum team; to Carey Tisdal, evaluator; and to our funder, the National Science Foundation.

What difference does an exhibition make?

Friday, August 1st, 2008 by Wendy Pollock

We were happy to see two new ExhibitFiles case studies posted by participants in the PI Summit, held July 25-26 in Washington, D.C. Liza Pryor of the Science Museum of Minnesota wrote about Science Buzz, and Elizabeth Fleming of the Museum of Life and Science in Durham, North Carolina, wrote about Flip It, Fold It, Figure It Out. Like a number of other exhibitions described in earlier case studies, both were funded by the National Science Foundation (NSF), a U.S. federal agency.

As anyone in the informal science education field with NSF funding now knows, documenting the impact of the work is an increasingly high priority. NSF issued a Framework for Assessing the Impact of Informal Science Education Projects (PDF) earlier this year to guide grantees, and future proposals will need to address impacts laid out in this publication.

But defining intended impacts can be a challenge in the rich and multifacted world of informal, lifelong learning, and assessing whether or not we’ve achieved these impacts can be even tougher. In a later reflection added as a comment about her case study, Liza notes one challenge she faces in assessing the impact of Science Buzz: “We’re working on our summative evaluation,” she says, “but we don’t have anything to compare our data TO. We’ve got the data from the Pew internet study, but it’s not too helpful. I’m particularly interested in studies of online communities. What’s a decent participation rate? Any way, without resorting to discourse analysis, to figure out what people are learning?”

Maybe this online community can give Liza some help. In fact, we’ll be posting soon about how we’re thinking about this in relation to ExhibitFiles itself.

Testing usability

Wednesday, April 18th, 2007 by Wendy Pollock

We were at Museums & the Web in San Francisco last week, getting feedback on ExhibitFiles in the usability lab. Many thanks to Michael Twidale and Paul Marty, who led the process, and Richard Urban, who played the role of user. We’re taking their comments into account in some last-minute adjustments we’re making to the beta site before opening next week. Meanwhile, the core group that’s been advising on development has started setting up accounts and posting case studies and reviews (and reporting bugs), so new users will be able to explore before jumping in. We already have a mix of old and new, large and small, funky and classic, science and art, with more in the works. We hope to see you on the site, Monday, April 23, when ExhibitFiles goes live.

April 23 ExhibitFiles BETA launch

Thursday, March 22nd, 2007 by Wendy Pollock

We’ve set the date. On April 23rd the ExhibitFiles website will be publicly available. Over the last nine months, we’ve shared part of the process with you via this blog. Now, in a month, you’ll be able to see (and participate in) ExhibitFiles for yourself. We appreciate all of the comments and suggestions we’ve received during this process; they have helped us improve the site’s design and functionality. Randi Korn & Associates will be carrying out a formal evaluation of the site during the summer, but meanwhile we hope you’ll continue to send your comments.
Once ExhibitFiles launches, we’ll continue to publish news here for the first few months as we consider whether and how to continue the blog. Meanwhile, we look forward to seeing you online in April!

East Coast Design Meeting

Wednesday, March 7th, 2007 by Jim Spadaccini

Site Structure 4.0
It’s been a while since we’ve posted anything on the blog, but we wanted to bring you up-to-date on our design process. In mid-February, we met at ASTC headquarters in Washington for a day-long meeting to review the ALPHA site.

Our core team has been working with the ALPHA build for the last few weeks, and their comments have helped us fine tune the structure of the site as work toward BETA. (The latest site diagram above is available as a PDF.) One of our main concerns has been to make this site as responsive to the needs and ways of working of exhibit practitioners — including time constraints. So we’ve been simplying and clarifying while working out bugs. Work has begun on the final phase which will lead to a BETA release. We’ll be publishing that release date soon.

ExhibitFiles structure emerges

Monday, October 23rd, 2006 by Jim Spadaccini

This site diagram (site diagram v2 PDF) illustrates the relationship between the various components of the proposed ExhibitFiles site. Since one of the key functions of the site is to “include shared records of exhibition descriptions,” the ability to add, browse, and search these records is a major part of the site structure. So far the design process has focused on the elements that make up these individual records. (See blog posts, “Personal Profiles” and “Case Studies and Reviews“).

Beyond the specifics, our discussions have focused on what is the “right amount” of information. Too much and users won’t want to add case studies or reviews, too little and we might be omitting important information. Obviously, we need to meet the needs of the members of the ExhibitFiles site. A major and ongoing concern has been how can we best encourage active participation.

Wendy’s last post, Tapping the wisdom of the crowd, explained in detail the concept of drawing ISEN-ASTC-L messages directly into the ExhibitFiles site. This is not the only place that ExhibitFiles members will have opportunities to converse. Comments (and ratings) can be added to any case study or review. Trying to anticipate conversation in a complex social site is difficult, so we intend to see how things go and make some adjustments as the site is launched and these conversations begin.

In our Berkeley design meeting, the issue of “ratings” was discussed and, for the most part, participants were concerned that employing such a system could trivialize contributions to the site. It was agreed that rating systems such as the starred Yahoo! ratings for news stories would not be appropriate in this environment.

As an alternative, we looked to favorites as way to sort reviews, case studies, and other content on the ExhibitFiles site. As members add items to their own favorites, a record of that addition will be associated with that item, so reviews or case studies that are favorited by many users could be presented in some way (e.g., most favorited). Also, each member’s list of favorites can be useful for others, in the same way the del.icio.us bookmarks can be made available to all members.

As our discussion progressed, we revisited the concept of “ratings.” This was due in part to addition of the ISEN-ASTC-L component. Having a way for ExhibitFiles members to help sort the messages that are most relevant to the site is essential, since not all of the messages on the Listserv are directly related to exhibit design. The example of Digg in which visitors give a simple thumbs up or thumbs down to content items as way to sort was discussed, as was Amazon’s Was this review helpful to you? Yes or No feature.

We’re looking to add a similar feature: “Did you find this useful? Yes or No.” This simple Yes/No rating along with favorites will help “score” the Listserv messages as well as case studies and reviews. While ratings won’t be the only way to browse (and search?) the ExhibitFiles content, having some methods for letting the community decide what information is most useful is important. We hope to have a large collection of records and conversations about exhibit design, so community members will need various methods to find the materials that are important to them.

After months of discussion and review we’re finally getting to the point of building out components of the ExhibitFiles site. While the pace will pick up dramatically over the next few weeks, we’re still very interested in your comments and ideas. The current push is for our first build to be ready in January 2007. We will be taking a look at the site again in late spring once the first members begin to contribute and use the site. As we design and develop elements for the ExhibitFiles, we’ll be posting page grids and other diagrams on this blog. As always, we welcome your comments and questions.

Personal Profiles

Monday, August 28th, 2006 by Jim Spadaccini

Since ExhibitFiles will be a community-built website, it was decided early on to require a membership (it will be free) for authoring and commenting. Since our design workshop, we’ve been actively exploring how we might structure “personal profiles” within the ExhibitFiles community.Based on the comments from the design workshop and our group of advisors, we’ve outlined the following elements as part of ExhibitFiles profile:—-NameWhat I do, where I workWhat I’ve done in the past (multiple-no more than five)City, State, CountryEmailExhibitFiles password (repeat password)Website(s) (multiple-no more than three)More about me (text area)My image (upload a JPEG or PNG)Resume and/or portfolio (upload a PDF, doc, or URL)What I’ve added to ExhibitFiles (automatically populates once a user adds content to the site)Things I’ve bookmarked in ExhibitFiles (automatically populates once a user bookmarks files in the site)Terms of Use, Privacy Policy, Email Use Agreement (check box)—–The purpose of the profiles is to allow ExhibitFile members to connect with each other, but also to learn from one another. We’re hoping that the “What I’ve added to ExhibitFiles” and “Things I’ve bookmarked in ExhibitFiles” will provide additional ways to access the “records” (the content of the ExhibitFiles site).Finding a balance between enough information and too much is an ongoing challenge in planning for the site. Too much and no one will want to fill in lengthy forms, not enough and we may be leaving out valuable information.The profiles (and language used) are intentionally informal and brief. We’re hoping that ExhibitFiles users will post additional information (resumes and porfolios) and/or link to additional sources using the “Website(s)” element. We’re not looking to recreate LinkedIn (who were part of our Competitive Analysis), but rather to create profiles that have a balance between professional in and personal information.As always, if you have any comments or suggestions, please feel free to post them here.