If you contributed to the ExhibitFiles evaluation by responding to a survey or agreeing to be interviewed about your perspective, we truly appreciate your help. You can find the report of the remedial evaluation on the InformalScience website. Carey will also post results of the summative evaluation that is now underway.

Once people have joined, why do they come back? For inspiration and help with a new project are two big reasons. But email from the site is number one – a reminder to come back and see what’s new.

That’s why we’ll be relaunching the newsletter soon. We’re looking forward to seeing members come back often, and contributing more.

These findings are based on a study carried out earlier this year by Carey Tisdal of Tisdal Consulting, St. Louis, Missouri.

When we asked earlier this year, most ExhibitFiles members said they were experienced (54.7%) or senior (25.0%) professionals. Another 14.7% considered themselves “entry level,” and some (4.7%) were students. (Fewer than one percent called themselves “retired.”) Members are involved with exhibitions in a wide variety of ways. Most listed as their primary responsibility exhibition development (33.5%) or exhibition management (13.0%), but we also have members who work primarily in new media, visitor studies, and fabrication and maintenance. (See right.)

Science centers and museums are the type of organization or context in which the largest single group of members work. That’s not surprising, because the site was started by ASTC with funding from the National Science Foundation. But from the beginning, the site has been open to everyone in the museum exhibition community, including independent firms and consultants, the next largest group (17.2%). (See left.)

These findings are based on a study carried out earlier this year by Carey Tisdal of Tisdal Consulting, St. Louis, Missouri. Methods included a February 2010 survey of ExhibitFiles members (N=286, a response rate of 22.2%); in-depth interviews with users with high, medium, and low levels of participation (N=15); and analysis of the ExhibitFiles member database. We’ll be reporting more findings in future posts – and letting you know more about the changes we’re making in response.

• Bits – Those short posts we added a few months ago are now visible on the home page.

• Profiles – We’ve added a tabbed section and place for you to pull in info from other social networking sites like LinkedIn.

• Members page - We’ve added a sorting feature that highlights top contributors. Users without full names are no longer prioritized.

• Search – Improvements include more targeted search and more result relevance.

• Other improvements – We’ve improved browser compatibility and boosted speed.

Over the next few weeks, we’ll be sharing some of the results of the recent evaluation that helped us plan these and other changes still ahead. For now, many thanks to James Kassemi, Jim Spadaccini, and other members of the Ideum team; to Carey Tisdal, evaluator; and to our funder, the National Science Foundation.

As anyone in the informal science education field with NSF funding now knows, documenting the impact of the work is an increasingly high priority. NSF issued a Framework for Assessing the Impact of Informal Science Education Projects (PDF) earlier this year to guide grantees, and future proposals will need to address impacts laid out in this publication.

But defining intended impacts can be a challenge in the rich and multifacted world of informal, lifelong learning, and assessing whether or not we’ve achieved these impacts can be even tougher. In a later reflection added as a comment about her case study, Liza notes one challenge she faces in assessing the impact of Science Buzz: “We’re working on our summative evaluation,” she says, “but we don’t have anything to compare our data TO. We’ve got the data from the Pew internet study, but it’s not too helpful. I’m particularly interested in studies of online communities. What’s a decent participation rate? Any way, without resorting to discourse analysis, to figure out what people are learning?”

Maybe this online community can give Liza some help. In fact, we’ll be posting soon about how we’re thinking about this in relation to ExhibitFiles itself.

Our core team has been working with the ALPHA build for the last few weeks, and their comments have helped us fine tune the structure of the site as work toward BETA. (The latest site diagram above is available as a PDF.) One of our main concerns has been to make this site as responsive to the needs and ways of working of exhibit practitioners — including time constraints. So we’ve been simplying and clarifying while working out bugs. Work has begun on the final phase which will lead to a BETA release. We’ll be publishing that release date soon.

This site diagram (site diagram v2 PDF) illustrates the relationship between the various components of the proposed ExhibitFiles site. Since one of the key functions of the site is to “include shared records of exhibition descriptions,” the ability to add, browse, and search these records is a major part of the site structure. So far the design process has focused on the elements that make up these individual records. (See blog posts, “Personal Profiles” and “Case Studies and Reviews“).

Beyond the specifics, our discussions have focused on what is the “right amount” of information. Too much and users won’t want to add case studies or reviews, too little and we might be omitting important information. Obviously, we need to meet the needs of the members of the ExhibitFiles site. A major and ongoing concern has been how can we best encourage active participation.

Wendy’s last post, Tapping the wisdom of the crowd, explained in detail the concept of drawing ISEN-ASTC-L messages directly into the ExhibitFiles site. This is not the only place that ExhibitFiles members will have opportunities to converse. Comments (and ratings) can be added to any case study or review. Trying to anticipate conversation in a complex social site is difficult, so we intend to see how things go and make some adjustments as the site is launched and these conversations begin.

In our Berkeley design meeting, the issue of “ratings” was discussed and, for the most part, participants were concerned that employing such a system could trivialize contributions to the site. It was agreed that rating systems such as the starred Yahoo! ratings for news stories would not be appropriate in this environment.

As an alternative, we looked to favorites as way to sort reviews, case studies, and other content on the ExhibitFiles site. As members add items to their own favorites, a record of that addition will be associated with that item, so reviews or case studies that are favorited by many users could be presented in some way (e.g., most favorited). Also, each member’s list of favorites can be useful for others, in the same way the del.icio.us bookmarks can be made available to all members.

As our discussion progressed, we revisited the concept of “ratings.” This was due in part to addition of the ISEN-ASTC-L component. Having a way for ExhibitFiles members to help sort the messages that are most relevant to the site is essential, since not all of the messages on the Listserv are directly related to exhibit design. The example of Digg in which visitors give a simple thumbs up or thumbs down to content items as way to sort was discussed, as was Amazon’s Was this review helpful to you? Yes or No feature.

We’re looking to add a similar feature: “Did you find this useful? Yes or No.” This simple Yes/No rating along with favorites will help “score” the Listserv messages as well as case studies and reviews. While ratings won’t be the only way to browse (and search?) the ExhibitFiles content, having some methods for letting the community decide what information is most useful is important. We hope to have a large collection of records and conversations about exhibit design, so community members will need various methods to find the materials that are important to them.

After months of discussion and review we’re finally getting to the point of building out components of the ExhibitFiles site. While the pace will pick up dramatically over the next few weeks, we’re still very interested in your comments and ideas. The current push is for our first build to be ready in January 2007. We will be taking a look at the site again in late spring once the first members begin to contribute and use the site. As we design and develop elements for the ExhibitFiles, we’ll be posting page grids and other diagrams on this blog. As always, we welcome your comments and questions.