jwilzewske: National Performing Arts Convention: Electronic Voting Systems (EVS) to Set Priorities

L et your voice be heard and your vote count, so that we will speak with one voice and together ensure a vital future for the performing arts.”
— NPAC 2008 brochure.

I usually gag at this sort of trendy management books. (I still bear scars from the ‘Who Moved My Cheese?’ era.) But [Collins’s ‘Good to Great and the Social Sectors’] is smart, down-to-earth wisdom.”
— Michael Bento, reviewer on Amazon.com .

W hen reproduction, distribution, and categorization were all difficult, as they were for the last five hundred years, we needed professionals to undertake those jobs, and we properly venerated those people for the service they performed. Now those tasks are simpler, and the earlier roles have in many cases become optional, and are sometimes obstacles to direct access, often putting the providers of the older service at odds with their erstwhile patrons.”
— Clay Shirky, Here Comes Everybody, p. 78.

W e’re not being atomized. We’re molecularizing, forming groups that create local culture. What’s happening falls between the expertise of the authorities in the editorial boardroom and the ‘wisdom of crowds’.”
— David Weinberger, Everything Is Miscellaneous, p. 131.

Management guru, Jim Collins, will speak on Thursday 12-JUN about ‘From Good to Great and the Social Sectors’ at the 2008 National Performing Arts Convention in Denver. Among the topics are ‘Getting things done within a diffuse power structure’ and ‘Getting the right people “on the bus” within social sector constraints’.

The NPAC is portrayed as a technology-enabled event. Without doubt, it is that. An Advocacy Center will be available with computers for Convention delegates to send messages to the federal, state and local legislators about key issues. Software will be provided that gives comprehensive access to contact information, updates on the most important legislation, and supplies standard formats for letters. “Here's your chance to act on your commitment to take action!” exclaims the NPAC brochure. Delegates can set up 20-minute appointments to learn from experts in one-on-one consultations at the ‘SmART Bar’ at the Convention. NPAC will run from June 11 through June 14.

But technology-enabled is not the same as ‘egalitarian’ or ‘fair’. For example, professional doctors and lawyers determine the parameters and credentialling of their peers, the prerequisites for acceptance and licensing. And professional chamber music presenters and agents determine the parameters for performers and ensembles. But what happens to the word ‘professional’ when works of comparable quality and skill can be conceived, produced, and distributed without expensive or centralized means of production? Oftentimes, not much changes. Filtering by the traditional ‘gatekeepers’ continues. The usual suspects show up in force, and the conventional wisdoms are usually replicated. Little ‘outside-the-box’ thinking occurs. Few real revelations emerge.

Organized by discipline, the fields of dance, music, opera, theater and others have been supported by national service organizations that provide a range of leadership and support services as needed by the respective fields. The combined effort of local arts organizations and their partner national service organizations has resulted in an exponential growth in the number of producing and presenting organizations and audiences. Technology-enabled surveys tend merely to reveal the extent of the market fragmentation that has set in.

In fact, while the traditional discipline-centric structure has merit for new and growing genres, it works at cross-purposes against goals that are inter-disciplinary, that are beyond the remit of any one service organization, or that reveal contested turf between disciplines. In the 2008 convention, NPAC will offer delegates an opportunity to address interdisciplinary issues in ways that aim to harness the energy and expertise of the whole non-profit performing arts sector.

Taking Action Together is the theme of the 2008 National Performing Arts Convention (NPAC). NPAC aims to lay a foundation for future cross-disciplinary collaborations and effective advocacy in terms of funding and public policy.

Three ‘caucus sessions’ will be held, one each day (Wednesday, Thursday, and Friday) during the convention. The caucuses are designed to elicit ideas and set the agenda for consideration in Saturday’s summation ‘21st Century Town Meeting®’, which is aimed at driving future planning and priorities for the performing arts in the years to come. The caucuses will be broken into round tables of ten delegates each. Each table will engage in a facilitated conversation to identify goals and action items toward enhancing performing arts’ prospects in the U.S. Each caucus will address a specific question and target a specific outcome. The responses of all of the round tables will be compiled every day and shared with the full convention to form the basis of the following day’s work. Industrial survey methodology at its intensive best.

The National Performing Arts Convention has engaged AmericaSpeaks Inc. to lead the process of caucuses leading to the Town Meeting, using methods and technologies that they have refined in previous public opinion analysis projects. Facilitators are charged with insuring a level playing field for all ideas, opinions, and suggestions.

The ideas that surface at the three caucuses will be addressed at Saturday morning’s ‘21st Century Town Meeting®’. Using the realtime radiofrequency (RF) enabled keypad survey Electronic Voting System (EVS) technology, the topics will be consolidated, discussed and voted upon by the 5,000 thousand NPAC conferees. Results will be projected on large video screens in order to identify and hone an agenda to be ratified by the assembled performing arts community. The aim is to help set the agenda for the performing arts for the coming years: by U.S. communities; by national, state, and local governments; by performing arts organizations’ supporters; and by audiences.

To promote spontaneous interactivity, an audience response system (ARS) or participant response system (PRS) or EVS that equips attendees with radio frequency keypads at their seats to record their real-time responses to multiple-choice questionnaire items, will be used for the first time at NPAC. The EVS system enables the audience to provide immediate information and responses to questions posed by presenters. The responses from the RF handhelds are tabulated by software. Bar-charts of those tabulations are immediately displayed on large projection screens for all to see.

These systems have been in use in medical professional society meetings for more than 10 years, as well as in other industry marketing and market research meetings. The EVS respondents are traceable and identifiable (via the permanent database records that are created by the system when the votes are cast), although their identities are not revealed by the displays.

One realtime effect of the anonymity is to remove respondents’ inhibitions—the opinions and votes tend to be more polarized than those collected by traditional survey methods and, if re-votes are taken, the respondents’ preferences tend to be more resistant to discussion and compromise than usual, because there is no accountability. This phenomenon makes sense in terms of psychology, but it runs ‘counter’ to what meeting organizers typically believe. Organizers typically see the real-time tabulations as wondrous and wholly factual representations of the prevailing opinions. The temptation is to ‘strike while the iron is hot’ and immediately act upon the wonderful statistical information that is so fresh and beautiful, without regarding it more carefully (and examining and tracing the votes to individual specific voters) and dispassionately later on. NPAC says it intends to “ratify” the results during the meeting. In my view, the RF keypad EVS survey technology is a “sword” that definitely has two edges—the capacity to help and the capacity to distort or harm. I hope that CMA and NPAC are mindful of this . . .

Besides special RF handheld EVS devices of the sort NPAC will use, cellphone SMS text-based EVS services have emerged as well, providing:

A short text number to dial (e.g., 41123) ;
Message content that each voter keys in (e.g., ‘CAST 82A SEND’);
Votes are caught, processed, and put on a dynamic web page that can be displayed to the conferees in real-time and embedded in Microsoft PowerPoint™.
Votes from SMS and web ASP/PHP HTML/XML apps can be combined. (Also can download spreadsheet form of the data.)

These might be considered for surveys at future classical music society meetings, including CMA. The cost is, in general, cheaper for cellphone SMS-based surveys than for surveys conducted using the older RF handhelds. A group need only arrange and pay for the service in advance and have a live internet (WWW) connection in the meeting hall. Audience members come with their regular mobile phones, and no other equipment or setup is necessary on the spot. PollEverywhere is one company offering SMS-based EVS services.

There are several issues, though, with SMS-based EVS. To cast each vote, each voter has to make about a dozen key presses, compared to one key press for conventional RF EVS keypads. This is a major issue in terms of ease-of-use and the time required for respondents to complete each survey question or an entire questionnaire.

Another factor is that mobile phone networks were not originally designed to handle hundreds of calls from a single room at once. This is not a fundamental technical barrier, but it rules out some off-the-shelf technologies that would otherwise be suitable. For example, Bluetooth™ is not currently able to handle 300+ respondents in a room. Even cell phone networks are not designed for EVS type loads with large groups of respondents all using their cell phones at once. For audiences of 300+ there is usually not enough cell capacity in any location. Each cell can handle 168 concurrent calls on digital cellular networks, 56 on analog. Concurrency will work differently for SMS texting, so those figures are context-agnostic, but for reliable EVS voting the phones may need to resend the message until they get a positive acknowledgment back from the server—which adds more latency time and causes problems with respondents in terms of ease-of-use (lack thereof), under-counts, and compliance.

In fact, a cell phone system could be deluged by a flood of SMS messages, which, if the EVS volume were big enough, would overwhelm the control channels, and prevent voice service as well as SMS from working in that area/cell. This means that it is likely the service provider will have defenses that would preclude large voting audiences.

Even if adequate coverage could be obtained, SMS messages (unlike voice calls, but like emails) are not guaranteed to be instantaneous or to be delivered within any specific time interval. Texts can be (and, in practice, often are—) delayed, and there are no prospects for an archictecture to allow text delivery for a vote within a guaranteed time (e.g., 2 minutes; vs. 2 seconds for the best regular RF EVS equipment). But not only is “prompt” delivery not guaranteed, it is frequently delayed beyond 2 minutes and drop-outs (uncounted votes) occur.

What’s more, the SMS service charging models are inappropriate. It’s probably impossible to arrange for the voting to be free to respondents/conference attendees, since they all use different providers. Local cells are expensive, and they will not be private but will fall under an existing licence and operator (AT&T, Sprint, T-Mobile, Vodafone, O2, etc.) who has paid for that part of the radio spectrum. Charging respondents to participate is not a good way to get this approach off the ground, even if it wouldn’t matter in a mature model.

EVS systems have a ‘ranking mode’ that is used to obtain first, second, etc., preferences from a list of options—for example, prioritizing the order in which possible activities should be done. These are tallied and displayed immediately, in barcharts and pie charts and other easy-to-understand formats.

Yes, formative, summative, and peer assessments can be made conveniently and quickly with EVS. And building a sense of community could get off to a quick start with EVS, especially in large groups that have factions that are mutually misapprehensive of each other.

But those results should best be considered a ‘beginning’, not an end. The results should not be considered “actionable” or “ratifiable” as-is, in the way that NPAC says it intends to do. Why? Because of the unique psychology of anonymous EVS, which encourages fait accompli ‘American Idol’-style relativism: “I like choice ‘A’ because I like ‘A’.” In other words, voting is the only responsibility of the anonymous EVS respondents, and, by contrast, discussion and explanation would require unmasking of identities and the additional duties of civility and discourse. Anonymity tends to bring out the worst in people, in terms of self-interested fractious behavior.

In fact, in some EVS surveys that I have personally designed and conducted, some respondents were recorded electronically voting for positions that they were known to oppose in reality, and did so expressly so as to be categorized into a particular focus group. Then they inserted themselves into the subsequent discussions as “wolves in sheeps’ clothing” and (a) passively ‘lurked’ in-caucus, to perform reconnaissance on those adversaries of interest to them, or (b) actively intimidated the members of the other camp.

In yet other instances of EVS surveys that I personally supervised in 1998 and 1999, some respondents had hidden motives such that they wished to surreptitiously undermine the legitimacy of the survey process itself. To accomplish their goal to sow seeds of doubt among the broader membership, they voted disingenuously, capriciously, seemingly irrationally—so that the displayed EVS survey results would, to the assembled conferees, appear to be scatological, unusable, unreliable. The broad group would then, they hoped, be thrown into disarray, and the decision-making and policy-setting process would be effectively derailed. Remember, these were EVS surveys of academic medicine faculty members. Don’t kid yourselves! This summer in Denver CMA and other NPAC-member organizations are likely to discover some of these idiosyncrasies of EVS survey psychology and sociology, about ten years after other professional societies learned these lessons.

If you are interested in these topics, please have a look at Aaron and colleagues’ recent book and look also at the whitepaper materials of some of the EVS vendors (links below). EVS is very useful technology, yes. But it can be co-opted, distorted, and manipulated by factions who want to interfere with the goals, privacy, and priorities of other factions——or who are determined to fabricate a “unified voice” when those [dominant; paternalistic] factions earnestly but mistakenly believe that only by so doing can they “ensure a vital future for the performing arts.” This is not a ‘hypothetical’; it is an unfortunate reality, especially when the stakes are high.

jwilzewske

Tuesday, May 6, 2008

National Performing Arts Convention: Electronic Voting Systems (EVS) to Set Priorities

No comments:

Post a Comment