jwilzewske: MusicXML

Showing posts with label MusicXML. Show all posts

Tuesday, April 19, 2011

Save The Music!: Writing Refactorable Compositions and ‘Refactor’ Methods

I n my previous CMT post, I confided a genuine ‘worry’ about the perishability of computer and electroacoustic compositions. And people laugh at this.

N otwithstanding the half-jesting comment above that delights in the fact that computer/electroacoustic works may become unperformable due to technology change, and notwithstanding that some musical works are written deliberately to be topical/of-a-time/perishable. They do not aspire to 200-year or 500-year relevance. Further notwithstanding that some composers’ goal is such prolific runtime indirection and aleatorics that there is no possibility of ‘rehearsals’ in the conventional sense, only performance—and ad hoc ‘thrownness’. Notwithstanding the fact that ‘you-shoulda-been-there!’ installation art and ‘one-time happenings’ may be a legitimate aim for some.

N otwithstanding these things, it is sad that works’ performance life ends needlessly, prematurely. Sad when all that is left of them is a recording or two, and no one will ever again hear them performed ‘live’, interpreted by different performers and realized in different settings than the ones on the recordings. Sad unless the expressive intent is to make an object-lesson of senseless carnage, waste, loss.

S orry, but a major facet of the hideous perishability of present-day computer music has to do with the fact that most musicians and composers do not have—nor do they have much desire to acquire—adequate expertise in engineering and fluency in refactoring code. On the one hand they will spend decades mastering multiple instruments and acquiring skill in orchestration, but on the other hand they will not spend a moment acquiring or maintaining skills in engineering that are needed to support preserving their achievements as computer musicians and composers. Boggles my mind.

S o what kinds of software tools and environments, if any, might be adequate for composer/users who lack technical knowledge in programming and software engineering, to enable their works to be performable ‘live’ 20 or more years from now, on different hardware and software than they were composed on? Or, given that computer and electroacoustic compositions involve multiple, complex, heterogeneous embedded (pervasive, ubiquitous) and parallel computing systems, is it even realistic to think that a ready-made end-user integrated development environment (IDE) is even feasible to create—one that would effectively immunize such musicians and their compositions against the ravages of time and technology obsolescence?

M y answer, as one who has lived for 25+ years as a professional software developer in a U.S. health informatics firm whose current-generation of applications comprise more than 37 million lines of sourcecode, is “I do not think so.” I do not think that any tool can comprehensively automate the porting and refactoring, now or ever.

I nstead, I believe that composers who want their works to remain performed and performable will either (a) have to acquire some skills of a code-refactoring engineer and suck it up and spend a modest part of their creative hours doing actual refactoring work, or (b) outsource those refactoring tasks to service contractors who do have those skills.

I n the (b) case, the composer needs to be knowledgeable enough about refactoring to be able to perform quality-assurance checks, to see that the contractor has done the job right, and to be sure that the new version does accurately reproduce the aesthetic choices of the original. Regression testing!

T he “Oh, yes, it [obsolescence, no refactoring, perishing] is wonderful!” quip on the iIRCAM twitter feed (jpeg above) [backhandedly?] feeds a kind of ego-centrism and sensation-mongering. I would love to think it’s harmless snark, a carefree “Here today, gone tomorrow, fine by me!” answer. Probably, that’s all that it is.

B ut the reality is, there’s a lot of remarkable, beautiful music—the result of countless hours of effort and musicianly skill—that today is being allowed to go headlong toward oblivion, all because nobody cares enough to engineer it in such a way as to prevent that. The book links below provide some useful sources that can guide you—that can enable you to learn how to do refactoring of your own compositions to future performance platforms, or to hire somebody else to do this.

Y es, some composers, by their own choosing, will simply decline to preserve their work, or preserve it only temporarily. Sort of like Damian Hirst and his infamous shark in formaldehyde or rotting copulating cow and bull. “Posterity? Not my problem! I won’t be alive then!” That’s fine. Being a Damian Hirst is not a disgrace. Transience is even an enviable fate. I like the idea of cremation, for example. I understand the wish to become atoms, dust.

B ut if you don’t mean to be a Damian Hirst, though, then implementing your version-to-version ‘uplifts’ changes without having to manually rewrite each and every data structure and interface design requires support by another goal: Clean separation and ‘pluggable’ pattern-based interfacing. If one has a synthesis algorithm that plays a particular soundfile or patch in a certain way and with certain timing and through certain channels, then a refactoring should allow that soundfile or patch to be accurately and consistently rendered in the new configuration, through the same or equivalent channels and with the same or indistinguishably different timing and processing, without having to manually reverse-engineer and manually specify each and every detail on the new config.

A ll of this requires an ontology for mapping synonymy of synths and waveform samples and libraries and channels/layers (and modules fan-ins and fan-outs topology, and inter-platform clock timedivision-resultion, and inter-platform sampling-freq and bit-depth skew, and EQ and reverb and other processing idioms, frequencies and pitch-bends, vocoder and compander FX, attack/decay envelopes, and timing, etc.) on different configs…

M ight be done in OWL/RDF or other environments, but such ontology does not yet exist so far as I am aware. Current-generation mapping paradigms range from 2D graphical ‘wire-patching’ of Max/MSP and Pd, to primitive source-code ‘text-based patching’ as in SuperCollider or ChucK, to WorldofWarcraft (and others’) OpenGLES patching of urMus elements, to Marsyas’s Qt4-based patching.

A s things are right now, you do in-line modding of your sourcecode, encapsulating the syntactic result of your work, but not the behaviors of your work. By contrast, true refactorings capture the behaviors and port those behaviors correctly to the new platform(s), not just the syntactic synonymy.

M any refactors that I do in Java/Eclipse are manual, really tedious. There is a refactor to convert an anonymous datatype to nested, and then nested to top-level, but there is nothing to convert a top-level type to anonymous (or nested). Lots of cut-and-paste required. Ultimately, you write your own source-modifying refactors. Chaining other refactors together into a script is only a start.

T ake Max/MSP, for example. Max is a data-flow procedural programming language in which programs are called “patches” and constructed by connecting “objects” within a “patcher”, the 2D Max/MSP GUI IDE. These objects are dynamically-linked libraries, each of which may receive input (through one or more “inlets”), generate output (through “outlets”), or both. Objects pass messages from their outlets to the inlets of connected objects.

M ax/MSP supports six basic atomic data types that can be transmitted as messages from object to object: int, float, list, symbol, bang, and signal (for MSP audio connections). A number of more complex data structures exist within the program for handling numeric arrays (table data), hash tables (coll data), and XML information (pattr data). An MSP data structure (buffer~) can hold digital audio information within main memory. In addition, the Jitter package adds a scalable, multi-dimensional data structure for handling large sets of numbers for storing video and other datasets (matrix data). Max/MSP is object-oriented and involves libraries of objects that are linked and scheduled and dispatched by a patcher executive. Most objects are non-graphical, consisting only of an object’s name and a number of arguments/attributes (in essence class properties) typed into an object box. Other objects are graphical, including sliders, number boxes, dials, table editors, pull-down menus, buttons, and other objects for running the program interactively. Max/MSP/Jitter comes with hundreds of these objects in the standard package; extensions to the program are written by composers and third-party developers as Max patchers (e.g., by encapsulating some of the functionality of a patcher into a sub-program that is itself a Max patch) or as objects written in C, C++, Java, or JavaScript. And those, too, require refactoring from time to time, just like your compositions do.

Refactored DSP Pd-Max/MSP code, PullUpMembers

R efactoring can change program behavior. For example, a ‘PullUpMembers’ refactoring changes this Max DSP method that previously was encapsulated in and extends a class ... pulls it up into the class that was previously its parent: that is the example that the A-to-B refactoring above illustrates.

B ut PullUpMembers refactoring causes concurrency bugs when it mishandles the ‘synchronized’ method. When that happens, methods in the parent and child can be time-interleaved in arbitrary ways by the operating system, different from what was the case in the original composition. It messes up the sound. You don’t want that. You need to take special engineerly steps to prevent it. You can’t just ‘plug-and-play’.

M oveMethod refactor methods can cause deadlock and starvation, with mutex and lock conflicts. Other refactoring can give rise to other sorts of bugs. The up-shots of this are that (a) creating an automated-unsupervised refactoring engine for any of the current-generation computer composition IDEs and the scripts/patches they emit would be incredibly difficult and nobody has yet done it [because there’s not enough money in it to make a viable business-case], and (b) doing it auto/semi-supervised or, worse, manually is a tough slog, mitigated only by some considerable skill and fluency in the tools and coding, hence, my remark above that it ain’t gonna happen unless you become a decent engineer as well as musician.

A ny program feature without an automated test simply doesn’t exist.”
— Kent Beck, Extreme Programming Explained, p. 57.

T he video game Half-Life 2 (HL2) uses realtime sound event calls to Pd as its sound OS. Game events are sent from HL2 to the Open Sound Control (OSC) engine, which triggers the sound by Pd via the network. Pd has the sample data and the specified sound-behaviors in-memory and can be instructed to modify those in realtime (perhaps by other game events and other processing of those) without recompiling. WorldofWarcraft uses Lua and Max/MSP.

T hink about it: if composers who are writing and maintaining video game scores across multiple years’ releases and across various gaming platforms do refactorings of their compositions out of economic necessity, why not you? Is your music any less deserving of future live performances than video game music?

W ould IRCAM or conservatories with computer music curricula please add some courseware by engineers who are knowledgeable about refactoring? Save the Music!

T he secret to creativity is knowing how to hide your sources [and efficiently refactor them in perpetuum].”
— Albert Einstein, amateur violinist.

Sunday, March 15, 2009

Tabula recompositoria: Amending/Improving Compositions Resembles Software ‘Refactoring’

W hile composers were [in earlier Centuries] certainly able to work out music in their minds without recourse to writing, at some point the music had to assume a graphic form so that it could be preserved, transmitted, and performed. The evidence suggests that composers employed two different kinds of surfaces for writing down their music: (1) an ‘erasable tablet’ such as a slate that could be used many times and (2) paper. In the period [1450-1600 C.E.] we are considering, the pencil had not yet been invented, so writing on paper meant using pen and ink. Erasure could only be achieved by scraping the ink from the surface of the paper, often resulting in holes in the paper. Paper was therefore a ‘write-once’ medium. Despite the differences in the properties of the two kinds of writing surfaces, composers seem to have used them both in much the same way for all the written stages of a composition, from the earliest sketches to the final version.”
— Jessie Ann Owens, p. 74.

[20-sec clip, Umesh Shankar et al, UC Berkeley, 2003, Mozart, K. 581, Mvt. 3, ‘Trio II’, 0.4MB MP3]

M usical sourcecode (the musical score for a composition) is not sacrosanct: we honor the text in our reading of it, but we also take advantage of the interpretive latitude that it affords.

A nd any composer/arranger performs edits upon edits, to clean-up and perfect a work over a period of days to years. Or you create new versions to suit new and different purposes. It’s matter of what software developers would call ‘emergent design’.

Y ou may want to generate a piano reduction, exploding and imploding sectional structures, changing durations and meters, moving layers, manage instrument doublings, etc. .... Johnson’s book (link at bottom of the CMT post) has a good synopsis of these use-cases for MusicXML.

A lternatively, you may want to programmatically change your orchestration and timbres/sonorities, simplify or obfuscate/complexify the rhythms and horizontal relationships, de-risk the voice-leading to make it more natural to perform or sing, change the key to address the realities with regard to black-key biomechanics on keyboards or string-change biomechanics on stringed instruments. More and more use-cases for arrangers and composers!

S uzanne Clercx was the first to document the use of ‘erasable tablets’ by composers when in 1953 she published the discovery of a slate with six staves on each side [termed a ‘tabula compositoria’, ‘cartella’, ‘palierten schiffer stein’, ‘palimpsestus compositorius’, or ‘ardoyse’, depending on the country and century]. MusicXML is simply a latter-day tabula compositoria, that’s all.

W orking as I do on software in my day-job, I am impressed that compositions are homeomorphic to today’s object-oriented software and today’s data networks, including so-called ‘cloud’ computing or ‘software-as-a-service’ (‘SaaS’). All can be complex and difficult to manage.

I suggest that the root of these problems lies in the complexity of the control and management planes—the orchestration and voice-leading and quasi-protocols coordinating the musical parts; the software and protocols coordinating the objects and network and distributed-services elements—and particularly the way the decision logic and the distributed-systems issues are inexorably intertwined.

I am impressed that ‘refactoring’ the functionality can improve a composition just as it can improve a piece of software. There is no great profundity or earth-shattering novelty in such a view. It is just something that naturally occurs to any person who spends her/his days working on both software and music.

I propose three key principles:

network-level objectives,
network-wide views, and
direct control

that I believe should underlie any [new/refactored] architecture, be it an object-oriented software system or a musical composition.

F ollowing these principles, I identify a design pattern that I call ‘4D’, after the architecture’s four planes: decision, dissemination, discovery, and data. A 4D architecture completely separates a network’s/composition’s decision logic from the compositional protocols and performance-practice/runtime protocols that govern the interactions among the network/musician elements.

I n creating a ‘4D’ design pattern, it’s important to remember that MusicXML score files do not represent ‘presentation-level’ primitives such as pages and staff-systems ... such details of formatting will change based on different paper and display sizes. But MusicXML does represent syntactic and semantic concepts of voices (parts) and synchronization among them, segmented into ‘movements’. In the MusicXML environment, in other words, formatting is handled separately from structure and semantics. The same applies for detailed interpretive performance information. Separate MusicXML supersets could be developed to represent individual printings and performances.

R efactoring of MusicXML can, of course, be done off-line—with generation of printed parts and score ahead of performance runtime. My MusicXML source for K.581 Mvt.3 Trio II is here. This (and several Beethoven string quartets) is what I have been testing my prototype MusicXML refactoring engine with this week.

O r the refactoring of MusicXML can be done in realtime—with refactored parts rendered with streaming MusicXML interpreter to be sight-read by the performers, with (I suppose) considerable surprise for artists who are deeply familiar with the original texts—and for listeners as well.

T he realtime MusicXML refactoring works okay so long as the lead-time for the canonical refactoring service is at least one measure ahead of where the musicians are playing. That, to me, seems to be about the minimum for proper scansion and so that phrasing and other effects are not too disrupted.

O f course, this depends on the meter and tempo and the prevailing complexity—things that are making cognitive demands on each performer. If a streaming realtime refactoring is too elaborate, then the aesthetics (and the fun) will undoubtedly suffer.

B eyond this, in principle I suppose one could mike each member of the ensemble and do analog-to-digital conversion of each signal—perform digital signal processing (DSP) and pattern-recognition to impute pitch and interval and rhythm and dynamics deviations from the score—and then use those imputed deviations to drive a refactoring server, to further permute the parts or effect meta-alteration of the downstream score. That’s beyond anything that I’m prepared to attempt right now, but CMT readers with enough studio gear and computational resources may be interested to try it.

K. 581 Clarinet Quintet (‘Stadler’ Quintet) [1789]

Allegro (A major)
Larghetto (D major)
Menuetto (A major), Trio I (A minor), Trio II (A major)
Allegretto con variazioni (A major)

T he clarinet predominates in K.581 as ‘primus inter pares’ (‘first amongst equals’) as Alfred Einstein once wrote, yet the parts’ roles are more discursive and autonomous than they would be if this were a ‘concertante’ form where the soloist rules.

M y simple experiments initially have dwelt on Mozart’s K. 581, primarily because it is straightforward and familiar. Its interpretive history and performance practice are pretty transparent and well-understood. I wondered, “What if we refactored the Trio II, to render it as an exuberant/obstinate ‘recovery’ from the A-minor Trio I of the 3rd movement?” Okay. Here’s what I get with my canonical XML parser—scanning for and phrase and repeat boundaries in the tokenized MusicXML source.

becomes

B asically, my “light” refactoring of K.581 treats every ‘repeat’ structure as a MusicXML ‘class’. You play it through the first time as originally written—that’s the first ‘invocation’ or programmatic ‘call’ to the ‘repeat’ class method with the contents of the bars between the repeat-delimiters as an argument. When you get to the end of the first time through, the method in the ‘main’ class (call it ‘A’) bumps an ‘excursion counter’ and calls the repeat class (call it ‘B’)—with a canonical refactoring argument to dynamically alter the articulation syntax in each part, according to rules I created for my refactoring engine to use.

B ut the simple as-written MusicXML source then involves a ‘cycle’ between the ‘main’ class ‘A’ and the ‘repeat’ class ‘B’. If this were a software refactoring, generally we try to eradicate cycles, to localize and manage features and dependencies and to improve reliability and for other reasons. How about doing the same thing in MusicXML, with similar justification? Sure...

DSM – removing cycles with dependency-inversion DIP

W hat my refactoring of the score does is this:

Extract a ‘repeat interface’ BI class from class B. BI contains the methods of class B that are needed by A, to implement the articulations (or other canonical changes that propagate through the quintet’s 5 parts) that we want to be associated with the repeats. The new, refactored class B implements BI.
Set all references in class A that aren’t required for object generation from B to BI.

A simple ‘toy’ example, to be sure, but it is non-trivial—non-trivial in terms of the parser and its embedded logic, and non-trivial in terms of the expressive effects that this has.

I n the end, what we get in this example is a canonical cascading propagation of accents, tenutos, and decrescendos—explicitly amending the expressive sense of the K.581 Trio II. Cool. It works. And it suggests how we could implement other, fancier refactorings and canonical edits.

Mozart K.581 Mvt.3 Trio II, mm. 1-12 repeat, after canonical XML mark-up with accents, tenutos, diminuendos

M y particular example refactoring of the MusicXML sourcecode for K.581 Trio II is directly analogous to what in the Roock-Lippert software refactoring book is called ‘dependency inversion’ or DIP (see. pp.130-43). Again, I was only exploring here what can be done with current-version MusicXML and refactoring techniques operating on music scores; I am not saying that anything wonderful has been done to K.581 with this attempt. I am only saying that it is interesting, and that it works, and that it may be of interest to you as a composer/arranger or as a music theorist. For CMT readers who are performing artists and avid users of PC-based digital music readers (like Hugh Sung or Nicholas Kitchen) maybe this stuff has interest as well.

T o me, the effort so far seems worthwhile. Yes, MusicXML is not ideal. It’s not a ‘causal encoding format’ and therefore and commands are [must be] used to encode single parts with multiple staves or multiple voices—and so on. While not ideal, MusicXML is the reigning standard that has market-share; it is the ‘bandwagon’ to jump on.

I t’s possible that these refactoring explorations will lead to new MusicXML enhancement-requests or new requirements-specifications for future versions of MusicXML. I’d welcome dialogue with you if you have observations or concerns in that regard. Please email me or comment here if you like. Thanks!

B rian Foote suggested the name [‘speculative generality’] for a smell to which we are very sensitive. You get this smell when people say, ‘Oh, I think we need the ability to do this kind of thing someday’ and consequently specify all sorts of ‘hooks’ and special-case requirements—to handle things that aren’t in fact required. The result is often harder to understand and maintain. If all this machinery were being used, it would be worth it. But if it isn’t being used, it isn’t [worth it]. The extra machinery [to provide abstract generality] just gets in the way [of clean, effective, testable, maintainable source-code and non-defective solutions] ... Any structure or class that isn’t doing enough to pay for itself should be eliminated ...”
— Martin Fowler, p. 83.

A ndrew Monk and others at University of York in the U.K. It is useful to think about software usability problems as forming a hierarchy with each successive level in the hierarchy requiring more contextualized knowledge. By contextualized knowledge we mean knowledge about a particular user and the goals and priorities which are relevant when the problems are identified. An example of a problem at the bottom of the hierarchy is a user having difficulty in correcting typographical errors because of inadequate delete facilities. This may be a quite general problem independent of the particular user and the task being performed. An example of a problem near the top of the hierarchy is the difficulty that a user might have in understanding how to carry out a database search task on some new system. The problem arises from the user's experience with the system, with other systems and with the task it is being used for and so depends very much on the context. Such problems often have a large impact on usability. By definition, these problems require information about the user's knowledge and beliefs in order to diagnose their cause, and to recommend the changes needed. Work at University of York has attempted to develop methods which elicit increasing amounts of context-sensitive data which can be used to diagnose problems at successively higher levels of this hierarchy and have identified four diagnostic levels. At the first level are problems that can be identified from system logs recorded from unknown users during free use. An approach was developed which identifies problems by isolating redundant user input. These problems can be diagnosed with minimal user-specific or task-specific information. A second level involved the analysis of log data from users completing tasks which have been set by the evaluator of the system. When these tasks are well constrained the system evaluator can use them to gain partial information about the users’ plans and goals when problems occur. This method is not always adequate since strategic differences amongst subjects can make plans and goals difficult to infer. A third level used verbal protocol methods to elicit users’ plans and goals directly. Two methods have been used re-enactment. Re-enactment proved particularly useful for diagnosing problems that could not be diagnosed from log data alone. In this method the user and evaluator discuss problems that the user had experienced during previous sessions.

Techniques that allow for more abstraction
Encapsulate Field - force code to access the field with getter and setter methods
Generalize Type - create more general types to allow for more code sharing
Replace type-checking code with State/Strategy
Replace conditional with polymorphism
Techniques for breaking code apart into more logical pieces
Extract Method, to turn part of a larger method into a new method. By breaking down code in smaller pieces, it is more easily understandable. This is also applicable to functions
Extract Class moves part of the code from an existing class into a new class
Techniques for improving names and location of code
Move Method or Move Field - move to a more appropriate Class or source file
Rename Method or Rename Field - changing the name into a new one that better reveals its purpose
Pull Up - in object-oriented programming (OOP), move to a superclass
Push Down - in OOP, move to a subclass
Shotgun surgery - in OOP, when a single change affects many classes—all affected code is kept in a single class, so that the change is made only in one place.”

Saturday, January 19, 2008

The Rest is Not ‘Noise’: Can Concerts with Performers’ Explications between Works Be ‘Mashups’?

Georges Braque, Violin & Candle Mashup, 1910

W o Menschen singen lass dich ruhig nieder.
Denn böse Menschen signen keine Lieder.
[Where people are singing, rest easy.
Because bad people sing no songs.]”
— German proverb/rhyme.

The ‘unity’ of a concert experience—its musical components and its narrative components—occasioned a remark by Allan Kozinn in the New York Times last week, in his review of a Lark Chamber Artists concert. The concert was comprised of a diverse array of [excerpts of] works, ranging from Brahms’s Quintet Op. 111, Jennifer Higdon’s ‘Soliloquy’, John Adams’s ‘Book of Alleged Dances’, and Giovanni Sollima’s ‘Waves’. Allan wrote, “That’s right, just the Allegro [movement of the Brahms Quintet]. Several works were represented by excerpted movements. Generally, this is a bad idea, unless you can argue that individual movements from diverse works add up to a kind of classical mashup. But that’s an assertion you can’t make if, as in this case, each work is given a spoken introduction.”

[In popular multimedia/web-software vernacular, a ‘mashup’ is a ‘composite’ mix of elements created and edited from more than one source, so as to appear as one thing.]

Why would Allan say those things? What authority ever defined once and forever what mashups are or could be? (If anything, the term ‘mashup’ deeply resists any crisp or immutable definition because the inherently ‘populist’ nature of the mashup idiom defies elites and top-down authority, top-down ‘definitions’.)

And, anyhow, presenters’ ‘spoken introduction’ elements are regularly written or extemporized with great poetical and editorial care. Presenters’ remarks are ‘composed’ as programming elements in their own right, to complement and inform the music. Presenters should not be backhandedly demeaned as mere shills/impresarios/educators making ‘noise’, as Kozinn’s comment appears to do!

What’s more, a presenter’s remarks need not be didactic, in the way that Kozinn’s comment implicitly presumes. The presenter may say things that are instead ironical or provocative or enigmatic—for the express purpose of adding drama or stimulation or poignancy to the concert experience.

Or the presenter (or the author of Program Notes, or others) may construct the narrative so as to manifest certain ‘illocutionary’ expressive acts, ones that complement the meanings and modulate the meanings of the musical elements in the conglomerated ‘mashup’. ‘Illocutionary’ is a fancy word that means indirect effects that go beyond the literal text. But it also means that the indirect effects or implications may be multi-layered or different from the superficial or literal sense of the expressions. In English, for example, the interrogative mood is supposed to indicate that the utterance is intended as a question; the directive mood indicates that the utterance is intended as a ‘directive’ illocutionary act (an order, a request, etc.); the words ‘I promise’ are supposed to indicate that the utterance is intended as a commitment.

R ecall that, after Schubert’s death, his brother cut some of Schubert’s scores into small pieces and gave each piece, consisting of a few bars, to Schubert’s favorite pupils. And this act, as a sign of deep piety, is just as understandable as the different act of keeping the scores untouched, accessible to no-one. And if Schubert’s brother had burned the scores, that, too, would have been understandable as a different act of piety.”
— Ludwig Wittgenstein.

Possible illocutionary gestures in English include word-order, stress, intonation contour, punctuation, the mood of the verb, and ‘performative’ verbs. And if I say ‘I promise’ in with an upward pitch inflection at the end (as though asking a question about promising) or with a sarcastic tone, then my expression may not be a commitment at all; it may be a repudiation of any commitment. ‘Illocutionary act’ is a technical term introduced in 1962 by John L. Austin in philosophy-of-mind and linguistics investigations concerning what he called ‘performative’ and ‘constative utterances’. [Of course, ‘utterances’ can be music or text or imagery, not just speech.]

According to Austin’s ‘How to Do Things With Words’, an illocutionary act is an act (1) for the performance of which I make it clear to some other person that the act is or has been or soon will be or should be performed, and (2) the performance of which involves ‘consequences’ like rights, commitments, interpretations, or obligations. For example, in order to successfully ‘perform’ a promise I must make clear to my audience that the promise occurs, and then undertake the obligation to do the promised thing (or induce somebody else to do it). It’s a compound expression. It’s a type of mashup.

Austin introduced the concept of ‘illocutionary [speech] act’ by means of a contrast with other kinds of acts: the illocutionary act, he said, is an act performed in the course of saying something else, as contrasted with a ‘locutionary act’ (the act of saying some literal thing), and also contrasted with a ‘perlocutionary act’ (an act performed by saying something).

Kozinn’s glib ‘mashup’ comment, though, inadvertently relegates presenters’ narratives and the Program Notes authors’ texts to the dust bin—as something to be ignored, not part of the concert experience-proper—if we follow the logic of his remark to its conclusion.

Backhandedly, Allan’s attitude and comment also exalt a kind of traditional and deliberate ‘monumentality’ in classical music programming—a sort of ‘unitary executive’ politics, somewhat like Bruckner’s politics. “There is a pecking-order, and I shall tell you what that pecking-order is, and you shall passively receive it and like it and abide by it.” It is a composer pecking-order and a tradition-bound, conventionality-obsessed pecking-order.

I’m sure that Allan means well, but his comment also seems broadly defamatory and hostile to the whole idea of “individual movements from diverse works, adding up to a kind of classical mashup”—as if the only legitimate way, the only aesthetically and morally acceptable way, to perform a composer’s work is in its entirety. You must perform all of its movements, with all of the movements in the original sequence, and with no apocryphal or extraneous material appended or inserted or layered on top of it, palimpsest-style. Well, damn. I guess all of us electroacoustic folks should go shrivel up and die then. I guess all of us who like to hear chamber music excerpts played in bars should roll over and die.

No. I won’t curl up and die. I like my eclectic tastes, and I bet you do, too. I like to create peculiar playlists on my iPod—sequences that no sane presenter or chamber music ensemble would ever put on a program together. I like the quirkiness of my Last.fm dynamic impromptu mashups. I like to explore different combinations and emotional paths that were never envisioned by the composers whose pieces I put back-to-back. I like to attend concerts by expansive-minded artists and presenters who push the envelope, chart new waters.

Don’t get me wrong: the composer’s intentions do have moral standing and should receive due respect; misusing/abusing a work is never a good idea. But I do favor a ‘multiple drafts’ model of musical interpretation and meaning production, like Steven Jan advocates in his book, ‘Memetics of Music’. The individual (performer; audience member; composer; presenter) is regarded as the sum of her/his genotype and its phenotypic effects, together with her/his ‘memotypes’ and their ‘phemotypic’ effects. Each individual is endowed with inalienable musicality, albeit of different types and in different amounts. Each of us brings to our musical experiences her/his genetic and memetic heritage, conditioning, and intentions. Instead of the composer’s musicality and genetic and musical memetic ‘fitness’ always trumping all the rest of us, the ‘multiple drafts’ model contends that musical performances inevitably shape themselves—through interactions with the intentionality and musicality that each of the rest of us possesses. Mashups empower the rest of us, not just the composers, presenters, and code-heads.

P opfly is a mashup tool for end-users, not for code-heads.”
— Steve Ballmer, Microsoft.

The idea of the composers and artists and audience members as vessels or interconnected conduits for the flow and production of individualized meaning is a centuries-old theme in music history. Indeed, Mozart’s rants against the Archbishop of Salzburg’s provincialism include implications that composers are memetic ‘vessels’, whose expressive authority and merit depend not only on talent but on what memes fill the vessels.

A fellow of mediocre talent will remain a mediocrity, whether he travels or not; but one of superior talent, which without impiety I cannot deny that I possess, will go to seed if he always remains in the same province.”
— E. Anderson, tr. Letters of Mozart and his Family. 3e. Macmillan, 1985. P. 612, letter 331.

We have web mashups and music mashups. The former combine data from more than one source into a single integrated app. An example is cartographic data from Google Maps plus concert data from IONARTS blog, thereby creating a new web service that was not originally provided by either source.

Mashups generally combine content-elements from multiple sources and hide the disparate sourcing behind a unified user interface that obscures the bricolage. ‘Mashups’ and ‘portals’ are both content-aggregation technologies. Portals are older technology designed as an extension to traditional dynamic web applications, in which the process of converting data content into marked-up web pages is split into two phases: generation of markup ‘fragments’. and software-based automatic composition of the fragments into ‘pages’ to be served up to the client app. Each of these markup fragments is generated by a ‘portlet’, and the portal app synthesizes them into a single web page. Portlets may be hosted locally on the portal server or remotely on another server.

Portal technology is server-side, presentation-tier aggregation. It can’t be used to drive higher-level app integration or handle illocutionary interactive user gestures, two-phase transaction commits and roll-backs, or messaging-based redirection of the app-based conversation or search. Portal technology is heavy-handed composer/presenter-dominated politics.

Mashups are Web 2.0 composer/presenter/performer/audience politics. Maybe understanding and accepting and programming mashups is a ‘generational’ thing, an ‘age’ thing, a ‘class’ thing. I hope not. Mashups differ from portals in the following respects:

	Portal	Mashup
Classification	Older technology, extension to web services model; data models coerce semantics	Newer Web 2.0 distributed agent/broker/swarm technology; RDF/OWL data models do not impose top-down ontology and semantics
Compositional Method and Form	Approaches aggregation by splitting role of web services into two phases (markup generation and aggregation of markup fragments), privileging the role of programmer/composer	Adopts an egalitarian approach to content aggregation without regard to markup or authorship
Content Dependencies	Aggregates presentation-oriented markup fragments (HTML, WML, XML, etc.)	Can operate on pure XML content and also on presentation-oriented content (e.g., MusicXML)
Location Dependencies	Content aggregation takes place on the server	Content aggregation can take place either on the server or on the client
Aggregation Style	‘Salad Bar’ style (aggregated content is presented ‘side-by-side’ without overlaps)	‘Melting Pot’ style (individual content may be combined in arbitrary ways, resulting in flexible hybrid content)
Event Model	‘Read’ and ‘update’ event models are defined through a specific portlet API	CRUD operations are based on REST architectural principles, but no formal API exists
Relevant Standards	Portlet behavior is governed by standards JSR 168, JSR 286 and WSRP, although portal page layout and portal functionality are undefined and vendor-specific	Base standard is XML Data Interchange. RSS and Atom are commonly used. More specific mashup standards are expected to emerge.

Incidentally, MusicXML does not yet have an explicit ‘mashup’ ontology and syntax and semantics spec. (Maybe it needs one, preferably as an RDF/OWL ontology, not just low-level XML!) MusicXML 2.0 does have an implicit way to compose a mashup. Currently, MusicXML™ has standard specs to represent—Partwise DTD, where the partwise.dtd file represents a score by part/instrument/voice, and Timewise DTD where timewise.dtd represents a score by time/measure. The partwise.dtd contains measures within each part, while timewise.dtd contains parts within each measure. XSLT stylesheets are provided to couple the two DTDs.

The partwise and timewise score DTDs represent a single movement of music or track. Multiple movements or other musical collections are presented using an Opus DTD opus.dtd file. An opus.dtd MusicXML file contains XLinks to the individual scores, and, by default, they are serial. But there is nothing currently in the MusicXML spec that forbids Timewise DTD from coupling two or more scores, instead of two or more parts. Or, alternatively, you could just instantiate two or more scores in a merged partwise.dtd to create a synthetic mashup with the excerpts and sequences and synchronization you want.

This post has been a very long-winded way of saying, Sing! Play! Devise exotic musical mashups, in whatever ways suit you! Rest easy! People are doing music! (Even us people who use MusicXML for more than just writing and publishing music with Finale™ and Sibelius™ and LilyPond™ software.)

Remember, as they say in Germany, bad people sing no songs. Bad people just don’t make chamber music mashups.

Ken Ueno v. Aaron Jay Kernis composer Flickr Fight Web 2.0 Mashup

L ast night we were finishing off a jam cake and a cheesecake that were in the fridge so my wife and I both had slim slices. I ate my jam cake first, then my cheesecake. My wife ate them alternating bites of one with bites of the other. I guess her approach stopped short of stacking them and mashing them irrevocably together with her fork. She commented on our different methods. And I said, ‘But I like cheesecake, and I like jam cake. And eating them the way you do, I don’t think I get either.’ But there is a flaw in this analogy. In jam cake or cheesecake, a single bite gives us the entire essence of the cake (you’ve got to get some icing on the fork too, though, with the former, and crust for the latter), whereas I’d have to spend time with Bach or Led Zeppelin—to the extent of a movement, in the case of a suite or concerto (arguably) or the whole of the Goldbergs; and, on the other hand, through the entirety of ‘The Rain Song,’ unless one is of the feeling that an entire album constitutes the artwork (which, in Led Zeppelin’s case is not so much the case). So it would depend, in the case of the cakes, on being able mentally to separate the two, and on how quickly one can make that happen. Perhaps my wife can make the shift completely much more quickly than I, and enjoy, savor, each essence completely with a each single bite; perhaps my capacity is such that I need the saturation of a greater amount of cake, given the imminent distraction of the next cake on my plate, or the previous one. Maybe I could be satisfied with the first four bars, or just the theme, of Goldberg, with memory providing the whole of the essence, before moving on to just the guitar solo of the live version of ‘Stairway to Heaven.’ Or maybe you could, but I couldn’t... I’d be sorry to hear just one movement of Brahms in concert, but what if one doesn't know any better? What if one hasn’t trained oneself ... to expect the other movements and not feel the presentation is complete without them? Nineteenth-century listeners would’ve been much less concerned about it, and twenty-first-century listeners who aren’t primarily immersed in this so-called concert music tradition aren’t concerned about it, given the fact that their iPod shuffles through their Minor Threat, Death Cab for Cutie, and Haydn playlists without value/stylistic judgment. [Even though I don’t feel like changing my way of listening], I can acknowledge that mine isn’t the only way to listen.”
— Robert Kirzinger, email, 19-JAN-2008.