Marc Cohen writes a challenging post on The Myth of Music Discovery. Citing a Digital Music News report of two venture capitalists agreeing that "the next big thing is going to be music discovery", Marc says this ought to be enough evidence that it won't be.

Having written a book which takes music discovery as a pointer to the changes in the forces shaping our cultural lives, you wouldn't expect me to be disinterested, or to be able to avoid rising to this bait. Perhaps that's just more evidence to support Marc's argument. But let me try and engage with his points anyway.

Marc reports evidence that radio remains the main route to music discovery, but online channels are growing in their influence. This is now a fairly well-established trend (here's one previous post supporting this, and another). He concludes from this:

people don't seek to discover new music — it just happens. They don't listen to the radio, watch TV or talk to friends for the purpose of discovering new music. This is a byproduct of the intended object of the interaction.

For some — actually I'd concede it's the majority — this is true. But sweeping generalisations about what "people" do or don't do are not helpful to our understanding here. There is a minority who do seek to discover music. These are the 'savants' and 'enthusiasts' in the classification I use. The thing is that a minority within this minority are quite influential for the rest of the "people". They are the first movers in the interactions from which discovery is a byproduct. And they're proud of it. They take kudos from people reading the blogs in which they assiduously document their new finds, and from the buzz they build on social networks.

The dynamics of discovery include a whole ecology of social recommendations, automated recommender systems, happenstance and serendipity — and the interactions between all of these influences.

[Update, 26 April 2008: Marc has posted on the reaction and I have commented, so see there for further discussion.]

Marc's second conclusion is that (commercial) radio is a kind of ad-supported music. Absolutely, yes. But then I don't entirely follow his inference:

Since downloaded music provides a superior user experience to streaming radio, I will argue that downloaded ad-supported music will be the superior vehicle for music discovery.

The first part of this is debatable. Another time, perhaps. But the second part is curious. If one experience is superior for music discovery, doesn't that mean that there's more to the story than just "discovery (like shit?) happens" of its own accord, independent of the design of services or the intentions of people?

I'm not sure how the business model of music discovery services can be written off as a "myth", as Marc claims. Some models are better than others. I don't think either radio or downloaded ad-supported music are the end of the story. There's a mixture of craft and science involved in making the different parts of the ecology work together. From this we can knit together the possibility for stimulating, social and diverse discovery experiences.

I wrote last year about Swarmteams cross-platform messaging service, and its application for coordinating networks of fans. Swarmteams is running a pilot project for the music industry this year, supported by NESTA, and going under the name of SwarmTribes®.

For many musicians, getting the first 10 or 20 dedicated fans is easy enough — but when it comes to multiplying this number things become more difficult. If and when their fan base does increase, they're faced with the challenges of managing it.

Musicians need a communication system to interact with their fans, which is adaptable and instantly reactive. They need to engage with their fans, using a means of communication that can be scaled up. This is where Swarmteams can help.

I'm pleased to say that I'll be working alongside Swarmteams as researcher, reporter and evaluator for the project (also funded by NESTA, but as an independent project). And I'm looking forward to working with Nancy Baym of University of Kansas and her colleague Ryan Milner.

The core of the Swarmteams concept is the combination of a "back to nature" communication patterns and the latest cross-platform messaging technologies.

Swarmteams founder Ken Thompson has researched biological/ecological perspectives on team organisation and coordination (laid out in his Bioteams book). Then Swarmteams have designed a communications system around this, combining SMS text messaging, email, instant messaging and RSS.

Starting with those 10 or 20 dedicated fans, bands and artists can use the techniques and technology first to build a broader base of fans and then to motivate and coordinate these fans around gigs, releases and special events.

Consider this quote from a talk on the music industry by marketing guru Seth Godin:

I have every record Ricky Lee Jones has ever made including the boot legs that she sells. Ricky Lee Jones should know who I am! (laughter) I have bought many of them (pause) well her agents, her people [should know who I am]. I’ve bought many of them directly from her site. I desperately want Ricky Lee to drop me a note telling me when she is going to be in town. I want her to ask me, "should I do a duets album with Willie Nelson, or should I do one with Bruce Springsteen?". I want to have these interactions. And I want her to say, "I'm making another bootleg, but not until I get 10,000 people to buy it as patrons before I make it". Because I'd sign up. I’d buy five if it would help.

It's this kind of interaction that the swarming platform can enable. And in the Swarmteams language, Seth would be one of Ricky Lee's 'alpha' fans, possibly with a role in passing on messages to other fans he knows, whom he has recruited to a 'swarm'.

Another recent reference point is Kevin Kelly's "Thousand True Fans" conjecture:

A creator, such as an artist, musician, photographer, craftsperson, performer, animator, designer, videomaker, or author — in other words, anyone producing works of art — needs to acquire only 1,000 True Fans to make a living. A True Fan is defined as someone who will purchase anything and everything you produce. They will drive 200 miles to see you sing. They will buy the super deluxe re-issued hi-res box set of your stuff even though they have the low-res version. They have a Google Alert set for your name…

But Kevin Kelly sees all these thousand people as separate individuals, talking of recruiting one fan a day for three years and then, "The key challenge is that you have to maintain direct contact with your 1,000 True Fans". That's not the way Swarmteams approaches the problem. Instead, the swarming model sees the fans as a collective (think as well of Mark Earls' Herd). This means that bands and artists can in theory share the strain of recruiting those 1,000 fans, hopefully do it much quicker, and also maintain a sense of intimate and direct contact without having to manage 1,000 relationships directly.

My job over the next months will be to test how well this theory works in practice. Beyond that, I'm interested in how the model might be adopted in other parts of the music industry ecology, such as independent gig and festival promoters, or unofficial fan communities. My work has its own project wiki.

SwarmTribes will be represented at the Association of Independent Music's Music Connected 2008 event on 28 April. I'll be there, too, recording some interviews and reactions to the concept, hopefully for a future podcast. Please come and say hello. Alternatively, if you're a band, an artist or a manager, you can sign up for the pilot directly.

[Photo credit: Max Westby, licensed under Creative Commons.]

Here's a short story — the fifth in my series of future scenarios from the first draft of my book, which got edited out of the published version — about a casual music listener trying to find some music to go with a home video, and being led through the minefield of finding music that you can use legally on your soundtracks. [I found the picture on the left on Flickr: it's by quasimime, and used under a Creative Commons licence.]

There were several things I was trying to combine in this scenario, aside from the licensing question.

• I wanted to describe the experience of someone who doesn't care a lot about music, and just sees it as a means to an end. Most of us who write about music discovery are pretty fanatical about it ('savants' in the classification I use), and have to remind ourselves that not everyone behaves like us.
• The idea of building up a collection of digital music almost by accident: the download-era equivalent of acquiring lots of promotional CDs and the ones they stick on the covers of magazines.
• Having a search tool that sorts through this slurry of inconsistently tagged music files and returns something reasonably coherent from this Everything is Miscellaneous mess. Of course, the tool could equally well have been searching the miscellaneous grab-bag of music files online, as SeeqPod does, but for this story you perhaps have to imagine that the Englobulators have won and closed down SeeqPod and its siblings.
• Finally I wanted to show search and recommendations for using music instrumentally as an accompaniment for other activities. My hunch is that recommending music for specific purposes — whether as a video soundtrack or for a gym workout — is going to be more effective and more widely used than for the general, and more ambitious, purpose of finding your next favourite band. This doesn't apply only to music: I wish Flickr and iStockPhoto had better search and recommendations to help me find images to accompany presentation ideas.

End of introduction. Continue reading for the story.

Angela shot some video at a family get-together last weekend. Everyone was in good spirits, and the weather was unseasonably pleasant, apart from a thunderstorm late in the afternoon. Angela got some great footage of her three young nieces playing in the garden and swimming. Several family members said they'd like to see some of the video, and Angela promised to edit it down and share it via the web.

Back at home, she has done a rough cut of the material, which comes in at about eight minutes. There's not a lot in the way of dialogue, and there's no need for a voice-over since most of the people who are going to see the video were there when it was shot, but it could do with some music to capture and reflect the halcyon mood of the day. The trouble is that Angela, who will admit to anyone who asks that her musical knowledge is far from encyclopaedic and her taste far from adventurous, isn't quite sure what would fit, or how to find it.

She doesn't really have anything she thinks of as a music collection any more, apart from about 40 or 50 of her favourite old CDs packed away in a cupboard, which she hangs on to for sentimental reasons and as back-up if anything goes wrong with her file copies. So, yes, the digital music files on her computer and iPod could be seen as a collection, but they're more fluid than collections used to be, with a slowly changing core of favourites, and much more rapidly changing periphery of tracks that Angela may have listened to only once, if at all.

How did this come about? Over the years, she subscribed to several 'feeds' to find out about new music. Some of them came from music and lifestyle magazines, some came direct from artists and labels, and some came from bloggers. Many came with free 'sample' tracks designed to promote new releases — the digital equivalent of the cover-mounted CDs that used to come with magazines. In keeping with her casual and slightly fickle approach to music, Angela would read each feed for a while, but then would fall behind when she found an interesting new one. She never cancelled the feeds, though, just in case she ever found time to come back to them. Sometimes she listened to the free tracks, to see if she liked them, but after a while there were too many of them, too. It all became like the bad old days of email, when you would subscribe to an email list and then not be able to unsubscribe, so the messages just piled up in folders, unread, like authorised 'spam'.

Then Angela's brother installed a new application on her computer. It 'scrapes' (as he put it) the free tracks out of the feeds without her even having to open them, and then it puts the tracks in a folder on the computer. At first, Angela feels it's slightly cheeky to scrape the feeds in this way — like tearing the free sample off the cover of a magazine and throwing away the magazine — but the clever bit is what the application does next with the tracks in the folder. Angela can tell the application to provide her with a playlist of a certain length (to accompany her on a long train or bus journey, say), whether she wants it mostly up-tempo or downbeat, what range of moods she wants, and what proportions of familiar and unfamiliar tracks she wants to hear. The application will then search through the folder of scraped tracks, and compile a playlist that combines new and familiar songs in a sequence with just the right transitions of tempo and mood between them. As Angela listens to the playlist on her iPod, she can tag those new tracks she'd like to hear again, which may become the familiar favourites of tomorrow, and those she's unconvinced by. Thus her music 'collection' is forever morphing. Angela still buys music occasionally (as much as she ever has done, which isn't a lot), but she feels this 'free' auditioning process is often a better way for her to discover what to buy than just listening to the radio.

However, when Angela tries using the application to fish for some appropriate music for the video soundtrack, the results are disappointing. She specifies a playlist with ten tracks, all new, with the characteristics 'peaceful', 'bright' and 'energetic', and medium or medium-fast tempo. Two or maybe three of the tracks in the playlist sound OK, but the lyrics don't fit with the images.

So Angela decides to take her challenge to the office, to see if some of her work colleagues can help. When she first got this job, she was intimidated by the way these new co-workers paraded their musical knowledge and wore their favourites like scaffolding for their personalities. She corners Kris at lunchtime, and asks if she could ask his advice on a musical matter. She can tell he feels flattered. She shows him a couple of scenes from the video, and plays him clips from the two tracks she found the previous evening.

"I'm looking for something that feels like these pieces, but has more appropriate lyrics — or no lyrics at all," Angela explains.

"I know a search site that might be able to help us," Kris responds, clicking on one of his bookmarks. "There are several criteria we can use. First let's see if we can find something that's acoustically similar to one of those tracks." He types in the title of the track. "Now, do you want to restrict the search to music that is almost identical, or just pretty similar?"

"Just pretty similar. The mood of that track is in the right ballpark for what I wanted, but it's not necessarily 100% perfect."

"Fine." Kris adjusts a slider very slightly on the screen. "Now, you said the mood you wanted as bright, peaceful and energetic. I'm not sure whether those last two might be in conflict sometimes, but I'll put them all in as tags, and we'll see what tracks other people have described using those terms. I can also put in the same terms for the lyrics: the search engine analyses the mood of the lyrics automatically. Are you bothered about rights?"

"Should I be?" Angela fires back, slightly nonplussed.

"Probably not, but I thought I'd ask. There are some composers and labels that don't like anyone using their material for anything without their permission, whereas others are fine with you using it for free as long as you're not going to go out and make a million bucks with your video and their music. But you're just showing this to family and friends, right? So it's very unlikely anyone would take action against you, though if your family is viewing over the web on one of the main video sharing sites, it's just possible that someone could get wind of it."

"Hell, if that's their attitude, they can keep their precious music! So you can search just for music that I can use for free without risking a 'cease and desist' letter?"

"Sure. You should still put an artist credit in your video titles somewhere, though: that's the deal for not having to pay a licence fee." Having completed all the criteria, Kris runs the search. "OK, now the results are ranked just like any other search — shall we sample them from the top?"

By the time they've heard clips of four of the tracks, Angela thinks she has found what she needs.

"But you may need another track if the video clip is eight minutes, right?" Kris asks. "And you said the final scenes were inside, during the thunderstorm."

"That's right."

"What we can do is take that first track as the 'seed' for a second search, and this time take out 'bright' from the search terms and replace it with 'mellow' or 'cosy'. And I'll make sure it's an instrumental track to be sure it's not out of sync with the first one... Here, now you can hear how your first track would segue into each of these new ones."

After Angela has provided her payment details, she can download the two tracks and incorporate them in her video edit.

We're getting used to the idea that fans create some of the most authoritative accounts of the objects of their fascination. I've cited Andy Aldridge's work as one example of this, and the They Might Be Giants wiki as another. Recently it was reported that fans are more likely to refer to Wikipedia than MySpace as first port of call to find out about a band or artist.

But earlier this month Laura Hale drew my attention to another twist on this, in the shape of a wiki site she oversees, fanhistory.com. If you check the Nine Inch Nails page on this site, you'll get only the briefest history of NIN themselves, but this is followed by a much more detailed historical account of all the fan-led sites, forums, fan fictions and other initiatives that relate to NIN over the last ten years.

I asked Nancy Baym for her take on it, and she told me that this kind of meta-fandom is quite common. Apparently the fanfic people have a highly developed self-reflexive culture that includes a lot of generating databases/analyses as well as a sense of ownership of the term 'fandom'. Nancy herself, I found via this page on Henry Jenkins, is a 'fancademic'… For an introduction to online fan communities, I recommend her account of the Swedish indie fan scene.

Fan History covers not just music, but movies, TV, actors, cartoons, games and sports. The music section is patchy, to say the least, but, as Nancy pointed out to me, these things take a lot of time and work to build. As a small gesture, I've added details of a fan site I created. A couple of days ago, the Fan History site added a blog.

Earlier today I recorded some comments about Songkick — who have just announced some new features and funding — for PM, BBC Radio 4's main afternoon news programme. You can listen to the three-minute broadcast feature, including my edited comments, below.

Songkick could be described as Last.fm for gigs. There are important differences between gigs and recorded music, however. Gigs are one-off events; they aren't available on-demand 24 hours a day. You can't try them out with a 30-second sample to see if you might like them. Crucially, the timing of recommendations can be critical for popular gigs. Previously I gave the example of Bandsintown.com recommending the Led Zeppelin reunion show to me on the same day that it was due to happen — not much use when all tickets had been allocated by a complex registration process months before.

Another feature that has attracted comment is Songkick's measure of the 'buzz' for each band, giving a means of comparing who's on the way up and who's star is on the wane.

It's interesting to track Songkick's combination of MySpace, Amazon and blog-tracking metrics. But how much do we need another chart, another index of popularity? Things will get interesting when music services embrace the full complexity of attention metrics (including, say, Last.fm's count of track plays and BigChampagne's figures for files most frequently shared) and do some serious number crunching that enables them to predict future trends. In particular, when is attention going to convert into revenue, and when is it not. In my book, I compare this to weather forecasting, where heavy duty monitoring of different measures helps predict which clouds are going to bring rain and which won't:

Clouds of Arctic Monkeys P2P activity in the north of England are building up a lot of high-pressure word-of-mouth, which will quickly spread across the country, and could turn into a heavy precipitation of sales. Once these storms clear, and sweep off west across the Atlantic, we’re expecting a return to the seasonal spells of Coldplay, with occasional outbreaks of Madonna in urban areas. Scotland will continue to see a light covering of Snow Patrol.

Back with Songkick, I worry that it could be the ticket touts who will be among the first to adopt Songkick's service, exploiting its potential to identify which tickets will be most scarce and most highly prized. Hopefully Songkick has thought about how to deal with this, so that fans won't suffer.

We'll all benefit if Songkick enhances the experience of live music, and makes the market for tracking and recommending gigs more competitive. As its stands today, the feature of Songkick that's genuinely unique is the BandSense technology for bloggers and other web publishers. This will identify when any band on tour is mentioned on your site, using semantic web techniques, and then insert links to relevant tour information for these bands and keep track of ticket vendor referrals. I hope it works better than Amazon's context links.

No doubt the new round of financing will lead to more features, and possibly some better execution of the existing ones. I'm waiting for some feedback on why Songkick's iTunes plug-in isn't working on my Mac. Also I notice that the Chumbawamba gig I'm going to tomorrow evening isn't listed on Songkick's band profile. It wasn't on Last.fm's site either, but Last.fm allows me to add it myself. If Songkick is going to reach down the long tail of live music, as the radio feature suggested, then the site will have to ensure that it has the potential to capture all gigs, including smaller ones than this.

[Update, 25 April 2008: Songkick has now added a feature for fans to add details of gigs.]

(Also there's something funny that I can't explain about the Songkick rankings. As shown in the graphic above and on this page at the time of writing, Billy Bragg has lower scores than The Beatles and The Rolling Stones on all the measures they cite — yet somehow he has a vastly better Songkick ranking, at 25, than either of them, at 306 and 594 respectively. I thought this might be because Billy is on tour, but in that case the Stones would surely have a higher ranking than the Beatles …)