At the age of 16 Kristin Hersh was knocked off her bike, sustained a double concussion and started hearing music no one else could hear. This music blew in from the great beyond and bent her to its will. So Hersh formed a band, Throwing Muses, and sang songs in a trance, her head swaying, her gaze cast eerily out over the crowd. It was hard to tell whether the music was a response to her demons, a symptom of demons or – worst of all – the demon itself.

For devotees, this only made the Muses seem all the more exciting. The fact that Hersh was wonky, disturbed and in the grip of dark forces was taken as proof of authenticity and exoticism; as central to her appeal as those tremendous riptide vocals or her deft way with a guitar. But the singer saw it differently, and now recoils from those early albums. "My response when people tell me they listen to Throwing Muses is 'why?'," she says. "Whatever would you do that for? It's so ratty. So unappealing, so unattractive. I didn't like being there and I had to be there. Why would you show up?"

In Edinburgh it's cold and rainy, a November day three months early. Hersh is at the book festival to discuss her memoir Paradoxical Undressing. The book, culled from her old diaries, covers a pivotal year in the mid-1980s. In one giddying 12-month spell we see her get lost in music, attempt suicide, land a record deal, bag a diagnosis ofschizophrenia (later downgraded to bipolar disorder), go on and off lithium and give birth to a son. Against the odds, it's a jaunty ride. In recounting the tale, Hersh made the decision to leave out "bad people" and avert her eyes when the going got too rough. She looks back not in anger but "with sweetness".

In the grip of dark forces ... Kristin Hersh, formerly of Thhrowing Muses. Photograph: Murdo Macleod

The songs are held at arm's length, as if they qualify as bad people, too. They arrive out of nowhere and then turn "evil", coloured by her bouts with depression, or warped by the Rhode Island squat where she crashes. They come threaded with references to suicide, dead rabbits, blow jobs and infanticide. I have the sense that she's still unwilling to claim them as her own. "Well, I don't take credit, but I don't take blame either," she explains. "There's a lot of shame attached. I've had people who I love crying because of what those songs made me say." She shrugs. "But basically I didn't have a choice."

From an outside vantage point, her life looks different now. Hersh has just turned 45. In person, she is warm, smart and funny, spilling over with merriment. In the mid-1990s she met and married Billy O'Connell, a former music executive who now serves as her manager. The pair went on to have three sons together to add to the now adult Dylan, whose birth brings Hersh's memoir to a close. She still writes and records and, praise be, her recent work is as rich and enthralling as those poisonous valentines of old. So all's well that ends well. Hersh came through a nightmare, pulled herself together and went on to better things.

Except that when I ask how she finally made peace with the music, the singer gently sets me straight. The fact is, she hasn't. "It's exactly the same," she says. "It's an entity that walks in the room. I now think of it as a gift as well as a curse. But it probably would have been better if I lived without it."

There's a line in the book, spoken by Hersh's stepsister and Throwing Muses bandmate Tanya Donelly. Kristin, she says, doesn't even like music.

Hersh grins. "Yeah, I hate music. Everyone knows that about me. Even my kids hate music. When they're watching a kids' show on TV, as soon as a song comes on, the TV is muted." She reconsiders. "Maybe hate is the wrong word. We can't bear it. The intensity of good music is too much to bear. And bad music is so offensive that that's also too much to bear. I'm in heaven when it's good, but that doesn't happen very often. And anyway, you don't want to be crying over the breakfast table. I don't want that life."

She is wary of the romantic notion of a link between great art and mental illness. Maybe, she concedes, in certain circumstances. But in the end the sums don't add up. "The disease is far more dangerous than the music is valuable."

She mentions her friend, the US singer Vic Chesnutt, who sang songs of love and loss and who died from an overdose two Christmases ago. "The fact that it killed Vic, it's not worth it for me," she says. "I think he'd have been a better man without music. And, even if not, he'd be here. He was more precious to me than he was to himself. And I know that I play that role for people too. My husband has begged me to stop. I've tried and it doesn't work. Vic didn't even want to. I want to."

For a few years she turned her back on songwriting, blotted out the noise and attempted to focus on other things. Except that the conversion didn't take, and the entity would not be denied. Music kicked down the door and stepped in to reclaim its bride. For the time being, it seems, they're stuck with each other.

Does she ever wonder what would have happened if she'd never fallen off that bike? If she'd never heard those sounds in her head, never sung songs like Delicate Cutters or Hate My Way? Once, she says, she longed to go into scientific research, into academia. "Without music I'd probably have gone on to be the scientist I wanted to be," she says. "I'd be like me but without this weird ethereal edge. I mean, I love the Earth. I want to be on the Earth. The fact that we're here, the fact that we're atoms held together: that's beauty and meaning enough. But that extra bump on the head was a bit too intense for me."

A pause. "That's why Billy wants me to stop. Because every time I write a song, or every time I hear a song that's real, it triggers a suicidal urge. And it's not subtle. It says, 'Do the math, this is right, you've got to go.' Every single time. And he thinks it isn't right that a mother of children has to fight a suicidal urge that is that attractive. That combination of beauty and death – it's inappropriate."

Is the urge as strong as it was in her teens? "I think it's a little stronger," she says.

'It's heartbreaking for my kids'

Away from the concerts and book readings, the Hersh family leads a life in motion. They have a habit of moving every year or so, rattling around the US in search of new scenery, fresh terrain. She insists that she has loved everywhere she lives, and that it's enchanting to not know how the lightswitch works. "But it's heartbreaking for kids. My youngest, especially, is not enamoured of it. I said to him, 'We have to keep moving or the adventure will end.' He said, 'Yeah. When will the adventure end?'" She giggles. "So I do try to settle down. But after a few days you notice that the view's not changing."

Interview over, we prepare to adjourn to the room next door. Hersh has agreed to a quick vox-pop with the Guardian's video producer, and it's time for her closeup. I tell her that I'm torn, because I love her music. But if the music is the problem – and assuming she's physically able to let it go – then she's probably done enough. Hersh nods and says that's what Billy says, too: that she's done enough and should quit while she's ahead. Halfway to the camera, apropos of nothing, she turns back with a question. "But what if we die?" she says. "What if we die and there's music everywhere?" And she laughs at what a great cosmic joke that would be.