This article first appeared in the Guardian. 

The Tories are on a roll, Labour’s lead has halved and Ed Miliband is failing to make himself heard on the big issues. Is the left in trouble or has politics changed? What must the Labour party do next?

Ed Miliband … facing an onerous challenge. Photograph: Murdo MacLeod for the Guardian

A month or so ago, I was discussing the increasingly uncertain outcome of the 2015 general election with a friend whose involvement with the Labour party stretches back more than three decades. “I’m scared of what will happen if we lose,” he said. “And I’m scared of what will happen if we win.”

In case you hadn’t noticed, there are now less than two years to go. For sure, the odds remain somewhat stacked against a Conservative win: thanks to the failure of the party’s bid to redraw constituency boundaries, it will have to be ahead of Labour by around six percentage points even to draw level on seats, the UK Independence party remains a threat to its chances and, in any case, it is now more than 20 years since the Conservative party had a parliamentary majority.

But thanks chiefly to the increasing signs of an economic recovery, it will go into this year’s conference season with that rather unbearable sense of born-to-rule arrogance restored. Who knows, in the wake of another uncertain election result, it may well renew its vows with the Lib Dems, and carry on as normal. Whatever, among its messages on so-called “welfare”, immigration and the rest, it looks likely to have a simple enough pitch to voters, fine-honed by such advisers as Lynton Crosby and its new guy Jim Messina, a former aide to Barack Obama: if the economy is growing again, do you want to hand it back to the people who busted it?

That may not be fair, but this is politics, not badminton. In the three years that have passed since the last election, Labour has never convincingly rebutted the accusation that the last government, as against a global financial crash, caused most of Britain’s problems. And now, the consequences of that political failure may be revealing themselves: in four months, Labour’s average poll lead has more than halved, coming down from 11 percentage points to five.

At the time of writing, Ladbrokes put Labour’s chances of a parliamentary majority at 5/4, with the Tories on 3/1. But even if Labour wins, it seems much more likely to tumble meekly over the finishing line than win in a triumphant blaze of glory. Whatever Labour does, the public seems to have fallen out of love with politicians so comprehensively that the idea of some 1997-esque new dawn seems laughable. Besides, the party leadership is putting a lid on any high-flown expectations. In the event that it wins, austerity, we know, will remain; ask any Labour insider what they might do about such totems of Tory misrule as the bedroom tax, and you will be met with furrowed brows, and an insistence that no commitments can be made.

The truth is, politics – and particularly centre-left politics – feels contorted and messed-up for a lot of deep-seated reasons. Chief among them is the strange turnabout since the financial crash of 2007-8. No mainstream politician has yet summed up what has happened, but the late novelist Iain Banks did a sterling job: “Your society’s broken, so who should we blame? Should we blame the rich, powerful people who caused it? No, let’s blame the people with no power and no money and these immigrants who don’t even have the vote – yeah, it must be their fucking fault.”

Note also last week’s flurry of headlines about zero-hours contracts: more proof that, to use old-school terms, capital is now lording it over labour to a quite amazing extent.

This is surely the defining feature of the early 21st century. The Labour party occasionally tries to put things the right way up, but too often, it seems to accept the toxic settlement the coalition have embedded. Witness, for example, the party’s work and pensions spokesman Liam Byrne taking issue with the new benefits cap because: “It won’t affect Britain’s 4,000 largest families and it does nothing to stop people living a life on welfare.”

Yet from time to time, you hear much better stuff. After a summer during which far too many of its front-benchers seem to have taken a vow of silence before setting off on holiday, Labour is belatedly trying to make the running on the crisis in people’s living standards. Policy-wise, there has been talk of building new houses, putting young unemployed people back to work, coming down hard on profiteering energy firms, and more.

The problem is that all this has not yet cohered into a consistent and primary-coloured message that can cut through such clunky Labour promises as “a recovery made by the many” (whatever that is). In the last week, a couple of backbenchers have bemoaned this ongoing fuzziness: the Scottish MP George Mudie says he has “difficulty knowing what we stand for” and reckons his party is “not setting any agenda”; Geraint Davies thinks that “the problem is that the electorate doesn’t yet see a clear choice between the parties”; the dependably grumpy Manchester MP Graham Stringer thinks the shadow cabinet is “slumbering”. None of these people are big hitters, but that isn’t the point: they are giving voice to very real unease about Labour’s predicament.

Meanwhile, Labour leader Ed Miliband has begun an uncertain tussle with the trade unions that may find him caught up in internal battles, and people who spend their lives picking through polling data still insist the average Briton cannot imagine him moving into Downing Street. Labour, then, is in a very uncertain place – and the reasons why run deep.

By nature, Labour is not radical

Commentary about the party’s predicament usually falls one of two ways. People on the right tend to pin all Labour’s problems on Miliband, and mock him as a toxic mixture of caution, confusion and woolly-minded north London socialism; on the left, there are regular calls for him to slough off the New Labour inheritance, be “bold”, and lead us all into some new social-democratic nirvana. But this is the Labour party, remember, not the Petrograd Soviet. It has arguably been successfully “bold” only once: in 1945, when in any case, a great deal of what it did went with the grain of Britain’s reshaping during the second world war. For most of its history, it has believed in incremental change at most – and, even at watershed moments in history, deferred to conventional wisdom: before splitting the party in the wake of the crash of 1929, let us not forget, Labour’s then leadership stuck with the supposed wisdom of balanced budgets and spending cuts. Some of that may sound familiar.

As may this: “The Labour party is hesitant in action, because [it is] divided in mind. It does not achieve what it could, because it does not know what it wants. It frets out of office and fumbles in it, because it lacks the assurance either to wait or to strike … If it neither acts with decision nor inspires others so to act, the principal reason is that it is itself undecided.” The Labour-aligned historian and thinker RH Tawney wrote those words, in 1932.

Ed Miliband is a lonely figure

At the end of May, a YouGov poll brought bad news for the Labour leader: 57% of people agreed that he dithered. More than half concurred with the suggestion that he is out of touch, weak or unclear about what he stands for. For what such things are worth, the  last Opinium/Observer poll put his approval rating at -22%: 22% of people surveyed thought he was doing a good job, as against 44% who didn’t (David Cameron’s respective ratings were 33% and 50%, meaning an approval rating of -17%). Obviously, none of this is good. There again, nobody seems to much respect any modern politician – and besides, Miliband’s supporters are fond of pointing out that in the run-up to the 1979 election, Mrs Thatcher was always rated lower than Jim Callaghan, and look what happened there.

A lot of the more positive cliches about Miliband are true: in person, he is warm, endearingly modest, and possessed of enviable people skills. When he does Q&A events, even with hostile people, he is frequently brilliant. But he still faces an onerous challenge, about which he often seems to be well aware: thinking his way out of his Westminster background, and trying to somehow embody a new approach to politics. Moreover, three years after he won the leadership, he remains a strangely lonely figure within his own party, short of shadow cabinet allies, and usually lacking what Westminsterspeak calls “outriders”: people who can float ideas and stake out territory on his behalf.

One thing is now surely clear: that for all its current problems, the party picked the best of 2010’s leadership candidates, something underlined by David Miliband’s last round of media appearances before he left for New York. His turn on BBC’s Andrew Marr show, said the Spectator editor Fraser Nelson, “reminded me of why the Blair project crashed: it found, in Miliband, an automaton when it needed a champion.

“Personally,” he said, “I find Ed Miliband far easier to listen to … Listening to David Miliband, gibbering on about Iraq and his self-help soundbites, seemed as if we’d been plunged back into 2006.” Quite so. But being much better than your elder brother may not be quite enough.

It’s only three years since the last election

Aside from his acting prowess and ability to believe in several contradictory things at once, Tony Blair had one massive advantage as a prospective prime minister: 18 years separated his first administration from the Labour government that fell in 1979, and for a few years at least, his top team could therefore give the impression of being box-fresh. Soon after winning in 1997, he nailed all this in a ludicrous soundbite dispensed to a meeting of European socialist leaders: “New, new, everything is new.” Cheers, Tony.

By contrast, if Labour’s more optimistic predictions prove correct and we are back in an era of one-term governments, that brings obvious problems – because as well as attacking the record of the current government, Labour is still answerable for its time in office, and straitjacketed by Blair and Gordon Brown’s record. Which, of course, is inevitable: most of the current Labour frontbench were directly involved. When Ed Balls is interviewed on TV, you can set your watch by the moment, usually around four minutes in, when he is questioned about what he and Brown got up to in the Treasury, and he turns loudly defensive. It is never pretty.

The reason Labour can’t savage George Osborne’s apparent attempt to base his “recovery” on a house-price bubble is because Brown’s boom years were much the same. Self-evidently, it is hard for Labour to coherently oppose what Michael Gove is doing to the education system, when he himself claims he is simply finishing off what the Blair governments started. And yes, the model of public-service “delivery” whereby government hands everything over to Serco and G4S is evidently rotting, but what can Labour say about that, since its people were the ones to give such companies their decisive breakthroughs?

The left and right have squeezed the middle

Many of the loudest voices in the Labour debate come from the remains of the Blairite right, or the trade union-backed left. Indeed, the recent dustup about supposedly fixed parliamentary elections was essentially a slanging-match between the Blairite pressure group Progress (largely funded by Lord Sainsbury, and founded by people close to such über-New Labour types as Peter Mandelson), and the trade union Unite, whose leader Len McCluskey has recently been heard bemoaning the power held by “Oxbridge Blairites”.

This runs way beyond the issue of how Labour chooses its candidates. The party’s debate on austerity – if there is still one – is always in danger of collapsing into a futile exchange of fire between people who think Labour should be as gung-ho about cuts as Osborne, and those who think there should be no cuts at all. It is much the same on so-called “welfare”: you hear Labour voices who want to kick people on benefits as hard as the Tories do, or others who think there is nothing wrong with the status quo.

Miliband, of course, comes from neither camp. If he has roots in an ideological tendency, it is what used to be called the “soft left”. But where is it? In some Labour circles, this position is known as the squeezed middle.

Brownism lingers on

Labour’s caution and timidity may be tangled up in its collective DNA, but it is also often traceable to dried-up electoral politics. No one in the party will fess up to it, but it often gives the appearance of following what Australian politicians call a “small target strategy”, giving away precious little about what it may or may not have planned, so as to give the Tories as little to attack as possible.

Indeed, the fact that the electoral system is geared in Labour’s favour encourages this approach: it has been furiously denied by party insiders, but earlier this year, there was speculation about a “35% strategy”, apparently aimed at winning power on the lowest possible share of the vote, and with the minimum of effort.

Whatever, this approach to politics seems to be reflected in the fact that some of Miliband’s more interesting rhetoric – his self-proclaimed belief in “responsible capitalism” is the best example – is singularly undeveloped. Whispers around Westminster suggest that this is the way Balls likes it: he is no fan of making things uncomfortable for the business world, and is perfectly happy that Miliband’s rhetoric on such issues is often vague and inconclusive.

There are echoes here of the man who gave both Eds their big break. When he was making mischief and trying to present himself as a cheated heir apparent who would topple Blair and revive the Labour party’s soul, much was made of Brown grandly claiming that “the public square is more than a marketplace, [and] we are bound together by more than contracts markets and exchange” – which some people took as being a sign of visionary centre-left genius, rather than a dull statement of fact. No flesh was ever put on these rhetorical bones – and when he took power, it rapidly became clear that Brown had no plan whatsoever. As the comedian Rory Bremner pointed out: “It’s like having an uncle who’s been building something in the shed at the bottom of the garden for the past 10 years. You look through the window and there’s nothing there.” This is a screamingly cautionary tale for the current leadership, but they do not seem to paying it nearly enough heed.

The Tories have framed the debate

Every summer, MPs go off on their holidays and take the latest fashionable American book about the art of politics, which they tend to devour enthusiastically, and then promptly forget. A perennial example of this seasonal political amnesia is 2004’s Don’t Think of an Elephant! Know Your Values and Frame the Debate, by the American linguist George Lakoff, a short but brilliant read about how to break through the ongoing dominance of the right wing. Its essential message is that even if you are attacking them, there is no point using the exact same language as your opponents – their “frame” – and thinking you can win the debate, which is exactly what Labour too often does on so-called “welfare”, sure to be one of the biggest issues in 2015. Its acceptance of the Tory frame on austerity and the deficit is as bad. Lakoff’s book is a mere 124 pages long, so there is still time. Maybe.

Labour has missed the digital revolution

Exploitative “work experience” schemes. Corporate tax avoidance. The campaign to outlaw Page 3. These issues have all flared into life in the past 18 months, and proved that left politics in the UK is alive and well. The Labour leadership has had almost nothing to do with them: instead, far away from Westminster, they have been put on the agenda by people and organisations who understand where civilisation is headed: towards more “horizontal” organisation via social media and activity that can amass huge momentum in a matter of hours. Labour feels as if it is light years behind, playing the political equivalent of progressive rock, while the world turns punk: Miliband’s favoured method of getting his message out still centres on set-piece speeches and coverage in the print press, and even such bog-standard things as viral videos seem to be beyond his party’s grasp.

At the time of writing, Balls’ second-to-last tweet was a week old and read thus: “Lots of media today – and meetings in Washington DC with Sir Alan Greenspan & Christine Lagarde, David Lipton & team at the IMF.” Miliband’s tweeting seemed to have stopped on 22 July, with this: “Many congratulations to the Duke and Duchess of Cambridge. I wish them and their son all happiness and good health.”

On the positive side …

The last time anyone looked, Labour was pledged to somehow repeal the coalition’s changes to the NHS, or rather, the parts of them that open the way to the full-blown marketisation and competition (to which, of course, the last government opened the door). Of late, Balls has been bigging up the idea of building 400,000 new homes – though, as ever, it is something he would hypothetically do “now”, and what the policy may be in 2015 is unclear. Miliband talks about things no Tory or Lib Dem tends to go near: the need to push for a living wage, the rise of casual work, new regional banks, some kind of crackdown on profiteering energy firms and train companies.

At the moment, too many of Labour’s ideas in these areas seem either trifling, or vague and underdeveloped, but who knows? Its ongoing policy review could yet harden them up, and propose more. And, yes, if Labour actually makes it into government, we will at least be governed by a lot of people with different instincts from the current lot – and, one hopes, a reasonably distinct set of priorities, albeit within an unbelievably tight spending framework.

Here though, are some questions. As ever, they are about the economy, stupid. Is Labour’s project to return belatedly to Miliband’s ideas about responsible capitalism and “pre-distribution” – or, as some people call it, what to do about inequality – and aim at starting to reshape Britain in a fairer and more sustainable way than the coalition have managed? If so, it will need a lot more than 35% of the vote to even begin to do it: between now and the election it will not only have to lay out its vision, but begin to amass momentum and support. In the US, they call this being “insurgent” – and there are currently few signs.

By contrast, if the plan is to avert its eyes from Britain’s underlying problems, tinker here and there and somehow get back to the kind of fragile stability that prevailed until 2007, the whole thing will surely tank, at speed – and, moreover, compound the sense that British politics offers no real choices at all. So, which is it? Twenty-one months to go …

FOUR VIEWS ON WHAT LABOUR NEEDS TO DO

Salma Yaqoob, former leader of Respect

“The first and most important thing is the need for a very strong and simple narrative. The Tory party has one: it’s all lies, but it’s been very effective. They say Labour created the mess, it is worse than anyone thought, and the Conservatives are clearing it up. Whenever you hear that, Labour seems to be silent. But they should be on the frontfoot, saying: ‘This crisis wasn’t created by public spending.’ That’s the corner they have to get out of.

“There should be a Labour mea culpa about the failure to regulate, and having too much faith in the financial sector. They should then talk clearly about how they would regulate things to make sure the kind of crash we have had never happens again. The argument should be about the long-term stability of the country. Another strong theme should be the extent to which the welfare state subsidises employers and landlords. The whole thing about shirkers and strivers needs to be turned around: where does the benefits bill actually go?

“Labour should support a financial transactions tax and propose a big house-building programme. How optimistic do I feel about them embracing any of that? Not very. They seem to be conceding everything and tying their own hands. But the facts and moral argument are strongly on the progressive side.”

Maurice Glasman, ‘Blue Labour’ thinker and Labour peer

“Justin Welby said the other week that he was going to open every church as an alternative to Wonga.com. Where is Labour, partnering that? We’ve got to work on our relationships with the people, in their lives. And what do they care about? They care about their families, their parents, their children, the place they live. And they care about work. And we are not seen as being on their side on any of those things. We’ve got to partner them up, and offer an alternative to the dominance of finance capital, and the state.

“A living wage, and interest rate caps – they should be put at the front, not as additions to other things. Another thing relates to public-sector institutions, such as hospitals: there must be representation of the patients, and the workers. The same thing in the private sector: the importance of work has to come to the fore, and workforces have to be represented in the governance of private companies. It’s straightforward: there’s got to be a politics of the common good, and we’ve got to be its leader.

“We’ve got to clarify our story, about what was wrong with New Labour, and how this will be different and better. The way it stands at the moment is, we’ve said lots of things – but the important thing is not to say lots of things: it’s to say a few things, and say them often.”

Neal Lawson, chair of leftwing pressure group Compass

“The fundamental problem is that party politics is linear, analogue and top-down, and the world is increasingly complex, horizontal and plural. Labour has a 20th-century culture and organisation in a world that’s being transformed, culturally and technologically. Can the old adapt and transform itself, or does it have to die before something new and relevant takes shape?

“The solutions lie beyond the Blairite right and the old statist left. If it wants to even start to move in the right direction, Labour needs to immerse itself in the new organisations where there’s energy and vitality. It ought to allow its members to decide what its next campaign priority ought to be, like 38 Degrees does: a windfall tax on the utilities? Public ownership of the railways? Banning advertising to children? For at least one day, it also ought to open up its conference to anyone who wants to attend: no barriers or security guards, or £500 passes. And the leadership ought to sit there, and listen to people telling them what they think of their form of politics.

“Labour’s mission ought to be to unleash the energy and potential of the British people, to transform their workplaces, communities and lives. How far is it from grasping that? A century.”

Deborah Mattinson, founder-director of “strategic advice” agency Britain Thinks, and former pollster for Gordon Brown

“What Labour is lacking is real clarity about what they care about and, very specifically, what they would do – how it would be different from the lot that are in now, and how it would be different from Labour before. Because there isn’t that clear definition, when they think of Labour, people now tend to refer backwards. They have no sense of what’s going to happen, so they go back to New Labour days, or even further.

“We’ve reached a point where something quite dramatic has to happen to get the public to look again and take notice. I don’t know what it is, but I can tell you the kind of boxes it needs to tick. It sounds like a statement of the bleeding obvious, but you have to focus on things the public really cares about. A lot of the so-called triumphs that Ed and Labour have had recently haven’t been like that: tackling News International, talking about the media – and, actually, going big on the unions and party funding. People’s finances are in a bad way, they’re worried – and they need to know that Labour cares about the same things they care about.

“Use language that people understand – again, that sounds obvious, but recently, when Labour has talked about the economy, they’ve tended to use language the public doesn’t understand, like ‘predistribution’ and so on. As far as Ed is concerned, we still don’t really have a sense of what his mission is. The focus all the time needs to be not on the machinery of what you do, and the process – but what you can deliver for the voters. We used to talk a lot about about symbolic policies. You don’t need many, but you have to have two or three. Mrs Thatcher selling council houses is the classic example. Another one would be [Labour’s] creation of the Open University. Has Labour got anything like that? I’m not aware of anything.”