Mayfair's Marlborough gallery effectively invented the modern art market in the 1960s, but the notorious Rothko case badly dented its image. Now a new space to showcase today's art world stars is giving it fresh direction

Gilbert Lloyd and Andrew Renton, in front of an Angela Ferreira sculpture, at Marlborough Contempory art gallery, London. Photograph: Karen Robinson

On Thursday night, Marlborough Fine Art, which has occupied the ground floor of a fine 18th- century terrace at 6 Albemarle Street, Mayfair, for the last 40 years, opened an upstairs gallery and celebrated the fact with a party attended by the London art world's most glamorous figures.This was more than a routine office expansion. It was, in the eyes of the Marlborough's managing director, Gilbert Lloyd, a long-awaited rebirth. Lloyd, a twinkling, bearded man of consummate charm, now 72, is an elusive, semi-mythical figure in his world. A long-time resident of Nassau in the Bahamas, with an accent that still betrays a little of his Austrian ancestry, he started work in the family firm, established by his father, Frank, 50 years ago. In the decades that followed, and before dealers such as Charles Saatchi or Larry Gagosian had their say, for better or worse Marlborough virtually invented the modern art market.

In the 1960s, the gallery was the dealer for Francis Bacon, Henry Moore, Frank Auerbach, Lucian Freud, Graham Sutherland and Barbara Hepworth, as well as establishing the international reputations of Egon Schiele, Gustav Klimt and others. In America, where Marlborough opened the doors of a New York gallery in 1963, it was the prime mover of abstract expressionism, taking on the estate of Jackson Pollock and working exclusively with Robert Motherwell, David Smith and, most infamously, Mark Rothko, an association that was to end in the "art trial of the century". Despite that scandal, Marlborough maintained its place at the top table of the art world. The London gallery continued to represent Auerbach and Paula Rego, among many others.

Gilbert Lloyd is reaching an age when some men could be thinking perhaps of winding down, but instead he has been planning for the shock of the new. With this in mind, he has turned to Andrew Renton, until recently professor of curating at Goldsmiths College, London, to move in upstairs and open Marlborough Contemporary.

At first sight, theirs is an unusual marriage. The academic Renton, 48, has no experience of selling paintings, whereas Lloyd has been happily swimming in the sharkish waters of art dealing for most of his life. Speaking to both of the new partners in advance of their adventure, however, it is hard to say who is the more enthralled. Renton says: "When we first began discussing plans, Gilbert asked, hopefully, 'I expect you will be having a lot of discotheques upstairs?' He pretends to be the old guard, but he is really excited about it all."

When Renton was approached for the job, at a time when he was "thinking of having leather patches sewn on his elbows" and settling into his tenure at Goldsmiths, alma mater to Damien Hirst and the Sensation group of young British artists, he thought they had the wrong man. He had spent his working life writing and teaching and curating mostly conceptual exhibitions, not whispering up seven-figure prices for painterly modern masters. Eventually, though, he went along to meet Lloyd, if only out of curiosity for "the legend" and "because as a lonely and difficult teenager I used to come to Marlborough to look at art, particularly Francis Bacon".

He was immediately seduced. "It was like meeting a new old friend," he says. "After about three months it became clear that the most interesting thing to do would be for me to create something brand new alongside the original thing. Marlborough London remains a fantastic business. But I suppose there was a feeling from Gilbert that the likes of Frank Auerbach and Paula Rego were not being replaced."

Lloyd, for his part, suggests that he had been "looking for years and years for the right man to take things forward. It so happened that I found Andrew at the very same time that it became possible to renovate this 18th-century building because all the sub-leases came up in the same month. All the tenants cleared out and we were able to gut the place."

Renton fills in for Lloyd the gaps in his knowledge of contemporary art that had inevitably grown, despite a peripatetic life that sees him and his wife travel the world's art fairs for much of the year. "I am not ashamed to say I don't know parts of this new world at all," Lloyd says. "At Frieze [art fair] in London for example, which I always find very invigorating, it is always very difficult for me to pick out from this enormous amount of art what we can sell with a good conscience. Andrew has that eye."

Lloyd, with his precisely clipped beard and immaculate tailoring, believes Renton, who has a shock of black hair and an air of practised dishevelment, can work in the Marlborough tradition. "We have always been after quality and beauty and desirability," Lloyd says. "There is a lot of trophy-hunting which goes on in the art world. At the moment for example, you have to own a Warhol Marilyn. We are not in the business of supplying trophies."

That, of course, is what all art dealers say. When I meet Lloyd at Brown's hotel over the road from his gallery, he is just back from Art Basel, where the Marlborough stand had a 1954 Rothko from a Swiss collection, a block of orange above a band of pale pink that Lloyd had priced at $78m (£48.7m). Though he had been talking to some interested parties, particularly from South America, the Rothko remained for sale. "It will find a good home this month or next though. It is a very great picture."

To understand what Renton calls the "legend" of Lloyd, it seems important to know a little of where he came from. He is, in some ways, the creation of his father, Frank, who established Marlborough when he was demobbed from the British army in 1946 along with his friend, and fellow Austrian emigre, Harry Fischer. The Marlborough name was chosen because it sounded like an establishment fixture, just as Frank Lloyd had changed his name from Franz Kurt Levai, prompted by the bank at which he opened his London account.

Before the war, Levai's parents had an antiques shop in the centre of Vienna. In 1938, they lost everything in the Anschluss and his parents eventually perished in Auschwitz. Levai, a Jew, escaped to France with his non-Jewish girlfriend. Gilbert was born in May 1940, the same day his father sailed from St Jean de Luz harbour for Britain. Mother and baby were repatriated by the Gestapo to Austria.

Lloyd never knew his father until he was five, when Frank arrived in a British uniform in the little village outside Salzburg where he and his mother had been hiding all that time. "It had been quite tough," Lloyd recalls, with understatement. "In the village money had no value. My mother's father taught all his children a trade and she was a seamstress. That saved our lives really. She could make some dresses for a farmer's wife and in return we could get a quarter of a pig and a kilo of butter for the winter. That was how we lived."

That early deal-making ran deep with Lloyd. They moved to London with his father. A baby sister was born and Lloyd eventually attended the Courtauld Institute of Art before joining what had become the family firm. By that time, Marlborough was already established as one of the biggest players in an expanding international art market. Lloyd and Fischer, and their British partner David Somerset, later the Duke of Beaufort, had seen that after the war there were many possible openings: for the sale of old masters from the British aristocracy fallen on hard times, for the marketing of European artists almost unknown in Britain and America such as Schiele and Klimt and, most crucially, in the establishment of a new generation of postwar masters, of whom there was a potentially unlimited supply.

"In the 60s," Lloyd recalls, "there was a lot of art to be had on the secondary market and living artists were not well looked after at all. There were very few collectors, very few competitors and hardly any money. Marlborough were good at what they did. David Somerset handled the PR and sales [the duke remains Marlborough chairman], Harry Fischer was the intellectual salesman and my father was the businessman who loved art and who concentrated on providing the capital."

The catalogues boxed in the basement of the current gallery are a testament to that endeavour. "When we opened premises at 39 Old Bond Street," Lloyd recalls, "we had an exhibition of 18 Van Gogh self-portraits. It would be impossible to do that now. The engineer Van Gogh, Vincent's nephew, who was heir to the Van Gogh estate, came to the opening. I remember him well, a charming old Dutchman."

For a short while, it seemed Marlborough had almost everything to itself. "For one thing, the auction houses were rather fuddy-duddy and not at all active," Lloyd says. "Though that all changed one night in the mid 60s, when for the first time we were invited by Peter Wilson to an evening auction at Sotheby's. Black tie. And all us dealers thought, 'What is going on here? Auctions happen at 11 in the morning and no one goes.' That was the beginning of the auction houses' rocket-like ascent and in a more modest way Marlborough took off alongside them."

It was not all high octane. Lloyd well remembers the years when "we used to celebrate for a week when we sold a Bacon; we would celebrate for two weeks when we sold an Auerbach. These British painters were totally out of fashion," he says, even, if you went to a client's home in Dallas, Texas, say, something of an embarrassment. "I remember one man in particular, Jim Clark, had a wonderful collection of Mondrians," Lloyd says. "At some point in our meeting, Mrs Clark would say, 'Show us some of your newer gallery art' and I would bring out a large Bacon of two men in the nude cavorting in a field. This would tend to cause a deathly hush."

Francis Bacon signed a 10-year contract with Marlborough in 1958 that began with Frank Lloyd's undertaking to settle a £5,000 gambling debt the artist had incurred and which guaranteed money against future paintings. In the terms of the contract, a painting measuring 20in by 24in was valued at £165; one of 65in by 78in £420. Bacon was contracted to supply the gallery with £3,600 worth of paintings each year. Bacon called his Marlborough employers "the old uncles" and was known to joke of Frank Lloyd: "I'd rather be in the hands of a competent crook than in the hands of an incompetent honest man." The gallery's administrator, Valerie Beston, became his celebrated helpmeet and protection from the world, and even after he withdrew from that original contract and his paintings were selling for hundreds of thousands of pounds, Bacon retained a loyal affiliation to the gallery.

"Bacon's exhibition in the Grand Palais in Paris in 1971 was one of the most stunning moments of my life," Lloyd says. "You knew that the explosion was going to happen for Francis and that was the moment. But I am also rather pleased for example to have been with Francis in Berlin for his retrospective in 1984 and walked along the Berlin Wall with him and, although I warned him against it, taken him to East Berlin. He wanted a slap-up meal there, though we didn't find one of course. He went to see the Pergamon Altar. We spent a lot of time together. I always thought being a dealer is a bit like being the oil in the gear case of the artist's life."

This efficient anonymity was undone for Marlborough in 1972 with the Rothko trial, which still attaches itself to the gallery's name 40 years on. When I bring it up with Lloyd he winces slightly and a little wearily.

"It is inevitable that people will talk about it," he says. "As far as I am concerned it was a very bad chapter for Marlborough, but it was dealt with by the judicial system and everything was cleared up; we came out of it with a black eye and we have long considered it history."

It is, however, history with a capital H. At the time, the trial was routinely called the Watergate of the art world, casting light on the murkier practices of millionaire dealers and their clients. When Mark Rothko died in 1970, he left behind 800 paintings in the hands of his appointed trustees, a triumvirate of men who were very close to Marlborough and Frank Lloyd. A hundred of those paintings were quickly sold to the gallery for $1.8m, a fraction of their market value. In 1972, Rothko's 20-year-old daughter, Kate, on behalf of herself and her eight-year-old brother, Christopher, sued the trustees and the gallery over the terms of that deal and the alleged exploitation of her father's work. The court case lasted eight months and became a cause celebre in liberal New York, as it exposed the ways in which Marlborough (in common with other dealers) manipulated the market and its artists to hugely rewarding advantage, using a base in Liechtenstein legally but shadily to avoid taxes on the art it bought and sold.

Frank Lloyd became a media pariah for his performance in the witness box, telling one reporter: "I collect money, not art." The critic Robert Hughes, commenting on the trial, suggested Lloyd senior was viewed by the New York public as a combination of "Fu Manchu and Goldfinger", the foreign plutocrat resident in Nassau defrauding the orphans of tortured artists. Marlborough and the trustees lost the case and Marlborough was ordered to pay more than $9m in damages and fines. Later, it was discovered Lloyd had sold some of Rothko's paintings in contravention of a temporary ruling and had tampered with the gallery's books to cover up the deals. Another consignment of Rothko paintings was reportedly bound for Liechtenstein before prosecutors, who had been tipped off, intercepted them.

Frank Lloyd originally moved to Nassau in part to escape American justice, although he returned to face a criminal trial in 1982, a conviction resulting in a fine and a requirement to teach in New York public art schools. He was also forced to leave the company, its reputation severely compromised, to be managed by Gilbert, his sister, Barbara, and their cousin, Pierre Levai, who remains president of Marlborough New York.

Gilbert Lloyd is necessarily practised in playing down the impact of the trial on Marlborough's reputation. "There were some consequences. Some of the more politically correct American artists rather sadly said, 'We have to leave.'" This is a reference to the loss of the Jackson Pollock estate and the abrupt departure of Robert Motherwell, who said of Frank Lloyd: "If you are in his power, he is ruthless" and: "He knows everyone has his price; Lloyd's potency is money."

"But generally speaking," Lloyd maintains, "there were not so many defections."

When I ask the about the frustrations in what seems a generally golden life, Lloyd talks about the fact that "I was never able to do exactly what I wanted when my father was in charge. He was a great man, but he ruled the place with a fist of iron. When he left the scene, half my life was gone and I really had to try very hard to change the direction of the gallery."

One of the difficulties of that change was that after the Rothko case a slew of other litigation followed, attempting to use the Rothko ruling as a precedent. Marlborough was faced with defending its actions against the estates of Naum Gabo, Kurt Schwitters and most famously that of Francis Bacon, which looked back at that original 10-year contract and sought retrospective compensation. None of these suits was successful; the Bacon case, pursued by John Edwards, the former publican who was Bacon's heir, was dropped before it came to court.

Not surprisingly, Lloyd is somewhat rueful about this part of his father's legacy. "He was an enormous influence on me. But he has been dead a long time now. I really like to think I turned over a new page. He started it all, but I feel we are very much a different generation."

One of the motivations for opening the new gallery, you imagine, is to emphasise that fact and to secure the wider legacy of the family firm. Lloyd is proud to report that the "builder has constructed Marlborough Contemporary to last for a couple of centuries". Having lived through precarious times, he is interested in permanence.

Andrew Renton, for his part, thinks it is "amazing that the Rothko case is still on the radar. Gilbert feels strongly they paid their dues. But people bring it up, artists are conscious of it. It is partly I think because we can't get our heads around the concept of someone like Rothko being represented in that way. If one painting is worth $78m, what would 100 paintings be worth?"

When he announced his new role, one artist friend gave Renton a gift of the book that details the events of the trial along with the instructions not to open it inside the gallery. When he read it, he says, it seemed to reflect a different era entirely. "Some of the things that Marlborough was accused of – manipulating the PR of museum exhibitions or supporting their artist for the Venice biennale – you think: 'Isn't that exactly what our job is?'"

Those long-blurred boundaries between the commercial and public, educational art worlds are personified in Renton's appointment. It is one thing to back your judgment academically, I suggest, but quite another pressure to put your money, or the gallery's money, where your mouth is.

Renton laughs. "Absolutely. This is a business. But having worked a large part of my life in the public sector I know how enabling collecting art is. The fact that people buy art makes art possible. You can wait for public funding for ever."

He has enjoyed sitting in inner-sanctum meetings with Lloyd and his fellow directors, who have all been with Marlborough since at least the 1960s, and in the case of David Somerset from the beginning. "What I get is that arc of history. They have seen four or five recessions come and go."

Although he has consulted on establishing several important collections, the closest Renton has previously come to running a commercial gallery was in a space in 1996 that someone had given him rent-free. "It was basically a corridor that had its own front door," he says. "One day, we had a crisis because one of the artists we showed sold a book. I called him up and said, 'I have 25 quid but I don't know what to do with the money.' He said: 'Do you want to go for a curry?'"

That artist, Ian Whittlesea, is among those that Renton has contracted to Marlborough Contemporary. He also plans exhibitions with the Belgian painter Koen van den Broek and the video artist Adam Chodzko. The gallery with an installation by the Mozambique-born artist Ângela Ferreira, who looks at the influence of modernism in Africa, in a documentary spirit, "and is about as far from a traditional Marlborough artist as you could get," Renton says.

Ferreira's show is upstairs from a display of new work by Frank Auerbach. Lloyd enjoys the contrast. "We have been a gallery dealing with easel paintings and bronzes in limited editions. I never dealt in works with motors and flashing lights and televisions because I was always worried about how you maintain them. I mean, if you have something featuring a television made in 1960, what do you do when it goes wrong? Nowadays, though, with digital media, they are to a certain extent indestructible."

Having seen at first hand the difficulties created by a domineering managing director in his father, Lloyd is determined not to cramp Renton's style. "I am not going to interfere one iota. That said, I am on the phone to him every day discussing plans."

The only point on which the two men seem slightly to diverge is the length of time it will take to make the new gallery a success. Renton talks in terms of five- and 10-year plans. I get the idea that Lloyd is a little less patient than that. "I am 72," he says. "I am looking forward to some buzz." By which I guess he means sales.

Does he still love the art of the deal? "It is an enormous thrill," he says. "It is not about the price, it is the making of a good sale. I don't like much these individuals who walk around in faded blue jeans and white shirts open to the navel saying they are 'gallerists' and not in it to make money." Money has always to be at the heart of it? "It has to be," he says, determinedly, before adding: "But only to make all the other good things happen."