Building types are not what they used to be. I've noticed a trend for building-society premises to become high-concept Cornish pasty shops and have lost count of banks translated into pubs. And galleries are shape-shifting too.

Rarely are Eastbourne and Chelsea mentioned in the same architectural (or cultural, social, artistic, sexual, gastronomic) paragraph, but right now the seaside Sussex gerontopolis and the perfumed, Audi-rich hinterlands of Sloane Square are competing with two of the most interesting galleries of recent years. (They'll be joined next year by London's Whitechapel, remodelled to the tune of £10 million.)

In Eastbourne, the old carpark site next to the Congress Theatre will soon be a new home for the Towner Gallery. In Chelsea, the drill hall of Pevsner's 'sturdy Tuscan portico' that fronts the 1801 Royal Military Asylum for the Children of Soldiers' Widows (more recently known as the Duke of York's) will next month open as the new permanent home of the notorious, wandering Saatchi.

We use the word 'gallery' to describe an art museum because of Italian precedents. In Rome the Galleria of the Vatican displayed earth-bound Papal wealth while in Florence the Corridoio Vasariano was (and is) a splendid overground gallery linking the Pitti with Uffizi, designed to display Medici art with the intention of boggling the minds of visiting ambassadors and supplicants. But makeovers rather than new-build defined the two most interesting London galleries of the late 20th century. The first was The Boilerhouse Project in the Victoria & Albert. This opened in what had been a fetid basement in early 1982. It was made possible by three people. First, Terence Conran who was passionate to proselytise about modern design and willing to put his money where his proselytiser was. Second, Sir Roy Strong, who said to me (I was the third person involved): 'Oooh, just get on with it, darling'.

Getting on with it involved stripping back the old boilerhouse yard to make it a coruscating white-tiled box so harshly zeitgeisty that it was used in the credits for The Money Programme on the telly and the part of it that was my office was trialled as background for commercials about headache cures. The Boilerhouse's exhibitions about design made it 'London's most successful gallery of the Eighties', according to Arena. Since Arena was the most successful magazine of the Eighties, this must be true.

The second significant gallery of the period was the Saatchi on Boundary Road in London's well-upholstered St John's Wood. This was an old paint factory superbly tightened and brightened by the late Max Gordon, an architect with connections to and influence in the New York art world. (Significantly, it was the same Max Gordon who found the site for Conran's Design Museum, another white box that opened in 1989.) Gordon's Saatchi Gallery was like the most magnificent imaginable SoHo loft.

It opened in 1985. Here Charles Saatchi exhibited artists difficult at the time to find in this country: Donald Judd, Brice Marden and Cy Twombly, for example. It was an astonishingly wonderful, dramatic and beautiful place whose influence was only limited because St John's Wood is a lush backwater. Cleverly, Saatchi extended his influence by outreach. After all, if an adman does not understand marketing, distribution and demographics who does? Saatchi properties displayed at the Royal Academy in 1992 created the Young British Artist phenomenon and all the rest was... market-making.

The Saatchi Gallery's move to London's County Hall in 2003 was only a mixed success. Although it became central, it lost credibility and energy. Being next to a tacky aquarium and a low-concept permanent Salvador Dali exhibition somehow located Saatchi in circus culture. Moreover, the architecture of County Hall was inappropriate and stifling and, besides, Charles himself seemed to have lost interest. It was rumoured he now preferred go-karting. So the new Saatchi Gallery is a welcome sign of resurrection.

Architects AHMM spent three years looking for an appropriate site and, coincidence or not, eventually found one very close to Saatchi's Eaton Square home. It is a fine historic building in a beautiful wooded and grassy setting. AHMM stripped back the innards to reveal the underlying geometry and created 15 spacious new galleries on three floors. They are boldly white, sparely finished and top-lit, designed to create a recessive background for London's reclusive modern Medici. If they are recessive it is not because of a limited budget: none has been declared, but we may be certain Charles Saatchi does not stint himself. Anyway, this sort of recession is surely the right thing because so many art gallery designs - the Guggenheims in New York and Bilbao and the National Gallery extensions in London and Washington - fail because the showy architecture overwhelms the subtleties of painting.

The new Saatchi Gallery will be a huge success, but the new Towner Gallery in Eastbourne is perhaps even more interesting since it can enjoy the strange pleasures and atmospheres which attend provincial galleries: many of them still carry that faint aroma of beeswax only barely disguising the shocking radicalism and missionary bohemianism represented by cautious nudes and rumours of subterranean life classes. Londoners, fatigued by excess, may not be shocked by Wolfgang Tillmans (and there is a striking one in the Towner's permanent collection). You can bet someone in Eastbourne will be.

The Towner Gallery has a fascinating history. From 1923 to 2005 it was in a fine 18th-century manor house in Eastbourne's Old Town. Its first curator, Arthur Reeve Fowkes, had connections to the St Ives School and, to complement a foundation of topographical pictures, slowly built up a fine collection of English art including Edward Burra, Edward Wadsworth and Christopher Wood. Best of all, the Towner has the world's outstanding collection of Eric Ravilious, a local teacher who for my money was Britain's outstanding 20th-century artist. Because of this, in 1962 The Observer described the Towner as 'the most go ahead municipal gallery' in the country.

So when in 2005 it was decided to turn the gallery's home into flats, a mighty engine of grant applications and local authority fixing for a new venue was started up. Curator Matthew Rowe came to Eastbourne from Tate St Ives. A competition was announced through the Official Journal of the European Union. More than 80 entries were received. Rick Mather Architects were selected not least because they were prepared to take Eastbourne seriously.

The site is next door to the Congress Theatre, a 1963 landmark of modernismo now Grade II-listed and much admired by amateurs of period concrete. To develop a brief, Rowe travelled with the architects to see some noteworthy new small gallery spaces in Europe: Peter Zumthor's Kunsthaus in Bregenz, 1992 and Herzog and de Meuron's Schaulager in Basel, 2003, among them. Back in Sussex, they built a small jewel.

Light is the enemy of art, so the street facade is a mixture of cladding with white-painted concrete and only exiguous daylight-openings. Major structural elements are in exposed fair-faced (that is to say: glass-smooth) concrete: the structure of sometimes daringly spanned posts and lintels is visible throughout. Outside, the square-on facade becomes voluptuous volumes around the side.

The Towner design is modest only in the intelligent sense: budget restraints mean that goods and people have to share one giant lift. Visitors are ingeniously led through the levels of the building because the handsome staircase (solid black steel plate where there might have been banisters) rises through one floor at a time and exits at different ends. Through the balcony of the cafe is a view of redbrick and tile-hung late 19th-century Eastbourne.

Through the biggest windows in the building: a fine view of the startlingly green championship tennis courts. Rowe has £1m from the Art International Fund to spend on new acquisitions, but Miss Joan Hunter Dunn and I would just as soon see this fine building devoted in its entirety to the delightful and whimsical Eric Ravilious. And topographical watercolours in polite gilt frames would look gorgeous against its severe white modern walls. Still, splendid to have the choice.

In the cab back to the station I pondered the nature of galleries. Agreeable spaces, controlled lighting, hints of the incongruous or transgressive licensed by the sanctity of art. Why do I find the Towner perhaps a little more interesting than the Saatchi? Something to do, perhaps, with the dissonant glory of great art in bingo and variety show territory. That mysterious surreal mood caught in Auden's 'Musée des Beaux Arts' or Eliot's 'Prufrock' in whose room the women 'come and go, talking of Michelangelo'. Eastbourne seems somehow more adventurous: Edward Burra on the South Coast is more of a challenge than Dinos Chapman in Chelsea. 'Oh do not ask "what is it"/Let us go and make our visit.' I think we should.