The artist Mira Schendel first saw Brazil, the country that would become her home, in August 1949. She was 30 years old, and a refugee: born Jewish in Switzerland but raised a Catholic in Italy; forced to move between Bulgaria, Austria and Sarajevo to avoid fascist persecution, and now seeking a new life in South America. After docking in Rio, Schendel and her husband Josep Hargesheimer would travel south to the city of Porto Alegre: two more stateless people in the great wave of postwar European émigrés.

But it was in São Paulo that Schendel ultimately established her life and work, after separating from Josep in 1953. Little remains now of the cosmopolitan city that Schendel would have found in the 1950s – the downtown district of Luz, then the city's beating heart, is now a scrubby, dilapidated area with a serious drug problem. But it isn't difficult to imagine the impact the first sight of Brazil must have had back then on unaccustomed European eyes. The rows of palm trees, the clotted traffic, the sense that the contours of life here are both familiar and utterly distinct; all still carry the same power now.

I travelled to Brazil's largest city to follow in Schendel's footsteps: to see the prolific body of work produced by this late, great artist (she died in 1988 aged 69, having produced countless paintings, sculptures, prints, notebooks and diaries); to meet her daughter, Ada; and to get a feel for her legacy. Schendel is a towering figure of Brazilian modernism, cited as a significant influence by many younger artists – and yet even in São Paulo, the majority of her works are in private collections, rather than on public display; and she has never been given a major solo retrospective outside Brazil.

Tate Modern is about to change all that: the gallery will this month open a wide-ranging exhibition of Schendel's work, in association with São Paulo's main public gallery, the Pinacoteca do Estado, where it will transfer next year. The driving force behind the show is Tanya Barson, curator of international art at Tate Modern, with a special interest in Latin America: she worked on the gallery's acclaimed 2007 show dedicated to Hélio Oiticica, another Brazilian modernist; and has spent eight years researching Schendel and her art, gaining the trust of the city's contemporary art lovers and gallery owners. With some of Schendel's most significant paintings hanging in collectors' living rooms – in one case, above a leading gallerist's bed.

En route to an art dealer's house in the affluent district of Morumbi, Barson tells me what first drew her to Schendel. "We had several of her works in the Oiticica show," she says. "I was really interested in the way they occupied the space: they seemed to have this incredible energy. The more I travelled to Brazil, and the more I immersed myself in Brazilian art, I realised that Mira's name was on everybody's lips. It's a curious thing: she's such an influential figure here, and yet it's really not that easy to view her work. I find that disjuncture very intriguing."

Schendel's artistic output is also full of disjunctures, taking in everything from colourful abstract canvases, in an almost figurative style reminiscent of the key modernist painter Alfredo Volpi (another Italian immigrant to Brazil), to sculptures made from intricately knotted rice paper. She also made huge, transparent "graphic objects" mounted on sheets of glass; hundreds of monotype prints; and complex paintings exploring diverse religious and philosophical themes, from the writings of Heidegger to Catholicism and the I Ching.

Schendel arrived in Brazil at a fascinating time. During the 1950s, modernism was taking its own uniquely Brazilian form, as the country underwent a rapid process of industrialisation, bringing a new affluence (for some) and a general sense of optimism that contrasted with dour postwar Europe. The architect Oscar Niemeyer began work on Brasilia, his city of swooping curves and white concrete, in 1956. Meanwhile, under the banner of arte concreta a group of São Paulo artists were asserting their belief in the raw, mechanical power of the abstract image; and in Rio de Janeiro, the neo-concrete movement – including Oiticica, Lygia Clark and Lygia Pape – were establishing their own, more emotive version of abstraction.

Links can naturally be drawn between the work of these artists and Schendel's. She was a close friend of the neo-concrete sculptor Sergio de Camargo, and a walk around the Pinacoteca do Estado's collection – where several works by Schendel hang alongside concrete and neo-concrete masterpieces – confirms their shared themes: an interest in the potential of pure shape and colour; and an exploration of writing as a visual, rather than solely communicative, tool.

But Schendel's work was too varied and too slippery to fit easily within these movements, and her own personality was too contrary to allow it to (one art dealer, whose home is lined with works by Schendel, remembers her as "scowling and chain-smoking"). In Barson's view, this makes Schendel's art all the more interesting. "What's remarkable about Mira," she says, "is that she was affirming a new line in Brazilian art: one preoccupied with ontology, and one that had a certain softness and delicacy. Her contemporaries respected her: a guestbook from one of her exhibitions in the 1960s reads like a Who's Who of Brazilian art. But her work is astonishingly complex. With Mira, it was never a simple story."

Schendel's story began in Zurich in 1919, where she was born the only daughter of Karl Leo Dub, a fabric merchant, and Ada Saveria Büttner, a milliner. She had an unsettled childhood, shuttling between Zurich and her grandparents' home in Berlin; and, in 1922, her mother took her to Milan, after separating from Karl. Mira's relationship with her mother was troubled: Ada would disappear for long periods, leaving her daughter in an austere Catholic convent school, before returning, guilt-ridden, to shower her with affection.

In São Paulo, we meet Schendel's daughter Ada (named, in a contradictory gesture she says was typical of her mother, after the grandmother who was so alternately warm and distant) and grandson Max for lunch. They say Mira's difficult relationship with her mother, together with her experiences as a refugee (she was forced to leave university in Milan in 1939 owing to Mussolini's new anti-semitic legislation, and later fled Italy for Sarajevo) left Mira with a lifelong sense of statelessness: of lacking her own home, and her own language. "She didn't feel at home either in Brazil or in Europe," Ada says, as Max nods. "She spoke many languages, but all of them with an accent," he adds. "She hoped when she arrived in Brazil that she could find a territory of her own – a place for her. But it was not that easy. She was a misplaced person. Territory, for her, was always an issue, and you can see it in her work."

The breakdown of language is certainly a defining theme of Schendel's art. Her preoccupation with writing, and its symbolic power, began in the mid-1960s, when she produced an extraordinary series of 2,000 "monotipias", or monotypes, in just three years. She would work in intense bursts, staying up all night in the living-room or kitchen of the house she shared with her second husband, the German bookseller Knut Schendel, and Ada, covering glass panels with oil paint and talcum powder. She would then lay a sheet of fine Japanese rice paper over the talc, and use her fingernails, or the blunt tip of a ballpoint pen, to scratch lines and shapes through the talc, into the paint.

In the Pinacoteca's basement workshop, Taisa Palhares – a curator at the gallery who is working with Barson on the Schendel exhibition – shows me a series of these monotypes, carefully laid out in tissue wrapping. In photographs, these delicate works lose much of their impact: but up close, with every smudge and inkblot visible, they have a mesmeric power. Many of the printed shapes resemble letters, or fragments of them: here is a stack of "c"s or "n"s; there, an elongated "z", or a reflected "s". Looking at them is like listening to a whispered conversation.

Letters are employed to even more powerful effect in Schendel's later series of "objetos graficos". Here, rolls of Japanese paper, covered in letters, symbols and type, are pressed between large sheets of transparent plastic. I see one graphic object hanging at an art dealer's living-room window: it is night-time, so the translucent effect is lost; but its owner tells me that, during the day, the light courses through the work, scattering shadowy fragments of language across the floor. It is impossible, looking at these works, not to think of Schendel, shuttling between countries: fluent in several languages, but fully at home in none of them.

Schendel did eventually find a home, not among São Paulo's artists, but its intellectuals: with Knut, she became part of a close-knit circle that included the theoretical physicist and art critic Mario Schenberg, the philosopher Vilém Flusser, and the concrete poet Haroldo de Campos; she also befriended the Dominican friar Paulo Celso. Later works would bear the influence of Schendel's interest in philosophy, of both the east and west; semiotics (she met Umberto Eco during a trip to Europe in 1967, but found him unsympathetic); and physics.

But, unsurprisingly for a woman raised between two religions, it was religion to which Schendel returned most often in her work; and in surprising, complex ways. In 1969, she exhibited an installation at the São Paulo Biennial with the unwieldy title Ondas paradas de probabilidade – Antigo Testamento, Livro dos Reis, 1, 19 (Still waves of probability – Old Testament, 1 Kings 19). It was made up of a series of hundreds of nylon threads, hanging from the ceiling like fine shafts of light or drops in a heavy rainstorm, and accompanied by a verse from the Book of Kings.

In 1975, Schendel produced a work called Homenagem a Dio – pai do Ocidente (Homage to God – father of the West), a series of 16 numbered sheets of paper, with typed Biblical phrases in Portuguese, German and Italian suspended beside brash swaths of spray-paint. I see this work displayed casually, in a dealer's upstairs office, but it will be travelling to Tate Modern, where Barson plans to allow it a narrow gallery to itself. I can see her reasoning: this is a difficult, complex work on which the eye needs to linger. Days later, I can still see in my mind the 12th frame, with the words "deus e amor" hovering beside a bloody seepage of red paint; and the 16th, where "der geist" hangs suspended above a watercolour tracery of flowers.

On leaving São Paulo, this is what stays with me most about Schendel: the contradictory nature of her character (loving but argumentative); of her work (delicate but profound); and of her identity (European but Brazilian; Jewish but also Catholic, or atheist, or maybe all of the above). Neither her life, nor her art, offers easy answers, but that is surely why both remain so intriguing: and why it is time, almost two decades after her death, for Schendel to become as familiar and respected here as she is 5,000 miles away, in the country she made her home.