Arthur Penn obit

Arthur Penn, who has died aged 88, was one of the major figures of US television, stage and film in the 1960s and 70s when the three disciplines actively encouraged experimentation, innovation and challenging subject matter. "I think the 1960s generation was a state of mind," he said, "and it's really the one I've been in since I was born." He will be best remembered for Bonnie and Clyde (1967), a complex and lyrical study of violent outsiders whose lives became the stuff of myth.

The film, starring Warren Beatty and Faye Dunaway, and based on the exploits of the bank-robbing Barrow Gang in the 1930s, became a cause celebre. It was praised and attacked for its distortion, bad taste and glorification of violence in equal measure. Newsweek's critic, Joseph Morgenstern, retracted his initial view of the film's violence, admitting that he had misread explicitness for exploitiveness. The film won two Oscars (best cinematography and best supporting actress) from a total of 10 nominations (including best picture and best director).

Penn was born in Philadelphia into a Russian-Jewish family, the younger of two sons. His brother, Irving, became a noted fashion photographer. His father was a watch repairer and engraver. By the time he was four, Penn's parents had divorced. The boys went first to New York with their mother, a nurse. When he was 14 Arthur went to live with his father in Philadelphia. It was at this time that he became fascinated by the theatre, acting in school productions and on local radio.

During the second world war, he was sent to Fort Jackson, South Carolina, to train as a rifleman, and in his spare time he set up a theatre group. It was there that Penn met Fred Coe, who was to play an important role in his professional life. Towards the end of the war, Penn spent time acting in Paris, then returned to the US to study at the experimental Black Mountain College in North Carolina. He continued to stage theatre productions before heading back to Europe to study literature in Italy.

On his return to the US, he joined the Los Angeles branch of the Actors Studio. His first professional work, in 1951, was with NBC TV in New York as a floor manager working on The Colgate Comedy Hour. He began to write plays for television, and in 1953 Coe, who was also with NBC in New York, asked him to direct a live experimental drama series called First Person.

During the 1950s Penn also became active in the theatre. His not terribly inspired Broadway debut, The Lovers (1956), ran for only four performances. His next production, however, was a success. Two for the Seesaw (1958), starring Henry Fonda and Anne Bancroft, ran for more than 700 performances. He had a Tony award-winning hit with William Gibson's The Miracle Worker (1959), the story of Helen Keller, which he had previously directed for TV. He also found success with Toys in the Attic and All the Way Home (both in 1960).

Coe produced some of Penn's stage work, and it was he who asked him to direct The Left Handed Gun (1958). This was based on Gore Vidal's television play which, rather than dealing with the outlaw Billy the Kid's notorious exploits, centred on him as a confused young man, desperately seeking love and recognition, who wreaks revenge on those who killed his boss, a kind rancher whom Billy has taken as a father figure. The film starred Paul Newman as Billy and was shot in only 23 days on an abandoned set. Warner Bros insisted on editing the film against his wishes and Penn always maintained that the treatment destroyed the rhythms of some of the scenes. Despite his reservations, it was an extraordinary debut by any standards and still resonates today, thanks to Newman's powerfully complex and touching performance.

The Left Handed Gun clearly signposted Penn's continuing preoccupations – family, father figures, the myths of American history and the contradictions they set up with reality. He was particularly interested in disillusioned outsiders in conflict with society and the law (albeit motivated more by emotion than logic), and their ensuing violence and pain, both of which were conveyed in a deeply sensuous way through the powerful performances Penn consistently drew from his actors.

His films can be seen as vividly allegorical, highlighting the traumas and conflicts of the times through which he and the nation were living. Penn openly admired the French new wave (the influence of directors such as François Truffaut and Jean-Luc Godard can be seen in his use of elliptical narratives and episodic structures) and Elia Kazan.

It was not until 1962 that he made another film, with his third interpretation of The Miracle Worker. Bancroft and Patty Duke won Oscars for their performances and Penn received a nomination for best director. This success was short-lived, however. His Broadway productions of 1962 and 1963 flopped, and only a week or two into shooting The Train, Burt Lancaster, for undisclosed reasons, insisted that he be taken off the film. Penn was always philosophical about this: "From that point they took this $2m film and proceeded to turn it into a $7m fiasco."

In 1965 he made Mickey One, a deeply paranoid noir thriller about a nightclub comedian (Beatty) on the run from mobsters who seems to be seeking punishment for an undefined sin. This was the first film on which Penn had full creative control and, to some extent, this may have proven his downfall. The film, shot in grainy black and white, was a strange mixture of naturalism and existentialism. Penn, who described it as "an allegory of a man's trip through purgatory", also said: "I was really operating on the symbolic and metaphorical level without engagement between audience and screen." The critic Robin Wood observed that the film "gives the impression of reversing Penn's usual method of working. In the other films he starts from the particular and the concrete ... and discovers the universal by a process of exploration. In Mickey One he appears to have started from an abstract conception and tried to impose the concrete on it."

The Chase (1966) was his first film in colour and, despite its problems, was rightly regarded by many as a (near) masterpiece. It perhaps most clearly enunciates Penn's stance on violence: "America is a country where people realise their views in violent ways – we have no tradition of persuasion, idealism or legality." In the film, Sheriff Calder (Marlon Brando) tries to protect an escaped convict, Bubber (Robert Redford), from the mob violence he has stirred up on his return to his home town.

The Chase's portrayal of small-town boredom fostering sexual philandering, racial and class hypocrisy and prejudice and random violence is deeply disturbing and often visually stunning. Penn's ability to give the feel of a wild west town, where the sheriff stands alone against lawlessness (albeit from within the town rather from outside), was impressive. Again, however, the editing was taken out of his control, which resulted in the loss of scenes in which Brando improvised his own dialogue. It has perhaps the most desolate ending of all of his films, none of which end on an optimistic note.

After two films which had been anything but commercially successful, Penn's film career looked bleak, but Beatty rescued it when he persuaded him to direct Bonnie and Clyde. Penn followed that film with Alice's Restaurant (1969), which he also co-wrote, a drama prompted by an Arlo Guthrie song. It was a highly episodic film which, for all its celebration of the protest movement and its rejection of Vietnam, racism and authoritarianism, remained pessimistic.

The Vietnam war clearly informed Penn's next – and greatest – film, Little Big Man (1970). Adapted from Thomas Berger's novel, this wildly comic, profoundly ironic and epic film recounts the memoirs of the 121-year old Jack Crabb (Dustin Hoffman) who survived the massace of his family and was brought up by a Cheyenne tribe. His story is the vehicle by which the traditional history and myths of white men and Native Americans are completly subverted, along with the conventions of the western genre.

Aside from contributing a section (on pole vaulting) to Visions of Eight, a documentary about the 1972 Munich Olympics, Penn took time out until he returned to cinema in 1975 with Night Moves, taking a straight genre script and rewriting it to embody the alienation of contemporary America. This deeply pessimistic film, in which one can almost touch the sense of malaise generated during the Watergate era, is as narratively elusive as any he made.

He had hardly finished Night Moves when he made another film with Brando, The Missouri Breaks (1976), which centres on the violent clashes between ranchers and rustlers in Montana in the 1880s. Brando plays the cold, hired gunman brought in to kill the rustlers; Jack Nicholson is the genial leader of one of the gangs. The film had some wonderful and eccentric moments, but opinion was divided. Penn himself was disappointed with both Night Moves and The Missouri Breaks and he returned to the theatre, directing Sly Fox (1976) and Golda (1977) on Broadway.

From his 1981 movie Four Friends onwards, his film career began to falter. Target (1985), Dead of Winter (1987), and Penn and Teller Get Killed (1989), starring the successful American magicians, suffered from mediocre scripts which clearly failed to ignite Penn's talents. He directed the television film Inside (1998), which dealt with the anti-apartheid struggle in South Africa, and in 2000 became an executive producer on Law and Order, some episodes of which his son Matthew directed. In 2002, after a break of some 20 years, he returned to the New York stage to direct Alan Bates in Fortune's Fool, an adaptation of Ivan Turgenev's A Poor Gentleman, for which Bates won a Tony.

Penn's unrealised projects included an adaptation of George Orwell's Burmese Days; a film on the Attica prison riot in New York; and The Last Cowboy, dealing with the takeover of the ranges by big business agriculture. In his latter years he maintained relationships with the Actors Studio and the Berkshire theatre in Stockbridge, Massachusetts. In 2007 he attended the Berlin film festival, which programmed a special tribute to his work.

In 1955 he married the actor Peggy Maurer. She survives him, along with Matthew, a daughter, Molly, and four grandchildren. Irving Penn died in 2009.