Colonel Payne's West Park Legacy
by Julie O'Connor
click on any photo to see enlargement
This is a reprint of an article published in Ulster Magazine in 1992. No
attempt has been made to update any of the statements. It is representative
of the feelings and opinions of the residents of Esopus who were associated
with the Payne Estate.
Monasteries apartment complexes and ranch houses line Ulster County's west
bank: of the Hudson today. But once upon a time,
some of the richest men and women in the world lived
along what is now Route 9W. West Park sported
residents like John Jacob Astor and Delores Ponce
de Leon, daughter of the Governor General of Cuba.
The biggest mansion of them all belonged to an
enigmatic recluse named Colonel Oliver Hazard
Payne; today it's tucked behind a small sign labeled
Marist Brothers.
When the Colonel, as the people of West Park still
refer to him, died in 1917, the only thing locals knew
was that he was treasurer of Standard Oil. Payne's holdings in that company
were second only to John D. Rockefeller. Although he was childless Payne's
eight years in West Park would have a far-reaching legacy, touching a multitude
of lives - from the fabulously wealthy Whitney family to the son of John
Burroughs to a future world heavyweight boxing champion.
When Payne was 70, he bought the former Astor estate, coupled it with
another, and commissioned society architects Carrere and Hastings to build him
a massive Mediterranean palazzo.
Though Payne was actually a brigadier general, he preferred his more modest
rank. He earned it the hard way. Payne left Yale to fight for the Union Army
during the Civil War, before it became fashionable. He was from a tightly knit,
patriotic Ohio family; his father became a senator, and his brother became the
mayor of Cleveland. Another relative penned the tune "Home Sweet Home."
The Ohio Paynes used to spell their name like the Revolutionary War agitator
Thomas Paine. But an Old World sense of duty began to filter through the
Payne genetic pool. So did an aura of secretiveness and mystery.
Dr. Karl Berntsen, a professor emeritus at Cornell Medical College, spent last
spring in Payne's ancestral Cleveland home, sifting through the Colonel's
letters. Payne's multi-million dollar benefactions created the medical school at
which Berntsen taught.
Payne was shot from his horse during the Battle of Chickamauga. No one
knows exactly where the bullet went. "Some people think he got shot in the
genitalia, which might explain why he never had any children," says Berntsen.



In any case, Payne was extremely modest, before and after the bullet. During
the Civil War, he never allowed himself to be photographed; he would only
submit to a sketch artist, and even then only in profile. "He viewed himself as
unattractive," Berntsen explains. "But Payne was very popular at all stages of
his life, especially as an officer."
Heading through Illinois, Payne allowed his troops to go back to Ohio and
recruit their friends. Membership in the 124th Ohio Volunteers grew rapidly. "I
am the idol of my troops," Payne wrote his brother ..
"Payne wanted to do things right, and he did them," says Berntsen, who has
taken a liking to his research subject. Such attention to detail was noticed by
one general, who termed Payne's camp "a model of sanitation."
Payne's scrupulous nature was firmly formed by the time he went to prep
school. He voluntarily mailed his father itemized expense reports, justifying the
whereabouts of each dollar.
The Colonel wasn't the most academic of the Paynes. His sister was. Flora
Payne studied at Cambridge in special classes conducted only for women. She
was a compulsive writer, well-traveled, outspoken and sharp-tongued. When
Payne was wounded in the war, his sister wrote him, "You darling Oh - I hardly
know which to write to, you or the bullet in you."
"Flora has a fine, masculine mind," Payne wrote later about his favorite sibling.
Payne had so much affection for his sister that he fixed her up with his
handsome Yale chum William C. Whitney. As a wedding present, he built them
a house on Fifth Avenue in Manhattan, and gave them a million dollars in start-
up capital. Payne could spare the money. The man who was called
affectionately "my little captain" by the soldiers he commanded emerged from
the Civil War with heroic status and $20,000 from his father.
Just at the beginning of the acquisitive age of the Robber Barons, he became
an industrialist. Payne got his feet wet with iron manufacturing and ended up in
oil refining. Mister Rockefeller bought out his oil concern, Clark, Payne &
Company, he became a ground-floor associate at Standard Oil. It was said that
Payne and a handful of others controlled the American tobacco industry and
helped start U.S. Steel.
Payne eventually moved his base of operations from Cleveland to New York,
where he spent a good deal of time with his sister Flora and her family.
Although Payne loved children and lavished gifts upon Flora's brood, he never
married. He had already announced at the age of 18 that he would be
"eventually, from choice and not necessity, obliged to choose an old bachelor's
life."
In a strange turn of events, Payne did become a bachelor father. Flora Whitney
died of heart trouble in 1893. Her husband remarried three years later, and
Colonel






**'""'
some of Julie's original manuscript was inadvertently omitted from the published
article here!
*****
Medical College newsletter on the Payne-Whitney legacy.
Pulitzer prizewinning biographer William A. Swanberg touched on the
Payne- Whitney relationship in his biography,
Whitney Father,
Whitney Heiress.
When Whitney collapsed in Washington in 1887,
Oliver Hazard Payne hurried to his bedside. "He [Payne] was militarily
erect, decisive and to outsiders so formal and possessed of such
seeming hauteur that Henry Flagler, another Rockefeller partner,
described him as kin to God," Swanberg writes.
Whereas Vidal implies that Payne's devotion to the Whitney family
was based on homosexual impulse, Swanberg does not. "He
epitomized the Payne family solidarity," Swanberg writes. "He loved
his sister with an affection so tender that some - perhaps unfairly -
were later to question its normality."
West Parker Elizabeth Kelley Burroughs, granddaughter of naturalist
John Burroughs, remembers Payne as a warm and open man. Never
for a minute would she believe that Payne was gay. "Everybody said
when the Colonel was young he was engaged," Burroughs recalls,
"but his fiance died." "The Colonel was charming and delightful,"
adds Burroughs. "1 can't say enough nice things about him."
Payne also owned a castle in Scotland, a mansion in Thomasville,
Georgia, an estate named Paynehurst in Bermuda and a house on
Fifth Avenue.
His West Park mansion was faced with imported French limestone and
built around a huge central courtyard lined
with frescoes. The mansion's 40-plus rooms
were walled in rich Circassian walnut, leather
and ebony with gold tracery. There were
greenhouses, formal gardens, huge barns, a
gatehouse and a stone boathouse, where Payne's Aphrodite, the
largest yacht in America, was moored.
Payne brought his Ming porcelains and Rubens and Turner paintings
to West Park. He brought in tapestries, bronzes and busts. The
Colonel spared no expense on his Hudson River villa.
In 1913, Burroughs' father, Julian Burroughs, became the
superintendent of Payne's West Park estate. Since Julian Burroughs
was selling Victrolas door to door at the time, the job offer at the
Colonel's, which included a house, motor car, maid and chauffeur,
came as somewhat of a surprise.
Andrew Mason, the estate's previous superintendent, died either in
what the Town of Esopus Story refers to as an automobile accident or
from what Elizabeth Burroughs recalls as blood poisoning. The






Colonel's right-hand person, a Swedish woman named Miss Larson,
adopted Mason's orphaned daughter.
Remembering what a helpful Victrola salesman Burroughs had been,
she convinced the Colonel to hire him.
"It
was like a fairy tale," Elizabeth Burroughs says. "I was so happy to
grow up there and know beauty of that sort."
From all reports in West Park, the Colonel was very generous. He
sent Elizabeth Burroughs and her sister Ursula to Newport on his
yacht. He gave Miss Larson the go-ahead to buy them expensive Fifth
Avenue dresses and lunches at New York City's Clermont. He bought
them toys from F.A.O. Schwartz, magic lanterns and slides.
The Colonel and their brother, John Burroughs, shared a birthday, so
every year they shared a cake. Payne also gave the Burroughs boy a
pedigreed Sealyham puppy from his nephew, Payne Whitney's,
kennels.
Emily Burroughs, the children's self-sacrificing mother, received a
diamond bow-shaped brooch from Tiffany's after the Colonel noticed
she didn't wear a lot of jewelry. He also lent her his box at the
Metropolitan Opera House.
But Julian Burroughs was, by far. the largest West Park beneficiary.
"Those were my father's best years," recalls Elizabeth Burroughs with
a bittersweet expression. "At Riverby [the family farm where Julian's
naturalist father John lived], it was always, My boy
.
He was always
dictated to by my grandfather."
Julian designed large barns, a poultry farm, a
boathouse and a dock for the Colonel. He
commissioned custom-made iron gates, stone
lfl!!l"ll~
engravings and tile mosaics. Payne had total
confidence in his inexperienced superintendent
and never quibbled over price. "I would go into
the library to see the Colonel and he would ask
me how much money I needed' Julian wrote. "Often I'd say fifteen
thousand and he'd write me out a check for twenty. "
But the Colonel's architectural firm from New York City had problems
with the young superintendent. They sneered at his sketches and told
the Colonel that they would never dream of having anyone other than
themselves design a building for his estate.
Payne had reason to doubt the competence of
his society architects. The Hastings-designed
central courtyard at times collected ten feet of
snow. The imported French limestone couldn't
handle the Hudson Valley winters. "When I first
went there, the limestone around the cellar





windows was all crumbling," Julian wrote. "So we cut out
replacements of our native stone, which is good to this day."
Payne's affection for native stone and native folk is still remembered.
Although be didn't attend the nearby Ascension Church, the Colonel
had it wired for electricity and built the sexton's house. Payne threw a
wedding reception for one of his servants on board the Aphrodite.
Every year his employees sailed to West Point and enjoyed an on-
board picnic. "Those were good days for West Park," Elizabeth
Burroughs says.
At the annual employees' Christmas party, Payne gave out gold
pieces to the children. Esopus resident John Mowell received a few of
those coins when he was a boy. His father was a farm helper at
Payne's, and his grandfather worked in the estate quarry. "They also
had a moving picture at Colonel Payne's, where the Marist Brothers'
chapel is today," Mowell says. "You could go to the movies every
week if you wanted to."
Mowell's grandparents lived in a group of stone structures built to
house the Payne estate workers. Because of their quaint design,
these buildings on the western side of Route 9W were known as the
English Village. Today they're called Black Creek Apartments.
Mowell's neighbor, Amy Markle, also remembers the glory days of
Colonel Payne. One day, she took a peek inside the "Big House."
"There were solid gold- handled doorknobs," Markle recalls. "And a
room was papered in silk that came from Japan. Money was no
object. Everything was so big."
Markle didn t work on the estate. She only knew people who did. But
Payne's reputation preceded him. "The Colonel was sort of an
eccentric, I heard," says Markle. "He spent all this money on the
place, and his nephew didn't want it."
Colonel Payne died June 27, 1917 at his home on 852 Fifth A venue.
He was 78 years old and had outlived his four siblings. The bulk of his
huge estate went to his favorite nephew, Payne Whitney. The niece,
Pauline Whitney Paget, who sided with the Colonel in his dispute with
William C. Whitney, died of heart trouble in 1914.
To keep his will from being contested, the Colonel had to leave
something to his estranged nephew, Harry Payne Whitney, William C.
Whitney's eldest son. Payne willed him a single painting that Harry
had once praised.
It
was Turner's
"Juliet and Her Nurse."
Dorothy
Whitney Straight, Harry's other sister, didn't get a penny.
Payne left the 645-acre Esopus estate and two million dollars to his
other sister's son, Harry Payne Bingham of Cleveland. When Bingham
and the first of his three wives, Harriette Gown Bingham, moved into
town, the trouble started.




"It
is almost a universal truth," Julian Burroughs wrote, "that women
like to change their surroundings, but men do not. Young Mrs.
Bingham wanted to change everything."
"The opulence seemed almost
inappropriate in today's day and
age.
Mrs. Bingham "told
Burroughs to hand over his
plans for the unfinished
barns to her architects.
She told the workmen to
destroy the four-by-five-
foot stone en- graving that
Burroughs had designed
with the Latin phrase
That's not the way we Brothers
live."
---Brother Stephen Martin
Horae pereunt et puniunt,
which means "the hours
perish and punish." She
had a Bingham family crest
put in its place.
The Burroughs family left the estate after less than a year, but not
before they witnessed Mrs. Bingham's renovations to the mansion. The
gold tracery was chemically removed, and the ebony walls were painted
a putty color. The long, custom-made silk curtains in the drawing room
were dyed.
"She was an utter fool," says Elizabeth
Burroughs. "She had so little knowledge about
art. She really just wanted to be with her friends
on Long Island."
Though the Binghams were divorced by 1927,
they did share some brief happiness in Long
Island, where Harry won the amateur golf championship in 1924. When
Bingham couldn't get his property taxes reduced, he gave the West Park
estate to the Episcopal Diocese of New York in 1933. He was a
Presbyterian at the time. Hundreds of workers in West Park had to find
new jobs. Slowly, they cleared out of the English Village.
"Dad worked there until the place closed up,"
recalls Mowell. "Then he went to Hercules.
Everybody was laid
off.
Mr. Buchanan, the man
who was in charge of the chickens, was left with
a brooder house and a thousand half-grown
chicks. I think the estate must have turned their
over to him."







Before Bingham left, he donated a good deal of the Colonel's art
collection, including Rubens'
"Venus and Adonis" to the Metropolitan
Museum of Art, where, coincidentally, he became director. Bingham also
became a director of the First National Bank of New York and an avid
collector of marine life. The Bingham Oceanic Foundation at Yale was
established with more than 3000 of his specimens.
The Episcopalians turned Colonel
Payne's estate into the English Village
for Convalescents. It was a home for
mental patients. "They were sort of the
nutty kind,
11
recalls Mowell.
11
I came
home one night and saw a patient
playing golf in the middle of 9W."
The place never caught on.
''I came home
one night
and saw a patient
playing golf
in the middle of
9W"
- John Mowell
The Episcopalians started an experimental summer camp for
delinquent and homeless black kids from New York City in 1937. The
big stone barns were turned into dormitories, and teepees lined the
slopes by the Colonel's former chicken farm. The twenty boys in the
pilot program benefited so greatly that the camp caught the attention
of Eleanor Roosevelt, Pulitzer Prize-winning author James Agee and
the State Department of Welfare, which bought the camp from the
church and turned it into the Wiltwyck School for Boys in 1942.
The English Village was renovated into three dormitories, a dining hall
and a gymnasium for the Wiltwyck School's interracial, non-sectarian
population of 8- to 12-year-old kids. Although the kids were placed in
West Park by the courts, there were no bars at Wiltwyck.
It's been more than 35 years since former heavyweight champion
Floyd Patterson walked the grounds of the old Wiltwyck School. At
this point, he's not sure exactly where it is. "Is it that left up there?"
he asks hesitantly as his air-conditioned Lincoln heads up Route 9W.
Patterson's squint widens with recognition as he pulls into Black
Creek Apartments. He starts naming dorms, pointing to where the
kids lined up for dinner, remarking that this was the first place he
tried on boxing gloves. He spent one of his two Wiltwyck Christmases
at Eleanor Roosevelt's house in Hyde Park. He didn't realize who she
was. "She came up and rubbed my head," he says. "But to me she
was just a nice lady.
11
Patterson came from a large, poor Bedford-Stuyvesant family. He ran
away from home a lot, slept under the subway and stole at night.
11
I
was a delinquent," he says.
11
I had been in court a hundred times.
The judge even knew my name. What's it now, Floyd? he used to
say."






Patterson came to Wiltwyck in September 1945. He and another kid
stole a neighbor's rowboat, paddled to Poughkeepsie and jumped a
train to the city. The cops brought him back to West Park in three
days.
There he met a teacher named Vivian Costen, one of the most
important people in his life. Every Friday, Costen asked her Wiltwyck
pupils a question. Whoever answered correctly first got a bag of
candy. One day, Patterson thought he knew the answer. "Oh no, don't
answer. You're way off key," he said to himself.
No one won the candy that day, and Costen told them the answer
Floyd al- ready knew. Patterson ran out of the classroom crying, only
to be followed by Costen. "I know you knew the answer, Floyd," she
said, giving him the bag of candy.
A little positive feedback went a long way, and Floyd started raising
his hand like crazy. Costen had a soft spot for him. Once in a while,
she let him stay at her place in New York City, where she took him to
movies and restaurants. Floyd never forgot her. He gave her tickets
to all his New York fights. She saw him win the Olympics at the age
of 17. But she died before he became world heavyweight champion.
Patterson received his first dose of nature at the Wiltwyck School.
Now the middle-aged Patterson wrestles with the brambles by Black
Creek, navigating a log and jumping up to the dam with a boxer's
grace. He can't hide his disappointment. The old swimming hole dried
up a long time ago.
Further through the woods. he stops abruptly in awe, gaping at the
huge, ivy- encrusted stone barns, the buildings Julian Burroughs was
working on when Colonel Payne died. To Patterson's childhood
memory they are "the castle."
"I always thought, if I ever make it in this life, I'm going to live in a
place just like this," Patterson says, eyeing the bucolic splendor
around him. Eventually, he moved to New Paltz, where he continues
to live today with his family. His adopted son, Tracy Harris Patterson,
is the super bantamweight champion of the world.
"The murals outside weren't quite
appropriate for what we had in mind,
what with the nudes and all. "
- Brother Stephen Martin
The Episcopalians sold
the rest of Colonel
Payne's estate to the
Catholic Church in 1942.
The Marist Brothers
made a few renovations
when they moved in
with their religious
preparatory students.
The frescoes of the
Muses, Pegasus and a


youth lining the central
courtyard were covered
with paint. "The murals
outside weren't quite
appropriate for what we
had in mind, what with
the nudes and all," says
Brother Steve Martin.
"So we painted over
them, which was our
prerogative."
The Marist Brothers transformed the mansion's great hall into a
chapel, and made the other rooms dormitories and classrooms.
The Brothers built cinderblock extensions to house future novitiates in
1952 on the huge Payne-era garage and electric plant. But the
number of brother aspirants dwindled, and the West Park Marist
Brothers ran out of students. The property became a retreat house
for handicapped and mainstream students from New York City in
1971.
The intricate wooden carvings upstairs in the great hall intrigued
Merker most. "I remember putting my finger in the mouth of a carved
lion and there were ridges inside," she says. "They went down to the
last detail."
Every week, Merker crossed an immense
wrought-iron threshold with her bags of
groceries. The door was so heavy, she recalls,
that it literally ushered visitors inside
.
"I used
to pretend that the mansion was my house and
that two hundred guests were coming for the
weekend' she says in her best mock-patrician
voice.
When Brother Steve Martin took a group of high school kids on a field
trip to Hyde Park's Vanderbilt Mansion, the students ran to catch the
river view. "'That's our mansion," they screamed.
Due to financial considerations, the Marist Brothers sold the mansion
and the boathouse to another reclusive millionaire in the late 1980s.
The brothers moved their activities over to Colonel Payne's garage
and power plant. "The opulence seemed almost inappropriate in
today's day and age," Brother Steve Martin says. "That's not the way
we Brothers live. When I used to visit my family during the summer,
my room felt like a bandbox. Forget what happened to the brothers
that came from apartments."
Today. the Marist Brothers use the Payne buildings and grounds they
still own to host children each week during the summer. Some of the



kids have terminal cancer. Others are mentally retarded. Merker
continues to cook.
"It
was nice to have a big, beautiful mansion, " Merker adds, "but the
spirit of the work is the real draw."
Karen Merker was there in the
beginning. She came up with her high
school class from the St. Helena's in
the Bronx in 1972. Today Merker
teaches third grade at St. Joseph's
Elementary School in Kingston during
the week and returns to the Payne
estate every weekend to cook for
hundreds of retreat guests.
She remembers what it was like to
slave away in Colonel Payne's kitchen,
where a few amenities from a bygone
era remained - a two-floor scullery with
a spiral staircase, a dumbwaiter and an
elevator where the Brothers stored
brooms. A speaking tube that once
connected the Colonel in the dining
room to the kitchen was there as well.
''It was nice to have
a big, beautiful
mansion."
-
Karen Merker
The intricate wooden carvings upstairs in the great hall intrigued
Merker most. "I remember putting my finger in the mouth of a carved
lion and there were ridges inside," she says. "They went down to the
last detail."
Every week, Merker crossed an immense wrought-iron threshold with
her bags of groceries. The door was so heavy, she recalls, that it
literally ushered visitors inside. "I used to pretend that the mansion
was my house and that two hundred guests were coming for the
weekend' she says in her best mock-patrician voice.
When Brother Steve Martin took a group of high school kids on a field
trip to Hyde Park's Vanderbilt Mansion, the students ran to catch the
river view. '"That's our mansion," they screamed.
Due to financial considerations, the Marist Brothers sold the mansion
and the boathouse to another reclusive millionaire in the late 1980s.
The brothers moved their activities over to Colonel Payne's garage
and power plant. "The opulence seemed almost inappropriate in
today's day and age," Brother Steve Martin says. "That's not the way
we Brothers live. When I used to visit my family during the summer,
my room felt like a bandbox. Forget what happened to the brothers
that came from apartments."


Today. the Marist Brothers use the Payne buildings and grounds they
still own to host children each week during the summer. Some of the
kids have terminal cancer. Others are mentally retarded. Merker
continues to cook.
"It
was nice to have a big, beautiful mansion, " Merker adds, "but the
spirit of the work is the real draw."
The Colonel's stone barns enjoyed a brief incarnation as a restaurant
called the Creamery and a confederation of quaint giftshops called
Wildwycke Village. A handful of other schools that tried to make a go
of it in the Wiltwyck School's old quarters proved short-lived. A
microfilm service for hospitals took up residence in the Colonel's
chicken farm, but it too has relocated.
The Colonel's legacy continues to ripple down through the ages. in
an ironic footnote, the Turner painting that he gave his estranged
nephew, Harry Payne Bingham, sold for $6.4 million at auction in
1980, fetching the highest price for any work of art "I'm just
staggered," said the owner, Flora Whitney Miller, at the time.
The Whitney family continued to prosper, in part because of the
Colonel's wealth. Payne Whitney formed a renowned brokerage firm.
He and his descendants continue to give heavily to the Cornell
Medical College. Subsequent Whitneys have other success stories,
including ownership of the New York Mets.
Miss Larson was set up with millions in stocks by Payne before he
died. But she was deceived by swindlers and left only with the house
in Staatsburgh the Colonel had helped her purchase and the more
modest income Payne's heirs gave her.
The legendary Aphrodite had an even sadder fate. Not long before
the Colonel died, he offered his favorite possession to the U.S. Navy
for wartime use. She was painted grey, and the contents of her
Stanford White-designed interior were stored in a Long Island
warehouse. Vincent Astor, whose great uncle had owned Colonel
Payne's West Park estate before him, served as a junior officer on
board.
But while the Aphrodite was away, the Long Island warehouse
burned, taking the Colonel's memories with it.









Payne Greenhouse
<<===These
photos which
appeared in the
Ulster Magazine
article was supplied
by the Burroughs
family collection.
Scanned from the
magazine, as
original photos were
unavailable. ===>>
Click on images for
enlargement.
Emma Larson and
family members
Julie O'Corozine, who now lives in Rosendale, used to reside on the
Payne estate in West Park. The author wishes to note the invaluable
assistance of Dot DuMond, Esopus historian and Melinda Terpening of
the Klyne-Esopus Historical Society Museum.
most recent revision:
return to
tOP- Of P-89~
end of original article
9 August 2009.
documents list
home P-89~