"The slaveholder's life is a life of utter and perpetual injustice. The worst wrongs to which men are subject from their fellow man, he is day by day inflicting."
--Papers on the Slave Power, 1846
Palfrey knew from his own experience what it was to be a slave-holder. His father and two brothers had prospered in Louisiana as plantation owners. Under Louisiana law, he would inherit one-third of his father's estate, including slaves. Although he loved his family and would never have considered himself a radical abolitionist, he regarded "the claim of property in human beings (however the same may be sanctioned by human laws) as utterly null & void before God." He began, even before his father's death, to take "the needful steps to divest myself of my legal relation to said slaves, & to grant them their unconditional emancipation." He hired a lawyer and petitioned the Louisiana state legislature in order to free those under thirty. Before their official emancipation, he paid their wages. Although a few of the older house servants remained in Louisiana, Palfrey paid for the "comfortable passage" in 1845 of sixteen former slaves to Massachusetts. After a ceremony in King's Chapel to formalize their emancipation, he arranged positions for them as servants in "suitable households," mostly in Massachusetts.
While Palfrey had wanted to keep his actions private, his friends were proud and his story was used as an example by the growing abolitionist movement.