Born in 1840 at Hechingen, Hohenzollern to Jewish parents, Ullman immigrated with his family to America to escape discrimination at the age of eleven. The Ullman family settled in Port Gibson, Mississippi. After briefly serving in the Confederate Army, he became a resident of Natchez, Mississippi. There, Ullman married, started a business, served as a city alderman, and was a member of the local board of education.

In 1884, Ullman moved to the young city of Birmingham, Alabama, and was immediately placed on the city's first board of education.

During his eighteen years of service, he advocated educational benefits for black children similar to those provided for whites. In addition to his numerous community activities, Ullman also served as president and then lay rabbi of the city's reform congregation at Temple Emanu-El. Often controversial but always respected, Ullman left his mark on the religious, educational, and community life of Natchez and Birmingham.

In his retirement, Ullman found more time for one of his favorite passions - writing letters, essays and poetry. His poems and poetic essays cover subjects as varied as love, nature, religion, family, the hurried lifestyle of a friend, and living "young." It was General Douglas MacArthur who facilitated Ullman's popularity as a poet - he hung a framed copy of a version of Ullman's poem "Youth" on the wall of his office in Tokyo and often quoted from the poem in his speeches. Through MacArthur's influence, the people of Japan discovered "Youth" and became curious about the poem's author.

In 1924, Ullman died in Birmingham, Alabama.

In 1994, the University of Alabama at Birmingham and the Japan-America Society of Alabama opened the Samuel Ullman Museum in Birmingham's Southside neighborhood. The museum is located in the former Ullman residence and is operated by the University of Alabama at Birmingham.