He studied at the Brooklyn Museum Art School and the Skowhegan School of Painting and Sculpture. After graduation, Williams left the United States for Denmark. The move was perhaps inspired by his grandfather who was from the Danish West Indies. Williams was captivated by the intrinsically beautiful island of Bornholm in Denmark and in later years, he returned to the scenic environment there.

Free from the stifling bigotry that he experienced in the United States, Williams "felt the freedom from racial prejudice was essential for his further development." After leaving Denmark, he traveled again—this time to Mexico from 1959-1963. Williams’ work was well-received there and he was included in several exhibitions, such as the Instituto Nacional de Bellas Artes in Mexico City. Williams’ eventual return to the U.S. was short-lived. He traveled back and forth between the U.S. and Mexico before making a second trip to (Copenhagen) Denmark, ultimately leading to his decision to become a Scandinavian citizen in 1979.

Williams' artwork, which covers his experience in the United States and abroad, is marked by Southern genre scenes that reflected ‘the experiences of being an African American living in the South” and by his idyllic views of lush Danish landscaped environments—awash with children playing in colorful fields of flowers. Paradoxically, one re-current motif or theme in many of these images is that of a Black, overturned tree-trunk with tentacle like roots. This is perhaps a salient reminder of the artist’s southern roots.

One of Williams’ most notable efforts was a show that he curated in Denmark. Ten American Negro Artists Living and Working in Europe featured expatriate artists such as Beauford Delaney, Sam Middleton, and Herbert Gentry. Despite his success abroad, returned to the United States in 1965, only to find that the racial animus had changed very little. His interracial marriage was not well received in some communities. Accustomed to a more welcoming atmosphere in Denmark, Williams returned Copenhagen, where his show at the Noa Gallery nearly sold out. His unsold works remained in Danish collections.

His "Girl With Butterflies #2" was purchased by the National Collection of Fine Arts of the Smithsonian Institution in 1965 for the Executive Wing of the White House—making him one of the few black artists to achieve that distinction. However, it was not enough to make him call the United States home again. Williams spent his final years in Copenhagen. In 1980, his studio was destroyed by fire, losing all his paintings and prints.By 1983 he stopped making art altogether.

Williams had become somewhat reclusive by the time of his death. Nevertheless, over the life of his career he achieved an international following that remains a part of his legacy as a Black artist.