Sidney B. Williams, the brother of Walter Baker’s second wife Eleanor, came from Philadelphia to begin working at Baker’s Chocolate in 1843. Williams had communicated to his sister that his work in Philadelphia was “extremely dull,” since his employers gave him little to do. At the time, Walter Baker found himself without a clerk, so he hired Williams initially as his personal assistant and replacement clerk. Walter’s plan for Williams was for him to learn everything about the company from the ground up so that he could manage the entire operation without Walter present. The agreement between Williams and Walter Baker began with $200 plus room and board for the first year, which gradually increased to $2,000 by the fifth year, and then one third of the company’s net profits after Walter retired. The only expectation in return was that Williams would pay Walter $15,000 if he ever decided to leave Baker’s in order to start a chocolate business on his own. This restriction resulted from a painful lesson Walter learned when he suspected his former clerk, John Mott, was collaborating with the Webb & Twombley Chocolate company. By all accounts, Williams quickly accepted Walter's offer and began working for him a few weeks later.
Nothing is known of how Williams conducted business during his time working for Walter Baker. His near decade of experience was put to the test, however, when Walter Baker died suddenly on May 7, 1852, at the age of fifty-nine. Under the terms of the will, the mills were turned over to the trustees of Walter Baker’s estate, who then leased the management of the company to Williams. Under the terms of the lease agreement Williams was to continue operations under the name of Walter Baker & Company.
Unfortunately, two years later on July 4, 1854, Sidney Williams died while on business in Montreal, Canada. The trustees then leased the company to Walter Bakers’ step-nephew, Henry L. Pierce, who had only worked at the company for a few years.