Frustrated and overwhelmed, Jerry abandoned his wife and five children. Alone, Mallie was unable to meet the physical demands of the farm, so she moved with the children to join her brother in Pasadena, California.

While life in California was an improvement over their impoverished southern life, the Robinsons lived in a cramped apartment, at times without enough food, and eventually had to turn to welfare. The family also had to suffer the onslaught of their white neighbors' verbal abuse and repeated attempts to drive them from their home. But, Mallie Robinson refused to give in to their racism. Through it all, Jackie said, they could rely on their mother's strength, love and attention.

Jackie's childhood was not exceptional. He was a fair student and held several part time jobs. However, for a while, he was involved in petty crimes with the Pepper Street gang. This didn't last long because he received "big brotherly" encouragement from Carl Anderson, a local mechanic and Reverend Karl Downs. Both men were able to refocus his energies in more positive pursuits-- athletics.