Cornelius was born into slavery at Tuckahoe on December 29, 1856. His parents were Tom and Amy Anderson and his siblings were Maria, Abner, Philip, and Willis. All of the family were listed as field hands.
After the Civil War, it seems that some of the Anderson family stayed in the area. Cornelius’ mother had died prior to 1866. By the 1880 census, Tom Anderson is listed as living alone so it is unknown whether Cornelius and his siblings still lived nearby or had left the area entirely.