Another Brick Wall - William Marcus Jeffrey

My great-grandfather, William Marcus Jeffery isn't so much a mystery, until, what we assume was his death.

• He was born in  1852 in Caldwell county, Texas, probably in Prairie Lea. The family appears on the 1860 census for Prairie Lea in Caldwell county. William was 8 years old.
• In 1870 the family was living in Crockett, in Houston county Texas. William was 18. 
• He's missing from the 1880 census where the family is listed living with Dr. George Forest. Will's father died sometime before this census was taken and his mother, Minerva remarried.
• There is a William  Jeffrey living in Milam county in 1880 with a Harvey family as a laborer. He is 28 years old which matches our William. 
• There is no surviving census record for 1890 since all the records were destroyed in a fire. 
• The next location for William in in Oakwood Texas (In both Freestone and Leon Counties) is marked by the birth of his first child Daisy in 1892 with Ellen Crane. 
• In September of 1894, Mattie Lena Jeffrey was born in either Leon or Montgomery county Texas.
• I found a marriage record for the marriage of William M. Jeffrey and Ellen Crain in Anderson county, Texas on October of 1894. Will's brother Andrew was living in Palestine (Anderson county) at the time.
• In June of 1900 the census for Precinct 1 Montgomery County Texas lists William, Ellen and children - Mannie and Maude listed along with Daisy and Mattie (it states that they lived "west of the R. R. track" which is not far from where I am right now.)
• The 1910 Huntsville, Walker County census lists Albert Moore, wife Ellen and 3 daughters.
• Ellen married Albert Moore in May of 1902. Mannie had disappeared, we have to think she died. No record for William Jeffrey. I have to assume he died between June of 1900 and May of 1902 and Ellen went to live with relatives in Huntsville where she met and married Albert.

I have not been able to find any death records for William or Mannie or where they might be buried. How can someone so close be so lost?
I have visited some cemeteries in the area. The Willis cemetery has a big field that I was told was full of unmarked graves.

Note: Albert worked as a guard at the Huntsville Prison. My dad remembers visiting them there.