The Civil War Soldiers and Sailors (CWSS) database shows three Elijah Roberts (all Privates) who fought in TN units: one in Co H, 42nd TN Infantry, a mixed state unit organized at Fort Cheatham; one in Co E, 43rd TN Infantry (also known as Gillespie's or the 5th East TN Volunteers) organized at Knoxville; and Co A, 3rd TN Cavalry (Forrest's), with the trooper also known as Elijah Robards.
I took a look at the Fold3 records and I see that the Elijah Roberts in Co E, 43rd TN (Mounted) enlisted from Rogersville, TN, on Nov 1, 1861 at age 35. He was shown absent, on sick furlough, from Jan-Apr 1862, present May-Oct, then absent without leave (AWOL) starting Oct 22 1862, then present Nov/Dec 1862 to Mar/Apr 1863, when the records end. No indication if he mustered out or died in service. According to the
Family Search webpage for this unit, Co E was formed with men from Hawkins County. Here's a website entitled
The Blue and Gray from Hawkins County, TN, 1861-1865.
The Elijah Roberts in Co H, 42nd TN Infantry enlisted at Camp Cheatham, TN, on Oct 24, 1861 at about age 25. He died on 28 Aug 1863 at Lauderdale (?) Springs Hospital, MS.
They don't appear to be the same person. If your relative came from East Tennessee--especially Hawkins County--he's likely the one listed in the 43rd TN. You might try going through that website for Hawkins County to see if you can find anything on Elijah Roberts.
Unfortunately, there isn't any good place to check for death records. If the records aren't in the soldier's service file (as on Fold3) or his gravesite listed on the FindAGrave site, or somewhere on Ancestry.com, you might have a hard time finding them. The fact that the Elijah Roberts from 43rd TN has no records after Apr '63 might indicate that he didn't survive the war, or possibly just that the records for the later period of the war didn't survive. (i.e. were lost or destroyed.)