The image is reversed I would say. It is common to see Civil War era photos that are of the negative and therefore reversed. This was what led to the popular story that Billy the Kid was left-handed (he wasn't, the image was just reversed). Not all period photographs we see are reversed, but it is definitely something to pay attention to when viewing an image.
Here is some evidence for this:
The Sharps Carbine hammer is on the right side of the gun.
View attachment 161242
In the photograph, it is on the left. This is a good constant to work off of. The hammer will always be on the right side. Whenever weapons are in the photograph, I use the hammer to determine if the image is reversed or not. The sharps in this photograph indicates that it is a reversed image.
Words are also helpful, but there don't seem to be any in this photograph (such as a US or CS belt plate).
So he is actually holding the rifle in his left hand and the knife in his right. Since left-handedness is not only the minority of the population but was also at the time (and for long before and after) associated with the devil, it makes sense that he would brandish a knife and pistol in his dominant, right hand.