What you are describing is clearly one path towards better performance. The system would need to deal with parallax, the effect caused by the different tubes being mounted in different locations and thus seeing things positioned slightly differently. You would also need some sort of driver board that would combine the information and decide what to show you for any given point. I suspect that a visor type display would give you more room for a high quality image and allow you to look under the display to see the dash instruments to operate a vehicle. Once you had a system that could do these tasks you could mix and match your inputs for the best results. I would start out with a combination like:
A starlight scope. there's usually a little visible light around to work with. This would give you a view you're familiar with and some resolution.
A thermal infrared sensor. This would show living things nicely and work even with no visible light availablilty but have poor resolution.
