I've noticed that when I add captions and credits to a photo that it shows up on the left and to the right of the photo. How can I get this text to appear at the bottom of the photo?

Hi. By default they do appear at the bottom. So something in your template made them go on the side of the images. You need to check your template's HTML and CSS code.