Is it possible to assign an Item layout to each Category?

For example, for Category A, I want to assign a certain item layout, which may include the Additional Info fields in a different location. For Category B, I might want to assign a different location for the image gallery and a different location for my Additional Info fields.

I basically want to create different item layouts for each category. Thanks in advance.

Yes, you can do that, goto simon.getk2.org and search for "templating"
Some of the things will be done by setting "item view" paramaters for the seperate categories and some of the things will be done as template overrides that are selected in the category paramaterss