As the subject suggest, I would like to make a page that would display all available K2 tags, is that possible?

The tag cloud can do that if I set it to a very high number but I was hoping for another alternative that would better display all the tags in a neater look.

What would be even more ideal is if I can display the tags by categories, so it would be like a site map of some sort.

Hi. This kind of functionality is not included in K2. Your only option is the mod_k2_tools tag cloud. You can also make a template override to the module and make it look exactly as you wish.