Funny you should ask as we have been working on updating the style guide.
Our registered trademark does have the Bang (!) and for improved branding recognition, The Joomla! Project does ask that you use it in prominent locations such as headings and in graphics. We agree that the bang gets cumbersome in text.
If you are writing for JCM, the editors are pretty strict about using the bang whenever practical. It is not practical if there is any other punctuation needed like a question mark or a comma.
In general, we have adopted the approach that in body text the bang should be used at least once at or near the first instance in a way that acknowledges it as a trademark, which means with the bang and the circle-R indicia in superscript. Refer to Joomla!® like this the first time and then to Joomla with or without the bang and without the indicia from then on.
There is an exception. There are a number of other trademarks and service marks associated with The Joomla! Project and we do ask that you ALWAYS use use the bang and capitalize the initials when the word Joomla is part of a proper name for a product or service such as the Joomla! Community Magazine, the Joomla! Framework or the recent release of Joomla! 3.3 last month. In these instances it is helpful to add the superscript TM indicia the first time a product or service name appears on a page.
Do not use the bang in keywords, SEF URLs / aliases or other places where search in third party websites or the ability to resolve the URL will be thrown off by the punctuation, such as in Linked-In.
Of course, these are guidelines, and the branding police will not be knocking at your door, but whatever you can do to consistently support our trademarks and keep them distinctive benefits everyone.
Thanks for asking,