French articles for monuments?

I know that you would use prepositions when you talk about the Eiffel Tower, like "Je visite la tour Eiffel." But how about other monuments like the Gravensteen in Ghent, Belgium or other monuments. How would you decide on the gender?