The best place to go to buy tools is a good hardware store.
I would begin by thinking of the literal definitions of modify and restrict. When something is modified or restricted, it changes.
If a home is modified, it is changed. If a policy is restricted, something changes (i.e., the total number of things you can do changes (my becoming smaller).
The same thing happens in language. When a word or phrase is modified or restricted, our concept of that word or phrase changes from lots of possibilities to fewer possibilities. For example,
In “My brother is an attorney,” the concept of brother is restricted only by how many brothers I have.
In “My brother who lives in Ohio is an attorney,” who lives in Ohio narrows the concept of which bother I’m referring to to one.
In “The best place to go to buy tools is a good hardware store,” to go restricts the our concept of place. It is not a place to eat or to swim. It is a place to go.
So if to go modifies our concept of place. And, if place is a noun, then to go is an adjective.
To buy tools narrow our concept of to go because it tells us why, not when or how. If something modifies our concept of to go by narrowing it to why we are going, it must be adverbial.
This way of thinking focuses on content or meaning rather than on position or part of speech. However, it provides one additional way of analyzing sentences that pure syntax doesn’t.
Hope this help. If not, pay it no attention.