The most likely origin of the West- in Westbury is simply that the town is near the western edge of the county of Wiltshire, the bounds of which have been much the same since the Anglo-Saxon period.
The -bury part of the name is a form of borough, which has cognates in many languages, such as the German-burg and the Greek-pyrgos. It carries the idea of a hill or fortified town. For a fuller explanation, see borough. In Wiltshire, -bury often indicates an Iron Age or Bronze Age fortified hill fort, and such a site is to be found immediately above the Westbury White Horse.