The Papa Stour sword dance is unique in Shetland, possibly of Norse origin and related to the long sword dance of north east England. Sir Walter Scott vividly describes the dance in his famous novel, "The Pirate."

The story of the prisoner of Papa Stour concerns the Honourable Edwin Lindsay, an officer in the Indian army , who was banished to Gardie House by his father Earl Balcarres, who considered his son dishonoured after refusing to fight a duel. His plight was made known to a lady missionary who was visiting the island and in 1835 he was released.

The Maiden or Frau stack is a single perpendicular rock to the north east of Housa Voe on top of which can be seen a metre-high structure built of stone. The legend associated with this structure concerns a Norwegian maiden who was confined there by her father to keep her from an unsuitable suitor. However the young man, though poor, was nonetheless brave and rescued his fair damsel. And so they eloped together and lived happily ever after.

There is the site of a Ting at the head of Housa Voe; which consisted of a circle of at least 46 stones and was the site of Norse meetings or assemblies convened to settle disputes. The circle appears in later island folklore with a story concerning Lord Tervil who was involved in a duel there.

The ruins of huts that according to tradition were the final refuge of lepers can be found along the slopes of the Hill of Fielie at the south western end of the airstrip. At one time people who were considered to have leprosy were sent to Papa from all over the west side of Shetland . Now however it is thought that these unfortunate folk did not suffer from the disease at all: but from malnutrition and vitamin deficiency.