Software scans your Facebook PHOTOS to work out if you're a hipster, goth or a surfer - and customises the adverts you see

  • The software uses what’s called a ‘multi-label classification algorithm’
  • It identifies elements in Facebook photos associated with select sub-cultures
  • These sub-cultures included bikers, goths, hipsters and surfers
  • The algorithm then scans other photos looking for these elements
  • Websites and advertisers could use this information to customise what these people see online

By Victoria Woollaston

PUBLISHED: 08:51 EST, 18 December 2013 | UPDATED: 09:22 EST, 18 December 2013

Your Facebook posts and the pages you like already give away a lot about you including what music you listen to, or your favourite restaurant.

But now researchers are using people’s Facebook photos to identify what sub-culture they belong to, from goths and bikers to hipsters and surfers.

They have created an algorithm that can identify certain elements associated with each of these groups, such as distinctive tattoos, hats and jewellery, before scanning other photos to find similar people.

By scanning photos for elements commonly associated with certain sub-cultures, including bikers, hipsters, goths, and surfers, researchers could categorise Facebook photos into each of these groups. The results could one day be used by advertisers to customise ads online

By scanning photos for elements commonly associated with certain sub-cultures, including bikers, hipsters, goths, and surfers, researchers could categorise Facebook photos into each of these groups. The results could one day be used by advertisers to customise ads online

HOW DOES THE SOFTWARE WORK?

Researchers used a so-called ‘multi-label classification algorithm’.

The algorithm scans photos that have already been tagged, and looks for identifying features. For example, if a picture is tagged as ‘dog’, it will pick out the shapes and sizes of objects in the image.

It will then scan other photos looking for similar features and elements that match, or closely resemble, the original dog.

The software assumes that the new image contains similar elements as the original image and tags it with the same label.

This is called a ‘parts and attributes’ approach. 

The University of California researchers used this model to scan photos of elements most commonly associated with different sub-cultures. 

It started by scanning the head, neck and arms for tattoos. It then checked for certain haircuts or jewellery, as well as the type of clothing and the colours the subjects wore.

These results could then be used by advertisers and websites to customise the type of content these people see, or the products they’re more likely to buy.

To do this, researchers from the University of California used a so-called ‘multi-label classification algorithm’. It is already used by certain photo editing software, for example, to help tag and categorise images.

 

More...

The algorithm scans photos that have already been tagged, and looks for identifying features from each of these different groups.

It started by scanning the head, neck and arms of the sample images for tattoos.

It then checked for certain haircuts or jewellery, as well as the type of clothing and the colours the subjects wore, as well as poses.

According to the algorithm, hipsters, for example, may wear glasses, have longer hair and beards.

Photos of Goths may predominantly feature the colour black and there may be tattoos or piercings.

Bikers are either stood near bikes, wearing bandanas or certain types of jackets or are in more urban environments.

Whereas surfers stand in more relaxed poses, for example, and wear shirts over T-shirts as well as caps.

The software uses a 'multi-label classification' algorithm that is often used in photo-editing programs to help tag and organise images. The green images pictured are those correctly identified as (from top to bottom) goths, formal and bikers. The elements in the red images don't match these respective groups

The software uses a 'multi-label classification' algorithm that is often used in photo-editing programs to help tag and organise images. The green images pictured are those correctly identified as (from top to bottom) goths, formal and bikers. The elements in the red images don't match these respective groups

THE ATTRIBUTES OF EACH GROUP

The software begins by scanning the head, neck and arms of the sample images for tattoos.

It then checks for certain haircuts or jewellery, as well as the type of clothing and the colours the subjects are wearing, as well as their poses.

According to the algorithm, hipsters, for example, may wear glasses, have longer hair and beards.

Photos of goths may predominantly feature the colour black and there may be tattoos or piercings.

Bikers are either stood near bikes, wearing bandanas or certain types of jackets, or are in more urban environments.

Surfers stand in more relaxed poses, for example, and wear shirts over T-shirts as well as caps.

They also looked at identifying ‘Formal photos’ by suits and long dresses, and the way groups of people pose in quite rigid, uniform stances.

They also looked at identifying ‘Formal photos’ by suits and long dresses, and the way groups of people pose in quite rigid, uniform stances.

Unfortunately, the researchers claim that the algorithm is only accurate in 48 per cent of cases and only correctly identified groups in 9 per cent.

This means that it is unlikely to be used by websites any time soon without the software being enhanced.

According to members of the Urban Tribes project: 'Image sharing via social networks has produced exciting opportunities for the computer vision community in areas including face, text, product and scene recognition.

'People can guess plenty of implicit information from the visual aspect of a group of people, but what can we automatically determine about the social subculture to which these people may belong?

'We propose a framework that integrates state-of-the art person and face detection and uses low- and mid-level features to capture the visual attributes distinctive to a variety of social groups.'

Comments (28)

Share what you think

 View all

The comments below have not been moderated.

Are goths still around? I thought that fad ended with the 90's.

0
1
 Click to rate

I could tell them that just by looking at 1 photo!!

0
1
 Click to rate

Oh dear... if I ever did use Facebook, which I don't, I'd be deliberately posting misleading photos just so I could screw up their image scanning system. The scary thing is that there's no reason why they can't just move to full on facial recognition software... and this is reasonably accurate - but there are ways of fooling even this. Beats me why anyone would want to use this ever more intrusive piece of software especially when it's probably hotwired to the NSA and through them to GCHQ.

0
0
 Click to rate

Im so glad that my soft ware blocks ALL adverts on every page I ever visit

1
1
 Click to rate

I don't really see the problem with this, at least now you get to see adverts you might actually find interesting.

3
0
 Click to rate

Whaaat?? Facebook has adverts? *adblock alert*

0
3
 Click to rate

Most people I know on Facebook have deleted many of the photos they previously had posted.

0
8
 Click to rate

Facebook keep sending me ads for some bunion plaster. I'll bear it in mind, should I need it in 20 years time.

0
3
 Click to rate

im confused, my pictures mostly consist of anime and animals with funny faces. the ads i saw were of games and product testers etc. should sue facebook for advertising scam companies

3
3
 Click to rate

Use - Ad block Plus Google how to use it, takes about 20 seconds to sort out and you'll never have adverts in FB again.

1
9
 Click to rate

The views expressed in the contents above are those of our users and do not necessarily reflect the views of MailOnline.

Add your comment

By posting your comment you agree to our house rules.

Who is this week's top commenter? Find out now