Aeric Maelstrom
Once, I encountered the funny story of an AI image descriptor with a sheep obsession. It had been trained on pictures of fields of sheep. Therefore, it tagged anything in a field as 'sheep', including an empty field, because they work on statistical probability. Therefore, it thinks "ah, a field! there's probably a sheep here." (It's a bit more complicated but basically that.) It also couldn't recognise sheep in places that weren't fields, such as petrol stations or barns. [cont]






Now, the alarming aspect of this story is that the very same technology is probably what tumblr is using to identify porn. Now, if it can’t tell that an empty field is not, in fact, full of sheep, what hope do we have that it can’t tell an empty room isn’t full of writing human forms engaged in passionate coitus?

this really does sound like an episode of black mirror

But wait, it’s even weirder than that!


This is gonna produce some absolutely baffling pornography.

…. oh my fucking god they actually are using open source software. They’re using a fucking one-layer unidirectional bicategory tag-trained neural network. This will never work. Literally, it will never work. There’s just not enough algorithmic complexity to do what they’re asking of it. I bet you I could prove on a mathematical level that this joke of a neural net fundamentally lacks the abstraction necessary to do its job.

This will never get better. Their algorithm will never stop fucking up, it will never actually flag porn reliably and it will always require a massive quantity of human hours to deal with the deluge of mistagged pictures. This isn’t just a case of an insufficiently trained algorithm, it’s just … this is the most basic neural network you can make. It probably hasa a lot of neurons and has loads of training data but like … you can’t just brute force this kind of stuff. One layer of neurons is just Not Enough.

Also, just to make this clear, Tumblr lied. I mean, we already know this, but I mean they liiiieeeeed. All that stuff they promised about what would or would not be censored? That cannot be delivered on with a system this simple. Nude classical sculptures, political protests, male-presenting nipples (really Tumblr?), nude art outside the context of sex, all that? You cannot train a bicategory one-layer neural network to exclude those things. It cannot be done. Tumblr never intended for those things to actually be permitted, they were just lying. Because the system they have cannot actually do what they said it would and never will be able to.

Also, this kind of system is super vulnerable to counter-neural strategies. I bet you before the end of the month someone hooks up their own open source one layer bicategory neural network which puts an imperceptible (to humans) layer of patterned static over arbitrary images, and trains it by having it bot-post static-ed images to Tumblr and reinforcing based on whether the images are labeled nsfw or sfw. Seriously, within a month someone will have an input-output machine which can turn any image ‘sfw’ in Tumblr’s eyes.

This is genuinely pathetic. Like, I have real pity for whoever implemented this, because it’s clear Tumblr doesn’t actually have any engineers with any expertise with machine learning left at all and they foisted the job off on some poor bastard who has no idea what they’re doing and is going to get all kinds of flak for their (perfectly reasonable and predetermined) failure from management.

1 week ago with 62,044 notes Via Source (+)
  1. theladysunami reblogged this from shipping-isnt-morality
  2. foldedantennae reblogged this from disturbingelephanteyes
  3. vetchshadow-4thwall reblogged this from adeadfreelancer
  4. my-bts-heart reblogged this from draumbooty
  5. cheskitten-personal reblogged this from tyrantofthefirmament
  6. plumppeachprincess reblogged this from tomcat-g
  7. disturbingelephanteyes reblogged this from brontozaurus
  8. ausefulblogforputtingthingsin reblogged this from nyardynn
  9. tyrantofthefirmament reblogged this from nyardynn
  10. nanatei reblogged this from sarahthecoat
  11. stabbitytuesday reblogged this from a-wooden-ring
  12. llaevateinn reblogged this from aughtpunk
  13. breezy-day reblogged this from neurolept
  14. slugdge-boy reblogged this from kimbergoat
  15. iguana-america reblogged this from mirron91
  16. littleredhotsridinghood reblogged this from seven-oomen
  17. awkwardbystander reblogged this from windypenguin
  18. nyardynn reblogged this from hongvanngh
  19. winterrcrow reblogged this from gaslightgallows
  20. tomcat-g reblogged this from v0mpy
  21. fromkansaswithlovesstuff reblogged this from thefriendlypigeon
  22. pulpo-fiction reblogged this from jajachaik
  23. betery reblogged this from aughtpunk
  24. nelkitty reblogged this from annawry and added:
    Maybe they’re intentionally making it a fucking disaster in the hopes we’ll yell and scream enough to overthrow the laws...
  25. lauralot89 posted this