Many of the good ideas below came in talking about personal information with Doug Fritz, the bad parts are my own ideas.
When Friendster and Flickr started getting traction (2002 and 2004 respectively), I wasn't a huge fan of the functionality, but the social aspect was totally addictive. I got seriously into each (and even got comfortable with the photostream concept eventually, though the fact that I can't easily control the order of the photos without using the organizr still bothers me). These sites ability to connect me with other people blew away all other shortcomings. At the time, the fact that I was putting so much personal information into a public space didn't bother me at all; I assumed a that the Web made old ideas of privacy kind of obsolete, and we would all eventually adapt and make new social norms to cope.
But now that social networks are taking on large chunks of people, I am not so sure. Facebook with a dozen real friends is a much different thing than Facebook with 140 "friends." The information I post there is, to me, the things that define me personally as I choose to represent myself. In the social network context however, the sites ultimately have more control over how I get represented than I do.
I had an experience with that when my story about a stolen cellphone and pictures from the thief unintentionally uploaded to Flickr briefly became news. There were some wrinkles in the story that caused people to be suspicious that I was doing marketing, or lying about the phone being stolen (as opposed to lost). I knew the facts of the matter and was not looking for any large amount of attention in posting the story. But once there was that attention, the people who read the story simply did not believe it and couldn't be convinced otherwise. They used personal information about me that was publicly available (on Flickr, my blog, and other sites) to make the case that I was perpetrating a hoax (for example the fact that I worked at Yahoo was used to say that I was marketing the Flickr service). Even the Reuters reporter that interviewed me asked "how can I know you're not deceiving me?" I didn't have an answer for her. The effortless flow of personal information (flow that I started) did not lead to anything like the truth, and caused some people to think that I was not a real person at all. Privacy as we knew may be gone, but the idea that everyone is now a public figure (with none of the protections of public figures) strikes me as wrong.
We can say that in the context of computer networks "information wants to be free," and I support the idea that copyright is an outmoded framework for intellectual property. But when it comes to representing ourselves, how can it be that personal information should flow everywhere and be used by anyone however they want? Open Social and the Data Portability initiatives are good starts for independent mechanisms, but I still have to give these systems a truckload of personal information, with no way to take it back after it's out there.
An alternate solution would be to allow people to own their personal information store, and choose to allow social network sites access to this store. Sites that behaved badly could be banned. This is much like OpenID and Oauth in concept, where one's identity is tied to a DNS-like way of creating a single namespace for unique user identifiers. It could take the form of a fancier version of an "Attention Profile Markup Language" file; a "Social Profile Markup Language" file, say. It would be stored on my own web server and under my direct control. If I wanted to share with Friendfeed or mybloglog (for example) what sites I've been posting to, saving, liking, or reading, I could allow them to access my SPML file under the condition that it be removed if I decided not to use the application any longer. (This is a geeky solution, but that's usually where these things start.) There should be a better solution to the new portability of social data than exists today, or my own understanding of my personal information will mean less and less.