xkcd #576

EDIT: Some of the information here is now out of date. See here for the changes.

Recently TradeMe released their API. This is pretty great, if only because it means I can finally give up maintaining the increasingly-complicated set of scraper scripts that I’d used to do TradeMe stuff up to this point. After converting the basics of metatrade.co.nz over to the API, I was looking for a new TM-related project, and I happened to stumble across this old comic from XKCD:

XKCD: "Packages"

Google didn’t reveal anyone doing this already, and after asking a few people, the general opinion was that doing this was a terrible idea that would inevitably end badly. So obviously I had to do it.

I initially ran into a couple small problems with the API, but as of this afternoon, the system is working! It’s on Twitter at @trademe_xkcd576, so everyone can see what it does. The first thing it automatically bought:

Bought '~ 5X Maxell SR626SW batteries BEST ...' for $2.00! - http://bit.ly/b8vRjJ
@trademe_xkcd576
trademebot.py

Problem solved!

I’ve had a bunch of people ask me how the script works. So here’s some info about the inner workings:

  • It’s using Python and python-oauth2. For anyone looking to do TM stuff with python, the code examples on that page basically just worked.
  • It runs every day at 8pm (although it was earlier today because I was testing it)
  • It gains $1 every day, and has a 1 in 3 chance of buying an item on any particular day. This means that it will save up money to buy some (slightly) more expensive items.

The method it uses to select items:

  • It has a bunch of top-level categories it looks in.
  • For each of these categories, it searches for the term “Free shipping”, specifying both pay-now and buy-now, sorting by newest listings, with a maximum of 100 items returned per category.
  • For each of these items, it filters on buy-now price. It tries to spend at least 50% of its savings.
  • For each of the surviving items, it looks up the individual auction details to find its shipping information so it can filter on free shipping. Despite searching for the term ‘free shipping’ to start, only a small number of items have this.
  • At this point I have a list of items that match the price requirements, and can be bought with a credit card buy-now.
  • I then sort this list by ‘rarity’ – doing a search for the item title, and finding the item that returns the least results. As the objective here is to buy strange and esoteric things, rarity is preferred.
  • Finally I buy the rarest item and subtract its cost from the bots savings.

There’s a bunch of improvements that could be made here, but I figure it’s good enough for the moment. I’ll keep an eye on it and see what happens.

Hopefully in a week or so I can post a “here is my collection of random stuff that’s turned up in the post” photo :)

Trackback

126 comments untill now

  1. Wow, I had no idea Trademe released their API! This is pretty huge news on their part IMO, they’ve stalled on releasing an API forever.

    Looking forward to the results of your bot :)

  2. [...] This post was mentioned on Twitter by Jonathan Oxer, Simon Lyall, Chris McDowall, Douglas, Joanna and others. Joanna said: What happens when it becomes self-aware? RT @fogonwater: TradeMe API xkcd gold from @paul_hunkin http://bieh.net/2010/11/08/xkcd-576/ [...]

  3. So it handles payment and shipping automatically (as in xkcd)? ie fully automated?

  4. This is great! Beautiful, even.

    Monetize it! I bet a ton of people would sign up for a service like this, especially if they could pick the categories and perhaps make it so its not necessarily so esoteric.

    Rock on.

  5. Sounds like a lot of fun and a great way to explore ebay’s api. I think an interesting next step would be to find items you could resell for a profit.

  6. Simply awesome. I waited for this since the xkcd was released.

  7. If you could find some way to sell those things for more then you bought them that would be sweet.

  8. Cool implementation of a bad idea!

    I might use this to buy random crappy Christmas presents as opposed to just crappy ones.

  9. I’ve found some great $1 items on ebay by searching for “USB” with free shipping. Lots of various cables, memory stick readers, bluetooth, audio adapters, etc. I’ve got a whole tupperware full of them now. They take 2 weeks to show up, but if you do one a day, you get a surprise everyday in the mail eventually.

  10. You are my hero

  11. Nice one,

    I hope you will not up with some debts ;-)

  12. Robert K @ 2010-11-09 04:47

    Dumb question(?) – do you have any logic to prevent it from buying the same thing twice? That the bot has to save up money to find something worth buying would seem to indicate this is a reasonable concern.

  13. spriggig @ 2010-11-09 04:55

    I want this.

  14. So, where’s the source? :D

  15. You are insane. I love it! Can’t wait to see the results.

  16. Ever since I saw this xkcd about a few months ago I thought the idea was brilliant, and you’ve even improved on it! Is there anyway that the common man can get a hold of this script?

  17. Dave Babbitt @ 2010-11-09 05:37

    I did this about 5 years ago using ColdFusion, though it wasn’t as complex as yours. Got some really interesting stuff!

  18. Wow… I wonder if you start buying risky stuff that would land you on the FBI list. LOL

  19. James Jackson @ 2010-11-09 05:38

    I will have to keep an eye on this thread :)

  20. You’re going to end up on every watchlist eer.

  21. Pure AWESOME

  22. That is awesome. So awesome.

  23. this is brilliant definitely gonna follow the script on twitter , nice job

  24. I’m sorry but I don’t understand from your post how it “gains $1 every day”.
    You only buy stuff, right? Or do you sell items as well?

  25. Awesome idea, would like to see the photo. I’m really interested in what type of stuff will turn up at your door step.

  26. Hopeful Guy @ 2010-11-09 06:34

    Please, please, please, please, tell me that you intend to release this so I can start collecting junk, too!

  27. Wow, that’s incredible! Good show.

  28. Holy cow, this is excellent. Anyway you could set this up as a service for the rest of us? I’d gladly spend a few bucks a week to get random, rare stuff in the post.

    BRILLIANT!

  29. is there a way to make it sell the things it purchased the same way but with say +2 bucks? if it worked like that you can actually get money every few days I bet you could get like 20 bucks a week for free… see now this sounds interesting

    1. Buy cheap useless junk
    2. Automatically sell it again for slightly more.
    3. Say this happens about 10-15 times a day
    4. ????
    5. Profit!

  30. You sir, win. “What do I win?” you ask, a nervous but excited quiver just barely detectable in your voice. “Why, you win absolutely everything my good man” is the immediate response.

    Yes, you win everything.

  31. Awesome idea to implement this, looking forward to see how the purchasing goes.

  32. This is a beautiful idea.

    Is there a site w/ pics?

  33. Stephanie @ 2010-11-09 06:59

    This is awesome! I’m really excited to see a catalog of the stuff you get.

  34. You, sir, are amazing. I look forward to following the results of this project.

  35. God I love Api’s and the internet.

  36. Wow! Great work Paul – hopefully you will rewarded with some decent stuff

  37. [...] jetzt hat ein fin­di­ger Ent­wick­ler den oben ange­spro­che­nen Algorhyt­mus umge­setzt und erstei­gert jetzt jeden Tag 1$ Schnäpp­chen auf eBay. Worum es sich gerade han­delt [...]

  38. [...] by lucaDuval to programming [link] [95 comments] reddit: the voice of the internet — news before it happens @ [...]

  39. Awesome idea, I was wanting to do an implementation of it as well but I didn’t have enough cash to put behind it. I suppose if I did get around to doing it, it’d just be a PHP cron job. Mm, maybe after Christmas I’ll give it a shot. Looking forward to seeing what you get out of it!

  40. As a platinum powerseller on eBay let me recommend a couple things to you.

    First – I would recommend you utilize the best match feature as a 50% alternative to rarity. These are items that people have bought and are happy with.

    Rarity on eBay in the $1 range, generally means proprietary and useless. Think a gear for a FSR294 model CD player. Especially shipped. That generally means something mass produced, ordered by a seller that has no demand. Generally. Maybe I’ll be wrong, but that’s my feel on the market for $1 & “rare.” If its $1 and people want to sell it (taking the time and effort to pack and ship etc) its mass produced. Meaning they can get it for $.10 or less.

    From what I understand this is increasing to $2 and then $3 and then $4 and then using the odds to create the $365 a year formula?

    That’s a good idea. I just think the rarity function is probably the issue in your script. I think its a great implementation, and I applaud your internet tomfoolery. I would just try hedging with the best match and see what works the best.

    I would think halfway useful and cool stuff is probably going to trump useless. Looking at the first purchase I come to a website sr626sw.net with some basic information. Again, this leads me to my first thought, niche, very proprietary and specific stuff that can be mass produced.

    This probably most closely mirrors the xkcd comic, however, if you really want this to break out and be something you can run for the next couple years bringing you massive amounts of unintended happiness, I would use that.

    Second – I would recommend filtering feedback below 99%. Exponentially increases your odds of getting items, getting the items described etc. Since low feedback sellers selling low dollar items have a much higher percentage of people who don’t really care about sending a part out if they don’t have it, or having it match the feedback.

    Lastly – Since this is the XKCD crowd, how about posting some of the data that the botscript is processing to come to its purchasing decision. This can lead to an increase in its usefulness.

    I don’t think you’ll be the last person to do something like this. Its badass! In fact I think you could even profit off it.

    What if you take a 10% rake of donated proceeds to it. And call it givememoneyyougetrandomcrap.net?

    I’ll send you $30 and see if your algo is cool enough for me. =)

    Anyways, Just wanted to send some geek love, script is cool!
    -Nick Chandler

  41. I’m looking into doing something similar, requiring logging into various website (PayPal included), was python-oauth2 sufficient enough and straight-forward enough to not require any previous experience with web-interfacing applications?

    Thanks.

  42. So you start at $1 and increase by $1 each day until something has been purchased — and then revert back to $1 for the following day?

  43. Jesus Christ @ 2010-11-09 08:44

    Oh my father, I approve of this.

  44. Arbitrage you fool

  45. Awesome! Can’t wait to see what items you get. My hope is that you get something of serious value.

    Great idea!

  46. Source code would be nice

  47. Is there any chance that, once you’ve tested it and feel comfortable with the results, you’d be willing to release the software? Because I know this sounds like something fun to give a try to.

  48. [...] Shared Implementing the auto-buying bot from xkcd #576. [...]

  49. Hilarious, can’t wait to see what it buys!

  50. That is so frigging awesome! I can’t wait to see what ends up on your doorstep :)
    Hopefully no bear traps.

  51. Fantastic! I was actually just about to start writing something like this inspired by the very same comic, is there any way to convince you to release the source code in github?

    Cheers!

  52. That is truly awesome. It’s rare that programming makes me laugh. Good job.

  53. [...] XKCD #576 made real. Filed under comics, craft Comment (RSS)  |  Trackback  |  Permalink [...]

  54. Has proper care been taken? We (human race) might be in mortal danger. http://xkcd.com/534/

  55. [...] of the Week – November 8, 2010 Published Monday, November 8th, 2010 at 3:00 pm bieh.net » xkcd #576 Super-Secret Delivery Procedures Document Revealed [PIC] hipstertracker:weirdfriends:I Like to [...]

  56. Awesome idea, I’m gonna have to check that twitter once it has a few items purchased. Are you considering open sourcing this?

  57. Awesum!

  58. This.. this is… omg this is *awesome* :)

  59. This is great, hope you end up with some very odd goods.

  60. good stuff sir!!!!!

    i went on a hacking bender a few months ago, same sort of “let me write a site that will scratch my own itch” thing. came up with http://foodtruckr.com. the irony is that since i built the site, i never bother bloody going to eat at the foodtrucks anymore!

    sod’s late mate.

    keep up the good work!

    -rickt

  61. Hi– I read about this on gawker. Can it be tweaked to search for specific things, like a CD from a specific artist, and then bid on it based on time it ends, like find the Regina Spektor Songs CD ending on midnight with low shipping and automatically bid on it? If so, this may be a cool marketable product. Otherwise, nice idea, good luck!

  62. Maybe add dynamic connectivity to paypal accounts and let us sign up for your ‘service’ of random products?

  63. This is GREAT.

    I wanted to write something like this after reading the comic myself, but had no easy options and couldn’t justify the time for something more complex.

    Please, if this works well, *please* consider releasing an app or the scripts for the rest of us =)

    Idea (for you, a team, or others if you do publish): build a light-weight web app that anybody can sign up for such a service. I’m sure there’s more people out there than just me who would gladly pay a flat $XX to receive gifts like this every Y days. You could even keep $1 or so of their down payment (be sure to state this, and don’t use %’s up front, it would push more people away) for funding of other projects.

  64. Just awesome, look forward to seeing what turns up :)

  65. Thank you for doing this. Now I just need time to figure out how to set it up here…

  66. This is a bad idea…. I can’t wait to see the results! :D

  67. That’s great !! :D

  68. [...] bieh.net » xkcd #576 [...]

  69. What if someone writes a script to list unique items that your bot will likely buy?! Your top level categories, once decrypted will be a lot of fun in the coding room!

  70. Josh from Auckland @ 2010-11-09 14:06

    Ingenious! Looking forward to an update post about your new treasures. Nice work on writing the bot, you should look in to doing search engine spam. :)

  71. Ohhmygod, this is the worst idea :0

    Following.

  72. [...] can be a well-known blogger. Take, for example, Paul Hunkin, a Ph.D. student in New Zealand. He set up an Internet bot to randomly buy low-priced stuff, with free shipping, from an online auction site. Paul, you see, likes to get packages. This way, he gets one, on [...]

  73. I have seen you all over the place you always give me a laugh. Then I always check your comment history and find myself laughing even more.

  74. [...] Google didn’t reveal anyone doing this already, and after asking a few people, the general opinion was that doing this was a terrible idea that would inevitably end badly. So obviously I had to do it. Delicious/tag/eBay auctions [...]

  75. John Carter @ 2010-11-09 21:34

    Well, I have this real scarface-claw of a tom cat….. I wonder…

  76. [...] bieh.net [...]

  77. [...] om det eller veta lite mer ingående hur det hela är uppbyggt så hoppa vidare på länken nedan. bieh.net Läs vidare och kommentera: [...]

  78. LOVE, LOVE this idea – will you sell the script? Would love have a script that sends me presents!!

  79. Good god man are you a wizard?

  80. I found this on CNET.com. Kudos to you for the innovation. I hope you are willing to help others out by sharing your code :)

  81. Niels de Vos @ 2010-11-10 04:24
  82. Source please!

  83. MatsSvensson @ 2010-11-10 08:28

    So you have created a spambot that sends physical items to your IRL-inbox?

    What are you going to do with all that homemade Viagra?

  84. Guys, the bot gets a new $1 each day to spend. However, each day it only has a 1/3 chance of buying things. Some days it won’t spend and then it will have n + 1 the next day to spend.

  85. NPR woke me up this morning with this blurb:

    “Feeling neglected by your honey? Wishing he or she would surprise you every once in a while with flowers or maybe a perfect CD? Forget about trying to change your mate. A man in New Zealand has programmed an Internet bot that showers him with gifts. Paul Hunkin has the bot buy him cheap items from eBay using his credit card to pay. Sadly, its first gift, batteries for a watch, was a bust. Hunkin doesn’t own a watch.”

    http://www.npr.org/templates/story/story.php?storyId=131183070

  86. [...] PhD student in New Zealand may have found the answer.  Paul Hunkin has invented an internet “bot” to automatically shower him with gifts by buying cheap items in online auctions.  Hunkin’s [...]

  87. [...] proved pretty popular on the internet for a random sunday-afternoon project! 162,434 views to the original post so far, 49 RTs of the associated tweet, and the bot has almost 3000 followers on [...]

  88. Now skynet can buy all it’s parts online without needing a human to pay for it. THANKS! :)

  89. [...] Heh, this is funny. [...]

  90. [...] Dude’s gonna need an eBay account just to sell back the weird stuff his computer buys.link:// bieh.netvia:// Gawkerrelated:// A Color-Coded Guide to XKCD’s New Online Communities [...]

  91. [...] A kiwi developer did a little work to make it happen and has been having it auto-tweet all the things it sends. Kinda cool. Wondering what is going to the be most interesting/coolest/weirdest/nastiest things he’s going to get. [...]

  92. [...] created a unique computer program that buys him random gifts online and has them delivered to him. Paul Hunkin from New Zealand has given the program a budget of $1 per day  to  automatically buy him cheap [...]

  93. [...] реализовал бота покупающего на аукционах дешевые лоты с бесплатной [...]

  94. Wicked! Any plans on releasing/sharing the code so that others can follow in your footsteps, and maybe compare what ‘tat’ people get in different countries.

  95. [...] a Python script that searches $1 NZD items in the New Zealand auction site TradeMe. Paul explains how the bot works here. You can even follow the bot on [...]

  96. not the real blinky @ 2010-11-11 20:43

    This is the best xkcd ever!

  97. Great work!

    But I just noticed it bought the same thing twice, maybe implement a feature that filters out previously bought items. Unless you want lots of batteries :)

  98. [...] xkcd #576 is a project that references this XKCD cartoon. The idea is to set up a program that will automatically locate and purchase something cheap every day or so from Amazon or eBay. It’s another neat idea, and it’s especially neat that it’s not that big a deal to code. While I couldn’t find the code, at least Paul (the guy behind bieh.net) summarized his approach to the problem. [...]

  99. Hopefully this will find more interesting stuff than mine has. After so many washers, screws, and stickers, I made a blog with a contest… What can I do with the 7 things purchased each week. Still, the result is rarely worth $7.

  100. [...] вот это. На первый взгляд кажется – ну и что такого? Бот [...]

  101. [...] Via bieh.net [...]

  102. [...] bieh.net » xkcd #576 Bot that buys dirt-cheap goods on TradeMe and tells Twitter what it's buying/bidding on. Seems we need a Rule 38: if software is described in an XKCD comic, the chance of it being brought into reality approaches 1 as t approahces infinity. (tags: xkcd shopping dumbai bot programming twitter ) [...]

  103. [...] trademe_xkcd576 · “Google didn’t reveal anyone doing this already, and after asking a few people, the general opinion was that doing this was a terrible idea that would inevitably end badly. So obviously I had to do it.” [...]

  104. Ben Sinclair @ 2010-11-16 18:06

    Awesome work Paul! Ill be looking for you on Campbell Live! :D

  105. [...] [...]

  106. [...] bieh.net » xkcd #576 [...]

  107. [...] explains the inner workings of the bot and the idea behind it on his blog. “I happened to stumble across this old comic from XKCD,” he writes. “Google [...]

  108. [...] informazioni sul post ufficiale e sul suo [...]

  109. [...] http://bieh.net/2010/11/08/xkcd-576/ Categories: 海外のNews Tags: Comments (0) Trackbacks (0) Leave a comment Trackback [...]

  110. [...] bieh.net » xkcd #576This is so cool [...]

  111. [...] Hunkin is using the API of the New Zealand auction site TradeMe to realize an XKCD strip about a guy who uses a script to [...]

  112. [...] Hunkin is using the API of the New Zealand auction site TradeMe to realize an XKCD strip about a guy who uses a script to [...]

  113. [...] service you want to use By Klint Finley | Published November 20th, 2010 Paul Hunkin is using the API of the New Zealand auction site TradeMe to realize an XKCD strip about a guy who uses a script to [...]

  114. [...] betydligt coolare mashup är utvecklat av en Nya Zeeländare och handlar automatiskt saker på den lokala auktionssajten TradeMe. För $1/dagen och genom [...]

  115. Haha, great stuff this! I can imagine the great moments of surprise every time you receive somethig you didn’t order.

  116. [...] Delicious hotlist [...]

  117. To start earning money with your blog, at first use Google adsense but steadily as your traffic
    increases, keep adding increasingly more funds generating programs to your site.
    thanks !! pretty useful publish!

  118. Google didn

  119. Google didn

  120. [...] das Webcomic xkcd hat, darüber kann man immer wieder staunen. Jetzt ist die Sache aber eskaliert: Inspiriert von einem xkcd-Comic hat jetzt jemand einen Bot gebaut, der automatisch beliebige Dinge online ersteigert. Und er [...]

  121. Would you like some more epic sauce on your win burger?

  122. [...] Пол Ханкин — программист из Новой Зеландии написал «бота-потребителя» или же бота-покупателя. Это спициальная программа, которая работает согласно интересного алгоритма, который позволяет программе закупать всякие мелочи в Сети на сумму не более нескольких долларов. Идея такой программы пришла из комикса (перевод). Цель программы — каждый день удивляться новому приобретенному товару. Получается, что каждый день этой программе-боту выделяется определённая небольшая сумма (два доллара), на которые она должна купить что-нибудь интересное. Покупки совершаются в местном онлайн-магазине. [...]

  123. [...] quite a long time of neglect, I’ve cleaned up a bunch of the dead widgets. As the bot is now offline and replaced with mysterybox.co.nz, I’ve taken away most of the inactive [...]

Add your comment now