Main Content:

How to enhance YOURLS for spam control/throttling, unshortening and weekly stats

So I've been using YOURLS link shortener PHP code to power mrte.ch.  YOURLS is a fantastic system created by Ozh Richard – @Ozh on Twitter and Lester Chan – @GamerZ on Twitter.

The latest updates are incredible and provide tons of features and control but there were a couple of spots that I've enhanced:
  • pre-validating URL/link,
  • site blacklisting,
  • domain spam throttling,
  • unshortening already shortened links and
  • top weekly stats.

Below is the code that I've added to both the shortener page and the API pages, just before the "yourls_add_new_link" calls.  There's nothing fancy about my approach (yet) but I figured this would help anyone who needs these features.  I'm more of function vs. form kind of person when it comes to some things, so please ignore the crass and simplistic approach, for me it just works.

Pre-validating Submitted URL

Since most of these enhancement are before the "yourls_add_new_link" has had a chance to clean up the url, make sure to check the submitted URL with "yourls_sanitize_url" function and use a URL validator like this:

Code:
function valid_url($str) {
return ( ! preg_match('/^(http|https|ftp):\/\/([A-Z0-9][A-Z0-9_-]*(?:\.[A-Z0-9][A-Z0-9_-]*)+):?(\d+)?\/?/i', $str))
? FALSE : TRUE;
}

$url = yourls_sanitize_url($_REQUEST['url']);

if (!valid_url($url)) {
die( 'Invalid url, go back and try again' );
}



Site Blacklisting
All I did here is create a crude array of top level domains that have spammed me before.  I added the array to the config.php file so that it's available from the general include.  It's populated manually for now as I await ozh's enhancements to add blacklisting, etc.  

config.php code:
Code:
$yourls_banned_URL = array(
'mrte.ch', 'yoursitesucksforspamming.org', 'diespammers.org', 'idontthinkso.org', 'nothanksloser.com'
);

Note: please note how the first site is mrte.ch, yes I found folks submitting mrte.ch shortened links to the mrte.ch shortener, not anymore....

Main index.php page and also your API page: include just before "yourls_add_new_link" call in the submit section:
Code:
$url = $_REQUEST['url'];

foreach ($yourls_banned_URL as $banned_url) {
if (stristr($url, $banned_url)) {
die( 'Banned site' );
}
}

Note: please let me know if there's a more optimal way of doing this, thanks.


Domain Spam Throttling
This just check for today's submitted sites and if more than 5 have been submitted from the same domain for that day, it just dies.  I may add a check to automatically blacklist if within a specific time period and a greater throttle number.  I'll also remove all the other existing links, etc. if they are spamming,  I'll update this posting if/when i do.

Main index.php page and also your API page: include just before "yourls_add_new_link" call in the submit section (I split $spamCheckQuery for easier reading only):
Code:
$spamCheckURLArray = parse_url(stripslashes($url));

$spamCheckQuery = "SELECT count(*) FROM `yourls_url_table` WHERE `url` LIKE '%";
$spamCheckQuery .= $spamCheckURLArray['host'] . "%' AND timestamp >= CURDATE()";
$spamLinkCount = $ydb->get_var($spamCheckQuery);

if ($spamLinkCount > 5) {
die( 'Nice try on spamming...' );
}


Unshortening Already Shortened Links
I found Slinky a PHP URL Unshortener or "lengthener" as they put it and decided to put this into place because I was finding folks submitting links that were already shortened by bit.ly, tr.im, etc.  I also wanted to make sure that I kept things nice and clean.  Here's how I use Slinky, note there's a syntax bug in their code (missing semi-colon or something, can't remember) once you fix that you are on your way.  Just download the code from Slinky and drop there include on your site.

Code:
require_once( dirname(__FILE__).'/includes/slinky.php' );

$url = $_REQUEST['url'];

$slinky = new Slinky( $url );
$slinky->set_service_from_url();
// you should be able to remove the 'login' and 'apiKey' lines if you don't want bit.ly unshortening, etc.
$slinky->service->set( 'login', 'YOUR_BIT.LY_API_LOGIN_IF_YOU_HAVE_ONE' );
$slinky->service->set( 'apiKey', 'YOUR_BIT.LY_API_KEY_IF_YOU_HAVE_ONE' );
$url = $slinky->long();


Top Weekly Stats
I wanted to show top weekly stats on my main shortener page so I whipped up this little ditty.

Code:
$urlt2 = '';

// Connect To Database
$sqlcode = "SELECT url, keyword, clicks FROM `yourls_url_table` WHERE timestamp >= SUBDATE(CURDATE(), 7) ";
$sqlcode .= "order by clicks desc limit 8";

$query = $ydb->get_results($sqlcode);

if ($query) {
foreach( $query as $query_result ) {
$thisURLArray = parse_url(stripslashes($query_result->url));
$urlt2 .= '<a href="http://mrte.ch/' . $query_result->keyword .'" target="blank">';
$urlt2 .= str_replace('www.', '', $thisURLArray['host']) . '</a> (' . $query_result->clicks . ') ';
}
}

echo '<b>Week\'s Top links:</b> ' . $urlt2 . "<br><br>\n\r";

To test or validate your result you can pull up your data like this, you can also change the limit from 8 to whatever you want to show:
Quote
SELECT url, keyword, clicks, timestamp, SUBDATE(CURDATE(), 7)
FROM `yourls_url_table` WHERE timestamp >= SUBDATE(CURDATE(), 7)
order by clicks desc limit 8

Note: SUBDATE should probably be 6 instead of 7 but I wanted to be a little more inclusive.

In Conclusion
Again I'd like to extend my honest gratitude to Ozh and YOURLS for a great system and hope this helps someone.  Please post a message if you were able to use any of this, if you find any bugs or have any performance enhancements, etc.

-- mel reyes: self-professed geek

« Last Edit: April 11, 2010, 03:44:42 PM by mrtech »



• mel reyesBlogPlaxoLinkedInTwitter
Support mrtech.com get our toolbar

December 08, 2009, 11:45:11 AM

About The Author

  • mrtech
  • Administrator
  • Posts: I am a geek!!
  • Email WWW

Comments

Dragon

Got pre-validating URL/link, and site blacklisting working but Top Weekly Stats is a no go, I only get the title and no data. 

mrtech

Do you have a public link?  Have you enabled PHP debugging, I might have missed a variable declaration or something like that.  I'll review the code and see if I missed anything.



• mel reyesBlogPlaxoLinkedInTwitter
Support mrtech.com get our toolbar

mrtech

just to confirm that you've changed the "yourls_url_table" to be your specific YOURLs url table?



• mel reyesBlogPlaxoLinkedInTwitter
Support mrtech.com get our toolbar

rudolfpietersma

I also tried to get the Weekly Top Links active. But only the Title is visible.

I also tested to get the data in the db. I get results:

url   keyword   clicks   timestamp   SUBDATE(CURDATE(), 7)
http://blblblb   12    53   2010-01-17 07:35:54   2010-01-15
http://blblblb   10   30   2010-01-21 01:00:22   2010-01-15
http://blblblb   0s   26   2010-01-17 11:31:28   2010-01-15
http://blblblb   20    20   2010-01-22 08:03:58   2010-01-15

rudolfpietersma

I did change your_url_table to the name of my table.

mrtech

Do you have a sample page?



• mel reyesBlogPlaxoLinkedInTwitter
Support mrtech.com get our toolbar

rudolfpietersma

http://urli.nl

mrtech

Can you send me the code snippet, you can change the table name if you want, and I'll check to see if there's something missing or wonky.  Also I'm assuming PHP 5.x, etc. thanks.

mel [at] mrtech.com



• mel reyesBlogPlaxoLinkedInTwitter
Support mrtech.com get our toolbar

rudolfpietersma

$urlt2    = '';

// Connect To Database
$sqlcode = "SELECT url, keyword, clicks, FROM `url` WHERE timestamp >= SUBDATE(CURDATE(), 7) ";
$sqlcode .= "order by clicks desc limit 8";

$query = $ydb->get_results($sqlcode);

if ($query) {
   foreach( $query as $query_result ) {
      $thisURLArray    = parse_url(stripslashes($query_result->url));
      $urlt2       .= '<a href="http://urli.nl/' . $query_result->keyword .'" target="blank">';
      $urlt2       .= str_replace('www.', '', $thisURLArray['host']) . '[/url] (' . $query_result->clicks . ') ';
   }
}

echo 'Week\'s Top links: ' . $urlt2 . "

\n\r";

The name of the table is url.

mrtech

can you confirm the version of PHP?



• mel reyesBlogPlaxoLinkedInTwitter
Support mrtech.com get our toolbar

rudolfpietersma

Sorry forgot to mention it. It's 5.2.11.

mrtech

Hmm, not sure, just to confirm you're using YOURLS 1.4?



• mel reyesBlogPlaxoLinkedInTwitter
Support mrtech.com get our toolbar

rudolfpietersma

Yes. I use version 1.4.

boygenius0709

Oh, that ish ow to do it then.. great.

Add a Comment

Only registered members can post comments, please click here to register.

Pages: [1] 2 3 4