Twitter recently announced
that it has implemented a new security system to scan all URLs posted in tweets to protect users from malicious sites. This follows a similar announcement from bit.ly in November 2009
Twitter, and the URL shorteners it has helped to popularize, have long been blamed for leading users to malicious sites. I posted on this topic
3 weeks ago and argued that this may not be true. I wanted to additionally do a thorough investigation of the Twitter links both before the security scan and after.
I have retrieved more than 1 million URLs (1,314,615 to be exact) from the public timeline over a couple of weeks before they put any protections in place. I then ran the links through the Zscaler infrastructure to find out which links lead to malicious sites.
The state of the Twitter links
Prevalence of hostnames on Twitter
As expected, URL shorteners are very popular on Twitter, and bit.ly
represents 40% of all links. TinyUrl
, one of the original URL shorteners, comes in 3rd
with only 5% of all URLs.
How many malicious links?
I looked for malicious sites - phishing sites, malware, etc. I did not look for spam, only for pages that present a security risk to users.
To my surprise, a very low number of links led to malicious pages - only 773, links, 0.06% of all links scanned, redirected to malicious content.
Types of malicious sites
Here is the distribution of malicious links by host name:
Bit.ly represents 40% of all links, and roughly the same proportion of malicious links. Same case for TinyUrl: 5% of all URLs and 6 % of all malicious sites. It does not look like bit.ly’s phishing and malware protection is making it any safer than other URL shorteners. Twitpic.com is used to share images, so it is unlikely to be used for malicious content. Mediafire is known for hosting malware and other viruses, even if it is not blocked by Google Safe Browsing
Note that these links may have been scanned up to 4 weeks after they were collected. Bad sites may already have been taken down, or cleaned up.
Can Twitter and bit.ly really protect their users?
But remember that only 0.06% of all the URLs tests represented a security risk. It is actually much safer to follow link s from Twitter that from some search results on Google