Perpetrating fraud on social networks

There has been a lot of talk recently about the Koobface virus (see the article from CNET and the article from Wired).  Viruses are definitely an internet scourge at the moment, but there is another, more subtle, security risk here.

The Koobface virus propagates on social network sites like MySpace and Facebook through the mechanism that allows the users of the site to send messages to their “friends”.  This is an example of what Silver Tail calls business process abuse and similarly what Jeremiah Grossman calls business logic flaws.  Social network sites want their users to be able to send messages to each other.   Turning off the ability for users to send messages to each other would disable an important function on the social network sites.  And yet, bad guys can take advantage this function to send malicious messages. These sites need to allow users to send messages to each other while protecting the users from people abusing the same functions to perpetrate bad activities (like sending links to websites that contain viruses and other types of malware).

Unfortunately, protecting users from business logic flaws is extremely challenging.  There is a huge amount of exposure due to the time it takes to identify the issue, confirm the nature of the issue, and develop and deploy a fix.  Each of these areas is so important that it wouldn’t do them justice to discuss them all in one posting.  I’ll cover the identification of business logic flaws here and save the investigation and mitigation of those flaws for future installments.

So, how do most websites identify business logic flaws?  Since business logic flaws are not errors in the code of a website, the traditional mechanisms for finding bugs don’t apply.  This means that most of the time a bad guy finds a business logic flaw on a website, it is brought to the attention of the website from the customers that are impacted, and often days or weeks later.

Let’s walk through an example.   Say that a bad guy starts sending messages to people through a social network site and those messages contain a link to a site hosting malware.  Some number of customers receives the message.  The more savvy customers will know the message is malicious and report it back to the social network site.  The customer support team of the social network site will investigate the complaints about the messages and once they determine there is a pattern of problematic messages, they will report them to the appropriate group inside the social network site – say the group concerned with customer security – so that an action can be taken to rectify the problem (we’ll discuss rectifying the problem in an upcoming post).

Looking at all of the actions above, how long could this take? If lots of users receive the malicious message – this is already a problem since those that are not savvy will likely get infected by the virus – it’s more likely that savvy customers will report the problem quickly.  Let’s say it takes 8 hours for 50 reports to come in about the same issue.  Since customer support is reviewing cases about all sorts of concerns from the users of the social network site, it may take a day or two to get to the 50 cases that were reported.  It wouldn’t be until the end of the second day that they realize there is a trend with the same malicious message being reported by many customers.  At this point, the message has been “in the wild” for two days.  After two days, customer support escalates the issue to another group to address the problem.  Unfortunately, since there has been customer impact for a couple of days, there could be lots of users impacted by the malicious link.

To make matters worse, investigating and fixing the problem can take even longer.  But we’ll get into the details of that in a future posting. It would be great to hear from those of you who have experience responding to these types of exploits.