Scratch

sparks · 2011-09-04 18:43:37

I tried to write a piece of php code that would search the Scratch database for users who have you as a friend to answer the long-sought-after "Who's added me?".

I failed. Dismally. The page processed 84 users in one minute (0.7 users a second) which isn't great for the 867,000 odd registered users. It would take it 10 days to run all the way through!

It was a nice thought, maybe I just did it really badly so here's the code, if anyone's interested:

Code:

<?php
set_time_limit(3600);
function nth_position($str, $letter, $n, $offset = 0){
    $str_arr = str_split($str);
    $letter_size = array_count_values(str_split(substr($str, $offset)));
    if( !isset($letter_size[$letter])){
        trigger_error('letter "' . $letter . '" does not exist in ' . $str . ' after ' . $offset . '. position', E_USER_WARNING);
        return false;
    } 
    else if($letter_size[$letter] < $n) {
        trigger_error('letter "' . $letter . '" does not exist ' . $n .' times in ' . $str . ' after ' . $offset . '. position', E_USER_WARNING);
        return false;
    }
    for($i = $offset, $x = 0, $count = (count($str_arr) - $offset); $i < $count, $x != $n; $i++){
        if($str_arr[$i] == $letter){
        $x++;
        }
    }
    return $i - 1;
}


$t1 = ltrim(file_get_contents('http://scratch.mit.edu/api/getregisteredusers '));
$t1 = str_replace(",", "", $t1);
if(isset($_GET['username'])){
    $image = ltrim(file_get_contents('http://scratch.mit.edu/api/getinfobyusername/' . $_GET['username']));
    if($image == '::'){
        echo "<p>Sorry, can't find that username! Try again!</p>";
    }
    if($image != '::'){
        $startpos = nth_position($image, ":",1) + 1;
        $endpos = nth_position($image, ":", 2);
        $image = substr($image, $startpos, $endpos - $startpos);
        echo "<p>Well it looks like there are currently " . $t1 . " people registered with Scratch, " . $_GET['username'] . " (" . $image . ") so this could take a while! <br />
        I'll take all the people I find who are friends with you and put a link into the space below along with their username!<br />
        <br/>
        I guess I'd better start!</p><hr/>";
        $count = 0;
        if(isset($_GET['start'])){
            $count = $_GET['start'];
        }
        while($count < $t1){
            $count ++;
            $id = ltrim(file_get_contents('http://scratch.mit.edu/api/getusernamebyid/' . $count));
            if(strpos(ltrim(file_get_contents('http://scratch.mit.edu/api/getfriendsbyusername/' . $id)), $image) != 0){
                echo "<a href = 'www.scratch.mit.edu/users/" . $id . "'><p>" . $id . "</p></a>";
            }
        }
        echo"<p>That's it! I'm done!</p>";
    }
}
if(!isset($_GET['username'])){
echo "<p>This experimental project searches the Scratch database for users with you as their friend!</p>
<br/>
<p>Please add your Scratch username into the top bar like so: ?username=yourUsernameHere</p>
<p>If you have searched before and know what number the search reached then you can add it to the url by putting &start=theNumber</p>";
}
?>

Last edited by sparks (2011-09-04 18:45:09)

samtwheels · 2011-09-04 18:50:43

aren't you notified of being friended?

sparks · 2011-09-04 18:51:20

Yes, and I delete them, as many do, because the inbox gets cluttered otherwise.

Greatdane · 2011-09-04 18:52:04

..ouch.

It might be a better idea to tell the Scratch Team to just display this on your profile as well, because it is already in the database.

However, the Scratch Team was talking about making friend-ing more like Facebook, a kind of request thing, which would make this useless.

gettysburg11 · 2011-09-04 18:52:25

samtwheels wrote:
aren't you notified of being friended?

Yes, but most people don't keep track of who friends them. They just delete the message and move on.
Edit: Grrrr, ninja'd. tongue

Oh well, at least you tried. That's something I certainly never could've come up with. tongue

Last edited by gettysburg11 (2011-09-04 18:52:52)

sparks · 2011-09-04 18:54:12

Greatdane wrote:
..ouch.

It might be a better idea to tell the Scratch Team to just display this on your profile as well, because it is already in the database.

However, the Scratch Team was talking about making friend-ing more like Facebook, a kind of request thing, which would make this useless.

Ah, even though the Scratch Team are discussing it, it isn't currently in the database at all. id's of friends are listed with the user id but not the other way around which would be much faster and would then probably already be implemented by the ST

sparks · 2011-09-04 18:56:18

Hmmm, either there's a faster way of doing it that I've not thought up, or this isn't going to work. I have a feeling that AJAX or some such language that I've not looked into may be better suited than php and I'm hoping LS97 or someone will come up with a faster method!

Any method would have to process at least 20 users a second simply to finish the search in half a day though...

Last edited by sparks (2011-09-04 18:59:37)

Greatdane · 2011-09-04 18:56:46

Ouch again.

Can't you just once again read it for 10 days and then compile the database, and future friend-ing supports this?

Well, I'm not the Scratch Team, so I wouldn't know.

sparks · 2011-09-04 19:01:19

Greatdane wrote:
Ouch again.

Can't you just once again read it for 10 days and then compile the database, and future friend-ing supports this?

Well, I'm not the Scratch Team, so I wouldn't know.

Using the current method you would have to run the 10 day cycle for every single user so you would be finished in around 8671840 days...

Greatdane · 2011-09-04 19:08:43

sparks wrote:
Greatdane wrote:
Ouch again.

Can't you just once again read it for 10 days and then compile the database, and future friend-ing supports this?

Well, I'm not the Scratch Team, so I wouldn't know.
Using the current method you would have to run the 10 day cycle for every single user so you would be finished in around 8671840 days...

I thought so..

So you're saying that every user takes 10 days to run?

...ouch once more...

Well, end of that idea! big_smile

Unless you get a horde of servers that somehow have access to the database that run this...

WindowsExplorer · 2011-09-05 03:08:52

I think he means the list is so long it takes 10 days to scroll down.

whizzer · 2011-09-05 13:15:28

You know, you should be happy that it at least WORKS. Even if it is painfully slow. (understatement of the year)

sparks · 2011-09-05 15:29:42

I built the same script in a Panther project and it is about the same speed, maybe a little slower so no improvement there. On the plus side the project shows progress and doesn't timeout!

If you ask me, php is missing some fundamental functions, namely you cannot do
$var1 contains $var2, you have to do

strpos($var1, $var2) != 0

and you cannot do index of $var1 in $var2.

Oddly users seem to have been stored alphabetically early on... I'm pretty sure my user id has always stayed the same so it can't be done any more as anyone with a user-name alphabetically earlier than mine would cause my id to increase.

Last edited by sparks (2011-09-05 16:21:24)

Greatdane · 2011-09-05 17:19:20

I'm guessing that earlier on for testing users were stored alphabetically and then chronologically.

sparks · 2011-09-05 17:28:20

Well currently my Panther system has scanned 3540 account and has just started on "L" with laxxkid14. Once it reaches Z I will be interested to see what happens. Numbers or the newer chronological order? It didn't start out alphabetically either, with Andresmh as the first registered person (#39 I think)

JSO · 2011-09-05 17:42:46

You could also try something like this:

http://www.google.com/search?q=site:scr … s/*+%2BJSO

although I'm afraid that's not very accurate.

sparks · 2011-09-05 17:54:23

JSO wrote:
You could also try something like this:

http://www.google.com/search?q=site:scr … s/*+%2BJSO

although I'm afraid that's not very accurate.

Interesting idea! I didn't know google could be used for that! However, I tried a more accurate method using the API and my user id: site:scratch.mit.edu/api/getfriendsbyusername/* +183006 which returned no results. Maybe google hasn't added all those pages to it's search range...

jji7skyline · 2011-09-05 20:37:24

How do you run the php? tongue

ohaiderstudios · 2011-09-05 22:21:53

you have to upload it to a server or establish a local host.

sparks · 2011-09-06 10:36:13

20890 people scanned, 2% complete, several hours running and three people who have added me as friends have been found tongue Does anyone want me to post the Panther project, btw?

cskidmsonic · 2011-09-06 11:05:56

WindowsExplorer wrote:
I think he means the list is so long it takes 10 days to scroll down.

It takes ten days to find all of the friends for each user, and assuming every user had this done, it would take around 8671840 days.

roijac · 2011-09-06 11:16:14

maybe just upload to something about 10 hosts, all with different range, and make it x10 faster? tongue

LS97 · 2011-09-06 13:22:08

sparks wrote:
If you ask me, php is missing some fundamental functions, namely you cannot do $var1 contains $var2, you have to do
strpos($var1, $var2) != 0
and you cannot do index of $var1 in $var2.

I think you're looking for strstr wink

sparks wrote:
I tried a more accurate method using the API and my user id: site:scratch.mit.edu/api/getfriendsbyusername/* +183006 which returned no results. Maybe google hasn't added all those pages to it's search range...

Google couldn't have added it to its search page simply because it's not linked anywhere and GoogleBot never found it! Maybe now with the link we might get some improvements tongue

sparks wrote:
20890 people scanned, 2% complete, several hours running and three people who have added me as friends have been found Does anyone want me to post the Panther project, btw?[/url]

Wow hmm I believe that with some faster script you could be able to finish it off in a couple days. Still that script needs to be found tongue

I'll be attempting something in the next days, and I'll tell you what I found! big_smile

sparks · 2011-09-06 13:25:32

LS97 wrote:
sparks wrote:
If you ask me, php is missing some fundamental functions, namely you cannot do $var1 contains $var2, you have to do
strpos($var1, $var2) != 0
and you cannot do index of $var1 in $var2.
I think you're looking for strstr

sparks wrote:
I tried a more accurate method using the API and my user id: site:scratch.mit.edu/api/getfriendsbyusername/* +183006 which returned no results. Maybe google hasn't added all those pages to it's search range...
Google couldn't have added it to its search page simply because it's not linked anywhere and GoogleBot never found it! Maybe now with the link we might get some improvements

sparks wrote:
20890 people scanned, 2% complete, several hours running and three people who have added me as friends have been found Does anyone want me to post the Panther project, btw?[/url]
Wow I believe that with some faster script you could be able to finish it off in a couple days. Still that script needs to be found

I'll be attempting something in the next days, and I'll tell you what I found!

Well I physically can't think of a faster way to do it, so I'm looking forward to you enlightening me!(searching #28240 - 3% complete tongue )

sparks · 2011-09-06 16:44:58

I wonder if this is the sort of problem the Scratch team are being presented with in changes for Scratch 2.0? They say they want to make friending more like actual friendship where it's mutual rather than one-sided as it is at the moment and though it should be quite easy for them to apply this to new friendships, they might end up having to do something similar to this if they want to convert old friendships...

The way I see it, old friendships will be recategorised as "watching"

(search #35575, 4% complete)

Last edited by sparks (2011-09-06 16:45:30)

Scratch

archived forums

#1 2011-09-04 18:43:37

Who friended me?

Code:

#2 2011-09-04 18:50:43

Re: Who friended me?

#3 2011-09-04 18:51:20

Re: Who friended me?

#4 2011-09-04 18:52:04

Re: Who friended me?

#5 2011-09-04 18:52:25

Re: Who friended me?

samtwheels wrote:

#6 2011-09-04 18:54:12

Re: Who friended me?

Greatdane wrote:

#7 2011-09-04 18:56:18

Re: Who friended me?

#8 2011-09-04 18:56:46

Re: Who friended me?

#9 2011-09-04 19:01:19

Re: Who friended me?

Greatdane wrote:

#10 2011-09-04 19:08:43

Re: Who friended me?

sparks wrote:

Greatdane wrote:

#11 2011-09-05 03:08:52

Re: Who friended me?

#12 2011-09-05 13:15:28

Re: Who friended me?

#13 2011-09-05 15:29:42

Re: Who friended me?

#14 2011-09-05 17:19:20

Re: Who friended me?

#15 2011-09-05 17:28:20

Re: Who friended me?

#16 2011-09-05 17:42:46

Re: Who friended me?

#17 2011-09-05 17:54:23

Re: Who friended me?

JSO wrote:

#18 2011-09-05 20:37:24

Re: Who friended me?

#19 2011-09-05 22:21:53

Re: Who friended me?

#20 2011-09-06 10:36:13

Re: Who friended me?

#21 2011-09-06 11:05:56

Re: Who friended me?

WindowsExplorer wrote:

#22 2011-09-06 11:16:14

Re: Who friended me?

#23 2011-09-06 13:22:08

Re: Who friended me?

sparks wrote:

sparks wrote:

sparks wrote:

#24 2011-09-06 13:25:32

Re: Who friended me?

LS97 wrote:

sparks wrote:

sparks wrote:

sparks wrote:

#25 2011-09-06 16:44:58

Re: Who friended me?

Board footer