Scratch

Thanks a lot for the suggestion! We are working on a system like what you describe.

there should be a way to make projects private, or sharable with only select members via a secret URL. This way inappropriate projects could be kept hidden from the general public. Or maybe perfectly safe projects created by school children may want to be hidden from anyone except their classmates.

So, if someone creates a project that is deemed offensive, is there any way for them to still display it?  Will it still be in their gallery for embedding in another site, or is it gone for good?

As a project administrator on a couple of Wikimedia projects (I'm currently an admin on Wikibooks and Wikiversity), this is an ongoing issue for any website that allows public contributions.

This task is far too large for any paid employee (or for a small team like the Scratch development group) to be able to regulate what content is seen.  The Los Angeles Times encountered this problem in a huge way when they tried to establish a Wiki and tried to have a full time moderator keep up with the torrent of contributions that were added.  And they had the money to keep hiring more moderators if they wanted.

What is really needed here is a form of "censorship", mainly to cut the junk out.  The most effective methods tend to be some sort of peer review, where other users who are participants can help to decide what should remain and what can be removed.  This can be done in a number of ways, both with "votes" to keep or remove stuff (aka Digg and Slashdot) or to promote "moderators" from the user community that has the opportunity to remove stuff that simply crosses the line and is unacceptable (aka Wikipedia).

The other major... even critical issue is to establish clear standards over what is desired vs. what is completely unacceptable.  There will always be content that sits on the border of what is acceptable, but if you are clear on these standards you will find that 90%+ of the content that is contributed will fit well within those standards.  So far I don't see any sort of standard at all on this website, other than "If you have made something in Scratch... let us know and share it with us!"  That is an open door for disaster.

Establishing these standards policies is hardly easy to do either (I'm in the middle of helping write a couple of them right now), but it does have a pay off in the long run as well.  I am assuming that you do want children to be able to share their ideas, but you don't want something like *.cx (link to Wikipedia article about topic... not the picture) appear as a Scratch project which is featured.  Be very specific and clear about what is considered acceptable.

Wikipedia, to give them some credit where it is due, has a few subtle policies that also removes some content as a side effect without being confrontational.  A good example of this is the "no original research" policy that also kicks off UFO and paranormal researchers from posting wild and crazy theories as respected Wikipedia articles.

There is also a small minority of people who for some reason or another simply can't conform with society and wish to destroy everything they touch.  They would set off a nuclear bomb in the middle of Manhattan just to see if it could be done, and because they think it would look pretty, not because of politics or anything else.  Unfortunately Scratch is becoming popular enough that you need to start protecting against these kind of individuals.

Last edited by Roberth (2007-05-22 10:07:42)

Hi,

Thanks for all these suggestions.  We are currently working on a system, similar to some of your suggestions, that will enable users to mark projects as inappropriate.

Re:
"So far I don't see any sort of standard at all on this website, other than "If you have made something in Scratch... let us know and share it with us!"  That is an open door for disaster."

Did you see the Terms of Use, linked from the bottom of each page on the website?  We just posted them a couple weeks ago.  Do you think it's necessary to be more specific than that?

Let us know if you have any more suggestions about how to handle inappropriate or offensive projects on the Scratch website.

Thanks!
-Tammy

As far as the terms of use... it is an excellent start.  Very simple, straight forward, and as I said, for 90% of the users they will stay well within the guidelines.  They are also very vague and subject to quite a bit of personal interpretation.  With experience, the moderation team can come up with what that expectation can be to fall within these standards, although my experience here is that even the most innocent changes to policies like this can have very large consequences... and depend largely on the personality of those performing the moderation.

Information is also a huge key here.  The more information you can give to those performing the moderation, the more accurate you can get at blocking the trolls and dealing with mistakes from new users.... and be able to distinguish between the two.  This may mean that moderators may have to agree to some sort of privacy code or rules as well if they have access to information that you may not want to be publicly released.

The problem always shows up when individuals try to push the edge of the policies as much as they can... as a sort of game.  The real trick is to make sure that you have tools available that make it much easier to kill the trolls than it is to clean up after them.  Regardless of the cleanup mechanism.  And treat those who are pushing the edge as trolls just as much as somebody who makes something blatantly offensive.

BTW, you might want to include these "standards" in regards to your terms of use right inside of Scratch itself when you click on the "Share" button.  I understand that on the website itself that you have the general terms of service, but once you have created the account there is no gentle reminder that "inappropriate projects" will be deleted immediately.  Certainly on the page where the information about the project shows up within Scratch (when you try to share it), a notice that all projects uploaded to the Scratch site must fall within the terms of service should be listed.

Again, please don't take these criticisms as something harsh against the Scratch development team.  You have a wildly successful project here that IMHO is only going to get even more attention if you keep up with what you are doing right now.  That is a very positive sign and something all of you who have been working on this should be proud of.  At the same time, merely due to the popularity of Scratch, you are also going to get those who would seek to completely destroy this project, say vile and hateful things, and deliberately try to scare parents and educators away from this website just to fit their perverted world view.  And not care that once it is trashed that they have destroyed something of significant value to many other individuals.

If you can effectively minimize these trolls, you can turn this into something that can be a very positive experience for children and those who might take offense at nearly everything.  And that is the real goal here.... not to invoke punishment on those who would try to destroy a community.

I've noticed the new feature!

stuff exploding is fine...this is not an "official" way to tell if something is appropriate, but it works in most cases: If it is ok for your grandmother and a 5 year old kid to see whats on your project, then its fine. Anything else will be taken down.

lordmgm wrote:

Would it be okay to post a project with mild violence if you state that the project will contian violence in your project notes?

Here goes pushing the terms of service to the edge of acceptability, as I was mentioning above.  Nothing personal Lordmgm, but it is an issue that does have to be dealt with in any larger community that is also of a highly transient nature.

It is also human nature to see just how far we can go before we get into trouble.

To answer this question, I would like to point out that there is a degree of blood and gore that go into the violence.  Destroying virtual asteroids, while certainly something of a violent nature, is abstract enough that I don't think anybody besides a hard core pacifist would really be worried about.  Showing decapitation and blood spurting everywhere, on the other hand, is perhaps going too far.

I should add that the MPAA (Motion Picture Association of America), who is the body that assigns the viewing ratings for movies in the USA (among other things they do as well), has only recently come up with hard standards for determining how the portrayal of violence in movies will affect ratings of movies.  For the gory details, you can look up these standards at this URL:

http://mpaa.org/Ratings_Rules.pdf

Go to page 10 of this book for the specific details, but I would recommend that you try to stick with "G" or at most "PG" type content if you are really pushing the envelope.  The rest of this document shows how complex it can be to even go through such a rating process, and what happens when you have multi-million dollar projects depending on the opinion of a small committee to assign these ratings.

if bats didn't eat bugs, they would starve...so eating them probably isn't violent, refer to my post above for how to "judge" if something is appropriate for the site.

Paulmedwal

I noticed the two inappropriate posts by sabird1 titled Naked Baby Torture.

I just taught my 9 year old son to use Scatch over the weekend, and have been allowing him to to post and look at other peoples projects online.

As a professional software developer myself, I think Scatch is an AWESOME educational tool for young children and my son learned a tremendous amount after just one day using this tool.

I am very pleased to see this particular post was pulled in a very short period of time today, but I am curious how frequently these types of things happen, and on average, how quickly they are typcially pulled?

I also would prefer these entire posts be removed, and not just the Scratch project itself.  In this case the remaining picture and the project description and title are inappropriate as well.

I would love for my son to be able to review and learn from others work, but if this inappropriate content cannot currenly be controlled, I will have to limit his access to the site until the improvements discussed above are implemented.

Thanks,

A concerned Dad....

I have noticed there is no way to flag comments in particular, I have just been flagging the entire projects instead, but there are some comments in galleries that are offensive also!