This is a read-only archive of the old Scratch 1.x Forums.
Try searching the current Scratch discussion forums.

#1 2012-04-20 13:22:42

Magnie
Scratcher
Registered: 2007-12-12
Posts: 1000+

Mesh Python Server - Host a Mesh Server on a VPS!

So one of the many (many) things that I have been wanting to make was a Mesh Server that can be run on a VPS (or any other server) without needing to run Scratch 24/7 on a computer.

Code:

#!/usr/bin/env python

"""
This is a Mesh Server for Scratch (and possibly any project)
that you can run in the background on a VPS of the sorts.

It's totally insecure and anyone can connect and attempt to
send messages (whether broadcasts or sensor-updates) to the
users connected.

This code doesn't work on Windows I'm afraid. I'll try to
work out a Windows supported Mesh server later.
"""

import select
import socket
import sys
import threading

class Server:
    def __init__(self):
        self.host = ''
        self.port = 42001
        self.backlog = 5
        self.size = 1024
        self.server = None
        self.threads = []

    def open_socket(self):
        try:
            self.server = socket.socket(socket.AF_INET,
                                        socket.SOCK_STREAM)
            self.server.bind((self.host,self.port))
            self.server.listen(5)
        except socket.error, (value,message):
            if self.server:
                self.server.close()
            print "Could not open socket: " + message
            sys.exit(1)

    def run(self):
        self.open_socket()
        input = [self.server,sys.stdin]
        running = 1
        while running:
            inputready,outputready,exceptready = select.select(input,
                                                               [],
                                                               [])

            for s in inputready:

                if s == self.server:
                    # handle the server socket
                    c = Client(self.server.accept())
                    c.start()
                    self.threads.append(c)

                elif s == sys.stdin:
                    # handle standard input
                    junk = sys.stdin.readline()
                    running = 0

        # close all threads

        self.server.close()
        for c in self.threads:
            c.join()

class Client(threading.Thread):
    def __init__(self,(client,address)):
        threading.Thread.__init__(self)
        self.client = client
        self.address = address
        self.size = 1024

    def run(self):
        running = 1
        while running:
            try:
                data = self.client.recv(self.size)
            except Exception, e:
                self.log(str(e))
                data = ''
            
            if data:
                for user in s.threads:
                    try:
                        user.client.send(data)
                    except Exception, e:
                        self.log(str(e))
            
            else:
                self.client.close()
                running = 0
    
    def log(self, string):
        try:
            log_file = open("mesh.log", 'a')
            log_file.write('\n' + string)
            log_file.close()
        except IOError, e:
            log_file = open("mesh.log", 'w')
            log_file.write(string)
            log_file.close()

if __name__ == "__main__":
    s = Server()
    s.run()

But as it says, it's totally insecure. I'm trying to think up of ways to validate data that comes through (make sure the length is valid, check to see if it's only a broadcast and sensor-update, etc) but I still need ideas on how to do that. Maybe even make it so the user has to "login" before being able to send all users a broadcast or sensor-update.

The EXE is located here.

A Java version is also being developed by SJRCS_011.

I'm also thinking of a way to close the server while it's in the background without using the 'kill' command. One of the ideas is sending something like 'broadcast "QUIT"' maybe with a password attached (broadcast "QUIT mYpa5sw0rD.").

Other possible plans are allowing multiple mesh networks on a single server. So someone could create their own Mesh network and people could login to that one.

I keep getting ideas while writing this so I quickly stop and say: All ideas and suggestions are welcome!

Last edited by Magnie (2012-04-27 14:51:19)

Offline

 

#2 2012-04-20 13:47:39

blob8108
Scratcher
Registered: 2007-06-25
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

Magnie wrote:

Other possible plans are allowing multiple mesh networks on a single server. So someone could create their own Mesh network and people could login to that one.

This is cool!  smile  Having multiple mesh networks is a good idea. You could broadcast "join <network>" from the client, or something similar.

I'm also thinking of a way to close the server while it's in the background without using the 'kill' command.

I still think you should try Twisted  tongue  It handles all the connection/threading/"select" stuff for you; and I think it has built-in handling of things like shutting down the server nicely after a Ctrl-C (or you could hack it to make it do so)... (:


Things I've made: kurt | scratchblocks2 | this cake

Offline

 

#3 2012-04-20 14:16:57

Magnie
Scratcher
Registered: 2007-12-12
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

blob8108 wrote:

Magnie wrote:

Other possible plans are allowing multiple mesh networks on a single server. So someone could create their own Mesh network and people could login to that one.

This is cool!  smile  Having multiple mesh networks is a good idea. You could broadcast "join <network>" from the client, or something similar.

Yeah, that's my plan. I'm also thinking of using a password for the networks as well (broadcast "join <network> <password>").

blob8108 wrote:

I'm also thinking of a way to close the server while it's in the background without using the 'kill' command.

I still think you should try Twisted  tongue  It handles all the connection/threading/"select" stuff for you; and I think it has built-in handling of things like shutting down the server nicely after a Ctrl-C (or you could hack it to make it do so)... (:

.... We both know I've had troubles with Twisted. It's just not something I want to use. I prefer keeping to all the default modules that come with Python.

Offline

 

#4 2012-04-20 14:39:58

bobbybee
Scratcher
Registered: 2009-10-18
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

I personally think that if you're doing mesh server, you shouldn't include a protocol. Only bare-old mesh. Have the VPS itself get IO commands.


I support the Free Software Foundation. Protect our digital rights!

Offline

 

#5 2012-04-20 15:17:45

Magnie
Scratcher
Registered: 2007-12-12
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

bobbybee wrote:

I personally think that if you're doing mesh server, you shouldn't include a protocol. Only bare-old mesh. Have the VPS itself get IO commands.

The Python script above is bare-old mesh at the moment. I'll probably keep the current code and then develop another one with features (like multiple mesh networks).

Offline

 

#6 2012-04-20 15:48:07

blob8108
Scratcher
Registered: 2007-06-25
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

Magnie wrote:

We both know I've had troubles with Twisted. It's just not something I want to use. I prefer keeping to all the default modules that come with Python.

sad  *sadface*. You rejected Twisted! Now you've hurt Twisted's feelings! D':<
tongue

That aside, am I being stupid, or is there a possible obscure bug in your code?

My understanding was that socket.recv doesn't always return your whole message; a message like 'sensor-update "note" 60 "seconds" 0.1' might get split up across multiple packets, and not all arrive at once. You're echoing the recieved bytes straight back to the other clients. So if more than one client sends data at the same instant, the messages might get split up, and so the parts might get sent from the server all mixed up. The data could get mangled.

I think you'd be very unlikely to notice this in practice, and it could be hard to find the problem — so I thought I'd suggest it, just in case.  smile  Please, correct me if I'm wrong somewhere...


Things I've made: kurt | scratchblocks2 | this cake

Offline

 

#7 2012-04-21 05:07:16

DigiTechs
Scratcher
Registered: 2011-04-30
Posts: 500+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

How long till' you have a window's version?  tongue  I'll download Python, so I can run the un-compiled version when it's done (Or the compiled version)


I'm back.
Maybe.

Offline

 

#8 2012-04-21 05:09:08

DigiTechs
Scratcher
Registered: 2011-04-30
Posts: 500+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

Magnie wrote:

Code:

#!/usr/bin/env python

I guess this is why it doesn't work on windows - since Windows doesn't use that kind of way to "manage" the place of Python.


I'm back.
Maybe.

Offline

 

#9 2012-04-21 09:09:38

Magnie
Scratcher
Registered: 2007-12-12
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

DigiTechs wrote:

Magnie wrote:

Code:

#!/usr/bin/env python

I guess this is why it doesn't work on windows - since Windows doesn't use that kind of way to "manage" the place of Python.

Actually, you can still run it (it'll work fine) the only problem is that the 'select' module doesn't support windows.  sad

Anyways, here is the Windows Mesh Server code:

Code:

#!/usr/bin/env python

"""
This is a Mesh Server for Scratch (and possibly any project)
that you can run in the background on a VPS of the sorts.

It's totally insecure and anyone can connect and attempt to
send messages (whether broadcasts or sensor-updates) to the
users connected.

This version supports Windows! :D
"""

import select
import socket
import sys
import threading

class Server:
    def __init__(self):
        self.host = ''
        self.port = 42001
        self.backlog = 5
        self.size = 1024
        self.server = None
        self.threads = []

    def open_socket(self):
        try:
            self.server = socket.socket(socket.AF_INET,
                                        socket.SOCK_STREAM)
            self.server.bind((self.host,self.port))
            self.server.listen(5)
        except socket.error, (value,message):
            if self.server:
                self.server.close()
            print "Could not open socket: " + message
            sys.exit(1)

    def run(self):
        self.open_socket()
        running = 1
        while running:
            
            try:
                c = Client(self.server.accept())
                c.start()
                self.threads.append(c)
            
            except KeyboardInterrupt:
                junk = sys.stdin.readline()
                running = 0

        # close all threads

        self.server.close()
        for c in self.threads:
            c.join()

class Client(threading.Thread):
    def __init__(self,(client,address)):
        threading.Thread.__init__(self)
        self.client = client
        self.address = address
        self.size = 1024

    def run(self):
        running = 1
        while running:
            try:
                data = self.client.recv(self.size)
            except Exception, e:
                self.log(str(e))
                data = ''
            
            if data:
                for user in s.threads:
                    try:
                        user.client.send(data)
                    except Exception, e:
                        self.log(str(e))
            
            else:
                self.client.close()
                running = 0
    
    def log(self, string):
        try:
            log_file = open("mesh.log", 'a')
            log_file.write('\n' + string)
            log_file.close()
        except IOError, e:
            log_file = open("mesh.log", 'w')
            log_file.write(string)
            log_file.close()

if __name__ == "__main__":
    s = Server()
    s.run()

I made a quick chat client (lol) to test it out and I realized that any message you send to the server is echo'ed back to you and I was debating if that's a good thing or a bad thing but I couldn't come up with a reasonable opinion. What are your guys' opinions? Should I keep the echo or remove it?

Offline

 

#10 2012-04-21 09:52:04

blob8108
Scratcher
Registered: 2007-06-25
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

Magnie wrote:

I made a quick chat client (lol) to test it out and I realized that any message you send to the server is echo'ed back to you and I was debating if that's a good thing or a bad thing but I couldn't come up with a reasonable opinion. What are your guys' opinions? Should I keep the echo or remove it?

I'd remove it; Mesh is quite slow as it is, and sending unnecessary traffic doesn't really sound like a good idea...  hmm


Things I've made: kurt | scratchblocks2 | this cake

Offline

 

#11 2012-04-21 10:27:19

Magnie
Scratcher
Registered: 2007-12-12
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

blob8108 wrote:

Magnie wrote:

I made a quick chat client (lol) to test it out and I realized that any message you send to the server is echo'ed back to you and I was debating if that's a good thing or a bad thing but I couldn't come up with a reasonable opinion. What are your guys' opinions? Should I keep the echo or remove it?

I'd remove it; Mesh is quite slow as it is, and sending unnecessary traffic doesn't really sound like a good idea...  hmm

Yeah, I'm also thinking to remove it as well because it can mess up scripts and stuff.

Offline

 

#12 2012-04-27 02:50:00

lallaway12
Scratcher
Registered: 2012-01-04
Posts: 500+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

Aotopost-


Is the EXE ready


http://i49.tinypic.com/2re4ied.png

Offline

 

#13 2012-04-27 07:42:18

slinger
Scratcher
Registered: 2011-06-21
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

Wow, this sounds so cool!  big_smile


http://s0.bcbits.com/img/buttons/bandcamp_130x27_blue.png

Offline

 

#14 2012-04-27 08:52:52

SJRCS_011
Scratcher
Registered: 2011-02-07
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

So what does this do again? Does it just echo back the variable declarations across all connections?
If so, I'd be willing to make the EXE for you in C++.
And maybe I'll make a Jar as well  tongue

EDIT: nvm about the EXE, apparently my IDE doesn't have a built-in compiler  sad

Last edited by SJRCS_011 (2012-04-27 09:55:15)


http://i.imgur.com/vQqtH.png
Learning to Program in a Nutshell:  "You're missing a closing parentheses" - LS97

Offline

 

#15 2012-04-27 09:56:18

Magnie
Scratcher
Registered: 2007-12-12
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

SJRCS_011 wrote:

So what does this do again? Does it just echo back the variable declarations across all connections?
If so, I'd be willing to make the EXE for you in C++.
And maybe I'll make a Jar as well  tongue

Yeah, that's basically all it does.  tongue  You can make the Jar file.  smile

lallaway: I'll upload an EXE in the next 10 hours.

Last edited by Magnie (2012-04-27 09:56:52)

Offline

 

#16 2012-04-27 10:38:04

DigiTechs
Scratcher
Registered: 2011-04-30
Posts: 500+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

SJRCS_011 wrote:

So what does this do again? Does it just echo back the variable declarations across all connections?
If so, I'd be willing to make the EXE for you in C++.
And maybe I'll make a Jar as well  tongue

EDIT: nvm about the EXE, apparently my IDE doesn't have a built-in compiler  sad

Upload the solution somwhere (or source files) and i'll compile them for you, since my C++ program has a built in compiler (thanks, Microsoft)


I'm back.
Maybe.

Offline

 

#17 2012-04-27 10:41:14

SJRCS_011
Scratcher
Registered: 2011-02-07
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

DigiTechs wrote:

SJRCS_011 wrote:

So what does this do again? Does it just echo back the variable declarations across all connections?
If so, I'd be willing to make the EXE for you in C++.
And maybe I'll make a Jar as well  tongue

EDIT: nvm about the EXE, apparently my IDE doesn't have a built-in compiler  sad

Upload the solution somwhere (or source files) and i'll compile them for you, since my C++ program has a built in compiler (thanks, Microsoft)

yeah, I'm using Eclipse (for C/C++), cause I don't have admin permissions on my computer  sad
But Magnie has already said that the EXE will be ready within 10 hours, so we should be fine.

Last edited by SJRCS_011 (2012-04-27 10:42:00)


http://i.imgur.com/vQqtH.png
Learning to Program in a Nutshell:  "You're missing a closing parentheses" - LS97

Offline

 

#18 2012-04-27 11:16:57

SJRCS_011
Scratcher
Registered: 2011-02-07
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

Magnie, do you want broadcasts to be echoed as well, or only variable declarations?
Oh, and I could make the multiple networks thing very easily in java.
(in my java code)It'd be as easy as calling a new instance of one class and keeping track of it via ArrayList.

EDIT: Great, now I also feel like making a PHP version of this.
I am really going insane (partly because I know I can do it very easily  tongue )

Last edited by SJRCS_011 (2012-04-27 11:34:12)


http://i.imgur.com/vQqtH.png
Learning to Program in a Nutshell:  "You're missing a closing parentheses" - LS97

Offline

 

#19 2012-04-27 13:01:01

Magnie
Scratcher
Registered: 2007-12-12
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

The Mesh Server just takes all input and sends it to everyone else connected. So technically both.

Offline

 

#20 2012-04-27 13:18:27

SJRCS_011
Scratcher
Registered: 2011-02-07
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

Magnie wrote:

The Mesh Server just takes all input and sends it to everyone else connected. So technically both.

ok, I'll build in a verifier though so only mesh commands go through.


http://i.imgur.com/vQqtH.png
Learning to Program in a Nutshell:  "You're missing a closing parentheses" - LS97

Offline

 

#21 2012-04-27 14:20:21

Magnie
Scratcher
Registered: 2007-12-12
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

EXE-build1 can be downloaded here.  smile

Offline

 

#22 2012-04-27 14:46:18

SJRCS_011
Scratcher
Registered: 2011-02-07
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

Here's my java alpha
On SourceForge!

Last edited by SJRCS_011 (2012-04-27 14:46:42)


http://i.imgur.com/vQqtH.png
Learning to Program in a Nutshell:  "You're missing a closing parentheses" - LS97

Offline

 

#23 2012-04-27 14:51:59

Magnie
Scratcher
Registered: 2007-12-12
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

SJRCS_011 wrote:

Here's my java alpha
On SourceForge!

Sweet! Thanks!

Offline

 

#24 2012-04-27 15:00:32

SJRCS_011
Scratcher
Registered: 2011-02-07
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

Magnie wrote:

SJRCS_011 wrote:

Here's my java alpha
On SourceForge!

Sweet! Thanks!

I'll add the multiple networks later.  Gotta work on a science project.....
could you tell me if you encounter any bugs?

Last edited by SJRCS_011 (2012-04-27 15:00:55)


http://i.imgur.com/vQqtH.png
Learning to Program in a Nutshell:  "You're missing a closing parentheses" - LS97

Offline

 

#25 2012-04-27 15:17:38

Magnie
Scratcher
Registered: 2007-12-12
Posts: 1000+

Re: Mesh Python Server - Host a Mesh Server on a VPS!

SJRCS_011 wrote:

Magnie wrote:

SJRCS_011 wrote:

Here's my java alpha
On SourceForge!

Sweet! Thanks!

I'll add the multiple networks later.  Gotta work on a science project.....
could you tell me if you encounter any bugs?

Okay, first bug! I can't seem to send broadcasts or sensor-updates to other clients that are connected.  tongue

It also took me a bit to realize that it's running in the background. xD

Offline

 

Board footer