Memcache - sharing variables
23 Jan 2012 00:08 #17029
by dab77
Memcache - sharing variables was created by dab77
Hi i just discovered python-memcache, and i've found it so simple and useful that i think it can be useful to others.
I have one gladevcp window with my buttons and LEDs, one vismach window and another gladevcp window with other infos on my machine.
I needed a way to make those three programs comunicate and i've found this memcache, which reserve some Mb of memory and let you save any datas in a very simple way.
just put:
import memcache
mc = memcache.Client()
into any python file you want to intercomunicate and then to write:
mc.set("value_name","value")
and to read:
my_value = mc.get("value_name")
Into ubuntu, you need just to install:
sudo apt-get install memcached python-memcache
memcached is the daemon that set th11211 port and reserve memory.
if interested you can find other infos here: blog.isotoma.com/2009/09/memcache-in-2-minutes/
Wish it's helpful
Davide.
I have one gladevcp window with my buttons and LEDs, one vismach window and another gladevcp window with other infos on my machine.
I needed a way to make those three programs comunicate and i've found this memcache, which reserve some Mb of memory and let you save any datas in a very simple way.
just put:
import memcache
mc = memcache.Client()
into any python file you want to intercomunicate and then to write:
mc.set("value_name","value")
and to read:
my_value = mc.get("value_name")
Into ubuntu, you need just to install:
sudo apt-get install memcached python-memcache
memcached is the daemon that set th11211 port and reserve memory.
if interested you can find other infos here: blog.isotoma.com/2009/09/memcache-in-2-minutes/
Wish it's helpful
Davide.
Please Log in or Create an account to join the conversation.
23 Jan 2012 07:13 #17035
by ArcEye
Replied by ArcEye on topic Re:Memcache - sharing variables
Hi
Interesting.
Looks like it is a socket server running on localhost:11201
Co-incidentally I am presently writing a socket server in C to interact with userspace modules and store the values of HAL pins in realtime.
I don't use python, but will have a look at it to see if there are any ideas in the mechanism of it that I can 'borrow'
regards
Interesting.
Looks like it is a socket server running on localhost:11201
Co-incidentally I am presently writing a socket server in C to interact with userspace modules and store the values of HAL pins in realtime.
I don't use python, but will have a look at it to see if there are any ideas in the mechanism of it that I can 'borrow'
regards
Please Log in or Create an account to join the conversation.
23 Jan 2012 10:03 #17037
by mhaberler
Replied by mhaberler on topic Re:Memcache - sharing variables
As you noted, there is a lack of a coherent, late binding IPC mechanism beyond HAL and NML, and this issue comes up in many corners of linuxcnc, not just between Python scripts. Also, persistent state (state retained across linuxcnc invocations) is an issue as this is currently, well, 'unstructured', so it isnt just a question of IPC.
I am addressing this issue as I work on a rewrite of the interpreter and task (mostly its structure, not its contents). I have decided to use redis as such an IPC mechanism, and have partially integrated it.
While likely will not be something tangible for others to try before summer, I encourage anybody seriously looking into such a mechanism install redis (easy) and use redis-py (easy to use) as a python binding. It can only help if we collectively gain some experience with it.
http://http://redis.io/
github.com/andymccurdy/redis-py
- Michael
I am addressing this issue as I work on a rewrite of the interpreter and task (mostly its structure, not its contents). I have decided to use redis as such an IPC mechanism, and have partially integrated it.
While likely will not be something tangible for others to try before summer, I encourage anybody seriously looking into such a mechanism install redis (easy) and use redis-py (easy to use) as a python binding. It can only help if we collectively gain some experience with it.
http://http://redis.io/
github.com/andymccurdy/redis-py
- Michael
Please Log in or Create an account to join the conversation.
23 Jan 2012 14:05 #17050
by dab77
Replied by dab77 on topic Re:Memcache - sharing variables
Michael, you say right.
going to study this redis, which i never heard before.
p.s. just an add-on with memcache, you can start a client from every programming language, so into C you can recall the vars saved from Python or whatelse.
going to study this redis, which i never heard before.
p.s. just an add-on with memcache, you can start a client from every programming language, so into C you can recall the vars saved from Python or whatelse.
Please Log in or Create an account to join the conversation.
23 Jan 2012 15:59 - 23 Jan 2012 16:11 #17055
by ArcEye
Replied by ArcEye on topic Re:Memcache - sharing variables
Hi Michael
The primary one is that connecting remotely reliably, necessitates data be serialised to byte level, otherwise there is no way to know the data sizes used on the remote machine ( is int 16 bit or 32? what padding is used, etc) so you cannot pass structs etc.
All the converting values to strings and back again is quite unwieldly and requires rigid message formats.
I will study this to see if it prevents me having to re-invent the wheel!
regards
This looks very promising, a key based database server would solve some of the problems of socket servers that I have discovered.It can only help if we collectively gain some experience with it.
The primary one is that connecting remotely reliably, necessitates data be serialised to byte level, otherwise there is no way to know the data sizes used on the remote machine ( is int 16 bit or 32? what padding is used, etc) so you cannot pass structs etc.
All the converting values to strings and back again is quite unwieldly and requires rigid message formats.
I will study this to see if it prevents me having to re-invent the wheel!
regards
Last edit: 23 Jan 2012 16:11 by ArcEye.
Please Log in or Create an account to join the conversation.
24 Jan 2012 11:03 - 24 Jan 2012 11:10 #17094
by ArcEye
Replied by ArcEye on topic Re:Memcache - sharing variables
For those who prefer to program in C, have found these libraries for redis which implement most of the client commands
code.google.com/p/credis/
Having a play with them now, the example from the wiki will throw a seg fault unless you spot that it uses an arbitrary port address of 6789 despite the default for redis being 6379.
Change that and it works fine so far
regards
code.google.com/p/credis/
Having a play with them now, the example from the wiki will throw a seg fault unless you spot that it uses an arbitrary port address of 6789 despite the default for redis being 6379.
Change that and it works fine so far
regards
Last edit: 24 Jan 2012 11:10 by ArcEye. Reason: add info
Please Log in or Create an account to join the conversation.
25 Jan 2012 00:09 #17106
by mhaberler
Replied by mhaberler on topic Re:Memcache - sharing variables
You pulled an dud
redis.io/clients clearly marks suggested clients, and credis isnt one of them
stick to the bindings which have a star marked on this page, those are actually used, and I still have to find a problem with them
My suggestion is: for C, use hiredis, and for Python: redis-py
- Michael
redis.io/clients clearly marks suggested clients, and credis isnt one of them
stick to the bindings which have a star marked on this page, those are actually used, and I still have to find a problem with them
My suggestion is: for C, use hiredis, and for Python: redis-py
- Michael
Please Log in or Create an account to join the conversation.
25 Jan 2012 12:41 - 25 Jan 2012 12:47 #17122
by ArcEye
Replied by ArcEye on topic Re:Memcache - sharing variables
Calling credis a dud is a little unkind, it works as it claims, even if the documentation consists solely of the credis.h file and a single wiki page example that does not work (as previously covered) so far as I can find.
However having looked a hiredis, I do like the printf style functions, keeps the input nearer a command line entry and handles some of the string formatting you would have to do before the function call anyway.
I only have one afternoon invested in credis, think I can probably switch to hiredis and port my code so far in about 10 minutes!
regards
However having looked a hiredis, I do like the printf style functions, keeps the input nearer a command line entry and handles some of the string formatting you would have to do before the function call anyway.
I only have one afternoon invested in credis, think I can probably switch to hiredis and port my code so far in about 10 minutes!
regards
Last edit: 25 Jan 2012 12:47 by ArcEye.
Please Log in or Create an account to join the conversation.
25 Jan 2012 17:28 #17129
by mhaberler
Replied by mhaberler on topic Re:Memcache - sharing variables
yes, my wording wasnt appropriate, sorry. I'll refrain from posting after beer #2
I should write down my current plans for restructuring LinuxCNC in a wiki page, to outline how redis fits in.
the status currently is:
- the following components already have basic redis capability: task, interpreter, TkInter Python applications like Axis, GladeVCP (in fact all PyGTK or GTK applications like Touchy), any userspace C or Python HAL component.
- they can both use the redis set/get and derived mechanisms to export/retrieve global variables
- they also can use the publish/subscribe mechanism to declare interest in what is called a 'channel', and get notified asynchronously when a channel-related event happens, for instance, when a variable changes value (this is why the async support is needed).
As mentioned, I use hiredis for C, and redis-py for Python for synchronous I/O. For publish/subscribe I currently use Twisted, which is the fasted path to get this to work on the Python side, but there are some minor issues left with TkInter apps.
The open issues are:
- design a redis data model which includes the tool table, and possibly other state components in an extensible way
- implement this without breaking backwards compatibility (it will be a lot of work until all user interfaces are adapted, and I dont know others than Axis and Touchy very well)
- export/import to/from external files like tooltable
I'd be interested to discuss any plans when accessing interpreter and task state.
- Michael
I should write down my current plans for restructuring LinuxCNC in a wiki page, to outline how redis fits in.
the status currently is:
- the following components already have basic redis capability: task, interpreter, TkInter Python applications like Axis, GladeVCP (in fact all PyGTK or GTK applications like Touchy), any userspace C or Python HAL component.
- they can both use the redis set/get and derived mechanisms to export/retrieve global variables
- they also can use the publish/subscribe mechanism to declare interest in what is called a 'channel', and get notified asynchronously when a channel-related event happens, for instance, when a variable changes value (this is why the async support is needed).
As mentioned, I use hiredis for C, and redis-py for Python for synchronous I/O. For publish/subscribe I currently use Twisted, which is the fasted path to get this to work on the Python side, but there are some minor issues left with TkInter apps.
The open issues are:
- design a redis data model which includes the tool table, and possibly other state components in an extensible way
- implement this without breaking backwards compatibility (it will be a lot of work until all user interfaces are adapted, and I dont know others than Axis and Touchy very well)
- export/import to/from external files like tooltable
I'd be interested to discuss any plans when accessing interpreter and task state.
- Michael
Please Log in or Create an account to join the conversation.
25 Jan 2012 18:32 #17131
by BigJohnT
Replied by BigJohnT on topic Re:Memcache - sharing variables
mhaberler wrote:
Hmmm, I might need to set my posting enable breath analyzer a bit lower.
John
I'll refrain from posting after beer #2
- Michael
Hmmm, I might need to set my posting enable breath analyzer a bit lower.
John
Please Log in or Create an account to join the conversation.
Time to create page: 0.095 seconds