Looking for feedback: Bridge to Python

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Looking for feedback: Bridge to Python

pascalwhoop
Hi everyone,
I wanna throw my hat in the ring this year. But I'd also like to use Python for my agent. So I intend to build a bridge to let Python fans join the party as well.

I created an issue. I am not sure which repo whould be best, it's in the core repo for now.

Some feedback would be welcome. Who would have interest in using Python? A lot of the hot libraries and frameworks are in python (at least in the deep RL environment). Also, what do you think is the best approach? I would think using the sample-broker would be the best idea and then just pass everything from that to a python runtime which works with the messages and does it's thing before answering the server through the sample-broker proxy.
Reply | Threaded
Open this post in threaded view
|

Re: Looking for feedback: Bridge to Python

grampajohn
Administrator
So far, we've been keeping all the issues, as well as the wiki, in the powertac-server repo. Just one place for the whole project. One set of milestones, one place to see status.

The broker-adapter repo contains an implementation that might give you a reasonable start on this - it's an out-of-process adapter that extracts the incoming xml and passes it through a fifo, and accepts xml going the other way through another fifo, packages it up with the key, and sends it off to the server. So it encapsulates the interface to the server but does not do much with the xml other than keep track of state. It should even allow you to throttle the server so you can stop your broker and inspect things while the server waits.

The alternative would be to re-implement all of broker-core, which seems like a lot of extra work for not much benefit. If you want to use it, I recommend you start by updating the pom.xml to 1.5.0-SNAPSHOT.

Cheers 

John

On Fri, Feb 16, 2018 at 2:26 AM pascalwhoop [via Power TAC Developers] <[hidden email]> wrote:
Hi everyone,
I wanna throw my hat in the ring this year. But I'd also like to use Python for my agent. So I intend to build a bridge to let Python fans join the party as well.

I created an issue. I am not sure which repo whould be best, it's in the core repo for now.

Some feedback would be welcome. Who would have interest in using Python? A lot of the hot libraries and frameworks are in python (at least in the deep RL environment). Also, what do you think is the best approach? I would think using the sample-broker would be the best idea and then just pass everything from that to a python runtime which works with the messages and does it's thing before answering the server through the sample-broker proxy.


If you reply to this email, your message will be added to the discussion below:
http://power-tac-developers.975333.n3.nabble.com/Looking-for-feedback-Bridge-to-Python-tp4026778.html
To start a new topic under Power TAC Developers, email [hidden email]
To unsubscribe from Power TAC Developers, click here.
NAML
Reply | Threaded
Open this post in threaded view
|

Re: Looking for feedback: Bridge to Python

grampajohn
Administrator
In reply to this post by pascalwhoop
Dear colleagues -

Today I pushed and deployed an update to powertac-core that adds IPC support to the broker-core module. This means you can write brokers in non-JVM languages or run them across machine boundaries. The mechanism is that you write a simple broker in Java that implements the samplebroker.interfaces.IpcAdapter interface, telling it which messages to pass through as raw xml, and which messages to process locally by unmarshaling them into the Java structures defined in the common module. You can freely do both to any message types you wish.

The broker-adapter repo contains an example of using this functionality. The CharStreamAdapter class implements the IpcAdapter interface and passes exported data through an entity in the local filesystem. I've been using it with fifos. It also reads from another filesystem entity and sends the incoming xml messages back to the server. The repo contains an example (r-broker/fork-prototype.R) written in R that forks a broker, connecting it to two fifos, then reads the broker-output fifo and prints the incoming xml messages from the server. The README describes briefly how to use it. The config file in src/main/resources/config/rbroker.xml shows how to configure the raw/cooked message lists, and it also shows how to run in interactive mode in which the broker always pauses the server in each timeslot, allowing it to take more time for decisions and even allow for human interaction. To use this you have to tell the server to allow broker-originated pause requests - see server-distribution/pause.props to see how to set this up.

As always, if you try this and have questions or run into problems, please let us know.

Cheers -

John
Reply | Threaded
Open this post in threaded view
|

Re: Looking for feedback: Bridge to Python

pascalwhoop
Hi everyone,
I have extended Johns work and added the ability to connect remotely via a socket technology (GRPC) as well as written a super light-weight sample in Python.

The Python Adapter is here
https://github.com/pascalwhoop/broker-python

The adaptation are currently in PRs. If anyone has actual interest in using Python it would be good to get a quick raised hand so we know if we need to hurry or can take the time we want to mature the functionality. Right now it’s hidden in branches and forks.