Internet Connectivity mandatory for powertac ?

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Internet Connectivity mandatory for powertac ?

easwar1977
Hello,

  I am trying to run to server and broker based on the boot data. But if I am not connected to Internet then I do see them working. I guess Internet is required for getting weather updates. Is there a way, I can just run my broker and server without having to connect to the Internet (using some static weather data). This is only during development phase. I understand that it is a different ball game during the tournament.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Internet Connectivity mandatory for powertac ?

gbuijs
Hi Easwar,

It's possible to get the weather from a file.
That way you can run a simulation offline.

To create a weather-file, you can use this Python script.
You'll need to adjust the params, localhost:8080 should become weather.powertac.org:8080.
If you want i can create it for you.

To use a weather-file, add the following to the command :
--weather-data your-weather-file-name

More about the cli options here

Let me know if you need more help.

Cheers,

Govert
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Internet Connectivity mandatory for powertac ?

easwar1977
Dear Govert,

  Thanks for your prompt response. I tried the solution that you proposed. The script runs well with the change of parameter from localhost to weather.powertac.org.

Few questions on the script itself:
a) What is the optimal value for DAYS  parameter (I had set it to 60 because leaving it at 764 took a long amount of time) ?
b) Is there a constraint for Start date which we give as command line argument ? Or can we give any start date ?

Anyways, I gave sixty for DAYS and some start date in 2009 and I have weather.xml file. Next I tried running the following command from server directory

mvn -Pcli -Dexec.args="--sim --boot-data boot.xml --weather-data weather.xml"

The first time it downloads all repositories (jars) so it my laptop has to be online. But I don't know why this is the case when I have ran several times mvn -P web from the same directory (Did it not download repositories then ?)

If I switch the internet off (this is after downloading the repositories), I get the following error trying to run the server using above command
[INFO] Scanning for projects...
[INFO]                                                                        
[INFO] ------------------------------------------------------------------------
[INFO] Building Power TAC distribution 1.4.1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- exec-maven-plugin:1.5.0:exec (default-cli) @ server-distribution ---
java.net.UnknownHostException: weather.powertac.org
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at java.net.Socket.connect(Socket.java:538)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
        at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
        at sun.net.www.http.HttpClient.New(HttpClient.java:308)
        at sun.net.www.http.HttpClient.New(HttpClient.java:326)
        at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1202)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032)
        at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:966)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
        at org.powertac.server.WeatherService$WeatherRequester.webRequest(WeatherService.java:342)
        at org.powertac.server.WeatherService$WeatherRequester.run(WeatherService.java:308)
        at org.powertac.server.WeatherService.activate(WeatherService.java:186)
        at org.powertac.server.CompetitionControlService.step(CompetitionControlService.java:782)
        at org.powertac.server.CompetitionControlService.access$900(CompetitionControlService.java:67)
        at org.powertac.server.CompetitionControlService$SimRunner.run(CompetitionControlService.java:1112)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 28.259 s
[INFO] Finished at: 2017-03-25T10:32:20+05:30
[INFO] Final Memory: 11M/166M
[INFO] ------------------------------------------------------------------------

If internet services are on, when I run the above mvn command, the server message that gets displayed is,
[INFO] --- exec-maven-plugin:1.5.0:exec (default-cli) @ server-distribution ---

And then I don't know what do after that. I tried running the broker (the broker runs well) on another terminal;   but I don't see how the simulation is running (visualizer or log somewhere ). I don't know if something is going on;  

Best Regards,
Easwar





Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Internet Connectivity mandatory for powertac ?

gbuijs
This post was updated on .
Hi Easwar,

It's best to set days to 1092 and set the start date to 2009-01-01.
That way you get all the available weather data.
I've created a weather file for you here (72M).
EDIT Better the zipped version (5M).
You need the whole period, because the sim takes a random start date from it.

You are correct, the first time it downloads all the repos.
I suspect the error comes from missing dates from your weather file.
If the date isn't in your file, the sim tries to get it online.

When you run a sim, the output is in the log files, in the log/ directory.
This goes for both for the sim and the broker.
If you want to use a visualizer, look at the new visualizer2.

Cheers,

Govert

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Internet Connectivity mandatory for powertac ?

easwar1977
Hi,

  Really thanks for your reply. Indeed I have the weather data downloaded and I don't have the previous error that I displayed in the previous message.  The trace and state files also get generated both for server and broker.

  But I don't really see if the broker connects or communicates to the server. In the visualizer, we input the name of the broker which is used to recognize the broker by the server. How does the server and broker communicate in this scenario. Is there some property file where I need to input something to the let the server and broker talk to each other.

  Apologies if I am being very silly with my questions.

FYI, log from the broker :

33991 INFO  core.MessageDispatcher: sending text:
<broker-authentication username="MySample" password="secret" brokerTime="1490537278926"/>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Internet Connectivity mandatory for powertac ?

grampajohn
Administrator
Hello, Easwar
easwar1977 wrote
I don't really see if the broker connects or communicates to the server. In the visualizer, we input the name of the broker which is used to recognize the broker by the server. How does the server and broker communicate in this scenario. Is there some property file where I need to input something to the let the server and broker talk to each other.
The broker communicates with the server through a tcp socket, regardless of whether the server is local or remote. If you look at your broker.properties file, you will see the line
samplebroker.core.jmsManagementService.jmsBrokerUrl = tcp://localhost:61616
So the server, by default, listens for tcp connections on port 61616. If you want to communicate with a remote server, you will have to change that line or override it on the command line, replacing localhost with a hostname or ip address. In a tournament setup, the broker gets the server connection information from the Tournament Scheduler.

Does this make sense?

John
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Internet Connectivity mandatory for powertac ?

easwar1977
Hi John,

  Yes, I understand this and those are the exactly the same in broker.properties file. But when I run the sim offline with weather data from offline, I don't see much log from the broker in broker.trace and it all ends

33991 INFO  core.MessageDispatcher: sending text:
<broker-authentication username="MySample" password="secret" brokerTime="1490537278926"/>

This line is repeated in the trace file several times with the first number being different. And I felt somehow broker and server aren't communicating. I don't see the boot.xml getting parsed at all by the broker.

Best Regards,
Easwar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Internet Connectivity mandatory for powertac ?

grampajohn
Administrator
Hello,  Easwar -

It appears that your broker is not connecting to the server. The question is whether the server was running and waiting for the broker to log in during this period. It is usually easiest, but not critical, to start the server before you start the broker. If you are using the server as mvn -Pweb2, you can go to the games tab to do this; otherwise (in server-distribution) the command line for the server is something like
mvn -Pcli -Dexec.args="--sim --boot-data name-of-boot.xml --weather-data name-of-weather-file --brokers Sample"
Details are in server-distribution/README.md.

The first number in the server and broker logs is the number of msec since the process started.

Cheers -

John
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Internet Connectivity mandatory for powertac ?

easwar1977
Hi John,

  Thanks for your reply. Indeed the server was running and waiting for the broker and the broker was also running.

  I fixed the error using your command

mvn -Pcli -Dexec.args="--sim --boot-data name-of-boot.xml --weather-data name-of-weather-file --brokers Sample"

The difference is I did not use the --broker option of the mvn. The README.md (as far as I read) file does not contain  any reference to that. The --broker option is the key thing for server and broker to communicate with each other (Just like how we input the broker name in the -Pweb option when we use the visualizer to run simulations).  

In fact if you want to run multiple brokers we need to do the below way,

mvn -Pcli -Dexec.args="--sim --boot-data name-of-boot.xml --weather-data name-of-weather-file --brokers Sample1 --brokers Sample2 --brokers Sample3"

The key is --brokers options have to be repeated for every broker that you want to run. Giving the --brokers option once and giving all the broker names at one shot doesn't really work. So, we need to give the command the above way in case we want offline simulations using multiple brokers.

I haven't checked the -Pweb2 option to do offline simulations but I will check and let you know.

Best Regards,
Easwar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Internet Connectivity mandatory for powertac ?

grampajohn
Administrator
Hello, Easwar -
easwar1977 wrote
  I fixed the error using your command

mvn -Pcli -Dexec.args="--sim --boot-data name-of-boot.xml --weather-data name-of-weather-file --brokers Sample"

....

In fact if you want to run multiple brokers we need to do the below way,

mvn -Pcli -Dexec.args="--sim --boot-data name-of-boot.xml --weather-data name-of-weather-file --brokers Sample1 --brokers Sample2 --brokers Sample3"

The key is --brokers options have to be repeated for every broker that you want to run. Giving the --brokers option once and giving all the broker names at one shot doesn't really work. So, we need to give the command the above way in case we want offline simulations using multiple brokers.
Actually, I believe the README.md in server-distribution is pretty clear about this. It says "--brokers broker,... is a comma-separated list (no whitespace allowed) of broker usernames that are expected to log in to the simulation before it starts. If this option is missing and --control is provided, then the broker list will be retrieved from controller-url/broker-list."

So the option is --brokers and not --broker, and the broker names are given as a comma-separated list, without embedded whitespace. It is possible that the option processor allows the --brokers option to appear multiple times in the command line, but I have never tried that. I know it works to give the option once with multiple broker names in a single comma-separated list, because that is how I usually test it.

All the best -

John
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Internet Connectivity mandatory for powertac ?

grampajohn
Administrator
In reply to this post by easwar1977
Hello, Easwar -
easwar1977 wrote
  I fixed the error using your command

mvn -Pcli -Dexec.args="--sim --boot-data name-of-boot.xml --weather-data name-of-weather-file --brokers Sample"

....

In fact if you want to run multiple brokers we need to do the below way,

mvn -Pcli -Dexec.args="--sim --boot-data name-of-boot.xml --weather-data name-of-weather-file --brokers Sample1 --brokers Sample2 --brokers Sample3"

The key is --brokers options have to be repeated for every broker that you want to run. Giving the --brokers option once and giving all the broker names at one shot doesn't really work. So, we need to give the command the above way in case we want offline simulations using multiple brokers.
Actually, I believe the README.md in server-distribution is pretty clear about this. It says "--brokers broker,... is a comma-separated list (no whitespace allowed) of broker usernames that are expected to log in to the simulation before it starts. If this option is missing and --control is provided, then the broker list will be retrieved from controller-url/broker-list."

So the option is --brokers and not --broker, and the broker names are given as a comma-separated list, without embedded whitespace. It is possible that the option processor allows the --brokers option to appear multiple times in the command line, but I have never tried that. I know it works to give the option once with multiple broker names in a single comma-separated list, because that is how I usually test it.

All the best -

John
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Internet Connectivity mandatory for powertac ?

easwar1977
Hi John,

  Yes, You are right. I was actually looking at the Readme.md of the broker by mistake when I wrote the reply. I apologize for not being attentive enough.

  My intent in asking or pointing out in some of these things here is to help newcomers (like me) with the issues they might face;  Not to point to out mistakes towards the powertac team.  If I faced an issue and it is solved and if it is documented well here or somewhere, then it will help others in future.

 I still have to test -P web2 visualizer. I will test it and get back to you.

  Thanks a lot for your patient answers.  Apology once again for the oversight error on my part.

Best Regards,
Easwar

Loading...