powertac-server intialisation

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

powertac-server intialisation

Sachi
Hi,

I am a student from Kings college london. I have been working on the power-tac project for my final dissertation topic. I have been downloaded the server and sample broker and imported them into STS. I followed the steps of starting the server and running the sample broker in STS as you have stated in your threads. The only output i got from that is -- Username not given - default is 'Sample'.

if i try running the server within STS itself, by setting up the run configuration for bootstrap ans sim sessions, it gives a number of errors. Could you please help me with how I should go about testing the sample broker.

I also wanted to know about working with the visualizer. Will i be able to run the smaple broker with the visualizer. When i tried it says, competition running already.
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

grampajohn
Administrator
Sachi wrote
I am a student from Kings college london. I have been working on the power-tac project for my final dissertation topic. I have been downloaded the server and sample broker and imported them into STS. I followed the steps of starting the server and running the sample broker in STS as you have stated in your threads. The only output i got from that is -- Username not given - default is 'Sample'.
This is a pretty sketchy account of what you saw, so I'm not sure what to say. I have been running games with the sample broker pretty much every day, and it's working fine.  However, we are working hard toward a new release, and there have been some changes. It's possible that you got an inconsistent set. You may need to do an update.

Also, I have not deployed any of the updated modules to the snapshot repo, so you will need to do mvn install on common before the sample broker will run correctly.
if i try running the server within STS itself, by setting up the run configuration for bootstrap ans sim sessions, it gives a number of errors. Could you please help me with how I should go about testing the sample broker.
Sure, but I would have to know what the errors are.
I also wanted to know about working with the visualizer. Will i be able to run the smaple broker with the visualizer. When i tried it says, competition running already.
I am running a game right now with the sample broker and visualizer, in "standalone" mode. To do that, you run from the visualizer directory using mvn jetty:run. This only works if you have done mvn install on ALL the server modules first.

The visualizer also runs in "remote" mode, but that's complicated to set up and only really of interest if you are setting up the tournament scheduler.

Does this help?

John
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

Sachi
hi,

Thanks for the reply. I recently downloaded all the modules again, and imported all into STS. When i run the server in bootstrap session in STS, with project - server-main and main class - org.powertac.server.PowerTacServer and arguments - bootstrap.xml. I starts at first, but then throws an exception of Java heap space. The console output is this:

1734 [Thread-3] INFO org.apache.activemq.broker.BrokerService - Using Persistence Adapter: MemoryPersistenceAdapter
1756 [Thread-3] INFO org.apache.activemq.broker.BrokerService - ActiveMQ 5.6.0 JMS Message Broker (simJmsProvider) is starting
1756 [Thread-3] INFO org.apache.activemq.broker.BrokerService - For help or more information please see: http://activemq.apache.org/
1857 [JMX connector] INFO org.apache.activemq.broker.jmx.ManagementContext - JMX consoles can connect to service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
1946 [Thread-3] INFO org.apache.activemq.transport.TransportServerThreadSupport - Listening for connections at: tcp://localhost:61616
1946 [Thread-3] INFO org.apache.activemq.broker.TransportConnector - Connector tcp://localhost:61616 Started
1948 [Thread-3] INFO org.apache.activemq.broker.BrokerService - ActiveMQ JMS Message Broker (simJmsProvider, ID:unknown-e0-f8-47-00-aa-20.home-57248-1342609848907-0:1) started
Exception in thread "InactivityMonitor WriteCheck" java.lang.OutOfMemoryError: Java heap space
        at java.util.zip.InflaterInputStream.<init>(InflaterInputStream.java:71)
        at java.util.zip.ZipFile$1.<init>(ZipFile.java:224)
        at java.util.zip.ZipFile.getInputStream(ZipFile.java:224)
        at java.util.zip.ZipFile.getInputStream(ZipFile.java:192)
        at java.util.jar.JarFile.getManifestFromReference(JarFile.java:167)
        at java.util.jar.JarFile.getManifest(JarFile.java:148)
        at sun.misc.URLClassPath$JarLoader$2.getManifest(URLClassPath.java:696)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:228)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        at org.apache.activemq.transport.AbstractInactivityMonitor.writeCheck(AbstractInactivityMonitor.java:142)
        at org.apache.activemq.transport.AbstractInactivityMonitor$2.run(AbstractInactivityMonitor.java:111)
        at org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
        at java.util.TimerThread.mainLoop(Timer.java:512)
        at java.util.TimerThread.run(Timer.java:462)
Exception in thread "Thread-3" java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:2734)
        at java.util.Vector.ensureCapacityHelper(Vector.java:226)
        at java.util.Vector.add(Vector.java:728)
        at org.powertac.householdcustomer.appliances.Dishwasher.createDailyPossibilityOperationVector(Dishwasher.java:97)
        at org.powertac.householdcustomer.appliances.Appliance.createWeeklyPossibilityOperationVector(Appliance.java:188)
        at org.powertac.householdcustomer.customers.Household.checkProbability(Household.java:301)
        at org.powertac.householdcustomer.customers.Household.fillAppliances(Household.java:404)
        at org.powertac.householdcustomer.customers.Household.initialize(Household.java:159)
        at org.powertac.householdcustomer.customers.Village.initialize(Village.java:287)
        at org.powertac.householdcustomer.HouseholdCustomerService.initialize(HouseholdCustomerService.java:298)
        at org.powertac.server.CompetitionControlService.configurePlugins(CompetitionControlService.java:583)
        at org.powertac.server.CompetitionControlService.setup(CompetitionControlService.java:344)
        at org.powertac.server.CompetitionControlService.runOnce(CompetitionControlService.java:298)
        at org.powertac.server.CompetitionSetupService$1.run(CompetitionSetupService.java:371)
45266 [ActiveMQ ShutdownHook] INFO org.apache.activemq.broker.BrokerService - ActiveMQ Message Broker (simJmsProvider, ID:unknown-e0-f8-47-00-aa-20.home-57248-1342609848907-0:1) is shutting down
46830 [ActiveMQ ShutdownHook] INFO org.apache.activemq.broker.TransportConnector - Connector tcp://localhost:61616 Stopped
46855 [ActiveMQ ShutdownHook] INFO org.apache.activemq.broker.BrokerService - ActiveMQ JMS Message Broker (simJmsProvider, ID:unknown-e0-f8-47-00-aa-20.home-57248-1342609848907-0:1) stopped

And then when I run it in Sim-session. With arguments --sim --boot-data=file:bootstrap.xml --brokers=Sample. I get this error

[Fatal Error] :1:1: Premature end of file.
preGame: Error reading boot dataset: javax.xml.xpath.XPathExpressionException
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

Sachi
Hi,

I am trying to update the server modules using this command - git submodule foreach git pull origin master.It stops at the first module accounting and says:

Your local changes to the following files would be overwritten by merge:
        .project
        src/main/java/org/powertac/tariffmarket/TariffMarketService.java
Please, commit your changes or stash them before you can merge.

So im trying to use the command: git checkout --filename (here using TariffMarketService.java) But it gives an error saying it did not match any file known to git.

Im not sure how Im suppose to stash or revert the changes. I hope you can help. Thanks
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

grampajohn
Administrator
Hello, Sachi -

Yes, the Power TAC setup needs more than the default memory allocation.

Sachi wrote
I am trying to update the server modules using this command - git submodule foreach git pull origin master.It stops at the first module accounting and says:

Your local changes to the following files would be overwritten by merge:
        .project
        src/main/java/org/powertac/tariffmarket/TariffMarketService.java
Please, commit your changes or stash them before you can merge.
This is a typical situation. If you want to keep your changes, you can put them on a branch (git stash; git checkout -b new-branch; git stash pop), or you can just git stash them and forget about them. The .classpath and .project files are automatically managed by Eclipse, and often get updated without your knowledge, so you can safely just stash and forget them. Eclipse can always regenerate them - just do Maven->Update Project Configuration on the project.
So im trying to use the command: git checkout --filename (here using TariffMarketService.java) But it gives an error saying it did not match any file known to git.
It's git checkout path/to/filename. Just copy the path from the git status display.
Im not sure how Im suppose to stash or revert the changes. I hope you can help. Thanks
Any time you have changes you don't want to keep, it's safe to stash them. There are other methods, but all of them that I know are more complex.

Does this help?

John
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

Sachi
Hi,

I used git checkout path/to/filename with the path displayed, but its still giving me the same error. Im trying to do this, since Im not sure if all the modules are consistent and updated for everything to run correctly.

I also did mvn install on all the modules. But there were build failures for the accounting, server-interface and household-customer modules. Is that the reason im not being able to run the server and test the sample broker? If so how should I be resolving this?

I tried these steps, in the server-distribution directory
1) mvn -Pcli -Dexec.args="--boot bootstrap.xml" (Result of this was build success)
2)mvn -Pcli -Dexec.args="--sim --boot-data bootstrap.xml --brokers Sample" (Result of this was also build success)
3) In STS set up run configuration with Project = sample-broker and Main class = org.powertac.samplebroker.BrokerMain

output of the third step in STS console was - Username not given - default is 'Sample'

Is this the correct way to run and test the sample broker? Im not sure if I have got the correct outputs everywhere.
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

grampajohn
Administrator
Sachi wrote
I also did mvn install on all the modules. But there were build failures for the accounting, server-interface and household-customer modules. Is that the reason im not being able to run the server and test the sample broker? If so how should I be resolving this?
This sounds correct, but for a while today there was an inconsistency that caused builds of accounting and server-interface to fail. There have been at least 8 updates today. The latest versions are now consistent again. One thing to be aware of is that there have been several changes in the last two days in the common module, and it appears as a dependency both directly and transitively in all the server modules except for server-interface. As a result, any time common changes, you have to rebuild server-interface (and you have to do a "clean" on it inside STS), and then you have to re-build all the other modules (and again, you have to do a "clean" on them in STS). The server-main module must always be last.
I tried these steps, in the server-distribution directory
1) mvn -Pcli -Dexec.args="--boot bootstrap.xml" (Result of this was build success)
This should have taken about 5 minutes and produced a file called bootstrap.xml.
2)mvn -Pcli -Dexec.args="--sim --boot-data bootstrap.xml --brokers Sample" (Result of this was also build success)
This should have waited for the Sample broker to log in, and then taken about two hours to run. It should wait indefinitely for broker logins unless you set the firstLoginTimeout and loginTimeout properties to non-zero values.
3) In STS set up run configuration with Project = sample-broker and Main class = org.powertac.samplebroker.BrokerMain

output of the third step in STS console was - Username not given - default is 'Sample'
This sounds like you have a very old version of the sample broker. There is no class org.powertac.samplebroker.BrokerMain in the current master branch in github. The correct class is org.powertac.samplebroker.core.BrokerMain, and the default username is not "Sample", but "broker". The username is not given in the command line, but in the config file. I believe the README file is up-to-date. The README file in server-distribution is also up-to-date in the current master branch on github.

Does this help?

John

Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

Sachi
Thank you. This does help.

I did have an old version of the broker. I downloaded the new version. I am now running the server in sim mode as:

mvn -Pcli -Dexec.args="--sim --boot-data bootstrap.xml --brokers broker

The server seems to have started, and in STS after running the configuration, it says starting session 1.

So i think it seems to be running fine. How long should I wait until i can get some result on the broker? Im not very sure how I should be evaluating results.

Also if i want to test my own broker, is it the same procedure as testing the sample broker?

Thanks a lot.
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

grampajohn
Administrator
Sachi wrote
So i think it seems to be running fine. How long should I wait until i can get some result on the broker? Im not very sure how I should be evaluating results.
Both the server and broker produce extensive log files, both a trace log and a state log. The state log is a full record of the simulation state, and it can also be a full record of the agent state if you use the logging aspects correctly.
Also if i want to test my own broker, is it the same procedure as testing the sample broker?
Yes. Please note that the interaction protocol between broker and simulation has changed. It is documented on GitHub for the case with a tournament scheduler, but if you are just running a broker against a local simulation server the main change is that the broker can no longer create its own JMS queue. By default, the broker's input queue is named from the broker's username, as supplied to the server. The default name of the server's input queue has not changed.

I'm glad you are making progress.

John
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

Sachi
Hi,

Im using the sample broker setup, to code my own broker.

Regarding the previous thread, while testing the sample broker, I used the username 'broker' and ran the sim session. But in the config files, the username is set as 'Sample'.

My question is: in order to test my own broker, if i change the username in the config files, is that the name I should be using to start my session? Im confused on how i should start testing my own broker.

Also while testing the sample broker, the broker.trace file result was something like this:

0 INFO  core.BrokerPropertiesService: adding broker.properties
270 INFO  core.JmsManagementService: registerMessageListener(Sample, org.powertac.samplebroker.core.BrokerMessageReceiver@1fdbb27d)
687 INFO  core.PowerTacBroker: Listening on queue Sample
756 INFO  core.MessageDispatcher: sending text:
<broker-authentication username="Sample" password="secret"/>
3858 INFO  core.MessageDispatcher: sending text:

the file continues in the same way. So im guessing this is not the right output, since it is not getting any response. Is there a resolve to this?
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

grampajohn
Administrator
Sachi wrote
Im using the sample broker setup, to code my own broker.

My question is: in order to test my own broker, if i change the username in the config files, is that the name I should be using to start my session? Im confused on how i should start testing my own broker.
The server will only communicate with brokers that are given in its command-line. Brokers with other usernames will be ignored.
While testing the sample broker, the broker.trace file result was something like this:

0 INFO  core.BrokerPropertiesService: adding broker.properties
270 INFO  core.JmsManagementService: registerMessageListener(Sample, org.powertac.samplebroker.core.BrokerMessageReceiver@1fdbb27d)
687 INFO  core.PowerTacBroker: Listening on queue Sample
756 INFO  core.MessageDispatcher: sending text:
<broker-authentication username="Sample" password="secret"/>
3858 INFO  core.MessageDispatcher: sending text:

the file continues in the same way. So im guessing this is not the right output, since it is not getting any response. Is there a resolve to this?
This is exactly what happens when the server is ignoring the broker. If you look at the server log, you will see messages that look more or less like "Unauthorized attempt to log in ...".

It sounds like you are almost there.

John
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

Sachi
Hi John,

My server log file i.e. in the server-distribution log file - sim.trace does have the message "Unauthorized attempt to log in Sample".

How am I suppose to give the broker in command line. Because I was only starting the sim session in command line, and then running the broker in STS (by setting up the run configuration).

when i start the sim session in command line, it says 'started' and then remains on that. And when i run the broker in STS, it says 'Starting session 1'.

Can you help me go about this. Thanks
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

grampajohn
Administrator
Sachi wrote
How am I suppose to give the broker in command line. Because I was only starting the sim session in command line, and then running the broker in STS (by setting up the run configuration).
Instructions for using the command-line interface are in server-distribution/README.txt. Is this not working for you?

John
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

Sachi
I have followed the instructions in server-distribution/README.txt file. Using these 2 commands:

1) mvn -Pcli -Dexec.args="--boot bootstrap.xml"
2)mvn -Pcli -Dexec.args="--sim --boot-data bootstrap.xml --brokers broker"

this works, but the result I am getting is what I have explained in the previous 2 threads i.e. regarding the broker.trace file and server log files.
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

grampajohn
Administrator
Sachi wrote
I have followed the instructions in server-distribution/README.txt file. Using these 2 commands:

1) mvn -Pcli -Dexec.args="--boot bootstrap.xml"
2)mvn -Pcli -Dexec.args="--sim --boot-data bootstrap.xml --brokers broker"

this works, but the result I am getting is what I have explained in the previous 2 threads i.e. regarding the broker.trace file and server log files.
This should work just in case the username of your broker is "broker". If the username is "Sample", then that last bit needs to be '--brokers Sample'.

Does this help?

John
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

Sachi
In reply to this post by Sachi
Hi,

I just updated all the modules. And there were changes in the common and few other files. I rebuild them all in STS. I am getting an error in CompetitionSetupService file in the server-main folder.

the error is this:

public void preGame ()
  {    
   
    //competitionId = competition.getId();
    String suffix = serverProps.getProperty("server.logfileSuffix", "x");
    logService.startLog(suffix);
    log.info("preGame() - start");
    IdGenerator.recycle();    ---------- In this line, since this method is not declared in IdGenerator class
    // Create competition instance
    competition = Competition.newInstance("game-" + gameId);

Is this the correct updated file?
   
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

grampajohn
Administrator
Sachi wrote
Hi,

I just updated all the modules. And there were changes in the common and few other files. I rebuild them all in STS. I am getting an error in CompetitionSetupService file in the server-main folder.

the error is this:

public void preGame ()
  {    
   
    //competitionId = competition.getId();
    String suffix = serverProps.getProperty("server.logfileSuffix", "x");
    logService.startLog(suffix);
    log.info("preGame() - start");
    IdGenerator.recycle();    ---------- In this line, since this method is not declared in IdGenerator class
    // Create competition instance
    competition = Competition.newInstance("game-" + gameId);

Is this the correct updated file?
There are two possibilities, depending on exactly where this error is arising. If it's in STS, then your setup probably depends on a deployed or installed version of common, not the version that's in STS. This method was added to common very recently, to deal with an overflow problem that can happen when using the server with the embedded visualizer. So assuming your version of common actually has this method, what you have to do is `mvn clean install` on common. If your version of common does not have this method, then you are not up to date.

Another possibility with STS is that it has not re-compiled deeply enough. You can fix this by forcing a rebuild (do a clean) on common, then on server-interface, then on all the other server modules except server-main, then on server-main.

If this is happening when using the server from the command line, or when compiling the server with maven, then the problem is that you have the transitive dependency problem I discussed earlier. When common changes, you have to re-compile and re-install all the server modules, in order, starting with server-interface and ending with server-main.

Just so you know, I do not anticipate any further changes in common between now and at least November. I'm working on issue #576, so most of the remaining changes are in the visualizer and the tournament scheduler, but it's possible there might be one or two more small changes in server-main. We are also hoping to add an additional genco model in the next few days (a wind-park model), but that will be a pretty simple re-build.

Does this help?
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

Sachi
Hi John,

Thank you. This did help. I am now getting a correct output in the broker.trace and server log files.

I would like to run the broker on the visualizer. But the visualizer is not starting, when i hit mvn jetty:run. It gives an error like:

Compilation failure
[ERROR] /Users/sachisolanki/powertac-server/visualizer/src/main/java/org/powertac/visualizer/services/competitioncontrol/WebCompetitionControlService.java:[85,11] simSession(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.util.List<java.lang.String>,java.lang.String) in org.powertac.common.interfaces.CompetitionSetup cannot be applied to (java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.util.List<java.lang.String>)

It had started initially, but since 2 days it is not working. How should i resolve this?  
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

grampajohn
Administrator
Sachi wrote
I would like to run the broker on the visualizer. But the visualizer is not starting, when i hit mvn jetty:run. It gives an error like:

Compilation failure
[ERROR] /Users/sachisolanki/powertac-server/visualizer/src/main/java/org/powertac/visualizer/services/competitioncontrol/WebCompetitionControlService.java:[85,11] simSession(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.util.List<java.lang.String>,java.lang.String) in org.powertac.common.interfaces.CompetitionSetup cannot be applied to (java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.util.List<java.lang.String>)

It had started initially, but since 2 days it is not working. How should i resolve this?
Sorry, this is my fault. I have been working on the server <-> tournament-scheduler interface, and had to add a parameter to the game-startup api. I have not gone back and fixed the embedded visualizer yet. This is now issue #600. I will try to get it fixed today.

John
Reply | Threaded
Open this post in threaded view
|

Re: powertac-server intialisation

grampajohn
Administrator
I have just now pushed an update to visualizer that fixes the API error (issue #600). It now runs games correctly in its embedded mode. I apologize for the inconvenience.

John