Consistent State File Results

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Consistent State File Results

frankyn
Hi,

I'm new here, thanks for having this forum available to us.

I have been working with PowerTAC attempting to get the same output state file when using a modified configuration of the VillageType file given different situations. I'm using the same boostrap-data, model seeds, and weather data. When I attempt running a simulation using this method, I provide two simulations the same modified VillageType file as well as the same files mentioned above.  I use one to verify the other, but I'm currently unable to get consistent powertac-sim state files.

I'm not sure why this is occurring, but I'd appreciate any help the PowerTAC dev team may provide.

Thank you,
Frank Natividad
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Consistent State File Results

grampajohn
Administrator
Hello, Frank -

Welcome to the Power TAC community.

frankyn wrote
I have been working with PowerTAC attempting to get the same output state file when using a modified configuration of the VillageType file given different situations. I'm using the same boostrap-data, model seeds, and weather data. When I attempt running a simulation using this method, I provide two simulations the same modified VillageType file as well as the same files mentioned above.  I use one to verify the other, but I'm currently unable to get consistent powertac-sim state files.
This is a well-known issue. Repeatability is much better than it was when we started, but it's not yet perfect. We think the remaining problem is in the factored-customer model, but we have not yet tracked it down. However, keep in mind that even if repeatability were perfect in the sim server, random broker behavior can also cause divergence. But even so, removing individual sources of variability still increases the statistical power of an empirical study with a simulation like Power TAC. See Sodomka's paper on this topic for more info.

Cheers -

John
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Consistent State File Results

frankyn
Thanks John!

I appreciate your help clearing that up and I'll review Sodomka's paper.

Frank N.


Loading...