Updated 1.1.0-SNAPSHOT deployed

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Updated 1.1.0-SNAPSHOT deployed

grampajohn
Administrator
The fix for issue #682 has caused a small regression; we apologize for that. The symptom is that weather data does not match what's expected in the server, and the server usually throws an exception and won't run. There are two easy fixes:

1. Use the 1.1.0-SNAPSHOT version of the server. The source is updated, and I have just deployed the fixed version so you don't need to download sources. All you need is the current server-distribution package.

2. Add a line to config/server.properties to set the forecastOffset to zero, as:
  server.weatherService.forecastOffset = 0

Please let us know if this does not work for you as expected.

John
Reply | Threaded
Open this post in threaded view
|

Re: Updated 1.1.0-SNAPSHOT deployed - Clarification

grampajohn
Administrator
Colleagues -

The two fixes for the weather-data regression are alternatives. If you don't want to use the current 1.1.0-SNAPSHOT version, you need to set the forecastOffset to zero. You don't need to do both. I apologize; the original message was not clear about this.

Cheers -

John
Reply | Threaded
Open this post in threaded view
|

Re: Updated 1.1.0-SNAPSHOT deployed

Jonathan
In reply to this post by grampajohn
Hi John... I used the version you provided me and did the forecastOffset = 0 edit and now it is working. Thank you very much.  
Reply | Threaded
Open this post in threaded view
|

Updated 1.1.0-SNAPSHOT deployed

grampajohn
Administrator
In reply to this post by grampajohn
Dear colleagues -

Today I have assembled and deployed another update to the 1.1.0-SNAPSHOT series. This is mostly a cleanup version in preparation for next month's trial. It breaks out the balancing market for better modularity, adds/updates documentation in a number of places, and fixes a few minor bugs. It also contains most of the representation for the storage feature (needed for electric vehicles, also usable for water heaters and cold-storage). My goal is to have all of the storage functionality available very quickly after the February trial.

As always, I encourage you to try it out. If you are using a server source environment, do a complete update and make sure you get the new balancing-market module (it's not a submodule of powertac-server). Otherwise, just download the server-distribution module, and follow the instructions in the README.

Please let me know if you have problems with this.

John
Reply | Threaded
Open this post in threaded view
|

Updated 1.1.0-SNAPSHOT deployed

grampajohn
Administrator
Colleagues -

Today I have deployed another update to the 1.1.0-SNAPSHOT series of the Power TAC server. There is also a small change in the sample broker code that logs an error when an attempt is made to send a message with an improper ID. There are a number of changes in this version, mostly to enhance stability and to prevent improper cross-broker manipulation. Here's a quick summary:

  • Brokers can no longer (attempt to) supersede each others tariffs (Issue #731).
  • Broker messages will no longer be sent to the server if their id values are not from the current series (Issue #734). This could happen in a multi-session setting, for example, if a broker saved and attempted to re-use artifacts like tariff specifications across games. Also, the server will no longer accept messages from brokers with id values outside the broker's assigned range. This prevents potential collisions across brokers, such as tariffs from different brokers having the same ids.
  • The sample-broker's supersede example was broken by the fix for Issue #722. This is now fixed (Issue #736).
  • The household-customer and officecomplex-customer models are now evaluating tariffs in the intended way (Issue #735). Prior to this fix, it was possible to get these models to pay high signup fees for tariffs with slightly lower per-kWh prices.
  • All the infrastructure for a general storage-device model is in place in the server (Issue #653), but there are as yet no customer models that use it. From the broker's standpoint, there is a new type of Rate called RegulationRate, and the scope of BalancingOrders and EconomicControlEvents is expanded to include down-regulation and up-regulation through battery discharge, in addition to simple curtailment. For the curious, the discussion of controllable capacity is updated to show how these features work in more detail.
  • Documentation of the sample-broker code has been improved in a number of areas (Issue #720) and the javadocs are now being generated on the Minnesota CI server (Issue #713).
  • There was sometimes a problem when loading weather data from a file (Issue #730). This is now fixed.

The updated server is deployed, so if you are running the 1.1.0-SNAPSHOT server, you should see them the next time you run the server. To use the latest sample-broker code, you will need to use the latest versions of the classes in samplebroker.core. Also, the example PortfolioManager is updated; if you are using it or code from it, you might want to see what has changed. As always, please let me know if you run into trouble with this version, or if you have suggestions for improving the Power TAC software or our processes. Cheers - John

Reply | Threaded
Open this post in threaded view
|

Updated 1.1.0-SNAPSHOT deployed

grampajohn
Administrator
Dear colleagues -

This evening I have deployed another snapshot. It contains several updates, and one of them may cause you to start seeing some WARN messages in your broker logs (also in the server logs). Here are the main changes:

  • Orders with invalid values no longer crash the sim (issues #738 and #739).
  • Clock slippage now aborts a sim (and will cause a game to be re-run in tournament mode) instead of just creating a mess (Issues #727 and #729).
  • If you configure expected game length equal to minimum game length, you get a game of exactly that length, rather than a divide-by-zero error (issue #741).
  • The final bit of infrastructure is now in place and tested for supporting storage customers, including electric vehicles and thermal storage devices. These customers now have a way to evaluate tariffs that include RegulationRates (issue #740). We are working on two customer models that will offer storage capacity; they should be available soon.
  • There is now a minimum order quantity that is acceptable to the wholesale market. This is a common feature of real-world markets that has been missing from Power TAC (issue #742). A slightly more detailed description of this feature is in the 2014 spec that should be out in a couple of days.
I updated the sample broker's MarketManagerService to receive the configured minimum order quantity from the Competition instance that is sent when the broker logs in. Because the MarketManagerService already had a concept of minimum order quantity, the change is very simple.

We are tentatively setting the minimum order quantity to 10 kWh. If you have thoughts on this concept or what the limit should be, please let us know.

As always, please let us know if you have trouble with this version, or if you have ideas for improving Power TAC or our processes.

John

Reply | Threaded
Open this post in threaded view
|

Updated 1.1.0-SNAPSHOT deployed

grampajohn
Administrator
Dear colleagues -

In preparation for the next trial, scheduled for 25-28 March, I have deployed another new server snapshot. Other than some improved tests, the primary change is that we have a new customer model that implements thermal storage behavior, and will subscribe to THERMAL_STORAGE tariffs. There are currently five moderately-large cold-storage warehouses in the model, with capacities ranging from 250 to 900 tonnes. These models respond to weather (they need more cooling at higher outdoor temperatures), and they can deliver both up-regulation (by allowing temperature to rise) and down-regulation (by dumping power into the refrigeration units). They can also respond to 100% curtailment for 20 consecutive timeslots without getting out of their control ranges, but a consequence of doing that is that they will then have no up-regulation capacity until they have recovered to their setpoints. The general control scheme for these models is described in the server wiki.

A warning: In order to integrate the new model, I ran into a naming conflict in the visualizer. If you are running the server from the server-distribution package, you will need to do mvn clean first; otherwise you will get a long backtrace from Spring and it won't work.

As always, don't hesitate to let us know if you have problems or suggestions.

Cheers -

John
Reply | Threaded
Open this post in threaded view
|

Re: Updated 1.1.0-SNAPSHOT deployed

grampajohn
Administrator
My previous message about the behavior of the thermal-storage model was not quite accurate: if you curtail one of these, its internal temperature will rise, and while it is recovering there is no available DOWN-regulation because the refrigeration unit is already running at full capacity.

I apologize for any confusion -

John
Reply | Threaded
Open this post in threaded view
|

Re: Updated 1.1.0-SNAPSHOT deployed

grampajohn
Administrator
In reply to this post by grampajohn
Colleagues -

To help you make use of storage devices, I have pushed an updated sample-broker that issues a tariff with a RegulationRate, and also the BalancingOrders that make use of the regulating capacity of the storage devices. The relevant code is in the updated PortfolioManagerService. I have also pushed and deployed several updates to the server relating to tariff evaluation and balancing market operation with respect to storage devices.

All the best -

John
Reply | Threaded
Open this post in threaded view
|

Updated 1.1.0-SNAPSHOT deployed

grampajohn
Administrator
Colleagues -

This evening I deployed another update to the Power TAC server 1.1.0-SNAPSHOT version. The only substantive change is the new genco model we have been discussing for some time. It produces a quadratic supply curve with a fair amount of variability, instead of a few fixed prices and relatively fixed quantities. The stochasticity might provide a bit more scope for arbitrage; I expect much of the variability could be damped out by competitive brokers. However, demand peaks will produce higher prices than what you may have seen in the past. The parameters of the curve and the variability of individual ask orders are configurable; if you think prices are too erratic, we can easily tone it down a bit.

This genco model is the second one we described in Section 5.3 of the new spec; the first one is not yet ready, unfortunately. Also, we are still working on a mapping of ISO price statistics to the parameters of this model, and progress on that has been slow. So the current parameters are a rough guess of what it should look like.

Please give it a try and let me know what you think.

John
Reply | Threaded
Open this post in threaded view
|

Updated 1.1.0-SNAPSHOT deployed

grampajohn
Administrator
In reply to this post by grampajohn
Dear colleagues -

Today I deployed an updated 1.1.0-SNAPSHOT of the Power TAC server. The main change is that the new genco model has been re-worked to produce more realistic prices. The March trial was the first attempt to use a genco model with a price curve close to what we see in real-world wholesale markets that use congestion pricing (LMP). Unfortunately, the first attempt at modeling a quadratic price curve was inadequate, because the price curve was fixed, and the noise component was too simplistic. There was as much noise in the price signal from timeslot to timeslot, and even between successive clearings in the same timeslot, as there was over days or weeks. So the effect of varying demand was hidden under the noise. Even worse, the large noise component between clearings for the same timeslot created very unrealistic arbitrage opportunities.

The revised genco model cuts down on the short-term noise by more than an order of magnitude, and adds long-period noise through a slow mean-reverting random walk that produces a standard deviation of around 10%. In addition, the noise from the random walk is between timeslots only, and not between successive clearings of the same timeslot. You are welcome to tweak the parameters to experiment - they are in genco/src/main/resources/config/genco.properties, and if you copy them over to server-distribution/config/server.properties you can easily change them to override the numbers from the genco module.

Kailash is working to characterize the price patterns in North American markets to derive a statistical model we can use with this code, but it's taking some time. In the meantime, the updated model appears to produce fairly reasonable patterns. The short-term relationship between demand and price is quite apparent, and there is a reasonable variation over days and weeks. In addition, the mean price is closer to the prices produced by the original genco model, so brokers will have more room to maneuver under the prices offered by the default broker.

As always, please let us know if you have questions, concerns, or suggestions for improvement.

John
Reply | Threaded
Open this post in threaded view
|

Updated 1.1.0-SNAPSHOT deployed

grampajohn
Administrator
In reply to this post by grampajohn
Dear Colleagues -

Today I deployed a new SNAPSHOT in the 1.1.0 release series. I expect this will be the last before next week's qualifying round, although there is one annoying but harmless defect (#759) that might yet get fixed by then.

The primary reasons for this snapshot are to incorporate fixes for #755 and #756. In other words, we corrected an error in pricing down-regulation in the balancing market, and we have eliminated discontinuities across the boot-sim boundary in market prices and in the charge-state of storage devices. So now if you examine the trend in market prices in a boot record, the trend will continue at the beginning of the sim session as you should expect.

Other updates include fixes for #743 and #753. The first of these supports older brokers that were compiled before we factored out the balancing market into a separate module; the second should make it easier for brokers to use the TariffRepo for tariffs that have been composed but not sent to the server for publication.

We apologize that the Electric Vehicle model has not made it into the Power TAC model yet. It is coded and mostly working, but it does not yet respond correctly to balancing orders based on storage capacity. It's available in github if anyone wants to play with it.

Best of luck in the qualifying round starting next week, and in the finals next month. Please get your agents registered as soon as possible in both the tradingagents.org site and in the powertac.org tournament manager.

John