Potential Bug in the Evaluation of Tariffs

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Potential Bug in the Evaluation of Tariffs

HarryRose
Hi,

I think I've spotted a bug in the way customers evaluate tariffs. I've checked out the latest code from your GitHub repos, and I've been looking at how the agents calculate their utilities.

In the household-customer source, customers evaluate their tariffs in the Village.possibilityEvaluationNewTariffs function. This calls the Village.costEstimation function, which in turn calls estimateFixedTariffPayments, Village.estimateShiftingVariableTariffPayment and Village.estimateVariableTariffPayment, which returns the expected cost of a tariff to the agents.

However, the units in which cost is returned seem to be mixed up between these functions. In estimateVariableTariffPayment, the sum of costs over all timeslots is taken and then divided by the number of days:

double estimateVariableTariffPayment (Tariff tariff, String type)
  {

    double finalCostSummary = 0;

    int serial =
      (int) ((timeService.getCurrentTime().getMillis() - timeService.getBase()) / TimeService.HOUR);
    Instant base =
      new Instant(timeService.getCurrentTime().getMillis() - serial
                  * TimeService.HOUR);
    int daylimit = (int) (serial / VillageConstants.HOURS_OF_DAY) + 1;

    for (int day: daysList) {
      if (day < daylimit)
        day = (int) (day + (daylimit / VillageConstants.RANDOM_DAYS_NUMBER));
      Instant now = base.plus(day * TimeService.DAY);
      double costSummary = 0;
      double summary = 0, cumulativeSummary = 0;

      for (int hour = 0; hour < VillageConstants.HOURS_OF_DAY; hour++) {

        summary =
          getBaseConsumptions(day, hour, type)
                  + getControllableConsumptions(day, hour, type);

        log.debug("Cost for hour " + hour + ":"
                  + tariff.getUsageCharge(now, 1, 0));
        cumulativeSummary += summary;
        costSummary -= tariff.getUsageCharge(now, summary, cumulativeSummary);
        now = now.plus(TimeService.HOUR);
      }
      log.debug("Variable Cost Summary: " + finalCostSummary);
      finalCostSummary += costSummary;
    }
    return finalCostSummary / VillageConstants.RANDOM_DAYS_NUMBER;
  }

which I believe gives a cost per day.

In estimateFixedTariffPayments, the value of the periodic payment (per timeslot) plus the lifecyclePayment / minDuration in milliseconds is returned. I think this gives some mix between cost per timeslot and cost per millisecond:

double estimateFixedTariffPayments (Tariff tariff)
  {
    double lifecyclePayment =
      -tariff.getEarlyWithdrawPayment() - tariff.getSignupPayment();
    
    double minDuration;

    // When there is not a Minimum Duration of the contract, you cannot divide
    // with the duration
    // because you don't know it.
    if (tariff.getMinDuration() == 0)
      minDuration = VillageConstants.MEAN_TARIFF_DURATION * TimeService.DAY;
    else
      minDuration = tariff.getMinDuration();

    log.debug("Minimum Duration: " + minDuration);
    return (-tariff.getPeriodicPayment() + (lifecyclePayment / minDuration));
  }

The estimateShiftingVariableTariffPayment also returns cost per day. I think that this means that comparisons of expected costs between tariff types will be incorrect.

For example, consider a fixed rate tariff whose minimum duration is 1 day, and whose signup fee is 864x10^5, withdrawal fee is 0, and periodic payments is 0 and assume no variable payments. estimateFixedTariffPayments would calculate the cost to be 1.

Compare this to a tariff that has no signup, withdrawal or periodic fees, but whose variable costs cause an agent to spend 15 in expectation over three days. This returns the cost as 5.

The agent would therefore believe that the fixed rate tariff was better insofar as the calculated cost per day is 1 (the entire cost over the minimum period of the contract) whereas the variable tariff is calculated to cost 5 per day. However, in reality, the cost for an agent on the fixed rate tariff for one day is 864x10^5.

Is this code correct?

Reply | Threaded
Open this post in threaded view
|

Re: Potential Bug in the Evaluation of Tariffs

achryso
Hello there.

Since I have implemented the models for the households and the office complexes, I have examined the code again to see the mistake that you are talking about, and it seems that you are right.

There were a lot of changes in the metrics of the tariff and tariff specifications over the passing of the versions that I have lost track of that part that was left unchanged.

I have made corrections over these lines and updated the repositories. Be as kind as to pull the latest changes and tell me if the corrections seems alright with you too.

Thank you in advance.

Reply | Threaded
Open this post in threaded view
|

Re: Potential Bug in the Evaluation of Tariffs

HarryRose
Hi.

Thanks, that looks correct now.  All cost estimations now appear to give the cost per day.

When I clone the server repo and then do "git submodule init" followed by "git submodule update", it still seems to pull in the old sources, so I guess that needs updating.  However, I've had a look at the source code on github for household-customer and officecomplex-customer, and they both seem to fix the bug.

Why, in costEstimation(...) in Village.java, do you divide the value returned by one million?

Thanks,

Harry
Reply | Threaded
Open this post in threaded view
|

Re: Potential Bug in the Evaluation of Tariffs

achryso
I am glad to hear that everything works correctly now.

The only reason that the cost Estimation is divided by one million is to make easier the evaluation process, since smaller numbers help me with the math without overflowing.

Let's say that the cost in this case is measured in MWh and thousands of dollars or euros. The only reason I do this is the aforementioned one.

Reply | Threaded
Open this post in threaded view
|

Re: Potential Bug in the Evaluation of Tariffs

grampajohn
Administrator
In reply to this post by HarryRose
HarryRose wrote
Thanks, that looks correct now.  All cost estimations now appear to give the cost per day.
Thanks for spotting this, Harry. It's a big help.

Obviously, we'll use the updated version for the tournament. There's also a small config change coming in the next day or two (which should not require a release), to set the slopes of the up-regulation and down-regulation prices in the balancing market to non-zero values. I still have to do some testing.

The question is, should we make a point release to include the change to the customer model? I think the answer is yes. I will try to get it done before the end of next week.

When I clone the server repo and then do "git submodule init" followed by "git submodule update", it still seems to pull in the old sources, so I guess that needs updating.  However, I've had a look at the source code on github for household-customer and officecomplex-customer, and they both seem to fix the bug.
Better to do
  git submodule foreach git pull origin master

I have not checked the "submodule update" function for a while, but I am not surprised that it does not work. Most likely the powertac-server repo has the wrong version ID values for the submodules. It's almost impossible to keep those up to date.

Hope this helps...

John
Reply | Threaded
Open this post in threaded view
|

Re: Potential Bug in the Evaluation of Tariffs

Prashant Reddy
In reply to this post by achryso
Hi Harry and all,

I've just committed the corresponding fix to factored-customer
(essentially copied Antonios' change).

Please also update the factored-customer module to pick up that change.

Thanks,
Prashant