Flaw in customer tariff evaluation model ?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Flaw in customer tariff evaluation model ?

easwar1977
Dear Powertac team,

    We suspect that there might be a flaw in the way a customer evaluates tariff. While we were experimenting with the Sample Broker code, on the tariff market, we were trying out high withdrawal penalties. For some reason, there are customers who sign up for such a Broker. After some time, customers switch tariffs (or change brokers) and this gives the broker a huge profit. In fact, every six hour some customers switch and the broker who signed them up ends up with lot of profit because of (high) withdrawal penalty.

    We looked up on the customer utility model in Section 4.1.1 of the Powertac 2017 specification document; it does take into account withdrawal penalty as a variable to evaluate utility. But for some reason, we find some customers sign up for this broker in spite of high withdrawal penalty and then switch when new tariffs are seen. A rational behavior would be that the customer is discouraged from signing up with brokers who have high withdrawal penalty or if they are already signed they should not switch brokers before the end of their contract period.

   Such brokers exploit the customer. We request the Powertac team to look into the issue and confirm if our observation is right and kindly provide a cure to the problem.

  We have attached a sample figure in the URL ( http://imgur.com/a/5NyWB ) highlighting the above issue. The brokers in the game are Sample Broker (ver 1.4.1) and Sample Broker with (high) withdrawal penalty.  

Thanks,
VidyutVanika team.


     


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Flaw in customer tariff evaluation model ?

grampajohn
Administrator
Dear Easwar -

Thank you very much for reporting this. I have recorded it as Issue #931, and I will dig into it quickly. You could help by adding a comment to this issue with two pieces of information:
* The terms of the tariff that is causing this behavior, and
* Which customers are falling for this trick. If my initial hypothesis is correct, it would be mainly the large-population models, like Centerville and Brookside.
For the tariff, you could just paste in the xml from your broker log when it sent the tariff.

Thanks much for your help. We'll do our best to get this fixed in the next day or two, and have it ready for Monday's qualifying round.

John
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Flaw in customer tariff evaluation model ?

easwar1977
Dear John,

   One need not do anything fancy to discover this flaw. Use the Sample Broker, modify the tariff of the Sample Broker to have high expiration interval, high withdrawal penalt (that's all we did actually). Play a game between this modified Sample Broker  and original Sample Broker (with of course the default broker in), you can replicate the picture of profit and the behavior as described above.  

  Actually, we did not the check what kind of customers signed-up with us; so we really have to look at the log to figure this out. But you can replicate the scenario from the above para.

 We are well past work-time here and hence immediate access to log file is difficult as the log files are in work location.

Cheers,
Easwar  
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Flaw in customer tariff evaluation model ?

grampajohn
Administrator
In reply to this post by easwar1977
Dear Easwar and all -

I have been able to reproduce the problem you reported, which I have recorded as Issue #931. It turns out that the only customer model that is misbehaving is the batteries, which means a workaround is to remove the battery models from the server configuration. You can do this by adding the line

   customer.model.battery.instances =

to your server.properties file in server-distribution, or just create a new config file with just this line and specify it when you run the server.

I did discover a small error in evaluating signup and withdrawal payments, but I seriously doubt it would have any observable impact except in cases of models that use non-standard profile lengths (there are a couple, including batteries) and situations where small signup/withdraw payments are influencing choices between two otherwise very similar tariffs. That fix is in version 1.4.3-SNAPSHOT, which I just deployed.

My recommendation to Govert is that he use the 1.4.3-SNAPSHOT version of the server for the start of the qualifying round tomorrow, and that he eliminate the batteries from the server configuration. I hope this does not create major problems for anyone. I will try to get tariff eval working correctly for batteries and put out a 1.4.3 release well before the start of the final round.

Best of luck in the upcoming competition.

John
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Flaw in customer tariff evaluation model ?

grampajohn
Administrator
In reply to this post by easwar1977
Dear colleagues -

I found the problem with tariff evaluation for the battery model -- the default storage tariff produced a positive expected value (because it offers regulation rates), while the competing tariffs all produced negative expected values. The utility calculation did not envision this case, and ended up preferring to leave a tariff with a large negative withdraw penalty. I have fixed and tested Issue #931, and deployed an update to 1.4.3-SNAPSHOT. If you download and run the current server-distribution master you can try it out.

I apologize for the confusion.

John
Loading...