I am a beginner in RL ,and I want to join the PowerTAC as a learning process.However, i have no idea how to modify the sample broker after reading sample broker code and some papers .
So ,if i want to implement a RL arithmetic like q-learning in wholesale market , where should i add my code?
If i want to implement a RL arithmetic like q-learning in wholesale market , where should i add my code?
The sample broker handles the wholesale market in the MarketManagerService, and the tariff and balancing markets in the PortfolioManagerService. There is, of course, some interaction between the two, specifically the MarketManager needs to know how much energy it should buy or sell in each of the 24 open timeslots. It gets this information by calling portfolioManager.collectUsage() -- see MarketManagerService line 285. Of course this is just an estimate, and not a very sophisticated one. Note that the estimate for a given timeslot can change as the lead-time counts down from 24 to 1; for example, when customers evaluate tariffs they may decide to switch, and so the number of customers may change.
Keep in mind that wholesale trading can also be used to make some money by learning how prices vary over the 24h horizon, and using that information to buy energy when prices are lower and sell when prices are higher. Note that the broker is not exactly a price-taker in this market; its activity can materially prices, so you may have to learn or at least estimate this effect as well.