Potential restructuring of Power TAC server source
Dear colleagues -
The amount of effort that went into getting the 1.4.1 release pushed out has convinced us to consider a simplification of the way we are managing the server sources on github. Currently there are 22 active repositories on github besides sample-broker and server-distribution. Both sample-broker and server-distribution are released in source form; in these cases, a release consists of updating a version ID and tagging the repo. The modules under powertac-core and powertac-server must be updated, built, and staged to the Sonatype staging repository along with sources and javadocs, from which they are eventually synced to maven central (the final sync is what failed earlier this week). Maven has a release plugin that manages this process, but it was not designed to work with git submodules, and so each release requires a somewhat complex and error-prone process to be repeated 22 times. It takes 6-8 hours if nothing goes wrong, and it's hardly ever that easy.
The original purpose of the module-level module breakdown in our github project space was to let different teams who were developing server components to work together without unnecessary interference from other groups working in different server components. It reduced the temptation to try to fix a problem in module A by changing something in module B without the need to convince the developers of module B that their module needed fixing. This purpose is much less valid today, since virtually all ongoing server development is now done by me, Erik, and Govert (although we would still welcome some help in several areas).
Our proposal is to collapse those 22 github modules into 2: powertac-core and powertac-server. All the modules below those two, such as common and balancing-market, would go away -- they would be folded into powertac-core and powertac-server respectively. Erik has worked out the details, including a way to bring forward all the change history. We have tested it, and the server runs correctly. Installs and deploys are marginally simpler, and release should be vastly simpler.
I can think of two potentially major negative consequences of this change. One is that it would be harder to work with a single module without getting the whole server. I noticed that someone recently forked balancing-market. I'm not sure why, but possibly they had a use for just that piece of code (or maybe they are hoping to fix something and issue a pull request?). The second is that commits for the various modules will be all lumped together in a single history, making it harder to see what has changed in a single module.
I am not sure I would have thought this a good idea if I had not just gone through an unusually painful release process. That makes me worry that this change could cause problems for others. If you have concerns or objections, please let us know quickly, because we will want to make another release soon to incorporate some improved plots in the new visualizer.
Thank you in advance for your thoughts on this matter.