[nova] Rebuild from cell0 limitation for personality files
I wanted to get some wider attention on what I think will be a
limitation in the rebuild from cell0 spec . The main idea in that
spec is the user tries to create a server which fails during scheduling
to find a host and puts the instance into ERROR state and "buries" it in
cell0. With a new microversion, the user can attempt to rebuild the
server from cell0 and go through scheduling again. This is part of a
bigger effort to support pre-emptible instances where scheduling fails
during server create, an external reaper service tries to make room by
destroying some pre-emptible instances, and then the server gets rebuilt.
The issue I've found is that if the server was created with personality
files using microversion 2.1, fails, and then the user goes to rebuild
it, their personality files will be lost because (1) we don't persist
personality files anywhere, not even the RequestSpec and (2)
microversion 2.57 deprecated the personality files parameter from the
rebuild API . Since you'll have to use a newer microversion > 2.57 to
rebuild from cell0, you can't specify the personality files again.
Are we OK with this? We deprecated personality files for a reason  so
this is a natural extension of that deprecation saying "going forward
you should not be using these things and if you do there could be
The user has an obvious recourse: delete the server and re-create it
rather than rebuild from cell0.
I don't want to consider un-deprecating personality files in the rebuild
API just for this new microversion to rebuild from cell0.
Another alternative is persisting personality files in the RequestSpec
but I'm not sure how much people would support that - although it would
fix some other known limitations with personality files, like the fact
that if you create a server with files and then shelve offload, the
files are gone when you unshelve. I have a similar issue with cross-cell
https://review.openstack.org/#/c/648686/1/specs/train/approved/enable-rebuild-for-instances-in-cell0.rst at 72
https://review.openstack.org/#/c/642807/3/specs/train/approved/cross-cell-resize.rst at 454