Allow replicated volumes to be recoverable

Allow replicated volumes to be recoverable

Currently, once they are in ‘error’ or ‘failed-over’ states you can’t
do anything with them. We need to allow enable_replication after the
admin has resolved any issues that may have prevented the operation.

Same thing for after a fail-over, you cannot enable replication on it.

This change allows for more states to be recoverable via enable
replication. It also more uniformly sets the state to ‘error’ when
something bad happens.

'enabled' --> replication is on.
'disabled' --> replication is off by design.
'failed-over' --> we have just failed over. Replication is off.
'error' --> an error occurred during the last operation.

allowed flow is:
'enabled' --> 'failed-over'/'disabled'/'error'
'error' --> 'enabled'/'error'
'disabled' --> 'enabled'/'error'
'failed-over' --> 'enabled'/'error'

Change-Id: Icbc22b8501b048b1755019305430177ba0a9385f
Partial-Bug: #1491631