Fix some issues around tempest in fullstack testing doc

The why section in the fullstack testing doc gives a good explanation
of the rational behind the testing and where it fits in the testing
pyramid. However, some of the drawbacks of tempest mentioned aren't
accurate or are misleading. This commit attempts to reword that
piece to clear up any potential sources of confusion.

The difficulty in running tempest doesn't change depending on the
nature of the deployment, since tempest is an external test suite that
interacts with any deployment only through the api. The configuration
and run mechanics do not change whether your cloud is 1 or multiple
nodes. The real difficulty lies in setting up a multinode deployment.

For the failure reporting, if you can't figure out why something
failed from a tempest run it's the same for any end user of the API.
It should be treated as a bug in the project if an end user
can't figure out why something failed from logs and what gets
returned by the API. But, since the fullstack tests are a bit lower
level its not necessarily trying to catch bugs like that. This commit
attempts to reword it to make that distinction clear.

Change-Id: Ie5b01047412deb84fe1457100ecd4af48a1d7efc

diff --git a/doc/source/devref/fullstack_testing.rst b/doc/source/devref/fullstack_testing.rst
index 565fda43c255997c620f4132fb6d40aeaf7f7108..68cb6a518b5056c2dc14dfb637488a1918f0cb65 100644 (file)
--- a/doc/source/devref/fullstack_testing.rst
+++ b/doc/source/devref/fullstack_testing.rst
@@ -28,12 +28,15 @@ Why?
 ----
 
 The idea behind "fullstack" testing is to fill a gap between unit + functional
-tests and Tempest. Tempest tests are expensive to run, difficult to run in
-a multi node environment, and are often very high level and provide little
-indication to what is wrong, only that something is wrong. Developers further
-benefit from full stack testing as it can sufficiently simulate a real
-environment and provide a rapidly reproducible way to verify code as you're
-still writing it.
+tests and Tempest. Tempest tests are expensive to run, and operate only
+through the REST API. So they can only provide an explanation of what went wrong
+gets reported to an end user via the REST API, which is often too high level.
+Additionally, Tempest requires an OpenStack deployment to be run against, which
+can be difficult to configure and setup. The full stack testing addresses
+these issues by taking care of the deployment itself, according to the topology
+that the test requires. Developers further benefit from full stack testing as
+it can sufficiently simulate a real environment and provide a rapidly
+reproducible way to verify code as you're still writing it.
 
 How?
 ----