[转]讨论 When to fix bugs

agiletesting上的一个讨论,看到这篇回复,觉得写得真不错,转载一下,自己加了点点评注,用“[XY]”开头。

From: agile-testing@yahoogroups.com
[mailto:agile-testing@yahoogroups.com] On Behalf Of ext STEURS
Stefan
Subject: RE: [agile-testing] When to fix bugs

I
think Boris Beizer has written a couple of articles about related subjects.  One
of them was called ‘Why software is different.’
One of
the issues he describes is the lack of a clear ’cause – effect’ path from
‘fault’ to ‘failure’.  This exposes itself in different ways, one of them being
that seemingly small ‘faults’ can lead to catastrophic failures.  It is nearly
impossible to tell from the fault what effect it will have ‘in the real world’. 
Another way this problem exposes itself is that there is no well defined
distance between cause and effect.  The ‘travel’ between the fault being
triggered and the failure revealing itself as a symptom may be long (we have all
experienced how painful a debugging process can be to come to the root of the
cause).
[XY] 通常要发现问题并非是最痛苦最困难的事情,而是在和开发人员沟通(有的公司可能没这个阶段)确认是否真的是一个bug的时候,以及确认是bug后要定位问题根源并修复的时候。中国人常讲“失之毫厘,谬之千里”,非常有道理。从另一方面说明,每个基层工作人员(开发、测试工程师)要注重细节,必须极力避免人为造成的error,因为此时的“差不多”、“没关系”、“无所谓”,很有可能在以后导致代价很高的bug修复。

What
are the measures we can take to expose these problems?  Most of our testing is
fixed result, fixed scenario testing.  We start a test from an initial state,
perform some actions/events, and look at an outcome after we usually go back to
the initial state.  All the tests are executed in a predictable and defined
order in a single stream.
The
reason why we do this is to make the debugging easier.  A well defined path with
a well determined set of actions/values is easy to repeat.  However this leads
to the DDT syndrome where the code becomes resilient against the ‘fixed’ tests
(another article that Boris wrote and deserves more attention).  Without
variation in the tests, no new bugs will surface and they will continue to live
in shelter until a user does something that hasn’t been tested
 
Even
without the code changing, bugs may still ‘erupt’.  We upgrade a third party
library, a platform, etc. and the software breaks.
 
If we
want to have more variation and better coverage, we have to provide for tests
that exercise more values and more paths.  Probably we will need also an oracle
that is more dynamic and allows to predict expected results on the fly, or at
least, will allow to assert that results are not invalid (which is
a difference).  Of course, given that oracles are usually also software
implementations, they suffer also from ‘bugs’.  And there we are again with our
oracle problem.
[XY] 测试的深度和广度是必须思考的问题,当然,是要和测试的预算之间做一个平衡:-)

Without being too pessimistic, we can save ourselves already some trouble
by making sure we cover all probable causes.  Stress testing without accurate
prediction of results can already check whether the software easily breaks for
random sequences of events and values.  That is doesn’t break will raise
confidence and if it breaks we may be in for some difficult debugging but with
proper implementation of logging and tracing, life of debugging can become a lot
easier.
[XY] 我认为testability可以概括此段的含义,软件的可测性更加好,调试工作会更容易,而软件测试也能更有效更便捷地进行。

Some
defensive programming may also be useful.  Make sure that problems don’t get a
chance to hide or shelter.  Assert on values and sequences.  Handle and Log any
exception that occurs.  Make sure that the software doesn’t continue in the
presence of a real error.  No ‘catch all’ clauses.
[XY] 从软件编程或设计的角度提出的建议。

And
like Michael says.  Lock-step approaches don’t allow for ‘rich’ testing.  We
need to partition the spectrum of testing and perform some ‘out-of-step’ testing
for specific purposes and needs that are difficult to tailor within a
micro-cycle of agile development.
 
Regards, Stefan.

About Kaveri, Yi XU

Agile Coach & Consutlant
This entry was posted in Agile&Scrum&Testing&TA. Bookmark the permalink.

发表评论

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / 更改 )

Twitter picture

You are commenting using your Twitter account. Log Out / 更改 )

Facebook photo

You are commenting using your Facebook account. Log Out / 更改 )

Google+ photo

You are commenting using your Google+ account. Log Out / 更改 )

Connecting to %s