* On 2021-09-02 12:13, Huw Davies wrote:
On Thu, Sep 02, 2021 at 11:44:36AM +0300, Saulius Krasuckas wrote:
I mean, if some questions arises about when that input pointer is handled (before other argument validation or maybe after that, or maybe at the end), the crash (and maybe additional crashes) would give some insights about the original algorithm.
Which is exactly why you should *not* write these sort of tests.
Huw, I disagree that there can be a wrong sort of tests in black-box testing. Really don't see this.
We're aiming to match the functionality of the API without copying its implementation. We don't want to know about its internal branching structure.
And I aim at the functionality too: not the implementation, but the exactness of the behavior.
IMO, it would be quite difficult to always ensure the project does the same function in the very different way. Difficult to prove and unnecessary. Because this is a black-box. Do I get this wrong?
Plus, in case a developer starts being afraid that one or another test would reveal some unavoidable details of the original function/algorithm, one could quickly get lost in the doubt and start dropping every second ok-check in the area. Which would be a disaster for a black-box testing.
I don't think these two things can be easily decoupled (even if the original implementation goes open source, even with the compatible license) or is it necessary at all. Just a PoV.
Of course if a particular app depends on Windows returning a specific error code then thats the time to add a test for it.
OK. And leaving the disabled test code only helps to do that (for everyone in doubt). I still saw no harm in that code.
S.
PS. As the removal patch already got accepted, I hope no one minds my response. :)