« first day (1229 days earlier)      last day (3736 days later) » 

2:00 PM
It's in the feature list.
 
Why would your stupid http network wrapper crap connect to a server, send the specified request, and then error out with a "connection refused" error?
 
The feature list is where I look first when I want to check if something has some feature.
 
I don't think "connection refused" means what you think it means
 
mentions nothing about custom reporters.
 
2:01 PM
1 min ago, by R. Martinho Fernandes
The feature list is where I look first when I want to check if something has some feature.
 
well why don't you show us what feature list you found
cause I've viewed the Catch docs a few times and not once did I see a feature list which included anything about custom reporters.
 
@rightfold an exported function can call functions that are not exported... right?
 
that's really not the same thing as having it be properly documented at all.
there's nothing about it in the place you would normally expect to find it- the reference pages.
 
It's enough to make "catch can't do it" a lie.
 
2:04 PM
probably true.
but I'll maintain that if that's all there is about it there, then it's quite reasonable to have that position
 
5 mins ago, by R. Martinho Fernandes
@AndyProwl Catch's docs! Granted there is no exhaustive description of how to write a custom reporter, but it's kinda trivial.
 
that page doesn't include any description at all, not just "no exhaustive". It just says "you can".
and there's no mention of the ability in the reference pages.
it's not unreasonable to check the reference and if it's not in there, conclude it does not exist.
 
Well, I've read this article on how to integrate Catch with MS and it doesn't seem to be as simple as "create a custom reporter".
Maybe that guy didn't know about customer reporters
 
I don't know, I was aware of this ability even before I read the docs.
(Even before it was listed there)
any_runner -h makes that feature obvious.
 
What's the reporter's interface? Is it getting called for every assertion and then you can do what you want in the callback (like notifying the VS framework)?
As @DeadMG pointed out, there's no description whatsoever on how to do that. And I don't think I should be forced to browse a ~10k line header file to figure it out myself.
 
2:10 PM
@AndyProwl I don't think you're forced to do anything whatsoever. Are you?
 
Ahaha you think you can avoid looking at the source
 
@jalf Well, if I want to write a custom reporter, then I'm forced
@CatPlusPlus I could for GMock
Or at least I was pointed by the documentation at the exact place where the interface is defined (don't remember exactly what the workflow was)
 
My point is stop having expectations
 
@AndyProwl Then what the hell did you mean with "And browsing GMock's output in raw text format is not so nice, especially on big projects. "?
 
2:14 PM
That NPE is really funny
 
@R.MartinhoFernandes I mean that I had to write a custom reporter for GMock that integrates with MS Test, so I don't have to browse the raw text of the test output
And there is a documented and simple way to do so
So I did it
 
so I guess the staggeringly surprising conclusion is "Catch does support custom reporters, so it coudl be integrated into MSTest just fine, but so far, no one here have been arsed to write that integration"
 
My point is: Catch alone is not enough for unit testing. I do need a mocking framework, and GMock is probably the best option around. And rather than integrating GMock with Catch and then Catch with VS I'd rather cut off Catch and use GMock and MS Test alone. I haven't yet found anything useful that cannot be done this way and can be done in Catch.
 
@jalf actually it was forked and integrated with MSTest
 
@Abyx And the result is not very good
 
2:20 PM
Because integrating anything into VS is a pain in the ass.
 
@AndyProwl maybe. I didn't try that
 
Also, there's no documentation about writing customer reporters with Catch, while there is documentation about writing customer reporters with GMock.
 
@AndyProwl I haven't found anything that can't be done without a unit test framework entirely. It's not about what can be done, but about how easy it is to do. :)
 
What's to integrate between a mocking framework and testing framework
 
@jalf So I shall rephrase: everything that needs to be done for smooth TDD can be done easily without Catch
 
2:21 PM
Smooth TDD is an oxymoron
 
And about which parts of the process you want to be made easy. Like I said, I don't care about IDE integration, I just want easy-to-write tests, and Catch > GTest there :)
 
@CatPlusPlus It's not
 
@CatPlusPlus Mocking in C++ is pretty crappy anyway.
 
@R.MartinhoFernandes Oh that I don't doubt :v
 
@AndyProwl "smooth" isn't a binary. Some approaches are smoother than others
 
2:22 PM
Need a mock? No mocking framework? Write a class.
Need a mock? Have a mocking framework? Write a class.
 
@jalf I don't think Catch makes TDD any smoother than GMock
 
@AndyProwl Different strokes. I find it requires less boilerplate, and less artificial structure, making it easier to just get to the point and write what you want tested
 
I'm a mocking framework for C++
 
@R.MartinhoFernandes Integrating GMock was not a pain in the ass
 
Mocking in C# is actually neat.
 
2:23 PM
I will mock everything C++
 
@CatPlusPlus No, you're a mock factory
 
@R.MartinhoFernandes That's true. Some framework like Typemock Isolator are very powerful
Don't even require mockable functions to be virtual
 
@AndyProwl aaaaand that is why I don't believe in C++ mocking frameworks
 
NSubstitute looked nice, but I never got to use it
 
@AndyProwl Ugh, Typemock is an abomination :<
 
2:24 PM
I don't subscribe to TDD and hardly ever used mocks
 
Encouraging poor code is not a good thing.
 
@jalf You can make them conditionally virtual
 
if you have to compromise your code to allow your mocking framework to mock, then I prefer not using it
@AndyProwl Oh, so your tests test different code than you'll use in production? Even better ;)
 
:lol:
But it's smooth
 
@jalf Testing the mock frameworks is all the rage these days.
 
2:25 PM
@jalf No they don't
 
@R.MartinhoFernandes as far as I'm concerned, they failed the test ;)
 
  ILoveThisFramework lovable = Mock.Of<ILoveThisFramework>(l =>
    l.DownloadExists("2.0.0.0") == true);
This is only decent method of describing mocks.
 
var dude
 
@jalf oh so he's Java based?
 
@CatPlusPlus conversion is intended there.
 
2:26 PM
@R.MartinhoFernandes I don't think it's an abomination. It gives you more freedom
 
If that generic doesn't return T it's bad
So, var or bust :v
 
You just have more options for verifying behavior
 
@CatPlusPlus It returns an object that you can use to specify more behaviour.
 
Yeah I guess
 
@AndyProwl making stuff virtual when it didn't need to be virtual just means you have more options for introducing bugs and fewer invariants about the behavior of the code you wrote
 
2:28 PM
@CatPlusPlus It implements ILoveThisFramework, but it has a more derived type.
 
I really don't use mocks much
 
Me neither, especially not in C++.
In C++ it's unvariably a pain.
@AndyProwl Freedom to write poor code, yes.
 
@jalf You make them virtual just for the purpose of mocking them. In production code, they don't need to by virtual. You can make them conditionally virtual without problems. The only situation that could lead to problems is if you have a derived class in production that defines a non-virtual function that hides the base class's name. But I don't think that's good practice.
 
It's still different code that you're testing
2
 
@CatPlusPlus Whether a non-overridden function is virtual or not makes no difference apart from performance
 
2:31 PM
@AndyProwl I don't want my code to be different under test than it is in production. That is a horrific solution, even worse than the problem you're trying to solve
@AndyProwl it potentially changes the semantics of your code
 
With C++ being as bad it is, anything but exactly the same thing is a potential hidden bug
 
@jalf How? If no production derived classes are meant to override those functions, I actually want them not to be virtual when not testing them
 
"when not testing them" is the wrong thing in that sentence.
 
@AndyProwl If no production derived classes are meant to override those functions, then I don't want them to be virtual, and then I don't want them to be virtual in the test either, because otherwise my test is useless
I want my test to test the code that is going to run in production
 
My code is not made for testing. It is inherently testable. That's a property of good code.
 
2:33 PM
If you have to rewrite your code when testing it, then what value are you getting out of your test?
 
@jalf So how do you verify that your SUT invokes that function?
 
it shows that your test code passes your tests. It doesn't tell you that your production code does
@AndyProwl SUT?
 
Suitably useless test
 
@jalf System Under Test - Meszaros convention
 
@AndyProwl You look at the expected results.
If there is no way for client code to tell the difference, what are you really testing?
 
2:35 PM
@R.MartinhoFernandes Yeah, so you do state verification. I like being able to do behavior verification
 
@AndyProwl by changing the API to something sane. Perhaps the called function should be a callback function, rather than a class with a virtual function. Or perhaps it should emit a Qt signal that I can connect to something different in my unit test. Or a half-dozen other solutions. It depends on the specific test
@AndyProwl ... by changing the behavior of the thing you're verifying
 
@R.MartinhoFernandes I'm testing that client code calls the function it is expected to call
 
@AndyProwl What? How can you test client code?
 
@jalf It doesn't change that behavior
 
There is no client code.
Client code is the one I'm going to write. You don't have it.
 
2:36 PM
@AndyProwl sorry, you're right, by changing arbitrary other parts of my system
 
@R.MartinhoFernandes I have an object X with a function foo that I want to test. I want to verify that function foo calls function bar on object Y. Then function Y has to be virtual if I want to mock it, but doesn't have any other reason for being virtual than that.
 
So did you hear that Magic the Gathering Online Exchange lost 750k butts and closed down
 
@AndyProwl No, no, no.
 
@jalf What?
 
If function bar on object Y is not virtual, you only look at its effects.
 
2:37 PM
the point I'm trying to make: if you can't imagine other ways to test that foo calls bar than making bar a virtual member of some class then you're using the wrong programming language.
 
That's what the code that uses X can see.
 
@CatPlusPlus I imagine we all did. AFAIK, no Loungers have committed suicide yet.
 
If there's nothing the code that uses X can see about that, you're not testing something useful.
 
@R.MartinhoFernandes Which makes unit testing of X a contradiction in terms, because now you're not testing X alone, but X together with Y
 
@AndyProwl That's called high coupling.
 
2:38 PM
They're looking for "serious exchanges" now :allears:
 
I think Andy has a point, as long as that's not the only testing you do. Yes, you need to test your code as it'll be delivered. But in some cases, doing some rewriting let's you carry out tests you couldn't without it, and those tests can let you get at some behaviors (and in some cases state as well) that quite rightly aren't so easily accessible as it would normally be delivered.
 
@AndyProwl making the callee virtual when it wasn't originally and didn't need to be. Changing other parts of your system.
 
5 mins ago, by R. Martinho Fernandes
My code is not made for testing. It is inherently testable. That's a property of good code.
 
@R.MartinhoFernandes What?
 
@AndyProwl The fact that X and Y are highly coupled.
@CatPlusPlus lol
@CatPlusPlus hehe
 
2:39 PM
@jalf There's no possible negative consequence of doing that if that function is not hidden by a function with the same name in a subclass, and that's not the case
 
It's amazing
You think you reached the bottom of this well of incompetence but it just keeps going
 
@R.MartinhoFernandes It don't get it. Every time a function foo in X calls a function bar in Y, that means X and Y are highly coupled? Also, transitively? No, I want to have a way to hijack dependencies so I can test in isolation. In C++, unfortunately, I am forced to make bar virtual if I want to hijack the communication between X and Y - and I may have good reasons for doing that. However, there's no other reason than mocking for having bar virtual.
 
@AndyProwl "the code being tested is different from the code used in production" is a negative consequence, and I basically can't imagine a worse negative consequence of writing tests
it means you waste a lot of effort writing tests without getting any value out of them
 
Also extremely funny that it wasn't more than two months ago when someone was touting butts as stable investment
 
you're better off not testing, then at least you save the time
 
2:42 PM
@jalf It is not different in any significant way
 
@AndyProwl It. is. different. That is significant
 
@jalf No it is not
 
Suit yourself
 
If this difference has no way whatsoever to alter the behavior of the program, than the difference is irrelevant
 
@AndyProwl .. and how do you test that it doesn't alter the behavior of the program?
 
2:43 PM
The problem with "having a way of hijacking dependencies" is that it should be default otherwise yes, you have highly coupled code
3
 
Oh right, you don't
 
user1804599
@thecoshman only in the same module.
 
That's basically why I mostly do integrated system testing. Fuck unit tests.
 
so your tests are valid, as long as an untested assumption holds
 
If you can't substitute Y without internal code hacks, your design sucks
 
2:44 PM
Great, that's super useful
 
@AndyProwl if X is an unit and you cannot isolate it as a unit that doesn't have Y without hacks like making things conditionally virtual, yes, X and Y are highly coupled.
That's pretty much the definition.
 
Dependency inversion man
 
Inject the shit
 
@jalf I really don't like this kind of sarcasm. There's no need of making fun of those who disagree with you.
 
It's fun to make fun
 
2:46 PM
@R.MartinhoFernandes So say you have a class Y, which you want by design to be final. Yet, you want to test the fact that other objects in your code call Y as expected - without actually triggering Y's behavior. How do you do that?
 
Also buttcoin.txt:
> Only if there are bitcoins to be bought and sold. Might just be imaginary numbers with no substance.
 
Low-coupled units are well... easy to have as isolated units.
 
@AndyProwl I'm not making fun of those who disagree with me. I am making fun of people who claim that "despite not testing the thing that all my tests depend on, I am going to assume that my tests are valid"
because it makes no sense.
 
@AndyProwl Do you?
 
@R.MartinhoFernandes Yes?
 
2:47 PM
Note that I didn't make fun of anything you said up until that point, because that was just disagreement, and I'm fine with that
 
@AndyProwl Your problem is depending on a concretion instead of abstraction
 
I usually want those other objects to perform their tasks as specified.
I don't care if they call Y.
 
As I've pointed out several times already, what makes a good test framework is highly subjective
@CatPlusPlus Better than depending on secretion
 
@jalf It is disagreement all the way down
 
@AndyProwl I disagree with internally contradictory arguments, yes.
 
2:48 PM
Where do you draw the line? Do you mock std::copy? Do you mock your fft implementation?
Because you are testing those as well if your code calls them!
 
@jalf And the fact that it's contradictory is exactly what we're disagreeing on
 
I'd also make fun of people who claim that 2+2=5
 
I mock C++... oh wait, I already did that joke
 
@AndyProwl So you see no problem in writing a test for different code than you use in production, without making sure that it behaves the same?
 
@R.MartinhoFernandes Where you draw the line is a matter of philosophy. It's state-driven vs behavior driven. We're not going to solve this today
 
2:49 PM
It's already a solved problem
 
@jalf I am making sure that it behaves the same
 
@CatPlusPlus You no longer write any C++ whatsoever?
 
@AndyProwl how?
 
Also note the "if" in "if your code calls them". Your tests are assuming your code calls them. Now you are coupling your tests to your implementation.
 
I have all my automated system test drivers built-in to my production code. It gets delivered. I can turn it on even on customer sites. What I test is what I deliver.
 
2:49 PM
@ScarletAmaranth Yup
 
you're changing the code and hoping and assuming that it didn't change any behavior
 
@jalf For instance, by not deriving from those classes?
 
@CatPlusPlus are you a happier person now ^^?
 
@ScarletAmaranth Slightly
 
@AndyProwl Aaaah, so you're assuming that the code under test is correct
 
2:50 PM
It's offset by a lot of things, but at least I don't have to touch this poop
 
As long as the code under test contains no bugs, my tests will work!
 
@R.MartinhoFernandes Yes that's the disadvantage of behavior verification. The advantage is that you have higher test coverage
 
Yes, I fully stand behind my earlier sarcasm in this case.
 
@jalf I assume that one condition about my production code which is extremely easy to verify and enforce holds, yes.
 
@AndyProwl you haven't told me how you verify it!
 
2:51 PM
It's not very easy to enforce it when your language doesn't have a sealed specifier :ssh:
 
You've only said "well, I just don't write code which contains that particular bug"
 
@jalf I don't need to verify that I don't accidentally write a subclass of a class I'm not supposed to derive from!
 
@AndyProwl what if someone else modifies the code later?
 
@jalf Yes, so let's stop this conversation, it's becoming unpleasant to me.
 
@AndyProwl sorry I haven't been following the conversation too well but final should let you do just that? or is there some profound reason you're avoiding it?
 
2:53 PM
Oh, C++ got final, I missed that
 
@ScarletAmaranth The only problem is that you have to mock that class so only for mocking purposes you need to derive from it. Production code is not ever expected to derive from that class.
 
Does anyone actually support it
 
@AndyProwl That's a false dichotomy.
 
@CatPlusPlus yeah that's why I was surprised when you said there's no sealed
@AndyProwl aaah, can you roll a conditional final with a macro?
 
@AndyProwl but what if production code does derive from it? You might not do it when initially writing the code, but what if someone else does it 18 months from now?
 
2:54 PM
My testing is interface driven.
 
My class is final except when it's not!
@jalf Conditional final, duh
 
Set up preconditions, make a call, check postconditions.
 
@CatPlusPlus I think it works in MSVC. puppy used it.
 
I don't care about what behaviour it takes to do that. (Come on, encapsulation!)
 
Invasive testing is crappy
 
user1804599
2:55 PM
xorg-server y u depend on LLVM.
 
Can your tests be written by someone that never saw the implementation?
 
@rightfold yeah
 
@jalf It is not even enough that production code derives from it, it also needs to hide that function. And in 95% cases I do prohibit doing that as part of a coding standard. Perhaps there's 5% where I'd allow it, and that means I'm not going to use this technique.
Most likely even less than 5%
 
Coding standard is not a real way to enforce anything
 
@R.MartinhoFernandes Yes, that's the pro of state verification.
 
2:56 PM
Encapsulation?
 
1 min ago, by Cat Plus Plus
Invasive testing is crappy
 
(It doesn't have to be state verification)
 
@AndyProwl it's irrelevant how often it happens. The point remains that your test makes an assumption about the code that your tests cannot verify. And that assumption might hold initially when you write the code, but you have no way to ensure that it will hold in the future. Code gets maintained, modified, rewritten, refactored, has features added and bugs fixed.
 
@ScarletAmaranth It is not enough to allow mocking
 
2:57 PM
Yeah, that, and also make it conditionally struct, so that tests can call private things easier
 
Those functions need to be virtual
 
heck, how about this: your compiler just so happens to generate correct code if the function is virtual, but incorrect code otherwise
Your tests pass perfectly, but in production....? Not so much
 
user1804599
Use templates instead of inheritance!
 
And shit like that happens way too often
 
2:58 PM
@R.MartinhoFernandes If you don't hijack the invocation of a function but verify post-conditions expressed on the state of the SUT or the DOC after exercising, then it is state verification
 
And yes, compiler bugs do exist
 
@AndyProwl well it would be possible to make it conditionally virtual but that sounds broken and wrong ^^
 
Also, martinfowler.com/articles/mocksArentStubs.html <- this article by Martin Fowler is interesting
 
@AndyProwl I often don't check any state in my tests.
 
Who cares if it's state or behavior verification, if it doesn't even verify the correct program?
 
2:59 PM
@jalf I already answered that objection
 
@AndyProwl no, you refused to acknowledge it
 
@R.MartinhoFernandes So what do you check?
 
@AndyProwl Postconditions. They're not always state.
 

« first day (1229 days earlier)      last day (3736 days later) »