Recently, Michal Mocny published an article applauding the C++ Committee’s recent decision to keep implicitly-generated move operations in the C++0x standard. I disagree, and I think it raises some really interesting questions about the standardization process and what’s next for C++, which I’ll discuss at the end.
For those of you who missed my previous article on the topic, the quick skinny is that Scott Meyers discovered that implicitly generating move-constructors and -assignment operators could break class invariants in correct code. The decision in Batavia was to tighten the conditions under which compilers could generate these operations, but not to remove the implicit move generation feature altogether.
So what’s wrong with that?
Still Broken
First, as was acknowledged by everybody in the room at the time of the vote,
Implicitly generated move operations will still break carefully-engineered, working C++03 code when recompiled under C++0x , no matter how we tighten the rules. The only way to avoid all such breakage is to tighten the rules “all the way,” i.e. drop the implicit move feature.
When I say “break”, I mean it in two ways:
- It’s going to cause actual crashes, asserts, and other visible misbehavior.
- It’s going to silently alter the invariants of some correct classes from what they have been designed and documented to be
I don’t know which one is worse, but I’m particularly worried about the effect of #2 on the ability of large software projects to transition to C++0x.
Cure: Worse Than Disease
Second, we’ve saved the patient’s life, but have left him in a condition so impaired that he really can’t be alone without supervision. By further limiting the cases in which we’ll implicitly generate move operations, the feature will only usefully “kick in” about as often as it will break code. As I said in the room on the day of the vote, it would have been much better to boldly break existing code and have implicit move generation that would actually be commonly useful, like implicit copy generation is now. Then everyone would know that it’s something they have to pay attention to, but it would be worth the trouble. The decision made in Batavia leaves us with an implicit move that’s hardly ever going to be a factor (and so will be easily overlooked), but when it is a factor, is just about as likely to hurt as it is to help. This is the kind of rare “gotcha” that keeps C++ an “experts-only language” in some peoples’ eyes.
How Did This Happen?
It’s very disappointing, but I see this a lot. While our emphasis on consensus is one of our great strengths, the C++ committee is extremely conflict-averse. The natural inclination of a group to choose what looks like a moderate compromise rather than anything that appears to be an extreme position can lead to bad decisions. When disagreements arise, there is a strong tendency to “cut a middle course,” even if it would be better to make a more definitive choice. In this case, the committee was offered a choice between
- leaving the implicit move feature alone
- limiting its effects, or
- removing it
and, naturally, we chose #2.
Another problem is that at the beginning of this 13-year design process, the committee did not reach a consensus about what kinds of code breakage would be acceptable. Some were absolutely convinced that we had to be able to maintain binary backward-compatibility, some thought source backward-compatibility was paramount, and some, like me, thought some code breakage should be on the table. However, no decision was reached at the time.
Since we’re apparently willing to break code, it seems clear now that the third group won the argument. Unfortunately, we missed the opportunity to decide what kinds of code we could break. For example, is it acceptable to break code if we aren’t sure we can generate compiler warnings to help people through the transition? Is it acceptable to break code that follows all the common coding guidelines, or do we have to be able to say with conviction about the cases that break, “of course, if you do that, you deserve what you get?” I’m not even sure these are all the right questions, but we haven’t really begun to ponder what the right questions are. So here we are near the end of this whole effort, having painted ourselves into a corner, with no agreed-upon rules on how to navigate our way out of it.
I hope, at least, that we figure it out before the next round of language changes.
Why not just make it so implicit move is auto generated like originally planned, and then have it mandated that the compiler should have the option to turn this off for those that are worried about breaking old code.
The original plan was never to have move operations written implicitly by the compiler, and the standard doesn’t even mandate the existence of compiler options. The idea is to define a single standard language (well, maybe two if you include non-hosted implementations), so that vendors have something concrete to which they can conform.
One other question regarding move constructors:
For a class which has more than one object which could benefit from a move constructor, do we have to write all N possible constructors? ie:
This is going to get ugly quickly.
Yes I know that we could use a template constructor, but that has issues as well. ie:
Any thoughts on this Dave?
You’re storing a copy anyway, so I think a correct way in C++0X is to pass by value.
By the way, your point made me wonder : does the new aggregate uniform initialisation use move when appropriate ? Example:
struct str_vec { std::string m_str; std::vector m_vec; };
str_vec sv { get_some_str_by_value(), get_some_vec_by_value()};
From n3225 [dcl.init.aggr] and [over.match.ctor], I understand that move is used.
Actually, that’s the correct way in C++03 too, as long as there’s a default constructor and a fast swap you can use to place the value where you want it. Otherwise, you lose copy elision.
Moreover, it appears that I replied too quickly on Gary’s point. Passing by value does not help at all here: There is a problem with such constructors because you have to write an initialization in them (unlike functions).
Gary, I think you’re stuck with a template constructor using std::forward on arguments used to initialize members.
Yep, it does appear that a template constructor is the “best” work around.
I did find this article on it the whole problem but it didn’t appear to have any better options.
http://www.artima.com/cppsource/rvalue.html
Essentially what I want is a copy, unless the argument is a R-Value in which case I want it to be a move. Which is of course why we want automatically move constructors. The template option isn’t bad but I vaguely remember that it has lookup problems. But that may only be if there are other non template constructors which get favored first.
You want to deal with different argument types in a single C++ function, it’s not a surprise that you have template in the end. I’d even say this is reassuring.
Is this the proper way to write that template constructor?
Is this the proper way to write that template constructor? Trying again. Some less than & greater than signs got squashed.
That’s what I would have written, if I really had to write such member-wise constructor, which makes little sense to me in your example since members are public
.
Now I have to admit I never carefully analyzed if/why this does work in all cases. I only know this has to do with template argument deduction and reference collapsing rules.
I really wish there was no need to call std::forward here, but well, the pattern is not that hard to remember.
I’m afraid that your template solution is not ideal. Consider this use of your solution above:
For me this prints out:
But ideally it would print out
To get the desired effect your templated constructor needs to be properly constrained. This could look like:
At about this time, lots of people gasp or giggle. Just how complicated does this have to get?! <shrug> For me it is just another tool in the toolbox. If I need it, I need it, and will use it. If you prefer something simpler, then passing your arguments by value looks perfectly reasonable to me. And I’m not seeing problems with this approach which were noted earlier in this thread:
This latter approach is slightly more expensive: when passing in lvalues you get an extra move construction. But the cost of that extra move construction is probably going to be lost compared to the cost of the copy construction that both techniques require anyway for lvalue arguments.
I’d be tempted to go with the by-value approach, and only if performance testing turned up a problem, then go with the constrained template approach, for this particular problem. Obviously one can construct other scenarios that would favor other solutions. Any time you write code, keep all of your tools handy!
I am more scared about all the unexpected ways in which is_convertible can make the program invalid.
I had never seen the argument passing presented quite like this example, it looks very nice indeed when you are certain that there is an efficient move constructor.
For example?
The usual SFINAE caveats. The standard got a whole new scary paragraph after the definition of is_convertible making it clear that some of these restrictions apply to is_convertible.
If To has a private constructor from From (or more likely From has a private operator To, in this example), usual implementations of is_convertible will fail. However, the note seems to imply this is handled (by compiler magic?). If trying to find a conversion causes the instantiation of a class that wasn’t supposed to be, then it may fail. But I’ll agree that it seems unlikely in this specific example.
Ah. Thanks Howard. Those constraints make sense.
I tested both the template-based constructor and the pass-by-value constructor on VC10 (without the constraints because VC10 doesn’t support default template arguments in functions) and gcc 4.5. I found that the pass-by-value version yields the extra move construction even when the initial arguments to the constructor are rvalue references:
The extra move is no longer overshadowed by a much more expensive copy operation, so it actually has a chance of causing a noticeable effect, right? But the template version has pretty big drawbacks too by the way of code bloat. If n is the number of parameters to a constructor, a potential n! different constructors are generated by the template.
Is there any reason why compilers are not allowed to copy elide all rvalue references whenever they want? Or is it just that these particular compilers don’t happen to implement this optimization?
It seems to me that we haven’t truly established perfect forwarding unless the thing being forwarded (an rvalue reference) is eligible for the same optimization as original thing (a temporary).
I mean 2^n.
They are likely inlined here.
The rules for copy elision are very strict. See for instance CWG issue 1049 for one try at relaxing them a little bit. And rules for implicitly using lvalues as temporaries are based on the same.
And after allowing the transformations, compilers would need to perform optimizations at a higher level that they currently do to manage them.
Thanks for replying, Marc. =)
Good point. I just took a look over CWG issue 1049 and I’m starting to see the problems. For example:
class MyCompoundClass { public: std::string m_str; std::vector<int> m_vec; MyCompoundClass(std::string s, std::vector<int> v) : m_str(std::move(s)), m_vec(std::move(v)) { s = "blah"; // What would this do? } };Assuming the the move construction of m_str from s had been elided, I can’t think of how the assignment would work. We pretend s is a temporary and construct it directly in the space for m_str, then we try to access the “temporary” using its name. It makes perfect sense that elision of the move construction be disallowed in this case.
Which leads to my next question: If the only thing we do with an object after its construction is pass it to another function by value, why can’t we pretend it’s a temporary and construct the object directly in the space for the called function’s parameter?
Is it really that much higher of a level? I’m no compiler writer, but as far as I can see, a given function only needs to worry about eliding copy/move construction in the arguments it passes to the functions it calls.
For example, why should this code elide
void foo1 (void) { bar (std::string ("blah")); }…when this code doesn’t?
void foo2 (void) { std::string str = "blah"; bar (str); }Is there any reason why these two snippets of code should compile any differently?
Wouldn’t it be nice if the below code (without the std::move’s) were actually the way to write the most efficient MyCompoundClass constructor?
class MyCompoundClass { public: std::string m_str; std::vector<int> m_vec; MyCompoundClass(std::string s, std::vector<int> v) : m_str(s), m_vec(v) {} };So, why is this disallowed? Would it be hard for a compiler to check that an object is only used once after it’s constructed and then transform it into a temporary? Are there side effects that we wouldn’t want?
I’m going to try to answer my own question. You can’t construct an object without having a location in which to construct it. A function 1 knows where in memory to put the arguments for a function 2 that it calls. However, function 1 doesn’t (or can’t) know where to put the arguments for a function 3 called by function 2. Only function 2 knows that. This means function 1 can’t construct a temporary directly inside the parameter of function 3.
Is that right?
Thanks Howard, With these sort of things, I’ll probably always write the template constructor with the is_convertable constraints, as long as the combining class has an object that could benefit from a move construction. I did try that, and it failed predictably, and then I though, hey, it should fail when it tries to construct those members without the test, if it’s not convertable and it does. The error message wasn’t really any clearer with or without the is_convertable stuff, so I was leaning toward omitting it. But now that I see where it could be used, I’ll just add those few lines in. Not a big issue.
I hadn’t thought about any code needed to call “is_constructable” as I don’t use that test often.
The last question is whether it’s necessary to have as the arguments “&&”? Without it does the compiler assume it must make a copy?
ie: template < class T > MyCompoundClass( T t);
A foo(); // fn returning a copy of an “A”
MyCompoundClass(foo() );
Shouldn’t the “type of T” in the template be a R-value? ie. “A &&”
And if I were to instead write it as:
template < class T > MyCompoundClass( T && t);
A a; MyCompountClass(a);
wouldn’t that fail because “a” is a L-Value ?
I’m just still a little confused as to what types a constructor which specifies a “&&” or R-Value can take, and what sort of automatic conversions will be applied to try and make it fit.
Hi Gary,
Case 1
In the above code, T deduces as A, not A&&, just as it does in C++03. Indeed, there is no situation I can think of in C++0x where a template parameter will be deduced as an rvalue reference.
Case 2
This case is the one that throws everyone at first (even me at first!
). But it is worth it because it turns out to be so useful. In this example, the code compiles and T deduces as A&. When you include reference collapsing (an rvalue reference of an lvalue reference is an lvalue reference), then this code is calling the following instantiation:
If you call the Case 2 code with foo() instead, then T deduces as A and you call:
Inside of MyCompoundClass(T&& t) you can tell whether you’ve been called with an lvalue or rvalue by inspecting the type of T: If T is an lvalue reference type, you’ve been called with an lvalue, else you’ve been called with an rvalue.
For the move-on-return case, can that be an optimization only performed when there are no other references left on the moved object? ie in the examples, the checkers have references (or pointers) to the moved object, and check in their destructors. But compilers are good at tracking references, aren’t they? Would it be too limiting to only use move in those safe cases?
Or is that no better than what RVO does?
Actually, can someone point to the paragraph in the latest draft explaining this move-on-return thing? I failed to find it…
Ah, found it. It is right after the paragraph on copy elision and starts with “When the criteria for elision of a copy operation are met”. So return choose?a:b; doesn’t fit the bill, but if(choose)return a;return b; does.
Call me naive, but I, for one, welcome our new implicit move constructor and implicit move assignment operator overlords (henceforth known as IMC’s and IMAO’s).
Here’s my reasoning:
If we didn’t have IMC’s and IMAO’s for aggregates (and other simple classes), we would lose a lot of implicit optimization that people are unlikely to add back in themselves.
This is when moving from an object is typically allowed:
A. IMC’s and IMAO’s do not break any invariants that are required to hold true in order to destroy an object. If they did, that object would require a destructor other than the compiler-generated desctructor, for which the only invariant is that the object has been properly constructed. In that case, the IMC’s and IMAO’s would not be generated. This means you don’t get any unexpected behavior in cases 1 and 2.
B. IMC’s and IMAO’s do not break any invariants that are required to hold true in order to assign to an object. Again, if they did, then you would need a user-defined constructor. This means you don’t get any unexpected behavior in case 3.
C. So we’re down to case 4, which is where all of Dave’s examples come from. Because they can result in sub-objects with indeterminate values, IMC’s and IMAO’s CAN break invariants that ensure that objects with indeterminate values are still usable for purposes other than being destroyed or receiving assignments.
But in reality, are we going to break anything useful other than testing and logging? What else would you do with an indeterminate value besides testing its invariants and printing it out?
Even if people really do test invariants of objects inside the destructors of OTHER objects, won’t this code break noisily every time? If I were an object, I’d test my invariants in my own destructor anyway, which again would prevent the generation of IMC’s and IMAO’s.
Is there anything I’m missing?
Replace that with “destructor.” =)
Silently breaking valid code is unfortunate, but in this case the code that will be broken is somewhat twisted (although valid) and I suspect is quite rare. I agree with the decision taken. Losing implicitly generated moves would be far worse.
Dave (and others) – from reading the latest comments here, I would like to highlight two different points that have IMHO been too implicit in the article above and in the discussion:
1st – Implicit move, as defined in N3203, can (still) break class invariants set up by any constructors of the class. (Just to be clear: A class invariant is something that always must hold from the end of any ctor to the run of the dtor.)
2nd – Move-at-return (not only the implicitly generated move-ctor) can break existing code that relies on the fact that hitherto the return statement did not modify the object being returned, while the move-at-return will modify the returned object in many cases. (This irregardless of any invariants a class may have.)
I think both points need to be considered separately.
The first point maybe could be fixed by further tightening the rules for implicit generation of move-ctor. (It still would be useful for aggregate structs, I think.)
The second point – well, you said yourself that move-at-return is very important. So is there any real word code (as opposed to a real-ish example) where this would break legitimate usage of accessing an object after return? (Note that I am not convinced that a post condition checker is a legitimate use that deserves to be preserved.)
Dave – I tend to agree that implicit move is still brittle (if not downright broken as you suggest).
However, I think loosing implicit move completely would be a great loss to the language. Consider the simple aggregate struct –
struct aggr1 { std::string name; std::vector<std::string> options; std::map<int, double> values; };– if this construct where to loose it’s implicitly defined move-ops and we were required to explicitly default them, I think it would be for the worse.
Note that I have no idea how this could be properly fixed at the moment, but completely removing implicit move from trivial classes seems to be a bad cure.
agreed, it would be unfortunate
but, sadly, it’s the best cure we have time for. There are any number of obvious ways to make the syntax for
more palatable, but it’s much too late in the standardization process to invent any new language features to handle this problem. The irony is that I’m sure I’ll end up wanting a shorthand for that anyway, to handle the myriad cases where memberwise move just works but is being outlawed by the new tighter restrictions. So in the end there will be a macro:
and then, for the other cases where it breaks something, we’ll probably want to add
at least until we can get around to writing proper move operations.
I think that the aggregate case is very very important, because in my experience the majority of classes written are not much more than that. Like Michal Mocny i’d consider it a wart if I had to do even a simple “DEFAULT_MOVE(Self)” in this case.
IMVHO it would be much better to place this wart onto the legacy code and force legacy code maintainers to add a “DELETE_MOVE(Self)” (or a “DELETE_MOVE_UNTIL_INSPECTED_AND_CONSIDERED_SAFE(Self)”) to all of their classes (they have doubts about). This isn’t too onerous, because it is a no-thought low-risk fix which could even be applied automatically.
Hi Dave, You can count me in the group that would prefer all the implicit operations (copy, assignment, conversion et.al.) to go away. But since that’s not an option, I’d prefer not to have to add to the list of declared but not defined operators the move operator. In my experience, code is written/changed about one time for every ten times it’s read. So writing a bit more verbosity in creating a new class is a trivial amount of work. What it looks like now is that we are going to have to go through all our existing code base and decide whether we want implicit move’s or not, and remove them or code them to avoid breaking existing code.
Oh well, but it looks like my day job will be secure for a few more years.
That is a very good example. Thanks. I think you are right. This is a serious flaw. The new rules are not enough. I wonder if the comitte members have seen an example like this, and still think its okay to break it. Is there any chance of a new vote?
In my humble opinion all these problems are due to the fact that destructors are called at all for moved from object. A destructor should tear down a valid invariant and release owned resources, thats its job. A moved from object no longer owns any resources, and has no invariants. So calling a destructor on it (still) seems fundamentally wrong to me.
I have been told that what I’m talking about is “destructive move semantics” and that while useful is not what is wanted. I would be curious to hear your take on “destructive move semantics” are.
(For a longer version of my “question” see http://tinyurl.com/5vvghg4)
Hi, petke:
I never showed this particular example to the committee. I usually trust those people can generalize adequately to come up with more cases on their own. Is there something special about this one that you think makes it more convincing than the others? On the chance of a new vote, that would probably only happen at this point if a National Body (e.g. ANSI) threatened not to ratify the standard.
Re: destructive move semantics, someone else just posted—and then answered—the same question. My take on it is that nobody knows how to make destructive move semantics work. Just to be clear, “owns no resources” ≠ “has no invariants,” and the status quo is that a moved-from object does indeed have invariants.
Thanks.
To me as a non-expert that example really hit home. I had somehow gotten the impression that with the new rules only really arcane code or code that deserved to be break would break. (I think I got that impression after reading one of Stroustrup’s papers on implicit move. That was probably me reading between the lines too much though).
But your latest example though, looks to me as an example of a good programming idiom. A non-intrusive invariant checker (say if you are not allowed to change the definition of Y, to add the invariant checking directly to it. Or if you are creating some kind of generic version). I could imagine it could equally well be a non intrusive Pre and post condition checker instead (pre in constructor, post in destructor). Or even some type of non-intrusive RAII wrapper. There has to be lots of code like this out there.
The traffic rules committee was at an impasse: half wanted the rule to be “drive on the right hand side of the road”, the other half wanted it to be the left hand side.
So of course they split the difference, and required driving in the middle of the road.
Dave, could you provide some examples that would break with the new tightened rules?
The first one in your previous post would no longer break with the new rules, but the second example (under Tweak #1: Destructors Suppress Implicit Move) would break. Right? That example relies on code that accesses an element in a collection after it has been removed. That was always in undefined behaviour land, right?
Thanks
Well, let’s see. The new rules say:
So I think, yes, that one would still break.
Not at all. The element hasn’t been removed; it has only been “
removed!” That is, theremovealgorithm has simply used the element as the source of an assignment, which in C++0x becomes a move assignment. The standard library is not currently allowed to make that element invalid in any way. However, just so you’re not distracted by the use ofremove, consider a simple case like this one:Y f(bool choose) { Y a, b; InvariantChecker xa(a), xb(b); // The use of ?: guarantees that there will be no copy elision. // Instead, either a or b will be *moved* into the return value. return choose ? a : b; // xb and xa get destroyed here // b and a get destroyed thereafter }where
InvariantCheckeris a class whose destructor checks the invariant of the object with which it was constructed. Thereturnstatement breaks the invariant of eitheraorb, and theInvariantCheckerdetects that breakage as it is being destroyed. Of course, you’re actually lucky if the problem shows up in an explicit invariant check; more likelyInvariantCheckeris replaced by something that tries to do real work withaorb.Dave,
Your InvariantChecker example is quite eye opening, but isn’t the “error” not due to the implicit generation of move (Y(Y&&) could have been explicitly provided in that example) but because of the return-by-value-can-return-by-move rule? I had not considered it, but perhaps return-by-value shouldn’t implicitly move when copy is available? Would this ensure that current code wouldn’t change?
Your std::remove example remains valid.
Only if you think it’s OK to explicitly provide a broken move constructor. I don’t.
If you code it by hand, you do so in such a way that it doesn’t break the class invariants, of course!
Sure, you could turn off one of the most valuable optimizations that we get with move semantics in order to rescue implicit move generation, which is already of dubious value. But then, it still wouldn’t be a complete rescue, because there would still be the
std::removeexample (and many others I can come up with at a moments’ notice). Your suggestion is yet another example of chipping away at the problem instead of fixing it. The result, again, is a slightly smaller problem at the cost of a weaker move semantics feature overall, and a nasty gotcha that will be all the more surprising when it bites.Thats not what I’m arguing here. The decisions for return-via-move and generate-implicit-move are orthogonal, and the example you provide doesn’t only apply to broken move constructors.
Consider the case where Y is std::string (which is well written, is widely used, and will soon have a move constructor come implicit move generation or not) and an InvariantChecker which asserts a == “some value” and b == “some other” in its destructor.
return by implicit move would cause assertion to always fail, even with an explicitly provided well written well behaving move constructor such as std::string’s.
The issue is that returning via move has different semantics than returning via copy, and could change the meaning of some code if it happens implicitly.
Yes, std::remove example still holds, but removing “one of the most valuable optimizations” isn’t about rescuing implicit move, its about identifying the actual cause of the problems in your example.
That’s a very good point. Implicitly returning any object with a move constructor (where the move constructor is implicitly generated or explitly defined) would cause the same problem in that example. Seems the example shows an even more fundamental problem. I’m not sure where this leaves us. I guess going forward one has to be extra careful with wrapper objects that only contain a reference to the wrapped object. The wrapped object may have moved elsewhere by the time the wrapper is destructed, only leaving some default state replacement in its place. I guess one can no longer assume one know the order that stack objects are destructed, as some might have moved beyond the closing “}” only leaving that very different default state object behind. Things sure seem a bit more complicated now with move semantics.
Yes, that assertion would fail. The string’s invariant would still be preserved, but expectations of the surrounding code would have been violated, which is arguably almost as bad.
Well, I think we may have an issue here. It seems like the fundamental problem is that the move should happen at the point of the returned object’s destruction, rather than at the
returnstatement.I was thinking about that, but was concerned that if the returned object’s move constructor somehow references or just relies on other stack objects, the delayed move operation may change the meaning of some programs.
I am not sure if this is a serious practical problem, but consider:
Y f(bool choose) {}
Imagine that example without move constructors, lock would be destroyed before a or b. Now imagine it with move constructor. If we always do move and destructive as as one unit. One after the other. Then we get the same semantics as if we where doing copy. lock gets destructed, and or b gets moved, and an b gets destructed.
If we change your example so the lock object is constructed before a and b. Then if we are doing a copy, a and b are destroyed before lock. If we are doing a move, then a or b is moved, a and b is destroyed, and lock is destroyed. We get the same semantics as with copy.
Right?
To anyone reading, disregard the above comment of mine as I was mistaken. Michal is very much correct in his observation. Currently all object that are in local scope gets destructed after move/copy has completed. Any delay of move from where copy happens now would mean a local mutex object might already have been destructed by the time of the move. So we cant “move” move. I think the only thing that could be done is to disable implicit return. Failing that we will have to be more wary about using aliases to outside objects in our destructors. As implicit return might have changed the outside object from what we expected it to be.
Interesting point.
Dave,
I had another thought this morning which I would appreciate your help in pondering.
Looking at your std::remove example, where T is a std::string, std::remove as currently defined would remain “broken” even without implicit move. Same applies to all standard library containers which will have move constructors come c++0x.
Implicit move or not, these very popular classes are going to “automatically” receive move constructors come c++0x, and we have to deal with the issues.
Now, given that move constructors are going to exist, the real fault I’m seeing is the use of a = std::move(b) in place of a = b where b is an l-value and the programmer didn’t ask for this behavior. Because b is an l-value, it can still be referred to after being moved-from, and this clearly may not be acceptable in some cases.
Implicit move increases visibility of the issue, as does implicit move without restrictions, but I don’t think it is the issue.
If std::remove (and other algorithms) were to be written without using move, or if it were to copy the trailing tail elements rather than move them, it would preserve current semantics (I think). Whether this should have been considered depends on your position on potentially breaking compatibility. The same holds for return-by-value-using-move and likely other cases we haven’t identified.
Is there any still valid example of an r-value being implicitly moved from causing issues?
Hi Michal,
No,
std::removewill not be broken by the addition ofstd::stringmove constructors. The moved-fromstrings will all satisfy their invariants, and currently their values are unspecified. So, while the moved-from strings will likely have different values under C++0x, they will still be valid values.Once the
std::stringvalues change on move, the example does become broken. Is that acceptable just because its broken for reasons that are not a violation of class invariants?Should code using a Collection of non-empty
std::strings have been written to first check that the last element is still non-empty after callingstd::remove? That would be accounting for the case of astd::strings spontaneously clearing itself even thought the documentation forstd::stringandstd::removegive no indication. Because that is what is going to happen now: poof goes your value.Specifically, which example becomes “broken” and in what way is it broken?
Absolutely not; my understanding is that it’s not guaranteed to remain non-empty. As I said, the values of those strings are unspecified.
Alright, this makes sense, thanks for the clarification. My example is not valid.
“It seems like the fundamental problem is that the move should happen at the point of the returned object’s destruction, rather than at the return statement.”
Yes that’s it. The move should happen at the point where the destruction of the object would have happened, had there been no move constructor, and no sooner. This makes a lot of sense, as a move is half a destructor. It destroys some invariants. Destructor should follow the move immediately. They are one unit. With move we just do the destruction in two stages. First we move to empty state, then we destruct properly. That way the semantics are the same for both copy and move.
So currently (as I understand it) the sequence is 1. a and b are constructed 2. xa an xb are constructed 3. a or b is moved (destroying the invariant of the empty state object left behind. Call them ‘a or ‘b) 4. xa and xb are destructed (invariant check fails for ‘a or ‘b) 5. a’ and b’ are destructed
If the sequence was instead 1. a and b are constructed 2. xa an xb are constructed 3. xa and xb are destructed (invariant check succeeds on a and b) 4.1. a or b is moved (destroying the invariant, but this is fine as no local object can depend on our invariants at this point. Call them ‘a or ‘b) 4.2. a’ and b’ are destructed (fine)
Here 4.1 and 4.2 belong together. Nothing should come between.
This looks like it solves the problem in general for both implicit and explicit move.
Anyone see any problem with this?
That’s a very dangerous way to think about this: it will lead you to write incorrect code because it clashes with the reality of how move is (meant to be) used.
The only invariants that a move can destroy are defined outside the class being moved from. But then, any mutating member function can destroy invariants defined outside the class being mutated. You wouldn’t say that
std::vector<int>::push_backis “half a destructor” or “destroys invariants.”No, the principle at work here is that you only move from an object when no other code can use its value. That means
std::sort)std::remove).It solves this case, which, as Michal has pointed out, really has nothing to do with implicit move. Implicit move still breaks code.
I realize now that “the fix” is no good, it took me a while. Thanks everyone that pointed it out. So currently the copy/move return happens after the local objects have been constructed but before they have been destructed. That way say any local mutex wrapper does not scope out until after the move/copy out is completed. So there can be no moving of move until after some local objects have already been destructed to fix that example.
I do realize the example is not about implicitly generated move constructors, but about implicit move return. They both got issues it seems.
“The principle at work here is that you only move from an object when no other code can use its value.”
Fair enough. I guess the issue with implicit move return, is that other local objects infact can use the moved from value in their destructor’s. I know its a bit of “throwing out the baby out with the bathwater”, but maybe it would be best if programmers had to be explicit about move return as well. After all they are the only ones who know if any other local object depends on the moved from value in their destructor’s, or not.
Thanks
I have posted on Usenet these three samples that (at least IMHO) indicate that the issue with move return is quote serious… (but it has nothing to do with implicit move constructor).
void f1(X & res) { X x = blabla... Y y(x); res = x; // y dtor sees the original value } X f2() { X x = blabla... Y y(x); return x; // y dtor sees the original value due to RVO } X f3() { X x = blabla... Y y(x); if (something_silly) return error_x; return x; // y dtor sees the moved-out value due to move }IMHO, f1, f2 and f3 should show the same behavior, at least with respect to Y. Y need not be invariance checker, it may be some encapsulated RAII logic. It is rather silly that one line of difference, that has got nothing to do with “x” and “y”, could change the behavior of the program.
Hope this helps…
“IMHO, f1, f2 and f3 should show the same behavior”
Very good point, again. Usenet is a bit slow on moderation and I’m unpatient.. so I’ll reply here also. Let me see if I understand you correctly. The implicit move return and the (implicit) move retrun have different semantics. Copy and RVO semantics though are the same. RVO has priority over implicit move return as its faster. But because both move and RVO are implicit, subtle changes like adding a different possible return value, can cause semantics differences. So its not easy just by looking at code to know if a value is returned with move semantics or returned with copy/RVO semantics. You would have to look at the defintion, if no move constructor is explicitly defined, then you have to look for the rules when implicit move constructors are generated by the compiler, finally you have to look if there are multiple possible retrun values. Only then will you know, and you might have overlooked something subtle step. A beginner would like to have an clear and unambigous answer to his question, if his big class is being move terurned or copy returned. It would be so much easier to underatand and explain, if the only way for move return to happen is if it was expliucitly stated with “return move(val);”. The way its now the simple looking return statement is one of the most confusing features of the languages.
“The implicit move return and the (implicit) move retrun have different semantics”
Oops. That should have read “The implicit move return and (implicit) RVO have different semantics”
f2 can be broken by RVO on/off toggle: suppose X is std::auto_ptr or any similar struct. I’ve never seen someone upset by that. Return-via-move is quite similar to me.
Regarding implicit move, I disagree that new rules are a compromise: they do make sense. I’m not sure I’d want different rules for “C++0X as a new language”. If they kept the old “generate them all” way of life, these rules would be written in all C++0X guidelines, wouldn’t they ?
I didn’t understand the logic behind title of this article as “W00t W00t Nix Nix!” , would appreciate if you could shed some light.
Thanks Javin
Hi Javin,
It’s just a reference to Michal Mocny’s article.
Is there at least a sensible way for the compiler to issue warnings in suspect cases?
Good question—one that nobody bothered to explore before deciding to retain implicit move. I guess we could warn anytime an move operation was implicitly generated. They’re all suspect, you know!
My day job is working on a C++ analysis tool. One of the most common complaints we see against such “preventative” advice usually comes down to: “But I know this code is perfectly safe, this is a false positive!”
I see three groups: The first group are those who want the implicit move and so such a warning annoys them. The second group understand the issue and so will fix their code. The final group have no idea what the warning means, but their program appears to run fine so they ignore it!
I fear the number of people in the latter group is several times larger than either of the first two!
Catching a dangerous use of an implicit move is possible, but this delays detection of the issue until there’s a problem. By that stage fixing the interface may not be an option and the onus is now on the client of the class to “workaround” the issue.
I’m a bit surprised that this made it in. Compiler vendors could have offered “implicit move generation” as an extension allowing it to be tested in the wild. With such user experience it would have been a formality to add this to the next version of C++.
David,
I would just like to say that I was neither at the Batavia meeting, nor any other. I am not a committee member, just an outsider looking in. So, as merely a user and fan of C++ I was and am still happy with the decisions of the committee, because it suits my biased personal preference.
I have also followed your (and others’) arguments against implicit move, and some remain valid, giving us a lot to think about.
From what I have read (in D&E for example) potentially breaking changes like this one were in the past tested for actual real world impact by actually compiling large code repositories available to the committee using the new feature. Was anything like this tried? Is there any data to show what the real world effects might be?
-Michal Mocny
Hi Michal,
Thanks for writing in! I know you’re not on the committee, and I don’t really see any connection between that and your satisfaction being based on your “biased personal preference.” Every committee member (including me of course!) comes to the meetings with his or her own biases. I’ve learned to appreciate the concerns of the other members and how accounting for a broad spectrum of ideas and application domains keeps C++ powerful and relevant and, frankly, on the right track. But I also realize I’m not always successful in overcoming my own personal biases, nor do I think I always should. Some of my biases (e.g. towards ease-of-use, simplicity, and safety), and I’m sure some of yours, are objectively good.
I’m interested to know a few things:
On testing changes for actual real-world impact: such tests have been so rare since I joined the committee in 1996 that I can’t remember a single one. Of course D&E was written in 1994, and things may have been different in the early years. It was certainly a lot easier to do a meaningful survey before there was quite so much C++ out in the world as there is today. The common wisdom on the committee (which I don’t think I can disagree with) seems to be that it’s impractical to know much of anything about the nature of existing C++ code in general, so the best we can do is to do some very specific tests and make educated guesses about what’s going to work out.
Dave,
Allow me to answer your questions, but first I must admit I am still learning “to appreciate the concerns of the other members and how accounting for a broad spectrum of ideas and application domains keeps C++ powerful and relevant and, frankly, on the right track”, as you say.
The personal preference of mine which this language decision suits is simple: aesthetics.
Would you not agree that implicit move complements current C++ features more than explicit?
Many years from now, what will be considered a wart on the language? Some things (vector<bool>, iterator categories, etc.) are warts because of unforeseen mistakes. Those are unfortunate, but unavoidable. I consider explicit move to be an avoidable wart on the language. A hack to work around a language bug. That is not to say that a wart on the language is the worst available choice, I may well be the most reasonable. Explicit move would not be the end of the world, but for me, these little warts add up.
As far as long term language relevance, the significant but temporary(?) cost of compatibility must be weighed against the minimal but permanent loss of aesthetics. Which way the scale tips will depend on who you ask. You know my opinion.
As far as technical arguments on implicit move, I have nothing substantial to share. I can only quote the arguments made by others, and I suspect that would not be worthwhile. Because I have not ywt studied the minutes from last meeting, all I have to go on is your word:
However, where I do know for a fact that your argument is very valid is here:
Thank you so much for articulating this. This is what I think gives us all a lot to think about. Yes, we are near the end of the current effort, but it is a worthy discussion to have next time.
Your rant on how the “cure is worse than the disease” seems too harsh. Do you think things are really that grim, or are you stretching to support your argument?
I just don’t see how this is true. I feel the limitations that were added are a tremendous benefit. I would have wanted copy to have had the same limitations (given an alternate past I mean).
Perhaps you are simply arguing that if implicit move is to remain broken, it should have the opportunity to break as often as possible, rather than rarely and thus more surprisingly?
If that is what you mean, then while I do agree that surprise is bad, I disagree with denouncing an entirely positive language change just because it limits exposure to another language bug.
To conclude briefly: I believe implicit move would be included in an ideal version of C++. As far as real C++, I can only hope that the committee find the appropriate resolution — one we are proud of a decade from now.
No. What complements current features well is what’s going to work.
I might agree if we actually allow implicit move to kick in quite often. As currently defined, it doesn’t really do much to complement the current feature set; it just pops up in a very obscure set of circumstances.
Why a wart? What language bug? So far, you’ve said that your aesthetics favor it, but that judgement is not just partly subjective; it’s 100% subjective. Nothing wrong with that, but it gives the rest of the world nothing to go on in understanding why we should agree with you.
Many people view the majority of the things C++ does implicitly, without any prompting, as language warts: implicit narrowing conversions, constructors that automatically become implicit conversions, the implicit generation of destructors and assignment operators when the copy constructor has been defined explicitly, the implicit overriding of
virtualfunctions, etc., etc. People see these things as warts because they lead to difficult-to-debug program errors that would be avoidable with a minimum of additional verbosity had the language been defined differently. So, that’s an example of an objective reason: you can disagree with its importance, but I presume you can at least relate to it on a logical level. What’s your objective reason for classifying explicit move as a wart?I just fear that, like last time, nobody will want to confront it.
I don’t know about “grim,” but I believe everything I’m saying and I’m not exaggerating for effect.
It’s simple: just look at all the limitations on when implicit move now applies! Unless classes having no user-declared copy ctor, copy assignment, move assignment or dtor, with implicitly-generated move ctors that are correct and more efficient than their copy ctors, are common, having implicit move is seldom going to be of much benefit.
Wouldn’t stronger limitations just confer more of the same benefit?
Not exactly. I’m arguing that if it’s to remain broken, it should be allowed to take effect in many more cases where it would actually be useful and correct at the risk of also being broken in a few more cases, thus increasing its benefit (and reducing its surprise). Introducing a feature that rarely has a benefit is hard to justify to start with, but if it also occasionally breaks code, IMO, it’s inexcusable. If we’re going to break code, let’s make sure we get maximum payoff for that breakage.
I would beg to differ:
It would not be defined for those classes that to move/copy-details themselves — but those classes are in for a review anyway if one wants move-semantics.
It would however be defined, for all those classes that do not do any resource management themselves, but rely on their members to do it for them. These include simple aggregate structs, but also more complicated classes, that may have a bunch of move-enabled std type members, but do not do any resource management themselves.
This leaves me with the feeling that the people on the committee that are advocating breaking existing code haven’t thought their argument through and the burden should have been on them to provide clear rules as to how far breakage should go. This isn’t intended to put the blame on anyone. I’m just left with the feeling that the need to ship has lead to some rushed decisions without the right amount of consideration. Anyway C++0x as a whole seems like a success and there are enough features to compensate for the lack of implicit move. Move semantics seem like a rather expert only feature anyway so I don’t see the “expert only” argument applying here.
@pmr: While I agree with you 100% about what’s been well thought-through and where the burden should lie, I actually don’t think having less time pressure could have led to a better decision here. As I suggested in the article, I think when faced with a choice between “leave implicit move as it is in the working paper,” “limit its power so it causes less damage,” and “rip it out completely,” people are simply most comfortable with the least extreme-sounding change to the WP that seems to improve the situation, even when the boldest move is actually a more careful and conservative choice.
Maybe you misunderstood—there’s no “lack of implicit move:” the result of the Batavia vote is to preserve (but weaken) implicit move.
The point is that if move semantics kick in silently and implicitly in a way that can break code, it becomes something non-experts have to deal with whether they like it or not.
I don’t think the “expert only” argument can be discounted so lightly.
Keep in mind that we’re talking about implicit generation of the move constructor. The experts will write move constructors explicitly when they want them.
It’s exactly the NON-experts who don’t (explicitly) write move constructors who will be affected by this.
It’s actually worse than that. Everyone will have to look at all of their classes and make a judgment as to whether an implicit move constructor will do the right thing. Only an expert will know for sure. And I’m assuming people will become experts after they make the wrong call a few times and figure out all of the subtle got-ya’s in the classes they use.
This I suspect the bugs will lead to people just hacking it out with a declared move operator but no implementation. That lead to linker problems where some standard algorithm tries to call it because it exists. So folks will then go back and write a move which does a move but in an exception unsafe way. That will eventually fail, resulting in people writing move as a copy. Or moving to JAVA in disgust.
In any case unless some National standards body puts their foot down, we are are in for a heap of trouble.