Clarify the treatment of named parameters whose name starts with `_`

[eernstg]

A declaration of a name that starts with _ is private to the current library, unless the declaration is

• a local variable (where privacy makes no difference),
• a label (where privacy also makes no difference, and which is in its own namespace anyway), or
• a formal parameter (where the meaning of privacy has never been fully resolved)

Concretely, the tools are currently dealing with named formal parameters whose name starts with _ in slightly different ways:

// ----- Library 'n011lib.dart'.
class A {
  A({_name});
  void m({_name}) {}
}

class B {
  B({required _name});
  void m({required _name}) {}
}

// ----- Library 'n011.dart'.
import 'n011lib.dart';

void main() {
  var a = A(_name: 0);
  a.m(_name: 0);
  var b = B(_name: 0);
  b.m(_name: 0);
}

With the analyzer (commit 92638bf5d3a5668920d0a40a4b24a6a97f982b3f), no issues are reported; that is, _name is treated like any other name when it occurs as the name of a named parameter, required or not (that is, it isn't treated as a private name).

But the common front end reports an error for both cases with the declaration of A (effectively saying "a named parameter name cannot be private", so we cannot even ask how it's treated at call sites), and allows class B (.. "except if it's required").

We could say that a formal parameter name starting with _ is simply a name like all others, with no special exceptions. In particular, call sites in different libraries can invoke the enclosing function/method/constructor and pass an actual argument to a named parameter with such a name.

We could also say that a formal parameter name starting with _ is a private name. So if it's the name of a named parameter then it can't be passed at call sites in other libraries.

The former makes it easy to use named initializing formals to initialize private instance variables (and the name _name in the parameter declaration this._name serves as documentation for the fact that we're initializing a private instance variable), but it is unfortunate that this usage will make it harder to rename said instance variable. (It would be nicer if the API would only reveal that we're initializing a private instance variable, and it wouldn't mention the name.)

The latter provides support for a very special kind of privacy: Every caller outside the library must use the default value for a non-required named formal parameter with that kind of name, and the whole function/method/constructor can't be called at all outside the library if the parameter is required. This feature is again potentially useful, but rather accidental in nature.

At the conceptual level it may seem inconsistent to claim that _name is just another name (not private to any library) when it's used as the name of a named initializing formal this._name, because it does refer to the instance variable whose name is definitely private.

So we need to clarify these issues and then ensure that implementations agree. We have the freedom to make different choices, because current code is (in practice) unable to depend on any particular behavior in the case where the tools behave differently.

[srawlins]

I love the idea of making a formal parameter which starts with _ just a name like others, and that they could be used in initializing formals.

There was a lot of momentum behind "make private initializing formals a thing," several months ago but it lost steam. I personally think this is a great feature, on par with super parameters.

[lrhn]

Names starting with _ can also be library names, which have no notion of privacy. (And are not accessible to the program without using dart:mirrors anyway.)

I'm fairly sure that allowing required positional parameters to be named with _-starting identifiers was just a plain specification mistake.

We never allowed it before introducing required named parameters. There was no good reason to allow it for required named parameters, but not for optional named parameters (if anything, the other direction makes more sense).

We just forgot to remove "optional" from

It is a compile-time error if the name of a named optional parameter
begins with an `_' character.

when we introduced required named parameters. Prior to that "named optional parameters" covered all named parameters, and the "optional" was just a reminder.

[eernstg]

Very good, @lrhn, so it could be argued that we "used to prohibit" formal parameter names starting with _.

But what are the disadvantages of simply saying that formal parameter names can start with _, and it's just another name (no privacy implied)? The use case is, of course, C({this._x}).

[Edit: That should not be C(this._x), it should be C({this._x}). Corrected.]

[lrhn]

The use case C(this._x) works today. We never prohibited private names for positional parameters, because the name is only visible inside the method anyway.
At least from a language perspective, it leaks into documentation too, which is why purists like me don't want to use that pattern for public APIs.

If DartDoc considered such a parameter as being named x instead, then I'd be happy to use C(this._x) in public API (it should allow using [x] in the doc comment too.)

The problem comes when you want to do C({required this._x});.
This name does matter to the language semantics.

If the parameter's name is a private name, then you can't call the constructor from outside the library. You can't express the name of the named parameter, and you need to do so in the argument list.
Might as well make it C._({required this._x}); then.
Which suggests that we could allow private-named required named parameters on private methods without issues.

If the parameter was not required, you could call the constructor. So we could also safely allow private optional named parameters on static methods. (I guess you could do a tear-off and using type inference to get a type you can't otherwise satisfy, but the same use-case breaks by adding a non-private optional parameter too.)

If the parameter name is not private, how does it then correspond to the int _x; field it initializes, which is private?
A private _x (with symbol #_x) and a non-private _x (with symbol const Synbol("_x")) are two different and unrelated names (#_x != const Symbol("_x")).

If the parameter name (non-private _x) and the initialized field name (private _x) are not the same name anyway, then we might as well make the parameter name just be x, without the _.

I hope that we will at some point, say if we do primary constructors, make this._x give the parameter the name x. That can be breaking, if someone already has Foo(this._x, this.x), so it should be carried by a larger feature which makes the breakage worth it. For a primary constructor, Foo(int _x) would also be able to take advantage of such a renaming to not mess up the public API.

With that, named parameters don't need to have private names at all.

Now, if you actually want to have secret unspeakable parameters that other people can't pass ... just make a separate privately named function for that, and let the public method forward the public parameters to it.

[eernstg]

If the parameter name is not private

That's the case that I asked about, with the extra assumption that C({this._x}) can both initialize a private instance variable, and this named parameter can be passed (using _x: e) in any library. So it's a public name, and we are able to specify that this._x is an initializing formal that initializes said private instance variable. It's just a tiny novelty: connect those two names for this particular purpose. Probably not too hard to implement either.

With respect to symbols, I agree that #_x would be the symbol for the private name of the instance variable, and const Symbol('_x') would be the symbol for the public name of the formal parameter. No problems, and nothing new btw.

might as well make the parameter name just be x, without the _.

We could do that, but I think it's a bad idea.

• This name translation step is capable of creating name clashes and complexity that we do not need (so do we stop reporting an error for this.x if x is a non-variable? .. does it matter whether or not there is a declaration named _x? .. do we report an error about a declaration named _x if it's not an instance variable, and there is an initializing formal this.x?).
• The name translation removes the information that the initializing formal will initialize a private instance variable.

You could claim that it's already a violation of encapsulation to use this.x, because it reveals that x is an instance variable. This information may be useful (especially if we add final getters, dart-lang/sdk#57934 ;-), but that part is in any case unchanged with respect to {this._x} and {required this._x}. In any case, you can just use the formal parameter declaration SomeType _x rather than this._x, and use an element in the initializer list to initialize _newNameOfX if you really, really have to rename that private variable.

The important point here is that we're removing information from the reader for no good reason!

If we allow C({this._x}) to initialize _x, the signature of the constructor immediately reveals that the resulting object has a private instance variable, and the x part of _x may be used to communicate the purpose of this variable. Both of those pieces of information may be useful (e.g., we know that we can't use that getter, so we need to look deeper into the entire interface of C if we wanted to use that getter, or perhaps to understand why we shouldn't have a getter for that variable).

If we use C({this.x}) to initialize _x then we're implicitly promising that we are initializing a public instance variable (perhaps final, perhaps not), and we may only discover that there is no x getter later on because it's an error to use it.

I can see the aesthetics, but I cannot see that the latter is more useful than the former.

[lrhn]

I do not propose that you can write this.x to initialize int _x;.
I propose that you write this._x to initialize int _x;, and then we give the parameter a different public/external name.
Instead of making that public name a non private _x, which is still a different name than the private _x, we just make it x.

Both names are different names than the private _x, one of them is actually pretty to look at.

It means that you can just do C(this._foo) instead of C(TheType foo) : _foo = foo;, and it means the same thing. (That'd get a big thank-you from me, I write that far too often).

For positional parameter names, it only matters for documentation. You can't refer to external the name from outside the class anyway. (Well, we have to decide the name of the implicit variable in the initializer list, and we can use the private _x there for backwards compatibility).

For named parameter names, it affects how you invoke the constructor. Doing C({this._foo}) instead of C({TheType? foo}) : _foo = foo; is shorter, and if it means the same thing, it is actually useful. In comparison {this._foo} is currently completely inaccessible, and if we just make it non-private _foo, having to do C(_foo: 42) is never going to be popular.

So about:

immediately reveals that the resulting object has a private instance variable,

That's just breaking abstraction. You should never reveal that a class has a private member through its public API, that's the entire point of making the member private. If anyone outside of the current library needs to know that information, your're doing something wrong.

The source name should match what the code does when you look at the source. You write this._x because this._x is the field you initialize.
The external name should be what other people would want to write, people who do not know, and should never know, the private details of the class. That should never be a private _x name.

Even if we allowed foo({int? _x}) as a function signature, that you could call from another library using foo(_x: 42), without any privacy issues because parameter names are not affected by privacy, then it would confuse me when I read it.
Why is it not private? If it's not private, what does it mean that it starts with _? Nothing? Why does it start with _ then?
That's just not going to help anybody.

[eernstg]

you write this._x to initialize int _x;, and then we give the parameter
a different public/external name. ... [let's] make it x.

That's a very interesting way to cut it!

This means that we preserve the convenience of declarations like C({this._x}), and we preserve the local documentation effect: If you look up the constructor, you get a hint that this particular constructor argument is used to construct the new C, but it's not going to be "a visible part" of that new C.

But I can't help commenting on the underlying understanding of encapsulation. Here we go:

That's just breaking abstraction. You should never reveal that a class has a
private member through its public API, that's the entire point of making the
member private. If anyone outside of the current library needs to know that
information, your're doing something wrong.

Of course, any particular choice in software engineering can be wrong. However, faced with "you are doing something wrong", I'd say that your thinking seems to be based on a rather inflexible reliance on traditional wisdom which is just not applicable in Dart.

First, we're not breaking abstractions by using a name like a non-private _x, because the developer who chose to use that name can do whatever they want with it (e.g., C({int _x}): y = _x;). The traditional (Parnas, 1972) perspective is that public APIs should be expressed in a way that allows evolution of the implementation. With OO subtyping we also want public APIs to maintain freedom for subtypes to choose different method implementations (as well as evolving them over time). The use of a non-private name starting with _ doesn't prevent any of these things, it's just another name.

However, I'm arguing that constructor parameters with names like non-private _x can serve as a hint to developers about the role played by actual arguments to that parameter. Consider an example like the following (from here):

    return Align(
      alignment: Alignment.topLeft,
      child: Padding(
        padding: const EdgeInsets.only(top: 10),
        child: SizedBox(
          height: 90,
          child: StoreCarouselList(
            documents: documents,
            mapController: mapController,
          ),
        ),
      ),
    );

In this example, the class Align has this.alignment and this.child (via super.child, like all single child widgets), Padding has this.padding and this.child, SizedBox has this.height, this.child, and StoreCarouselList has this.documents and this.mapController.

It just goes on and on, and the point is that the named constructor arguments (in Flutter, in particular) correspond quite faithfully to properties of the newly created object. So the developer who writes this code will constantly reinforce the awareness of properties of the many different kinds of objects. It's necessary to use the correct name for each named parameter in order to pass the actual arguments to the correct parameters, but it is also useful to maintain the tight coupling between object construction and object properties: It makes sense that we're choosing the right mapController for our StoreCarouselList using the label mapController:, because that's the name of the property for that object (technically: it's the name of the getter which will return it when we need it).

This is just a convention, of course, but I have no doubt that it's possible to get into serious trouble by declaring something that violates this convention:

class Point {
  final int x, y;
  Point({int x, int y}): y = x, x = y; // Hehe.
}

So what I'm suggesting that developers could do is to use parameter names like non-private _x to indicate that

1. you are now providing an actual argument that will play the "x" role in the construction of the new object, and
2. that "x" role is not a public property of the object.

For instance, you might want to create an instance of Account with a "password", and perhaps there is no password getter on the new object, because there shouldn't be any such getter.

My take on this is that we can support developers in making this distinction automatically and conveniently by allowing parameter declarations like C({this._x}). The instance variable _x has been made private for a reason, and it is relevant to developers who are going to use instances of C that x is not part of the public API of the new object.

Again, we haven't actually lost any implementation flexibility, and we could of course declare a getter named x in C, but the point is that the straightforward declaration yields the useful hints.

I think it would be useful to allow the syntax _x: e at call sites, and it would allow us to maintain a simpler and more consistent semantics. But it's not so terrible if call sites have x: e, because a simple mouse hover will show this._x if that's indeed how the parameter is declared.

In summary, I vehemently disagree that

That's just breaking abstraction.

because we still have all the degrees of freedom in the implementation that we need in order to maintain encapsulation. I'm just arguing that it can be helpful for developers to build the correct understanding of the newly created object if we allow the author of the constructor to use these non-private _ parameter names, and it happens to be very convenient and appropriate in the special case where the argument is actually used to initialize a private instance variable.

[munificent]

So what I'm suggesting that developers could do is to use parameter names like non-private _x to indicate that

1. you are now providing an actual argument that will play the "x" role in the construction of the new object, and
2. that "x" role is not a public property of the object.

We could do that, yes. But developers are not asking to be able to express that, and I see little motivation to encourage it. This sounds like a solution in search of a problem. Meanwhile, we do have a real annoyance:

class Fruits {
  String? apple;
  String? _banana;

  Fruits({this.apple, String banana}) : _banana = banana;
}

main() {
  Fruits(apple: 'Fuji', banana: 'Cavendish');
}

There's no compelling reason why it must be so much harder to initialize the private _banana field from a named parameter compared to apple. The only reason that users write code like this—which they do, all the time—is to manually shave off that _. It's pointless busy work. The author, the reader, and the language knows exactly what the obvious name for this parameter is: it's banana. It's exceedingly unlikely to collide with another parameter name.

The author of Fruits does not want to see _banana at every constructor callsite. They aren't trying to express that. They just want to initialize the "banana" field with a "banana" parameter. The fact that the field happens to be private is immaterial to the class's API, and the fact that the parameter name and field name aren't actually 100% identical is entirely because of Dart's unfortunate choice to bake privacy into the field name.

If we're going to allow private names in named initializing formals, we should have their semantics map to what users already do, which is use a public parameter name to initialize that field.

[leafpetersen]

I do not propose that you can write this.x to initialize int _x;. I propose that you write this._x to initialize int _x;, and then we give the parameter a different public/external name. Instead of making that public name a non private _x, which is still a different name than the private _x, we just make it x.

I worry a lot that this will be super confusing. It's fine if you go the doc page, but it's going to be super confusing if you go to definition from a call site which is passing x and don't see x listed as a parameter.

[leafpetersen]

I wonder if we could make this explicit in the syntax for the parameters? A strawman might be to use _this.x to mean "initializing formal, with name x, which initializes the field _x. It's still pretty opaque if you haven't seen it before, but at least you know something is going on, and it has a searchable name.

[eernstg]

I wonder if we could make this explicit in the syntax for the parameters?

I considered this._x as someName (except that I suppose @lrhn might want to use as with parameters for dart-lang/sdk#57845).

[eernstg]

@eernstg then @munificent wrote:

So what I'm suggesting that developers could do is to use parameter names like non-private _x to indicate that

• you are now providing an actual argument that will play the "x" role in the construction of the new object, and
• that "x" role is not a public property of the object.

We could do that, yes. But developers are not asking to be able to express that, and
I see little motivation to encourage it.

The motivation I see is that (1) we avoid introducing confusing and surprising rules about implicit renaming of parameters, and (2) there is a meaningful interpretation of those non-private _ names (and it's not a violation of encapsulation to have them, so why not?).

I know I know, the answer to "why not?" would be "everybody hates _". ;-)

But we could use this._x as x: The renaming is explicit and developer-controllable, and anyone who looks up the declaration of the constructor will know what is going on. All good!

[lrhn]

I would indeed want as for dart-lang/sdk#57845.
I also like it for this, but the two do not combine well (the dart-lang/sdk#57845 as changes the internal variable, this one changes the public parameter, so we can't just make it do both.)

I'm not sure I worry about readers being confused. The rule of "this._x has parameter name x" feels like something you'll learn quickly as a code writer, because you'll be using it all the time.
I'm more worried about people not realizing that it applies to local DartDoc too.

It would be better if we could combine this., _x and x in some way, and as does have a nice ring to it. C(x as this._x) doesn't work as well. Could be readable as C(this._x = x), but that conflicts with default values.
C(x is this._x). Not confusing at all :)