Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "JDT Core/Java9/JLS9"

m (JPMS)
 
(9 intermediate revisions by the same user not shown)
Line 1: Line 1:
This page is a working document of attempts to map current drafts of JLS to the semantics as it should be implemented by the compiler.
+
{{note|This page was a working document of attempts to map drafts of JLS to the semantics as it should be implemented by the compiler.}}
  
 
=Fresh start as of Public Review documentation=
 
=Fresh start as of Public Review documentation=
 
<u>Sources</u>:
 
<u>Sources</u>:
 
* http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html (''version 2017/2/23'')
 
* http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html (''version 2017/2/23'')
* http://cr.openjdk.java.net/~mr/jigsaw/spec/java-se-9-jls-pr-diffs.pdf (''version 2017-02-22'')
+
* http://cr.openjdk.java.net/~mr/jigsaw/spec/java-se-9-jls-pr-diffs.pdf (''versions 2017-02-22'' - ''2017-06-26'')
  
 
==Capture rules and their dependencies as logic formulae==
 
==Capture rules and their dependencies as logic formulae==
Line 12: Line 12:
 
;[6.5.3.1-2] namedPackage (name, pack, m):
 
;[6.5.3.1-2] namedPackage (name, pack, m):
 
::name = prefix . n
 
::name = prefix . n
::namedPackage(prefix, parent, m)
+
::<strike>namedPackage(prefix, parent, m)</strike> // #prob6
::hasNamedMember_P_P(parent, pack, n) // #prob4
+
::hasNamedMember_P_P(parent, pack, n) // <strike>#prob4</strike>
 
::uniquelyVisible_P(pack, m) // <strike>#prob1</strike>
 
::uniquelyVisible_P(pack, m) // <strike>#prob1</strike>
 
:or
 
:or
Line 42: Line 42:
 
::reads(m, m2)
 
::reads(m, m2)
 
::exportTo(pack, m2, m)
 
::exportTo(pack, m2, m)
 +
::unique(m2)
  
 
;[6.3] inScope_P(pack, m):
 
;[6.3] inScope_P(pack, m):
Line 56: Line 57:
 
::}     
 
::}     
  
;read(m1, m2) -> mentioned in 7.4.3 and Configuration.resolve() // #prob3
+
;read(m1, m2) -> 7.3 now refers to javadoc of package <code>java.lang.module</code> // <strike>#prob3</strike>
  
 
===Intuitive foundation===
 
===Intuitive foundation===
Line 92: Line 93:
 
:intention is probably that ownership is ensured via the chain of hasNamedMember_P_T and exportedTo. See #prob5.
 
:intention is probably that ownership is ensured via the chain of hasNamedMember_P_T and exportedTo. See #prob5.
  
; #prob3
+
; <strike>#prob3</strike>
:read is mentioned in 7.4.3, but not explicitly defined, neither in JLS nor API doc (only in SOTMS)
+
:<strike>read is mentioned in 7.4.3, but not explicitly defined, neither in JLS nor API doc (only in SOTMS)</strike>
:API: "A readability graph is constructed"
+
:<strike>API: "A readability graph is constructed"</strike>
 +
:<strike>An interim version of 7.3 has been extended to refer to [http://cr.openjdk.java.net/~mr/jigsaw/spec/api/java/lang/module/Configuration.html#resolve-java.lang.module.ModuleFinder-java.util.List-java.lang.module.ModuleFinder-java.util.Collection- method] <code>java.lang.module.Configuration.resolve()</code> to define "reads".</strike>
 +
:[[File:Ok_green.gif]] As of "2017-06-26" JLS 7.3 refers to the [http://cr.openjdk.java.net/~mr/jigsaw/spec/api/java/lang/module/package-summary.html package documentation] of <code>java.lang.module</code>, which seems sufficient for a definition.
  
 +
<strike>
 
; #prob4
 
; #prob4
 
:Text still says "The package name Q.Id names a package that is the member named Id within the package named by Q". So, if two modules declare package p0.p1, how many packages p0 do we have?
 
:Text still says "The package name Q.Id names a package that is the member named Id within the package named by Q". So, if two modules declare package p0.p1, how many packages p0 do we have?
Line 101: Line 105:
 
:* if ''two'', then "uniquely visible to" must be applied to disambiguate even the toplevel package. "Uniquely visible" (7.4.3), however, requires an ordinary compilation unit in the package. Where a toplevel (or parent) package has no compilation units, it is not "uniquely visible".
 
:* if ''two'', then "uniquely visible to" must be applied to disambiguate even the toplevel package. "Uniquely visible" (7.4.3), however, requires an ordinary compilation unit in the package. Where a toplevel (or parent) package has no compilation units, it is not "uniquely visible".
 
:workaround: ignoring the "the" in "the member named Q.Id" seems to fix this.
 
:workaround: ignoring the "the" in "the member named Q.Id" seems to fix this.
 +
</strike>
 +
: [[File:Ok_green.gif]] [http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012796.html Confirmed]: ''Conceptually,'' one package Q can contain multiple sub packages named "Id", as long as only one of them is uniquely visible at the location of reference. The uniquely visible sub package is "the" member named Id. ''Technically,'' the spec doesn't differentiate different packages Q.Id, by the rule for unique visibility it just ensures that a split package (with contributions from multiple modules) never creates a conflict during name resolution.
  
 
; #prob5
 
; #prob5
:7.7.2 doesn't even attempt to define ''which'' package of a given name is exported (given we know that a name does not uniquely identify a package).
+
:<strike>7.7.2 doesn't even attempt to define ''which'' package of a given name is exported (given we know that a name does not uniquely identify a package).</strike>
:Still with common sense reasoning the intention is clear.
+
:At a closer look, JLS only '''emulates''' per-module packages. Technically, packages exist independent of modules, and exporting a package factually means to export all types of compilation units that meet both following criteria:
 +
:# the CU is associated to the exporting module
 +
:# the CU declares the exported package
 +
 
 +
; #prob6
 +
:As per [http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012488.html this post], we are not allowed to apply recursion in 6.5.3.2 - in particular the requirement of unique visibility does not apply to any prefix Q. As a result the seemingly recursive part of rule 6.5.3.2 does not provide any clarification to anything.
 +
 
 +
==Resolving package-vs-type conflicts==
 +
 
 +
When a qualified name Q could identify both a package and a type, then JLS 6.5.4.2 generally favors types over packages.
 +
 
 +
It is an open issue, however, whether or not visibility / accessibility can influence this selection.
 +
 
 +
===Java 8-===
 +
Consider:
 +
* package-private type p1.p2.t3 (possibly containing a member type t4)
 +
* public type p1.p2.t3.t4
 +
* a qualified type reference p1.p2.t3.t4
 +
 
 +
(''need separate compilation of both types to avoid error regarding clash between package and type'').
 +
 
 +
;javac:
 +
:accept
 +
;ecj:
 +
:The type p1.p2.t3 is not visible
 +
 
 +
6.5.3.2 finds that package p1.p2 has a member type named t3 and concludes that p1.p2.t3 is a TypeName. Usage of that type fails due to lack of access permissions. That's what ecj concludes. I have no idea what lets javac accept it.
 +
 
 +
''Since full compilation lets both compilers complain about the package-vs-type clash, the answer concerning the type reference ''could'' be seen as irrelevant.''
 +
 
 +
===Java 9===
 +
Similar to the above, but apply accessibility at the package level: make all types public and let one module export package p1.p2.t3, while p1.p2 (containing type t3) in another module is not exported.
 +
 
 +
''Now the input is legal and hence the answer regarding the type reference p1.p2.t3.t4 ''is'' relevant.
 +
 
 +
;javac:
 +
:accept
 +
;ecj:
 +
:The type p1.p2.t3 is not visible
 +
 
 +
Open question: Can we let ecj accept this program without creating an obvious conflict with JLS?
  
 
=Previous attempts=
 
=Previous attempts=

Latest revision as of 16:04, 18 December 2018

Note.png
This page was a working document of attempts to map drafts of JLS to the semantics as it should be implemented by the compiler.


Fresh start as of Public Review documentation

Sources:

Capture rules and their dependencies as logic formulae

Driving question: What's the meaning of a {type,package} name?

[6.5.3.1-2] namedPackage (name, pack, m)
name = prefix . n
namedPackage(prefix, parent, m) // #prob6
hasNamedMember_P_P(parent, pack, n) // #prob4
uniquelyVisible_P(pack, m) // #prob1
or
namedToplevelPackage(name, pack)
inScope_P(pack, m)
uniquelyVisible_P(pack, m)
[6.5.5.2] namedType (name, type, m)
name = prefix . n
namedPackage(prefix, pack, m) // includes 'uniquelyVisible_P'
hasNamedMember_P_T(pack, type, n)
accessible(type, m)
unique(type)
[7.3] visible_CU (cu, m)
observable_CU(cu)
associated(cu, m2)
read(m, m2)
[7.4.3] visible_P (pack, m)
declares(cu, pack) // syntax
visible_CU(cu, m)
[7.4.3] uniquelyVisible_P (pack, m)
declares(cu, pack)
associated(cu, m)
xor
reads(m, m2)
exportTo(pack, m2, m)
unique(m2)
[6.3] inScope_P(pack, m)
uniquelyVisible_P(pack, m) // only for toplevel packages (?!) -- is this a problem?
[6.6.1] accessible (type, m) ("type can be accessed by code in module m")
public(type)
{
associated(cu(type), m)
or
hasNamedMember_P_T(pack, type, _)
exportedTo(pack, m1, m) // #prob2
visible_CU(cu(type), m)
}
read(m1, m2) -> 7.3 now refers to javadoc of package java.lang.module // #prob3

Intuitive foundation

Let's rely on intuitive interpretation of the following:

  • Observability: what cus / packs are provided to the compiler (JDT: NameEnvironment):
    • observable_CU ...
  • associated(cu mod): is cu a member of mod?
  • hasNamedMember: 7.1
    • hasNamedMember_P_P (package contains package)
    • hasNamedMember_P_T (package contains type)

Questions

  • Why are dependencies considered at two levels:
    • visible_CU is based on read
    • accessible is based on exportedTo
    + accessible is based on visible_CU (?)

Problems

#prob1
namedType says "accessible", but namedPackage only "visible", which is too weak.
this issues seams to be resolved by the introduction of uniquelyVisible_P.

#prob1b
even though namedPackage is based on visible_CU -> read, this only establishes this fact for one arbitrary CU

#prob2
accessible(type,m) does not ensure that the exporting module (m1) owns the type
intention is probably that ownership is ensured via the chain of hasNamedMember_P_T and exportedTo. See #prob5.
#prob3
read is mentioned in 7.4.3, but not explicitly defined, neither in JLS nor API doc (only in SOTMS)
API: "A readability graph is constructed"
An interim version of 7.3 has been extended to refer to method java.lang.module.Configuration.resolve() to define "reads".
Ok green.gif As of "2017-06-26" JLS 7.3 refers to the package documentation of java.lang.module, which seems sufficient for a definition.

#prob4
Text still says "The package name Q.Id names a package that is the member named Id within the package named by Q". So, if two modules declare package p0.p1, how many packages p0 do we have?
  • if one, then p0 does not have "the member named p1", but is has two such members.
  • if two, then "uniquely visible to" must be applied to disambiguate even the toplevel package. "Uniquely visible" (7.4.3), however, requires an ordinary compilation unit in the package. Where a toplevel (or parent) package has no compilation units, it is not "uniquely visible".
workaround: ignoring the "the" in "the member named Q.Id" seems to fix this.

Ok green.gif Confirmed: Conceptually, one package Q can contain multiple sub packages named "Id", as long as only one of them is uniquely visible at the location of reference. The uniquely visible sub package is "the" member named Id. Technically, the spec doesn't differentiate different packages Q.Id, by the rule for unique visibility it just ensures that a split package (with contributions from multiple modules) never creates a conflict during name resolution.
#prob5
7.7.2 doesn't even attempt to define which package of a given name is exported (given we know that a name does not uniquely identify a package).
At a closer look, JLS only emulates per-module packages. Technically, packages exist independent of modules, and exporting a package factually means to export all types of compilation units that meet both following criteria:
  1. the CU is associated to the exporting module
  2. the CU declares the exported package
#prob6
As per this post, we are not allowed to apply recursion in 6.5.3.2 - in particular the requirement of unique visibility does not apply to any prefix Q. As a result the seemingly recursive part of rule 6.5.3.2 does not provide any clarification to anything.

Resolving package-vs-type conflicts

When a qualified name Q could identify both a package and a type, then JLS 6.5.4.2 generally favors types over packages.

It is an open issue, however, whether or not visibility / accessibility can influence this selection.

Java 8-

Consider:

  • package-private type p1.p2.t3 (possibly containing a member type t4)
  • public type p1.p2.t3.t4
  • a qualified type reference p1.p2.t3.t4

(need separate compilation of both types to avoid error regarding clash between package and type).

javac
accept
ecj
The type p1.p2.t3 is not visible

6.5.3.2 finds that package p1.p2 has a member type named t3 and concludes that p1.p2.t3 is a TypeName. Usage of that type fails due to lack of access permissions. That's what ecj concludes. I have no idea what lets javac accept it.

Since full compilation lets both compilers complain about the package-vs-type clash, the answer concerning the type reference could be seen as irrelevant.

Java 9

Similar to the above, but apply accessibility at the package level: make all types public and let one module export package p1.p2.t3, while p1.p2 (containing type t3) in another module is not exported.

Now the input is legal and hence the answer regarding the type reference p1.p2.t3.t4 is relevant.

javac
accept
ecj
The type p1.p2.t3 is not visible

Open question: Can we let ecj accept this program without creating an obvious conflict with JLS?

Previous attempts

Sources:

To determine legal program structures a number of concepts must be considered, including:

  • scope
  • visibility
  • importing
  • accessibility
  • observability


Terms & Concepts

modular compilation unit
compilation unit containing a ModuleDeclaration
ordinary compilation unit
compilation unit that is not a "modular compilation unit" (7.3)
associated
a compilation unit is said to be "associated" with a module (membership)
  • An observable modular compilation unit is associated with the module that it declares (7.3)
  • For regular compilation units membership in a module is defined by the host system.

Accessibility of a type

Let's try to define the algorithm for determining accessibility of a type.

Start at 6.6.1 Determining Accessibility:

If a top level class or interface type is declared public and is a member of a package that is exported by a module, then the type may be accessed by any code in the same module, and by any code in another module to which the package is exported, provided that the compilation unit in which the type is declared is visible to that other module (§7.3).

so our checklist for cross-module accessibility is

  • type is public
  • containing package is exported by a module
  • referring code location is within another module to which the package is exported
  • compilation unit is visible to the other module

When is a compilation unit visible to a given module?

7.3 Compilation Units

[...] The ordinary compilation units that are visible to M are the observable ordinary compilation units associated with modules read by M. […]

Most of this is determined by "the host system" (i.e., not specified by JLS):

  • observability of a compilation unit (7.3)
apparently, this is where JEP 261 comes into the picture, which states (under "Module paths"):
  • "The module definitions present on these paths, together with the system modules, define the universe of observable modules.".
  • association of a compilation unit to a module (7.3)

Remains: source and target modules have a reads connection.

The host system must use the Java Platform Module System to determine which modules are read by M (§7.7.1).

Dealing with ambiguous names

Let's now try to find out how a compiler is supposed to handle ambiguous names, in particular with regard to package names and qualified type names, possibly referring to elements from different modules.

Determining the meaning of a name

Let's assume that this question will be answered in section 6.5 ("Determining the meaning of a name").

6.5.3.2 Qualified Package Names
If a package name is of the form Q.Id, then Q must also be a package name. The package name Q.Id names a package that is the member named Id within the package named by Q.
If Q.Id does not name a package visible to the current module, then a compile-time error occurs.

Nothing hints at the possibility for multiple packages of the same name (per module), so if two modules would try to declare the same package we should probably consider this as one split package. This is also in line with this sentence:

6.4.1 Shadowing
[...] A package declaration never shadows any other declaration.

implying that in particular a package from one module cannot shadow a same-named package from another module.

From hear-say we know that split packages should be banned from Java 9, but where would this be defined and what kind of error should be signalled?

6.5.5.2 Qualified Type Names
If a type name is of the form Q.Id, then Q must be either a type name or a package name.
If a type name is of the form Q.Id, then Q must be either the name of a visible type or the name of a visible package.
If Id names exactly one accessible type (§6.6) that is a member of the type or package denoted by Q, then the qualified type name denotes that type.

The last sentence could accommodate the situation that a package Q might contain multiple accessible types named Id, in which case the qualified type name Q.Id cannot be resolved.

  • Does this mean a name clash (wrt fully qualified names) between two types from different modules should be flagged as a compile error?
    • If both are accessible: Yes
  • Does this mean a name clash involving one accessible type and one or more inaccessible types is not an error?
    • Correct
  • If the latter, does this mean that split packages (across modules) without clashing type names are not an error?
    • Correct
Idea.png
Solution
The trick here is, that resolving of a qualified name applies access control: if two types have the same qualified name, but one is not accessible, then it's not a name clash.
Consequences
The compiler has to distinguish same-named types from different modules.
  • It must detect that these are not compatible
  • It must detect (and allow) when an accessible sub type S of an inaccessible type super type T is passed where the inaccessible type T is required.
  • As a consequence it must be reported as an error if any (inaccessible) type is not available at compile time which is used, e.g., in the signature of a candidate method of any invocation being compiled. To be continued in bug 148844.

The above definitions are also based on visibility of packages and accessibility of types. We already discussed accessibility of types, but how is visibility of packages defined?

When speaking of "visible", the JLS text typically refers to 6.4.1 (Shadowing), with the following exception:

6.3 Scope of a declaration
[...] The scope of the declaration of an observable top level package (§7.4.3) is all observable compilation units associated with modules to which the package is visible (§7.3).

Hence, visibility of packages should be defined in 7.3.

7.3 Compilation Units
[...] The ordinary compilation units that are visible to M are the observable ordinary compilation units associated with modules read by M. The host system must use the Java Platform Module System to determine which modules are read by M (§7.7.1).
The ordinary compilation units that are visible to M drive the packages that are visible to M (§7.4.3), which in turn drives the top level packages in scope for code in compilation units associated with M (§6.3).

The first sentence was already discussed above. What's new is the fact that a package p is visible to M if any compilation unit declaring the package p is visible to M.

Ergo, in the hypothetical case that multiple modules declare types in the same package p, p will be visible to M if only one type from a module M1 is visible to M, disregarding the possibility that M2 may define more types in p, which may be invisible to M.

Let's look into the section referenced from 7.3:

7.4.3 Observability of a Package
A package is visible to a module if and only if an ordinary compilation unit containing a declaration of the package is visible to the module.

Yep, no contradiction to my hypothesis.

Imports

7.5.1 Single-Type-Import Declarations
The TypeName must be the canonical name of a class type, interface type, enum type, or annotation type (§6.7).

Similar in 7.5.2 (Type-Import-on-Demand Declarations). Compile errors are specified only in terms of visibility and accessibility. => This doesn't seem to require detection of ambiguity / duplicate types.

Extent of exports

If several modules M0, M1,... have compilation units declaring package P, and if M0 export P, which types are affected? Asked differently, under which conditions is M1.P.T1 covered by the export?

  • if M0 requires M1 (which results in all compilation units of M1 being visible in M0)?
  • if M0 requires and M1 exports P?
  • ...

Which section in JLS defines this?

Questions

Package Names

Does JLS

  • prevent split packages, or
  • handle ambiguity of package names (ambiguous membership of a package in several modules)?

File System Organization

7 Packages and Modules
Modules and packages may be stored in a file system or in a database (§7.2). Modules and packages that are stored in a file system may have certain constraints on the organization of their compilation units to allow a simple implementation to find module and type declarations easily.
7.3 Compilation Units
The host system also determines which observable ordinary compilation units are associated with a module, except for ...
  • Does JLS provide any help in coordinating "constraints" between implementations?
Exploded-module directory (see JEP 261)
  • Under "Module paths" we have
"An exploded-module directory whose name is, by convention, the module's name and whose content is an "exploded" directory tree corresponding to a package hierarchy."
  • Later under "Compile time" we see that the module's name may be used as a path segment anywhere on the source-module-path, e.g. {src/*/{share,<os>}/classes,build/gensrc/*} where '*' represents the module name.

This looks inconsistent, but the second, more detailed description could be useful, e.g., for using s.t. like </code>/${project}/src</code> as a module path in Eclipse (provided project name and module name match).

Remaining problem: JEP 261 does not allow class files to be created into separate output folders (per module). Ergo: sticking to JEP 261 would not allow us to compile modules in different projects in one go while conforming to the Eclipse strategy of per-project output folders.

JPMS

7.3 Compilation Units
The host system must use the Java Platform Module System to determine which modules are read by M (§7.7.1).
  • Alex promised to amend 7.3 with a reference to API doc of j.l.module.Configuration
  • Ok green.gif As of 2017-05-25 JLS 7.3 ("Compilation Units") does contain the expected reference.
Still, I'm not sure, why "reads" is not a first-class concept in JLS.
JLS 7.4.3 uses "reads" with no obvious link to what this means.
But those are just "cosmetic" deficiencies.

Automatic Modules

There have been long public discussions about "automatic modules", but the following seems to be all that JLS defines in this regard (7.7):

 The Java programming language does not distinguish between named modules specified
 explicitly in module declarations versus named modules specified by the Java Platform
 Module System when it detects a JAR file on the modulepath ("automatic modules"). A
 module declaration refers to other named modules without being aware of the manner in
 which they were specified.

Ok green.gif The algorithm for automatic modules is described in API doc of j.l.module.ModuleFinder.of(Path...).

Task

Check JDT implementation

  • Section 6.5.3 does not mandate a compile-time error if no module with that name is observable. This is because exports and opens statements in a module declaration are tolerant of non-existent module names, while the requires statement in a module declaration performs its own validation of the module name given as its operand.
Ok green.gif given as of bug 500636#c4
  • PR version of JLS still says about the name of a module "every identifier in the name must end with a Java letter". But this issue seems to have been resolved differently. I don't think JDT ever implemented the now-obsolete rule?

Critique

Confusion

Also the term "observability" comes in different flavors, as 7.4.3 introduces a distinction between "'technically' observable" and "'really' observable" (the latter called "visible"). I believe this distinction only works around the legacy that packages are defined to span a containment tree (packages as members of other packages), whereas JPMS wants to regard all packages as independent, disregarding "containment" / "membership".


Non-JMPS issues

  • JLS 13.1 ("The Form of a Binary") in its 2nd item 6 speaks of "code for the field initializers, the instance and static initializers,", which could be read as requiring that field initializers should be encoded separately from instance and static initializers. Do we have any indications that a change is intended here?

Back to the top