Revision as of 06:38, 9 June 2020

About environment variables

What is the problem

For CDT environment variables are very important. They are used both as input and output. As we are currently working on environment variables in CDT the minds get refreshed about the problems and it is a good time to write up some documentation about it.

Basically there are 2 components at play: The OS environment variables and there are CDT environment variables. They both live in their own space and have their own tools and rules to be handled and maintained. If the rules were the same this article would not exists. The main trouble some rule difference is "case insensitivity on Windows". (Note that the same problem exists for filenames which is out of scope of this page)

The relationship between the 2 main components can best be described as OS->CDT->OS

To make it clear what part of the relationship we are talking about we'll use OSin->CDT->OSout

The sandwiching and the different rules makes this a non trivial problem (to say the least).

An example

To make this a bit more concrete lets look at the build. The build itself is one or more tools(gcc) that run at the OSout level; and uses the environment variables known in the shell OSout. The tools do not know they are started by CDT.

There is no guarantee that gcc is in the path -or the gcc you want to use is the first in the path-. A common solution is to modify the path environment variable in OSout to point to the "correct gcc". Sometimes you can simply say "PATH=gcc_folder" but more likely you will need to tweak it like "PATH=gcc_folder;${PATH}". ... did you notice this just blew up into your face? No? Well, here are some problems

1) ; is windows separator not Linux/mac they use :

2) Windows is case insensitive and uses by default "Path" (on my Windows 10 system). After reading the environment strings from OSin ${Path} will resolve to the PATH but ${PATH} will resolve to null because java/eclipse/CDT/linux/macos are case sensitive by default. A workaround could be something like "PATH=gcc_folder;${PATH};${Path}"

3) "PATH=gcc_folder;${PATH}" could work in OSin and OSout but not inside CDT. This is because the OS immediately resolves the environment variable and CDT does not. To make this work in CDT this would be something like set("PATH","gcc_folder;"+resolve("PATH"))

I say could work but I should say won't work as it should be "PATH=gcc_folder;%PATH%" on windows and "PATH=gcc_folder:${PATH}" on Linux and Mac

4)From java/CDT point of view "PATH=gcc_folder;${PATH}" is recursive and as such unresolvable

In short: "There is no problem in getting/setting/adding/removing environment variables but there is a problem resolving environment variables is a OS compliant way".

Note that windows OS does not uppercase the environment variables. This means that tools like make and CDT can decide to be case sensitive

There is also some good news: "Environment variables received from the OS are resolved.".

For 1) just use java.io.File.pathSeparator and you'll be fine. This is not part of the scope of this document.

The current CDT solution

Currently CDT uppercases environment variables on the windows OS and remains internally case sensitive.

This creates a behaviour that is nor Windows nor Linux like.

For instance

Creating the environment variables

Test=Hello

Testing=pre_${Test}_post

The resolving of "Testing" will result in an empty string because CDT uppercased Testing. (There may be cases the resolve uppercased and this works like in the next step)

The resolving of "TESTING" will result in pre__post because the environment variable name Test has been uppercased but the reference ${Test} not.

So on windows the creation of the environment variables could be done as

TEST=Hello

TESTING=pre_${TEST}_post

When this is done by tooling this can be fixed with regular expression and all is fine, right? ...... no.

Suppose we have

HELLO_JANTJE=Welcome to the club Jantje

USER=Jantje

TESTING=${HELLO_${USER}}

When expanding TESTING level one resolve becomes ${HELLO_Jantje} which does not match any environment variable because it is not uppercased.

The proposed new CDT solutions

There is agreement to remove the uppercasing from environment variables on windows. However this leaves us with the problem to find the environment variables of OSin (like PATH)

Provide uppercase links

What is it

On windows when adding OS environment variables we add 1 or 2 environment variables

KEY=value

key=${KEY}

Pro's and cons

I only found 2 locations that contain System.getenv() /org.eclipse.cdt.core/src/org/eclipse/cdt/core/build/CBuildConfiguration.findCommand and /org.eclipse.cdt.core.native/src/org/eclipse/cdt/utils/spawner/EnvironmentReader.init

init is very easy to fix, findCommand will be a bit harder.

However there may be other ways to get to the environment variables.

There may be code that pushes KEY to OSout but not key in which case the solution will fail.

Use system.genEnv("PATH") to get to the value

What is it

For environment variables that are read from OSin use system.genEnv("PATH") (need to check but according to [1] this should work.

Pro's and con's

This only works if you want the value and not ush it back to the environment variables.

I don't think this is viable for CDT

use a environmentNameProvider

What is it

Instead of using "PATH" use something like private final string PATH=EnvironmentReader.getPathEnvironmentName();

Pro's and con's

This is from a user perspective the cleanest solution because there are no extra environment variables created.

However this will probably require a case insensitive search.

Known keys to "map" are PATH CYGWIN_HOME MINGW_HOME MSYS_HOME QMAKE

This solution has the disadvantage that when there are multiple paths variables the user does not know which environment variable is being used

only Uppercase environment variables from windows OSin

What is it

On windows; only the environment variables of OSin are uppercased. When pushing the environment variables to OSout the OSin provided environment variables only the uppercased variants are set to handle duplicates.

Pro's and con's

It is clear which variables are being used Tools used on windows will have to take into account that OSin environment variables are uppercased in CDT Very little code impact

only Uppercase environment variables from windows OSin that are explicitly used by CDT

What is it

Only the environment variables of OSin that are explicitly used by CDT are uppercased. These are probably the same as the keys from the environmentNameProvider solution When pushing the environment variables to OSout these can be set again after the push overwriting potential duplicates.

Pro's and con's

As the keys are known one could do a check for "unwanted" environment variables It is clear (though maybe not always obvious to the end user) which variables are being used It is very unlikely these are being referenced by tool content created environment variables (What I mean is that if the tool is using them it is at the configuration level and should be easy to fix) Very little code impact