A potential bug in the Project Oberon compiler

[Guy T.]

Hello All,

I'm currently playing with the Project Oberon compiler. Adding code from the Extended Oberon effort, I found a potential bug in the current compiler. In file ORP.Mod, inside the Declarations procedure, there is the following lines:

IF (x.type.form = ORB.String) & (x.b = 2) THEN ORG.StrToChar(x) END ;
ORB.NewObj(obj, id, ORB.Const); obj.expo := expo;
IF x.mode = ORB.Const THEN obj.val := x.a; obj.lev := x.b; obj.type := x.type

When a string constant is declared, x.b receives the string length (including the null character at the end). On the third line above, the obj.lev is receiving the string length, not the level of the declaration. Is it the right intent? If not, it would then be preferable to get the level as follow maybe:

IF (x.type.form = ORB.String) & (x.b = 2) THEN ORG.StrToChar(x) END ;
ORB.NewObj(obj, id, ORB.Const); obj.expo := expo;
IF x.mode = ORB.Const THEN obj.val := x.a; obj.lev := level; obj.type := x.type

Is there a place on the Internet where I can signal this kind of issue?

Cheers

Guy

[c***@gmail.com]

Post by Guy T.
Hello All,
I'm currently playing with the Project Oberon compiler. Adding code from the Extended Oberon effort, I found a potential bug in the current compiler.

Can you provide the source code of the *simplest* test example that convinces yourself that it is actually a bug?

Regards,
Chris Burrows
https://www.astrobe.com

[Guy T.]

Post by c***@gmail.com

Post by Guy T.
Hello All,
I'm currently playing with the Project Oberon compiler. Adding code from the Extended Oberon effort, I found a potential bug in the current compiler.

Can you provide the source code of the *simplest* test example that convinces yourself that it is actually a bug?
Regards,
Chris Burrows
https://www.astrobe.com

To "see" the problem, you have to modify the compiler. One way is to compile the following:

MODULE Test;
CONST a = "HELLO!";
END Test.

After having modified the compiler by adding a trace in the Log (The last two lines below):

IF (x.type.form = ORB.String) & (x.b = 2) THEN ORG.StrToChar(x) END ;
ORB.NewObj(obj, id, ORB.Const); obj.expo := expo;
IF x.mode = ORB.Const THEN obj.val := x.a; obj.lev := x.b; obj.type := x.type;
Texts.WriteString(W, "===> "); Texts.WriteInt(W, obj.lev, 0);
Texts.WriteLn(W); Texts.Append(Oberon.Log, W.buf);

I found this as I was adding more strict verification code in the "qualident" procedure (at the end of it):

ELSIF (obj.lev > 0) & (obj.lev # level) &
((obj.class # ORB.Const) OR (obj.type.form # ORB.Proc)) THEN
ORS.Mark("not accessible: ");
END
END quasiment;

So compiling the following raised the error "not accessible":

MODULE Test;
CONST a = "HELLO!";

PROCEDURE P;
VAR aa : ARRAY 12 OF CHAR;
BEGIN aa := a; END P;

END Test.

[Guy T.]

Post by c***@gmail.com

Post by Guy T.
Hello All,
I'm currently playing with the Project Oberon compiler. Adding code from the Extended Oberon effort, I found a potential bug in the current compiler.

Can you provide the source code of the *simplest* test example that convinces yourself that it is actually a bug?
Regards,
Chris Burrows
https://www.astrobe.com

To "see" the problem, you have to modify the compiler. One way is to compile the following:

MODULE Test;
CONST a = "HELLO!";
END Test.

After having modified the compiler by adding a trace in the Log (The last two lines below):

IF (x.type.form = ORB.String) & (x.b = 2) THEN ORG.StrToChar(x) END ;
ORB.NewObj(obj, id, ORB.Const); obj.expo := expo;
IF x.mode = ORB.Const THEN obj.val := x.a; obj.lev := x.b; obj.type := x.type;
Texts.WriteString(W, "===> "); Texts.WriteInt(W, obj.lev, 0);
Texts.WriteLn(W); Texts.Append(Oberon.Log, W.buf);

When compiling, the example, you will get ===> 7 in the log.

I found this as I was adding more strict verification code in the "qualident" procedure (at the end of it):

ELSIF (obj.lev > 0) & (obj.lev # level) &
((obj.class # ORB.Const) OR (obj.type.form # ORB.Proc)) THEN
ORS.Mark("not accessible: ");
END
END qualident;

So compiling the following raised the error "not accessible", but it was expected to be ok:

MODULE Test;
CONST a = "HELLO!";

PROCEDURE P;
VAR aa : ARRAY 12 OF CHAR;
BEGIN aa := a; END P;

END Test.

[c***@gmail.com]

Post by Guy T.
MODULE Test;
CONST a = "HELLO!";
PROCEDURE P;
VAR aa : ARRAY 12 OF CHAR;
BEGIN aa := a; END P;
END Test.

The new test you have added checks for access to constants that have been declared at an intermediate level i.e. they are not global or strictly local. The current implementation of the official Project Oberon 2013 compiler is not concerned about this. There was an extensive debate as to whether or not that is a correct interpretation of the language report in the ETH Oberon Mailing list a couple of years ago:

[Oberon] Intermediate scopes in Oberon-07:

http://lists.inf.ethz.ch/pipermail/oberon/2018/011549.html

[Guy T.]

Post by c***@gmail.com

Post by Guy T.
MODULE Test;
CONST a = "HELLO!";
PROCEDURE P;
VAR aa : ARRAY 12 OF CHAR;
BEGIN aa := a; END P;
END Test.

http://lists.inf.ethz.ch/pipermail/oberon/2018/011549.html

Wow!! I was not aware of such a debate (I missed this forum for a couple of years...). I understand the difficulty to interpret the Oberon-07 report vs the implementations of it and I'm not incline to continue or add to this debate!!

I'm currently building a compiler for the ESP32 chip, adding functionalities to the Project Oberon compiler, both in line with the report (as long as my interpretation is correct...) and functionalities to optimise the code generated. Still a work in progress but going well. You can find it here (https://github.com/turgu1/esp32-oberon-compiler).

My point is still valid but with no direct consequence (it seems to me) to the way that the Project Oberon compiler is working. The obj.lev does not receive the level of the constant string but it's length. It maybe not a big deal as the current implementation doesn't care about it. Or there is some insight reason to have the string length put in the obj.lev that is not visible to me.

Cheers!

Guy

[a***@gmail.com]

Post by Guy T.
My point is still valid but with no direct consequence (it seems to me) to the way
that the Project Oberon compiler is working. The obj.lev does not receive the
level of the constant string but it's length. It maybe not a big deal as the current
implementation doesn't care about it. Or there is some insight reason to have
the string length put in the obj.lev that is not visible to me.

For the record, the meaning of the field obj.lev for string constants is as follows:

1. In Project Oberon 2013, the field obj.lev is "abused" to hold the length (len) of string constants. The reason is (probably) that there simply aren't any other fields available in obj to hold the string length. This choice may also be one of the reasons, why exporting and importing string constants is, in fact, not implemented in Project Oberon 2013, i.e. it just does not work (unlike in Extended Oberon).

2. In Extended Oberon, the field obj.lev consistently holds the scope level for all (!) declared identifiers, i.e. also for constants, types and procedures, not just variables (see ORP.Declarations). This makes it trivial to disallow access to all (!) intermediate identifiers with a single ELSIF clause at the end of ORP.qualident (instead of ORG.MakeItem), and in all (!) cases when an item is used, not just when it is initially created.

3. In Extended Oberon, the length of a string constant (len) is encoded together with its string buffer position (strx) in the single object field obj.val. This was necessary because no other fields were available in obj (except exno perhaps, but there are other reasons not to abuse that field). The encoding of obj.val (20bits for strx, 12 bits for len for globally declared strings) was chosen such that implementing exporting and importing string constants is made easy, i.e. such that the *same* code in ORG.MakeItem can be used for global and imported string constants (where the least significant 20bits are the exno):

x.a := y.val MOD 100000H; (*strx/exno*) x.b := y.val DIV 100000H (*len*)

The field x.r is used to determine whether a string is global (x.r = 0) or imported (x.r = mno). This is why ORG.MakeStringItem (which is only ever called for global, but not for imported strings) has been adapted to include the assignment x.r := 0, and ORG.MakeItem (which is used for imported strings), the assignment x.r := y.lev automatically sets x.r to the mno for imported strings (because ORB.Import sets y.lev to mno).

For the above reasons, one cannot simply mix and match code from Project Oberon 2013 and Extended Oberon. One has to take "all or nothing" from ORP.Declarations, ORP.qualident, ORP.loadStringAdr, ORG.MakeStringItem, ORG.MakeItem, ORB.Export and ORG.Close.

All this is explained in more detail at:

https://github.com/andreaspirklbauer/Oberon-importing-string-constants

[Guy T.]

Post by a***@gmail.com

Post by Guy T.
My point is still valid but with no direct consequence (it seems to me) to the way
that the Project Oberon compiler is working. The obj.lev does not receive the
level of the constant string but it's length. It maybe not a big deal as the current
implementation doesn't care about it. Or there is some insight reason to have
the string length put in the obj.lev that is not visible to me.

1. In Project Oberon 2013, the field obj.lev is "abused" to hold the length (len) of string constants. The reason is (probably) that there simply aren't any other fields available in obj to hold the string length. This choice may also be one of the reasons, why exporting and importing string constants is, in fact, not implemented in Project Oberon 2013, i.e. it just does not work (unlike in Extended Oberon).
2. In Extended Oberon, the field obj.lev consistently holds the scope level for all (!) declared identifiers, i.e. also for constants, types and procedures, not just variables (see ORP.Declarations). This makes it trivial to disallow access to all (!) intermediate identifiers with a single ELSIF clause at the end of ORP.qualident (instead of ORG.MakeItem), and in all (!) cases when an item is used, not just when it is initially created.
x.a := y.val MOD 100000H; (*strx/exno*) x.b := y.val DIV 100000H (*len*)
The field x.r is used to determine whether a string is global (x.r = 0) or imported (x.r = mno). This is why ORG.MakeStringItem (which is only ever called for global, but not for imported strings) has been adapted to include the assignment x.r := 0, and ORG.MakeItem (which is used for imported strings), the assignment x.r := y.lev automatically sets x.r to the mno for imported strings (because ORB.Import sets y.lev to mno).
For the above reasons, one cannot simply mix and match code from Project Oberon 2013 and Extended Oberon. One has to take "all or nothing" from ORP.Declarations, ORP.qualident, ORP.loadStringAdr, ORG.MakeStringItem, ORG.MakeItem, ORB.Export and ORG.Close.
https://github.com/andreaspirklbauer/Oberon-importing-string-constants

Thank you very much for these explanations. It helps a lot understanding some implementation aspects.

I'm puzzle about what to implement or not in regard of extending the Project Oberon Compiler. Up to now, I have limited these extensions to the numerical CASE statement based on your own code in Extended Oberon, and string <-> char constant assignments. I also added some `standard` functions to get access to specific instructions available with the ESP32. That's part of the fun to create a new compiler I guess.

Thanks again for your help!

Guy

[a***@gmail.com]

Post by Guy T.
I'm puzzle about what to implement or not in regard of extending the
Project Oberon Compiler. Up to now, I have limited these extensions
to the numerical CASE statement based on your own code in Extended
Oberon, and string <-> char constant assignments. I also added some
`standard` functions to get access to specific instructions available
with the ESP32. That's part of the fun to create a new compiler I guess.

Yes, it's generally helpful to experiment with certain features, it greatly
helps to increase the understanding of some of the nitty-gritty of a
compiler.

I tend to stick to the language report as closely as possible. This was
the case with limiting access to intermediate objects and with the
numeric case statement.

And where I deviate, I at least try to make it a strict superset of
Oberon-07. This was the case with the ELSE clause of the case
statement and with type-bound procedures.

[a***@gmail.com]

If you have copied code from Extended Oberon, keep in mind that there the meaning of obj.lev is different from the official Oberon-07 compiler. The Extended Oberon compiler strictly disallows access to all intermediate objects and it allows exporting/importing of string constants.

See

https://github.com/andreaspirklbauer/Oberon-no-access-to-intermediate-objects

https://github.com/andreaspirklbauer/Oberon-importing-string-constants

-ap

[Guy T.]

Post by a***@gmail.com
If you have copied code from Extended Oberon, keep in mind that there the meaning of obj.lev is different from the official Oberon-07 compiler. The Extended Oberon compiler strictly disallows access to all intermediate objects and it allows exporting/importing of string constants.
See
https://github.com/andreaspirklbauer/Oberon-no-access-to-intermediate-objects
https://github.com/andreaspirklbauer/Oberon-importing-string-constants
-ap

Hello Andreas,

At first that is what I did in terms of strict access constraint. At the end, I've decided to relax the constraint. Up to now, no body explained the meaning of obj.lev if it's not the level of the object. That's why I think that there is a potential bug there, even if there is no consequence to the compiler execution.

As I'm working on my ESP32 compiler, I'm seeing the limitations of both the report and the Project Oberon Compiler in term of formal definition. I'm trying to get my compiler as close as possible to the report definition, but I understand the difficulties related to its interpretation.

I've documented changes I did to the compiler, both to get it closer to the report, and added functionalities I consider useful for IOT development.

Thanks!

Guy

[a***@gmail.com]

At the very end - in the appendix - of

https://github.com/andreaspirklbauer/Oberon-no-access-to-intermediate-objects

there is ONE way to implement relaxed shadowing rules. It‘s one line of code in ORB.thisobj.

The relaxed shadowing rules would allow accessing an object of the global scope, even if an object with the same name is also declared at an intermediate scope, but not the local one (‚piercing through the shadow‘).

Such a rule would make nested procedures self-contained in the sense that they can be moved around more freely.

The reasons why such relaxed shadowing rules have NOT been adopted in Extended Oberon are:

1. A nested procedure (as in the familiar expression - term - factor construct) may, and typically does, call the surrounding procedure that contains it. It is therefore typically NOT self-contained anyway.

2. It would break with a long language tradition.