[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Ptolemy] Re: limited success with JDK1.4 and soot
At 05:27 PM 10/11/2002 +0200, Stephen Andrew Neuendorffer wrote:
ok, I have the start of a solution... in soot.SootClass:
public SootMethod getMethod(String name, List parameterTypes, Type
returnType)
{
SootClass theClass = this;
while(theClass != null) {
Iterator methodIt = getMethods().iterator();
while(methodIt.hasNext()) {
SootMethod method = (SootMethod) methodIt.next();
if(method.getName().equals(name) &&
parameterTypes.equals(method.getParameterTypes()) &&
returnType.equals(method.getReturnType())) {
return method;
}
}
if(theClass.hasSuperclass()) {
theClass = theClass.getSuperclass();
} else {
theClass = null;
}
}
This method is used to resolve method invocations when bytecode is being
parsed... This code does the link time search up the class tree to
resolve the method, which should result in placing the actual method that
is being called placed into the soot data structures.
To be complete, the same fix needs to be made for getField as well.
I tried this, and successfully ran the code generator after recompiling
with -target 1.2. Probably the other methods that get fields and methods
should also do this resolution. On the other hand the soot guys might want
to do this in a more backward compatible way...
(Add 2 new methods, and change coffi to call the new methods when
jimplifying.) Diff of my changes is attached.
Furthermore, the bytecode output routines need to be changed. Instead of
generating the invocation/field reference verbatim, it should probably
output code that is based on the inferred type of the local variable that
the invocation occurs on...
Haven't done this yet... But in order to be in compliance with the JLS, it
needs to be done.
Steve
At 08:33 PM 9/23/2002 -0700, cxh@eecs.berkeley.edu wrote:
Edward asked about upgrading to JDK1.4, and as far as I know,
the only thing that is holding us back is codegen.
I'm working on getting codegen to work under JDK1.4. Steve and I had
a chat on Sunday about this, and I hope to get codegen working under
JDK1.4 while he is away.
Basically, the class file format changed between 1.1 and 1.2, but
the 1.2 and 1.3 compilers generated 1.1 format byte code.
In 1.4, the default output style is 1.2
However, if we use the -target 1.1 javac option, then we can generate
1.1 compatible code with 1.4.
I'm working on a small test case now, and I'll ping the soot folks
shortly.
The longer answer:
If class A defines an int a and class B extends class A and uses int
a, then JDK1.3.1_04 (which defaults to -target 1.1 or ClassFile
version 45.3) produces a different constant pool value than JDK1.4.1
(which defaults to -target 1.2, or ClassFile version 46.0)
JDK1.3.1_04 produces:
[2] fieldref=soot/coffi/A.a
JDK1.4.1 produces
[2] fieldref=soot/coffi/B.a
This causes no end of problems in Ptolemy II when we try to fold
subclasses into their parent classes because with JDK1.4.1, we can no
longer determine that a is part of the superclass by looking
at the fieldref.
Here's how to reproduce this:
Download and build soot, then create A.java and B.java in
soot/src/soort/coffi
and compile B.java with JDK1.3.1_04 and then JDK1.4.1:
bash-2.05a$ pwd
c:/cxh/src/soot-1.2.3/soot/src/soot/coffi
bash-2.05a$ cat A.java
package soot.coffi;
public class A {
public int a;
}
bash-2.05a$ cat B.java
package soot.coffi;
public class B extends A {
public B() {
a = 1;
}
public static void main(String [] args) {
ClassFile classFile = new ClassFile("B");
classFile.loadClassFile();
classFile.listFields();
classFile.listConstantPool();
}
}
bash-2.05a$ rm B.class
bash-2.05a$ c:/jdk1.3.1_04/bin/javac -classpath
"../..;../../../classes/;$PTII/lib/jasminclasses.jar;" B.java
bash-2.05a$ c:/jdk1.3.1_04/bin/java -classpath
"../..;../../../classes/;$PTII/lib/jasminclasses.jar;" soot.coffi.B >&
1.3.1.txt
bash-2.05a$ cat 1.3.1.txt
Magic number ok
Version: 45.3
Constant pool count: 36
Access flags: 33 = public, super
Has 0 field(s)
Has 2 method(s)
Has 1 attribute(s)
[1] methodref=soot/coffi/A.<init>
[2] fieldref=soot/coffi/A.a
[3] class=soot/coffi/ClassFile
[4] string="B"
[5] methodref=soot/coffi/ClassFile.<init>
[6] methodref=soot/coffi/ClassFile.loadClassFile
[7] methodref=soot/coffi/ClassFile.listFields
[8] methodref=soot/coffi/ClassFile.listConstantPool
[9] class=soot/coffi/B
[10] class=soot/coffi/A
[11] utf8=<init>
[12] utf8=()V
[13] utf8=Code
[14] utf8=LineNumberTable
[15] utf8=main
[16] utf8=([Ljava/lang/String;)V
[17] utf8=SourceFile
[18] utf8=B.java
[19] nameandtype=<init>
[20] nameandtype=a
[21] utf8=soot/coffi/ClassFile
[22] utf8=B
[23] nameandtype=<init>
[24] nameandtype=loadClassFile
[25] nameandtype=listFields
[26] nameandtype=listConstantPool
[27] utf8=soot/coffi/B
[28] utf8=soot/coffi/A
[29] utf8=a
[30] utf8=I
[31] utf8=(Ljava/lang/String;)V
[32] utf8=loadClassFile
[33] utf8=()Z
[34] utf8=listFields
[35] utf8=listConstantPool
bash-2.05a$ rm B.class
bash-2.05a$ c:/j2sdk1.4.1/bin/javac -classpath
"../..;../../../classes/;$PTII/lib/jasminclasses.jar;" B.java
bash-2.05a$ c:/jdk1.3.1_04/bin/java -classpath
"../..;../../../classes/;$PTII/lib/jasminclasses.jar;" soot.coffi.B >&
1.4.1.txt
bash-2.05a$ cat 1.4.1.txt
Magic number ok
Version: 46.0
Constant pool count: 36
Access flags: 33 = public, super
Has 0 field(s)
Has 2 method(s)
Has 1 attribute(s)
[1] methodref=soot/coffi/A.<init>
[2] fieldref=soot/coffi/B.a
[3] class=soot/coffi/ClassFile
[4] string="B"
[5] methodref=soot/coffi/ClassFile.<init>
[6] methodref=soot/coffi/ClassFile.loadClassFile
[7] methodref=soot/coffi/ClassFile.listFields
[8] methodref=soot/coffi/ClassFile.listConstantPool
[9] class=soot/coffi/B
[10] class=soot/coffi/A
[11] utf8=<init>
[12] utf8=()V
[13] utf8=Code
[14] utf8=LineNumberTable
[15] utf8=main
[16] utf8=([Ljava/lang/String;)V
[17] utf8=SourceFile
[18] utf8=B.java
[19] nameandtype=<init>
[20] nameandtype=a
[21] utf8=soot/coffi/ClassFile
[22] utf8=B
[23] nameandtype=<init>
[24] nameandtype=loadClassFile
[25] nameandtype=listFields
[26] nameandtype=listConstantPool
[27] utf8=soot/coffi/B
[28] utf8=soot/coffi/A
[29] utf8=a
[30] utf8=I
[31] utf8=(Ljava/lang/String;)V
[32] utf8=loadClassFile
[33] utf8=()Z
[34] utf8=listFields
[35] utf8=listConstantPool
bash-2.05a$
2c2
< Version: 45.3
---
> Version: 46.0
9c9
< [2] fieldref=soot/coffi/A.a
---
> [2] fieldref=soot/coffi/B.a
The Java Language Spec Clarification page at
http://java.sun.com/docs/books/jls/clarify.html
says:
--start--
Compilation of Symbolic References
Note: this change only applies to compilers targeted at virtual
machines that perform a resolution time lookup (see changes to 12.3.3
above). These include the JDK1.1 and 1.2 VMs (but not the JDK1.0.2
VM), and any virtual machine whose version number is 46.0 and above.
The third bullet of JLS 13.1 should be corrected to state the
following:
Given a legal field access expression in a class C referencing a field
f declared in another class:
If the expression is of the form
Primary.f
then let T be the type of Primary. T must be some reference type
denoting a class or interface.
If the expression is of the form
super.f
then let T be the superclass of C.
The reference to f must be compiled into a symbolic reference to the
class or interface T, plus the simple name of the field, f. The
reference must also include a symbolic reference to the declared type
of the field so that the verifier can check that the type is as
expected. References to fields that are static, final, and initialized
with compile-time constant expressions are resolved at compile time to
the constant value that is denoted. No reference to such a constant
field should be present in the code in a binary file (except in the
class or interface containing the constant field, which will have code
to initialize it), and such constant fields must always appear to have
been initialized; the default initial value for the type of such a
field must never be observed. See -13.4.8 for a discussion.
Similarly, the fourth bullet of JLS 13.1 should be corrected as
follows:
Given a method invocation expression in a class C referencing a method
m declared in another class:
If the expression is of the form
Primary.m
then let T be the type of Primary. T must be some reference type
denoting a class or interface.
If the expression is of the form
super.f
then let T be the superclass of C.
If the expression is of the form
X.m
then X must be the name of a class or interface. Let T be the class or
interface denoted by X.
The reference to a method or constructor must be resolved at compile
time to a symbolic reference to T, plus the signature of the method or
constructor. A reference to a method must also include either a
symbolic reference to the return type of the denoted method or an
indication that the denoted method is declared void and does not
return a value.
Here is an explanation why these corrections are necessary.
http://java.sun.com/docs/books/jls/public-symref-compilation-rules.html
Obviously, much of JLS 13.4.5 must change in light of the changes to
JLS 13.1 and 12.3.3.
--end--
http://java.sun.com/docs/books/jls/public-symref-compilation-rules.html
says
--start--
Rationale for the changes to JLS 13.1
This change corrects an error in the original JLS. The
original policy required that references be compiled so that a
symbolic reference was made to the declaring class. However, in some
cases it was not possible to compile legal Java programs. For
example, consider the following program:
package pack;
class OffLimits { // Note this has default access
public int x = 11;
}
public class Allowed extends OffLimits {}
package outside;
import pack.*;
public class Outsider extends Allowed {
public static void main(String[] args) {
System.out.println((new Outsider()).x);
}}
This program is legal, and yet, if we compile it according to
the directives in JLS 13.1, it will fail to execute on a Java
virtual machine because it violates the rules for access
control. Specifically, the expression
(new Outsider()).x
will be compiled as a symbolic reference to the declaring
class of x, pack.OffLimits. When resolving this symbolic
reference, the Java virtual machine will raise an
IllegalAccessError, because the
class pack.OffLimits is not accessible from the class
outside.Outsider. Under the corrected rule, the code executes
correctly.
Another benefit of the revised rules is that the structure of
the inheritance graph of a class is hidden from its
clients. This enables additional binary compatible
changes. For example, fields and methods can be moved up and
down the hierarchy. As long as a class continues to expose a
field or method to its clients, those clients will function
regardless of where the field or method is declared.
Unfortunately, on older virtual machines such as the JDK1.0.2
VM, this strategy is unacceptable, because such VMs do not perform
a link time search up the hierarchy.
Therefore, this strategy applies to compilers targeted at VMs
that do perform such a link time search (such as the JDK 1.1
and JDK 1.2 VMs).
--end--
* http://forum.java.sun.com/thread.jsp?forum=31&thread=275690
says:
javac v1.4 emits by default Java 1.2 compatible .class files
while all javac implementations since between 1.1.0 and 1.3.1 emit
byte default Java 1.1 .class files.
Check to see what effect using the -target option in javac to
get the two versions to produce Java 1.1, 1.2, or 1.3 .class
files.
* Description of the ClassFile format at
http://java.sun.com/docs/books/vmspec/2nd-edition/html/ClassFile.doc.html
Steve writes:
--------
I'm guessing that the semantics of reflection has changed in 1.4
somehow?
Then soot would be using reflection internally to resolve which class
methods are
declared in and such. For instance, I can imagine that the
virtualinvoke
problem at the
bottom is occuring because Parameter reports that it has a setExpression
method in 1.4,
even though it is inherited from Variable?
This will likely cause many errors if it is not fixed properly
within Soot,
since alot of what
I do is matching method names by signatures...
Steve
At 10:19 AM 9/17/2002 -0700, you wrote:
>Ok, I took a look at JDK1.4 and soot.
>
>The first error is documented in ptII/doc/codegen.htm, where
>a small test case barfs.
>
>There was a response to Steve's email to the soot mailing list:
>
> > Hi Stephen,
> >
> > I haven't try soot-1.2.2 on JDK1.4, but tried soot-1.2.3, I did
> > not find the source of the problem, but here is a quick hacker to
> > make it runable on JDK1.4
> >
> > In soot/jimple/toolkits/invoke/ClassHierarchyAnalysis.java
> > replace line 67 :
> > Iterator methodsIt = c.getMethods().iterator();
> > by:
> > Iterator methodsIt = c.getMethods().snapshotIterator();
> >
> > It is not verified that this change won't affect others. (look
> > at the code, it is not supposed to be).
> >
> > Cheers,
> > ===========================================================
> > Feng Qian fqian@sable.mcgill.ca
>
>I made this change to a copy of the soot-1.2.3 tree.
>It looks like this change does not break (cd copernicus/java; make
demo)
>under JDK1.3.1_04, but I'll verify this and update
>ptII/lib/sootclasses.jar and jasminclasses.jar if everything is ok.
>
>I also had to modify KernelMain/Main.java so that it no longer
>throws an error if we are running under JDK1.4
>
>Now the problem is that I get:
>
>Soot started on Mon Sep 16 22:13:00 PDT 2002
>WatchDogTimer.internalTransform(wjtp.watchDog, {time=600000,
cancel=false}
)
>ModelTransformer.internalTransform(wjtp.mt,
>{targetPackage=ptolemy.copernicus.java.cg.OrthogonalCom})
>ActorTransformer: Creating actor class
>ptolemy.copernicus.java.cg.OrthogonalCom.CGbitSource
>for actor .OrthogonalCom.bitSource based on
>ptolemy.actor.lib.DiscreteRandomSource
>Code generation of
>'file:/C:/cxh/ptII/ptolemy/domains/sdf/demo/OrthogonalCom/OrthogonalCom.xm
> l'failed:
>java.lang.RuntimeException: No field output in class
>ptolemy.copernicus.java.cg.OrthogonalCom.CGbitSource
> at soot.SootClass.getFieldByName(SootClass.java:230)
> at
>
ptolemy.copernicus.kernel.SootUtilities.changeTypesInMethods(SootUtilitie
s.java:462)
> at
>
ptolemy.copernicus.kernel.SootUtilities.foldClass(SootUtilities.java:1127
)
> at
>
ptolemy.copernicus.java.ActorTransformer._createAtomicActor(ActorTransfor
mer.java:357)
> at
>
ptolemy.copernicus.java.ActorTransformer.createActorsIn(ActorTransformer.
java:179)
> at
>
ptolemy.copernicus.java.ModelTransformer._composite(ModelTransformer.java
:250)
> at
>
ptolemy.copernicus.java.ActorTransformer.createCompositeActor(ActorTransf
ormer.java:490)
> at
>
ptolemy.copernicus.java.ModelTransformer.internalTransform(ModelTransform
er.java:191)
> at soot.SceneTransformer.transform(SceneTransformer.java:46)
> at soot.Pack.apply(Pack.java:79)
> at soot.Main.run(Main.java:1609)
> at
> ptolemy.copernicus.kernel.KernelMain.generateCode(KernelMain.java:167)
>
>
>I looked into this a little, and it looks if actor.lib is compiled
>with JDK1.3 instead of JDK1.4, then the above problem does not occur,
>but a similar problem with sdf occurs later because sdf was compiled
>with JDK1.4
>
>Looking in to the problem itself, I added printlns to
>soot.SootClass.getFieldByName() and
>soot.SootClass.addField()
>
>Under JDK1.4, addField() is being called with
>
>soot.SootClass.addField(<ptolemy.actor.lib.DiscreteRandomSource:
>ptolemy.actor.TypedIOPort output>)
>
>from copernicus.kernel.SootUtilitiies.foldClass()
>
>However, under JDK1.3, addField() does not get called with
>this parameter.
>
>In foldClass(), the call to addField() is caused by the marked line
>
>
> // Loop through all the methods again, this time looking for
> // method invocations on the old superClass... Inline
these call
s.
> // This code is similar to inlineCallsToMethod, but avoids
iterat
ing
> // over all the methods in the class twice, which could get
> // very expensive.
> for (Iterator methods = theClass.getMethods().iterator();
> methods.hasNext();) {
> SootMethod newMethod = (SootMethod)methods.next();
> Body newBody = newMethod.retrieveActiveBody();
>
> // use a snapshotIterator since we are going to be
manipulati
ng
> // the statements.
> Iterator j = newBody.getUnits().snapshotIterator();
> while (j.hasNext()) {
> Stmt stmt = (Stmt)j.next();
> if (stmt.containsInvokeExpr()) {
> InvokeExpr invoke =
(InvokeExpr)stmt.getInvokeExpr();
> if (invoke.getMethod().getDeclaringClass() ==
> superClass) {
> // Force the body of the thing we are
inlining to
be
> // loaded
>---> invoke.getMethod().retrieveActiveBody();
> SiteInliner.inlineSite(invoke.getMethod(),
> stmt, newMethod);
> }
> }
> }
> }
IAnother difference is that under JDK1.4, I see lines like:
>value = virtualinvoke $r11.<ptolemy.data.expr.Parameter: void
>setExpression(java.lang.String)>("{0.5, 0.5}")
>
>Yet under JDK1.3, I see lines like:
>value = virtualinvoke $r11.<ptolemy.data.expr.Variable: void
>setExpression(java.lang.String)>("{0.5, 0.5}")
>
>To put this in context, below are some of the lines around the 1.4 run:
>
>fieldRef = r3.<ptolemy.copernicus.java.cg.OrthogonalCom.CGbitSource:
>ptolemy.data.expr.Parameter pmf>
>value = r3.<ptolemy.copernicus.java.cg.OrthogonalCom.CGbitSource:
>ptolemy.data.expr.Parameter pmf>
>class = soot.jimple.internal.JInstanceFieldRef
>value = $r11
>class = soot.jimple.internal.JimpleLocal
>unit = $r11 =
>r3.<ptolemy.copernicus.java.cg.OrthogonalCom.CGbitSource:
>ptolemy.data.expr.Parameter pmf>
>value = $r11
>class = soot.jimple.internal.JimpleLocal
>value = "{0.5, 0.5}"
>class = soot.jimple.StringConstant
>---> value = virtualinvoke $r11.<ptolemy.data.expr.Parameter: void
>setExpression(java.lang.String)>("{0.5, 0.5}")
>class = soot.jimple.internal.JVirtualInvokeExpr
>unit = virtualinvoke $r11.<ptolemy.data.expr.Parameter: void
>setExpression(java.lang.String)>("{0.5, 0.5}")
>value = r3
>class = soot.jimple.internal.JimpleLocal
>
>
>-Christopher
--------
_______________________________________________
Ptolemy maillist - Ptolemy@gigascale.org
http://www.gigascale.org/ptolemy/listinfo/ptolemy
_______________________________________________
Ptolemy maillist - Ptolemy@gigascale.org
http://www.gigascale.org/ptolemy/listinfo/ptolemy
178,180c178
< System.out.println("searching for field " + name + " in " + this);
< SootClass theClass = this;
< while(theClass != null) {
---
> Iterator fieldIt = getFields().iterator();
182,195c180,185
< Iterator fieldIt = theClass.getFields().iterator();
<
< while(fieldIt.hasNext()) {
< SootField field = (SootField) fieldIt.next();
<
< if(field.name.equals(name) && field.type.equals(type))
< return field;
< }
< if(theClass.hasSuperclass()) {
< theClass = theClass.getSuperclass();
< } else {
< theClass = null;
< }
< System.out.println("Not found.. superclass = " + theClass);
---
> while(fieldIt.hasNext())
> {
> SootField field = (SootField) fieldIt.next();
>
> if(field.name.equals(name) && field.type.equals(type))
> return field;
360,378c350,360
< System.out.println("searching for method " + name + " in " + this);
< SootClass theClass = this;
< while(theClass != null) {
<
< Iterator methodIt = theClass.getMethods().iterator();
<
< while(methodIt.hasNext()) {
< SootMethod method = (SootMethod) methodIt.next();
<
< if(method.getName().equals(name) &&
< parameterTypes.equals(method.getParameterTypes()) &&
< returnType.equals(method.getReturnType())) {
< return method;
< }
< }
< if(theClass.hasSuperclass()) {
< theClass = theClass.getSuperclass();
< } else {
< theClass = null;
---
> Iterator methodIt = getMethods().iterator();
>
> while(methodIt.hasNext())
> {
> SootMethod method = (SootMethod) methodIt.next();
>
> if(method.getName().equals(name) &&
> parameterTypes.equals(method.getParameterTypes()) &&
> returnType.equals(method.getReturnType()))
> {
> return method;
380d361
< System.out.println("Not found.. superclass = " + theClass);
382c363
<
---
>
384,385c365,366
< {
< SootMethod m = new SootMethod(name, parameterTypes, returnType);
---
> {
> SootMethod m = new SootMethod(name, parameterTypes, returnType);
<<< text/plain; charset="us-ascii"; format=flowed: Unrecognized >>>