Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Classes needed for lambda merging in ProGuard #36

Open
wants to merge 40 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
ca6936c
Prevent DominatorCalculator from crashing when a method has an empty …
Masrepus Dec 21, 2021
f0ed6b2
Fix CallResolver to not use unimplemented interface methods as viable…
Masrepus Dec 21, 2021
7f1f0af
Create getter for target method in ConcreteCall
capoz Dec 21, 2021
b626bfc
Remove error logging for abstract call targets
Masrepus Dec 21, 2021
e92a27b
Add ClassMethodFilter from Nadeesh
Dec 28, 2021
2acc29c
Move PackageGrouper to ProGuard-CORE
Jan 5, 2022
e61c38d
Move MethodCopier to ProGuard-CORE
Jan 5, 2022
e39c05e
Create separate & extra tests for PackageGrouper
Jan 5, 2022
5f24388
Make MethodCopier public to be used in KotlinLambdaClassMerger
Jan 5, 2022
3338829
Add Kotlin Unit name, type, INSTANCE field to ClassConstants
Jan 6, 2022
4f74d43
Visitor that visits inner and/or outer class constants of InnerClassInfo
Jan 6, 2022
0d9cb78
Visitor that visits referenced class of ClassConstant
Jan 6, 2022
8c81564
Merge remote-tracking branch 'Guardsquare/master'
Jan 7, 2022
ea25197
ModifiedAllInnerClassesInfoVisitor: revisit inner classes attribute w…
Jan 9, 2022
fc68b07
Merge branch 'Guardsquare:master' into master
heckej Feb 8, 2022
cd4a6a3
Merge branch 'Guardsquare:master' into master
heckej Feb 18, 2022
5550c96
Merge branch 'Guardsquare:master' into master
heckej Feb 24, 2022
9860f9e
Merge branch 'Guardsquare:master' into master
heckej Mar 2, 2022
fcfa75e
New classes: FieldCopier & FieldRenamer
Mar 22, 2022
a33e5da
Use a FieldCopier in the MethodCopier
Mar 22, 2022
bf8155b
FieldRenamer: use descriptor + index as new name
Mar 27, 2022
b2877ea
Don't print warning in FieldCopier
Mar 27, 2022
9d3bb0c
Merge remote-tracking branch 'Guardsquare/master'
Mar 27, 2022
8c82796
ClassConstantToClassVisitor: don't visit referenced class if it is null
May 7, 2022
758be93
PackageGrouper: pretty print package names in log
May 7, 2022
528e7a7
Merge remote-tracking branch 'Guardsquare/master'
May 7, 2022
7821839
Test added for MethodCopier
Jun 6, 2022
b542b9b
Apply ktlint formatting to new test classes
Jun 6, 2022
99065f7
Merge branch 'Guardsquare:master' into master
heckej Jun 6, 2022
4ef0e59
Inner classes of lambda merging moved to ProGuard CORE
Jun 8, 2022
67cde7f
New ConstantVisitor: FieldReferenceFinder
Jun 8, 2022
72a80b9
Move Kotlin specific constants from ClassConstants to KotlinConstants
Jun 29, 2022
7eeff48
Add PGC header to ClassConstantToClassVisitor
Jun 29, 2022
02e8fd0
Open '{' on new line in ClassConstantToClassVisitor
Jun 29, 2022
660a391
Remove redundant ClassMethodFilter
Jun 29, 2022
59f7999
Move visitors to ProGuard
Jun 29, 2022
7d34770
Move ModifiedAllInnerClassesInfoVisitor from PGC to ProGuard
Jun 29, 2022
7441f81
Move InnerClassRemover to proguard.classfile.editor + update docs
Jun 29, 2022
082b3a8
Move FieldCopier to ProGuard
Jun 29, 2022
31f231d
MethodCopierTest moved to ProGuard
Jun 29, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/main/java/proguard/classfile/ClassConstants.java
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,10 @@ public class ClassConstants
public static final String NAME_ANDROID_WEBKIT_WEB_VIEW = "android/webkit/WebView";
public static final String NAME_ANDROID_SUPPORT_V4_APP_FRAGMENT = "android/support/v4/app/Fragment";

public static final String NAME_KOTLIN_UNIT = "kotlin/Unit";
public static final String TYPE_KOTLIN_UNIT = "Lkotlin/Unit;";
public static final String FIELD_NAME_KOTLIN_UNIT_INSTANCE = "INSTANCE";
heckej marked this conversation as resolved.
Show resolved Hide resolved

public static final String NAME_JAVA_UTIL_CONCURRENT_ATOMIC_ATOMIC_INTEGER_FIELD_UPDATER = "java/util/concurrent/atomic/AtomicIntegerFieldUpdater";
public static final String NAME_JAVA_UTIL_CONCURRENT_ATOMIC_ATOMIC_LONG_FIELD_UPDATER = "java/util/concurrent/atomic/AtomicLongFieldUpdater";
public static final String NAME_JAVA_UTIL_CONCURRENT_ATOMIC_ATOMIC_REFERENCE_FIELD_UPDATER = "java/util/concurrent/atomic/AtomicReferenceFieldUpdater";
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
package proguard.classfile.attribute.visitor;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add the ProGuardCORE header to all the classes?


import proguard.classfile.Clazz;
import proguard.classfile.constant.ClassConstant;
import proguard.classfile.constant.Constant;
import proguard.classfile.constant.visitor.ConstantVisitor;
import proguard.classfile.visitor.ClassVisitor;

public class ClassConstantToClassVisitor implements ConstantVisitor {

private final ClassVisitor classVisitor;

public ClassConstantToClassVisitor(ClassVisitor classVisitor)
{
this.classVisitor = classVisitor;
}

public void visitAnyConstant(Clazz clazz, Constant constant) {}

@Override
public void visitClassConstant(Clazz clazz, ClassConstant classConstant) {
heckej marked this conversation as resolved.
Show resolved Hide resolved
if (this.classVisitor != null && classConstant.referencedClass != null)
{
classConstant.referencedClass.accept(this.classVisitor);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
package proguard.classfile.attribute.visitor;

import proguard.classfile.Clazz;
import proguard.classfile.ProgramClass;
import proguard.classfile.attribute.InnerClassesInfo;
import proguard.classfile.constant.visitor.ConstantVisitor;

public class InnerClassInfoClassConstantVisitor implements InnerClassesInfoVisitor {
heckej marked this conversation as resolved.
Show resolved Hide resolved

private final ConstantVisitor innerClassConstantVisitor;
private final ConstantVisitor outerClassConstantVisitor;

public InnerClassInfoClassConstantVisitor(ConstantVisitor innerClassConstantVisitor, ConstantVisitor outerClassConstantVisitor)
{
this.innerClassConstantVisitor = innerClassConstantVisitor;
this.outerClassConstantVisitor = outerClassConstantVisitor;
}

@Override
public void visitInnerClassesInfo(Clazz clazz, InnerClassesInfo innerClassesInfo) {
if (this.innerClassConstantVisitor != null)
{
innerClassesInfo.innerClassConstantAccept(clazz, this.innerClassConstantVisitor);
}
if (this.outerClassConstantVisitor != null)
{
innerClassesInfo.outerClassConstantAccept(clazz, this.outerClassConstantVisitor);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
package proguard.classfile.attribute.visitor;

import proguard.classfile.Clazz;
import proguard.classfile.attribute.InnerClassesAttribute;

/**
* This {@link AllInnerClassesInfoVisitor} revisits each {@link InnerClassesAttribute} everytime its amount of
* referenced classes has been modified in the meantime.
*/
public class ModifiedAllInnerClassesInfoVisitor extends AllInnerClassesInfoVisitor {
heckej marked this conversation as resolved.
Show resolved Hide resolved

public ModifiedAllInnerClassesInfoVisitor(InnerClassesInfoVisitor innerClassesInfoVisitor) {
super(innerClassesInfoVisitor);
}

public void visitInnerClassesAttribute(Clazz clazz, InnerClassesAttribute innerClassesAttribute)
{
int originalClassesCount = -1;
while (originalClassesCount != innerClassesAttribute.u2classesCount)
{
originalClassesCount = innerClassesAttribute.u2classesCount;
super.visitInnerClassesAttribute(clazz, innerClassesAttribute);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
package proguard.classfile.visitor;

import proguard.classfile.Clazz;
import proguard.classfile.constant.ClassConstant;
import proguard.classfile.constant.Constant;
import proguard.classfile.constant.visitor.ConstantVisitor;

public class ClassConstantReferenceUpdater implements ConstantVisitor
heckej marked this conversation as resolved.
Show resolved Hide resolved
{
private final Clazz originalClass;
private final Clazz replacingClass;
public ClassConstantReferenceUpdater(Clazz originalClass, Clazz replacingClass)
{
this.originalClass = originalClass;
this.replacingClass = replacingClass;
}

@Override
public void visitAnyConstant(Clazz clazz, Constant constant) {}

@Override
public void visitClassConstant(Clazz clazz, ClassConstant classConstant) {
if (classConstant.referencedClass == originalClass)
{
classConstant.referencedClass = replacingClass;
}
}
}
52 changes: 52 additions & 0 deletions src/main/java/proguard/classfile/visitor/ClassMethodFilter.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
/*
* ProGuard -- shrinking, optimization, obfuscation, and preverification
* of Java bytecode.
*
* Copyright (c) 2002-2021 Guardsquare NV
*/

package proguard.classfile.visitor;

import proguard.classfile.*;

/**
* This {@link ClassVisitor} delegates its visits to given
* acceptVisitor when the visited class has method with
* given name and descriptor and delegates to rejectVisitor when
* none of the class methods match.
*/
public class ClassMethodFilter
implements ClassVisitor
{
private final String methodName;
private final String descriptor;
private final ClassVisitor acceptVisitor;
private final ClassVisitor rejectVisitor;

public ClassMethodFilter(String methodName,
String descriptor,
ClassVisitor acceptVisitor,
ClassVisitor rejectVisitor)
{
this.methodName = methodName;
this.descriptor = descriptor;
this.acceptVisitor = acceptVisitor;
this.rejectVisitor = rejectVisitor;
}

@Override
public void visitAnyClass(Clazz clazz)
{
ClassVisitor delegateVisitor = delegateVisitor(clazz);
if (delegateVisitor != null)
{
delegateVisitor.visitAnyClass(clazz);
heckej marked this conversation as resolved.
Show resolved Hide resolved
}
}

private ClassVisitor delegateVisitor(Clazz clazz)
{
Method method = clazz.findMethod(methodName, descriptor);
return method != null ? acceptVisitor : rejectVisitor;
}
}
heckej marked this conversation as resolved.
Show resolved Hide resolved
32 changes: 32 additions & 0 deletions src/main/java/proguard/classfile/visitor/ClassReferenceFinder.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package proguard.classfile.visitor;

import proguard.classfile.Clazz;
import proguard.classfile.constant.ClassConstant;
import proguard.classfile.constant.Constant;
import proguard.classfile.constant.visitor.ConstantVisitor;

public class ClassReferenceFinder implements ConstantVisitor
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be replacable by the following Filter class which can delegate to another ClassVisitor if the the class is referenced.

It's extracted from proguard.backport.StaticInterfaceMethodConverter.MyReferencedClassFilter in ProGuard.

Usable like the following:

        this.programClassPool.classesAccept(
                new ClassNameFilter(regularExpression,
                new ReferencedClassFilter(
                    lambdaClass,
                    enclosingMethodUpdater)));

But I see that currentLambdaClass gets updated, so might not work. Can also be used with a ClassCounter:

ClassCounter counter = new ClassCounter();
programClass.accept(new ReferencedClassFilter(currentLambdaClass, counter))
package proguard.classfile.visitor;

import proguard.classfile.*;
import proguard.classfile.constant.*;
import proguard.classfile.constant.visitor.ConstantVisitor;
import proguard.classfile.visitor.ClassVisitor;

/**
 * This ClassVisitor delegates its visits to classes that
 * reference a given class via any RefConstant.
 */
public class ReferencedClassFilter
    implements ClassVisitor,
               ConstantVisitor
{
    private final Clazz        referencedClass;
    private final ClassVisitor classVisitor;

    private boolean referenceClassFound;


    public ReferencedClassFilter(Clazz referencedClass,
                                 ClassVisitor classVisitor)
    {
        this.referencedClass = referencedClass;
        this.classVisitor = classVisitor;
    }


    // Implementations for ClassVisitor.


    @Override
    public void visitAnyClass(Clazz clazz) {}


    @Override
    public void visitProgramClass(ProgramClass programClass)
    {
        referenceClassFound = false;
        programClass.constantPoolEntriesAccept(this);

        if (referenceClassFound)
        {
            programClass.accept(classVisitor);
        }
    }


    // Implementations for ConstantVisitor.


    public void visitAnyConstant(Clazz clazz, Constant constant) {}


    public void visitAnyRefConstant(Clazz clazz, RefConstant refConstant)
    {
        if (refConstant.referencedClass == referencedClass)
        {
            referenceClassFound = true;
        }
    }
}

{
private final Clazz referencedClass;
private boolean classIsReferenced = false;

public ClassReferenceFinder(Clazz referencedClass)
{
this.referencedClass = referencedClass;
}

public void visitAnyConstant(Clazz clazz, Constant constant) {}

public void visitClassConstant(Clazz clazz, ClassConstant classConstant)
{
if (classConstant.referencedClass != null && classConstant.referencedClass.equals(referencedClass))
{
this.classIsReferenced = true;
}
}

public boolean classReferenceFound()
{
return this.classIsReferenced;
}
}
29 changes: 29 additions & 0 deletions src/main/java/proguard/classfile/visitor/CodeSizeCounter.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
package proguard.classfile.visitor;

import proguard.classfile.Clazz;
import proguard.classfile.Method;
import proguard.classfile.attribute.Attribute;
import proguard.classfile.attribute.CodeAttribute;
import proguard.classfile.attribute.visitor.AttributeVisitor;

public class CodeSizeCounter
implements AttributeVisitor
{
private int totalCodeSize = 0;

// Implementations for AttributeVisitor

@Override
public void visitAnyAttribute(Clazz clazz, Attribute attribute) {}

@Override
public void visitCodeAttribute(Clazz clazz, Method method, CodeAttribute codeAttribute)
{
totalCodeSize += codeAttribute.u4codeLength;
}

public int getCount()
{
return totalCodeSize;
}
}
Comment on lines +9 to +29
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public class CodeSizeCounter
implements AttributeVisitor
{
private int totalCodeSize = 0;
// Implementations for AttributeVisitor
@Override
public void visitAnyAttribute(Clazz clazz, Attribute attribute) {}
@Override
public void visitCodeAttribute(Clazz clazz, Method method, CodeAttribute codeAttribute)
{
totalCodeSize += codeAttribute.u4codeLength;
}
public int getCount()
{
return totalCodeSize;
}
}
public class CodeAttributeSizeCounter
implements Counter,
AttributeVisitor
{
private int size = 0;
// Implementations for Counter.
public int getCount()
{
return size;
}
// Implementations for AttributeVisitor.
public void visitAnyAttribute(Clazz clazz, Attribute attribute) {}
public void visitCodeAttribute(Clazz clazz, Method method, CodeAttribute codeAttribute)
{
size += codeAttribute.u4codeLength;
}
}

We have a class like this already but it's not in ProGuardCORE at the moment. There's already the Counter interface to use with these counting visitors.

Can you move this also into the attribute/visitor package, since it's an AttributeVisitor?

Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
package proguard.classfile.visitor;

import proguard.classfile.Clazz;
import proguard.classfile.constant.Constant;
import proguard.classfile.constant.Utf8Constant;
import proguard.classfile.constant.visitor.ConstantVisitor;

public class DescriptorTypeUpdater implements ConstantVisitor
{
private final String originalType;
private final String replacingType;
public DescriptorTypeUpdater(String originalType, String replacingType)
{
this.originalType = originalType;
this.replacingType = replacingType;
}

@Override
public void visitAnyConstant(Clazz clazz, Constant constant) {}

@Override
public void visitUtf8Constant(Clazz clazz, Utf8Constant utf8Constant) {
utf8Constant.setString(utf8Constant.getString().replace(originalType, replacingType));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we should be using a ConstantPoolEditor to add a new constant. You can use a ConstantPoolShrinker after to remove unused ones.

        programMember.u2nameIndex =
            new ConstantPoolEditor(programClass).addUtf8Constant(newName);

}
}
78 changes: 78 additions & 0 deletions src/main/java/proguard/classfile/visitor/FieldCopier.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
package proguard.classfile.visitor;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import proguard.classfile.*;
import proguard.classfile.constant.FieldrefConstant;
import proguard.classfile.constant.visitor.ConstantVisitor;
import proguard.classfile.editor.ClassBuilder;
import proguard.classfile.editor.ClassEditor;

public class FieldCopier implements MemberVisitor, ConstantVisitor {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one seems quite specific to the lambda merging, can you move it to ProGuard?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should all visitors copied to proguard.classfile.visitor or would you like them to be part of the package that contains the lambda merging code?

Copy link
Collaborator

@mrjameshamilton mrjameshamilton Jun 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now, most of them (like this one) seem to belong in the lambda package since they aren't ready for general use.


private final ClassBuilder classBuilder;
private final ClassEditor classEditor;
private final FieldRenamer fieldRenamer;
private ProgramField lastCopiedField;
private boolean hasCopiedField = false;
private static final Logger logger = LogManager.getLogger(FieldCopier.class);

public FieldCopier(ClassBuilder builder, FieldRenamer renamer)
{
this.classBuilder = builder;
this.classEditor = new ClassEditor(builder.getProgramClass());
this.fieldRenamer = renamer;
}

@Override
public void visitProgramField(ProgramClass programClass, ProgramField programField)
{
String fieldName = programField.getName(programClass);
if (this.fieldRenamer != null)
{
fieldName = fieldRenamer.getNextFieldName();
}

String fieldDescriptor = programField.getDescriptor(programClass);
Field oldField = classBuilder.getProgramClass().findField(fieldName, null);
Field oldFieldSameDescriptor = classBuilder.getProgramClass().findField(fieldName, fieldDescriptor);
if (oldField != null && oldFieldSameDescriptor == null)
{
String oldFieldDescriptor = oldField.getDescriptor(classBuilder.getProgramClass());
//logger.warn("Field " + fieldName + " already exists in class " + classBuilder.getProgramClass() + " with different descriptor: " + oldFieldDescriptor + " <-> " + fieldDescriptor + ". The field will be duplicated with different descriptors.");
// Merge the field types: generalise to a common super type
//fieldDescriptor = ClassConstants.TYPE_JAVA_LANG_OBJECT;
}
else if (oldFieldSameDescriptor != null)
{
this.classEditor.removeField(oldFieldSameDescriptor);
}
ProgramField copiedField = classBuilder.addAndReturnField(programField.u2accessFlags, fieldName, fieldDescriptor);
if (this.fieldRenamer != null)
{
this.fieldRenamer.visitProgramField(classBuilder.getProgramClass(), copiedField);
}
this.lastCopiedField = copiedField;
this.hasCopiedField = true;
}

@Override
public void visitFieldrefConstant(Clazz clazz, FieldrefConstant fieldrefConstant) {
fieldrefConstant.referencedFieldAccept(this);
}

public ProgramField getLastCopiedField()
{
return this.lastCopiedField;
}

public boolean hasCopiedField()
{
return this.hasCopiedField;
}

public void reset()
{
this.hasCopiedField = false;
}
}
46 changes: 46 additions & 0 deletions src/main/java/proguard/classfile/visitor/FieldReferenceFinder.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
package proguard.classfile.visitor;

import proguard.classfile.Clazz;
import proguard.classfile.constant.Constant;
import proguard.classfile.constant.FieldrefConstant;
import proguard.classfile.constant.visitor.ConstantVisitor;
import proguard.classfile.util.ClassUtil;
import proguard.util.ListParser;
import proguard.util.NameParser;
import proguard.util.StringMatcher;

public class FieldReferenceFinder implements ConstantVisitor {
heckej marked this conversation as resolved.
Show resolved Hide resolved

private final Clazz referencedClass;
private final StringMatcher regularExpressionMatcher;
private final ConstantVisitor constantVisitor;
private boolean fieldReferenceFound = false;

public FieldReferenceFinder(Clazz referencedClass,
String fieldNameRegularExpression,
ConstantVisitor constantVisitor)
{
this.referencedClass = referencedClass;
this.regularExpressionMatcher = new ListParser(new NameParser(null)).parse(fieldNameRegularExpression);
this.constantVisitor = constantVisitor;
}

@Override
public void visitAnyConstant(Clazz clazz, Constant constant) {}

@Override
public void visitFieldrefConstant(Clazz clazz, FieldrefConstant fieldrefConstant)
{
if (fieldrefConstant.referencedClass == referencedClass
&& this.regularExpressionMatcher.matches(fieldrefConstant.getName(clazz)))
{
this.fieldReferenceFound = true;
fieldrefConstant.accept(clazz, this.constantVisitor);
}
}

public boolean isFieldReferenceFound()
{
return this.fieldReferenceFound;
}
}
Loading