[#5116][#5106][#4616][#5135] improve(auth-ranger): The owner of catalog/metalake should have all the privileges of schemas/tables #5113

xunliu · 2024-10-11T13:56:22Z

What changes were proposed in this pull request?

The owner of catalog/metalake should have all the privileges of schemas/tables.

Why are the changes needed?

Fix:

[Improvement] Upper-layer metadata sets lower-layer privilege #5116
[Improvement] Filter Catalog securiable object in authorization Ranger #5106
[Subtask] Support select table privilege from catalog/metalake in the RangerAuthorization #4616
[Subtask] The owner of catalog/metalake should have all the privileges of schemas/tables #5135

Does this PR introduce any user-facing change?

N/A

How was this patch tested?

Add ITs.

...r/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationHivePlugin.java

...anger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java

…in the onOwnerSet

yuqi1129 · 2024-10-18T06:26:45Z

.github/workflows/access-control-integration-test.yml

-          ./gradlew -PskipTests -PtestMode=embedded -PjdbcBackend=h2 -PjdkVersion=${{ matrix.java-version }} -PskipDockerTests=false :authorizations:authorization-ranger:test --tests "org.apache.gravitino.authorization.ranger.integration.test.**"
-          ./gradlew -PskipTests -PtestMode=deploy -PjdbcBackend=mysql -PjdkVersion=${{ matrix.java-version }} -PskipDockerTests=false :authorizations:authorization-ranger:test --tests "org.apache.gravitino.authorization.ranger.integration.test.**"
-          ./gradlew -PskipTests -PtestMode=deploy -PjdbcBackend=postgresql -PjdkVersion=${{ matrix.java-version }} -PskipDockerTests=false :authorizations:authorization-ranger:test --tests "org.apache.gravitino.authorization.ranger.integration.test.**"
+          ./gradlew -PtestMode=embedded -PjdbcBackend=h2 -PjdkVersion=${{ matrix.java-version }} -PskipDockerTests=false :authorizations:authorization-ranger:test --tests "org.apache.gravitino.authorization.ranger.**"


Why do we remove parameter -PskipTests?

Because access-control-integration-test.yml needs to test all of the Authorization mode.

yuqi1129 · 2024-10-18T06:42:33Z

...anger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java

  @Override
  public void close() throws IOException {}
+
+  public boolean validAuthorizationOperation(List<SecurableObject> securableObjects) {


The package access level is sufficient.

yuqi1129 · 2024-10-18T06:48:02Z

...anger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java

+    return securableObjects.stream()
+        .noneMatch(
+            securableObject -> {
+              AtomicBoolean match = new AtomicBoolean(true);
+              securableObject.privileges().stream()
+                  .forEach(
+                      privilege -> {
+                        if (!allowPrivilegesRule().contains(privilege.name())) {
+                          LOG.error(
+                              "Authorization to ignore privilege({}) on metadata object({})!",
+                              privilege.name(),
+                              securableObject.fullName());
+                          match.set(false);
+                          return;
+                        }
+
+                        if (!privilege.canBindTo(securableObject.type())) {
+                          LOG.error(
+                              "The privilege({}) is not supported for the metadata object({})!",
+                              privilege.name(),
+                              securableObject.fullName());
+                          match.set(false);
+                        }
+                      });
+              return !match.get();
+            });


Suggested change

return securableObjects.stream()

.noneMatch(

securableObject -> {

AtomicBoolean match = new AtomicBoolean(true);

securableObject.privileges().stream()

.forEach(

privilege -> {

if (!allowPrivilegesRule().contains(privilege.name())) {

LOG.error(

"Authorization to ignore privilege({}) on metadata object({})!",

privilege.name(),

securableObject.fullName());

match.set(false);

return;

}

if (!privilege.canBindTo(securableObject.type())) {

LOG.error(

"The privilege({}) is not supported for the metadata object({})!",

privilege.name(),

securableObject.fullName());

match.set(false);

}

});

return !match.get();

});

return securableObjects.stream()

.allMatch(

securableObject -> {

AtomicBoolean match = new AtomicBoolean(true);

securableObject.privileges().stream()

.forEach(

privilege -> {

if (!allowPrivilegesRule().contains(privilege.name())) {

LOG.error(

"Authorization to ignore privilege({}) on metadata object({})!",

privilege.name(),

securableObject.fullName());

match.set(false);

return;

}

if (!privilege.canBindTo(securableObject.type())) {

LOG.error(

"The privilege({}) is not supported for the metadata object({})!",

privilege.name(),

securableObject.fullName());

match.set(false);

}

});

return match.get();

});

yuqi1129 · 2024-10-18T06:49:25Z

...anger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java

+                      privilege -> {
+                        if (!allowPrivilegesRule().contains(privilege.name())) {
+                          LOG.error(
+                              "Authorization to ignore privilege({}) on metadata object({})!",


Since you are going to ignore privilege, why do you use the error level log and return false here?

The input privilege is not in the list of allowed privileges, so it needs to be ignored, and the error log output.

If so, I suggest the information 'xxxx privileges are not allowed/appliable for securable object xxxx'

yuqi1129 · 2024-10-18T06:52:33Z

...anger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java

  /**
   * Create a new role in the Ranger. <br>
   * 1. Create a policy for metadata object. <br>
   * 2. Save role name in the Policy items. <br>
   */
  @Override
  public Boolean onRoleCreated(Role role) throws RuntimeException {
-    rangerHelper.createRangerRoleIfNotExists(role.name());
+    if (!validAuthorizationOperation(role.securableObjects())) {
+      return false;


So I wonder what the return value means for onRoleCreated? what does the true and false means here?

Tells the caller that the execution succeeded or failed

yuqi1129 · 2024-10-18T06:54:24Z

...thorization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerHelper.java

+                          })
+                      .collect(Collectors.toList());
+
+              if (matchPolicyItems.size() == 0) {


yuqi1129 · 2024-10-18T07:02:38Z

...ization-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerPrivileges.java

+    DROP("drop"),
+    ALTER("alter"),
+    INDEX("index"),
+    LOCK("lock"),


What's the meaning of lock privilege?

This is hive privilege.

yuqi1129 · 2024-10-18T07:07:16Z

...on-ranger/src/main/java/org/apache/gravitino/authorization/ranger/RangerSecurableObject.java

+ * you need.
+ */
+@Unstable
+public interface RangerSecurableObject extends MetadataObject {


I guess RangerSecurableObject is the corresponding representing object of SecurableObject in the Rander service, is that right? please add more to clarify it.

xunliu · 2024-10-19T14:05:57Z

hi @yuqi1129 Please help me review this PR again, Thanks.

xunliu requested a review from yuqi1129 October 11, 2024 13:56

xunliu self-assigned this Oct 11, 2024

xunliu force-pushed the issue-5106 branch from e4b5127 to 0c32387 Compare October 12, 2024 07:13

xunliu marked this pull request as draft October 12, 2024 07:23

xunliu force-pushed the issue-5106 branch 4 times, most recently from dcafa1d to b2cb8f2 Compare October 16, 2024 14:16

xunliu changed the title ~~[#5106] improve(auth-ranger): Filter Catalog securiable object in authorization Ranger~~ [#5116][#5106][#4616][#5135] improve(auth-ranger): The owner of catalog/metalake should have all the privileges of schemas/tables Oct 16, 2024

xunliu requested a review from jerqi October 16, 2024 14:29

xunliu marked this pull request as ready for review October 17, 2024 01:10

jerqi reviewed Oct 17, 2024

View reviewed changes

...r/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationHivePlugin.java Outdated Show resolved Hide resolved

jerqi reviewed Oct 17, 2024

View reviewed changes

...anger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java Show resolved Hide resolved

jerqi reviewed Oct 17, 2024

View reviewed changes

...anger/src/main/java/org/apache/gravitino/authorization/ranger/RangerAuthorizationPlugin.java Show resolved Hide resolved

xunliu added 4 commits October 18, 2024 11:44

[apache#5106] improve(auth-ranger): Filter Catalog securiable object …

d2ef4c5

…in the onOwnerSet

updateSecurableObject

8e52d93

USE_CATALOG,USE_SCHEMA

a48fbaa

rename variable

213f440

xunliu force-pushed the issue-5106 branch from 42f4d68 to 213f440 Compare October 18, 2024 03:44

yuqi1129 reviewed Oct 18, 2024

View reviewed changes

jerqi approved these changes Oct 18, 2024

View reviewed changes

xunliu added 2 commits October 18, 2024 23:37

improve code

5715376

fix CI

76be018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[#5116][#5106][#4616][#5135] improve(auth-ranger): The owner of catalog/metalake should have all the privileges of schemas/tables #5113

[#5116][#5106][#4616][#5135] improve(auth-ranger): The owner of catalog/metalake should have all the privileges of schemas/tables #5113

xunliu commented Oct 11, 2024 •

edited

Loading

yuqi1129 Oct 18, 2024

xunliu Oct 18, 2024

yuqi1129 Oct 18, 2024

xunliu Oct 18, 2024

yuqi1129 Oct 18, 2024

xunliu Oct 18, 2024

yuqi1129 Oct 18, 2024

xunliu Oct 18, 2024

yuqi1129 Oct 19, 2024

yuqi1129 Oct 18, 2024

xunliu Oct 18, 2024

yuqi1129 Oct 18, 2024

xunliu Oct 18, 2024

yuqi1129 Oct 18, 2024

xunliu Oct 18, 2024

yuqi1129 Oct 18, 2024

xunliu Oct 18, 2024

xunliu commented Oct 19, 2024

[#5116][#5106][#4616][#5135] improve(auth-ranger): The owner of catalog/metalake should have all the privileges of schemas/tables #5113

Are you sure you want to change the base?

[#5116][#5106][#4616][#5135] improve(auth-ranger): The owner of catalog/metalake should have all the privileges of schemas/tables #5113

Conversation

xunliu commented Oct 11, 2024 • edited Loading

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xunliu commented Oct 19, 2024

xunliu commented Oct 11, 2024 •

edited

Loading