Version 1.6.0

.github/workflows/main.yml

@@ -40,6 +40,7 @@ jobs:
     - name: Install PostgreSQL client
       run: |
         sudo apt-get -yqq install libpq-dev
+
     - name: Build app
       env:
         PGHOST: localhost
@@ -50,6 +51,7 @@ jobs:
         pushd spec/apps/dummy
         bundle exec rake db:create db:schema:load db:migrate
         popd
+
     - name: Run tests
       env:
         PGHOST: localhost

CHANGELOG.md

@@ -1,3 +1,15 @@
+# 1.6.0 (2023-09-25)
+
+Many thanks to `@xjunior`, who contributed a series of improvements and fixes back-ported into this version. New features:
+
+* Allow writable complex types in custom extensions via [#61](https://github.com/RIPAGlobal/scimitar/pull/61)
+* Allow complex queries via table joins via [#62](https://github.com/RIPAGlobal/scimitar/pull/62)
+
+Fixes:
+
+* Much better error message raised if `PatchOp` misses operations in [#65](https://github.com/RIPAGlobal/scimitar/pull/65)
+* Combined logical groups generate working queries with [#66](https://github.com/RIPAGlobal/scimitar/pull/66)
+
 # 1.5.3 (2023-09-16)
 
 * Fix warning messages for Rails 6 and Zeitwerk. Thanks to `@sobrinho` for the contribution.

Gemfile.lock

@@ -9,7 +9,7 @@ GIT
 PATH
   remote: .
   specs:
-    scimitar (1.5.3)
+    scimitar (1.6.0)
       rails (~> 6.0)
 
 GEM

README.md

@@ -6,6 +6,8 @@
 
 A SCIM v2 API endpoint implementation for Ruby On Rails.
 
+For a list of changes and information on major version upgrades, please see `CHANGELOG.md`.
+
 
 
 ## Overview
@@ -77,6 +79,16 @@ Scimitar.engine_configuration = Scimitar::EngineConfiguration.new({
 
 When it comes to token access, Scimitar neither enforces nor presumes any kind of encoding for bearer tokens. You can use anything you like, including encoding/encrypting JWTs if you so wish - https://rubygems.org/gems/jwt may be useful. The way in which a client might integrate with your SCIM service varies by client and you will have to check documentation to see how a token gets conveyed to that client in the first place (e.g. a full OAuth flow with your application, or just a static token generated in some UI which an administrator copies and pastes into their client's SCIM configuration UI).
 
+**Important:** Under some more recent versions of Rails 6, you may need to wrap any Scimitar configuration with `Rails.application.config.to_prepare do...` to avoid `NameError: uninitialized constant...` exceptions arising due to autoloader problems:
+
+```ruby
+Rails.application.config.to_prepare do
+  Scimitar.engine_configuration = Scimitar::EngineConfiguration.new({
+    # ...
+  end
+end
+```
+
 ### Routes
 
 For each resource you support, add these lines to your `routes.rb`:
@@ -185,11 +197,20 @@ class User < ActiveRecord::Base
     return nil
   end
 
+  # The attributes in this example include a reference to the same hypothesised
+  # 'Group' model as in the HABTM relationship above. In this case, in order to
+  # filter by "groups" or "groups.value", the 'column' entry must reference the
+  # Group model's ID column as an AREL attribute as shown below, and the SCIM
+  # controller's #storage_scope implementation must also introduce a #join with
+  # ':groups' - see the "Queries & Optimisations" section below.
+  #
   def self.scim_queryable_attributes
     return {
-      givenName:  :first_name,
-      familyName: :last_name,
-      emails:     :work_email_address,
+      givenName:        { column: :first_name },
+      familyName:       { column: :last_name },
+      emails:           { column: :work_email_address },
+      groups:           { column: Group.arel_table[:id] },
+      "groups.value" => { column: Group.arel_table[:id] },
     }
   end
 
@@ -211,6 +232,8 @@ end
 
 ### Controllers
 
+#### ActiveRecord
+
 If you use ActiveRecord, your controllers can potentially be extremely simple by subclassing [`Scimitar::ActiveRecordBackedResourcesController`](https://www.rubydoc.info/gems/scimitar/Scimitar/ActiveRecordBackedResourcesController) - at a minimum:
 
 ```ruby
@@ -235,6 +258,26 @@ end
 
 All data-layer actions are taken via `#find` or `#save!`, with exceptions such as `ActiveRecord::RecordNotFound`, `ActiveRecord::RecordInvalid` or generalised SCIM exceptions handled by various superclasses. For a real Rails example of this, see the [test suite's controllers](https://github.com/RIPAGlobal/scimitar/tree/main/spec/apps/dummy/app/controllers) which are invoked via its [routing declarations](https://github.com/RIPAGlobal/scimitar/blob/main/spec/apps/dummy/config/routes.rb).
 
+#### Queries & Optimisations
+
+The scope can be optimised to eager load the data exposed by the SCIM interface, i.e.:
+
+```ruby
+def storage_scope
+  User.eager_load(:groups)
+end
+```
+
+In cases where you have references to related columns in your `scim_queryable_attributes`, your `storage_scope` must join the relation:
+
+```ruby
+def storage_scope
+  User.left_join(:groups)
+end
+```
+
+#### Other source types
+
 If you do _not_ use ActiveRecord to store data, or if you have very esoteric read-write requirements, you can subclass [`Scimigar::ResourcesController`](https://www.rubydoc.info/gems/scimitar/Scimitar/ResourcesController) in a manner similar to this:
 
 ```ruby
@@ -446,7 +489,7 @@ Whatever you provide in the `::id` method in your extension class will be used a
 ```json
 {
   "schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
-  "Operations":[
+  "Operations": [
     {
       "op": "replace",
       "path": "urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:organization",
@@ -458,7 +501,41 @@ Whatever you provide in the `::id` method in your extension class will be used a
 
 Resource extensions can provide any fields you choose, under any ID/URN you choose, to either RFC-described resources or entirely custom SCIM resources. There are no hard-coded assumptions or other "magic" that might require you to only extend RFC-described resources with RFC-described extensions. Of course, if you use custom resources or custom extensions that are not described by the SCIM RFCs, then the SCIM API you provide may only work with custom-written API callers that are aware of your bespoke resources and/or extensions.
 
+Extensions can also contain complex attributes such as groups. For instance, if you want the ability to write to groups from the User resource perspective (since 'groups' collection in a SCIM User resource is read-only), you can add one attribute to your extension like this:
 
+```ruby
+Scimitar::Schema::Attribute.new(name: "userGroups", multiValued: true, complexType: Scimitar::ComplexTypes::ReferenceGroup, mutability: "writeOnly"),
+```
+
+Then map it in your `scim_attributes_map`:
+
+```ruby
+  userGroups: [
+    {
+      list: :groups,
+      find_with: ->(value) { Group.find(value["value"]) },
+      using: {
+        value:   :id,
+        display: :name
+      }
+    }
+  ]
+```
+
+And write to it like this:
+
+```json
+{
+  "schemas": ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
+  "Operations": [
+    {
+      "op": "replace",
+      "path": "urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:userGroups",
+      "value": [{ "value": "1" }]
+    }
+  ]
+}
+```
 
 ## Security
 

app/controllers/scimitar/application_controller.rb

@@ -25,7 +25,6 @@ class ApplicationController < ActionController::Base
       #
       # ...to "globally" invoke this handler if you wish.
       #
-      #
       # +exception+:: Exception instance, used for a configured error reporter
       #               via #handle_scim_error (if present).
       #

app/models/scimitar/lists/query_parser.rb

@@ -192,7 +192,7 @@ def parse_expr
 
             ast.push(self.start_group? ? self.parse_group() : self.pop())
 
-            unless ! ast.last.is_a?(String) || UNARY_OPERATORS.include?(ast.last.downcase)
+            if ast.last.is_a?(String) && !UNARY_OPERATORS.include?(ast.last.downcase) || ast.last.is_a?(Array)
               expect_op ^= true
             end
           end
@@ -601,9 +601,15 @@ def apply_scim_filter(
           column_names   = self.activerecord_columns(scim_attribute)
           value          = self.activerecord_parameter(scim_parameter)
           value_for_like = self.sql_modified_value(scim_operator, value)
-          all_supported  = column_names.all? { | column_name | base_scope.model.column_names.include?(column_name.to_s) }
+          arel_columns   = column_names.map do |column|
+            if base_scope.model.column_names.include?(column.to_s)
+              arel_table[column]
+            elsif column.is_a?(Arel::Attribute)
+              column
+            end
+          end
 
-          raise Scimitar::FilterError unless all_supported
+          raise Scimitar::FilterError unless arel_columns.all?
 
           unless case_sensitive
             lc_scim_attribute = scim_attribute.downcase()
@@ -615,8 +621,7 @@ def apply_scim_filter(
             )
           end
 
-          column_names.each.with_index do | column_name, index |
-            arel_column    = arel_table[column_name]
+          arel_columns.each.with_index do | arel_column, index |
             arel_operation = case scim_operator
               when 'eq'
                 if case_sensitive
@@ -641,7 +646,7 @@ def apply_scim_filter(
               when 'co', 'sw', 'ew'
                 arel_column.matches(value_for_like, nil, case_sensitive)
               when 'pr'
-                arel_table.grouping(arel_column.not_eq_all(['', nil]))
+                arel_column.relation.grouping(arel_column.not_eq_all(['', nil]))
               else
                 raise Scimitar::FilterError.new("Unsupported operator: '#{scim_operator}'")
             end

app/models/scimitar/resources/base.rb

@@ -112,7 +112,7 @@ def self.find_attribute(*path)
       end
 
       def self.complex_scim_attributes
-        schema.scim_attributes.select(&:complexType).group_by(&:name)
+        schemas.flat_map(&:scim_attributes).select(&:complexType).group_by(&:name)
       end
 
       def complex_type_from_hash(scim_attribute, attr_value)

app/models/scimitar/resources/mixin.rb

@@ -220,24 +220,36 @@ module Resources
     # allow for different client searching "styles", given ambiguities in RFC
     # 7644 filter examples).
     #
-    # Each value is a Hash with Symbol keys ':column', naming just one simple
-    # column for a mapping; ':columns', with an Array of column names that you
-    # want to map using 'OR' for a single search on the corresponding SCIM
-    # attribute; or ':ignore' with value 'true', which means that a fitler on
-    # the matching attribute is ignored rather than resulting in an "invalid
-    # filter" exception - beware possibilities for surprised clients getting a
-    # broader result set than expected. Example:
+    # Each value is a hash of queryable SCIM attribute options, described
+    # below - for example:
     #
     #     def self.scim_queryable_attributes
     #       return {
     #         'name.givenName'  => { column: :first_name },
     #         'name.familyName' => { column: :last_name  },
     #         'emails'          => { columns: [ :work_email_address, :home_email_address ] },
     #         'emails.value'    => { columns: [ :work_email_address, :home_email_address ] },
-    #         'emails.type'     => { ignore: true }
+    #         'emails.type'     => { ignore: true },
+    #         'groups.value'    => { column: Group.arel_table[:id] }
     #       }
     #     end
     #
+    # Column references can be either a Symbol representing a column within
+    # the resource model table, or an <tt>Arel::Attribute</tt> instance via
+    # e.g. <tt>MyModel.arel_table[:my_column]</tt>.
+    #
+    # === Queryable SCIM attribute options
+    #
+    # +:column+::  Just one simple column for a mapping.
+    #
+    # +:columns+:: An Array of columns that you want to map using 'OR' for a
+    #              single search of the corresponding entity.
+    #
+    # +:ignore+::  When set to +true+, the matching attribute is ignored rather
+    #              than resulting in an "invalid filter" exception. Beware
+    #              possibilities for surprised clients getting a broader result
+    #              set than expected, since a constraint may have been ignored.
+    #
     # Filtering is currently limited and searching within e.g. arrays of data
     # is not supported; only simple top-level keys can be mapped.
     #
@@ -406,8 +418,11 @@ def from_scim!(scim_hash:)
         def from_scim_patch!(patch_hash:)
           frozen_ci_patch_hash = patch_hash.with_indifferent_case_insensitive_access().freeze()
           ci_scim_hash         = self.to_scim(location: '(unused)').as_json().with_indifferent_case_insensitive_access()
+          operations           = frozen_ci_patch_hash['operations']
+
+          raise Scimitar::InvalidSyntaxError.new("Missing PATCH \"operations\"") unless operations
 
-          frozen_ci_patch_hash['operations'].each do |operation|
+          operations.each do |operation|
             nature   = operation['op'   ]&.downcase
             path_str = operation['path' ]
             value    = operation['value']

lib/scimitar/version.rb

@@ -3,11 +3,11 @@ module Scimitar
   # Gem version. If this changes, be sure to re-run "bundle install" or
   # "bundle update".
   #
-  VERSION = '1.5.3'
+  VERSION = '1.6.0'
 
   # Date for VERSION. If this changes, be sure to re-run "bundle install"
   # or "bundle update".
   #
-  DATE = '2023-09-16'
+  DATE = '2023-09-25'
 
 end

spec/apps/dummy/app/models/mock_user.rb

@@ -15,6 +15,7 @@ class MockUser < ActiveRecord::Base
     work_phone_number
     organization
     department
+    mock_groups
   }
 
   has_and_belongs_to_many :mock_groups
@@ -90,7 +91,17 @@ def self.scim_attributes_map
       # "spec/apps/dummy/config/initializers/scimitar.rb".
       #
       organization: :organization,
-      department:   :department
+      department:   :department,
+      userGroups: [
+        {
+          list:      :mock_groups,
+          find_with: ->(value) { MockGroup.find(value["value"]) },
+          using: {
+            value:   :id,
+            display: :display_name
+          }
+        }
+      ]
     }
   end
 
@@ -105,6 +116,8 @@ def self.scim_queryable_attributes
       'meta.lastModified' => { column: :updated_at },
       'name.givenName'    => { column: :first_name },
       'name.familyName'   => { column: :last_name  },
+      'groups'            => { column: MockGroup.arel_table[:id] },
+      'groups.value'      => { column: MockGroup.arel_table[:id] },
       'emails'            => { columns: [ :work_email_address, :home_email_address ] },
       'emails.value'      => { columns: [ :work_email_address, :home_email_address ] },
       'emails.type'       => { ignore: true } # We can't filter on that; it'll just search all e-mails

spec/apps/dummy/config/routes.rb

@@ -6,12 +6,12 @@
 Rails.application.routes.draw do
   mount Scimitar::Engine, at: '/'
 
-  get    'Users',     to: 'mock_users#index'
-  get    'Users/:id', to: 'mock_users#show'
-  post   'Users',     to: 'mock_users#create'
-  put    'Users/:id', to: 'mock_users#replace'
-  patch  'Users/:id', to: 'mock_users#update'
-  delete 'Users/:id', to: 'mock_users#destroy'
+  get    'Users',      to: 'mock_users#index'
+  get    'Users/:id',  to: 'mock_users#show'
+  post   'Users',      to: 'mock_users#create'
+  put    'Users/:id',  to: 'mock_users#replace'
+  patch  'Users/:id',  to: 'mock_users#update'
+  delete 'Users/:id',  to: 'mock_users#destroy'
 
   get    'Groups',     to: 'mock_groups#index'
   get    'Groups/:id', to: 'mock_groups#show'