"Cannot register notification blocks from within write transactions." when making changes in quick succession

[bogdan-pechounov]

How frequently does the bug occur?

Always

Description

I have a List of TextField. When I add items and edit their names quickly, I get a "Cannot register notification blocks from within write transactions" error.

The error doesn't happen if I only update the item name while typing fast. If I press the plus button while typing fast, the error sometimes occurs.

Stacktrace & log output

2023-08-07 21:24:05.203359-0400 TestFocus[28018:1317372] *** Terminating app due to uncaught exception 'RLMException', reason: 'Cannot register notification blocks from within write transactions.'
*** First throw call stack:
(0x1ce0c948c 0x1c7421050 0x10128395c 0x101104638 0x1012ff7fc 0x10131a6b4 0x10131a7f0 0x10140ce2c 0x10140d480 0x1d17d363c 0x1d17b2364 0x1d17be934 0x1d17fb804 0x1d17bb7ac 0x1d1cb0380 0x1efafcef8 0x1efafc5c0 0x1efafb478 0x1d178cf04 0x1d23f2ffc 0x1d17b7ba0 0x1d1794514 0x1d18a7754 0x1d179502c 0x1d6044edc 0x1d6035ca8 0x10131ddf4 0x10131dd10 0x10108a3e0 0x10108a24c 0x1016d96e8 0x1016d9638 0x1016cc900 0x1016cc828 0x1016cd440 0x101719db0 0x101719a1c 0x1016f9d64 0x101822860 0x101823df4 0x101284340 0x10128430c 0x1013d9268 0x1013d90bc 0x101408a2c 0x10141d8c4 0x10141f14c 0x100ffbc98 0x1d1f48aa8 0x1d1f48f90 0x1d1f48eb8 0x1d18328d0 0x1d17a4b50 0x1d18328d0 0x1d204db94 0x1d1788f90 0x1d178583c 0x1d27aa7c4 0x1d1788504 0x1d178abac 0x1d010b748 0x1d0108418 0x1d098a190 0x1d0146cdc 0x1d014b5e8 0x1d014a8e4 0x1d0148b58 0x1d018e524 0x1d046e884 0x1ce189154 0x1ce194dc8 0x1ce12009c 0x1ce1351b8 0x1ce139da0 0x204c4b998 0x1d03cefd8 0x1d03cec50 0x1d18f54f0 0x1d186f7ac 0x1d185c7fc 0x10100a8a4 0x10100a950 0x1eb87f344)
libc++abi: terminating due to uncaught exception of type NSException
(lldb)

Can you reproduce the bug?

Always

Reproduction Steps

Click on the "+" button on the top right repeatedly until the error occurs (on a "iPhone 14 Pro Max - iOS 16.4" emulator)
Optionally, type at the same time (in this case, the plus button doesn't need to be pressed as quickly)

struct ContentView: View {
    
    @FocusState var focusedId: ObjectId?
    
    var body: some View {
        NavigationStack {
            ItemsView2(focusedId: $focusedId)
                .navigationTitle("Items")
        }
    }
}

class Item: Object, ObjectKeyIdentifiable {
    @Persisted(primaryKey: true) var _id: ObjectId
    @Persisted var name: String
}

struct ItemsView2: View {
    @ObservedResults(Item.self) var items
    var focusedId: FocusState<ObjectId?>.Binding

    var body: some View {
        List {
            // Items
            ForEach(items) { item in
                ItemView(item: item, focusedId: focusedId)
            }
            .onDelete(perform: $items.remove)
        }
        .toolbar {
            // Add
            Button {
                let item = Item()
                item.name = "Item"
                $items.append(item)
                focusedId.wrappedValue = item._id
            } label: {
                Label("Add", systemImage: "plus")
            }
        }
    }
}

struct ItemView: View {
    @ObservedRealmObject var item: Item
    
    var focusedId: FocusState<ObjectId?>.Binding
    
    var body: some View {
        TextField("Item", text: $item.name)
            .focused(focusedId, equals: item._id) // focus
    }
}

Version

13.17.1

What Atlas Services are you using?

Local Database only

Are you using encryption?

No

Platform OS and version(s)

macOS 13.4.1

Build environment

Xcode version: 14.3.1
Dependency manager and version: ...

[jeffypooo]

I'm also seeing this in our app recently.

[aehlke]

same here. eek

[bogdan-pechounov]

I tried asyncWrite based on this comment, but it didn't make a difference.

struct ContentView2: View {
    
    @ObservedResults(Item.self) var items
    @Environment(\.realm) var realm

    @FocusState var focusedId: ObjectId?

    var body: some View {
        NavigationStack {
            List {
                // Items
                ForEach(items) { item in
                    ItemView(item: item, focusedId: $focusedId)
                        .onSubmit {
                            addItem()
                        }
                }
            }
            .navigationTitle("Items")
            .toolbar {
                // Add
                ToolbarItem(placement: .navigationBarTrailing) {
                    Button {
                        addItem()
                    } label: {
                        Label("Add item", systemImage: "plus")
                    }
                }
                
                // Delete
                ToolbarItem(placement: .navigationBarLeading) {
                    Button(role: .destructive){
                        deleteAll()
                    } label: {
                        Label("Delete all", systemImage: "trash")
                    }
                }
            }
        }
    }
    
    func addItem(){
        Task {
            try await realm.asyncWrite {
                let item = Item()
                realm.add(item)
                focusedId = item._id
            }
        }
    }
    
    func deleteAll(){
        $items.remove(atOffsets: IndexSet(integersIn: items.indices))
    }
}

[jeffypooo]

Can we get some eyes on this? It's a blocker bug for us.

[jeffypooo]

I tried asyncWrite based on this comment, but it didn't make a difference.

struct ContentView2: View {
    
    @ObservedResults(Item.self) var items
    @Environment(\.realm) var realm

    @FocusState var focusedId: ObjectId?

    var body: some View {
        NavigationStack {
            List {
                // Items
                ForEach(items) { item in
                    ItemView(item: item, focusedId: $focusedId)
                        .onSubmit {
                            addItem()
                        }
                }
            }
            .navigationTitle("Items")
            .toolbar {
                // Add
                ToolbarItem(placement: .navigationBarTrailing) {
                    Button {
                        addItem()
                    } label: {
                        Label("Add item", systemImage: "plus")
                    }
                }
                
                // Delete
                ToolbarItem(placement: .navigationBarLeading) {
                    Button(role: .destructive){
                        deleteAll()
                    } label: {
                        Label("Delete all", systemImage: "trash")
                    }
                }
            }
        }
    }
    
    func addItem(){
        Task {
            try await realm.asyncWrite {
                let item = Item()
                realm.add(item)
                focusedId = item._id
            }
        }
    }
    
    func deleteAll(){
        $items.remove(atOffsets: IndexSet(integersIn: items.indices))
    }
}

Affirm. We only use asyncWrite actually, and still see this bug. All of our realms are used with actor isolation.

[ianpward]

Hmm - we did merge a PR here -
realm/realm-core#6560

Are you using the latest?

[bogdan-pechounov]

@ianpward I think so. I don't know how to get the exact version of "Realm" (core), but I see "10.42.0 Release notes (2023-07-30)" in the changelog.

[ianpward]

Have you followed the recommendations here -
#4818 (comment)

and here -
https://www.mongodb.com/docs/realm/sdk/swift/react-to-changes/

[bogdan-pechounov]

@ianpward What should I observe?

    func addItem(){
        let token = items.observe { change in
            print(change)
            guard case let .change(items, _) = change else {
                print("something happened")
                return
            }
        }
    }

Type 'RealmCollectionChange<Results<Item>>' has no member 'change'

I am trying to add a new item to a collection, the recommendation seems to apply only to existing items.

Edit:
I can't use realm.add(item):

    func addItem(){
        let item = Item()
        let token = item.observe { change in
            print(change)
            guard case let .change(item, _) = change else {
                print("something happened")
                return
            }
            realm.writeAsync {
                realm.add(item)
            }
        }
    }

Instance method 'add(_:update:)' requires that 'RLMObjectBase' conform to 'Sequence'

The item coming from .change(item, _) doesn't have the correct type. (RLMObjectBase instead of Item)

[dianaafanador3]

@bogdan-pechounov
Some suggestions, first, you cannot observed unmanaged objects, this definitely is gonna throw an error
Only objects which are managed by a Realm support change notifications

let item = Item()
let token = item.observe { change in

Second, you are getting the second error because you are not supposed to add the same item to the realm and item in this case is of type RLMObjectBase which is not what realm.add(:) expects.

I'm a little bit confused about your sample.
Are you trying something more like this

func addItem(){
        let realm = try! Realm()
        let newItem = TestObject()
        // Adds the item
        try! realm.write {
            realm.add(newItem)
        }

       // Observe changes on item, this will trigger when we change the name
        let token = newItem.observe { change in
            print(change)
            guard case let .change(item, _) = change else {
                print("something happened")
                return
            }
            // Async write within the notification block which will committed asynchronously after the block completes, 
            realm.writeAsync {
               // Do any operation within the notification block
            }
        }
        try! realm.write {
            newItem.name = "test"
        }
    }

[bogdan-pechounov]

@dianaafanador3 I was trying to replicate this code, but for adding a new item:

let token = dog.observe(keyPaths: [\Dog.age]) { change in
    guard case let .change(dog, _) = change else { return }
    dog.realm!.writeAsync {
        dog.isPuppy = dog.age < 2
    }
}

If I only change the item name, then there are no errors.

    @ObservedRealmObject var item: Item
    
    var body: some View {
        TextField("Item", text: $item.name)
    }

The error occurs when adding a new item. (e.g. pressing the plus button repeatedly, typing and then pressing the plus button)

        .toolbar {
            // Add
            Button {
                let item = Item()
                item.name = "Item"
                $items.append(item)
                focusedId.wrappedValue = item._id
            } label: {
                Label("Add", systemImage: "plus")
            }
        }

[jeffypooo]

Hmm - we did merge a PR here - realm/realm-core#6560

Are you using the latest?

I am using 10.42.0 of RealmSwift, but it doesn't appear to pull in a newer core artifact than 13.7.1? I'll try updating the RealmDatabase package by itself

[jeffypooo]

@ianpward the realm-swift repo's Package.swift explicitly calls for a version:

let coreVersion = Version("13.17.1")

[dianaafanador3]

@bogdan-pechounov I was able to reproduce your issue. The error mentioned on this issue can be a little bit misgiving, this is not happening because you are registering a notification in a write transaction, is because your write is happening while the realm is in a transaction, which we verify later during the write commit and throw an exception. @jeffypooo do you have the same use case as the one described above, do you get this error while on SwiftUI View while using ObservedResults and ObservedRealmObject

[bogdan-pechounov]

@dianaafanador3 To clarify, does the binding $item.name for ObservedRealmObject create a write transaction for each character typed? (or is there some debounce)

Is the reason such a check exist is in case we create a new item that depends on the entire collection, e.g items.max? (if 2 transactions happen at the same time, the 2 items would have the same value for the order field)

                let order = (items.max(of: \Item.order) ?? 0) + 1
                let item = Item(name: name, order: order, userId: app.currentUser?.id)
                $items.append(item)

(the check is only for ObservedResults and not ObservedRealmObject?)

Is there a way to queue transactions?

[tgoyne]

There's currently no debouncing. We looked into adding it and concluded that it was going to be sufficiently complicated that we might as well put the effort into making small writes work better instead.

Write transactions can't happen "at the same time" as they involve a global (per-Realm) lock. However, in that code you're checking for the max outside of a write transaction, and so may be reading a stale value and end up with a duplicate. Wrapping the whole thing in an explicit write transaction is required for correct results:

realm.write {
    let order = (items.max(of: \Item.order) ?? 0) + 1
    let item = Item(name: name, order: order, userId: app.currentUser?.id)
    $items.append(item)
}

Async writes are the mechanism to queue writes. If you call asyncWrite from within a write transaction the block will be invoked after the current write transaction completes.

[bogdan-pechounov]

@tgoyne Are actors necessary, or can I just use Task? Since the realm comes from the environment, I am not sure how I would define a custom actor.

struct ContentView2: View {
    
    @ObservedResults(Item.self) var items
    @Environment(\.realm) var realm
    
    @FocusState var focusedId: ObjectId?
    
    var body: some View {
        NavigationStack {
            List {
                // Items
                ForEach(items) { item in
                    ItemView(item: item, focusedId: $focusedId)
                        .onSubmit {
                            addItem()
                        }
                }
            }
            .navigationTitle("Items")
            .toolbar {
                // Add
                ToolbarItem(placement: .navigationBarTrailing) {
                    Button {
                        addItem()
                    } label: {
                        Label("Add item", systemImage: "plus")
                    }
                }
                
                // Delete
                ToolbarItem(placement: .navigationBarLeading) {
                    Button(role: .destructive){
                        deleteAll()
                    } label: {
                        Label("Delete all", systemImage: "trash")
                    }
                }
            }
        }
    }
    
    func addItem(){
        Task {
            try await realm.asyncWrite {
                print("COUNT", items.count)
                let item = Item()
                realm.add(item)
                focusedId = item._id
            }
            print("COUNT AFTER", items.count)
        }
    }
    
    func deleteAll(){
        $items.remove(atOffsets: IndexSet(integersIn: items.indices))
    }
}

(doesn't fix the error)

Also, I noticed that the error doesn't occur if I remove the line focusedId = item._id. (or if the new item is too far down the list to be visible and gain focus)

    func addItem(){
        print(Thread.current)
        Task {
            try await realm.asyncWrite {
                print("COUNT", items.count, Thread.current)
                let item = Item()
                realm.add(item)
                //focusedId = item._id
            }
            print("COUNT AFTER", items.count, Thread.current)
        }
    }

<_NSMainThread: 0x2819280c0>{number = 1, name = main}
COUNT 49 <_NSMainThread: 0x2819280c0>{number = 1, name = main}
COUNT AFTER 50 <_NSMainThread: 0x2819280c0>{number = 1, name = main}

Since the closure is executed on the main thread, maybe animating the keyboard up interferes with realm in some say?

COUNT 0
COUNT AFTER 1
COUNT 1
COUNT AFTER 2
COUNT 2
COUNT AFTER 3
COUNT 3
COUNT AFTER 4
COUNT 4
COUNT AFTER 5
COUNT 5
COUNT AFTER 6
COUNT 6
COUNT AFTER 7
COUNT 7
COUNT AFTER 8
COUNT 8
COUNT AFTER 9
COUNT 9
COUNT 10
COUNT AFTER 11
COUNT 11
COUNT AFTER 12
COUNT AFTER 12
COUNT 12
COUNT 13
COUNT AFTER 14
COUNT AFTER 14
COUNT 14
COUNT AFTER 15
COUNT 15
COUNT AFTER 16
COUNT 16
COUNT AFTER 17
COUNT 17
COUNT AFTER 18
COUNT 18
COUNT AFTER 19
COUNT 19
COUNT AFTER 20
COUNT 20
COUNT AFTER 21
COUNT 21
2023-08-21 18:14:46.406612-0400 TestFocus[96011:3507884] *** Terminating app due to uncaught exception 'RLMException', reason: 'Cannot register notification blocks from within write transactions.'
*** First throw call stack:
(0x1b6d3148c 0x1b0081050 0x10122e8e4 0x1010af5c0 0x1012aa784 0x1012c563c 0x1012c5778 0x1013b7db4 0x1013b8408 0x1ba438db8 0x1ba417ae0 0x1ba4240b0 0x1ba460f80 0x1ba420f28 0x1ba915b00 0x1d87c6ef8 0x1d87c65c0 0x1d87c5478 0x1ba3f2680 0x1bb4b3cb4 0x1bac83eec 0x1ba3f0258 0x1ba3eac5c 0x1ba3e5ab8 0x1bb4b3c80 0x1bb4b3bb8 0x1ba49804c 0x1ba3e38a0 0x1ba3e3804 0x1ba3e398c 0x1b6db1968 0x1b6d4159c 0x1b6d9d294 0x1b6da1da0 0x1ed920998 0x1b903380c 0x1b9033484 0x1ba55ac68 0x1ba4d4f1c 0x1ba4c1f6c 0x100fb582c 0x100fb58d8 0x1d4540344)
libc++abi: terminating due to uncaught exception of type NSException
(lldb) 

[bogdan-pechounov]

If I simply use $items.append(Item()), there are no issues, so queuing transactions might not be necessary. The problem is when an ItemView gains focus.

        let item = Item()
        $items.append(item)
        focusedId = item._id

Edit: This only applies to pressing the plus button repeatedly. Typing while pressing the plus button will still produce an error.

@jeffypooo @aehlke How does the error occur for you? (by adding repeatedly while using @FocusState or by adding while updating a field)

[jeffypooo]

We usually encounter this when trying to start change observation as a result of a write, but it does seem to be limited to situations where the presentation mode is changing. The most common occurrence for us is when a button is clicked that:

• dismisses a sheet
• writes some data to realm

[jeffypooo]

Interesting that both reproduction cases mentioned involve buttons as well

[gongzhang]

I've also recently encountered this exception in the AppIntent.perform() implementation of my app (an iOS Shortcuts action). It consistently crashes at Realm.beginWrite(). However, if I call realm.refresh() just before initiating the write transaction, the crash doesn't occur. Everything is on main queue.

I'm not entirely sure whether this is an issue with my usage, or if it's related to the original issue. But I haven't been able to identify the cause myself. I've also tried versions before and after realm/realm-core#6560, and they reproduce the same crash.

[davidkessler-ch]

We are also encountering this issue. When asnyWrite (or normal writes, issue happens either way) are triggered in rapid succession (in our case caused by quick server requests) and there is a notification block observing this collection (no writes in the notification block). This happens even with everything realm on the @mainactor.

[aehlke]

I'm running into this on latest realm-swift, iPadOS 16.5 with debugger attached. No movement on this in 3+ months :/ I am checking immediately beforehand isInWriteTransaction as well...

[aehlke]

Is anyone seeing this more in development vs production? I am getting hit with it constantly in development due to a write transaction beginning but unclear what else is happening concurrently/elsewhere at the same time that could be conflicting with that... I'm worried about releasing to production in this state given how often I get it on macOS during development.

@dianaafanador3 I get this with many views having both ObservedResults and ObservedRealmObject, per your earlier question... Is there something else we can provide?

[davidkessler-ch]

This no longer happens for us when performing all writes on a (single) background actor. We use a global actor for this like so:

@globalActor
actor RealmBackgroundActor {
    static var shared = RealmBackgroundActor()
}

We also observe on this actor only, and use exclusively asyncWrite though I believe we have also tested with normal write and it did also work.

So we observe using:

@RealmBackgroundActor
func observeCollections() async throws {
        let realm = try await Realm(actor: RealmBackgroundActor.shared)
        let results = realm.objects(SomeProtocol.self)
        resultNotificationToken = results.observe { [weak self] (changes: RealmCollectionChange) in
            self?.objects = Array(results.sorted(by: \.createdAt).freeze())
        }
}

and write using:

@RealmBackgroundActor
func write(object: SomeProtocol) async throws {
        let realm = try await Realm(actor: RealmBackgroundActor.shared)
        try await realm.asyncWrite {
            realm.create(type(of: object), value: object, update: .modified)
        }
    }

Hope this helps someone:)

[aehlke]

Thanks for sharing. That would require a full rewrite of my RealmSwift usage, to remove all SwiftUI usage and so on... Difficult choice for a large project. I hope there is another solution to this constant crashing.

edit: did you test having more than one realm bg actor? it's too bad to not be able to parallelize even different .realm files

[aehlke]

I tried asyncWrite based on this comment, but it didn't make a difference.

@bogdan-pechounov that commenter suggests using writeAsync not asyncWrite but not sure that would make a difference

edit: I've begun implementing @davidkessler-ch 's solution and it's working for me as well. Too bad it means a full rewrite of all my realm usage to async actors, and maybe ditching ObservedResults. I hope others don't wander into this problem with huge lock-in on non-async actor approaches that are still advocated for in official docs.

[aehlke]

update: I've converted all Realm writes to asyncWrite on a new globalActor and inside Task.detached but still facing constant crashing (which has worsened recently). Now removing all usage of ObservedResults, ObservedRealmObject, StateRealmObject which I suspect are problematic.

This makes me want to abandon Realm ASAP - without being able to use any of the SwiftUI conveniences and inaction for years on this issue, the upside is getting hard to appreciate

[davidkessler-ch]

@aehlke In our case I believe the core problem was exactly the parallelisation that lead to the unexpected behaviour, which is fair, considering that opening many realms in async context without specifying any actors.
Imagine this: parallel code is a) writing an object "dog" to a realm collection and b) registering a change listener to this realm collection. Whether a notification for this dog fires is then no longer deterministic, since it depends on which parallel code runs first. (I imagine this could be happening with something like ObservedResults as well)

Now that being said, I also think this could be handled way better, e.g. by having the registration for the change listener be async as well, waiting for any write locks to finish before registering. Or waiting to finish any change listener registration before allowing to open any write transaction.

What frustrates me the most, is that realm does not allow for us to catch errors, but rather crashes the app, since they say it's "considered to be programmer error". In many cases this makes sense, but with these very fine grained, sometimes hard-to-reproduce problems in concurrency, going into production is just a huge pain.

[tgoyne]

It turns out that while 10.39.1 was supposed to make it so that notification blocks could be registered from within write transactions until changes were actually made, this didn't actually work. #8439 makes it so that it does.

[aehlke]

@davidkessler-ch appreciate the thoughts

@tgoyne great news, thanks for the update

For my code, I found a spot I'd missed with observation happening off the custom globalActor and fixed that; seems stable now that ALL writes AND observations happen on a custom globalActor and in Task.detached, without needing to finish ridding my code of ObservedRealmObject etc

[bogdan-pechounov]

@tgoyne I updated the package (RealmDatabase 13.25.0, iOS 17.2) and there are no longer any issues. Thank you!