1

I'm storing ~100.000 dictionary entries in a realm database and would like to display them. Additionally I want to filter them by a search field. Now I'm running in a problem: The search function is really inefficient although I've tried to debounce the search.

View Model:

class DictionaryViewModel : ObservableObject {

    let realm = DatabaseManager.sharedInstance

    @Published var entries: Results<DictionaryEntry>?
    @Published var filteredEntries: Results<DictionaryEntry>?
    @Published var searchText: String = ""
    @Published var isSearching: Bool = false
    var subscription: Set<AnyCancellable> = []
    
    init() {
        $searchText
            .debounce(for: .milliseconds(800), scheduler: RunLoop.main) // debounces the string publisher, such that it delays the process of sending request to remote server.
            .removeDuplicates()
            .map({ (string) -> String? in
                if string.count < 1 {
                    self.filteredEntries = nil
                    return nil
                }

                return string
            })
            .compactMap{ $0 } 
            .sink { (_) in   
            } receiveValue: { [self] (searchField) in
                filter(with: searchField)
            }.store(in: &subscription)

        self.fetch()
}

    public func fetch(){
        self.entries = DatabaseManager.sharedInstance.fetchData(type:   DictionaryEntry.self).sorted(byKeyPath: "pinyin", ascending: true)
        self.filteredEntries = entries
    }

    public func filter(with condition: String){
        self.filteredEntries = self.entries?.filter("pinyin CONTAINS[cd] %@", searchText).sorted(byKeyPath: "pinyin", ascending: true)
    }

In my View I'm just displaying the filteredEtries in a ScrollView

The debouncing works well for short text inputs like "hello", but when I filter for "this is a very long string" my UI freezes. I'm not sure whether something with my debounce function is wrong or the way I handle the data filtering in very inefficient.

EDIT: I've noticed that the UI freezes especially when the result is empty.

EDIT 2: The .fetchData() function is just this here:

func fetchData<T: Object>(type: T.Type) -> Results<T>{
    let results: Results<T> = realm.objects(type)
    return results
}

All realm objects have a primary key. The structure looks like this:

@objc dynamic var id: String = NSUUID().uuidString
@objc dynamic var character: String = ""
@objc dynamic var pinyin: String = ""
@objc dynamic var translation: String = ""

override class func primaryKey() -> String {
    return "id"
}

EDIT 3: The filtered results are displayed this way:

ScrollView{
    LazyVGrid(columns: gridItems, spacing: 0){
                    if (dictionaryViewModel.filteredEntries != nil)  {
                                 ForEach(dictionaryViewModel.filteredEntries!){ entry in
                                    Text("\(entry.translation)")
                        
                        }
                    } else {
                        Text("No results found")
                    } 
                }
kirkyoyx
  • 313
  • 2
  • 12
  • 1
    Realm is incredibly efficient working with large datasets as the objects are lazily loaded. This debounce code appears to be overriding that built in functionality and loading in all of the objects to perform filtering. e.g. as soon as you 'convert' realm results into an array like this `.map({ (string) -> String? in`, all of the objects are loaded. We regularly filter way over 100,000 objects with no lag to speak of. We also don't know what `.fetchData` does. Can you include it? Do your objects include primary keys? Did you set up indexing on the search property? – Jay Jan 03 '21 at 17:46
  • @Jay I've added some more information. So the bottle neck is the debouncing function? I'm not sure how I can solve this for my use case without the debounce. Since I would like to have a "real time filtering function" the filtered results should appear right after the user types something in the textfield (or maybe with a small delay ~0.8 seconds). But when I remove the debounce my UI freezes after each letter for ~0.5 seconds. – kirkyoyx Jan 03 '21 at 17:55
  • 1
    Hmm. We are not having that experience. You can add indexing to the `pinyin` property which will make it faster. Are you sure the delay is getting the data from Realm and not updating the tableView or however you're displaying the list? – Jay Jan 03 '21 at 18:30
  • Honestly I'm not sure whether it's a realm problem or an UI problem. But I've tried to replace the LazyVGrid with a List and it was worse. I've added the code for listing the objects in my view. Do you know an efficient way to display a long list in SwiftUI? – kirkyoyx Jan 03 '21 at 18:46
  • @Jay I've refactored my code a little bit and tested whether it's a realm or UI problem. And it's definitely an UI problem. I just printed the number of the filtering results and got the value in ~0.1 seconds. Doesn't matter how many results. But when I tried to display them in a list it's making trouble (CPU and memory explodes). Since my problem itsnot anymore a realm specific topic I create a new question. Thank you for your help! – kirkyoyx Jan 04 '21 at 08:29

0 Answers0