Fetching data really fast from Google cloud firestore

Question

I am making a flutter application and I’m using cloud firestore as my online database. One of the features in my app is looking for nearby users and showing their profile to the current user in a custom widget on screen. The way I do it is I get current user’s location (either live location or the address saved in database) and then go through every single user in my database collection for users. I get the address of the user from the stored data, calculate the distance using Distance Matrix API and then if distance is less than a specific number (e.g 10000 meter) I create the profile widget for the user to show it on screen.

There are 2 problems:

1- if my number of users grows (for example a million users), By going through every user detail and calculating the distance, It can take a really long time to get the results on the screen . Right now, I only have 20 users for testing purposes and when I search for nearby users, it can take 30 seconds for the results to show up on the screen.

2- With a slow internet connection, the waiting time can be much much more and it can use lots of user’s data for this simple task.

How can I improve this feature and make it faster?

(The current idea that I have is dividing user’s in different documents with respect to their location and then going through only one of the documents by using current user’s location. The problem is that how to divide the addresses efficiently and find the best addresses to look for.)

Below is the code where I find nearby users and add them to a list which I pass to my custom widget class.

final List<UserBoxDesign> listOfBoxes = [];
final FirebaseUser currentUser = await auth.currentUser();
final String currentUserId = currentUser.uid;
if (_userLocationSwitchValue == false) { //use default address of the user
  currentUserAddress = await _databaseManagement.getUserCollectionValues(
      currentUserId, "address");
} else {
  //currentUserAddress = //to do, get device location here.
}
if (_searchValue == SearchValues.Users) {
  final List<String> userIds = await _databaseManagement.getUserIds();
  for (String id in userIds) {
    final String otherUserLocations =
        await _databaseManagement.getUserCollectionValues(id, "address");
    final String distanceMeters = await _findDistanceGoogleMaps(
        currentUserAddress, otherUserLocations);
    if (distanceMeters == "Address can't be calculated" ||
        distanceMeters == "Distance is more than required by user") {
      //if it's not possible to calculate the address then just don't do anything with that.
    } else {
      final double distanceValueInKilometers = (double.parse(
                  distanceMeters) /
              1000)
          .roundToDouble(); 
      final String userProfileImageUrl =
          await _databaseManagement.getUserCollectionValues(id, "image");
      final String username =
          await _databaseManagement.getUserCollectionValues(id, "username");
      listOfBoxes.add(
        UserBoxDesign( //it creates a custom widget for user if user is nearby
          userImageUrl: userProfileImageUrl,
          distanceFromUser: distanceValueInKilometers,
          userId: id,
          username: username,
        ),
      ); //here we store the latest values inside the reserved data so when we create the page again, the value will be the reservedData value which is not empty anymore

    }
    print(listOfBoxes);
  }
  listOfBoxes.sort((itemA,itemB)=>itemA.distanceFromUser.compareTo(itemB.distanceFromUser)); //SORTs the items from closer to more far from user (we can reverse it to far comes first and close goes last)
  setState(() {
    _isSearchingForUser = false;
  });
  return listOfBoxes;

Here is the code where I calculate the distance between origin address and destination address.

Future<String> _findDistanceGoogleMaps(
  String originAddress, String destinationAddress) async {
final String url =
    "https://maps.googleapis.com/maps/api/distancematrix/json?units=metric&origins=$originAddress&destinations=$destinationAddress&key=$GoogleMapsAPIKey";
try {
  final response = await http.get(url);
  final responseDecoded = json.decode(response.body);

  final distanceInMeters = double.parse(responseDecoded["rows"][0]
          ["elements"][0]["distance"]["value"]
      .toString()); //this is the value in meters always so for km , divide by 1000.
  if (distanceInMeters < 100000) {
    return distanceInMeters.toString();
  } else {
    return "Distance is more than required by user";
  }
} catch (e) {
  return "Address can't be calculated";
}

}

this is how my screen looks when I find nearby users.

Answer 1

if you give code example it will be easy to answer. I can suggest (we had similar task) to use coordinates longitude and latitude and make requests in your range.

So you don't need Distance Matrix API (I think it will be expensive) and your queries will be fast and cheap.

I googled and found out answer here How to run a geo "nearby" query with firestore?

==after updated question==

1. You try to use all the logic inside screen and do it synchronously. Because of that you have such a long rendering time. You calculate everything on user device and pass to a widget return listOfBoxes; Instead you can try to use Streambuilder, example is here How to efficiently access a firestore reference field's data in flutter?

2. Organise data in your database in a such way, that you can make request for your purposes: "Find all users within X range sorted by distance AND ...". In this case Firebase will do it very quickly and pass data to your Streambuilder asynchronously. I guess that you can keep longitude, latitude and work with them instead of addresses.

Sorry, I cant rewrite your code, not enough info. Just look example by link, there is a good example.

==update 2==

The package https://pub.dev/packages/geoflutterfire allow to store geo data to Firestore and how to make requests

// Create a geoFirePoint
GeoFirePoint center = geo.point(latitude: 12.960632, longitude: 77.641603);

// get the collection reference or query
var collectionReference = _firestore.collection('locations');

double radius = 50;
String field = 'position';

Stream<List<DocumentSnapshot>> stream = geo.collection(collectionRef: collectionReference)
                                        .within(center: center, radius: radius, field: field);