20

This is an SSCCE, shows research, isn't a dupe and is on topic!!!


Spring Boot REST service and MySQL here. I have the following Profile entity:

@Entity
@Table(name = "profiles")
public class Profile extends BaseEntity {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "profile_given_name")
    private String givenName;

    @Column(name = "profile_surname")
    private String surname;

    @Column(name = "profile_is_male")
    private Integer isMale;

    @Column(name = "profile_height_meters", columnDefinition = "DOUBLE")
    private BigDecimal heightMeters;

    @Column(name = "profile_weight_kilos", columnDefinition = "DOUBLE")
    private BigDecimal weightKilos;

    @Column(name = "profile_dob")
    private Date dob;

    // Getters, setters & ctor down here
}

I also have a ProfileController and I want to expose a GET endpoint that provides a really flexible/robust way to search for Profiles based on a large range of criteria:

# Search for women between 1.2 and 1.8 meters tall.
GET /v1/profiles?isMale=0&heightMeters={"gt": 1.2, "lt": 1.8}

# Search for men born after Jan 1, 1990 who weigh less than 100 kg.
GET /v1/profiles?isMale=1&dob={"gt" : "1990-01-01 00:00:00"}&weightKilos={"lt": 100.0}

etc.

So here's my controller:

@RestController
@RequestMapping("/v1/profiles")
public class ProfileResource {
  @Autowired
  ProfileRepository profileRepository;

  @GetMapping
  public ResponseEntity<Set<Profile>> searchProfiles(@RequestParam(value = "isMale", required = false) String isMaleVal,
                                              @RequestParam(value = "heightMeters", required = false) String heightMetersVal,
                                              @RequestParam(value = "weightKilos", required = false) String weightKilosVal,
                                              @RequestParam(value = "dob", required = false) String dobVal) {

      Integer isMaleVal;
      BooleanCriteria isMaleCriteria;
      if(isMaleVal != null) {
        // Parse the value which could either be "0" for female, "1" for male or something like
        // ?isMale={0,1} to indicate

        // BooleanCriteria would store which values male, female or both) to include in the search
      }

      BigDecimal heighMeters;
      BigDecimalCriteria heightCriteria;
      if(heightMetersVal != null) {
        // Parse the value which like in the examples could be something like:
        // ?heightMeters={"gt" : "1.0"}

        // BigDecimalCriteria stores range information
      }

      BigDecimal heighMeters;
      BigDecimalCriteria weightCriteria;
      if(weightKilosVal != null) {
        // Parse the value which like in the examples could be something like:
        // ?weightKilos={"eq" : "100.5"}

        // BigDecimalCriteria stores range information
      }

      // Ditto for DOB and DateCriteria

      // TODO: How to pack all of these "criteria" POJOs into a
      // CrudRepository/JPQL query against the "profiles" table?
      Set<Profile> profiles = profileRepository.searchProfiles(
        isMaleCriteria, heightCriteria, weightCriteria, dobCriteria);
    }
}

My thinking for, say, BigDecimalCriteria would be something like:

// Basically it just stores the (validated) search criteria that comes in over the wire
// on the controller method
public class BigDecimalCriteria {
  private BigDecimal lowerBound;
  private Boolean lowerBoundInclusive;
  private BigDecimal upperBound;
  private Boolean upperBoundInclusive;

  // Getters, setters, ctors, etc.
}

Since all of these search criteria are optional (and thus can be null), I'm stuck on how to write the JPQL query in the ProfileRepository:

public interface ProfileRepository extends CrudRepository<Profile,Long> {
  @Query("???")
  public Set<Profile> searchProfiles();
}

How can I implement the @Query(...) for ProfileRepository#searchProfiles in such a way that enables all of my search criteria (given all the permissible ranges and criteria values to search for), and allows any criteria to be null/optional?

Of course, if there are any nifty little libraries or if Spring Boot/JPA already has a solution for this, I'm all ears!

smeeb
  • 27,777
  • 57
  • 250
  • 447
  • Have you looked into querydsl http://www.querydsl.com/ – Olantobi Jan 19 '18 at 17:24
  • 1
    The easiest way to handle a query with complex, conditional criteria is to use a `CriteriaQuery`. Criteria queries can be integrated smoothly into Spring Data repositories using a `Specification` (see example here: https://spring.io/blog/2011/04/26/advanced-spring-data-jpa-specifications-and-querydsl/) – crizzis Jan 19 '18 at 17:26
  • Why the downvote? This is an [SSCCE](https://sscce.org), shows research, isn't a dupe and is on topic! – smeeb Jan 29 '18 at 19:47
  • It is very much a dupe and shows no research, also contains tons of irrelevant cruft. https://stackoverflow.com/questions/32728843/spring-data-optional-parameter-in-query-method?noredirect=1&lq=1 https://stackoverflow.com/questions/45716923/passing-optional-parameters-to-query-of-a-spring-data-repository-method and many more deal with variations of this topic. – Jens Schauder Feb 01 '18 at 12:53
  • @JensSchauder many more you too big boy! Sounds like Spring Data doesn't have the capability to address this type of problem and *someone* is upset that its being called out.... – smeeb Feb 01 '18 at 15:15
  • I'm sorry that I left the impression of being upset about the limitations you see in in Spring Data. I'm not, but thanks for worrying. My intention was just to help you solve the problem stated in the question by pointing out that variations of it are solved many times on this site, and to answer your question in the comment why it was downvoted. – Jens Schauder Feb 01 '18 at 15:50

4 Answers4

35

You can achieve complex queries with specifications by JpaSpecificationExecutor in spring data. Repository interface must extend the JpaSpecificationExecutor<T> interface so we can specify the conditions of our database queries by creating new Specification<T> objects.

The trick is in the use of the Specification interface in combination with a JpaSpecificationExecutor. here is the example:

@Entity
@Table(name = "person")
public class Person {

 @Id
 @GeneratedValue(strategy = GenerationType.AUTO)
 private Long id;

 @Column(name = "name")
 private String name;

 @Column(name = "surname")
 private String surname;

 @Column(name = "city")
 private String city;

 @Column(name = "age")
 private Integer age;

        ....

}

Then we define our repository:

public interface PersonRepository extends JpaRepository<Person, Long>, JpaSpecificationExecutor<Person> {

}

As you can see we have extended another interface the JpaSpecificationExecutor. This interface defines the methods to perform the search via a Specification class.

What we have to do now is to define our specification that will return the Predicate containing the constraints for the query (in the example the PersonSpecification is performing the query select * from person where name = ? or (surname = ? and age = ?) ):

public class PersonSpecification implements Specification<Person> {

    private Person filter;

    public PersonSpecification(Person filter) {
        super();
        this.filter = filter;
    }

    public Predicate toPredicate(Root<Person> root, CriteriaQuery<?> cq,
            CriteriaBuilder cb) {

        Predicate p = cb.disjunction();

        if (filter.getName() != null) {
            p.getExpressions()
                    .add(cb.equal(root.get("name"), filter.getName()));
        }

        if (filter.getSurname() != null && filter.getAge() != null) {
            p.getExpressions().add(
                    cb.and(cb.equal(root.get("surname"), filter.getSurname()),
                            cb.equal(root.get("age"), filter.getAge())));
        }

        return p;
    }
}

Now it is time to use it. The following code fragment shows how to use the Specification we just created:

...

Person filter = new Person();
filter.setName("Mario");
filter.setSurname("Verdi");
filter.setAge(25);

Specification<Person> spec = new PersonSpecification(filter);

List<Person> result = repository.findAll(spec);

Here is full example present in github

Also you can create any complex queries using Specification

george_h
  • 1,562
  • 2
  • 19
  • 37
Bhushan Uniyal
  • 5,575
  • 2
  • 22
  • 45
  • Great answer, suppose I want to filter the records based on 3 parameters and each are optional to be passed to filter records, so do I need to write 6 cases in `toPredicate` right? – Romil Patel Nov 15 '19 at 10:05
  • it should be 3 if you have just needed 3, means each has the individual condition and don't need to group with others, if you have the group-based condition on you parameter then you can create any number of cases with the predicate. – Bhushan Uniyal Nov 15 '19 at 10:50
  • yes, as parameters are optional, so there might be group condition and might be not if only one parameter is passed. So I have to check for all the condition and write cases for 6 conditions as I have 3 parameters – Romil Patel Nov 15 '19 at 11:39
  • Thank you @bhushan for the time, just one more doubt, I have to filter on entity which has manytomany relationship, how can I pass parameter for that. can you suggest on that please – Romil Patel Nov 15 '19 at 12:18
  • you can Join with your entity "Join join = root.join("targetEntiiesVaribaleNameInSourceClass");" and add the predicate with it – Bhushan Uniyal Nov 15 '19 at 12:27
  • Thank you for your time @bhushanuniyal, it is working as expected – Romil Patel Nov 15 '19 at 13:47
9

Almost what you need is already implemented in Spring Data with help of Querydsl and Web support Spring Data extensions.

You should extend your repo as well from QuerydslPredicateExecutor and, if you are using Spring Data REST, you can query your repo data right 'from the box' with base filtering, paging and sorting support:

/profiles?isMale=0&heightMeters=1.7&sort=dob,desc&size=10&page=2

To implement more complex filter you should extend your repo from the QuerydslBinderCustomizer and use its customize method (right in your repo).

For example you can implement 'between' filter for heightMeters and 'like' filter for surname:

public interface ProfileRepository extends JpaRepository<Profile, Long>, QuerydslPredicateExecutor<Profile>, QuerydslBinderCustomizer<QProfile> {

    @Override
    default void customize(QuerydslBindings bindings, QProfile profile) {

      bindings.excluding( // used to exclude unnecessary fields from the filter
          profile.id,
          profile.version,
          // ...
      );

      bindings.bind(profile.heightMeters).all((path, value) -> {

          Iterator<? extends BigDecimal> it = value.iterator();
          BigDecimal from = it.next();
          if (value.size() >= 2) {
              BigDecimal to = it.next();
              return path.between(from, to)); // between - if you specify heightMeters two times
          } else {
              return path.goe(from); // or greter than - if you specify heightMeters one time
          }
      });

      bindings.bind(profile.surname).first(StringExpression::containsIgnoreCase);        
    }
}

Then you can query your profiles:

/profiles?isMale=0&heightMeters=1.4&heightMeters=1.6&surename=doe

i.e. - find all females which height is between 1.4 and 1.6 meters and surename contains 'doe'.

If you are not using Spring Data REST you can implement your own rest controller method with QueryDSL support:

@RestController
@RequestMapping("/profiles")
public class ProfileController {

    @Autowired private ProfileRepository profileRepo;

    @GetMapping
    public ResponseEntity<?> getAll(@QuerydslPredicate(root = Profile.class, bindings = ProfileRepository.class) Predicate predicate, Pageable pageable) {

        Page<Profile> profiles = profileRepo.findAll(predicate, pageable);
        return ResponseEntity.ok(profiles);
    }
}

Note: don't forget to add QueryDSL dependency to you project:

<dependency>
    <groupId>com.querydsl</groupId>
    <artifactId>querydsl-jpa</artifactId>
</dependency>

<dependency>
    <groupId>com.querydsl</groupId>
    <artifactId>querydsl-apt</artifactId>
    <scope>provided</scope>
</dependency>

<build>
    <plugins>
        <plugin>
            <groupId>com.mysema.maven</groupId>
            <artifactId>apt-maven-plugin</artifactId>
            <version>1.1.3</version>
            <executions>
                <execution>
                    <goals>
                        <goal>process</goal>
                    </goals>
                    <configuration>
                        <outputDirectory>target/generated-sources/annotations</outputDirectory>
                        <processor>com.querydsl.apt.jpa.JPAAnnotationProcessor</processor>                                                       
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Then compile your project (for example mvn compile) to let it make 'Q' classes.

Cepr0
  • 28,144
  • 8
  • 75
  • 101
  • Can you also give the gradle dependencies for query dsl? – Jagrati Gogia Jun 13 '18 at 11:08
  • @JagratiGogia No, but I hope these posts can help you: https://discuss.gradle.org/t/integrating-spring-boot-querydsl-into-gradle-build/15421/3, https://stackoverflow.com/questions/22773639/querydsl-spring-boot-gradle – Cepr0 Jun 13 '18 at 14:52
  • Small note: in your `customize` method, there is `Optional.of(` missing on both return statements. – Simon Lenz Nov 30 '18 at 22:36
  • 1
    @Graslandpinguin Yes, it has been changed since Spring Boot 2 (see my related ['how-to' post](https://stackoverflow.com/questions/51127468/how-to-easy-implement-rest-api-query-language-with-querydsl-and-spring-data-to)) – Cepr0 Dec 01 '18 at 09:19
4

The answer is quite eaiser and you can use the query-by-example in spring.

and even more you do not need to list all the Profile properties in your controller, you just take the Profile as the parameter, spring will take care of it.

And as you want to validate the request params, here is eaier to integrate with bean validator, take "givenName" as an example. add the NotNull in the entity, and add @Valid in the controller, in case the "givenName" is not in the request params, you will get the "Bad Request" response.

Here are the working codes:

@Entity
@Table(name = "profiles")
public class Profile {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "profile_given_name")
    @NotNull
    private String givenName;

    @Column(name = "profile_surname")
    private String surname;

    @Column(name = "profile_is_male")
    private Integer isMale;

    @Column(name = "profile_height_meters", columnDefinition = "DOUBLE")
    private BigDecimal heightMeters;

    @Column(name = "profile_weight_kilos", columnDefinition = "DOUBLE")
    private BigDecimal weightKilos;

    @Column(name = "profile_dob")
    private Date dob;
}

ProfileResource

@RestController
@RequestMapping("/v1/profiles")
public class ProfileResource {
    @Autowired
    ProfileRepository profileRepository;

    @GetMapping
    public ResponseEntity<List<Profile>> searchProfiles(@Valid Profile profile) {
        List<Profile> all = profileRepository.findAll(Example.of(profile));
        return ResponseEntity.ok(all);
    }
}

ProfileRepository

public interface ProfileRepository extends JpaRepository<Profile, Long> {
}

Then send the GET /v1/profiles?isMale=0 HTTP method as you wanted.

Liping Huang
  • 4,378
  • 4
  • 29
  • 46
  • Thanks @Liping (+1) - a few followup questions, if you don't mind: **(1)** is this documented anywhere? **(2)** This is a GET endpoint that takes an argument in the controller method...does the user have to send anything in the request entity/body? **(3)** Your example shows `isMale=0`, what are all the different types of supported comparison operators? For instance, how could I search for `Profiles` with `heightMeters > 1.5` AND `heightMeters <= 2.0`? – smeeb Jan 30 '18 at 10:17
  • 1
    @smeeb for (1) , yes, you can refer to the link. for (2) I just update the answer to add the the bean validator sample, you do not need to list all the `Profile` properties with the GET method. for (3) spring will convert the path vars to the correspondind Object, you can refer to spring type convestion part. – Liping Huang Jan 30 '18 at 10:22
  • Thanks again so much @Liping!!! For #3 above, I just looked at your link and searched for "*type conversion*" and nothing turned up. Any chance you could modify your example to show me how I could search for `Profiles` with `1.5 < heightMeters <= 2.0` (as an example)? Thanks again! – smeeb Jan 30 '18 at 10:24
  • 1
    @smeeb Sorry, for (3) my mistake, you can impmement it with the bean validator, add `@Range(min=, max=)` in your entity. – Liping Huang Jan 30 '18 at 10:29
  • 1
    @smeeb Here is a good example for the bean validator - http://www.baeldung.com/javax-validation – Liping Huang Jan 30 '18 at 10:32
  • Thanks again @Liping, but that's only for **validation**, not **querying**...I need to be able to *query* for `Profiles` whose `heightMeters` property is between, say, 1.5 and 2.0 meters. I don't believe that your *Query By Example* approach above allows me to do that, correct? – smeeb Jan 30 '18 at 10:59
  • Yes, this is the validator for the entity, so will impact the save behavior. for query, one is you can add the logic into the controller section. – Liping Huang Jan 30 '18 at 11:53
  • Or add the property with the corresponding annotation just like what your already did. – Liping Huang Jan 30 '18 at 12:08
  • Thanks again :-) Last question (I promise!) Does Spring/QueryByExample natively support query string params like `?heightMeters={"gt" : "1.0"}` or are you saying that I'll need to add support for that and add an `ExampleMatcher` into the controller logic myself? – smeeb Jan 30 '18 at 13:00
  • it ok smeeb, let's say JPA/Hibernate, as the search by example is actually the hibernate behavior, so basis it will parse to native sql then query it from the database. so for you, the query string is a json string, so you need to do the logic self. – Liping Huang Jan 30 '18 at 13:32
  • According to the reference document between queries are not possible using this method. > >Query by Example also has several limitations: > No support for nested or grouped property constraints, such as firstname = ?0 or (firstname = ?1 and lastname = ?2). > Only supports starts/contains/ends/regex matching for strings and exact matching for other property types. – HamedKhan Apr 04 '20 at 07:40
3

Check out “query by example” in spring data. Seems to fit the bill for what you need...

https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#query-by-example

LetsBeFrank
  • 774
  • 11
  • 31