1

I have a Meeting class that is persisted to DB using JPA hibernate in an Spring Boot Application interfaced by a REST API, I have performance concerns on operations that must be thread safe. This is the Meeting class:

@Entity
public class Meeting {
    @Id
    @GeneratedValue(strategy= GenerationType.AUTO)
    private Long id;

    @ManyToOne(optional = false)
    @JoinColumn(name = "account_id", nullable = false)
    private Account account;

    private Integer maxAttendees;
    private Integer numAttendees; // current number of attendees
    ...
}

As you can see I have an Account entity, an account can have many meetings associated. A meeting has a maximum number of attendees and an account has a maximum number of scheduled meetings, in a similar fashion an Account has a maxSchedules and numSchedules variables.

The basic workflow is: A meeting is created, then scheduled, then attendees are registered individually.

Note: The main goal here is avoiding to exceed the maximum number of allowed operations (schedules or registers).

I initially was more focused on the business logic than performance, initially my business service for scheduling and registering attendees looked like this:

@Service
public class MeetingService {
    ...

    @Transactional
    public synchronized void scheduleMeeting(Long meetingId, Date meetingDate) {
        Meeting meeting = repository.findById(meetingId);
        Account account = meeting.getAccount();
        if(account.getNumSchedules() + 1 <= account.getMaxSchedules() 
            && meeting.getStatus() != SCHEDULED) {
            meeting.setDate(meetingDate);
            account.setNumSchedules(account.getNumSchedules()+1);
            // save meeting and account here
        }

        else { throw new MaxSchedulesReachedException(); }
    } 

    @Transactional
    public synchronized void registerAttendee(Long meetingId, String name) {
        Meeting meeting = repository.findById(meetingId);
        if(meeting.getNumAttendees() + 1 <= meeting.getMaxAttendees() 
            && meeting.getStatus() == SCHEDULED) {
            meeting.setDate(meetingDate);
            meeting.setNumAttendees(account.getNumAttendees()+1);
            repository.save(meeting);
        }

        else { throw new NoMoreAttendeesException(); }
    } 
    ...
}

The problem with this approach is that the lock of the synchronized methods is the object (this), the service is a singleton instance so when multiple threads are trying to perform any of the two synchronized operations, they need to wait for the lock to be released.

I came with a second approach using separated locks for scheduling and registration:

...
private final Object scheduleLock = new Object();
private final Object registerLock = new Object(); 
...

@Transactional
public void scheduleMeeting(Long meetingId, Date meetingDate) {
    synchronized (scheduleLock) {
        Meeting meeting = repository.findById(meetingId);
        Account account = meeting.getAccount();

        if(account.getNumSchedules() + 1 <= account.getMaxSchedules() 
            && meeting.getStatus() != SCHEDULED) {
            meeting.setDate(meetingDate);
            account.setNumSchedules(account.getNumSchedules()+1);
            // save meeting and account here
        }

        else { throw new MaxSchedulesReachedException(); }
    }
} 

@Transactional
public void registerAttendee(Long meetingId, String name) {
    synchronized (registerLock) {
        Meeting meeting = repository.findById(meetingId);
        if(meeting.getNumAttendees() + 1 <= meeting.getMaxAttendees() 
            && meeting.getStatus() == SCHEDULED) {
            meeting.setDate(meetingDate);
            meeting.setNumAttendees(account.getNumAttendees()+1);
            repository.save(meeting);
        }

        else { throw new NoMoreAttendeesException(); }
    }
} 
...

With this I solved the inter-operations blocking problem, which means that a thread that wants to register shouldn't be blocked by a thread that is scheduling.

Now the issue is that a thread that is working on scheduling a meeting in one account shouldn't be locked by a thread trying to schedule a meeting from a different account, the same can be said for registering attendees to different meetings in the same account.

For fixing that, I came with a design that I haven't implemented yet but the idea is to have a lock provider, something like:

@Component
public class LockProvider {
    private final ConcurrentMap<String, Object> lockMap = new ConcurrentHashMap();

    private Object addAccountLock(Long accountId) {
        String key = makeAccountKey(accountId);
        Object candidate = new Object();
        Object existing = lockMap.putIfAbsent(key, candidate);
        return (existing != null ? existing : candidate);
    }

    private Object addMeetingLock(Long accountId, Long meetingId) {
        String key = makeMeetingKey(accountId, meetingId);
        Object candidate = new Object();
        Object existing = lockMap.putIfAbsent(key, candidate);
        return (existing != null ? existing : candidate);
    }

    private String makeAccountKey(Long accountId) {
        return "acc"+accountId.toString();
    }

    private String makeMeetingKey(Long accountId, Long meetingId) {
        return "meet"+accountId.toString()+meetingId.toString();
    }

    public Object getAccountLock(Long accountId) {
        return addAccountLock(accountId);
    }

    public Object getMeetingLock(Long accountId, Long meetingId) {
        return addMeetingLock(accountId, meetingId);
    }
}

However this approach involves lot of extra work on maintaining the map, for example making sure to dispose no longer used locks when an account, meeting is deleted or they come to a state where no more synchronized operations can be accomplished.

The question is if it is worth implementing that or if there is a more efficient way of doing it.

raspacorp
  • 5,037
  • 11
  • 39
  • 51
  • Going back to the start, why does the `service` need to be a singleton? – Scary Wombat Mar 08 '17 at 01:39
  • It is a Spring bean so by default it is a singleton, that makes the framework more efficient, especially because those services are wired with rest controllers that are singletons as well, that way we reuse instances when having high number of concurrent services running. But besides that, even if the service was not a singleton and we had a new instance every time, I don't see how that can solve the issue. – raspacorp Mar 08 '17 at 01:45
  • All your synchronization stuff is flawed. You don't need to synchronize the methods of your service, because it might happen (and it is highly desirable) that different meetings can be scheduled at the same time. Same for registering atendees: you want to let different atendees register to different meetings at the same time. And your synchronized methods precisely avoid this. Remove all this stuff and let the database decide instead, ideally with an optimistic locking mechanism (i.e. Hibernate implements one by default). – fps Mar 08 '17 at 01:50
  • @raspacorp see http://stackoverflow.com/questions/2637864/singleton-design-pattern-vs-singleton-beans-in-spring-container – Scary Wombat Mar 08 '17 at 01:51
  • @ScaryWombat Pure singleton vs Spring singleton has nothing to do here. The real problem is that synchronizing is not needed in a well-designed web application, as long as your services and domain entities are stateless. This is because the state is stored in the database, transactionally. What most ORMs do is to use a check-and-set mechanism, which checks if another transaction has modified the entities involved since the current transaction has loaded the them. – fps Mar 08 '17 at 01:59
  • @FedericoPeraltaSchaffner I agree with you. I posted the link to hopefully instruct the OP that what he thinks he needs is not at all necessary or even correct. – Scary Wombat Mar 08 '17 at 02:05
  • @ScaryWombat Ups, I misunderstood your comments ;) – fps Mar 08 '17 at 02:07
  • @FedericoPeraltaSchaffner we don't want to exceed the maximum number of schedules or attendees, that is the main point behind synchronizing those operations. Can you elaborate your comment or post an answer? what method would you use to make the db or hibernate to accomplish that validation? – raspacorp Mar 08 '17 at 02:07
  • @ScaryWombat your link is about Spring singleton, how does that help to solve my problem? or maybe you don't understand what my problem is or I am not explaining myself properly, it is not about the service being a singleton. – raspacorp Mar 08 '17 at 02:11
  • If you spring beans are request scoped then Spring will take care of multi-threading them - no synchronization is required. – Scary Wombat Mar 08 '17 at 02:15
  • see [this](http://stackoverflow.com/questions/33852382/constraint-on-table-to-limit-number-of-records-to-be-stored) regarding limiting the amoun of child rows – Scary Wombat Mar 08 '17 at 02:18
  • @raspacorp Use either a `CHECK` constraint in the database (if it's supported) or a trigger. My point is: let the database do all the sync stuff and do not keep state in your services. Regarding the optimistic locking mechanism, Hibernate does all the job for you if all your entities have a column named `version` in the database. See [here](https://docs.jboss.org/hibernate/orm/5.0/devguide/en-US/html/ch05.html) for optimistic locking in Hibernate. Other ORMs use similar approaches. – fps Mar 08 '17 at 02:28
  • @FedericoPeraltaSchaffner maybe u are not getting the problem, its not limiting the amount of child rows, yes we could use triggers or sprocs as you suggest, but I am not able to follow that path, this a microservice application that goes through a series of envs before prod, some environments use in memory or embedded databases to avoid db servers, prod will use a dbserver, different dbs that may not support triggs in the other hand toadd logic in the db layer adds more maintenance burden especially in the db layer and that is something we try to avoid and one of the reasons of using an ORM. – raspacorp Mar 08 '17 at 02:40
  • @ScaryWombat this is not about limiting amount of child rows and unfortunately cannot add logic to the db layer as I don't want to add another layer of logic. That is why I use synchronized methods, please read the full question, the problem is specifically avoiding having more schedules than allowed per account and more attendees per meeting than allowed. – raspacorp Mar 08 '17 at 02:43
  • @raspacorp well I am really confused as you wrote *The main issue here is avoiding to exceed the maximum number of allowed operations* It looks like I can not help you. – Scary Wombat Mar 08 '17 at 02:47
  • @ScaryWombat I know about spring beans I've worked with Spring for many years and used all the kinds of bean scopes, the issue has nothing to do with the nature of the Spring beans or if they are singleton or not. – raspacorp Mar 08 '17 at 02:48
  • like I said *It looks like I can not help you.* – Scary Wombat Mar 08 '17 at 02:50
  • @ScaryWombat Sorry maybe my question is to verbose, what I am concerned is that two threads A and B try to schedule a meeting (one account can have many meetings associated but it allows only a max number of meetings to be scheduled) so lets suppose that thread A is in the middle of scheduling a meeting for account 1 and that the current number of schedules is 9 but the max number of allowed schedules is 10, so thread A will schedule the last meeting. But at the same time and before A finishes thread B asks for the current number (9) and tries to schedule the meeting, the max will be exceeded. – raspacorp Mar 08 '17 at 02:54
  • @raspacorp Fair enough, I understand you don't want to have logic inside the database, plus the non production environments, etc. However, I think you're not getting the point of optimistic locking. You don't need to synchronize anything if you use it (and you should!). If thread B somehow modifies one of the entities involved in thread A, the transaction of thread B is assured to fail by the underlying mechanism. – fps Mar 08 '17 at 03:01
  • @raspacorp Can you seriously create a `Meeting` without a reference to an `Account` or is an `Account` non-optional in this case? Your mapping seems to indicate it can be null. – Naros Mar 08 '17 at 03:33
  • @Naros Account is not optional it must be set for creating a meeting, I will add the optional = false to the manytoone annotation, the column annotation already has the nullable = false – raspacorp Mar 08 '17 at 03:53
  • 1
    You should do that because if you dont, the JPA provider will always use a LEFT OUTER JOIN rather than an INNER JOIN. In other words, by not specifying optional = false, you end up hurting the query performance :) – Naros Mar 08 '17 at 03:55
  • Good call @Naros thanks – raspacorp Mar 08 '17 at 03:58
  • @FedericoPeraltaSchaffner optimistic locking may work, for example I could use JPA mechanism by annotating numSchedules and numAttendees with Version annotation, it will make fail the concurrent transactions that are updating a just updated version, it only solves the part of the concurrent updates, but the problem with this approach as I am reading it is the retry logic that I would need to implement. It seems that the auto retry mechanism is problematic when working with Spring transactional methods. I need to evaluate if that approach has a real advantage or not over the other idea. – raspacorp Mar 08 '17 at 04:06
  • 2
    Its not that difficult to catch the `OptimisticLockException` in the controller and have it resend the request to your service again. Put this inside a loop that tries X times before you break with a user-failure message. I wouldn't define this inside the service because there may be situations where you'd rather call scheduleMeeting and fail without any retries. – Naros Mar 08 '17 at 04:30
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/137515/discussion-between-naros-and-raspacorp). – Naros Mar 08 '17 at 05:41

1 Answers1

2

Perhaps this is partly a domain logic problem.

Given that a Meeting cannot be created without an actual reference to an Account, I'd recommend a slight change in your business logic here:

@Transactional
public void scheduleMeeting(MeetingDTO meetingDto) {
  // load the account
  // force increment the version at commit time.
  // this is useful because its our sync object but we don't intend to modify it.
  final Account account = accountRepository.findOne( 
      meetingDto.getAccountId(),
      LockMode.OPTIMISTIC_FORCE_INCREMENT
  );

  if ( account.getMeetingCount() > account.getMaxMeetings() ) {
    // throw exception here
  }

  // saves the meeting, associated to the referenced account.
  final Meeting meeting = MeetingBuilder.from( meetingDto );
  meeting.setAccount( account );      
  meetingRepository.save( meeting );
}

So what does that code do exactly?

  1. Fetch using OPTIMISTIC_FORCE_INCREMENT.

    This basically tells the JPA provider that at the end of the transaction, the provider should issue an update statement for that Account incrementing the @Version field by one.

    This is what forces the first thread that commits its transaction to win while all others will be considered late to the party and therefore will have no choice but to rollback due to an OptimisticLockException being thrown.

  2. We verify that the maximum meeting size hasn't been met. If it has, we throw an exception.

    I illustrate here that perhaps #getMeetingCount() might use a @Formula to calculate the number of meetings associated with the Account rather than relying on fetching a collection. This proves the fact that we don't need to actually modify Account for any of this to work.

  3. We save the new Meeting associated with the Account.

    I assume the relationship here is uni-directional, thus I associate the Account to the meeting prior to saving it.

So why do we not need any synchronization semantics here?

It all comes back to (1). Basically whichever transaction commits first will succeed, others will end up throwing an OptimisticLockException, but only if multiple threads try to schedule a meeting associated to the same account simultaneously.

If two calls to #scheduleMeeting come in serially to where their transactions do not overlap, there won't be an issue.

This solution will out perform any other solution because we avoid using any form of database or kernel-mode locks.

Naros
  • 19,928
  • 3
  • 41
  • 71
  • Looks good, but it is still not clear to me how it could be selective only to the specific account or meeting, you are forcing the increment of version when fetching, however isn't it true that when annotating the field w @Version it will take any update and increment the version? we have more than those 2 operations, I am showing only schedule and register the synchronized ones but if another operation is updating the other fields in the same account, we don't want the schedule operation to fail, as the former is not affecting the current number of schedules. – raspacorp Mar 08 '17 at 05:22
  • 1
    Then apply data normalization here. Create an `AccountMeetingDetails` that is a `@OneToOne` to `Account` and add the `@Version` and meeting stuff there associated to the account. This way you can increment that version without impacting other business domain things. – Naros Mar 08 '17 at 05:40