97

How do I design the database to calculate the account balance?

1) Currently I calculate the account balance from the transaction table In my transaction table I have "description" and "amount" etc..

I would then add up all "amount" values and that would work out the user's account balance.


I showed this to my friend and he said that is not a good solution, when my database grows its going to slow down???? He said I should create separate table to store the calculated account balance. If did this, I will have to maintain two tables, and its risky, the account balance table could go out of sync.

Any suggestion?

EDIT: OPTION 2: should I add an extra column to my transaction tables "Balance". now I do not need to go through many rows of data to perform my calculation.

Example John buys $100 credit, he debt $60, he then adds $200 credit.

Amount $100, Balance $100.

Amount -$60, Balance $40.

Amount $200, Balance $240.

001
  • 62,807
  • 94
  • 230
  • 350
  • 1
    What are your expected volumes for transactions ? – iDevlop Dec 07 '10 at 06:37
  • wtf is that question where everyone gets negative points ?? – iDevlop Dec 07 '10 at 07:14
  • no idea iDevelop, I didn't give anyone a positive or a negative point :), now im confused, one guy says yes, another says no. What is going on! ?? – 001 Dec 07 '10 at 07:18
  • @iDevlop, to answer your question...since the transactions are never deleted from the previous years, I would expect a few million records. – 001 Dec 07 '10 at 07:20
  • 7
    The problem with the Balance field in Transactions table, is that you not only build a transitive dependency at row level, which is not "Normal" but you add a transitive dependency at column level, which will be a headache if you ever have a problem (trigger failing or other). My advice is to write down your *normalised* structure, then write every "Use case" you plan to have, discuss them with others, then review your structure at the light of your use cases to see if some denormalisation is required. Anyway, design stage is crucial, taking your time at that point is not "losing" time! – iDevlop Dec 07 '10 at 08:11
  • 7
    "transactions are never deleted from the previous years"...I would thing twice about this. You might consider moving the old transactions to an archive table after some time, + create a special type of transactions (initialBalance) in the active table. That could be part of a yearly process (or any appropriate time frame). And you *should* include that point in your "Use cases" ;-) – iDevlop Dec 07 '10 at 08:16
  • Move it now, doesn't seem to be a good choice, it makes things complicated and time consuming which will work out to be more costly than it has to... Maybe something to consider later on! right now its in 1 table :) – 001 Dec 07 '10 at 17:06
  • I highly recommend everyone see [this](https://stackoverflow.com/a/29713230/3650835) answer. It is the correct way to do things, from a DBA who has worked in the financial sector for decades. – KayakinKoder Mar 26 '19 at 05:15

9 Answers9

83

An age-old problem that has never been elegantly resolved.

All the banking packages I've worked with store the balance with the account entity. Calculating it on the fly from movement history is unthinkable.

The right way is:

  • The movement table has an 'opening balance' transaction for each and every account. You'll need this in a few year's time when you need to move old movements out of the active movement table to a history table.
  • The account entity has a balance field
  • There is a trigger on the movement table which updates the account balances for the credited and debited accounts. Obviously, it has commitment control. If you can't have a trigger, then there needs to be a unique module which writes movements under commitment control
  • You have a 'safety net' program you can run offline, which re-calculates all the balances and displays (and optionally corrects) erroneous balances. This is very useful for testing.

Some systems store all movements as positive numbers, and express the credit/debit by inverting the from/to fields or with a flag. Personally, I prefer a credit field, a debit field and a signed amount, this makes reversals much easier to follow.

Notice that these methods applies both to cash and securities.

Securities transactions can be much trickier, especially for corporate actions, you will need to accommodate a single transaction that updates one or more buyer and seller cash balances, their security position balances and possibly the broker/depository.

smirkingman
  • 6,167
  • 4
  • 34
  • 47
  • 7
    "Personally, I prefer a credit field, a debit field and a signed amount, this makes reversals much easier to follow.", why not just have 1 field "Amount" with the sign? – 001 Dec 07 '10 at 17:02
  • "All the banking packages I've worked with store the balance with the account entity." how do they do that extactly? this would produce multiple account records as the balance are updated. Im looking at this diagram right now, http://www.databaseanswers.org/data_models/online_banking/index.htm It seems to be incomplete, but if this person was to store the balance information he would store it under the "Balance_history" table (to me this feels more like the right thing to do). – 001 Dec 07 '10 at 17:25
  • From his diagram, ... Transaction table stores the transaction information, including the "amount" field, and the balance history table, stores the updated balance. It feels somewhat right, however not feeling that its completely correct, ie How would I know which transaction was the transaction that affected the balance? – 001 Dec 07 '10 at 17:30
  • @001 "why not just have 1 field "Amount" with the sign?" I don't think you read my answer carefully. I said: "Personally, I prefer a credit field, a debit field and a signed amount" – smirkingman Dec 07 '10 at 20:18
  • @001 "databaseanswers.org/data_models/online_banking/index.htm". Don't take everything you find on the internet as gospel truth. That diagram, is, how should I put it, **very** naïve... – smirkingman Dec 07 '10 at 20:21
  • >>>"@001 "why not just have 1 field "Amount" with the sign?" I don't think you read my answer carefully. I said: "Personally, I prefer a credit field, a debit field and a signed amount"<<<---- Credit field, Debit field, and Amount field, this would mean you would have 3 fields, what exactly do you use the credit and debit field for? example. – 001 Dec 08 '10 at 03:53
  • 11
    @001 Each transaction contains 3 fields. 1/ The ID of the account being debited. 2/ The ID of the account being credited. 3/ The amount, positive=transfer from creditor to debitor, negative=transfer from debitor to creditor (usually a reversal). I suggest you read http://en.wikipedia.org/wiki/Double-entry_bookkeeping_system – smirkingman Dec 08 '10 at 09:05
  • If its done your way with credit and debit account id, do note that when money moves, sometimes there are more than two accounts involved. Example $5500 total from Account X, $5000 to Account Y, $500 to Account Z (TAX). What are you going to do in this case? – 001 Dec 09 '10 at 04:05
  • 1
    @001 That is 2 movements. 1/ X credits 5000 to Y. 2/ X pays 500 tax to Z. It must be done this way, as later, someone will ask 'show all the payments to the tax account Z' and you don't want the 5000 in that answer. I really suggest that you do a bit of background reading on book-keeping, you'll find it very helpful – smirkingman Dec 09 '10 at 09:21
  • In your solution... the amount is only negative for resverals everywhere else even in a payment or refund it's positive! Confirm if this is correct. – 001 Dec 09 '10 at 14:01
  • @smirkingman I like your solution better than mine actually, its more efficient than mine, just got to understand a little more about the amount sign. – 001 Dec 10 '10 at 05:15
  • @smirkingman, in your solution, amount sign could actually be all positive, even for transactions that are reserved, since you would have to add another entry anyways for reserval ie you will want to know how much and how often you reverse transactions. Is this correct? if not please state if the amount could be negative in any other entry besides reserved transactions and why exactly is it better? to have a negative sign. – 001 Dec 10 '10 at 08:09
  • 1
    I think the discussion is getting out of scope, hence I opened a new topic to address this question. http://stackoverflow.com/questions/4415022/database-design-accounting-transaction-table – 001 Dec 11 '10 at 03:29
  • @smirkingman and Mr. 001, thanks for this inspirational dialog. Just want to confirm, if I also need to record how many transactions ever happened for a specific account, would I have to create ONE EXTRA record for each transaction therefore double the transaction table size? For example, along with "$100 from account X (to account Y)", do I need "1 transaction happened for account X" too? Because I can not simply sum-up how many rows in transaction tables for that, when in future some of the old transactions would be moved out. – RayLuo Aug 05 '13 at 05:37
  • @iceberg If you need to know, forever, how many transactions were made, then you logically can never archive old ones: Suppose the client sees '560 transactions on this account'; it's then reasonable that he'll say 'show me them all'. If you must archive then keep the transaction count as a field in the account table (and be prepared for headaches a few years later ;-) – smirkingman Aug 19 '13 at 19:08
  • @smirkingman Sorry did not see your feedback in time. But your feedback doesn't comply with your own answer. Your answer suggests "an open balance on each account" and then "move old movements out of the active movement table to history table". In this case the client sees for example "560 dollars on my balance" but they won't get answer if they ask "show me how I end up this figure since I open this account 20 years ago", will they? And I am just following your idea. My question was specifically "do I need to store transaction counter as one more account". Can you share thoughts on this one? – RayLuo Apr 04 '14 at 06:05
  • @iceberg Your 'history' table is no different to your current transaction table. It's just in a different file, with a final balance record which allows you to 'pick up where you left off' without re-calculating the balance from those historical transactions. If you need a complete transaction history, you have to keep each transaction; the history serves to avoid re-reading them all on the occasional "what did I have 20 years ago?" question. The number of transactions is, in itself, of no interest; it's the balance that counts – smirkingman Apr 06 '14 at 20:58
  • 11
    This answer is incorrect, code-dependent, and cumbersome. For a method that (a) **complies with audit requirements** and legislature, (b) **doesn't** require triggers; offline safety nets; duplicate columns; duplicated data, and (c) performs well regardless of table population, look at **[this Answer](http://stackoverflow.com/a/29713230/484814/)**. – PerformanceDBA Apr 18 '15 at 06:33
  • @smirkingman Can you please explain how the notion of "locking" money on a credit card can be added here? Often before money are taken as a payment they a locked first, which means they are still on your account but you cannot use them (as far as i know.) – vwvolodya Jan 04 '18 at 09:03
  • @vwvolodya My account balance is $500 on the 1st of January. I reserve a room for the 10th, the Hilton locks $100 for my room, an anticipated debit. My available balance is $400, until either 1/ Hilton debits the $100 or 2/ the Hilton re-credits the blocked $100. On the 2nd of January I reserve a $100 room at the Ritz, my balance is $300, etc. On the 4th I cancel the room at the Hilton, they re-credit and my balance is $400. 'Locking' is a way of expressing "promise to pay at some future date"; promise=guarantee, so my account must be debited until the promise is either fulfilled or rescinded. – smirkingman Jan 07 '18 at 21:10
  • @smirkingman yes, i understand how this concept works. I wanted to ask about applying this principle in the scope of this discussion and transaction table. Should locking be a different type of transaction? How would you implement locking in DB design? Thanks. – vwvolodya Jan 07 '18 at 21:48
  • @vwvolodya I think that 'locking' is not the best word to describe money set aside for a future debit, as it suggests 'locking' at the database level. There is no 'lock' in a database sense, the fact that $100 have been debited ensures that the money is set aside. There is no difference between $100 debited "ahead of time" by the hotel and a straight $100 cash withdrawal, the money is debited up front. The only difference is when you cancel the booking and the hotel *re-credits* the 100$ later on. – smirkingman Jan 08 '18 at 16:08
4

This is a database design I got with only one table for just storing a history of operations/transactions. Currently working as charm on many small projects.

This doesn't replace a specific design. This is a generic solution that could fit most of the apps.

id:int standard row id

operation_type:int operation type. pay, collect, interest, etc

source_type:int from where the operation proceeds. target table or category: user, bank, provider, etc

source_id:int id of the source in the database

target_type:int to what the operation is applied. target table or category: user, bank, provider, etc

target_id:int id of the target in the database

amount:decimal(19,2 signed) price value positive or negative to by summed

account_balance:decimal(19,2 signed) resulting balance

extra_value_a:decimal(19,2 signed) [this was the most versatile option without using string storage] you can store an additional number: interest percentage, a discount, a reduction, etc.

created_at:timestamp

For the source_type and target_type it would be better to use an enum or tables appart.

If you want a particular balance you can just query the last operation sorted by created_at descending limit to 1. You can query by source, target, operation_type, etc.

For better performance it's recommended to store the current balance in the required target object.

Heroselohim
  • 1,241
  • 1
  • 18
  • 23
4

You should store the current account balance and keep it up to date at all times. The transaction table is just a record of what has happened in the past and shouldn't be used at a high frequency just to fetch the current balance. Consider that many queries don't just want balances, they want to filter, sort and group by them, etc. The performance penalty of summing every transaction you've ever created in the middle of complex queries would cripple even a database of modest size.

All updates to this pair of tables should be in a transaction and should ensure that either everything remains in sync (and the account never overdraws past its limit) or the transaction rolls back. As an extra measure, you could run audit queries that check this periodically.

Marcelo Cantos
  • 181,030
  • 38
  • 327
  • 365
  • Tell that the peopel coming to audit your bookkeeping. You will be surprised how long they laugh. Every transaction to an account musst be numbered (to maintain an order) and you can just put the new account balance right in there. No need for a second table actually. No performance penalty. Doubles, btw., how bookkeeping is done. Which this is all about. – TomTom Dec 07 '10 at 06:51
  • @TomTom: now I see your point. Your idea is good. Too bad it was not clearly said in your answer, and too bad your answers look so agresssive ! – iDevlop Dec 07 '10 at 08:00
  • 1
    @TomTom, Marcelo's wording is a bit ambiguous, but your comments on auditing are not helpful, nor right. Audit is about establishing correctness and have little to do with optimisation of speed. Marcelo is a bit imprecise because he talks about 'current balance', where in reality most financial systems will keep balances according to account and other analytical dimensions summed up by certain date granularity. Your idea of keeping a running balance on transaction level is useless for any report that would need to filter transactions on anything but attribute that conforms to order of booking. – Unreason Dec 07 '10 at 08:58
2

Of course you need to store your current balance with each row, otherwise it is too slow. To simplify development, you can use constraints, so that you dont need triggers and periodic checks of data integrity. I described it here Denormalizing to enforce business rules: Running Totals

A-K
  • 16,804
  • 8
  • 54
  • 74
2

A common solution to this problem is to maintain a (say) monthly opening balance in a snapshot schema. Calculating the current balance can be done by adding transactional data for the month to the monthly opening balance. This approach is often taken in accounts packages, particularly where you might have currency conversion and revaluations.

If you have problems with data volume you can archive off the older balances.

Also, the balances can be useful for reporting if you don't have a dedicated external data warehouse or a management reporting facility on the system.

ConcernedOfTunbridgeWells
  • 64,444
  • 15
  • 143
  • 197
0

Here is would like to suggest you how can you store your opening balance with a very simple way:-

  1. Create a trigger function on the transaction table to be called only after update or insert.

  2. Create a column having name in the master table of account naming Opening Balance.

  3. save your opening balance in array in the opening balance column in master table.

  4. you even not need to use server side language use this store array simply you can use database array functions like available in PostgreSQL.

  5. when you want to recalculate you opening balance in array just group your transaction table with array function and update the whole data in the master table.

I have done this in PostgreSQL and working fine.

over the period of time when your transaction table will become heavy then you can partition for your transaction table on the base of date to speed up the performance. this approach is very easy and need not to use any extra table which can slow performance if joining table because lesser table in the joining will give you high performance.

Mark Taylor
  • 1,843
  • 14
  • 17
  • hi, I am not clear with no.3. "opening balance" is in the same transaction table or a separate table ? – Axil Aug 04 '14 at 05:21
0

Your friend is wrong and you are right, and I would advise you don't change things now.
If your db ever goes slow because of this, and after you have verified all the rest (proper indexing), some denormalisation may be of use.
You could then put a BalanceAtStartOfYear field in the Accounts table, and summarize only this year records (or any similar approach).
But I would certainly not recommend this approach upfront.

iDevlop
  • 24,841
  • 11
  • 90
  • 149
  • Ah... no, seriously. Accounting isa tricky busines to get right with possibly some hefty legal requirements. Ad hoc summations do not cut it. – TomTom Dec 07 '10 at 06:53
  • 2
    @TomTom. Of course they do, if the Transactions are implemented according to Accounting Standards (yours aren't), in which case "ad hoc" is false. Besides, duplicating the CurrentBalance in every single row is plain stupid: when an adjustments has to be made, masses of rows have to be updated. – PerformanceDBA Apr 18 '15 at 06:22
0

My approach is to store the debits in a debit column, credit in the credit column and when fetching the data create two arrays, debit and credit array. Then keep appending the selected data to the array and do this for python:

def real_insert(arr, index, value):
    try:
        arr[index] = value
    except IndexError:
        arr.insert(index, value)


def add_array(args=[], index=0):
    total = 0
    if index:
        for a in args[: index]:
            total += a
    else:
        for a in args:
            total += a
    return total

then

for n in range(0, len(array), 1):
    self.store.clear()
    self.store.append([str(array[n][4])])
    real_insert(self.row_id, n, array[n][0])
    real_insert(self.debit_array, n, array[n][7])
    real_insert(self.credit_array, n, array[n][8])
    if self.category in ["Assets", "Expenses"]:
        balance = add_array(self.debit_array) - add_array(self.credit_array)
    else:
        balance = add_array(self.credit_array) - add_array(self.debit_array)
-3

Simple answer: Do all three.

Store the current balance; and in each transaction store the movement and a snapshot of the current balance at that point in time. This would give something extra to reconcile in any audit.

I've never worked on core banking systems, but I have worked on investment management systems, and in my experience this is how It's done.

Dog Ears
  • 9,637
  • 5
  • 37
  • 54
  • 1
    This give something *extra* to reconcile in an audit. Which helps audit the software (in terms of correct calculations and checking if transactions get deleted; and is not requirement=the only way to do it), which where I work is part of an IT audit; but this is not directly related to accounting audit (IT audit is just a part of financial audit). – Unreason Dec 07 '10 at 09:16
  • Correct, but it's not unlikely with large interoperateing system to encounter situations where certain process (an overnight batch for instance) creates anomalies; a bit of denormalisation, will help ensure that they can be tracked down. – Dog Ears Dec 07 '10 at 10:18
  • @Dog Ears, I see - so most of your entries come from data exchange/interoperation and then IT audit is much stronger component of financial audit. Interesting. Still I keep my position: this is an extra control value which would exist over all transactions. If I wanted to audit the data exchange process I would put controls that concentrate on it (using some sort of checksums of the data exchange: calculated by initiator and simply received, calculated on the received data in the received format - checks the integrity of transfer, calculated on the imported data - checks your import procedure) – Unreason Dec 07 '10 at 10:38
  • @dogs ears If you've never worked on core banking systems, you should refrain from giving (awful) advice – smirkingman Dec 07 '10 at 20:23
  • 1
    @smirkingman for the benefit of the community what's awful about my advice, for that matter what makes it different from your advice? Seems that we basically suggested the same thing? – Dog Ears Dec 07 '10 at 23:00
  • 3
    @dogs ears the awfulness is storing the balance in every movement. More duplicated data and a nightmare to fix when something goes wrong. – smirkingman Dec 08 '10 at 09:00