last day (14 days later) » 

9:02 AM
0
A: Calculating running balance from join table [SQL Database Design]

EstyIt depends. Suppose you have a report like transaction report where accounts' running balance will be shown. If you want to show real time data then always method 1 will be preferable. And I will suggest to use Quirky Update for this rather than using cursors, loops, sub-queries or recursions. ...

 
Hi, thank you for your response! The problem method 1 has is that what if you query transaction report for last week? Don't you have to calculate all of the rows just so you get your last week report?
 
For this scenario we use both methods. In our system we keep both methods. I am updating my answer for this.
@Zanko, please see update.
 
thanks for the help! One more question will it be more complicated if I were to add another field call customerID to the CumulativeBalance table. So the table will hold many customers cumulative data. I will read up on the quirky update and those stuff fully when I get back from work! Thanks. I want to clarify something, you stated that "real time data then always method 1 will be preferable with QUIRKY UPDATE". My method 1 does not have any additional table, what am I updating exactly? Thanks!
 
I would suggest to make different table to keep date wise cumulative balance by customer ID like (CustomerID, SystemOpenDate, CumulativeBalance).
And for method 1 you are not using any tables. But if you do e.g. in my solution a temp table was used for internal calculation where I used quirky update. If you don't use any tables means you are making output by direct select then obviously you can't use that. I will suggest to use window functions for that. What I meant that running total by window functions or quirky update is always give you better performance than loop, cursor, sub quires or CLRs.
Another idea about your customer ID and Date wise cumulative balance. If your customer ID balance for a date equal to summation of accounts' balance owned by that customer for that date then one table will be suffice e.g. (AccountNO, SystemOpenDate, CumulativeBalance ) and you may have already a table like (CustomerID, AccountNO). You could make a JOIN with SUM to get date wise customers cumulative balance. It would be smarter than keeping another table for customers date wise cumulative balance.
 
Thank you so much! It will take me some times to digest all the information here. I am still a little confuse on your last suggestion haha.
 
9:03 AM
No issues. Take your time. As per your information I am assuming that you are working with a core banking system.
 
Thanks, I am working for my personal office financial management system
 
oops. However almost same. Good luck.
 
But it simply, customer can pay us, we can pay customer, customer can owe us, we can owe customer
So that is like 4 tables
and there are more custom tables
the hard part is to have a central sheet that join all these information and compute a running balance.
So I think quirky update will help alot but I am still new to the concept. Does it make use of local table?
 
Before saying something about your table structure & modeling
I may need to kow your business first
Means business requirement..
 
Sure, I am always thrown off by that term haha
what do you really mean : P
 
9:07 AM
Quirky update is just a faster technique to update lots of rows and good to calculate running total
I mean that your system table structure is very closely relates to your requirement
without knowing it properly it would be tough to comment why you are having hard situation..
thats it.
mine is a core banking financial system
 
Ill try, one of the business requiremnt is that Admin can look up current customer ID and be able to the running balance
 
And your problems seem similar to me.
 
in the picture there are like 5 columns before running balance
We query by time span, Customer Name and record Type.
and the data will be shown to us
currently the read is fast because a bigger table is computed beforehand with running total
 
this is a very simple requirement.
let me see your tables first
 
my sql table?
 
9:12 AM
yap
I dont need image
just say here name and columns.
those are used to get that data
 
ok ill try haha
```
test
```
let me type in note pad first ok thanks
```
RECORD_SHEET
record_id PK
record_date
record_type (THAI or MALAY)

RECORD_USER_RELATIONSHIPS
record_id FK to RECORD_SHEET
manager_id
worker_id

RECORD_EXTERNAL_PARTY_RELATIONSHIPS
record_id FK to RECORD_SHEET
worker_id
external_party_id
customer_paper
customer_website
partner_paper
partner_website

RECORD_DATA
record_data_id PK
record_id FK to RECORD_SHEET
external_party_id
customer_paper_money_in
customer_paper_money_out
customer_website_money_in
customer_website_money_out
partner_paper_money_in
wow thats long srry about that I might have some typo
so the tricky part is joining minor details
such as CUSTOM_DATA into RECORD_DATA before including in the main table for running balance calculation
there is alot of joining going on to make the BIG table
 
9:27 AM
What is the primary key to CUSTOM_DATA table??
 
oops it should be custom_data_id, Since I can have more than one custom_data per record
 
means record data has one to many relation to custom data table.
 
yes that is correct
 
In that case your join should be simple.
 
Liek i feell like I have to join everything first and sort by date
the Sorting feels really heavy
 
9:31 AM
oopps
now I see the whole picture.
 
Let me know, If I can provide you with more details. Thank youy so much though I really appreciate you time and help!
 
All amounts are being kept in Payment IN and Payment out table.
yeah last thing is your script
means query you are currently using to generate that report.
 
If you see the image link I send you some entries are BOLDED, the BOLDED entry provide additional info. In that case it repreat the one to many custom_data
 
you r welcome and dont mention that.
yeah got it.
 
Oh the thing is I am migrating from mongoDB to PostgreSQL so my script is currently using for mongodB
I can still poost here if it helps
 
9:34 AM
wait
I am sorry then
I am no expert to MONGO
 
haha its ok, mayb I can come back after I get PostgreSQL working!
 
ok
 
maybe you can provide me with conceptual answer?
like where do I perform the quirky update
after joining everything?
 
no
 
and you gave a link on Haloween protection. Is that a supplement to quirky update that I should do too?
 
9:37 AM
If you are using postgreSQl or Mongo I doubt if any of quirky update or window functions will gonna work for you.
those are specific to sql server
 
oh my : (
 
sorry I haven't see your tags properly..!!!
:)
 
I mean I dont mind using MySQL, but people seems to like PostgreSQL for some reason
 
yeah I know that.
calculating running total is really difficult in postgre
 
When I posted I was expected table design, it seems like my problem lies in proper scripting of query
 
9:39 AM
I dont know about mongo but as far I know it should be tough too..
 
if thats the case I think I am migrating over to MySQL
I am still creating table as of now so its not much reworking..
 
look db migrating is not a solution to get performance.
you definately get more performance in mongo than sql server or mysql
 
I mean I am sureing migrating to SQL either way... sick of MongoDB lack of transaction
 
almost 4 times higher
I know
But it would be greta if you can master mongo
One of my target is to learn mongo
whatever you felling sick about it, it will be a precious to you later time.
means in future.
 
yea mongodb is nice >.< Im just not sure its for my project
 
9:42 AM
hahaha
 
I rely on transaction and roll back alot
right now if error occur halfway my mongo just saves halfway haha
 
I know ..
 
are we allow to discuss other thing while we are here?
I want to ask you about searching table haha
 
sure
 
full-text search
 
9:44 AM
go on
 
like if my table store amount
and user want to be able to search 10,000
with a comma
wait ill jsut post a picture
the blue circles are like formmated text
number is decimal formatted to text with commas, and date is just formatted to that format
 
I understand..
 
database doesnt contain these formatted text so searching directly from database doesnt help
so now I need to convert whole table into array of formatted input
and search against that array
really slow haha
 
In which datatype you are keeping your amounts in DB??
 
number
integer
 
9:47 AM
ok
Then your search should not be full text.
 
but the search field doesnt know im searching for amount or whatever
it just search whatever you type in
 
Perhaps you are getting all your data, then get it in an array with formatted input with comma, then doing full text search
 
yes thats what I am doing haha
 
Good practice is to make your param 10, 000 to 10000. remove the comma then convert it to integer.
then do search from your DB within integers
searching among String / text is lot slower than integer/number
 
yea but the search field is for general text
removing comma would be like assuming we are only searching for amount : (
 
9:52 AM
dynamic search field always creates such problem.
 
I see... so people normally dont use dynamic field? haha
Being fancy come with great cost haha
 
it is not good to use dynamic search like you are searching amount, customerType etc. with a single field.
Come on man...fancy but very bold.
 
what is the performance of full text search against database? haha
I think I can let go of searching formatted data
 
I can't say you about performance estimation btn 2 but its cost obviously high. just dont know how high
and also you are using financial data here.
 
haha thanks
btw do you prefer MySQL or PostgreSQL?
 
9:57 AM
this idea would be good for blogs but never for financial solution.
PostgreSql
of course.
However I have to go now.
Will talk later
 
Ok thanks for the chat
 
good luck.
 
thanks!
 
wc. :)
 

last day (14 days later) »