last day (26 days later) » 

5:38 PM
Hi
Maybe you can first explain me what the relationship is between hashes and emails. Is it one-to-many, many-to-one, or many-to-many?
 
5:59 PM
Sure!
For all child tables, it's a many to one relationship
So many emails to one id
 
but
you said that in hashes the id is not unique, so it really is many emails belong to one group of several hashes, no? That is many-to-many
 
Well many emails belong to one id which belongs to many hashes
The hashes table was supposed to act like a bridge to avoid many-to-many
But I think I set it up incorrectly
 
I think you should create an intermediate table. Something like hashgroup. Then link the emails to that. But also individual belonging to the same group will be linked to it.
*individual hashes,I meant to say
 
that was the point of the id column though. it groups all the many hashes to one id, and then many children point to one id
Let me back up a bit and explain the whole database
 
Yes, that is true. I listen...
 
6:12 PM
The hashes represent unique data. Many hashes are grouped together to uniquely identify a single entity (the entities could be businesses, for instance, so many hashes uniquely identify a business). All these hashes belong to the same entity (identified by the id column), so anytime a new group of hashes are added, I merge the entity ids together. Then, child tables have data that all point back to the entity ids, but not the hashes.
 
OK, what about this:
I would rename id to entityid, but consider the entity itself something that does not have its own proper table... it is just the entityid. Then as you insert a new group of hashes, first check if one of these hashes already exists in the hash table. If so, don't create a new entityid, but use the found one. In the other case, create a new one. That way you don't have to cascade update, right?
 
Haha that was my original implementation, but happens if there are two entity ids present (entity1, entity2), then a hash comes along that belongs to both entity1 and entity2. If the new batch inherits, say, entity1 for its entityId, then all rows referencing entity2 must be updated to entity1, bringing us back to the problem we started with
 
But:
If that happens you would want all three groups to become one. Is there no way to already know when you were inserting the second group that it actually belonged together with the first?
 
6:28 PM
That's the trickiness of this database: I do not know what groups are linked until a subsequent group comes along and links them. This is probably unconventional, but I'm dealing with (eventually) a large amount of data and need to limit redundancies.
Thinking of this in terms of business entities is a good example: Let's say UserA has the name and website of ABC corp. UserB has the email and phone number of ABC corp. On their own, there's no way to say "these two entities are the same". But UserC comes along and has UserA's website and UserB's email, therefore linking all three groups together.
 
So that means that while UserC has not yet come along, emails and other related data are not identified as belonging to the same entity. But apparently your application can live with that. So why does this need to change when UserC comes a long? Is data redundancy really the reason? My point is: if you could live with it before UserC pass by, why not after?
 
That's a good point, but yes, data redundancy is really the reason. The front-end that ends up interfacing with the database takes note of any late in the game merges. Essentially the front-end updates when a merge occurs. Also, the same process of UserC coming along is the same as UserB coming along with UserA's website (if that makes any sense)
 
6:44 PM
OK, I see. I will now have to think this over :)
 
Haha thank you so much, I honestly appreciate you helping me, even if it's just talking it through like we just did
 
A question:
You speak of compositie key and the issue when updating the id you could get a duplicate key. I understand this, but does this also apply for the emails table?
 
Yes, so if you look at the example I gave in my question, assume the value column contains emails and not numbers, then its the same situation
 
ah so email is part of the composite key. It is not defined as such in your table creation script, but it makes sense of course.
 
Ah yeah, I must've removed it by accident. The composite key is the id and then value for whatever child table
 
6:58 PM
But lets say you are about to insert a record with same as existing email, but with different id. I suppose that leads to the same hash and thus a merge? If you would first do the hash table insert and only then the insert in dependent tables, then the original problem with composite keys would be reduced to a plain duplicate when inserting, no? ... which is simpler to handle.
To expand on this:
I mean when you do the hash table insert, and all the merges needed, getting the new id, and somehow applying that new id as replacement of existing ids in dependent tables, then ...
the (id, email) you are about to insert will already exist.
 
7:13 PM
In the case of the email that would lead to the same hash. Although most hashes are combinations of data, like business name+phone number. Does that make sense?
 
OK, and what would be the composite key in such a table? id, business_name, phone?
 
Well those are two different tables, business name and phone
 
I see, business_name would form a composite key with id?
 
7:30 PM
Correct, as would phone.
 

  last day (26 days later) »