last day (15 days later) » 

1:32 PM
0
Q: How to ensure validity of foreign keys in Postgres

ChrisMMUsing Postgres 10.6 The issue: Some data in my tables violates the foreign key constraints (not sure how). The constraints are ON DELETE CASCADE ON UPDATE CASCADE On a pg_dump of the database, those foreign keys are dropped (due to being in an invalid state?) A pg_restore is done into a blank ...

 
lau
Unless the table was first filled, then the foreign key was later added with the NOT VALID option, right?
 
The foreign key existed from the beginning. I've edited the post to include the relevant part of one of the tables.
 
lau
What does the following query returns for your table SELECT relname, relrowsecurity FROM pg_class where relname='tasks' ?
 
relrowsecurity is false.
 
lau
Can you qualify all the tables from your query with the schema name? I am talking about SELECT * from myschema.tasks. It could have been a bug in version 10.6 but I have never heard of postgresql allowing a primary key and a foreign key to have different types. Hence I believe the most probable so far is that you are not selecting from the tables you believe.
 
1:32 PM
Sorry, I'm confused by your comment. From the attached image in the post, you can see the query I'm using, and indeed it is selecting from the tasks and dependencies table. If I do a SELECT * from tasks WHERE id = 1573022 (first result from screen shot), I get no results. FYI, this seems take have been happening with versions of at least 10.4, I've tried updating twice since and still get the issue.
 
lau
What I'm saying is that you need to replace dependencies with yourschema.dependencies and tasks with yourschema.tasks in your query (with the correct schema name, obviously). You are using pgAdmin so the schema name (for possibly several schemas) will be on the left.
 
Ah, sorry, I understand now. Everything is just in public. and if I add that in, its the same results.
 
lau
Stupid question: are you sure task_id is not NULL?
 
Yes, it's a primary key, serial, not null in the tasks table.
 
lau
nope nope it's not the primary key
from the CREATE TABLE you posted, task(id) is the primary key, not task(task_id)
You must have 2 fields and are confused because they have similar name
 
1:35 PM
Oh, yes, sorry. Too many with similar names. task_id is TEXT NOT NULL
I also did the same query returning tt.id and ft.id and they're also null
 
lau
OK ....
EXPLAIN ANALYZE executed from pgadmin does not show anything fishy?
everything in public like you expect?
 
Nothing fishy.
max(id) from tasks is only 27532 though. So the Ids from dependencies are way way off.
 
lau
I have only 2 ideas left but technically speaking this is more advanced
First is about the tablespace, anything worth mentionning in SELECT * FROM pg_tablespace ?
 
I've never used pg_tablespace, but there's just two records. pg_default and pg_global
 
lau
ok then for my last idea (which is a bit unlikely), let me first try to see if I can reproduce a situation
 
1:47 PM
Okay.
I'm actually not even sure how to reproduce the issue. One of my users keeps causing it randomly, then when she sends us projects, I can see the affect, but I can never see the cause.
 
lau
Let's try this
SELECT relname, relforcerowsecurity, relispartition from pg_class where relname in ('tasks', 'dependencies')
 
"dependencies" false false
"tasks" false false
 
lau
Try to start a transaction, run the query here and then re-run the query you have built
SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL READ COMMITTED
and try again after SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
 
still the same.
 
lau
yet I'm just like Raymond, I can't believe there's a bug on foreign keys
 
2:01 PM
I agree, it does seem very odd to me. It only seems to happen to one user (who in turn spreads it with the pg_dumps).
 
lau
either the data is correct and we're missing something obvious or some PK values are indeed missing but the only way this could happen is by dropping the FK first
mmm
task(id) is of type bigint too?
not integer?
 
Yes, everything was updated to bigint a while back (but after the initial creation as int)
The FK exists in the DB, but does not exist in the dump. So it seems that when the data gets output, and loses the FKs, then all the UPDATEs of the IDs in the dumped DB do not affect the foreign keys. Therefore, when reimported into the DB which has the FKs, there's mis-matches.
So I can understand how the DB gets corrupted when I import hers, but I cannot understand how hers gets to that point.
 
lau
I think I need to spend some time thinking on the case
Everything that could explain what you see, we've ruled them out already
I would suggest you edit your question with everything we've tried, in case somebody else starts trying to answer you with the same suggestions
 
Will do, thank you for your help.
 
lau
In the meantime, if you can avoid deleting data ...
also, change the screenshot to show id, not task_id :)
btw I was reading again your message
do you know what options were used when executing pg_dump?
Some (e.g. --section= ...) may not keep the foreign keys
 
2:19 PM
Nothing special:
arguments << "--file=" + fileName << args << "--username=" + connection.userName() << databaseName << "--format=c"
 
lau
ok
This is going to prevent me from sleeping, I'll give it more thoughts for sure and let you know if I have any interesting ideas
see you bye
 
Thanks again.
Try to get some sleep. It's not that important!
 

last day (15 days later) »