Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

As a Metric user I want opendata DB primary key to use bigint instead of integer #102

Open
VitaliStupin opened this issue Oct 30, 2023 · 1 comment

Comments

@VitaliStupin
Copy link
Contributor

VitaliStupin commented Oct 30, 2023

Opendata DB primary key sequence runs out of integer (int4) values in larger X-Road instances. Bigint should be used instead.

Software version:
1.1.1, 1.2.0

Host OS and version:
Ubuntu 20.04

The schema in anonymizer defines bigint as id column type:

But id column type is discarded during schema object creation:

Then during creation of table id column type is hardcoded to be SERIAL:

cursor.execute(f'CREATE TABLE {self._table_name} (id SERIAL PRIMARY KEY{column_schema});')

Example error log of anonymizer:

File \"/usr/lib/python3/dist-packages/opmon_anonymizer/iio/postgresql_manager.py\", line 58, in add_data
cursor.execute(query)psycopg2.errors.SequenceGeneratorLimitExceeded: nextval:
reached maximum value of sequence \"logs_id_seq\" (2147483647)

Proposed solution:

Replace id SERIAL PRIMARY KEY with id BIGINT PRIMARY KEY GENERATED ALWAYS AS IDENTITY in postgresql_manager.py because SERIAL types are soft-deprecated:
https://wiki.postgresql.org/wiki/Don't_Do_This#Don.27t_use_serial

Hotfix for existing installations (NB! This is slow operation that locks tables, may take hours to complete and requires at least 50% of free drive space):

ALTER TABLE logs ALTER COLUMN id TYPE BIGINT;
ALTER SEQUENCE logs_id_seq AS BIGINT MAXVALUE 9223372036854775807;
@raits
Copy link
Contributor

raits commented Oct 30, 2023

Hello @VitaliStupin!

Thank you for reporting this. We will look into it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants