Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postgres support #5355

Draft
wants to merge 55 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
a1189b4
Retry postgres
NodudeWasTaken Oct 2, 2024
a45658f
Better init
NodudeWasTaken Oct 3, 2024
876ad84
Query fixes
NodudeWasTaken Oct 3, 2024
2ac9768
Migration changes
NodudeWasTaken Oct 3, 2024
9f42dd1
Remove prepared
NodudeWasTaken Oct 3, 2024
5790ec1
SQL Boolean fixes
NodudeWasTaken Oct 4, 2024
61c3605
PGSQL Fingerprints fix
NodudeWasTaken Oct 4, 2024
17c5036
PGSQL fix duplicate ids in the wake of removing distinctIDs select
NodudeWasTaken Oct 4, 2024
08750b3
Undo remove prepared
NodudeWasTaken Oct 4, 2024
9d72fdd
Change rebind func
NodudeWasTaken Oct 4, 2024
b743bf5
fix prepared for postgresql
NodudeWasTaken Oct 4, 2024
0332276
nicer init
NodudeWasTaken Oct 4, 2024
58603ca
Dont fuck up the sorting
NodudeWasTaken Oct 6, 2024
0cdc191
regexp changes
NodudeWasTaken Oct 8, 2024
ae1759c
PGSQL migration fix
NodudeWasTaken Oct 8, 2024
bae1463
PGSQL Revert go.mod changes (or try to)
NodudeWasTaken Oct 8, 2024
b94e7b7
Add the lock back
NodudeWasTaken Oct 9, 2024
0ce1226
Fix linter issues
NodudeWasTaken Oct 9, 2024
f810b3b
Close being nil fix
NodudeWasTaken Oct 9, 2024
aaaad79
fix anonymiser
NodudeWasTaken Oct 9, 2024
c54540b
Readbility nits around getDBBoolean
NodudeWasTaken Oct 9, 2024
1d50d45
Infer db type
NodudeWasTaken Oct 9, 2024
16e0e36
Missed a readability nit
NodudeWasTaken Oct 9, 2024
9b55dcd
Another nit
NodudeWasTaken Oct 9, 2024
21f6d16
No more psql lock
NodudeWasTaken Oct 9, 2024
8e4ce3a
Multiple lines in pgsql schema nit
NodudeWasTaken Oct 9, 2024
5403431
Use a single "database" configuration as connection string too
its-josh4 Oct 9, 2024
1520e47
Fix values criterion for pgsql.
NodudeWasTaken Oct 10, 2024
36c8b8d
Merge pull request #1 from its-josh4/connstring
NodudeWasTaken Oct 10, 2024
edc3c7d
Better pgsql open
NodudeWasTaken Oct 10, 2024
817da3e
Fix DISTINCT aggregates must have exactly one argument
NodudeWasTaken Oct 10, 2024
2a79c63
Keep disableForeignKeys
NodudeWasTaken Oct 10, 2024
3959e4f
Better DB abstraction, still ugly though
NodudeWasTaken Oct 11, 2024
688e365
Added distinctIDs back and obey group by rules (if postgresql)
NodudeWasTaken Oct 13, 2024
2260ab5
Some small cosmetic query changes
NodudeWasTaken Oct 14, 2024
56fdfff
postgres backup system
NodudeWasTaken Oct 14, 2024
878c8e8
Cleanup PGSQL test
NodudeWasTaken Oct 14, 2024
088dd14
groupby
NodudeWasTaken Oct 14, 2024
235b2bc
Distinct on
NodudeWasTaken Oct 14, 2024
b019f3f
Fix images pagination bug
NodudeWasTaken Oct 14, 2024
4bf5d2a
Git fix
NodudeWasTaken Oct 14, 2024
f3c830b
pagination and sort
NodudeWasTaken Oct 14, 2024
1c537b8
Revert: postgres backup system
NodudeWasTaken Oct 15, 2024
992f481
Fix some tests for pgsql
NodudeWasTaken Oct 15, 2024
390059d
more test fixes for pgsql
NodudeWasTaken Oct 15, 2024
9cd1e5d
for this compilation fix for tests for pgsql
NodudeWasTaken Oct 15, 2024
2edc214
tests for sqlite bugfix
NodudeWasTaken Oct 15, 2024
8fa2b38
pgsql fix tests (suffering)
NodudeWasTaken Oct 17, 2024
aa4f257
pgsql fix more tests
NodudeWasTaken Oct 18, 2024
df48fc7
pgsql more test fixes
NodudeWasTaken Oct 19, 2024
45bbb76
pgsql: more work on passing tests
NodudeWasTaken Oct 20, 2024
2a9b8db
small remove
NodudeWasTaken Oct 20, 2024
db0cbc2
pgsql remove distinct (again), and use group by instead
NodudeWasTaken Oct 21, 2024
1ab44eb
pgsql missing some groupby's (some like title are unnecessary, but it…
NodudeWasTaken Oct 21, 2024
7540304
pgsql fix more sql bugs
NodudeWasTaken Oct 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ require (
github.com/gorilla/sessions v1.2.1
github.com/gorilla/websocket v1.5.0
github.com/hashicorp/golang-lru/v2 v2.0.7
github.com/jackc/pgx/v5 v5.3.1
github.com/jinzhu/copier v0.4.0
github.com/jmoiron/sqlx v1.4.0
github.com/json-iterator/go v1.1.12
Expand Down Expand Up @@ -82,7 +83,10 @@ require (
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/lib/pq v1.10.9 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
Expand Down
29 changes: 29 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,15 @@ filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4
github.com/99designs/gqlgen v0.17.2/go.mod h1:K5fzLKwtph+FFgh9j7nFbRUdBKvTcGnsta51fsMTn3o=
github.com/99designs/gqlgen v0.17.49 h1:b3hNGexHd33fBSAd4NDT/c3NCcQzcAVkknhN9ym36YQ=
github.com/99designs/gqlgen v0.17.49/go.mod h1:tC8YFVZMed81x7UJ7ORUwXF4Kn6SXuucFqQBhN8+BU0=
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 h1:L/gRVlceqvL25UVaW/CKtUDjefjrs0SPonmDGUVOYP0=
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/DATA-DOG/go-sqlmock v1.5.0 h1:Shsta01QNfFxHCfpW6YH2STWB0MudeXXEWMr20OEh60=
github.com/DATA-DOG/go-sqlmock v1.5.0/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM=
github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ=
github.com/Microsoft/go-winio v0.6.1 h1:9/kr64B9VUZrLm5YYwbGtUJnMgqWVOdUAXu6Migciow=
github.com/Microsoft/go-winio v0.6.1/go.mod h1:LRdKpFKfdobln8UmuiYcKPot9D2v6svN5+sAH+4kjUM=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/PuerkitoBio/goquery v1.9.2 h1:4/wZksC3KgkQw7SQgkKotmKljk0M6V8TUvA8Wb4yPeE=
github.com/PuerkitoBio/goquery v1.9.2/go.mod h1:GHPCaP0ODyyxqcNoFGYlAprUFH81NuRPd0GX3Zu2Mvk=
Expand Down Expand Up @@ -173,11 +177,21 @@ github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/denisenkom/go-mssqldb v0.10.0/go.mod h1:xbL0rPBG9cCiLr28tMa8zpbdarY27NDyej4t/EjAShU=
github.com/dgryski/trifles v0.0.0-20200323201526-dd97f9abfb48 h1:fRzb/w+pyskVMQ+UbP35JkH8yB7MYb4q/qhBarqZE6g=
github.com/dgryski/trifles v0.0.0-20200323201526-dd97f9abfb48/go.mod h1:if7Fbed8SFyPtHLHbg49SI7NAdJiC5WIA09pe59rfAA=
github.com/dhui/dktest v0.3.16 h1:i6gq2YQEtcrjKbeJpBkWjE8MmLZPYllcjOFbTZuPDnw=
github.com/dhui/dktest v0.3.16/go.mod h1:gYaA3LRmM8Z4vJl2MA0THIigJoZrwOansEOsp+kqxp0=
github.com/disintegration/imaging v1.6.2 h1:w1LecBlG2Lnp8B3jk5zSuNqd7b4DXhcjwek1ei82L+c=
github.com/disintegration/imaging v1.6.2/go.mod h1:44/5580QXChDfwIclfc/PCwrr44amcmDAg8hxG0Ewe4=
github.com/dlclark/regexp2 v1.4.1-0.20201116162257-a2a8dda75c91/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc=
github.com/dlclark/regexp2 v1.7.0 h1:7lJfhqlPssTb1WQx4yvTHN0uElPEv52sbaECrAQxjAo=
github.com/dlclark/regexp2 v1.7.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/docker/distribution v2.8.2+incompatible h1:T3de5rq0dB1j30rp0sA2rER+m322EBzniBPB6ZIzuh8=
github.com/docker/distribution v2.8.2+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v20.10.24+incompatible h1:Ugvxm7a8+Gz6vqQYQQ2W7GYq5EUPaAiuPgIfVyI3dYE=
github.com/docker/docker v20.10.24+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/go-connections v0.4.0 h1:El9xVISelRB7BuFusrZozjnkIM5YnzCViNKohAFqRJQ=
github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE=
github.com/dop251/goja v0.0.0-20211022113120-dc8c55024d06/go.mod h1:R9ET47fwRVRPZnOGvHxxhuZcbrMCuiqOz3Rlrh4KSnk=
github.com/dop251/goja v0.0.0-20231027120936-b396bb4c349d h1:wi6jN5LVt/ljaBG4ue79Ekzb12QfJ52L9Q98tl8SWhw=
Expand Down Expand Up @@ -249,6 +263,7 @@ github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5x
github.com/gofrs/uuid/v5 v5.1.0 h1:S5rqVKIigghZTCBKPCw0Y+bXkn26K3TB5mvQq2Ix8dk=
github.com/gofrs/uuid/v5 v5.1.0/go.mod h1:CDOjlDMVAtN56jqyRUZh58JT31Tiw7/oQyEXZV+9bD8=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang-jwt/jwt/v4 v4.5.0 h1:7cYmW1XlMY7h7ii7UhUyChSgS5wUJEnm9uZVTGqOWzg=
github.com/golang-jwt/jwt/v4 v4.5.0/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=
Expand Down Expand Up @@ -409,6 +424,12 @@ github.com/ianlancetaylor/demangle v0.0.0-20220319035150-800ac71e25c2/go.mod h1:
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a h1:bbPeKD0xmW/Y25WS6cokEszi5g+S0QxI/d45PkRi7Nk=
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.3.1 h1:Fcr8QJ1ZeLi5zsPZqQeUZhNhxfkkKBOgJuYkJHoBOtU=
github.com/jackc/pgx/v5 v5.3.1/go.mod h1:t3JDKnCBlYIc0ewLF0Q7B8MXmoIaBOZj/ic7iHozM/8=
github.com/jinzhu/copier v0.4.0 h1:w3ciUoD19shMCRargcpm0cm91ytaBhDvuRpz1ODO/U8=
github.com/jinzhu/copier v0.4.0/go.mod h1:DfbEm0FYsaqBcKcFuvmOZb218JkPGtvSHsKg8S8hyyg=
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
Expand Down Expand Up @@ -512,13 +533,17 @@ github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RR
github.com/mitchellh/reflectwalk v1.0.0/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
github.com/mitchellh/reflectwalk v1.0.2 h1:G2LzWKi524PWgd3mLHV8Y5k7s6XUvT0Gef6zxSIeXaQ=
github.com/mitchellh/reflectwalk v1.0.2/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
github.com/moby/term v0.5.0 h1:xt8Q1nalod/v7BqbG21f8mQPqH+xAaC9C3N3wfWbVP0=
github.com/moby/term v0.5.0/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/mschoch/smat v0.0.0-20160514031455-90eadee771ae/go.mod h1:qAyveg+e4CE+eKJXWVjKXM4ck2QobLqTDytGJbLLhJg=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
Expand All @@ -530,6 +555,10 @@ github.com/npillmayer/nestext v0.1.3/go.mod h1:h2lrijH8jpicr25dFY+oAJLyzlya6jhnu
github.com/nu7hatch/gouuid v0.0.0-20131221200532-179d4d0c4d8d h1:VhgPp6v9qf9Agr/56bj7Y/xa04UccTW04VP0Qed4vnQ=
github.com/nu7hatch/gouuid v0.0.0-20131221200532-179d4d0c4d8d/go.mod h1:YUTz3bUH2ZwIWBy3CJBeOBEugqcmXREj14T+iG/4k4U=
github.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQA=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.0.2 h1:9yCKha/T5XdGtO0q9Q9a6T5NUCsTn/DrBg0D7ufOcFM=
github.com/opencontainers/image-spec v1.0.2/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde h1:x0TT0RDC7UhAVbbWWBzr41ElhJx5tXPWkIHA2HWPRuw=
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0=
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
Expand Down
4 changes: 1 addition & 3 deletions internal/api/resolver.go
Original file line number Diff line number Diff line change
Expand Up @@ -319,20 +319,18 @@ func (r *queryResolver) Latestversion(ctx context.Context) (*LatestVersion, erro

func (r *mutationResolver) ExecSQL(ctx context.Context, sql string, args []interface{}) (*SQLExecResult, error) {
var rowsAffected *int64
var lastInsertID *int64

db := manager.GetInstance().Database
if err := r.withTxn(ctx, func(ctx context.Context) error {
var err error
rowsAffected, lastInsertID, err = db.ExecSQL(ctx, sql, args)
rowsAffected, err = db.ExecSQL(ctx, sql, args)
return err
}); err != nil {
return nil, err
}

return &SQLExecResult{
RowsAffected: rowsAffected,
LastInsertID: lastInsertID,
}, nil
}

Expand Down
4 changes: 2 additions & 2 deletions internal/api/resolver_mutation_configure.go
Original file line number Diff line number Diff line change
Expand Up @@ -127,8 +127,8 @@ func (r *mutationResolver) ConfigureGeneral(ctx context.Context, input ConfigGen
return nil
}

existingDBPath := c.GetDatabasePath()
if input.DatabasePath != nil && existingDBPath != *input.DatabasePath {
existingDBUrl := c.GetDatabaseUrl()
if input.DatabasePath != nil && existingDBUrl != *input.DatabasePath {
if err := checkConfigOverride(config.Database); err != nil {
return makeConfigGeneralResult(), err
}
Expand Down
2 changes: 1 addition & 1 deletion internal/api/resolver_mutation_migrate.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ func (r *mutationResolver) MigrateBlobs(ctx context.Context, input MigrateBlobsI
mgr := manager.GetInstance()
t := &task.MigrateBlobsJob{
TxnManager: mgr.Database,
BlobStore: mgr.Database.Blobs,
BlobStore: mgr.Database.GetRepo().Blobs,
Vacuumer: mgr.Database,
DeleteOld: utils.IsTrue(input.DeleteOld),
}
Expand Down
2 changes: 1 addition & 1 deletion internal/api/resolver_query_configuration.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ func makeConfigGeneralResult() *ConfigGeneralResult {

return &ConfigGeneralResult{
Stashes: config.GetStashPaths(),
DatabasePath: config.GetDatabasePath(),
DatabasePath: config.GetDatabaseUrl(),
BackupDirectoryPath: config.GetBackupDirectoryPath(),
GeneratedPath: config.GetGeneratedPath(),
MetadataPath: config.GetMetadataPath(),
Expand Down
23 changes: 16 additions & 7 deletions internal/autotag/integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,20 +33,27 @@ var existingStudioID int

const expectedMatchTitle = "expected match"

var db *sqlite.Database
var db sqlite.DBInterface
var r models.Repository

func testTeardown(databaseFile string) {
err := db.Close()
err := db.Remove()

if err != nil {
panic(err)
}
}

err = os.Remove(databaseFile)
if err != nil {
panic(err)
func getNewDB(databaseFile string) sqlite.DBInterface {
dbUrl, valid := os.LookupEnv("PGSQL_TEST")
if valid {
db = sqlite.NewPostgresDatabase(dbUrl, true)
} else {
sqlite.RegisterSqliteDialect()
db = sqlite.NewSQLiteDatabase(databaseFile, true)
}

return db
}

func runTests(m *testing.M) int {
Expand All @@ -58,10 +65,12 @@ func runTests(m *testing.M) int {

f.Close()
databaseFile := f.Name()
db = sqlite.NewDatabase()
if err := db.Open(databaseFile); err != nil {
db = getNewDB(databaseFile)

if err := db.Open(); err != nil {
panic(fmt.Sprintf("Could not initialize database: %s", err.Error()))
}
db.TestMode()

r = db.Repository()

Expand Down
4 changes: 2 additions & 2 deletions internal/manager/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -691,7 +691,7 @@ func (i *Config) GetMetadataPath() string {
return i.getString(Metadata)
}

func (i *Config) GetDatabasePath() string {
func (i *Config) GetDatabaseUrl() string {
return i.getString(Database)
}

Expand All @@ -703,7 +703,7 @@ func (i *Config) GetBackupDirectoryPathOrDefault() string {
ret := i.GetBackupDirectoryPath()
if ret == "" {
// #4915 - default to the same directory as the database
return filepath.Dir(i.GetDatabasePath())
return filepath.Dir(i.GetDatabaseUrl())
}

return ret
Expand Down
2 changes: 1 addition & 1 deletion internal/manager/config/config_concurrency_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ func TestConcurrentConfigAccess(t *testing.T) {
i.SetInterface(Cache, i.GetCachePath())
i.SetInterface(Generated, i.GetGeneratedPath())
i.SetInterface(Metadata, i.GetMetadataPath())
i.SetInterface(Database, i.GetDatabasePath())
i.SetInterface(Database, i.GetDatabaseUrl())

// these must be set as strings since the original values are also strings
// setting them as []byte will cause the returned string to be corrupted
Expand Down
39 changes: 27 additions & 12 deletions internal/manager/init.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,20 @@ import (
func Initialize(cfg *config.Config, l *log.Logger) (*Manager, error) {
ctx := context.TODO()

db := sqlite.NewDatabase()
var db sqlite.DBInterface

dbUrl := cfg.GetDatabaseUrl()
upperUrl := strings.ToUpper(dbUrl)
switch {
case strings.HasPrefix(upperUrl, string(sqlite.PostgresBackend)+":"):
db = sqlite.NewPostgresDatabase(dbUrl, true)
case strings.HasPrefix(upperUrl, string(sqlite.SqliteBackend)+":"):
db = sqlite.NewSQLiteDatabase(dbUrl[len(sqlite.SqliteBackend)+1:], true)
default:
// Assume it's the path to a SQLite database - for backwards compat
db = sqlite.NewSQLiteDatabase(dbUrl, true)
}

repo := db.Repository()

// start with empty paths
Expand All @@ -46,30 +59,32 @@ func Initialize(cfg *config.Config, l *log.Logger) (*Manager, error) {

pluginCache := plugin.NewCache(cfg)

dbRepo := db.GetRepo()

sceneService := &scene.Service{
File: db.File,
Repository: db.Scene,
MarkerRepository: db.SceneMarker,
File: dbRepo.File,
Repository: dbRepo.Scene,
MarkerRepository: dbRepo.SceneMarker,
PluginCache: pluginCache,
Paths: mgrPaths,
Config: cfg,
}

imageService := &image.Service{
File: db.File,
Repository: db.Image,
File: dbRepo.File,
Repository: dbRepo.Image,
}

galleryService := &gallery.Service{
Repository: db.Gallery,
ImageFinder: db.Image,
Repository: dbRepo.Gallery,
ImageFinder: dbRepo.Image,
ImageService: imageService,
File: db.File,
Folder: db.Folder,
File: dbRepo.File,
Folder: dbRepo.Folder,
}

groupService := &group.Service{
Repository: db.Group,
Repository: dbRepo.Group,
}

sceneServer := &SceneServer{
Expand Down Expand Up @@ -227,7 +242,7 @@ func (s *Manager) postInit(ctx context.Context) error {
})
}

if err := s.Database.Open(s.Config.GetDatabasePath()); err != nil {
if err := s.Database.Open(); err != nil {
var migrationNeededErr *sqlite.MigrationNeededError
if errors.As(err, &migrationNeededErr) {
logger.Warn(err)
Expand Down
2 changes: 1 addition & 1 deletion internal/manager/manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ type Manager struct {

DLNAService *dlna.Service

Database *sqlite.Database
Database sqlite.DBInterface
Repository models.Repository

SceneService SceneService
Expand Down
2 changes: 1 addition & 1 deletion internal/manager/task/migrate.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ type migrateJobConfig interface {
type MigrateJob struct {
BackupPath string
Config migrateJobConfig
Database *sqlite.Database
Database sqlite.DBInterface
}

type databaseSchemaInfo struct {
Expand Down
5 changes: 5 additions & 0 deletions pkg/models/relationships.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package models

import (
"context"
"slices"

"github.com/stashapp/stash/pkg/sliceutil"
)
Expand Down Expand Up @@ -86,6 +87,10 @@ func (r RelatedIDs) Loaded() bool {
return r.list != nil
}

func (r RelatedIDs) Sort() {
slices.Sort(r.list)
}

func (r RelatedIDs) mustLoaded() {
if !r.Loaded() {
panic("list has not been loaded")
Expand Down
16 changes: 10 additions & 6 deletions pkg/sqlite/anonymise.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,24 @@ const (
)

type Anonymiser struct {
*Database
*SQLiteDB
}

func NewAnonymiser(db *Database, outPath string) (*Anonymiser, error) {
if _, err := db.writeDB.Exec(fmt.Sprintf(`VACUUM INTO "%s"`, outPath)); err != nil {
func NewAnonymiser(db DBInterface, outPath string) (*Anonymiser, error) {
if dbWrapper.dbType == PostgresBackend {
return nil, fmt.Errorf("anonymise is not yet implemented for postgres backend")
}

if _, err := db.GetWriteDB().Exec(fmt.Sprintf(`VACUUM INTO "%s"`, outPath)); err != nil {
return nil, fmt.Errorf("vacuuming into %s: %w", outPath, err)
}

newDB := NewDatabase()
if err := newDB.Open(outPath); err != nil {
newDB := NewSQLiteDatabase(outPath, false)
if err := newDB.Open(); err != nil {
return nil, fmt.Errorf("opening %s: %w", outPath, err)
}

return &Anonymiser{Database: newDB}, nil
return &Anonymiser{SQLiteDB: newDB}, nil
}

func (db *Anonymiser) Anonymise(ctx context.Context) error {
Expand Down
Loading
Loading