Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade embedded Postgres and fix compatibility issues. #1199

Closed
wants to merge 5 commits into from
Closed

Upgrade embedded Postgres and fix compatibility issues. #1199

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
40b9e85
Upgrade embedded Postgres and fix compatibility issues.
bcipriano Sep 16, 2022
99886a6
Cleanup.
bcipriano Sep 16, 2022
ccf7d39
Cleanup.
bcipriano Sep 16, 2022
65e09b3
Merge branch 'master' into postgres-upgrade
bcipriano Sep 29, 2022
3400c55
Re-bump version number.
bcipriano Sep 29, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion VERSION.in
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.21
0.22
6 changes: 4 additions & 2 deletions cuebot/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ targetCompatibility = 1.8

ext {
activemqVersion = '5.12.0'
postgresVersion = '14.5.0'
}

configurations {
Expand Down Expand Up @@ -63,11 +64,12 @@ dependencies {
testCompile group: 'junit', name: 'junit', version: '4.12'
testCompile group: 'org.springframework.boot', name: 'spring-boot-starter-test'
testCompile group: 'org.assertj', name: 'assertj-core', version: '3.8.0'
testCompile group: 'io.zonky.test', name: 'embedded-postgres', version: '1.3.1'
testCompile group: 'io.zonky.test', name: 'embedded-postgres', version: '2.0.1'
testCompile group: 'io.zonky.test.postgres', name: 'embedded-postgres-binaries-linux-arm64v8', version: postgresVersion
testCompile group: 'org.flywaydb', name: 'flyway-core', version: '5.2.0'

// Use newer version of Postgres for tests: https://github.com/zonkyio/embedded-postgres/issues/78
implementation enforcedPlatform('io.zonky.test.postgres:embedded-postgres-binaries-bom:11.13.0')
implementation enforcedPlatform('io.zonky.test.postgres:embedded-postgres-binaries-bom:' + postgresVersion)
}

compileJava {
Expand Down
24 changes: 12 additions & 12 deletions cuebot/src/main/java/com/imageworks/spcue/dao/postgres/DispatchQuery.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ public class DispatchQuery {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading postgres docs raised some concerns regarding the performance of this conversion:

https://www.postgresql.org/docs/current/datatype-numeric.html :

The type numeric can store numbers with a very large number of digits. It is especially recommended for storing monetary amounts and other quantities where exactness is required. Calculations with numeric values yield exact results where possible, e.g., addition, subtraction, multiplication. However, calculations on numeric values are very slow compared to the integer types, or to the floating-point types described in the next section.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to dig a bit deeper to understand where this is coming from. Was the issue impacting every query or a specific subset?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seemed to impact every query that used a SUM on a bigint field. I didn't test all of these queries but once I saw it a few times I just went through and changed all of them, it seemed consistent.

According to postgres docs SUM(bigint) always returns a numeric, so I'm assuming something is going wrong in that conversion process somewhere.

It could be a problem specific to the embedded postgres, I don't think I was able to reproduce it anywhere else. And I can't get an older version of the embedded postgres on ARM so I wasn't able to try that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DiegoTavares Any update here? Possible to do a test on your side to see if it produces any performance issues of note?

Copy link
Collaborator

@DiegoTavares DiegoTavares Oct 28, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've submitted a ticket to EDB asking about the error.
It looks there's a change this update will actually have performance implications. We can try a different version of postgres and see if this is a bug specific for this version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bcipriano I have few questions for you

  1. Did you installed postgres 14.5 using following link https://www.enterprisedb.com/downloads/postgres-postgresql-downloads or using some other source ?
  2. Did you got similar issue when you was on version 12.12 ? Are you able to test it on Postgresql12.12 on your new macbook ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bcipriano Can you confirm the source of postgres download? can you try https://www.enterprisedb.com/downloads/postgres-postgresql-downloads for download postgres ? Let me know how it goes with EDB 14.5?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing to mention -- this only seems to happen in Docker. When I run tests directly on my ARM Macbook, they pass fine, in fact this PR isn't needed at all. It's only when I try to build the Cuebot Docker image (which does the Gradle build/test as part of the build process) that this problem appears.

Did you installed postgres 14.5 using following link https://www.enterprisedb.com/downloads/postgres-postgresql-downloads or using some other source ?

Postgres is installed via Gradle/Maven. You can see the versions defined in build.gradle: https://github.com/AcademySoftwareFoundation/OpenCue/pull/1199/files#diff-7e57ab670111c27c23069909d0cc32c88d266c9a1f3e6ca38052c973418146fe

I don't know if it's possible use Postgres binaries from some other source with this embedded Postgres thing we're using.

Did you got similar issue when you was on version 12.12 ? Are you able to test it on Postgresql12.12 on your new macbook ?

Same issue on 11.13.0 and 12.12.0. But only on Docker.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a bit more research into this today:

  • Tried the latest Postgres 15 binaries. Same issue.
  • Tried a different base image (almalinux instead of gradle). Same issue.
  • Created a Postgres container from the standard postgres image on my mac. Was not able to reproduce the issue.

Still kind of stumped. I filed zonkyio/embedded-postgres#99 to see if those folks have any idea.

"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down Expand Up @@ -276,7 +276,7 @@ private static final String replaceQueryForFifo(String query) {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down Expand Up @@ -382,7 +382,7 @@ private static final String replaceQueryForFifo(String query) {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down Expand Up @@ -485,7 +485,7 @@ private static final String replaceQueryForFifo(String query) {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down Expand Up @@ -599,7 +599,7 @@ private static final String replaceQueryForFifo(String query) {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down Expand Up @@ -717,7 +717,7 @@ private static final String replaceQueryForFifo(String query) {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down Expand Up @@ -826,7 +826,7 @@ private static final String replaceQueryForFifo(String query) {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down Expand Up @@ -935,7 +935,7 @@ private static final String replaceQueryForFifo(String query) {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down Expand Up @@ -1051,7 +1051,7 @@ private static final String replaceQueryForFifo(String query) {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down Expand Up @@ -1169,7 +1169,7 @@ private static final String replaceQueryForFifo(String query) {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down Expand Up @@ -1278,7 +1278,7 @@ private static final String replaceQueryForFifo(String query) {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down Expand Up @@ -1387,7 +1387,7 @@ private static final String replaceQueryForFifo(String query) {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down
2 changes: 1 addition & 1 deletion cuebot/src/main/java/com/imageworks/spcue/dao/postgres/FrameDaoJdbc.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ public boolean updateFrameCleared(FrameInterface frame) {
"LEFT JOIN (" +
"SELECT " +
"limit_record.pk_limit_record, " +
"SUM(layer_stat.int_running_count) AS int_sum_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_sum_running " +
"FROM " +
"layer_limit " +
"LEFT JOIN limit_record ON layer_limit.pk_limit_record = limit_record.pk_limit_record " +
Expand Down
4 changes: 2 additions & 2 deletions cuebot/src/main/java/com/imageworks/spcue/dao/postgres/JobDaoJdbc.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,13 +167,13 @@ public DispatchJob getDispatchJob(String uuid) {

private static final String IS_JOB_COMPLETE =
"SELECT " +
"SUM (" +
"SUM(CAST( " +
"int_waiting_count + " +
"int_running_count + " +
"int_dead_count + " +
"int_depend_count + " +
"int_checkpoint_count " +
") " +
" AS numeric)) " +
"FROM " +
"job_stat " +
"WHERE " +
Expand Down
4 changes: 2 additions & 2 deletions cuebot/src/main/java/com/imageworks/spcue/dao/postgres/LayerDaoJdbc.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,12 +108,12 @@ public boolean isLayerDispatchable(LayerInterface l ) {

private static final String IS_LAYER_COMPLETE =
"SELECT " +
"SUM ( " +
"SUM(CAST( " +
"int_waiting_count + " +
"int_running_count + " +
"int_dead_count + " +
"int_depend_count "+
") " +
" AS numeric)) " +
"FROM " +
"layer_stat " +
"WHERE " +
Expand Down
2 changes: 1 addition & 1 deletion cuebot/src/main/java/com/imageworks/spcue/dao/postgres/LimitDaoJdbc.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ public void setMaxValue(LimitInterface limit, int maxValue) {
"limit_record.pk_limit_record, " +
"limit_record.str_name, " +
"limit_record.int_max_value," +
"SUM(layer_stat.int_running_count) AS int_current_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_current_running " +
"FROM " +
"limit_record " +
"LEFT JOIN " +
Expand Down
41 changes: 18 additions & 23 deletions cuebot/src/main/java/com/imageworks/spcue/dao/postgres/ProcDaoJdbc.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,30 +264,25 @@ public void updateProcMemoryUsage(FrameInterface f, long rss, long maxRss,
"SELECT pk_frame FROM proc WHERE pk_frame=? FOR UPDATE",
String.class, f.getFrameId()).equals(f.getFrameId())) {

getJdbcTemplate().update(UPDATE_PROC_MEMORY_USAGE,
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure this was a merge mistake. This block and the one below it seem to duplicate each other, and this block is missing one of the placeholder values.

rss, maxRss, vss, maxVss,
usedGpuMemory, maxUsedGpuMemory, f.getFrameId());
}
Comment on lines -267 to -270
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, this is was a weird merge issue. Not sure where it came from. Good catch.

getJdbcTemplate().update(new PreparedStatementCreator() {
@Override
public PreparedStatement createPreparedStatement(Connection conn)
throws SQLException {
PreparedStatement updateProc = conn.prepareStatement(
UPDATE_PROC_MEMORY_USAGE);
updateProc.setLong(1, rss);
updateProc.setLong(2, maxRss);
updateProc.setLong(3, vss);
updateProc.setLong(4, maxVss);
updateProc.setLong(5, usedGpuMemory);
updateProc.setLong(6, maxUsedGpuMemory);
updateProc.setBytes(7, children);
updateProc.setString(8, f.getFrameId());
return updateProc;
}
}
);
@Override
public PreparedStatement createPreparedStatement(Connection conn)
throws SQLException {
PreparedStatement updateProc = conn.prepareStatement(
UPDATE_PROC_MEMORY_USAGE);
updateProc.setLong(1, rss);
updateProc.setLong(2, maxRss);
updateProc.setLong(3, vss);
updateProc.setLong(4, maxVss);
updateProc.setLong(5, usedGpuMemory);
updateProc.setLong(6, maxUsedGpuMemory);
updateProc.setBytes(7, children);
updateProc.setString(8, f.getFrameId());
return updateProc;
}
});
}
catch (DataAccessException dae) {
} catch (DataAccessException dae) {
logger.info("The proc for frame " + f +
" could not be updated with new memory stats: " + dae);
}
Expand Down Expand Up @@ -608,7 +603,7 @@ public boolean increaseReservedMemory(ProcInterface p, long value) {
"AND " +
"proc.int_mem_reserved != 0 " +
"ORDER BY " +
"proc.int_virt_used / proc.int_mem_pre_reserved DESC " +
"CAST(proc.int_virt_used AS numeric) / proc.int_mem_pre_reserved DESC " +
") AS t1 LIMIT 1";

@Override
Expand Down
9 changes: 4 additions & 5 deletions cuebot/src/main/java/com/imageworks/spcue/dao/postgres/WhiteboardDaoJdbc.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@

import com.google.common.collect.Lists;
import com.google.common.collect.Sets;
import com.google.protobuf.ByteString;
import org.apache.log4j.Logger;
import org.springframework.jdbc.core.RowMapper;
import org.springframework.jdbc.core.support.JdbcDaoSupport;
Expand All @@ -49,12 +48,12 @@
import com.imageworks.spcue.OwnerEntity;
import com.imageworks.spcue.ShowInterface;
import com.imageworks.spcue.dao.WhiteboardDao;
import com.imageworks.spcue.dao.criteria.FrameSearchInterface;
import com.imageworks.spcue.dao.criteria.FrameSearchFactory;
import com.imageworks.spcue.dao.criteria.FrameSearchInterface;
import com.imageworks.spcue.dao.criteria.HostSearchInterface;
import com.imageworks.spcue.dao.criteria.JobSearchInterface;
import com.imageworks.spcue.dao.criteria.ProcSearchInterface;
import com.imageworks.spcue.dao.criteria.ProcSearchFactory;
import com.imageworks.spcue.dao.criteria.ProcSearchInterface;
import com.imageworks.spcue.grpc.comment.Comment;
import com.imageworks.spcue.grpc.comment.CommentSeq;
import com.imageworks.spcue.grpc.department.Department;
Expand Down Expand Up @@ -1806,7 +1805,7 @@ public Show mapRow(ResultSet rs, int rowNum) throws SQLException {
"limit_record.pk_limit_record, " +
"limit_record.str_name, " +
"limit_record.int_max_value, " +
"SUM(layer_stat.int_running_count) AS int_current_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_current_running " +
"FROM " +
"limit_record " +
"LEFT JOIN " +
Expand All @@ -1821,7 +1820,7 @@ public Show mapRow(ResultSet rs, int rowNum) throws SQLException {
"limit_record.pk_limit_record, " +
"limit_record.str_name, " +
"limit_record.int_max_value, " +
"SUM(layer_stat.int_running_count) AS int_current_running " +
"SUM(CAST(layer_stat.int_running_count AS numeric)) AS int_current_running " +
"FROM " +
"limit_record " +
"LEFT JOIN " +
Expand Down
Loading