Painting performance improvements - Tiled buffer (#1776)

* Canvaspainter improvement WIP * Implement tiled buffer for faster painting WIP * CanvasPainter painting improvements - Split painting for current frame and onion skinning - Introduce blitRect for updating only the dirty portion of the frame - Optimization: Only do an expensive fill when the size changes, otherwise rely on the blitRect to clear the dirty portion. * Only update the dirty potion Aside from cleaning up various extra update methods that are no longer necessary, I've made a minor change in vectorImage, in order to get it's correct dirty region. * Fix vector selection being slightly wider than the overall bounds This was a necessary change in order to calculate the correct bounds that wouldn't draw artifacts. * Fix stroke placement for vector polyline This also fixes some inconsistency in the CanvasPainter that I couldn't make sense of.. * Implement faster hashing * Do some cleanup * Fix next onion skin being drawn on current frame when it's the last * Re-add loadFile if the bitmap image hasn't been loaded yet Although i don't like that this exists in our painter, I believe we added it because for some reason the image hasn't been loaded yet, so let's leave it for now... ideally though i've really like to get rid of such weird mutable things that has little to do with painting. * Update CanvasPainter interface to better match its usage * Cleanup TiledBuffer * Make polyline work * Fix blur logic of smudge tool Additionally this will fix some artifacts when smudging * Replace use of BitmapImage buffer with TiledBuffer - Note that VectorImage hasn't been implemented yet. * Make TiledBuffer work for VectorImage * Cleanup dead code in TiledBuffer * Fix not painting bitmap layers when not on current frame * Add missing license * Remove dead code * Fix drawImage would not get proper update bounds * Cleanup TiledBuffer and fix warnings * Fix Eraser should use destinationOut composition * Fix anti aliasing was causing unwanted floating precision error This was causing the gaps to be drawn between the tiled buffer tiles in some cases... particularly when erasing and you had zoomed in or out. * Fix Polyline can't be removed after applying on vector * Fix transformed selection was being painted in wrong order * Cleanup after testing * Fix tabletRestorePrevTool would always ask for a full canvas update * Remove unnecessary clear of cache when setting tool * Fix update artifacts when using dotted cursor - In addition this fix makes it possible to ignore an additional update event which would previously be done by updateCanvasCursor, but because the TiledBuffer will handle this now, we only update the cursor when a tool is not active. * Refactor TiledBuffer - Also add missing onClearTile callback, which is unused but implementation wise is correct now. * Some cleanup * Remove redundant clear of tile before deleting it * Update signal/slot naming convention * Apply some refactoring to tile * Fix blitRect::extend would make tiles odd numbered * Remove unused methods * Avoid invalidating cache till we're done drawing * Remove various dead code from ScribbleArea * Fix ScribbleArea repaint when tiled buffer does not cover update rect * Fix canvas pixmap being too big when using devicePixelRatio > 1 * Try to fix seams appearing in Qt 6 * Fix canvas not being updated when using camera tool * Cleanup CanvasPainter * Adjust blitRect explanation * Remove the need to allocate an empty QPointF to draw pixmaps * TiledBuffer: Remove unused signals * Fix typo * Fix compiler warning --------- Co-authored-by: Jakob Gahde <[email protected]>
pencil2d · Oct 1, 2023 · a4d6330 · a4d6330
1 parent 1395c86
commit a4d6330
Show file tree

Hide file tree

Showing 35 changed files with 643 additions and 366 deletions.
diff --git a/app/src/actioncommands.cpp b/app/src/actioncommands.cpp
@@ -885,7 +885,7 @@ void ActionCommands::changeKeyframeLineColor()
         QRgb color = mEditor->color()->frontColor().rgb();
         LayerBitmap* layer = static_cast<LayerBitmap*>(mEditor->layers()->currentLayer());
         layer->getBitmapImageAtFrame(mEditor->currentFrame())->fillNonAlphaPixels(color);
-        mEditor->updateFrame(mEditor->currentFrame());
+        mEditor->updateFrame();
     }
 }
 
@@ -900,7 +900,7 @@ void ActionCommands::changeallKeyframeLineColor()
             if (layer->keyExists(i))
                 layer->getBitmapImageAtFrame(i)->fillNonAlphaPixels(color);
         }
-        mEditor->updateFrame(mEditor->currentFrame());
+        mEditor->updateFrame();
     }
 }
 

diff --git a/app/src/colorpalettewidget.cpp b/app/src/colorpalettewidget.cpp
@@ -636,7 +636,7 @@ void ColorPaletteWidget::clickRemoveColorButton()
     {
         fitSwatchSize();
     }
-    mEditor->updateCurrentFrame();
+    mEditor->updateFrame();
 }
 
 bool ColorPaletteWidget::showPaletteWarning()

diff --git a/app/src/mainwindow2.cpp b/app/src/mainwindow2.cpp
@@ -879,7 +879,7 @@ void MainWindow2::importImage()
         return;
     }
 
-    ui->scribbleArea->updateCurrentFrame();
+    ui->scribbleArea->updateFrame();
     mTimeLine->updateContent();
 }
 
@@ -1449,7 +1449,7 @@ void MainWindow2::makeConnections(Editor* editor, ColorInspector* colorInspector
 
 void MainWindow2::makeConnections(Editor* editor, ScribbleArea* scribbleArea)
 {
-    connect(editor->tools(), &ToolManager::toolChanged, scribbleArea, &ScribbleArea::setCurrentTool);
+    connect(editor->tools(), &ToolManager::toolChanged, scribbleArea, &ScribbleArea::updateToolCursor);
     connect(editor->tools(), &ToolManager::toolChanged, mToolBox, &ToolBoxWidget::onToolSetActive);
     connect(editor->tools(), &ToolManager::toolPropertyChanged, scribbleArea, &ScribbleArea::updateToolCursor);
 

diff --git a/core_lib/core_lib.pro b/core_lib/core_lib.pro
@@ -34,6 +34,8 @@ HEADERS +=  \
     src/corelib-pch.h \
     src/graphics/bitmap/bitmapbucket.h \
     src/graphics/bitmap/bitmapimage.h \
+    src/graphics/bitmap/tile.h \
+    src/graphics/bitmap/tiledbuffer.h \
     src/graphics/vector/bezierarea.h \
     src/graphics/vector/beziercurve.h \
     src/graphics/vector/colorref.h \
@@ -119,6 +121,8 @@ HEADERS +=  \
 
 SOURCES +=  src/graphics/bitmap/bitmapimage.cpp \
     src/graphics/bitmap/bitmapbucket.cpp \
+    src/graphics/bitmap/tile.cpp \
+    src/graphics/bitmap/tiledbuffer.cpp \
     src/graphics/vector/bezierarea.cpp \
     src/graphics/vector/beziercurve.cpp \
     src/graphics/vector/colorref.cpp \

diff --git a/core_lib/src/canvaspainter.cpp b/core_lib/src/canvaspainter.cpp
@@ -17,12 +17,13 @@ GNU General Public License for more details.
 #include "canvaspainter.h"
 
 #include <QtMath>
-#include <QSettings>
 
 #include "object.h"
 #include "layerbitmap.h"
 #include "layervector.h"
 #include "bitmapimage.h"
+#include "tile.h"
+#include "tiledbuffer.h"
 #include "vectorimage.h"
 
 #include "painterutils.h"
@@ -40,13 +41,17 @@ void CanvasPainter::reset()
 {
     mPostLayersPixmap = QPixmap(mCanvas.size());
     mPreLayersPixmap = QPixmap(mCanvas.size());
+    mCurrentLayerPixmap = QPixmap(mCanvas.size());
+    mOnionSkinPixmap = QPixmap(mCanvas.size());
     mPreLayersPixmap.fill(Qt::transparent);
     mCanvas.fill(Qt::transparent);
-    mCurrentLayerPixmap = QPixmap(mCanvas.size());
     mCurrentLayerPixmap.fill(Qt::transparent);
     mPostLayersPixmap.fill(Qt::transparent);
-    mOnionSkinPixmap = QPixmap(mCanvas.size());
     mOnionSkinPixmap.fill(Qt::transparent);
+    mCurrentLayerPixmap.setDevicePixelRatio(mCanvas.devicePixelRatioF());
+    mPreLayersPixmap.setDevicePixelRatio(mCanvas.devicePixelRatioF());
+    mPostLayersPixmap.setDevicePixelRatio(mCanvas.devicePixelRatioF());
+    mOnionSkinPixmap.setDevicePixelRatio(mCanvas.devicePixelRatioF());
 }
 
 void CanvasPainter::setViewTransform(const QTransform view, const QTransform viewInverse)
@@ -92,7 +97,7 @@ void CanvasPainter::paintCached(const QRect& blitRect)
     QPainter mainPainter;
     initializePainter(mainPainter, mCanvas, blitRect);
     mainPainter.setWorldMatrixEnabled(false);
-    mainPainter.drawPixmap(blitRect, mPreLayersPixmap, blitRect);
+    mainPainter.drawPixmap(mPointZero, mPreLayersPixmap);
     mainPainter.setWorldMatrixEnabled(true);
 
     paintCurrentFrame(mainPainter, blitRect, mCurrentLayerIndex, mCurrentLayerIndex);
@@ -107,7 +112,7 @@ void CanvasPainter::paintCached(const QRect& blitRect)
     }
 
     mainPainter.setWorldMatrixEnabled(false);
-    mainPainter.drawPixmap(blitRect, mPostLayersPixmap, blitRect);
+    mainPainter.drawPixmap(mPointZero, mPostLayersPixmap);
     mainPainter.setWorldMatrixEnabled(true);
 }
 
@@ -121,6 +126,9 @@ void CanvasPainter::initializePainter(QPainter& painter, QPaintDevice& device, c
 {
     painter.begin(&device);
 
+    // Only draw inside the clipped rectangle
+    painter.setClipRect(blitRect);
+
     // Clear the area that's about to be painted again, to avoid painting on top of existing pixels
     // causing artifacts.
     painter.setCompositionMode(QPainter::CompositionMode_Clear);
@@ -151,7 +159,7 @@ void CanvasPainter::renderPostLayers(QPainter& painter, const QRect& blitRect)
     }
 }
 
-void CanvasPainter::setPaintSettings(const Object* object, int currentLayer, int frame, QRect rect, BitmapImage* buffer)
+void CanvasPainter::setPaintSettings(const Object* object, int currentLayer, int frame, QRect rect, TiledBuffer* tiledBuffer)
 {
     Q_UNUSED(rect)
     Q_ASSERT(object);
@@ -160,7 +168,7 @@ void CanvasPainter::setPaintSettings(const Object* object, int currentLayer, int
     CANVASPAINTER_LOG("Set CurrentLayerIndex = %d", currentLayer);
     mCurrentLayerIndex = currentLayer;
     mFrameNumber = frame;
-    mBuffer = buffer;
+    mTiledBuffer = tiledBuffer;
 }
 
 void CanvasPainter::paint(const QRect& blitRect)
@@ -176,7 +184,7 @@ void CanvasPainter::paint(const QRect& blitRect)
     preLayerPainter.end();
 
     mainPainter.setWorldMatrixEnabled(false);
-    mainPainter.drawPixmap(blitRect, mPreLayersPixmap, blitRect);
+    mainPainter.drawPixmap(mPointZero, mPreLayersPixmap);
     mainPainter.setWorldMatrixEnabled(true);
 
     paintCurrentFrame(mainPainter, blitRect, mCurrentLayerIndex, mCurrentLayerIndex);
@@ -186,7 +194,7 @@ void CanvasPainter::paint(const QRect& blitRect)
     postLayerPainter.end();
 
     mainPainter.setWorldMatrixEnabled(false);
-    mainPainter.drawPixmap(blitRect, mPostLayersPixmap, blitRect);
+    mainPainter.drawPixmap(mPointZero, mPostLayersPixmap);
     mainPainter.setWorldMatrixEnabled(true);
 
     mPreLayersPixmapCacheValid = true;
@@ -230,7 +238,7 @@ void CanvasPainter::paintBitmapOnionSkinFrame(QPainter& painter, const QRect& bl
     initializePainter(onionSkinPainter, mOnionSkinPixmap, blitRect);
 
     onionSkinPainter.drawImage(bitmapImage->topLeft(), *bitmapImage->image());
-    paintOnionSkinFrame(painter, onionSkinPainter, blitRect, nFrame, colorize, bitmapImage->getOpacity());
+    paintOnionSkinFrame(painter, onionSkinPainter, nFrame, colorize, bitmapImage->getOpacity());
 }
 
 void CanvasPainter::paintVectorOnionSkinFrame(QPainter& painter, const QRect& blitRect, Layer* layer, int nFrame, bool colorize)
@@ -245,10 +253,10 @@ void CanvasPainter::paintVectorOnionSkinFrame(QPainter& painter, const QRect& bl
     initializePainter(onionSkinPainter, mOnionSkinPixmap, blitRect);
 
     vectorImage->paintImage(onionSkinPainter, mOptions.bOutlines, mOptions.bThinLines, mOptions.bAntiAlias);
-    paintOnionSkinFrame(painter, onionSkinPainter, blitRect, nFrame, colorize, vectorImage->getOpacity());
+    paintOnionSkinFrame(painter, onionSkinPainter, nFrame, colorize, vectorImage->getOpacity());
 }
 
-void CanvasPainter::paintOnionSkinFrame(QPainter& painter, QPainter& onionSkinPainter, const QRect& blitRect, int nFrame, bool colorize, qreal frameOpacity)
+void CanvasPainter::paintOnionSkinFrame(QPainter& painter, QPainter& onionSkinPainter, int nFrame, bool colorize, qreal frameOpacity)
 {
     // Don't transform the image here as we used the viewTransform in the image output
     painter.setWorldMatrixEnabled(false);
@@ -272,7 +280,7 @@ void CanvasPainter::paintOnionSkinFrame(QPainter& painter, QPainter& onionSkinPa
         onionSkinPainter.setBrush(colorBrush);
         onionSkinPainter.drawRect(painter.viewport());
     }
-    painter.drawPixmap(blitRect, mOnionSkinPixmap, blitRect);
+    painter.drawPixmap(mPointZero, mOnionSkinPixmap);
 }
 
 void CanvasPainter::paintCurrentBitmapFrame(QPainter& painter, const QRect& blitRect, Layer* layer, bool isCurrentLayer)
@@ -283,27 +291,43 @@ void CanvasPainter::paintCurrentBitmapFrame(QPainter& painter, const QRect& blit
     if (paintedImage == nullptr) { return; }
     paintedImage->loadFile(); // Critical! force the BitmapImage to load the image
 
-    const bool isDrawing = mBuffer && !mBuffer->bounds().isEmpty();
+    const bool isDrawing = mTiledBuffer && !mTiledBuffer->bounds().isEmpty();
 
     QPainter currentBitmapPainter;
     initializePainter(currentBitmapPainter, mCurrentLayerPixmap, blitRect);
 
     painter.setOpacity(paintedImage->getOpacity() - (1.0-painter.opacity()));
     painter.setWorldMatrixEnabled(false);
 
-    currentBitmapPainter.drawImage(paintedImage->topLeft(), *paintedImage->image());
+    if (isCurrentLayer && isDrawing)
+    {
+        // Certain tools require being painted continuously, for example, the Polyline tool.
+        // The tiled buffer does not update the area outside which it paints,
+        // so in that case, in order to see the previously laid-down polyline stroke,
+        // the surrounding area must be drawn again before
+        // applying the new tiled output on top
+        if (!blitRect.contains(mTiledBuffer->bounds()) || mOptions.bIgnoreCanvasBuffer) {
+            currentBitmapPainter.setCompositionMode(QPainter::CompositionMode_Source);
+            currentBitmapPainter.drawImage(paintedImage->topLeft(), *paintedImage->image());
+        }
 
-    if (isCurrentLayer) {
-        if (isDrawing) {
-            // paint the current stroke data which hasn't been applied to a bitmapImage yet
-            currentBitmapPainter.setCompositionMode(mOptions.cmBufferBlendMode);
-            currentBitmapPainter.drawImage(mBuffer->topLeft(), *mBuffer->image());
-        } else if (mRenderTransform) {
-            paintTransformedSelection(currentBitmapPainter, paintedImage, mSelection);
+        const auto tiles = mTiledBuffer->tiles();
+        for (const Tile* tile : tiles) {
+            currentBitmapPainter.drawPixmap(tile->posF(), tile->pixmap());
         }
+    } else {
+        // When we're drawing using a tool, the surface will be painted by the tiled buffer,
+        // and thus we don't want to paint the current image again
+        // When we're on another layer though, the tiled buffer is not used
+        currentBitmapPainter.drawImage(paintedImage->topLeft(), *paintedImage->image());
     }
 
-    painter.drawPixmap(blitRect, mCurrentLayerPixmap, blitRect);
+    // We do not wish to draw selection transformations on anything but the current layer
+    if (isCurrentLayer && mRenderTransform) {
+        paintTransformedSelection(currentBitmapPainter, paintedImage, mSelection);
+    }
+
+    painter.drawPixmap(mPointZero, mCurrentLayerPixmap);
 }
 
 void CanvasPainter::paintCurrentVectorFrame(QPainter& painter, const QRect& blitRect, Layer* layer, bool isCurrentLayer)
@@ -318,15 +342,19 @@ void CanvasPainter::paintCurrentVectorFrame(QPainter& painter, const QRect& blit
     QPainter currentVectorPainter;
     initializePainter(currentVectorPainter, mCurrentLayerPixmap, blitRect);
 
-    const bool isDrawing = mBuffer && !mBuffer->bounds().isEmpty();
+    const bool isDrawing = mTiledBuffer->isValid();
 
     // Paint existing vector image to the painter
     vectorImage->paintImage(currentVectorPainter, mOptions.bOutlines, mOptions.bThinLines, mOptions.bAntiAlias);
 
     if (isCurrentLayer) {
         if (isDrawing) {
             currentVectorPainter.setCompositionMode(mOptions.cmBufferBlendMode);
-            currentVectorPainter.drawImage(mBuffer->topLeft(), *mBuffer->image());
+
+            const auto tiles = mTiledBuffer->tiles();
+            for (const Tile* tile : tiles) {
+                currentVectorPainter.drawPixmap(tile->posF(), tile->pixmap());
+            }
         } else if (mRenderTransform) {
             vectorImage->setSelectionTransformation(mSelectionTransform);
         }
@@ -336,9 +364,9 @@ void CanvasPainter::paintCurrentVectorFrame(QPainter& painter, const QRect& blit
     painter.setWorldMatrixEnabled(false);
     painter.setTransform(QTransform());
 
-    // Remember to adjust opacity based on addition opacity value from image
+    // Remember to adjust opacity based on additional opacity value from the keyframe
     painter.setOpacity(vectorImage->getOpacity() - (1.0-painter.opacity()));
-    painter.drawPixmap(blitRect, mCurrentLayerPixmap, blitRect);
+    painter.drawPixmap(mPointZero, mCurrentLayerPixmap);
 }
 
 void CanvasPainter::paintTransformedSelection(QPainter& painter, BitmapImage* bitmapImage, const QRect& selection) const

diff --git a/core_lib/src/canvaspainter.h b/core_lib/src/canvaspainter.h
@@ -30,6 +30,8 @@ GNU General Public License for more details.
 #include "onionskinpainteroptions.h"
 #include "onionskinsubpainter.h"
 
+
+class TiledBuffer;
 class Object;
 class BitmapImage;
 class ViewManager;
@@ -39,6 +41,10 @@ struct CanvasPainterOptions
     bool  bAntiAlias = false;
     bool  bThinLines = false;
     bool  bOutlines = false;
+
+    /// When using a tool that can't rely on canvas buffer,
+    /// for example Polyline because we're continously clearing the surface
+    bool  bIgnoreCanvasBuffer = false;
     LayerVisibility eLayerVisibility = LayerVisibility::RELATED;
     float fLayerVisibilityThreshold = 0.f;
     float scaling = 1.0f;
@@ -61,7 +67,7 @@ class CanvasPainter
     void setTransformedSelection(QRect selection, QTransform transform);
     void ignoreTransformedSelection();
 
-    void setPaintSettings(const Object* object, int currentLayer, int frame, QRect rect, BitmapImage* buffer);
+    void setPaintSettings(const Object* object, int currentLayer, int frame, QRect rect, TiledBuffer* tilledBuffer);
     void paint(const QRect& blitRect);
     void paintCached(const QRect& blitRect);
     void resetLayerCache();
@@ -88,7 +94,7 @@ class CanvasPainter
 
     void paintBitmapOnionSkinFrame(QPainter& painter, const QRect& blitRect, Layer* layer, int nFrame, bool colorize);
     void paintVectorOnionSkinFrame(QPainter& painter, const QRect& blitRect, Layer* layer, int nFrame, bool colorize);
-    void paintOnionSkinFrame(QPainter& painter, QPainter& onionSkinPainter, const QRect& blitRect, int nFrame, bool colorize, qreal frameOpacity);
+    void paintOnionSkinFrame(QPainter& painter, QPainter& onionSkinPainter, int nFrame, bool colorize, qreal frameOpacity);
 
     void paintCurrentBitmapFrame(QPainter& painter, const QRect& blitRect, Layer* layer, bool isCurrentLayer);
     void paintCurrentVectorFrame(QPainter& painter, const QRect& blitRect, Layer* layer, bool isCurrentLayer);
@@ -102,7 +108,7 @@ class CanvasPainter
 
     int mCurrentLayerIndex = 0;
     int mFrameNumber = 0;
-    BitmapImage* mBuffer = nullptr;
+    TiledBuffer* mTiledBuffer = nullptr;
 
     QImage mScaledBitmap;
 
@@ -119,6 +125,10 @@ class CanvasPainter
     bool mPreLayersPixmapCacheValid = false;
     bool mPostLayersPixmapCacheValid = false;
 
+    // There's a considerable amount of overhead in simply allocating a QPointF on the fly.
+    // Since we just need to draw it at 0,0, we might as well make a const value for that purpose
+    const QPointF mPointZero;
+
 
     OnionSkinSubPainter mOnionSkinSubPainter;
     OnionSkinPainterOptions mOnionSkinPainterOptions;

diff --git a/core_lib/src/graphics/bitmap/bitmapimage.cpp b/core_lib/src/graphics/bitmap/bitmapimage.cpp
@@ -24,6 +24,8 @@ GNU General Public License for more details.
 #include "util.h"
 
 #include "blitrect.h"
+#include "tile.h"
+#include "tiledbuffer.h"
 
 BitmapImage::BitmapImage()
 {
@@ -98,7 +100,7 @@ BitmapImage* BitmapImage::clone() const
     b->setFileName(""); // don't link to the file of the source bitmap image
 
     const bool validKeyFrame = !fileName().isEmpty();
-    if (validKeyFrame && !isLoaded()) 
+    if (validKeyFrame && !isLoaded())
     {
         // This bitmapImage is temporarily unloaded.
         // since it's not in the memory, we need to copy the linked png file to prevent data loss.
@@ -204,6 +206,28 @@ void BitmapImage::paste(BitmapImage* bitmapImage, QPainter::CompositionMode cm)
     modification();
 }
 
+void BitmapImage::paste(const TiledBuffer* tiledBuffer, QPainter::CompositionMode cm)
+{
+    if(tiledBuffer->bounds().width() <= 0 || tiledBuffer->bounds().height() <= 0)
+    {
+        return;
+    }
+    extend(tiledBuffer->bounds());
+
+    QPainter painter(image());
+
+    painter.setCompositionMode(cm);
+    auto const tiles = tiledBuffer->tiles();
+    for (const Tile* item : tiles) {
+        const QPixmap& tilePixmap = item->pixmap();
+        const QPoint& tilePos = item->pos();
+        painter.drawPixmap(tilePos-mBounds.topLeft(), tilePixmap);
+    }
+    painter.end();
+
+    modification();
+}
+
 void BitmapImage::moveTopLeft(QPoint point)
 {
     mBounds.moveTopLeft(point);
@@ -624,7 +648,10 @@ void BitmapImage::drawRect(QRectF rectangle, QPen pen, QBrush brush, QPainter::C
         painter.setRenderHint(QPainter::Antialiasing, antialiasing);
         painter.setPen(pen);
         painter.setBrush(brush);
-        painter.drawRect(rectangle.translated(-mBounds.topLeft()));
+
+        // Adjust the brush rectangle to be bigger than the bounds itself,
+        // otherwise there will be artifacts shown in some cases when smudging
+        painter.drawRect(rectangle.translated(-mBounds.topLeft()).adjusted(-1, -1, 1, 1));
         painter.end();
     }
     modification();