Element Consumers and Suppliers

https://www.ojalgo.org/2021/08/common-mistake/

---

A relatively new design feature of ojAlgo is the functionality provided by the ElementsSupplier and ElementsConsumer interfaces - they bring important benefits.

Primarily they make it possible to setup chains of operations and have those executed all at once without the need to allocate memory for intermediate results. Further the end result can be "consumed" by predefined memory structures. The chain of operations are encapsulated by ElementsSupplier instances and ElementsConsumer:s are the memory structures that accept/receive results.

Example code

import static org.ojalgo.function.PrimitiveFunction.*;

import org.ojalgo.OjAlgoUtils;
import org.ojalgo.matrix.decomposition.QR;
import org.ojalgo.matrix.store.ElementsConsumer;
import org.ojalgo.matrix.store.ElementsSupplier;
import org.ojalgo.matrix.store.PrimitiveDenseStore;
import org.ojalgo.netio.BasicLogger;
import org.ojalgo.random.Normal;
import org.ojalgo.random.Uniform;

/**
 * A small example to show what can be done with {@link ElementsSupplier}s and {@link ElementsConsumer}s. To
 * realy "see" the benefit of using these you should take this example, scale up the matrix dimensions and
 * extend the loop. Then run the code in a profiler and monitor memory allocation and garbage collection.
 *
 * @author apete
 */
public class ElementConsumersAndSuppliers {

    static final int ITERATIONS = 3;
    static final int MATRIX_SIZE_SCALE = 1;

    public static void main(final String[] args) {

        BasicLogger.debug();
        BasicLogger.debug(ElementConsumersAndSuppliers.class.getSimpleName());
        BasicLogger.debug(OjAlgoUtils.getTitle());
        BasicLogger.debug(OjAlgoUtils.getDate());
        BasicLogger.debug();

        final Normal tmpNormalDistribution = new Normal();
        final Uniform tmpUniformDistribution = new Uniform();

        // Assume you have the matrices [A],[B],[C] and [D]
        final PrimitiveDenseStore tmpMtrxA = PrimitiveDenseStore.FACTORY.makeZero(5 * MATRIX_SIZE_SCALE, 7 * MATRIX_SIZE_SCALE);
        final PrimitiveDenseStore tmpMtrxB = PrimitiveDenseStore.FACTORY.makeZero(7 * MATRIX_SIZE_SCALE, 9 * MATRIX_SIZE_SCALE);
        final PrimitiveDenseStore tmpMtrxC = PrimitiveDenseStore.FACTORY.makeZero(5 * MATRIX_SIZE_SCALE, 9 * MATRIX_SIZE_SCALE);
        final PrimitiveDenseStore tmpMtrxD = PrimitiveDenseStore.FACTORY.makeZero(9 * MATRIX_SIZE_SCALE, 5 * MATRIX_SIZE_SCALE);

        final QR<Double> decompQR = QR.PRIMITIVE.make();

        // Imagine the matrices involved to be gigantic and the iterations (loop) many
        for (int l = 1; l <= ITERATIONS; l++) {

            BasicLogger.debug();
            BasicLogger.debug("Iteration {}", l);

            tmpMtrxA.fillAll(tmpUniformDistribution);
            tmpMtrxB.fillAll(tmpNormalDistribution);

            // [E] = [A][B]
            final ElementsSupplier<Double> tmpIntermediateE = tmpMtrxB.premultiply(tmpMtrxA);
            // Note that the multiplcation is not yet performed - it's a placeholder for a deferred operation.
            BasicLogger.debug("The matrix [E] = [A][B] will have {} rows and {} columns.", tmpIntermediateE.countRows(), tmpIntermediateE.countColumns());

            tmpMtrxC.fillAll(tmpUniformDistribution);

            // [F] = [A][B] - [C]
            final ElementsSupplier<Double> tmpIntermediateF = tmpIntermediateE.operateOnMatching(SUBTRACT, tmpMtrxC);
            // Still, nothing is actually done - no calculations and no memory/array allocation.
            BasicLogger.debug("The matrix [F] = [A][B] - [C] will have {} rows and {} columns.", tmpIntermediateF.countRows(), tmpIntermediateF.countColumns());

            // [G] = [F]t
            final ElementsSupplier<Double> tmpIntermediateG = tmpIntermediateF.transpose();
            // Still nothing...
            BasicLogger.debug("The matrix [G] will have {} rows and {} columns.", tmpIntermediateG.countRows(), tmpIntermediateG.countColumns());

            // Now it happens all at once!
            tmpIntermediateG.supplyTo(tmpMtrxD);
            // ...and you get to decide where to put the results.

            if (tmpIntermediateG.count() <= 100) {
                BasicLogger.debug("Resulting D", tmpMtrxD);
            }

            // You can supply it directly to a decomposition's internal storage
            // (as you decompose it)
            decompQR.decompose(tmpIntermediateG);
            // The matrix that was decomposed never "existed" other than as an intermediate state of the matrix decomposition

            BasicLogger.debug();
            BasicLogger.debug("Is it full rank? {}", decompQR.isFullRank());
        }

    }

}

Console output

Not very interesting at all...

What you should do take the code and start experimenting yourself.

ElementConsumersAndSuppliers
ojAlgo
2018-02-28


Iteration 1
The matrix [E] = [A][B] will have 5 rows and 9 columns.
The matrix [F] = [A][B] - [C] will have 5 rows and 9 columns.
The matrix [G] will have 9 rows and 5 columns.
Resulting D
  0.025434 -0.424117  0.130926  0.651991  1.216903
 -1.782048 -1.103734 -1.211828 -1.598301 -1.816968
 -1.174533 -1.527534  0.966207 -0.972809  -1.24818
  0.397046  0.488031   0.06442  0.447535  0.176725
  0.596547  0.779567  0.703685  0.443411  0.120253
 -2.299555  -1.89674 -1.165506 -2.370726 -2.891807
  3.378697  2.826955  2.798604  1.855877  2.328441
 -1.370901 -0.005162  -0.12012  0.665445  0.043749
  1.841179  1.806975  1.856241  2.676544  3.521646

Is it full rank? true

Iteration 2
The matrix [E] = [A][B] will have 5 rows and 9 columns.
The matrix [F] = [A][B] - [C] will have 5 rows and 9 columns.
The matrix [G] will have 9 rows and 5 columns.
Resulting D
  0.450999  0.046015 -1.274544 -0.481033  0.358168
 -1.976392  -2.77942  -3.59073 -1.724603 -2.898888
 -2.440217 -1.285213  -1.66719 -1.086323 -1.803063
   0.00665 -0.325289 -0.476657 -0.165752 -2.221526
 -2.667492 -2.937119 -4.047953 -1.703481 -3.466048
  0.127031  0.320149   1.56246 -0.145031  1.101951
  -1.08035 -0.222369 -1.614321  -0.43837 -1.205335
 -0.287134 -0.987389  -0.48585 -1.761696 -0.948811
 -1.840792 -1.576714 -3.325838  -2.26672 -2.899239

Is it full rank? true

Iteration 3
The matrix [E] = [A][B] will have 5 rows and 9 columns.
The matrix [F] = [A][B] - [C] will have 5 rows and 9 columns.
The matrix [G] will have 9 rows and 5 columns.
Resulting D
 -3.664938  1.030576 -0.278562 -2.488402 -0.949271
 -3.152721 -4.819521 -2.564122 -5.928387 -3.256222
 -1.874802  0.388996   0.01465 -0.776643 -0.372932
 -1.983605 -0.703812 -0.428568 -1.130364  0.638484
  0.104787 -1.102666 -0.834411  1.239207  0.182051
  0.433684   0.27971  0.323491  1.300327  1.527809
 -1.112619  -0.41358  -0.49789     0.129 -0.461661
  1.164316 -0.053617 -0.265128  0.412615 -0.612532
 -0.500644 -1.063022 -0.700942  -1.75428 -1.151864

Is it full rank? true