Unable to find mistake when using PosteriorMean acquisition function

[sambitmishra98]

I am trying to use PosteriorMean acquisition function to get the best possible candidate from my model (goal is to get the candidate with the lowest cost). I tested the model with one datum [1, 1, 1, 1] from the training dataset and got result as expected. I tried to test the model with a candidate [1, 2, 2, 2] which is known to perform better than [1, 1, 1, 1] and I am getting a lower cost as expected.

Issue is that the candidate I am being suggested by the optimiser isn't really a good suggestion. The following are two candidates from the dataset itself, the second candidate is the best candidate from the dataset.

cycle_1	cycle_2	cycle_3	cycle_4	cost-avg	cost-var	cost-sem
1.0	1.0	1.0	1.0	54.0	11.62	0.001689
0.48221924901008606	5.345752716064453	14.231993675231934	10.189181327819824	9.01	0.003355	0.01075591283752577

I am probably using the acquisition function wrongly.

import numpy as np

from botorch.optim.optimize import optimize_acqf
from botorch.models import FixedNoiseGP
from botorch.acquisition import PosteriorMean as PM
from gpytorch.mlls import ExactMarginalLogLikelihood
from botorch.models.transforms import Normalize, Standardize

import torch
torch_kwargs = {'dtype': torch.float64, 'device': torch.device("cpu")}

cv_args = {'model_cls': FixedNoiseGP,
           'mll_cls': ExactMarginalLogLikelihood,
           }

class optimiser:
    def __init__(self, train_X_Y_Yvar, bounds, track=None):
        if track:
            torch.random.manual_seed(track)

        self.initialise_data_model(train_X_Y_Yvar, bounds)

    def initialise_data_model(self,  train_dat: np.ndarray, bounds: np.ndarray):
        self.bnds = torch.tensor(bounds, **torch_kwargs)

        self.Y_standardiser = Standardize(m=1)
        self.X_normaliser = Normalize(
            d=train_dat.shape[-1]-3, bounds=self.bnds)

        self.train_X = self.X_normaliser.forward(
            torch.tensor(train_dat[:, :-3], **torch_kwargs))
        #self.train_Y, self.train_Yvar = self.Y_standardiser(torch.tensor(train_dat[:, -3:-2], **torch_kwargs),
        #                                                    torch.sqrt(
        #                                                        torch.tensor(train_dat[:, -2:-1], **torch_kwargs)
        #                                                        )
        #                                                    )

        self.train_Y, self.train_Yvar = self.Y_standardiser(torch.tensor(train_dat[:, -3:-2], **torch_kwargs),
                                                            torch.tensor(train_dat[:, -2:-1], **torch_kwargs))

        self.model = FixedNoiseGP(
            train_X    = self.train_X,
            train_Y    = self.train_Y,
            train_Yvar = self.train_Yvar,
        )
        mll = ExactMarginalLogLikelihood(
            likelihood = self.model.likelihood,
            model      = self.model,
        )
        mll = mll.to(self.train_X)
        fit_model(mll)
        

    def __optimise_now(self, raw_samples=512, num_restarts=10)-> torch.Tensor:
        self.X_cand, _ = optimize_acqf(
            acq_function = self.__acquisition_function,
            bounds       = self.bnds,
            q            = 1,
            num_restarts = num_restarts,
            raw_samples  = raw_samples,
        )
        return self.X_cand

    def evaluatePM(self, raw_samples=256, num_restarts=10):
        """
            Get the minimum from posterior mean.
            You may want HeteroskedasticSingleTaskGP here.
        """

        self.__acquisition_function = PM(
            self.model, 
            #posterior_transform = ScalarizedPosteriorTransform(weights=torch.tensor([-1.0])),
            maximize = False,
            )

        return self.__optimise_now(raw_samples = raw_samples, num_restarts = num_restarts)

    def evaluate(self, point:list[float]|torch.Tensor) -> dict[str, float]:

        if isinstance(point, torch.Tensor):
            tens = point
        else:
            try:
                tens = torch.tensor(point, dtype=torch.float32)
            except:
                raise Exception("not the right format")
    
        best = self.X_normaliser.forward(tens)

        with torch.no_grad():

            mean = self.model.posterior(best).mvn.mean
            lower, upper = self.model.posterior(best).mvn.confidence_region()

            return {'lower': self.Y_standardiser.untransform(lower)[0].tolist()[0][0],
                    'mean' : self.Y_standardiser.untransform(mean)[ 0].tolist()[0][0], 
                    'upper': self.Y_standardiser.untransform(upper)[0].tolist()[0][0],
                    }

if __name__ == "__main__":

    train_X_Y_Yvar = np.array([[1.0, 1.0, 1.0, 1.0, 11.62, 0.001689, 0.007383067072450739],
[8.98597526550293, 3.515949010848999, 8.994438171386719, 7.538417339324951, 9.805, 0.0672, 0.04814659402930398],
[9.849862098693848, 4.600430965423584, 4.970165252685547, 6.998086929321289, 9.52, 0.00063, 0.004661084461994176],
[11.369607925415039, 5.492735862731934, 6.927404403686523, 8.192280769348145, 9.836, 0.01596, 0.02346126403970321],
[4.154900074005127, 6.704091548919678, 8.69089126586914, 2.360156774520874, 14.18, 0.3765, 0.11397414066823935],
[7.243962287902832, 3.5690810680389404, 6.943271160125732, 9.905169486999512, 9.58, 0.001121, 0.006216668273322324],
[0.9397330284118652, 4.105175971984863, 12.018691062927246, 11.487043380737305, 9.336, 0.000606, 0.004570412910149912],
[15.057971954345703, 7.794610023498535, 1.1539400815963745, 8.244453430175781, 9.72, 0.000713, 0.004958600491483166],
[7.383619785308838, 0.8112460374832153, 6.504706382751465, 12.615680694580078, 9.54, 0.001907, 0.008109436918071327],
[13.669303894042969, 6.111311435699463, 1.1081206798553467, 0.10148591548204422, 44.94, 6.434, 0.44165405290322557],
[12.19154167175293, 6.026062965393066, 3.8678860664367676, 4.764541149139404, 11.17, 0.1183, 0.06387810827428371],
[14.36771297454834, 7.604015350341797, 9.374358177185059, 3.6606736183166504, 17.3, 0.9473, 0.16940271890610634],
[9.484539031982422, 6.484375, 13.779655456542969, 9.529952049255371, 9.836, 0.05963, 0.0453357759221318],
[7.42392635345459, 6.527213096618652, 8.775080680847168, 15.500592231750488, 9.32, 0.00924, 0.0178509617693394],
[4.884446620941162, 1.0657007694244385, 1.1518265008926392, 0.4736602306365967, 15.375, 0.4512, 0.12476405533770672],
[18.573400497436523, 6.670269966125488, 6.131234645843506, 11.22675895690918, 9.8, 0.03824, 0.03631395651362757],
[11.491645812988281, 7.015653610229492, 11.831451416015625, 4.287281036376953, 14.16, 0.41, 0.1188704044678296],
[4.205890655517578, 7.009191989898682, 4.638201713562012, 15.623321533203125, 9.76, 0.02339, 0.028402863615215577],
[15.536617279052734, 3.443092107772827, 6.728132724761963, 3.279031753540039, 15.67, 0.5444, 0.1370047148366823],
[3.3668606281280518, 4.602254390716553, 6.811132431030273, 12.518314361572266, 9.26, 0.00468, 0.012705351202177438],
[7.28032922744751, 1.9703459739685059, 11.872417449951172, 9.901084899902344, 9.14, 0.0009027, 0.005579133924417346],
[6.08379602432251, 0.44067779183387756, 15.608699798583984, 5.698777198791504, 10.59, 0.2382, 0.09062621606834148],
[0.920843780040741, 3.092834949493408, 1.0248475074768066, 12.079353332519531, 10.14, 0.01779, 0.02477600154144503],
[17.745704650878906, 7.1188249588012695, 12.792621612548828, 17.499713897705078, 10.39, 0.00345, 0.010908921081262965],
[10.108514785766602, 6.877035617828369, 2.8820695877075195, 16.6420841217041, 9.766, 0.006496, 0.014972139998284029],
[10.246114730834961, 6.400475025177002, 9.293664932250977, 6.572184085845947, 10.3, 0.0928, 0.05657904835082049],
[15.438278198242188, 9.521648406982422, 11.278777122497559, 15.920896530151367, 9.88, 0.01926, 0.02577338861173193],
[16.136524200439453, 3.88053297996521, 14.667619705200195, 16.64692497253418, 10.09, 0.003183, 0.010478231210002714],
[17.258230209350586, 8.568090438842773, 14.553056716918945, 7.396767616271973, 12.63, 0.3462, 0.10925921997233765],
[16.909381866455078, 4.888452529907227, 13.723743438720703, 14.476818084716797, 9.83, 0.00276, 0.009752858795248604],
[15.781805038452148, 3.5793685913085938, 11.718746185302734, 15.334735870361328, 9.7, 0.002771, 0.00977552668320967],
[20.074981689453125, 3.294785499572754, 1.104918360710144, 8.011820793151855, 10.02, 0.134, 0.0680036638831977],
[2.1058764457702637, 7.891506195068359, 7.775669574737549, 13.336424827575684, 9.78, 0.00909, 0.01770362049759247],
[3.6109678745269775, 7.487239837646484, 8.11462688446045, 5.621033191680908, 9.984, 0.00996, 0.018530998408171375],
[6.676307201385498, 9.70628547668457, 10.997719764709473, 16.95246124267578, 9.27, 0.007725, 0.01632087933196745],
[6.181928634643555, 8.28036880493164, 5.796778678894043, 3.6736092567443848, 11.43, 0.1632, 0.07503070915112814],
[8.553505897521973, 7.6846208572387695, 4.2577924728393555, 0.19275233149528503, 35.88, 2.537, 0.2957706021159879],
[6.860222816467285, 9.25721549987793, 14.607281684875488, 3.9291837215423584, 14.66, 0.5503, 0.13773008725143643],
[4.370328903198242, 4.283802032470703, 13.207708358764648, 10.069365501403809, 9.586, 0.001149, 0.006296005881186054],
[14.377578735351562, 9.256193161010742, 13.383161544799805, 4.6994547843933105, 15.55, 0.633, 0.1477039579543054],
[5.304084300994873, 3.590442657470703, 3.252551317214966, 13.088006973266602, 9.35, 0.005196, 0.013385387841009415],
[4.6886186599731445, 8.879745483398438, 0.24510399997234344, 4.303295612335205, 10.52, 0.00703, 0.015572839029252276],
[17.942468643188477, 1.364927887916565, 9.601430892944336, 3.1250810623168945, 18.8, 0.8364, 0.16982781660430574],
[17.455320358276367, 3.289930582046509, 3.8872628211975098, 8.096877098083496, 9.88, 0.03726, 0.03583793086644519],
[4.314944744110107, 6.873885631561279, 1.4737558364868164, 9.918482780456543, 9.63, 0.002764, 0.009764192739229138],
[16.74081039428711, 0.644906222820282, 11.166403770446777, 11.25849437713623, 10.1, 0.002054, 0.008415453405545717],
[17.704763412475586, 9.523462295532227, 9.615211486816406, 17.804189682006836, 9.9, 0.00572, 0.014042756591880326],
[13.656855583190918, 8.498209953308105, 10.917299270629883, 13.819327354431152, 9.54, 0.1998, 0.08300980571342334],
[11.65049934387207, 6.797582149505615, 8.023750305175781, 0.70789635181427, 34.9, 3.428, 0.3438265245934476],
[8.088726043701172, 6.792218208312988, 8.28805923461914, 14.839882850646973, 9.445, 0.00534, 0.013566730944697942],
[11.87939453125, 1.1403306722640991, 7.195440292358398, 6.2738213539123535, 10.15, 0.08527, 0.054221588002869636],
[19.523244857788086, 4.723472595214844, 7.559463024139404, 9.016617774963379, 10.17, 0.14, 0.06949974448862806],
[10.932744979858398, 8.843971252441406, 4.296719074249268, 5.5199294090271, 10.66, 0.0653, 0.047466557390472],
[17.81005859375, 4.646005630493164, 3.17171311378479, 8.192991256713867, 9.92, 0.02591, 0.029898944220645925],
[13.099006652832031, 4.632573127746582, 5.837216377258301, 7.994953155517578, 9.92, 0.000652, 0.004740422069857907],
[14.116206169128418, 1.7966647148132324, 6.1606879234313965, 20.160436630249023, 9.766, 0.02225, 0.02770015908842253],
[1.936470866203308, 6.144766807556152, 9.671148300170898, 2.029266595840454, 14.125, 0.4158, 0.11977711998627223],
[5.808743000030518, 4.0196533203125, 13.93490982055664, 3.6554200649261475, 13.375, 0.4282, 0.12149987947131324],
[4.468235969543457, 6.729650497436523, 8.575510025024414, 9.440250396728516, 9.39, 0.004112, 0.011911975123540131],
[10.513311386108398, 2.3951456546783447, 17.50869369506836, 4.186591148376465, 15.266, 0.5703, 0.1402688907030758],
[6.162827968597412, 2.6590917110443115, 9.820477485656738, 17.03623390197754, 9.42, 0.01327, 0.021387152291265678],
[4.582155227661133, 3.908531427383423, 2.775120258331299, 16.87796401977539, 9.664, 0.086, 0.0544482668824803],
[10.845364570617676, 3.1593456268310547, 8.928672790527344, 3.375709295272827, 14.586, 0.4473, 0.12422002602664114],
[8.322317123413086, 5.087471008300781, 12.4819974899292, 12.745932579040527, 9.19, 0.04648, 0.040031490139242384],
[5.887582302093506, 1.4788665771484375, 11.586141586303711, 14.051727294921875, 9.19, 0.002293, 0.0088914790527281],
[20.304655075073242, 3.4043896198272705, 16.647188186645508, 9.921747207641602, 10.86, 0.0855, 0.0543122595547139],
[10.2387056350708, 7.915661334991455, 5.353455066680908, 15.866303443908691, 9.42, 0.007744, 0.016343547219928516],
[13.202863693237305, 2.1520559787750244, 16.53597068786621, 5.285725116729736, 13.98, 0.4612, 0.12612412861537067],
[0.48221924901008606, 5.345752716064453, 14.231993675231934, 10.189181327819824, 9.01, 0.003355, 0.01075591283752577],
[6.538642883300781, 3.7341134548187256, 1.5639355182647705, 10.918636322021484, 9.48, 0.04092, 0.0375606903514862],
[9.890704154968262, 5.808599948883057, 1.0367724895477295, 0.7666738033294678, 22.6, 0.782, 0.1642061803899614],
[21.976346969604492, 3.699249267578125, 13.59854793548584, 12.116483688354492, 10.76, 0.003778, 0.011413281588396681],
[2.183483839035034, 1.5974398851394653, 3.483039140701294, 20.865076065063477, 9.836, 0.03452, 0.0345005254767423],
[0.04266921803355217, 6.036769866943359, 15.788650512695312, 16.746706008911133, 9.66, 0.01107, 0.019539719422438807],
[10.860538482666016, 6.396950721740723, 6.37175989151001, 4.698273658752441, 11.8, 0.2435, 0.09162360313862838],
[9.756772994995117, 5.217047691345215, 6.913456916809082, 19.86431312561035, 9.83, 0.0129, 0.021092469747771823],
[21.630878448486328, 8.836173057556152, 7.321562767028809, 13.012723922729492, 10.055, 0.1042, 0.05993389576905825],
[18.197792053222656, 7.845048427581787, 3.3553056716918945, 7.4780354499816895, 10.83, 0.1785, 0.07843089234528802],
[2.9301812648773193, 1.4144682884216309, 6.814985275268555, 2.8646297454833984, 10.555, 0.02037, 0.02649876102648604],
[8.907309532165527, 10.874872207641602, 9.649421691894531, 3.8322598934173584, 14.56, 0.488, 0.12975099068914123],
[0.8418487906455994, 5.075315952301025, 4.196861743927002, 15.207015037536621, 9.87, 0.00948, 0.01808897459293059],
[5.032914638519287, 0.07103432714939117, 6.624518394470215, 9.031562805175781, 9.58, 0.0007215, 0.004986935351434498],
[22.40474510192871, 9.23752498626709, 14.790619850158691, 11.60804557800293, 11.445, 0.1985, 0.08273779105789054],
[11.447603225708008, 2.3128576278686523, 17.193979263305664, 2.594630479812622, 22.7, 1.14, 0.19820801233156024],
[8.8720703125, 7.624041557312012, 14.313468933105469, 20.899351119995117, 9.45, 0.003937, 0.011651294411987874],
[4.634416580200195, 2.0752358436584473, 5.727588176727295, 13.068854331970215, 9.13, 0.002071, 0.008449455237487314],
[23.782514572143555, 1.2269810438156128, 12.032306671142578, 11.903139114379883, 10.83, 0.000567, 0.004420238152407851],
[5.600986003875732, 5.877364635467529, 6.598578453063965, 15.207565307617188, 9.11, 0.005093, 0.01324938051324302],
[16.0549259185791, 1.2416613101959229, 6.260197639465332, 17.40581703186035, 9.73, 0.06976, 0.049053309547746615],
[8.771440505981445, 7.926077842712402, 12.989253997802734, 16.09871482849121, 9.73, 0.001872, 0.008035766282197862],
[19.653440475463867, 10.16956901550293, 15.158709526062012, 5.370415210723877, 17.66, 0.7905, 0.16511289590840403],
[19.195606231689453, 5.945725917816162, 16.602209091186523, 4.584000587463379, 18.86, 0.875, 0.17372669333360907],
[14.638741493225098, 9.943954467773438, 4.182914733886719, 18.769437789916992, 9.85, 0.001565, 0.007344395699385353],
[7.358680248260498, 4.938865661621094, 7.120260715484619, 9.651960372924805, 9.734, 0.002121, 0.008551460733312111],
[24.74808120727539, 11.730527877807617, 6.015953540802002, 19.55328369140625, 9.59, 0.00348, 0.010954256857185098],
[3.0672061443328857, 8.03030776977539, 11.745855331420898, 14.896661758422852, 9.16, 0.00786, 0.016456886659733844],
[20.413095474243164, 6.172313690185547, 1.1965612173080444, 16.916645050048828, 9.59, 0.05954, 0.04531310803417074],
[10.657082557678223, 10.44234561920166, 2.5617947578430176, 18.243303298950195, 9.29, 0.00888, 0.017499609505942877],
[22.140531539916992, 4.409979820251465, 12.010472297668457, 4.891209602355957, 17.86, 0.758, 0.161667376938322],
[18.585830688476562, 8.13814640045166, 5.098474025726318, 16.643367767333984, 9.414, 0.007637, 0.016230207780123185],
[18.74945640563965, 7.1748433113098145, 12.394200325012207, 8.02175235748291, 11.945, 0.3054, 0.10264019668770641],
[7.7858686447143555, 10.257271766662598, 15.06873893737793, 16.111974716186523, 9.414, 0.0882, 0.055128303521312275],
[13.915267944335938, 7.201131820678711, 5.65164852142334, 8.96745777130127, 10.03, 0.001519, 0.0072367232315702894],
[3.179813861846924, 1.6187782287597656, 5.5972795486450195, 18.280376434326172, 9.305, 0.01156, 0.019959075349718528],
[5.893837928771973, 3.501549005508423, 4.616600036621094, 10.995903968811035, 9.76, 0.0256, 0.0297176011169574],
[3.2646050453186035, 8.231409072875977, 2.343456983566284, 4.430354118347168, 10.53, 0.03488, 0.03468186858043083],
[18.05000114440918, 4.804300785064697, 13.326313018798828, 1.3778246641159058, 38.53, 3.963, 0.3574566460259446],
[8.218573570251465, 9.800504684448242, 12.48350715637207, 22.51791763305664, 9.1, 0.006237, 0.014666123510809639],
[2.3738927841186523, 8.870429992675781, 12.737567901611328, 7.594089984893799, 9.96, 0.07074, 0.04937065997920154],
[22.03862190246582, 9.397616386413574, 2.0672411918640137, 14.788460731506348, 10.28, 0.0994, 0.05857382249139429],
[18.63474464416504, 9.218847274780273, 11.59217643737793, 22.214468002319336, 9.65, 0.006664, 0.015164817045953089],
[7.748866558074951, 3.007596969604492, 15.654098510742188, 5.685500621795654, 11.74, 0.1885, 0.08060700958955035],
[20.687320709228516, 3.773313045501709, 13.000941276550293, 17.7746524810791, 9.63, 0.005318, 0.013544063056736876],
[3.073549509048462, 8.632601737976074, 0.1892542690038681, 7.128441333770752, 10.46, 0.03348, 0.03397916405363779],
[2.909754514694214, 7.262402534484863, 7.573165416717529, 4.4111175537109375, 10.41, 0.0915, 0.056171026367521304],
[6.114389896392822, 11.792911529541016, 17.493436813354492, 1.3621068000793457, 33.44, 3.246, 0.32895446568913783],
[5.652334213256836, 1.410930871963501, 14.785002708435059, 17.071420669555664, 9.41, 0.006138, 0.01455278407100431],
[16.570981979370117, 11.11379623413086, 3.690096616744995, 12.639854431152344, 9.766, 0.00795, 0.01655889215555864],
[15.939230918884277, 4.415098667144775, 16.731239318847656, 11.21186351776123, 10.234, 0.000874, 0.005491295858568215],
[8.13746166229248, 10.573674201965332, 12.343387603759766, 4.298381328582764, 14.01, 0.4514, 0.12476405533770672],
[10.833751678466797, 12.203790664672852, 0.39462462067604065, 8.195001602172852, 10.195, 0.001195, 0.006420679264971917],
[2.3215506076812744, 12.292582511901855, 10.5802640914917, 21.646203994750977, 9.93, 0.02672, 0.030352301979867245],
[26.158283233642578, 3.331131935119629, 4.080850601196289, 8.564895629882812, 11.34, 0.2361, 0.0902181940850423],
[20.69098663330078, 5.702590465545654, 10.281820297241211, 5.7267937660217285, 14.984, 0.6387, 0.14356146455948757],
[4.121107578277588, 1.5943760871887207, 16.651256561279297, 11.117679595947266, 9.805, 0.013855, 0.021863177938448063],
[23.929548263549805, 10.763228416442871, 3.8210997581481934, 10.367431640625, 11.234, 0.02934, 0.031803046809375464],
[11.935973167419434, 9.756528854370117, 13.567544937133789, 22.957887649536133, 10.0, 0.01227, 0.020571108324667308],
[15.377453804016113, 9.035322189331055, 12.128118515014648, 21.220945358276367, 10.14, 0.00722, 0.01577685002090187],
[1.4372848272323608, 1.343258261680603, 4.298761367797852, 21.739248275756836, 9.77, 0.01987, 0.026181410595031117],
[14.251113891601562, 11.599821090698242, 6.961690425872803, 15.406325340270996, 9.78, 0.003134, 0.010393226630148716],
[14.33815860748291, 8.806207656860352, 0.6823679208755493, 1.0596727132797241, 25.78, 1.394, 0.2192438123594294],
[1.7475149631500244, 3.5922482013702393, 4.911562442779541, 0.6504946351051331, 16.66, 0.6553, 0.15033343295778906],
[19.3388614654541, 3.132617473602295, 14.62157917022705, 2.5413784980773926, 27.48, 1.822, 0.2506161692975446],
[4.463589668273926, 3.282318353652954, 14.2955322265625, 9.401823043823242, 9.61, 0.1547, 0.07303593501055433],
[13.074141502380371, 5.719539165496826, 12.974993705749512, 19.082170486450195, 9.51, 0.05127, 0.04204893216777725],
[23.215272903442383, 10.894474029541016, 15.13498306274414, 14.66445541381836, 10.03, 0.0963, 0.05762177119702953],
[0.8488757610321045, 9.471115112304688, 5.910003185272217, 5.299385070800781, 10.18, 0.0172, 0.02434531167018478],
[21.68509292602539, 3.8984532356262207, 3.1962649822235107, 21.912593841552734, 9.86, 0.005146, 0.013317384177126218],
[10.032824516296387, 9.435309410095215, 15.374613761901855, 1.4087802171707153, 33.5, 3.363, 0.32939331758426105],
[7.319126605987549, 0.11056067794561386, 2.249762535095215, 18.421709060668945, 9.47, 0.01134, 0.01977773224603],
[19.8909854888916, 1.3525668382644653, 11.558869361877441, 13.285350799560547, 9.83, 0.011795, 0.02016308634136812],
[11.031196594238281, 5.980291843414307, 17.834720611572266, 20.73859405517578, 10.29, 0.012886, 0.021081135803791288],
[17.05265998840332, 1.9963074922561646, 1.8995400667190552, 22.11697006225586, 9.51, 0.01082, 0.01931304054282815],
[13.142910957336426, 2.5221450328826904, 17.509305953979492, 1.34976327419281, 37.03, 4.004, 0.3592106040535498],
[2.933830738067627, 5.568336486816406, 2.3819420337677, 17.43478012084961, 9.555, 0.02184, 0.02745081232085081],
[4.432783603668213, 7.402700424194336, 8.114336967468262, 1.6551861763000488, 18.17, 0.7534, 0.1612140191791007],
[9.804427146911621, 12.733393669128418, 7.248822212219238, 8.98603630065918, 10.03, 0.1219, 0.06483015956864849],
[16.926925659179688, 4.1044111251831055, 7.361841678619385, 11.792774200439453, 9.84, 0.0758, 0.051138755240164674],
[5.721221923828125, 3.7213757038116455, 17.72142791748047, 12.21454906463623, 9.75, 0.07764, 0.05172812032715239],
[23.294225692749023, 9.833447456359863, 12.858981132507324, 22.35579490661621, 9.98, 0.010216, 0.018769011231762566],
[22.507795333862305, 12.89669132232666, 4.72636604309082, 25.516273498535156, 9.38, 0.00606, 0.014462112519160046],
[3.5095136165618896, 11.29843807220459, 2.974088430404663, 21.58329200744629, 9.29, 0.05972, 0.045381111698053936],
[12.29919147491455, 4.70257043838501, 6.415342807769775, 7.966999053955078, 10.08, 0.03592, 0.03520323000353535],
[7.213357925415039, 2.0374741554260254, 4.742248058319092, 23.849931716918945, 9.77, 0.02304, 0.028176184735604917],
[8.368751525878906, 4.941156387329102, 6.701932430267334, 13.360447883605957, 9.76, 0.00985, 0.01842899291234658],
[8.098868370056152, 5.686507225036621, 17.092374801635742, 12.968772888183594, 10.25, 0.006855, 0.015380161981583216],
[2.0902395248413086, 0.9945117831230164, 2.671900749206543, 18.27457618713379, 9.59, 0.029, 0.031621703705686934],
[10.375020027160645, 0.7085985541343689, 9.856907844543457, 15.21552848815918, 9.94, 0.00811, 0.0167175673712861],
[2.0190470218658447, 12.146888732910156, 3.2130355834960938, 3.7883431911468506, 11.875, 0.0519, 0.042298278935348974],
[17.765663146972656, 4.746570110321045, 11.300056457519531, 21.715639114379883, 10.34, 0.02922, 0.03173504314549226],
[5.649542808532715, 2.605647325515747, 6.361091613769531, 1.8912757635116577, 15.39, 1.212, 0.176234794275716],
[23.81405258178711, 2.212109327316284, 15.860865592956543, 22.6563663482666, 10.445, 0.05344, 0.042932979798258816],
[7.670902252197266, 11.112091064453125, 7.904172420501709, 16.657543182373047, 9.7, 0.01064, 0.019154365327100686],
[10.89069938659668, 10.039435386657715, 18.870227813720703, 11.981986999511719, 9.98, 0.010796, 0.019290372654867084],
[1.0177513360977173, 5.050332546234131, 10.23421859741211, 6.1937432289123535, 9.62, 0.01556, 0.02315524755222882],
[21.151126861572266, 9.878508567810059, 2.21356201171875, 9.501884460449219, 10.78, 0.1787, 0.07852156389713229],
[12.178799629211426, 5.389883518218994, 5.434438705444336, 20.188276290893555, 10.164, 0.01839, 0.025184023524744217],
[17.9094295501709, 4.170403957366943, 0.04301454871892929, 10.92287826538086, 9.35, 0.02351, 0.028470867279098772],
[13.484028816223145, 2.8446779251098633, 13.07066822052002, 14.292488098144531, 9.41, 0.010994, 0.01947171575855561],
[19.21687126159668, 8.48055362701416, 5.592600345611572, 24.720043182373047, 9.87, 0.0162, 0.023642607143391735],
[24.533113479614258, 7.481379508972168, 8.519625663757324, 4.964827537536621, 17.72, 0.543, 0.13682337173299378],
[18.886022567749023, 0.22358338534832, 0.4332773685455322, 8.063755989074707, 10.164, 0.02724, 0.0306469845233611],
[15.961647987365723, 0.8039363622665405, 11.127763748168945, 7.2693047523498535, 11.45, 0.0374, 0.035928602418289456],
[19.533733367919922, 3.01826548576355, 6.367339134216309, 12.671701431274414, 10.125, 0.01628, 0.023687942919313868],
[18.129491806030273, 4.357052326202393, 16.409265518188477, 25.299915313720703, 9.48, 0.01855, 0.025297362964549548],
[20.30126953125, 2.2271199226379395, 3.8050174713134766, 16.498329162597656, 9.99, 0.01811, 0.024980012533094623],
[5.896067142486572, 5.246772766113281, 13.438998222351074, 6.17702054977417, 10.58, 0.1349, 0.06818500698688623],
[22.732006072998047, 1.0641369819641113, 19.842662811279297, 10.88791275024414, 11.92, 0.3909, 0.11605958636065743],
[9.69402027130127, 12.178669929504395, 9.363137245178223, 18.466106414794922, 9.51, 0.00792, 0.016524890323617043],
[14.282175064086914, 12.03320598602295, 5.5931620597839355, 0.9558091759681702, 34.1, 2.979, 0.3204332642176276],
[18.949949264526367, 3.996544122695923, 16.327653884887695, 18.87925148010254, 9.97, 0.01082, 0.01931304054282815],
[22.931297302246094, 7.622402667999268, 1.3513906002044678, 26.205347061157227, 9.6, 0.003891, 0.011583290748104677],
[19.461400985717773, 4.800961494445801, 16.841035842895508, 19.07670783996582, 9.57, 0.003658, 0.011231938484708154],
[5.307003974914551, 1.722469687461853, 4.341961860656738, 16.36591148376465, 9.39, 0.01712, 0.024299975894262648],
[11.153217315673828, 4.138976573944092, 16.01484489440918, 11.878710746765137, 10.234, 0.1315, 0.06732362724436573],
[14.120287895202637, 2.8278892040252686, 11.071091651916504, 27.70237922668457, 9.79, 0.02696, 0.03048830930763364],
[17.06544303894043, 8.509339332580566, 19.266489028930664, 5.93134069442749, 16.12, 0.5947, 0.14317038036209223],
[24.11313819885254, 11.297439575195312, 8.517021179199219, 11.685956954956055, 10.805, 0.01762, 0.024662662101639702],
[5.050083637237549, 7.0630202293396, 15.939864158630371, 20.262657165527344, 9.94, 0.00866, 0.01728426457031275],
[20.90810775756836, 12.0457763671875, 16.81951332092285, 16.04167366027832, 10.06, 0.00899, 0.017601615001767674],
[17.06654167175293, 9.966872215270996, 9.890437126159668, 8.35505199432373, 11.45, 0.3987, 0.11339338648467771],
[4.407581329345703, 6.578131675720215, 4.0810112953186035, 11.767367362976074, 9.4, 0.0334, 0.033933828277715655],
[1.7006417512893677, 9.833636283874512, 16.682754516601562, 5.460238933563232, 11.28, 0.108, 0.06102195439118941],
[21.715932846069336, 5.446423530578613, 11.738332748413086, 1.9629361629486084, 32.5, 2.559, 0.29704000384180756],
[14.778461456298828, 3.388528823852539, 6.156041145324707, 18.069265365600586, 9.92, 0.007618, 0.016207539892162122],
[3.769395351409912, 1.4543412923812866, 9.86981201171875, 2.2908437252044678, 13.52, 0.514, 0.13319650965922325],
[25.907297134399414, 2.913574695587158, 2.1598899364471436, 26.114940643310547, 10.31, 0.01299, 0.021171807355635553],
[14.424359321594238, 4.420619964599609, 4.25377893447876, 23.551956176757812, 9.734, 0.02089, 0.02683877934590203],
[25.98914909362793, 12.432368278503418, 10.2712984085083, 3.8912296295166016, 23.84, 1.208, 0.20401099164959313],
[5.365392208099365, 3.8055717945098877, 10.205010414123535, 3.7030272483825684, 12.1, 0.277, 0.09774393288811617],
[20.704824447631836, 9.297612190246582, 1.3262628316879272, 6.957981586456299, 12.36, 0.325, 0.10581370100225564],
[19.839460372924805, 10.169946670532227, 2.895620107650757, 26.014183044433594, 9.766, 0.071, 0.0494613315310458],
[3.0529916286468506, 3.4862308502197266, 3.847447395324707, 11.620223045349121, 9.36, 0.0168, 0.02407329701465199],
[19.378400802612305, 11.538090705871582, 5.713446617126465, 23.93198585510254, 10.08, 0.02684, 0.03042030564375044],
[25.670886993408203, 0.5457128286361694, 19.72296714782715, 2.5742757320404053, 33.8, 3.236, 0.3340339969942671],
[7.765011787414551, 8.025897026062012, 6.268764495849609, 17.436603546142578, 9.164, 0.00547, 0.013736740104405938],
[6.077215671539307, 2.533625364303589, 6.069260120391846, 17.26119613647461, 9.66, 0.00623, 0.014654789566829106],
[10.796287536621094, 1.542526364326477, 17.64642333984375, 7.402411460876465, 11.586, 0.1783, 0.07843089234528802],
[5.074842929840088, 10.939129829406738, 20.22471809387207, 15.556997299194336, 9.81, 0.01509, 0.02281522923281283],
[10.629094123840332, 4.0070929527282715, 5.236382484436035, 22.565675735473633, 9.82, 0.02058, 0.026634768354252436],
[25.660991668701172, 11.239570617675781, 4.152639865875244, 13.006429672241211, 10.31, 0.0477, 0.0405528515623469],
[5.929766654968262, 7.762668132781982, 7.86545991897583, 10.248272895812988, 9.695, 0.07166, 0.04968801041065646],
[7.375267505645752, 6.4835944175720215, 14.600894927978516, 18.381738662719727, 9.84, 0.004963, 0.013079371353535025],
[0.7596781849861145, 1.876862645149231, 0.03776213899254799, 17.189714431762695, 10.22, 0.010475, 0.01900702405535376],
[12.027226448059082, 13.093915939331055, 2.2943274974823, 15.438506126403809, 9.51, 0.00436, 0.012263327386936654],
[15.388399124145508, 6.072832107543945, 7.429886817932129, 16.254316329956055, 9.805, 0.01494, 0.022690555849026968],
[5.428075313568115, 6.8203301429748535, 20.427663803100586, 23.53197479248047, 9.73, 0.012634, 0.020877124812141695],
[4.837095260620117, 4.036604404449463, 8.280713081359863, 11.022173881530762, 9.6, 0.02988, 0.032097729352869316],
[3.0800538063049316, 0.18660150468349457, 11.994233131408691, 26.21763038635254, 9.695, 0.05392, 0.04311432290194735],
[17.289356231689453, 2.5605714321136475, 19.033023834228516, 23.51763343811035, 9.62, 0.005516, 0.013793409824308602],
[10.545456886291504, 1.661823034286499, 18.970386505126953, 23.998355865478516, 10.414, 0.011086, 0.01955105336641934],
[6.714019298553467, 6.952317714691162, 9.58885669708252, 16.55411148071289, 9.56, 0.02512, 0.02942291857346354],
[24.08850860595703, 6.294763088226318, 16.90302276611328, 27.031070709228516, 11.33, 0.03284, 0.0336391457342218],
[2.675339937210083, 5.500548839569092, 5.736316680908203, 17.072633743286133, 9.55, 0.01245, 0.020718449596414234],
[21.276262283325195, 10.383407592773438, 18.440650939941406, 21.34499740600586, 10.49, 0.00902, 0.01763561683370927],
[16.704021453857422, 3.7706456184387207, 4.818150997161865, 8.079504013061523, 10.28, 0.02061, 0.026657436242213502],
[25.310134887695312, 3.5032365322113037, 8.130770683288574, 18.301259994506836, 10.38, 0.3066, 0.10109332360202773],
[27.94025421142578, 12.677963256835938, 4.207671642303467, 24.683195114135742, 9.61, 0.01367, 0.021715836666701134],
[5.489509582519531, 5.164991855621338, 11.033693313598633, 5.22015905380249, 10.67, 0.10693, 0.06070460395973449],
[10.352096557617188, 4.038761615753174, 14.371241569519043, 21.09249496459961, 9.64, 0.02034, 0.026476093138524975],
[28.752843856811523, 2.5085866451263428, 9.00772762298584, 9.160147666931152, 12.31, 0.477, 0.12213819523669002],
[19.830814361572266, 7.632664203643799, 9.24547004699707, 1.5914164781570435, 33.78, 3.096, 0.31606323657446134],
[7.054588317871094, 7.023423194885254, 6.1760687828063965, 24.259122848510742, 9.84, 0.0317, 0.03307244853519515],
[6.070498466491699, 6.497753143310547, 8.316017150878906, 2.4443976879119873, 15.58, 0.69, 0.1542323096870924],
[28.306018829345703, 8.885969161987305, 10.048260688781738, 15.855472564697266, 9.96, 0.09326, 0.05671505567858689],
[0.8970933556556702, 4.451648712158203, 14.430407524108887, 9.432147026062012, 9.73, 0.05438, 0.04329566600563588],
[1.0525786876678467, 11.82876205444336, 12.936854362487793, 4.99757194519043, 11.0, 0.1765, 0.07802287036198884],
[20.54768180847168, 10.65259075164795, 12.563803672790527, 4.038280487060547, 20.56, 1.251, 0.20414789334095546],
[20.92030906677246, 2.177260398864746, 2.6433215141296387, 4.47040319442749, 14.12, 0.461, 0.12603345706352642],
[18.243406295776367, 0.33440694212913513, 2.850785493850708, 15.334529876708984, 9.22, 0.004192, 0.012025314563345461],
[6.840036869049072, 9.802363395690918, 20.20130729675293, 13.982579231262207, 9.99, 0.0335, 0.03400183194159885],
[1.1564875841140747, 14.147162437438965, 17.79444122314453, 6.246200084686279, 11.24, 0.1471, 0.07122250397366907],
[11.477593421936035, 13.070152282714844, 16.337526321411133, 2.770270824432373, 24.77, 1.658, 0.2391915537651674],
[15.08060359954834, 5.806467533111572, 5.221383094787598, 20.51164436340332, 9.58, 0.00719, 0.015742848188960268],
[19.584259033203125, 13.750139236450195, 0.4979493021965027, 27.94239616394043, 9.914, 0.003738, 0.011356611868494017],
[10.635551452636719, 9.62377643585205, 18.142980575561523, 7.167160987854004, 12.49, 0.4731, 0.12356634304478802],
[27.998580932617188, 8.34493637084961, 13.44279956817627, 22.94693374633789, 11.49, 0.1768, 0.07806820613791096],
[11.209851264953613, 5.489734172821045, 14.350153923034668, 18.23004722595215, 9.69, 0.04608, 0.039872814923514924],
[12.919857025146484, 7.635162830352783, 17.951059341430664, 6.6695756912231445, 13.14, 0.281, 0.09846930530287028],
[25.090978622436523, 8.751540184020996, 12.401233673095703, 27.873626708984375, 10.59, 0.01718, 0.02434531167018478],
[1.1455028057098389, 13.969325065612793, 6.665939807891846, 16.65782356262207, 9.46, 0.03177, 0.03309511642315622],
[17.1646728515625, 7.465495586395264, 0.7485865950584412, 25.907976150512695, 9.67, 0.03093, 0.03266442655189596],
[12.698299407958984, 14.220677375793457, 11.240488052368164, 23.604896545410156, 9.91, 0.003695, 0.01128860820461082],
[8.649404525756836, 7.0686821937561035, 12.011761665344238, 25.00148582458496, 9.25, 0.019, 0.025592045508043403],
[24.953630447387695, 11.571120262145996, 0.1623953878879547, 4.8949689865112305, 16.62, 0.6836, 0.1535069372723383],
[6.696237564086914, 13.289169311523438, 0.6712778806686401, 18.471778869628906, 9.39, 0.002949, 0.01008154317068406],
[27.41204261779785, 5.697915077209473, 9.384078979492188, 7.749012470245361, 13.83, 0.5156, 0.12900361293036422],
[5.668290138244629, 4.60410213470459, 7.479183197021484, 4.550114154815674, 10.44, 0.04587, 0.039759475483709596],
[17.060474395751953, 11.346375465393066, 4.2818922996521, 29.951566696166992, 9.64, 0.02786, 0.030987002842777087],
[22.778026580810547, 6.228560924530029, 20.93468475341797, 14.37533187866211, 10.85, 0.0199, 0.026204078482992183],
[4.815083026885986, 6.652061939239502, 0.4337722957134247, 2.9431238174438477, 10.945, 0.02332, 0.028357527839293444],
[21.962352752685547, 5.449646472930908, 13.460321426391602, 12.05775260925293, 11.16, 0.0214, 0.02715612977735695],
[8.390705108642578, 8.860522270202637, 5.352258205413818, 22.122617721557617, 9.67, 0.01021, 0.018757677287782035],
[3.510619878768921, 6.463396072387695, 10.77855396270752, 6.502497673034668, 9.76, 0.03085, 0.03261909077597383],
[3.4279825687408447, 9.262845993041992, 3.8557612895965576, 7.012554168701172, 9.95, 0.03375, 0.034115171381404186],
[13.459784507751465, 4.568241119384766, 2.578082323074341, 2.7454733848571777, 15.74, 0.3826, 0.11488085618668199],
[17.52276039123535, 10.44411563873291, 8.997193336486816, 27.94095802307129, 10.31, 0.2048, 0.08405252855963237],
[25.332433700561523, 9.870955467224121, 7.493541240692139, 20.001667022705078, 9.98, 0.02225, 0.02770015908842253],
[28.09488868713379, 4.040063381195068, 17.146486282348633, 11.760104179382324, 11.79, 0.676, 0.1316678764686361],
[1.1845860481262207, 5.889479637145996, 5.423652172088623, 24.150238037109375, 9.95, 0.02121, 0.027042790337551623],
[15.121674537658691, 3.3921520709991455, 1.1132926940917969, 4.832014560699463, 11.43, 0.1271, 0.06619023284631244],
[14.13918685913086, 7.211899757385254, 8.3306245803833, 0.9719146490097046, 35.53, 3.72, 0.3464067104520317],
[8.717438697814941, 13.148149490356445, 5.802769660949707, 13.67928695678711, 9.57, 0.04117, 0.03767402979129153],
[19.178842544555664, 9.67149829864502, 16.284467697143555, 28.768577575683594, 10.35, 0.0256, 0.0297176011169574],
[17.237825393676758, 0.7082005739212036, 20.840606689453125, 20.44125747680664, 9.74, 0.0227, 0.027972173743955324],
[29.63863754272461, 10.75675106048584, 17.059940338134766, 3.7476563453674316, 28.89, 2.47, 0.2870551164008195],
[2.2014176845550537, 13.81320571899414, 6.7030863761901855, 2.5041072368621826, 13.87, 0.305, 0.10254952513586214],
[22.65642547607422, 8.644953727722168, 1.050228238105774, 27.088970184326172, 9.84, 0.02129, 0.027088126113473752],
[0.40505507588386536, 3.4619028568267822, 5.77980899810791, 2.203725814819336, 10.65, 0.1709, 0.07675346863616915],
[4.679834365844727, 1.4783096313476562, 12.142334938049316, 10.944846153259277, 9.82, 0.02242, 0.027790830640266797],
[21.198734283447266, 4.969048976898193, 18.439184188842773, 5.730982780456543, 17.52, 0.9272, 0.17294026152187505],
[8.59658432006836, 8.867834091186523, 9.295132637023926, 11.173819541931152, 9.93, 0.0009637, 0.0057633105141010055],
[30.096717834472656, 0.3704378604888916, 20.729948043823242, 4.058557510375977, 26.92, 2.08, 0.2678437641479547],
[26.847322463989258, 13.910696983337402, 18.09627342224121, 9.757781982421875, 13.83, 0.524, 0.13437523983319866],
[16.793787002563477, 8.97684097290039, 18.942907333374023, 27.12104606628418, 10.34, 0.02861, 0.03141769271403734],
[15.602289199829102, 5.934171199798584, 10.479414939880371, 20.2506046295166, 9.56, 0.007324, 0.015890189460707197],
[3.458665609359741, 5.31710958480835, 6.708564281463623, 11.608939170837402, 9.4, 0.05273, 0.042638297254764965],
[7.474071502685547, 6.17165994644165, 19.91966438293457, 23.120946884155273, 10.49, 0.138, 0.06895571517756248],
[11.739043235778809, 0.9398846626281738, 18.529863357543945, 22.922758102416992, 10.79, 0.04636, 0.03998615436332025],
[33.29726791381836, 2.9353411197662354, 4.005119323730469, 28.218618392944336, 10.29, 0.01796, 0.02488934098125036],
[34.3421630859375, 11.853626251220703, 21.769306182861328, 17.831974029541016, 12.29, 0.5728, 0.12277500134408298],
[23.97841453552246, 2.8505139350891113, 4.338203430175781, 7.046377658843994, 12.27, 0.1843, 0.07974562984702985],
[6.015542507171631, 1.8475507497787476, 3.073737621307373, 2.0433475971221924, 11.89, 0.3108, 0.10354691220614905],
[0.30137255787849426, 11.635442733764648, 12.651541709899902, 3.0209085941314697, 13.56, 0.3867, 0.11551555704959184],
[20.353328704833984, 11.979901313781738, 4.0300068855285645, 11.054471969604492, 10.26, 0.001336, 0.006789032444339238],
[19.42940902709961, 12.70743179321289, 5.852179050445557, 8.73660659790039, 11.22, 0.2463, 0.09216763244969396],
[31.248294830322266, 2.8516101837158203, 14.863533020019531, 29.740859985351562, 10.48, 0.2325, 0.08953815744621031],
[13.759539604187012, 1.1074209213256836, 1.1359575986862183, 25.854543685913086, 9.56, 0.02467, 0.029173571805891817],
[32.61806869506836, 10.772001266479492, 18.774311065673828, 12.990249633789062, 12.68, 0.04495, 0.03937412138837147],
[4.4729437828063965, 12.610641479492188, 21.63446044921875, 30.90533447265625, 9.76, 0.02827, 0.031213681722387747],
[12.167366027832031, 2.6741371154785156, 12.9700345993042, 20.84014129638672, 9.67, 0.02176, 0.02738280865696761],
[27.3504638671875, 9.676445960998535, 12.42516040802002, 31.65106201171875, 9.49, 0.01453, 0.022384539361552578],
[12.697709083557129, 0.7860007286071777, 14.663851737976074, 25.524747848510742, 10.05, 0.00797, 0.016581560043519707],
[24.712751388549805, 1.5095256567001343, 19.73548126220703, 3.411588430404663, 27.98, 2.299, 0.27238968168709127],
[20.78339958190918, 9.939838409423828, 16.676345825195312, 29.932313919067383, 11.04, 0.02872, 0.03146302848995947],
[23.555688858032227, 4.832556247711182, 17.985258102416992, 5.563566207885742, 18.86, 0.871, 0.17327333557438776],
[9.955491065979004, 7.5132246017456055, 7.080166816711426, 20.3085880279541, 9.23, 0.00523, 0.013430723616931548],
[13.61115837097168, 4.845157623291016, 11.879392623901367, 21.739179611206055, 9.56, 0.01936, 0.02584139227561513],
[4.617108345031738, 7.82882022857666, 13.871577262878418, 0.06471415609121323, 56.25, 14.75, 0.6148670905875026],
[22.39591407775879, 12.709549903869629, 14.831254959106445, 26.899627685546875, 10.82, 0.01706, 0.024254640118340515],
[10.38247299194336, 4.127026081085205, 17.852588653564453, 15.439671516418457, 9.94, 0.02585, 0.029853608444723793],
[2.370311737060547, 8.026403427124023, 17.046920776367188, 5.092191219329834, 11.82, 0.2382, 0.09062621606834148],
[8.29548168182373, 9.752336502075195, 11.608744621276855, 9.702553749084473, 10.19, 0.03485, 0.03465920069246976],
[11.60104751586914, 7.450238227844238, 14.710851669311523, 17.748210906982422, 9.555, 0.001665, 0.007576741550986278],
[27.545156478881836, 6.320235252380371, 9.110542297363281, 4.402825355529785, 20.97, 1.099, 0.19458115025778971],
[4.859279155731201, 6.6626458168029785, 5.3149847984313965, 3.980121612548828, 10.61, 0.08606, 0.05449360265840243],
[25.345720291137695, 4.580177307128906, 16.11130142211914, 4.2268266677856445, 22.44, 1.547, 0.2234542527169055],
[18.647563934326172, 6.876091957092285, 21.999189376831055, 19.59282112121582, 10.71, 0.02135, 0.027133461889395884],
[9.191402435302734, 8.63733196258545, 22.572433471679688, 21.941749572753906, 10.06, 0.05942, 0.04526777225824861],
[25.63987922668457, 6.903411388397217, 10.788987159729004, 10.389264106750488, 12.13, 0.0936, 0.056805727230431154],
[32.776485443115234, 6.797540664672852, 6.6455793380737305, 30.00859832763672, 11.266, 0.01834, 0.025138687748822088],
[14.404139518737793, 14.653424263000488, 14.435742378234863, 24.12673568725586, 9.63, 0.02344, 0.028425531503176643],
[15.125469207763672, 9.87134838104248, 0.07851316779851913, 3.309009075164795, 14.84, 0.4324, 0.12213458033422309],
[9.837047576904297, 8.812833786010742, 19.973119735717773, 2.736973285675049, 24.75, 1.571, 0.23284454513606895],
[6.984395980834961, 4.489354133605957, 15.249032974243164, 16.405052185058594, 9.53, 0.006287, 0.014722793230712303],
[15.310869216918945, 0.18378014862537384, 15.318511009216309, 25.420568466186523, 10.62, 0.03006, 0.03218840090471358],
[25.671581268310547, 10.940381050109863, 3.034933090209961, 14.654753684997559, 10.12, 0.002285, 0.008874478136757302],
[18.485332489013672, 12.853079795837402, 8.519634246826172, 11.724630355834961, 10.164, 0.0449, 0.03935145350041041],
[13.279614448547363, 6.621898651123047, 2.9871625900268555, 10.714553833007812, 9.81, 0.026, 0.029944279996568058],
[13.887011528015137, 13.340230941772461, 7.973103046417236, 31.268083572387695, 10.04, 0.02498, 0.029354914909580344],
[29.54301643371582, 3.299743890762329, 1.5214660167694092, 18.47065544128418, 10.36, 0.002201, 0.008710135949039573],
[34.34190368652344, 3.6052603721618652, 0.5367701649665833, 23.024728775024414, 10.125, 0.02707, 0.030556312971516838],
[0.3320748507976532, 3.24440598487854, 19.629119873046875, 7.196138858795166, 10.29, 0.0325, 0.033480470518494336],
[30.033687591552734, 5.7157368659973145, 10.513272285461426, 24.442764282226562, 10.83, 0.00963, 0.018224981920696985],
[12.637772560119629, 7.075038909912109, 22.202241897583008, 13.434246063232422, 11.086, 0.256, 0.0939357277106571],
[32.75688934326172, 5.565921783447266, 10.355769157409668, 4.601371765136719, 22.27, 1.123, 0.19675726750205202],
[1.2709625959396362, 3.780609607696533, 7.692920207977295, 11.069774627685547, 9.34, 0.001609, 0.007446401195210149],
[18.4191951751709, 7.639839172363281, 17.439329147338867, 32.30640411376953, 10.51, 0.02267, 0.027972173743955324],
[20.772199630737305, 11.965041160583496, 1.7248977422714233, 33.95304489135742, 10.555, 0.06122, 0.04594780889708058],
[6.672083854675293, 5.694490432739258, 5.986464023590088, 8.141797065734863, 9.94, 0.003246, 0.01058023670582751],
[25.306293487548828, 12.22387981414795, 13.091459274291992, 13.817896842956543, 10.55, 0.1829, 0.07942827941557493],
[5.767237186431885, 11.405099868774414, 11.706892967224121, 1.454675555229187, 24.69, 1.39, 0.21888112615205235],
[27.083776473999023, 8.609731674194336, 19.794673919677734, 3.1191134452819824, 32.0, 2.996, 0.315938923144256],
[21.710309982299805, 3.186647415161133, 6.169195652008057, 33.497901916503906, 9.695, 0.04965, 0.04136889552894527],
[14.137910842895508, 0.9565305113792419, 22.465553283691406, 30.38596534729004, 10.18, 0.01852, 0.025274695076588482],
[11.885661125183105, 15.040669441223145, 0.4237748682498932, 15.875863075256348, 10.08, 0.03665, 0.03554324832295133],
[29.169261932373047, 0.03993339091539383, 13.278441429138184, 8.945048332214355, 13.195, 0.5024, 0.13165509327787075],
[32.75108337402344, 14.448637008666992, 1.732838749885559, 15.309479713439941, 10.984, 0.3555, 0.10707913758529891],
[13.057276725769043, 11.90056037902832, 17.48455810546875, 12.146758079528809, 10.32, 0.001107, 0.006176999469390459],
[20.411874771118164, 9.820064544677734, 13.509835243225098, 24.940481185913086, 9.734, 0.01811, 0.024980012533094623],
[3.600942850112915, 6.494490146636963, 15.379231452941895, 7.464773178100586, 10.21, 0.05542, 0.04372635587689613],
[5.743381023406982, 13.493535995483398, 10.424019813537598, 0.06440399587154388, 55.62, 16.69, 0.6542736284379732],
[21.991159439086914, 12.240256309509277, 15.959982872009277, 27.653467178344727, 11.44, 0.0993, 0.05852848671547216],
[6.100683212280273, 7.202699661254883, 22.229434967041016, 26.541671752929688, 9.44, 0.1307, 0.0671422841406772],
[19.526729583740234, 7.99241828918457, 20.746110916137695, 25.625261306762695, 11.12, 0.0139, 0.02189717977038966],
[13.956147193908691, 8.821842193603516, 23.96216583251953, 32.64017105102539, 11.09, 0.005028, 0.013170042905379289],
[14.230182647705078, 13.252193450927734, 9.960463523864746, 2.556769371032715, 23.34, 1.212, 0.20437367785697017],
[32.24570083618164, 14.729891777038574, 0.8380342125892639, 14.994135856628418, 11.54, 0.05893, 0.04508642915456008],
[31.381372451782227, 9.54916000366211, 18.22266387939453, 32.70785140991211, 11.42, 0.821, 0.14503795181076007],
[15.870590209960938, 12.103693962097168, 7.992776393890381, 23.87118911743164, 9.73, 0.011925, 0.02027642578117345],
[6.402368068695068, 7.027665138244629, 17.485319137573242, 15.089552879333496, 10.125, 0.01802, 0.024934676757172494],
[14.614047050476074, 7.392373561859131, 10.89968204498291, 21.287904739379883, 9.664, 0.02477, 0.02921890758181395],
[30.001943588256836, 15.065593719482422, 13.36631965637207, 11.612709045410156, 12.17, 0.476, 0.12010338274677786],
[34.58189010620117, 8.606572151184082, 11.089231491088867, 15.509320259094238, 11.43, 0.001122, 0.00622233524531259],
[1.4456669092178345, 12.556680679321289, 0.8417296409606934, 1.9433246850967407, 17.83, 0.03842, 0.03640462806547184],
[4.442512035369873, 11.509516716003418, 24.191621780395508, 11.435004234313965, 10.18, 0.02774, 0.030918999178893892],
[19.805505752563477, 7.93574333190918, 16.44108009338379, 29.128562927246094, 10.45, 0.0233, 0.028334859951332378],
[2.557556629180908, 8.558647155761719, 10.496679306030273, 3.006185293197632, 12.98, 0.443, 0.1235853251637313],
[0.6335641741752625, 3.51729154586792, 10.36254596710205, 12.261551856994629, 9.68, 0.03732, 0.03588326664236732],
[9.442922592163086, 6.236413955688477, 9.777307510375977, 25.933292388916016, 9.9, 0.04108, 0.0376286940153694],
[22.810640335083008, 10.229330062866211, 17.50928497314453, 4.8365583419799805, 20.92, 1.024, 0.1878714554213142],
[9.004109382629395, 7.488907337188721, 24.167634963989258, 14.381523132324219, 10.86, 0.01677, 0.024050629126690922],
[25.074060440063477, 14.35179328918457, 11.057295799255371, 27.35733413696289, 10.54, 0.003462, 0.010925921997233764],
[10.278902053833008, 12.88414192199707, 12.536877632141113, 34.021942138671875, 10.4, 0.02367, 0.028561538830943037],
[28.93208122253418, 3.3621366024017334, 12.181500434875488, 30.824691772460938, 10.87, 0.305, 0.10254952513586214],
[10.116182327270508, 2.276341199874878, 22.087553024291992, 2.3297746181488037, 27.1, 1.814, 0.25007213998647904],
[15.685651779174805, 12.424015998840332, 11.348796844482422, 21.894746780395508, 10.41, 0.012955, 0.021137805523693955],
[24.18361473083496, 4.642943382263184, 9.349998474121094, 6.611024379730225, 14.25, 0.317, 0.10454429927643594],
[5.433197021484375, 5.053918838500977, 11.595952987670898, 34.628108978271484, 10.46, 0.09467, 0.057123077661886075],
[31.92284393310547, 4.394848823547363, 19.4155330657959, 6.7714080810546875, 19.17, 0.526, 0.13464725448873147],
[31.995012283325195, 14.666862487792969, 22.65811538696289, 19.382471084594727, 12.66, 0.4187, 0.1093583386245395],
[32.9555778503418, 1.8517897129058838, 14.51671028137207, 23.7664794921875, 11.12, 0.5044, 0.11376292177070399],
[31.281652450561523, 2.57781982421875, 15.983536720275879, 16.810165405273438, 11.37, 0.001894, 0.008081102058119995],
[27.32438850402832, 11.240340232849121, 2.522021770477295, 32.74884033203125, 9.49, 0.003124, 0.010381892686168183],
[1.4003421068191528, 10.036096572875977, 2.838379144668579, 5.823761463165283, 10.77, 0.0008807, 0.005511130260534148],
[32.79072570800781, 12.849947929382324, 6.745648384094238, 27.411333084106445, 11.14, 0.1642, 0.0752573880307388],
[16.62950897216797, 7.3876237869262695, 13.936875343322754, 6.703859329223633, 13.14, 0.2673, 0.09602117340307516],
[18.481382369995117, 5.06436824798584, 18.032169342041016, 11.56579303741455, 11.31, 0.05466, 0.043409005445441205],
[11.642894744873047, 14.637593269348145, 10.7555513381958, 23.738819122314453, 9.695, 0.007683, 0.016275543556045317],
[27.13592529296875, 5.38073205947876, 16.788497924804688, 14.0263090133667, 10.69, 0.002026, 0.00835878368564305],
[36.0480842590332, 10.94201946258545, 4.636951446533203, 18.791715621948242, 10.84, 0.001909, 0.008115103890061592],
[26.92148780822754, 4.500820636749268, 6.761307239532471, 29.032732009887695, 9.81, 0.02165, 0.02731480499308441],
[0.11014856398105621, 3.765242576599121, 4.848781108856201, 28.66058349609375, 9.62, 0.0482, 0.04075686255399649],
[22.978933334350586, 9.184601783752441, 10.92775821685791, 24.493633270263672, 9.82, 0.02858, 0.03139502482607628],
[0.2878802418708801, 7.878915309906006, 2.7710094451904297, 9.708654403686523, 10.47, 0.1049, 0.060160574648668905],
[11.762201309204102, 10.375874519348145, 16.86212158203125, 32.44910430908203, 9.25, 0.005596, 0.013895415320133398],
[26.120790481567383, 6.799890041351318, 0.8452260494232178, 31.23368263244629, 9.32, 0.00411, 0.011900641179559598],
[9.80132007598877, 11.68682861328125, 12.218311309814453, 18.511491775512695, 9.48, 0.03238, 0.03341246685461114],
[0.6497275829315186, 9.032448768615723, 9.69645881652832, 11.44369125366211, 10.05, 0.0655, 0.047511893166394134],
[17.354248046875, 8.674577713012695, 5.319127559661865, 16.56060218811035, 10.13, 0.0415, 0.03783270500701899],
[1.2859787940979004, 6.207844257354736, 5.076277732849121, 1.9011279344558716, 11.93, 0.11346, 0.06256337077254188],
[26.288272857666016, 4.966822624206543, 11.280312538146973, 23.620820999145508, 10.1, 0.02391, 0.028720214046670498],
[21.91585350036621, 13.79041576385498, 2.9939441680908203, 1.0858265161514282, 39.6, 4.285, 0.3844473798196777],
[29.798524856567383, 6.079392910003662, 14.023999214172363, 30.66559410095215, 10.26, 0.2634, 0.09529580098832105],
[33.356082916259766, 2.7421767711639404, 16.735618591308594, 12.898870468139648, 11.91, 0.1454, 0.07081448199036988],
[21.250431060791016, 6.728883743286133, 16.619365692138672, 31.386693954467773, 10.8, 0.00775, 0.016343547219928516],
[13.344538688659668, 13.640532493591309, 12.790766716003418, 36.064754486083984, 10.1, 0.02463, 0.02915090391793075],
[22.754159927368164, 9.916655540466309, 19.08536148071289, 12.156993865966797, 11.67, 0.7056, 0.13448262917224116],
[11.368486404418945, 9.905193328857422, 6.368355751037598, 17.119382858276367, 9.625, 0.08746, 0.05490162464170162],
[33.65716552734375, 1.359689474105835, 11.727993965148926, 32.157958984375, 10.54, 0.0699, 0.04909864532366875],
[20.021581649780273, 4.923123836517334, 4.511862754821777, 7.615957260131836, 10.914, 0.00825, 0.01686490864303303],
[29.049022674560547, 0.977588951587677, 20.953458786010742, 17.979433059692383, 11.266, 0.5464, 0.11837598870161227],
[6.99312162399292, 11.266953468322754, 8.66460132598877, 23.067537307739258, 9.57, 0.0392, 0.03676731427284889],
[28.551599502563477, 9.599556922912598, 21.094043731689453, 33.263126373291016, 11.37, 0.6353, 0.12760212256342882],
[22.618257522583008, 15.181815147399902, 24.818063735961914, 26.9487247467041, 11.8, 1.44, 0.19218505959614463],
[14.616678237915039, 13.458113670349121, 12.515966415405273, 10.408218383789062, 11.1, 0.2615, 0.094933114780944],
[24.68894386291504, 1.1500134468078613, 18.249181747436523, 2.8486077785491943, 30.73, 2.064, 0.26675570552582356],
[20.039533615112305, 2.6892778873443604, 6.012594699859619, 19.168193817138672, 9.88, 0.02463, 0.02915090391793075],
[14.611262321472168, 11.053725242614746, 23.149141311645508, 13.301898956298828, 10.9, 0.1779, 0.07829488501752163],
[29.764314651489258, 7.422133445739746, 19.724529266357422, 10.41288948059082, 14.03, 0.3052, 0.10254952513586214],
[26.825172424316406, 6.3547539710998535, 13.33987045288086, 17.224018096923828, 10.26, 0.01785, 0.024821337317367163],
[29.259044647216797, 1.5994147062301636, 18.473388671875, 23.686220169067383, 11.35, 0.8193, 0.14495976423565993],
[32.19058609008789, 9.83259105682373, 14.374537467956543, 28.391576766967773, 11.22, 0.9746, 0.15809527685248348],
[12.427630424499512, 13.541558265686035, 5.805683135986328, 9.418872833251953, 10.37, 0.05966, 0.04535844381009287],
[30.678510665893555, 13.964926719665527, 24.540912628173828, 14.970312118530273, 12.56, 0.7417, 0.1379228824766473],
[6.3485307693481445, 5.267223834991455, 5.969882011413574, 31.713029861450195, 9.65, 0.02031, 0.02645342525056391],
[18.502685546875, 6.196670055389404, 3.538734197616577, 9.208564758300781, 10.42, 0.001533, 0.007270725063511888],
[3.0883917808532715, 13.973569869995117, 14.143239974975586, 4.707990646362305, 12.63, 0.312, 0.10372825530983756],
[36.46449279785156, 5.894942283630371, 15.11865520477295, 26.61975860595703, 12.016, 0.5947, 0.12345818108312137],
[21.1854248046875, 6.845003128051758, 9.694130897521973, 12.098082542419434, 10.086, 0.0006933, 0.004890596827599968],
[4.695919036865234, 9.709100723266602, 15.976552963256836, 35.965518951416016, 10.03, 0.0396, 0.03694865737653742],
[2.1613078117370605, 12.757160186767578, 21.417512893676758, 15.213412284851074, 9.375, 0.00636, 0.014813464782556567],
[7.699887752532959, 4.190908908843994, 14.314733505249023, 34.37489318847656, 9.77, 0.0559, 0.04390769898058465],
[28.04141616821289, 14.040595054626465, 23.44459342956543, 16.47285270690918, 11.34, 0.3591, 0.11125399411291145],
[2.79140305519104, 2.9435243606567383, 7.936112403869629, 16.556093215942383, 9.14, 0.00102, 0.005933319673809],
[36.01722717285156, 3.8974220752716064, 13.393989562988281, 6.126778602600098, 19.83, 1.275, 0.19651735379373703],
[0.9406472444534302, 6.949175834655762, 2.1162166595458984, 1.118415355682373, 13.836, 0.0724, 0.04996002506618925],
[4.929233074188232, 4.372629642486572, 12.600790023803711, 10.325292587280273, 9.59, 0.11816, 0.06383277249836158],
[1.9141923189163208, 1.9562618732452393, 3.673517942428589, 11.718426704406738, 9.9, 0.00276, 0.009752858795248604],
[7.916863441467285, 3.4134316444396973, 2.576261281967163, 29.707212448120117, 10.2, 0.0746, 0.05073073325686549],
[11.146451950073242, 15.41972541809082, 12.240756034851074, 0.5134106278419495, 51.38, 7.73, 0.4995272462619677],
[25.252592086791992, 2.913386821746826, 8.987464904785156, 4.76997184753418, 17.98, 0.7915, 0.1652035674602483],
[10.909573554992676, 1.528741717338562, 20.570526123046875, 6.14040470123291, 13.14, 0.2644, 0.09547714409200958],
[15.401835441589355, 4.648359298706055, 10.184371948242188, 14.279998779296875, 10.29, 0.004215, 0.01205931639528706],
[36.43913650512695, 2.253706216812134, 15.137203216552734, 19.03251838684082, 11.18, 0.1267, 0.06609956129446817],
[10.053557395935059, 15.456311225891113, 5.907608509063721, 8.7243013381958, 10.55, 0.04816, 0.04075686255399649],
[35.81194305419922, 5.832279682159424, 10.332169532775879, 10.26175594329834, 13.73, 0.2798, 0.0981972906473375],
[30.71352195739746, 9.774332046508789, 3.048797845840454, 30.358766555786133, 10.336, 0.01549, 0.023109911776306686],
[27.894746780395508, 1.9203487634658813, 18.853050231933594, 9.081074714660645, 14.17, 0.3884, 0.11569690015328037],
[12.671351432800293, 4.602235317230225, 8.863850593566895, 18.9078311920166, 9.88, 0.04175, 0.03794604444682432],
[8.961718559265137, 0.1149262934923172, 10.692625999450684, 13.38698959350586, 9.805, 0.1486, 0.07158519018104612],
[3.928591251373291, 15.492582321166992, 19.02832794189453, 8.39162540435791, 11.11, 0.2311, 0.08926614279067753],
[31.35831069946289, 3.1742501258850098, 11.293211936950684, 25.8150634765625, 11.16, 0.01665, 0.02395995757484666],
[36.58989715576172, 0.25445717573165894, 6.397183418273926, 22.791236877441406, 10.83, 0.008705, 0.01732960034623488],
[14.906850814819336, 5.149161338806152, 12.381394386291504, 10.615562438964844, 10.54, 0.01117, 0.019630390974283072],
[19.324512481689453, 14.508803367614746, 11.279651641845703, 33.637451171875, 10.03, 0.04074, 0.037492686687603],
[22.138164520263672, 3.7951059341430664, 8.76376724243164, 12.006875038146973, 10.06, 0.0975, 0.05798445740440658],
[18.826810836791992, 10.807039260864258, 13.434221267700195, 8.132780075073242, 12.7, 0.2098, 0.08504991562991926],
[17.798336029052734, 4.355737686157227, 17.14862823486328, 17.783998489379883, 9.95, 0.02068, 0.02670277201813563],
[2.570646286010742, 9.026263236999512, 25.34752655029297, 34.502681732177734, 10.07, 0.03937, 0.036835317936732094],
[14.907303810119629, 8.912403106689453, 0.49566978216171265, 28.260175704956055, 9.85, 0.02153, 0.027246801329201216],
[6.844114303588867, 13.661410331726074, 24.188846588134766, 17.69150733947754, 9.85, 0.02957, 0.031939054137141855],
[19.450151443481445, 2.969493865966797, 0.854810893535614, 12.815380096435547, 9.89, 0.002491, 0.009265499204085688],
[30.31672477722168, 9.64556884765625, 17.296640396118164, 16.425458908081055, 11.84, 0.02295, 0.028130848959682785],
[20.629487991333008, 12.74227237701416, 7.346215724945068, 17.72953987121582, 10.32, 0.005432, 0.013691404328483805],
[36.300880432128906, 8.978607177734375, 4.406140327453613, 4.994290828704834, 21.5, 1.368, 0.2135975338187464],
[31.266023635864258, 10.753429412841797, 13.21583366394043, 11.442865371704102, 13.0, 0.4065, 0.11453345920262112],
[4.344005107879639, 8.771949768066406, 1.532438039779663, 9.23165512084961, 9.75, 0.001317, 0.006738029696426839],
[33.41280746459961, 4.969394683837891, 13.64025592803955, 7.846285820007324, 16.03, 0.605, 0.14443978208791194],
[32.82566833496094, 3.092005491256714, 23.77216911315918, 9.64282512664795, 16.27, 0.6304, 0.14743194329877263],
[29.838815689086914, 8.477625846862793, 5.696166038513184, 17.3865966796875, 10.32, 0.0159, 0.023415928263781076],
[26.2531681060791, 6.863449573516846, 19.407682418823242, 1.3986716270446777, 54.34, 8.12, 0.5291591765631224],
[22.059329986572266, 13.636262893676758, 23.780963897705078, 0.6406328082084656, 80.75, 20.72, 0.8044720703928983],
[20.423545837402344, 13.884984016418457, 5.161299705505371, 8.700858116149902, 11.5, 0.3142, 0.10409094151721462],
[26.12102699279785, 0.09420622140169144, 14.94897174835205, 4.726233005523682, 20.52, 1.335, 0.20422542044523576]]
    )

    bounds = np.array([
        [4.26692180e-02, 3.99333909e-02, 3.77621390e-02, 6.44039959e-02],
        [36.58989716   , 15.49258232   , 25.34752655   , 36.06475449]
                    ])

    opt = optimiser(train_X_Y_Yvar, bounds, 1)
    optimiser_PM   = opt.evaluate(opt.evaluatePM(raw_samples=4096, num_restarts=10))
    print(f"PM: {optimiser_PM['lower']:0.3f} < {optimiser_PM['mean']:0.3f} < {optimiser_PM['upper']:0.3f}")

    tested_candidate = opt.evaluate([1.0, 1.0, 1.0, 1.0])
    print(f"\nTESTED  : {tested_candidate['lower']:0.3f} < {tested_candidate['mean']:0.3f} < {tested_candidate['upper']:0.3f}")

    manual_best      = opt.evaluate([1.0, 2.0, 2.0, 2.0])
    print(f"MANUAL  : {manual_best['lower']:0.3f} < {manual_best['mean']:0.3f} < {manual_best['upper']:0.3f}")

    print(f"DATASET : 8.900 <  9.010 < 9.120")  # Approximate, based on the given dataset itself

CODE OUTPUT:

PM: 12.550 < 13.728 < 14.907


TESTED  : 11.531 < 11.613 < 11.695
MANUAL  : 8.879 < 9.528 < 10.177
DATASET : 8.900 <  9.010 < 9.120

Please let me know what I am doing wrong with the optimisation process. I may have been incorrectly using the BoTorch package for a while now and didn't realise it because I was always getting a suggestion and did not really pay attention to the details.

[Balandat]

Hmm my suspicion is that your model fits are wrong. You're doing the following:

        self.train_Y, self.train_Yvar = self.Y_standardiser(torch.tensor(train_dat[:, -3:-2], **torch_kwargs),
                                                            torch.tensor(train_dat[:, -2:-1], **torch_kwargs))

so you're standardizing the observations, but you are not scaling the variance observations accordingly (you are leaving those on the original scale). This could result in the model being completely wrong.

I would fix that and then take a look at the model fits to understand what's going on. See e.g. this tutorial for how to do cross validation: https://github.com/pytorch/botorch/blob/main/tutorials/batch_mode_cross_validation.ipynb

[sambitmishra98]

I do scale the variance observations.

train_dat[:, -3:-2] = array([[11.62 ],
       [ 9.805],
       [ 9.52 ],
       [ 9.836],
       [14.18 ],
       [ 9.58 ],
       [ 9.336],
       [ 9.72 ],
       [ 9.54 ],
       [44.94 ]])
self.train_Y = tensor([[-0.1981],
        [-0.3625],
        [-0.3883],
        [-0.3597],
        [ 0.0337],
        [-0.3829],
        [-0.4050],
        [-0.3702],
        [-0.3865],
        [ 2.8197]], dtype=torch.float64)
train_dat[:, -2:-1] = array([[1.689e-03],
       [6.720e-02],
       [6.300e-04],
       [1.596e-02],
       [3.765e-01],
       [1.121e-03],
       [6.060e-04],
       [7.130e-04],
       [1.907e-03],
       [6.434e+00]])
self.train_Yvar = tensor([[1.3855e-05],
        [5.5126e-04],
        [5.1681e-06],
        [1.3092e-04],
        [3.0885e-03],
        [9.1959e-06],
        [4.9712e-06],
        [5.8490e-06],
        [1.5644e-05],
        [5.2780e-02]], dtype=torch.float64)

I will try to look at CV plots.

[sambitmishra98]

I got the mistake I was making. I was standardising the train_X dataset but wasn't standardising the bounds. That's why I was getting a suggestion that looked reasonable.