Skip to content

Commit

Permalink
merge feature/mds branch into main (#739)
Browse files Browse the repository at this point in the history
* Support MDS on List, Detail, Dashboard, Overview pages (#722)

* update snapshots

Signed-off-by: Jackie Han <[email protected]>

* add snpshot

Signed-off-by: Jackie Han <[email protected]>

* add data source client

Signed-off-by: Jackie Han <[email protected]>

* test

* neo List Detectors page

Signed-off-by: Jackie Han <[email protected]>

* add opensearch_dashboards.json file

Signed-off-by: Jackie Han <[email protected]>

* neo List Detectors page

Signed-off-by: Jackie Han <[email protected]>

* test

Signed-off-by: Jackie Han <[email protected]>

* Support MDS on DetectorDetails page

Signed-off-by: Jackie Han <[email protected]>

* change 4/7

Signed-off-by: Jackie Han <[email protected]>

* version change

Signed-off-by: Jackie Han <[email protected]>

* list detector list change

Signed-off-by: Jackie Han <[email protected]>

* Support MDS on List, Detail, Dashboard, Overview pages

Signed-off-by: Jackie Han <[email protected]>

* change version back to 3.0

Signed-off-by: Jackie Han <[email protected]>

* revert version change

Signed-off-by: Jackie Han <[email protected]>

* make dataSourceId optional in DetectorListItem type

Signed-off-by: Jackie Han <[email protected]>

* update imports

Signed-off-by: Jackie Han <[email protected]>

* remove used function

Signed-off-by: Jackie Han <[email protected]>

* cleanup

Signed-off-by: Jackie Han <[email protected]>

* add getter and setter for dataSource plugin

Signed-off-by: Jackie Han <[email protected]>

* read dataSourceId from the url instead of passing props

Signed-off-by: Jackie Han <[email protected]>

* addressing comments and run prettier

Signed-off-by: Jackie Han <[email protected]>

* addressing comments

Signed-off-by: Jackie Han <[email protected]>

* make getDataSourceManagementPlugin() optional

Signed-off-by: Jackie Han <[email protected]>

* add comment

Signed-off-by: Jackie Han <[email protected]>

* make dataSourceId type safe

Signed-off-by: Jackie Han <[email protected]>

---------

Signed-off-by: Jackie Han <[email protected]>

* add MDS support for detector creation&Update flow (#728)

* update snapshots

Signed-off-by: Jackie Han <[email protected]>

* add snpshot

Signed-off-by: Jackie Han <[email protected]>

* add data source client

Signed-off-by: Jackie Han <[email protected]>

* test

* neo List Detectors page

Signed-off-by: Jackie Han <[email protected]>

* add opensearch_dashboards.json file

Signed-off-by: Jackie Han <[email protected]>

* neo List Detectors page

Signed-off-by: Jackie Han <[email protected]>

* test

Signed-off-by: Jackie Han <[email protected]>

* Support MDS on DetectorDetails page

Signed-off-by: Jackie Han <[email protected]>

* change 4/7

Signed-off-by: Jackie Han <[email protected]>

* version change

Signed-off-by: Jackie Han <[email protected]>

* list detector list change

Signed-off-by: Jackie Han <[email protected]>

* Support MDS on List, Detail, Dashboard, Overview pages

Signed-off-by: Jackie Han <[email protected]>

* change version back to 3.0

Signed-off-by: Jackie Han <[email protected]>

* revert version change

Signed-off-by: Jackie Han <[email protected]>

* make dataSourceId optional in DetectorListItem type

Signed-off-by: Jackie Han <[email protected]>

* update imports

Signed-off-by: Jackie Han <[email protected]>

* remove used function

Signed-off-by: Jackie Han <[email protected]>

* cleanup

Signed-off-by: Jackie Han <[email protected]>

* add getter and setter for dataSource plugin

Signed-off-by: Jackie Han <[email protected]>

* read dataSourceId from the url instead of passing props

Signed-off-by: Jackie Han <[email protected]>

* addressing comments

Signed-off-by: Jackie Han <[email protected]>

* addressing comments and run prettier

Signed-off-by: Jackie Han <[email protected]>

* addressing comments in neo1 branch

Signed-off-by: Jackie Han <[email protected]>

* addressing comments

Signed-off-by: Jackie Han <[email protected]>

* make getDataSourceManagementPlugin() optional

Signed-off-by: Jackie Han <[email protected]>

* add comment

Signed-off-by: Jackie Han <[email protected]>

* make dataSourceId type safe

Signed-off-by: Jackie Han <[email protected]>

* add MDS support on creation page

Signed-off-by: Jackie Han <[email protected]>

* updated methods to get data source query param from url

Signed-off-by: Jackie Han <[email protected]>

* using helper to construct url with dataSourceId

Signed-off-by: Jackie Han <[email protected]>

* add update page, and update urls on detector detail page

Signed-off-by: Jackie Han <[email protected]>

* update update call router

Signed-off-by: Jackie Han <[email protected]>

* update reducer and router

Signed-off-by: Jackie Han <[email protected]>

* clean up

Signed-off-by: Jackie Han <[email protected]>

* addressing comments

Signed-off-by: Jackie Han <[email protected]>

* update enums

Signed-off-by: Jackie Han <[email protected]>

---------

Signed-off-by: Jackie Han <[email protected]>

* update actionOption attribution for all data soure components (#732)

Signed-off-by: Jackie Han <[email protected]>

* Add MDS UTs and bug fixes (#734)

* working on ut

Signed-off-by: Jackie Han <[email protected]>

* update ut

Signed-off-by: Jackie Han <[email protected]>

* update UTs and add bug fixes

Signed-off-by: Jackie Han <[email protected]>

---------

Signed-off-by: Jackie Han <[email protected]>

* bug fixes (#737)

Signed-off-by: Jackie Han <[email protected]>

---------

Signed-off-by: Jackie Han <[email protected]>
  • Loading branch information
jackiehanyang authored Apr 29, 2024
1 parent 41a1703 commit 48acb93
Show file tree
Hide file tree
Showing 79 changed files with 2,571 additions and 852 deletions.
1 change: 1 addition & 0 deletions opensearch_dashboards.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"configPath": [
"anomaly_detection_dashboards"
],
"optionalPlugins": ["dataSource","dataSourceManagement"],
"requiredPlugins": [
"opensearchDashboardsUtils",
"expressions",
Expand Down
6 changes: 3 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,13 @@
"devDependencies": {
"@testing-library/user-event": "^12.1.6",
"@types/react-plotly.js": "^2.6.0",
"@types/redux-mock-store": "^1.0.1",
"@types/redux-mock-store": "^1.0.6",
"babel-polyfill": "^6.26.0",
"eslint-plugin-no-unsanitized": "^3.0.2",
"eslint-plugin-prefer-object-spread": "^1.2.1",
"lint-staged": "^9.2.0",
"moment": "^2.24.0",
"redux-mock-store": "^1.5.3",
"redux-mock-store": "^1.5.4",
"start-server-and-test": "^1.11.7"
},
"dependencies": {
Expand All @@ -56,4 +56,4 @@
"browserify-sign": "^4.2.2",
"axios": "^1.6.1"
}
}
}
6 changes: 5 additions & 1 deletion public/anomaly_detection_app.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,17 @@ export function renderApp(coreStart: CoreStart, params: AppMountParameters) {
} else {
require('@elastic/charts/dist/theme_only_light.css');
}

ReactDOM.render(
<Provider store={store}>
<Router>
<Route
render={(props) => (
<CoreServicesContext.Provider value={coreStart}>
<Main {...props} />
<Main
setHeaderActionMenu={params.setHeaderActionMenu}
{...props}
/>
</CoreServicesContext.Provider>
)}
/>
Expand Down
14 changes: 12 additions & 2 deletions public/components/CreateDetectorButtons/CreateDetectorButtons.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,24 @@
import { EuiFlexGroup, EuiFlexItem, EuiButton } from '@elastic/eui';
import React from 'react';
import { APP_PATH, PLUGIN_NAME } from '../../utils/constants';
import { useLocation } from 'react-router-dom';
import { constructHrefWithDataSourceId, getDataSourceFromURL } from '../../pages/utils/helpers';

export const CreateDetectorButtons = () => {
const location = useLocation();
const MDSQueryParams = getDataSourceFromURL(location);
const dataSourceId = MDSQueryParams.dataSourceId;

const createDetectorUrl = `${PLUGIN_NAME}#` + constructHrefWithDataSourceId(`${APP_PATH.CREATE_DETECTOR}`, dataSourceId, false);

const sampleDetectorUrl = `${PLUGIN_NAME}#` + constructHrefWithDataSourceId(`${APP_PATH.OVERVIEW}`, dataSourceId, false);

return (
<EuiFlexGroup direction="row" gutterSize="m" justifyContent="center">
<EuiFlexItem grow={false}>
<EuiButton
style={{ width: '200px' }}
href={`${PLUGIN_NAME}#${APP_PATH.OVERVIEW}`}
href={sampleDetectorUrl}
data-test-subj="sampleDetectorButton"
>
Try a sample detector
Expand All @@ -29,7 +39,7 @@ export const CreateDetectorButtons = () => {
<EuiButton
style={{ width: '200px' }}
fill
href={`${PLUGIN_NAME}#${APP_PATH.CREATE_DETECTOR}`}
href={createDetectorUrl}
data-test-subj="createDetectorButton"
>
Create detector
Expand Down
6 changes: 6 additions & 0 deletions public/models/interfaces.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import { DATA_TYPES } from '../utils/constants';
import { DETECTOR_STATE } from '../../server/utils/constants';
import { Duration } from 'moment';
import moment from 'moment';
import { MDSQueryParams } from '../../server/models/types';

export type FieldInfo = {
label: string;
Expand Down Expand Up @@ -326,3 +327,8 @@ export interface ValidationSettingResponse {
message: string;
validationType: string;
}

export interface MDSStates {
queryParams: MDSQueryParams;
selectedDataSourceId: string | undefined;
}
24 changes: 19 additions & 5 deletions public/pages/AnomalyCharts/containers/AnomalyDetailsChart.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -81,14 +81,19 @@ import {
} from '../utils/constants';
import { HeatmapCell } from './AnomalyHeatmapChart';
import { ANOMALY_AGG, MIN_END_TIME, MAX_END_TIME } from '../../utils/constants';
import { MAX_HISTORICAL_AGG_RESULTS } from '../../../utils/constants';
import {
DATA_SOURCE_ID,
MAX_HISTORICAL_AGG_RESULTS,
} from '../../../utils/constants';
import { searchResults } from '../../../redux/reducers/anomalyResults';
import {
DAY_IN_MILLI_SECS,
WEEK_IN_MILLI_SECS,
DETECTOR_STATE,
} from '../../../../server/utils/constants';
import { ENTITY_COLORS } from '../../DetectorResults/utils/constants';
import { useLocation } from 'react-router-dom';
import { getDataSourceFromURL } from '../../../pages/utils/helpers';

interface AnomalyDetailsChartProps {
onDateRangeChange(
Expand Down Expand Up @@ -118,6 +123,9 @@ interface AnomalyDetailsChartProps {
export const AnomalyDetailsChart = React.memo(
(props: AnomalyDetailsChartProps) => {
const dispatch = useDispatch();
const location = useLocation();
const MDSQueryParams = getDataSourceFromURL(location);
const dataSourceId = MDSQueryParams.dataSourceId;
const [showAlertsFlyout, setShowAlertsFlyout] = useState<boolean>(false);
const [alertAnnotations, setAlertAnnotations] = useState<any[]>([]);
const [isLoadingAlerts, setIsLoadingAlerts] = useState<boolean>(false);
Expand Down Expand Up @@ -174,7 +182,9 @@ export const AnomalyDetailsChart = React.memo(
zoomRange.endDate,
taskId
);
dispatch(searchResults(anomalyDataRangeQuery, resultIndex, true))
dispatch(
searchResults(anomalyDataRangeQuery, resultIndex, dataSourceId, true)
)
.then((response: any) => {
// Only retrieve buckets that are in the anomaly results range. This is so
// we don't show aggregate results for where there is no data at all
Expand All @@ -193,7 +203,9 @@ export const AnomalyDetailsChart = React.memo(
taskId,
selectedAggId
);
dispatch(searchResults(historicalAggQuery, resultIndex, true))
dispatch(
searchResults(historicalAggQuery, resultIndex, dataSourceId, true)
)
.then((response: any) => {
const aggregatedAnomalies = parseHistoricalAggregatedAnomalies(
response,
Expand Down Expand Up @@ -229,7 +241,9 @@ export const AnomalyDetailsChart = React.memo(
zoomRange.endDate,
taskId
);
dispatch(searchResults(anomalyDataRangeQuery, resultIndex, true))
dispatch(
searchResults(anomalyDataRangeQuery, resultIndex, dataSourceId, true)
)
.then((response: any) => {
const dataStartDate = get(
response,
Expand Down Expand Up @@ -318,7 +332,7 @@ export const AnomalyDetailsChart = React.memo(
try {
setIsLoadingAlerts(true);
const result = await dispatch(
searchAlerts(monitorId, startDateTime, endDateTime)
searchAlerts(monitorId, startDateTime, endDateTime, dataSourceId)
);
setIsLoadingAlerts(false);
setTotalAlerts(get(result, 'response.totalAlerts'));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ import {
FAKE_ANOMALIES_RESULT,
FAKE_DATE_RANGE,
} from '../../../../pages/utils/__tests__/constants';
import { Router } from 'react-router-dom';
import { createMemoryHistory } from 'history';

const DEFAULT_PROPS = {
onDateRangeChange: jest.fn(),
Expand All @@ -47,17 +49,32 @@ const DEFAULT_PROPS = {
entityAnomalySummaries: [],
} as AnomaliesChartProps;

const history = createMemoryHistory();

const renderDataFilter = (chartProps: AnomaliesChartProps) => ({
...render(
<Provider store={mockedStore()}>
<CoreServicesContext.Provider value={coreServicesMock}>
<AnomaliesChart {...chartProps} />
<Router history={history}>
<AnomaliesChart {...chartProps} />
</Router>
</CoreServicesContext.Provider>
</Provider>
),
});

describe('<AnomaliesChart /> spec', () => {
beforeAll(() => {
Object.defineProperty(window, 'location', {
value: {
href: 'http://test.com',
pathname: '/',
search: '',
hash: '',
},
writable: true
});
});
test('renders the component for sample / preview', () => {
console.error = jest.fn();
const { getByText, getAllByText } = renderDataFilter({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,18 @@ import {
} from '../../../../pages/utils/__tests__/constants';
import { INITIAL_ANOMALY_SUMMARY } from '../../utils/constants';
import { getRandomDetector } from '../../../../redux/reducers/__tests__/utils';
import { createMemoryHistory } from 'history';
import { Router } from 'react-router-dom';

jest.mock('../../../../services', () => ({
...jest.requireActual('../../../../services'),

getDataSourceEnabled: () => ({
enabled: false
})
}));

const history = createMemoryHistory();

const renderAnomalyOccurenceChart = (
isNotSample: boolean,
Expand All @@ -30,27 +42,41 @@ const renderAnomalyOccurenceChart = (
...render(
<Provider store={mockedStore()}>
<CoreServicesContext.Provider value={coreServicesMock}>
<AnomalyDetailsChart
onDateRangeChange={jest.fn()}
onZoomRangeChange={jest.fn()}
bucketizedAnomalies={false}
anomalySummary={INITIAL_ANOMALY_SUMMARY}
anomalies={FAKE_ANOMALY_DATA}
detector={getRandomDetector()}
dateRange={FAKE_DATE_RANGE}
isLoading={false}
showAlerts={isNotSample}
isNotSample={isNotSample}
isHCDetector={isHCDetector}
anomalyGradeSeriesName={'testAnomalyGradeSeriesName'}
confidenceSeriesName={'testConfidenceSeriesName'}
/>
<Router history={history}>
<AnomalyDetailsChart
onDateRangeChange={jest.fn()}
onZoomRangeChange={jest.fn()}
bucketizedAnomalies={false}
anomalySummary={INITIAL_ANOMALY_SUMMARY}
anomalies={FAKE_ANOMALY_DATA}
detector={getRandomDetector()}
dateRange={FAKE_DATE_RANGE}
isLoading={false}
showAlerts={isNotSample}
isNotSample={isNotSample}
isHCDetector={isHCDetector}
anomalyGradeSeriesName={'testAnomalyGradeSeriesName'}
confidenceSeriesName={'testConfidenceSeriesName'}
/>
</Router>
</CoreServicesContext.Provider>
</Provider>
),
});

describe('<AnomalyDetailsChart /> spec', () => {
beforeAll(() => {
Object.defineProperty(window, 'location', {
value: {
href: 'http://test.com',
pathname: '/',
search: '',
hash: '',
},
writable: true
});
});

test('renders the component in case of Sample Anomaly', () => {
console.error = jest.fn();
const { getByText } = renderAnomalyOccurenceChart(false, false);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,42 +16,67 @@ import { mockedStore } from '../../../../redux/utils/testUtils';
import { CoreServicesContext } from '../../../../components/CoreServices/CoreServices';
import { coreServicesMock } from '../../../../../test/mocks';
import { AnomalyOccurrenceChart } from '../AnomalyOccurrenceChart';
import { createMemoryHistory } from 'history';
import { Router } from 'react-router-dom';
import {
FAKE_ANOMALY_DATA,
FAKE_DATE_RANGE,
} from '../../../../pages/utils/__tests__/constants';
import { INITIAL_ANOMALY_SUMMARY } from '../../utils/constants';
import { getRandomDetector } from '../../../../redux/reducers/__tests__/utils';

jest.mock('../../../../services', () => ({
...jest.requireActual('../../../../services'),

getDataSourceEnabled: () => ({
enabled: false
})
}));

const history = createMemoryHistory();

const renderAnomalyOccurenceChart = (
isNotSample: boolean,
isHCDetector: boolean
) => ({
...render(
<Provider store={mockedStore()}>
<CoreServicesContext.Provider value={coreServicesMock}>
<AnomalyOccurrenceChart
onDateRangeChange={jest.fn()}
onZoomRangeChange={jest.fn()}
title="test"
bucketizedAnomalies={false}
anomalySummary={INITIAL_ANOMALY_SUMMARY}
anomalies={FAKE_ANOMALY_DATA}
detector={getRandomDetector()}
dateRange={FAKE_DATE_RANGE}
isLoading={false}
showAlerts={isNotSample}
isNotSample={isNotSample}
isHCDetector={isHCDetector}
anomalyGradeSeriesName={'testAnomalyGradeSeriesName'}
confidenceSeriesName={'testConfidenceSeriesName'}
/>
<Router history={history}>
<AnomalyOccurrenceChart
onDateRangeChange={jest.fn()}
onZoomRangeChange={jest.fn()}
title="test"
bucketizedAnomalies={false}
anomalySummary={INITIAL_ANOMALY_SUMMARY}
anomalies={FAKE_ANOMALY_DATA}
detector={getRandomDetector()}
dateRange={FAKE_DATE_RANGE}
isLoading={false}
showAlerts={isNotSample}
isNotSample={isNotSample}
isHCDetector={isHCDetector}
anomalyGradeSeriesName={'testAnomalyGradeSeriesName'}
confidenceSeriesName={'testConfidenceSeriesName'}
/>
</Router>
</CoreServicesContext.Provider>
</Provider>
),
});

describe('<AnomalyOccurrenceChart /> spec', () => {
beforeAll(() => {
Object.defineProperty(window, 'location', {
value: {
href: 'http://test.com',
pathname: '/',
search: '',
hash: '',
},
writable: true
});
});
test('renders the component in case of Sample Anomaly', () => {
console.error = jest.fn();
const { getByText } = renderAnomalyOccurenceChart(false, false);
Expand Down
Loading

0 comments on commit 48acb93

Please sign in to comment.