Skip to content

Commit

Permalink
feat: Tapis v3 Redesign (#1184)
Browse files Browse the repository at this point in the history
* task/DES-2702: Tapis v3 Auth (#1174)

* add tapipy; Tapis v3 OAuth token and flow

* add docstrings to test methods

* update service account to use Tapis

* remove future imports

* uncomment agave imports

* add TapisOAuthToken model migration

* fix bugs

* remove attrdict for agavepy

* pylint: ignore test files

* fix ignore unit test regex

* fix login url

* formatting; remove AgaveOAuthToken references

* add test settings for tapisv3

* fix auth/views_unit_test.py

* fix backends unit test

* formatting

* task/DES-2654: Tapis v2/v3 apps model (#1158)

* v3 apps model

* make appId required field

* add AppBundle model

* updated model admin

* update models and add migration

* rework app tray models

* squash appitem and appbundle

* un-squash migrations

* remove faulty app description migration

* Use an inline admin form to manage app variants within bundles (#1164)

Co-authored-by: Sal Tijerina <[email protected]>

* model adjustments

* model adjustments; add category pop migration

---------

Co-authored-by: Jake Rosenberg <[email protected]>

* fix fixture import

* skip tapisv2 tests; add todov3 notes; fix fixture

* remove unnecessary commented code

---------

Co-authored-by: Jake Rosenberg <[email protected]>

* Add Google Oauth secrets.

* task/DES-2709: Tapis v3 Apps Views (#1177)

* add apps views

* add AppDescriptionView

* TapisTokenRefreshMiddleware: redirect to login if expired

* formatting

* fix bugs; add AuthenticatedApiView class

* update apps call for v3

* separate linting from unit tests in workflow

* formatting

* formatting

* add uuidv4

* tests

* fix uuid import

* tests

* fix formatting

* set allowJS to true in tsconfig

* add uuid types

* rename docker tags and volumes to not conflict with main branch

* fix volumes

* add tags, description, is_simcenter; some reorg

* remove broken source mapping

* squash migrations

* split websockets; clean up nginx.debug.conf

* change staticfiles dirs to align with deployed config; nginx: /var, django: /srv

* formatting

* task/DES-2710: v3 Jobs Views & Workspace UI updates (#1182)

* add apps views

* add AppDescriptionView

* TapisTokenRefreshMiddleware: redirect to login if expired

* formatting

* fix bugs; add AuthenticatedApiView class

* update apps call for v3

* separate linting from unit tests in workflow

* formatting

* formatting

* add uuidv4

* tests

* fix uuid import

* tests

* fix formatting

* set allowJS to true in tsconfig

* add uuid types

* rename docker tags and volumes to not conflict with main branch

* fix volumes

* add tags, description, is_simcenter; some reorg

* remove broken source mapping

* squash migrations

* split websockets; clean up nginx.debug.conf

* change staticfiles dirs to align with deployed config; nginx: /var, django: /srv

* formatting

* add jobs views; update workspace UI

* fix client linting and tests

* fix some server side tests

* add getUseJobs hook

* add job detail spinner

* add usePostJobs

* fix webhooks tests

* server side linting and tests

* formatting

* fix unit test

* add CSRF_TRUSTED_ORIGINS setting

* add m1 makefile targetsg

* Use Django to serve static files in debug mode (#1186)

* Use Django to serve static files in debug mode

* clean up volumes; remove local nginx static and media locations

* add back /data/media mount

* add media back to nginx; same locations as main

---------

Co-authored-by: Sal Tijerina <[email protected]>

* task/DES-2708: Build and serve React assets on deploy (#1173)

* Build and serve React assets

* fix template string

* Add custom render block for react assets

* re-add agavepy to requirements for legacy migration purposes (#1187)

* update dockerfile to copy static assets to correct location

* add V3 file operation module and file listing util (#1189)

* Dockerfile build targets and cleanup (#1190)

* clean up unused files; add development docker build target; fix workspace react-assets ref

* fix data depot react-assets ref

* docker-compose fix; combine python-base and builder-base steps (#1191)

* docker-compose fix; combine python-base and builder-base steps

* revert rabbitmq images

* parity between intel/m1 rabbit versions

* Revert "parity between intel/m1 rabbit versions"

This reverts commit 9f5b709.

* add newline

* revert .gitignore changes

---------

Co-authored-by: Jake Rosenberg <[email protected]>

* gitignore compiled react assets

* TV3/DES-2706: Add view decorator using Tapis JWT auth (#1192)

* add view decorator using Tapis JWT auth

* Update designsafe/apps/api/decorators.py

Co-authored-by: Nathan Franklin <[email protected]>

* Add method decorator on project instance view

---------

Co-authored-by: Nathan Franklin <[email protected]>

* use wsgi for django server locally with working staticfiles (#1200)

* Quick: fix tapis client import in filemeta

* hotfix/update build info in README.md (#1227)

* Remove obsolete version param

* Update README for building development images

* Allow JWT-authed requests for filemeta routes (#1229)

* Milestone: Project creation/publication works end-to-end (#1236)

* Get project creation, updates, and type changes working end-to-end

* projects creation/curation/publication works end-to-end

* import error fix

* remove print/console.log statements

* DES-2684: NEES Detail View (#1219)

* formatting/type errors

* tweak for pi array check, updated placeholder text

* initial work minus file listing for nees publication details

* reworking top level data styling

* updated citation modal title

* first pass at adding file listing

* fix: app card design bugs DES-2741 through DES-2745 (#1223)

* fix: DES-2471, DES-2472, DES-2473 bugs (#1221)

* fix: DES-2744 hover and focus card UI (#1224)

* hotfix: des 2745 global font icon size tweaks (#1225)

* refactor: des-2745 rename DS font icons stylesheet

* fix: des-2745 load DS font icons + allow customize

* chore: des-2745 obvious app-card font icon space

* fix: des-2745 use rem not em for icon–title space

* fix: des-2745 DS font icons not found

* feat: des-2745 re-align certain DS font icons

* feat: des-2745 re-position certain DS font icons

* chore: des-2745 use (proper) `::before` syntax

* fix: des-2745 title alignment after icon resize

* fix: DES-2766 app with 1 variant needs diff list (#1232)

* fix: DES-2766 app with 1 variant needs diff list

* fix: DES-2766 more padding (designer request)

* formatting

* add nees listing breadcrumbs

* add breadcrumbs

* fix breadcrumbs

---------

Co-authored-by: Jake Rosenberg <[email protected]>
Co-authored-by: Wesley B <[email protected]>

* task/DES-2767: fix file metadata route path issue (#1235)

* Normalize path to be consistent

* Remove some unneeded fixgtures

---------

Co-authored-by: Jake Rosenberg <[email protected]>

* use async task for publication

* task/DES-2705: Update post-login task to use TAPIS V3 to configure the two systems (#1183)

* Add encryption and system access util from CEP

* Add pycryptodome package

* Update login to use tapisv3 to configure the two systems

* Fix linting

* Fix linting/black issues

* Fix lockfile

* Refactor to have parameter path and also use AGAVE_WORKING_SYSTEM via keyservice

* Update test settings for +AGAVE_WORKING_SYSTEM

* Add retry

---------

Co-authored-by: Jake Rosenberg <[email protected]>
Co-authored-by: Sal Tijerina <[email protected]>

* Working search bars in data files/published areas (#1240)

* working search bars in data files/published areas

* update test settings

* Working file operations/Data Depot buttons. (#1242)

* working search bars in data files/published areas

* Working V3 move/copy modals

* Get toolbar buttons working and enabled/disabled correctly depending on context.

* Migrate listings/breadcrumbs to Common for use in apps (#1243)

* migrate file listing table and breadcrumbs to common component module for use in Apps

* pass pipeline without tests

* Working pipeline for amend/revise (#1248)

* fix search filters for hyb sim/field recon

* formatting

* Refinements to Data Depot UI and pub ingest (#1256)

* Add placeholder/error UI for listings; add Data Diagram for pubs

* improve line wrapping in Data Diagram for entities with long names

* refinements to publication ingest and display

* task/WG-258: allow jwt access for project metadata updating (#1255)

* Move project fixture higher up to use in view testing

* Allow jwt access for updating project metadata

---------

Co-authored-by: Jake Rosenberg <[email protected]>

* task/DES-2629: v3 Apps Form (#1185)

* add apps views

* add AppDescriptionView

* TapisTokenRefreshMiddleware: redirect to login if expired

* formatting

* fix bugs; add AuthenticatedApiView class

* update apps call for v3

* separate linting from unit tests in workflow

* formatting

* formatting

* add uuidv4

* tests

* fix uuid import

* tests

* fix formatting

* set allowJS to true in tsconfig

* add uuid types

* rename docker tags and volumes to not conflict with main branch

* fix volumes

* add tags, description, is_simcenter; some reorg

* remove broken source mapping

* squash migrations

* split websockets; clean up nginx.debug.conf

* change staticfiles dirs to align with deployed config; nginx: /var, django: /srv

* formatting

* add jobs views; update workspace UI

* fix client linting and tests

* fix some server side tests

* add getUseJobs hook

* add job detail spinner

* add usePostJobs

* fix webhooks tests

* server side linting and tests

* formatting

* fix unit test

* add useGetApps hook; app routing

* move tapis types to types file

* add spinner; cleanup

* add CSRF_TRUSTED_ORIGINS setting

* add m1 makefile targetsg

* formatting

* wip: appswizard and layout

* wip: apps routing; prefetch app listing data

* remove await/defer

* fix m1 django command

* formatting; linting; 1s stale time

* use wsgi for django server locally with working staticfiles

* wip: cleanup

* remove /applications sub path for tools & apps url

* fix backend response for private apps

* install zod; tanstack form

* add workspace utils; apps submission form

* cleanup unused styles

* wip

* install zod; tanstack form

* add workspace utils; apps submission form

* cleanup unused styles

* remove version from docker compose files

* data loads

* working back

* state persists

* clean up

* remove unused dependencies

* working reset

* working submission

* update antd; add SystemsPushKeysModal

* add ssh keys manager; system push keys endpoint

* push keys backend

* reset push keys form on success

* reorg apps view components

* set defaultSystem in app response

* better param handling; fix apps view settins

* add systems hooks and views

* use prefetching to speed up load time

* fix submit status

* add app icon

* add job submit messages

* Task/DES-2656  apps sidebar (#1202)

* -Adds accordion ant sidebar
-Need to update to allow more than one category to be open
at once
-Needs styles

* Change from Panels to Menu Ant components

* -Adds ant menu and submenus with children rather
than with item prop passed to menu

* Adds unit test to app AppSideNav

* - Fixed unit test

* WIP ant d Menu

* WIP

* side nav submenus

* - Adds styles per mockup
- Adds WIP collapse icon and state to manage it
- Adds working subnav of application bundles

* - Removes unnecessary toggle because antd menu
handles appropriately
- Removes broken expand icon logic

* fix unbundled item render; formatting

---------

Co-authored-by: Sal Tijerina <[email protected]>

* Task/des 2631 apps breadcrumb (#1188)

* Add AppsBreadcrumb component and integrate it into WorkspaceBaseLayout

* Refactor AppsBreadcrumb and AppsSideNav components

* Add appsListingJson import and modify breadcrumb rendering

* Refactor path modification logic in WorkspaceBaseLayout.tsx

* Update subproject commit hash

* Fix archiveSystemDir formatting in JobsView

* Update SITE_ID values in site configuration files

* Update subproject commit hash

* Fix formatting issues in code

* Update app names and formatting

* Update SITE_ID values in site configuration files

* Remove unused code and imports

* Add Tapis operations to datafiles handlers

* Remove unused React assets from base template

* Update React and database configurations

* Update datafiles handlers and workspace views

* Replace Tapis operations with Agave operations

* Update import statement for service_account

* Fix formatting in data_depot.j2 template

* Fix formatting issues in data_depot.j2 template

* Fix formatting and add missing newline at end of file

* retrigger checks

* Refactor JobsDetailModal component

* Add AppsBreadcrumb component and update layout styles

* Refactor AppsBreadcrumb module CSS and fix function formatting

* Remove unused import statement in WorkspaceBaseLayout.tsx

* Fix modifiedPath for Job Status

* Refactor path modification in WorkspaceBaseLayout.tsx

* Add app data and loading state to AppsBreadcrumb component

* updated AppsBreadcrumb

* update to Job Status label

* refactor breadcrumb code from WorkspaceBaseLayout into AppsBreadcrumb

* disable app form for testing on this branch

* Refactor AppsBreadcrumb component to improve breadcrumb rendering; no link for last item

* Refactor AppsBreadcrumb component to improve breadcrumb rendering and remove last item link; change separator from / to >

* Refactor AppsBreadcrumb component to improve breadcrumb rendering and remove last item link; change separator from / to >

* remove suspense query

* revert reversion of suspense

* remove portal dir

---------

Co-authored-by: Sal Tijerina <[email protected]>

* linting

* linting and refactor breadcrumb

* linting

* layout adjustments

* more styling

* fix user guide link

* fix tests; fix tags and labels for description list

* Bug DES-2762: Adding key to avoid optimization from Antd (#1233)

Co-authored-by: Sal Tijerina <[email protected]>

* fix linting

* add typing

* add more typing

* working Edit button

* fix type

* fix username render

* more linting fixes

* more typing

* finish typing

* linting

* fix content hash

* Tasks/DES-2630: App Form styling (#1241)

* App Form Styling according to design (first version)

* Lint fix for FormField

* More lint fixes

* Common Component Button and font increase

* Fix typescript error

* Fix type error

---------

Co-authored-by: Sal Tijerina <[email protected]>

* fix accessor error

* fix workspace prod template

* fix is_bundled computed value; fix missing label

* Support html apps (#1244)

* support html apps

* linting

* task/DES-2758, DES-2747: Tapis v3 Job Notifications (#1245)

* fix job sumbit error render

* fix tapipy error propogation

* job status notifications

* fix state redirect on notification update by removing angular code from workspace

* only render data_depot events

* propogate error for ApiException

* add message to notification events for bell dropdown

* DES-2749: Handle changes to queue (#1246)

Co-authored-by: Sal Tijerina <[email protected]>

* fix common exports

* Bug/DES-2778: Handle number validation in App Form (#1250)

* Bug/DES-2778: Fix validation for number input in App Form

* Fix merge issue

* Lint fix

* fix app variant href

* fix persistent previous app state in step

* fix: allow compute max run time when no queue in VM case

* fix toasts for data_depot events

* more explicitly ignore jobs events

* fix errant root: declaration

* default open app

* Bug/DES-2786: Use key to make sure app instance is created on navigation (#1251)

* Bug/DES-2786: Use key to make sure app instance is created on navigation

Fix state issues

* Remove un-necessary field init

* App Form step and summary rendering fixes (#1252)

Initial working version

* task/des-2657--job-history-listing (#1239)

* Add support for displaying node count and cores per node in JobsListing component

* Add support for displaying archive system directory in JobsListing component

* interactive session modal; styling

* wip: notification badge

* linting

* add todo notes

---------

Co-authored-by: Sal Tijerina <[email protected]>

* disable notifs in job status nav

* task/DES-2779: Jobs Listing Notification Badge & Update Job Listing Upon Notification (#1253)

* update notification badge and listing when job updates; enable job listing buttons

* linting

* linting

* fix app ordering

* remove unused import

* tasks/DES-2655: Select file modal in Apps Form (#1254)

* Create SelectModal for file selection

* Handle public vs private

* Use isMyData to check if this is private or public

---------

Co-authored-by: Sal Tijerina <[email protected]>

* task/DES-2713 job history modal (#1234)

* - Adds View Details modal for job status listings
- Adds working output link button (may need to check I'm directing
to the correct path)
- Does not add a functioning delete button
- Adds modal with the design based of current DS Job Detail Modal

* -Adds closing/cancel action
- Doesn't handle state correctly for closing
- Buttons and links don't work yet

* - Adds state handling for open/close modal

* Adds linting changes

* Adds linting

* Uses PrimaryButton component

* Task/des 2713 job history modal  adjustments (#1257)

* url nav for job modal; working buttons

* fix data files links

---------

Co-authored-by: Sal Tijerina <[email protected]>

* render job detail on top of listing

* DES-2629: Use portal names to build list. (#1258)

* fix Tools & Applications link

* DES-2789: Only validate current steps on continue button (#1260)

* Apps form styling fixes (#1262)

* DES-2818: add missing license prompt

* fix submit button; submission details css; standard button width; standard font family

* job status header; small job action buttons

* larger summary item label width

* adjust spacing

* revert helvetica default

* fix button size on job detail modal

* use base Button for link type

* linting

* Remove htmltype submit on continue button (#1263)

* task/DES-2755 + DES-2756: Allocations backend + hooks (#1247)

* Allocations backend w/ TAS hookup

* Add Allocations elasticsearch model + add model to elasticsearch settings

* Add allocation type

* Add useGetAllocations hook

* Add TAS env variables to django settings

* Add UserAllocations model

* Use UserAllocations model in view

* Update designsafe/apps/workspace/api/views.py

Co-authored-by: Jake Rosenberg <[email protected]>

* Pylint errors fixes

* Create util function for grabbing user allocations and creating/updating their cached allocations

* Add async allocations cache update to login flow

* Roll back elasticsearch implementation

* Pylint fixes for allocations model

* fix unit test

* Roll back elasticsearch changes to IndexedApp

* Resolve pylint errors on allocations model

* Fix client linting errors

* Fix poetry black reformatting checks

* task/DES-2820: Allocations in App Form (#1264)

* use allocations in app form

* use dynamic user model; fix task

* fix linting

* fix type error

* fix test

* linting

---------

Co-authored-by: Garrett Edmonds <[email protected]>
Co-authored-by: Sal Tijerina <[email protected]>
Co-authored-by: Jake Rosenberg <[email protected]>
Co-authored-by: Garrett Edmonds <[email protected]>

---------

Co-authored-by: sophia-massie <[email protected]>
Co-authored-by: Van Go <[email protected]>
Co-authored-by: Chandra Y <[email protected]>
Co-authored-by: Jake Rosenberg <[email protected]>
Co-authored-by: Garrett Edmonds <[email protected]>
Co-authored-by: Garrett Edmonds <[email protected]>
Co-authored-by: Garrett Edmonds <[email protected]>

* fix cache_allocations task; linting

* Fixes for publication session items (#1270)

* Fixes for project/publication testing session items

* layout fixes and explanation for pipeline select

* fixes for file tags and operations on published files

* use filetype-specific icons

* various pipeline fixes

* add Best Practices modal and fix ingest for legacy simulations

* add version changes modal

* linting/type fixes

* task/DES-2823: Fix interactive modal (#1266)

* update placeholder job listing message

* fix interactive modal state

* Task/DES-2801 [UI] Apps Nav Bar Spacing (#1271)

* - Adds spacing styles and header to AppsSideNav
- Removes unused style with comment

* - Adds border to sider nav menu

* - Linting

* - Linting

* - Removes AppNavLink component
- Removes display block from Title div so that
justify content style applies
- Change font for title to match design
- Change color of right nav border to match design

* -Change item height of nav bar
- Removes css module for AppsSideNav

---------

Co-authored-by: Sal Tijerina <[email protected]>

* docs: tapis v3 redesign readme step №3 (#1273)

* docs: Tapis v3 changes to "First time setup" №3

* docs: Do not use `[!NOTE]` syntax

* Bug/layout sider fix (#1274)

* - Moves app sider styles to workspace specificity
to avoid changing Data Depot sider style

* - Linting

* DES-2819: Select Modal Design Updates (Part 1 of 2) (#1276)

* DES-2655: Select Modal Design work

* Adjust modal body to new global styles

* Address review comments

* fix lint

* Fix isRequired for nested fields (#1277)

* Task/des 2849 cite this data (#1272)

* commit to main

* Add useCitationMetrics hook for fetching citation metrics in publications module

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Add MetricsModal to project modals and update Datacite API endpoints

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor ProjectCitation module and styles, and add DownloadCitation component

* Refactor ProjectCitation module and styles, and add DownloadCitation component

* chore: Refactor ProjectCitation module, add DOI link to PublishedCitation

* Refactor ProjectCitation module, add DOI link to PublishedCitation

* chore: Update Docker images and volumes configuration for development environment

* Refactor ProjectPreview module, fix useState declaration

* Refactor useState declaration in ProjectPreview module

* Refactor ProjectCitation module, optimize DOI retrieval and usage

* feat: Optimize DOI retrieval and usage in ProjectCitation module

* Refactor ProjectCitation module, optimize DOI retrieval and usage

* Refactor ProjectCitation module, remove commented out code for MetricsModal

* Update index.ts

---------

Co-authored-by: Jake Rosenberg <[email protected]>

* Apps form testing session fixes (#1279)

* fix SystemsPushKeysModal: error handling; onSuccess callback; layout; info text

* show missing allocation message

* rm unuseful comment

* open View User Guide in new tab

* point notifications to job history; remove apps and jobs from dashboard

* task/DES-2726: configure cloud data on login (#1265)

* Configure cloud.data on login

* Update test settings and todo comments

* Rename setting

* Remove listing check in from view

Checkin and configuring occurs only in the task now

* Move task to an onboarding queue

---------

Co-authored-by: Sal Tijerina <[email protected]>

* Add modals for publication feedback/archive download (#1275)

* Add feedback and project download modals

* link formatting

* form/modal overflow fixes

* Bug: Handle zod optional fields for undefined or empty values (#1281)

* include App Overview link in breadcrumbs (#1283)

* task/DES-2759 : Apps side nav styling updates (#1280)

* Add shortLabel for rendering in app side nav - add shortLabel to AppVariant model

* Add shortLabel for rendering in app side nav - add shortLabel to relevant app views

* Add shortLabel for rendering in app side nav - add shortLabel to frontend app types

* Optionally render shortLabel for apps in side nav + update hover and selected colors for side nav menu items

* Remove unused import

* Linting fixes

* include app notes.shortLabel in _get_public_apps response

* fix submenu open key

* update apps side nav border colors and weight (#1282)

* Modify main themeConfig for design's #cbdded hover & select menu item color

* Client linting fix

* Change new appVariant field name to 'short_label'

* Update references to 'shortLabel' to renamed 'short_label for AppVariant model

* Linting fixes for new migration

* Update designsafe/apps/workspace/api/views.py

---------

Co-authored-by: Garrett Edmonds <[email protected]>
Co-authored-by: Sal Tijerina <[email protected]>

* Bug DES-2876: Select Modal bug fixes (#1284)

* Select Modal fixes

* Update usePathDisplayName.ts

* Fix from merge issue - getPathName

* Breadcrumb use system label

* Bug/download citation not working in next (#1285)

* commit to main

* Add useCitationMetrics hook for fetching citation metrics in publications module

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Add MetricsModal to project modals and update Datacite API endpoints

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor ProjectCitation module and styles, and add DownloadCitation component

* Refactor ProjectCitation module and styles, and add DownloadCitation component

* chore: Refactor ProjectCitation module, add DOI link to PublishedCitation

* Refactor ProjectCitation module, add DOI link to PublishedCitation

* chore: Update Docker images and volumes configuration for development environment

* Refactor ProjectPreview module, fix useState declaration

* Refactor useState declaration in ProjectPreview module

* Refactor ProjectCitation module, optimize DOI retrieval and usage

* feat: Optimize DOI retrieval and usage in ProjectCitation module

* Refactor ProjectCitation module, optimize DOI retrieval and usage

* Refactor ProjectCitation module, remove commented out code for MetricsModal

* Refactor DOI retrieval and usage in ProjectCitation module

* test commit

* Update index.ts

* fix app version href

* Bugs/DES-2828 and DES-2829: Select Modal system selection and validation bugs (#1287)

* Retain system select based on input and validation fix

* Fix test failure

* update default DS allocation prefix

* also accept DesignSafe-DCV

* fix allocation check on non-batch exec system

* full length workspace content; apps side nav scrolls (#1290)

* quick: fix hidden overflow on app side nav

* task/DES-1989 - Redirect guest users when accessing non-published project (#1292)

* If an unauthenticated user now attempts to view a project, they will now be prompted to log in. Some styling changes done to make the JSX messages look like the Django error messages produced in file listings.

* Ran a formatter

* Minor change to make sure projectID and data were defined when used.

* One last linting change.

* fixes to pipeline publish operations (#1294)

* scope .o-site__body changes to workspace

* revert changes to alert styling

* - Uses selectedKey property in Menu to correctly (#1296)

select/deselect active Menu Item based on route

* Workspace fixes from 6/12 testing session (#1295)

* unread notifs: only count unique jobs

* disable JobActionButton after success; make cancel button red

* fix scrolling on long app form

* expand app side nav to 250px

* disable app form while submitting

* fix notifications fetching; highlight newest notification

* remove DesignSafe-DCV as default allocation

* fix next url param redirect; update login redirect setting to reflect production

* use formatDateTimeFromValue util; do not render date for null values

* fix eventTypes array parsing

* fix notification bell event links

* Update apps placeholder text

* Hide "Edit" button for step when that step is active

* fix style override on ant btn disabled state

* fix long app form scroll

* allow small buffer for long app param labels

* no horizontal scrolling

* add right margin to user guide link to avoid scrollbar ui conflict

* Task/des 2862 update form validation messages (#1297)

* commit to main

* Add useCitationMetrics hook for fetching citation metrics in publications module

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Add MetricsModal to project modals and update Datacite API endpoints

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* add Total Request values to table

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor useCitationMetrics hook and update MetricsModal component

* Refactor useCitationMetrics hook and update MetricsModal component

* Refactor MetricsModal component and add Total Request values to table

* Refactor ProjectPreview module, remove DownloadCitation component

* update variable names

* format write

* Refactor ProjectPreview module, update MetricsModal props

* refactor: Add custom error messages to form validation rules

* refactor: Update form validation error messages

* Remove file from the pull request

* refactor: Update form validation error messages

* refactor: Update form validation error messages

* refactor: Update form validation error messages

* Delete client/modules/datafiles/src/projects/modals/MetricsModal.tsx

* refactor: Update form validation error messages

* task/WG-294: improve links from DS projects to hazmapper maps (#1291)

* Fix links from DS projects to hazmapper maps

* Fix linting issues

* TV3: disable move/rename on files with metadata associations. (#1293)

* disable move/rename on files with metadata associations

* remove logging

* Task/des 2666 metrics modal (#1237)

* commit to main

* Add useCitationMetrics hook for fetching citation metrics in publications module

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Add MetricsModal to project modals and update Datacite API endpoints

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* add Total Request values to table

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor useCitationMetrics hook and update MetricsModal component

* Refactor useCitationMetrics hook and update MetricsModal component

* Refactor MetricsModal component and add Total Request values to table

* Refactor ProjectPreview module, remove DownloadCitation component

* update variable names

* format write

* Refactor ProjectPreview module, update MetricsModal props

* Update docker-compose-dev.all.debug.yml

* Update docker-compose-dev.all.debug.yml

---------

Co-authored-by: Jake Rosenberg <[email protected]>

* pipeline bugfixes; redirect to project after publishing (#1298)

* close dropdown menus after selection

* replace registration link with redirect to TAM

* accept jwt auth for license view (#1300)

* DES-2896: Do not block continue to next step on invalid fields (#1301)

* styling for job status nav when active and hover (#1303)

* Bugs/DES-2907: Exclude allocations at portal level (#1302)

* Bug/DES-2907: Allow exclusions from allocations at portal level

* Fix formatting

* Use env for settings

* add default to ALLOCATIONS_TO_EXCLUDE

* task/DES-2911: Render `jobAttributes` `notes.label` as form field label if it exists (#1304)

* Render job attributes notes.label if it exists

* linting

* linting

* Task/des 2905 cite this data type other (#1299)

* Add citation download options and metrics to ProjectPreview and PublishedDetailLayout

* Refactor Download Citation link in ProjectPreview and PublishedDetailLayout

* Refactor useCitationMetrics hook and remove unused error variable

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor useCitationMetrics hook and update ProjectPreview module and styles

* Refactor useCitationMetrics hook and remove MetricsModal from ProjectPreview module

* Refactor useCitationMetrics hook and remove unused error variable

* Refactor import path for MetricsModal in ProjectCitation and PublishedDetailLayout

* DES-2909 and DES-2914 - bug fixes (#1306)

Co-authored-by: Garrett Edmonds <[email protected]>

* Bugs/DES-2888: Select Modal - regex fix to detect storage system and file selection modes (#1308)

* Regex fix and file selection modes

* Lint and formatting fixes

* Bug/DES-2835,2855: Ribbon Buttons and Amend text (#1311)

* chore: Refactor RenameModal to handle file extensions correctly

* feat: Add step to change metadata in curation directory before publishing

* Fixes: Data Depot testing session  2024-06-17 (#1310)

* data depot fixes- dropdown usability and error UI for forms

* pipeline display fixes

* fixes for Trash button

* working downloads for private data

* disable downloads from frontera.work

* don't indicate success on submission of async file transfer

* DES-2931, DES-2932, DES-2929: Project edit modal fixes (#1314)

* allow editing of type None; fix guest member input on type Other

* enforce URL formatting on referenced data/related works

* Show tags in Select Files listing for type Other (#1312)

* Bug/DES-2923: Remove Download Data from My Projects (#1318)

* chore: Refactor PipelineSelectForPublish component

- Add line break for clarity in the process description
- Include instructions for submitting a ticket with project number, dataset name(s), and author order

* feat: Add line break for clarity in the process description and include instructions for submitting a ticket with project number, dataset name(s), and author order

* feat: Add hybrid simulation steps to project type modal

* chore: Refactor PipelineSelectForPublish component

* Refactor

* chore: Refactor DownloadCitation component to include preview option

* remove previous ticket files

* bug/DES-2922: word/spelling/typo issues (#1313)

* chore: Refactor PipelineSelectForPublish component

- Add line break for clarity in the process description
- Include instructions for submitting a ticket with project number, dataset name(s), and author order

* feat: Add line break for clarity in the process description and include instructions for submitting a ticket with project number, dataset name(s), and author order

* set mydata acls for other and mask (#1315)

* feat: DES-2903 add id to app listing headings (#1316)

* DES-2940 - handle env var with labels (#1319)

* feat: DES-2806 cms breadcrumbs (#1268) (#1320)

* feat: DES-2903 add id to app listing headings

* feat: DES-2631 cms breadcrumbs

- added link state mixin
- added link state styles
- moved CSS variables
- added CSS variables
- added CMS breadcrumbs template
- included CMS breadcrumbs template
- changed DjangoCMS breadcrumb template

* fix: DES-2631 font size & distance from nav

* fix: DES-2809 lighter 2nd-level breadcrumb text

* fix: DES-2809 only add breadcrumbs on app page

* feat: DES-2809 load CSS sooner

* refactor: DES-2806 no new breadcrumb classes

I.e. skin Bootstrap's.

* chore: DES-2806 remove outdated vars

Added in DES-2806, but then not used after later commits for DES-2806.

* fix: DES-2806 incorrect margin ref. thus spacing

* fix: DES-2806 font size should match feat/Tapis-v3

* feat: DES-2806 move bs3 breadcrumb style to .css

* fix: DES-2806 core-styles update & match workspace

* fix: DES-2806 match workspace top padding

* feat: DES-2806 improve cms focus ui

* fix: DES-2806 match workspace colors

* fix: DES-2806 use `.breadcrumb-item` not `li`

Bootstrap uses `.breadcrumb-item`.

* Revert "fix: DES-2806 use `.breadcrumb-item` not `li`"

This reverts commit b58cc96.

Bootstrap 4 uses `.breadcrumb-item`, but not Bootstrap 3.

* chore(cms): update core-styles breadcrumb

* chore(cms): use core-styles breadcrumb at v2.27.0

* Bug/des 2990  Fix fr protected data options (#1321)

* show protected data options when changing project type to FR

* improve variable naming

* formatting

* fix prop typing

* Persist HazMapper maps when changing project type (#1323)

* bug/DES-3000: Handle DOI not provided in useCitationMetrics (#1325)

* feat: Handle DOI not provided in useCitationMetrics

* feat: Add cache page decorator to PublicationDetailView and PublicationDataCiteView

* feat: Add cache page decorator to PublicationDetailView and PublicationDataCiteView

* Bug/DES-2982: text/typo/wording/spelling issues (#1329)

* Fix typo in curation office hours link

* chore: Fix typo in Publish/ Amend/ Version

* Pipeline fixes and Fedora representation for publications (#1332)

* Pipeline fixes and Fedora representation for publications

* formatting

* task/DES-2924: Add metadata and move/copy/download buttons to preview modal (#1334)

* Add metadata and move/copy/download buttons to preview modal

* formatting

* Data Files/Projects styling fixes (#1336)

* Scope scroll overflow behavior in modals to Tools & Applications (#1338)

* Scope overflow scroll in modal to Tools & Applications

* clean up diff

* Task/DES-2984: Use a single form for creating and updating categories (#1341)

* use a single form for both creating and editing categories

* set smooth scrolling; remove error dialog when switching edit mode

* fix type error

* successCallback typo fix in preview/move modals

* Bugs/DES-2872: [AppForm] Handle axios request errors without crashing client (#1322)

* DES-2872: AppViews error handling

* Add key

* Task/WA-199: Uniform margin/padding for HTML app content. (#1342)

* Add uniform padding to HTML app container

* add top margin to app HTML

* Include all job submission values in job detail modal (#1333)

* Job Details Modal

* Fix label and remove unnecessary env variables

* Fix unit test error

* Switch to antd description instead of core's descriptionlist

* Filter display items with no data

---------

Co-authored-by: Garrett Edmonds <[email protected]>
Co-authored-by: Sal Tijerina <[email protected]>

* Update file metadata on move/copy/rename (#1344)

* [AppForm] Use bundle_icon field for app icon in App Form header (#1345)

* [SelectModal] New features added - back button and first row (#1340)

* [SelectModal] Back button and related features

* Fix lint error

* Disable selection for root if it is first row

* Update SelectModal.tsx

---------

Co-authored-by: Sal Tijerina <[email protected]>

* add search bars to NEES layouts; add links to files in experiments (#1347)

* task/DES-2935: Notifications fine tuning (#1346)

* hover pointer; duration 5s; onClick to job history

* only fetch notifications once - subsequent notifications will come through as webhooks

* Bug/des-2986: description error messages (#1328)

* Refactor form validation rules for project description length

* Refactor form validation rules for project description length

* Handle `memoryMb` value change on queue change (#1350)

* update docker compose command in makefile; show execution dir while running (#1351)

* split datacite metrics and events hooks; fix caching (#1349)

* reset 'download datset' modal on close

* add dropdown with help links (#1355)

* [Apps] Translate Early Termination TAPIS Failure Status to Success if interactive (#1358)

* [Job History] Implement expected functionality of "Reuse Inputs" (#1357)

* [Job History] Implement expected functionality of "Reuse Inputs"

* Fix build error

* Address review comments - shared component, suspense

* Fix lint error

---------

Co-authored-by: Sal Tijerina <[email protected]>

* add category to breadcrumb (#1360)

* task/DES-3020: Fix scroll in HTML apps (#1361)

* fix html app overflow

* remove unnecessary css

* Task/DES-2601: DOI logging for file/publication operations (#1356)

* log DOIs when published files are downloaded/copied/previewed

* re-add logging when entity dropdown is opened

* Add instructions for bulk data transfer into projects (#1365)

* Bugs/DES-3033 [Job Details Modal] Do not show hidden fields and show fields with empty strings (#1364)

* DES-3033: Handle hidden parameters and inputs

* more consolidion

* task/DES-2936 - Hide Outputs section in form for interactive jobs (#1359)

* Outputs form files are hidden from interactive job submission, still needs to be hidden from side panel.

* Hid Outputs field from Job Submission Summary sidebar.

* Refactored to bring app definition in as prop, to prevent accidentally double-loading the app and to make the code more readable.

---------

Co-authored-by: Sal Tijerina <[email protected]>

* Task/WG-316 Data Depot disable menu hazmapper (#1352)

* Disabled menu on preview modal and on listing for
hazmapper files

* - Disables Copy file option in preview PreviewModal
- Adds display of response in preview modal for testing

* - Adds functionality to open hazmapper map in hazmapper
in PreviewContent
- Exports Hazmapper base urls
- Handles cancel of preview modal once the hazmapper
map opens in new tab

* - Adds handleCancel prop to PreviewContent in PreviewModal

* - Front end linting

* - Fixing merge issues

* - Linting

* - Addressing error Expected a 'break' statement
before 'default'  no-fallthrough
- Addresses lexical delcaration error in Hazmapper
case block

* - Linting

* - Linting error

* - Combines file menu logic in PreviewModal

---------

Co-authored-by: Sal Tijerina <[email protected]>

* task/DES-2939 -  New App Notes Field "hideQueue"  (#1326)

* Adding hideQueue flag for apps to hide the queue selection from users. Should only be used with Compress/Extract for now.

* Can now hide queue from job subission summary

* As suggested, Queue and Allocation now hide independently of each other, and the fields are hidden from the Summary if the app definition requires it.

* Improve handling of error responses for projects/publications (#1367)

* improve error boundaries on projects/pubs

* remove logging

* task/DES-2938 & DES-2941: App form updates (#1366)

* Add user_guide_link to app entry model

* Render user_guide_link from app model in app form header, and move app form header to capture html apps

* Move Suspense one more level up from Header

* Fix element arrangement + clean up unused vars

* linting

* linting

* linting

* move user_guide_link to AppListingEntry model; send value to client side; ignore blank entry

* minor comment adjustment; capitalize Queue in job detail modal

* fix app form vertical scroll

* add max_length to charfield

---------

Co-authored-by: Sal Tijerina <[email protected]>

* formatting

* update to Usage Breakdown

* Revert "update to Usage Breakdown"

This reverts commit fe80f0d.

* Bug/des 2528 update datacite metrics (#1371)

* chore: Fix MetricsModalBody data display issue

* chore: Fix MetricsModalBody data display issue

* Revert "update to Usage Breakdown"

This reverts commit fe80f0d.

* chore: Update MetricsModalBody to display most recent year by default

* chore: Update MetricsModalBody to display most recent year by default

* chore: Update MetricsModalBody to display most recent year by default

* chore: Refactor MetricsModalBody to improve data display and filtering

* update missing allocation message (#1372)

* Task/DES-2707: add script to migrate project systems to v3 tenant (#1180)

* Add tapipy

* Use tapipy 1.6.3 due to bug in 1.6.2

* Add command to migrate project systems

* Log any issues

* Fix pylint issues

* Format with 'black' code formatter

* Fix linting issues after black formatting changes

* Update v3 system using any users who only exist in v2 system roles

* Fix linting issues

* Refactor how community data is handled

---------

Co-authored-by: Sal Tijerina <[email protected]>

* disable injection of path information into anchor elements on error pages (#1374)

* fix ingest script for projects

* Quick: Fix `check_or_configure_system_and_user_directory` `retry()`; Update `<SystemsPushKeysModal />` wording. (#1376)

* fix retry call; update wording on push keys modal

* use bind=True

* fix tombstone ingest and add tombstone UI (#1378)

* hide move button on public systems; hide copy button for unauth'd users (#1379)

* DES-2626: NCO TTC Updates (#1368)

* next batch of nco ttc grants page updates

* removing category filter, adding subjects as keywords to details modal

* updating button styles

* fixing bug in abstract details modal

* adding pi name to the text field search

* fix deprecated external_resource_secrets setting

* log DOI when downloading project archives (#1382)

* Quick: Maintenance toggle with staff bypass (#1381)

* Add setting to toggle maintenance page display

* refactor logic

* don't show page for requests coming from TACC VPN

* move VPN IP prefix to settings

* fix last-updated dates on projects after migrating (#1385)

* list all app versions (#1386)

* Fedora: Ingest on publish (#1387)

* add Fedora ingest step to publication pipeline

* update test settings

* fix webhook settings for local dev

* formatting

* always use www in prod redirect uri (#1388)

* get nco dashboard working for unauth'd users (#1389)

* NCO fix

* Fix/prod redirect uri (#1391)

* always use www in prod redirect uri

* add www replacement in tapis_oath call

* Update quarters in MetricsModal to match correct time periods

* Revert "Update quarters in MetricsModal to match correct time periods"

This reverts commit 5db4818.

* Update quarters in MetricsModal to match correct time periods (#1393)

* Use Zod refine for regex (#1392)

* curation fixes for entity ordering/file trashing (#1395)

* task/WP-647: Remove portal name filter for jobs by default (#1394)

* remove portalname filter by default for jobs

* remove old test description from setfacl job

---------

Co-authored-by: Garrett Edmonds <[email protected]>

* Now apps will list simcenter apps after popular apps, but before the rest of the apps. (#1397)

Co-authored-by: Sal Tijerina <[email protected]>

---------

Co-authored-by: Jake Rosenberg <[email protected]>
Co-authored-by: Nathan Franklin <[email protected]>
Co-authored-by: Sarah Gray <[email protected]>
Co-authored-by: Wesley B <[email protected]>
Co-authored-by: sophia-massie <[email protected]>
Co-authored-by: Van Go <[email protected]>
Co-authored-by: Chandra Y <[email protected]>
Co-authored-by: Garrett Edmonds <[email protected]>
Co-authored-by: Garrett Edmonds <[email protected]>
Co-authored-by: Garrett Edmonds <[email protected]>
Co-authored-by: fnets <[email protected]>
  • Loading branch information
12 people authored Aug 14, 2024
1 parent 6e297f8 commit 3cdb782
Show file tree
Hide file tree
Showing 481 changed files with 24,327 additions and 9,312 deletions.
6 changes: 5 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,8 @@ coverage
.docs
.github
.pytest_cache
data
data

client/node_modules
client/.nx
client/dist
8 changes: 0 additions & 8 deletions .docs/source/designsafe.apps.auth.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,6 @@ designsafe.apps.auth.backends module
:undoc-members:
:show-inheritance:

designsafe.apps.auth.context_processors module
----------------------------------------------

.. automodule:: designsafe.apps.auth.context_processors
:members:
:undoc-members:
:show-inheritance:

designsafe.apps.auth.middleware module
--------------------------------------

Expand Down
5 changes: 4 additions & 1 deletion .flake8
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
[flake8]
# E501: line is too long.
# H101: Use TODO(NAME)
ignore = E501, H101
# W503: line break before binary operator. Ingore as black will break this rule.
ignore = E501, H101, W503
exclude = __pycache__,
tests.py,
migrations

extend-ignore = W503
74 changes: 57 additions & 17 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
name: CI

# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
# Controls when the action will run. Triggers the workflow on pushes to main or on pull request events
on:
push:
branches: [ master ]
branches: [ main ]
pull_request:
branches: [ '**' ]

Expand All @@ -17,7 +16,7 @@ jobs:
- uses: actions/checkout@v4

- name: Fetch base and install Poetry
run: |
run: |
git fetch origin ${{github.base_ref}}
pipx install poetry
Expand All @@ -34,6 +33,30 @@ jobs:
- run: |
poetry install
- name: Run Server-side unit tests and generate coverage report
run: |
poetry run pytest --cov-config=.coveragerc --cov=designsafe --cov-report=xml -ra designsafe
Server_Side_Linting:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4

- name: Fetch base and install Poetry
run: |
git fetch origin ${{github.base_ref}}
pipx install poetry
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
cache: 'poetry'

- name: Install Python Packages
run: |
poetry install
- name: Run Server-side linting with pytest
# Only run on new files for now-- for all changes, filter is ACMRTUXB
# Check manage.py to prevent a crash if no files are selected.
Expand All @@ -44,16 +67,12 @@ jobs:
run: |
poetry run black $(git diff --name-only --diff-filter=A origin/${{github.base_ref}} | grep -E "(.py$)") manage.py --check
- name: Run Server-side unit tests and generate coverage report
run: |
poetry run pytest --cov-config=.coveragerc --cov=designsafe --cov-report=xml -ra designsafe
Client_Side_Unit_Tests:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Setup Node.js for use with actions
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: 16.x
cache: npm
Expand All @@ -64,12 +83,12 @@ jobs:
React_NX_unit_tests:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Setup Node.js for use with actions
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
Expand All @@ -78,14 +97,35 @@ jobs:
working-directory: client

- uses: nrwl/nx-set-shas@v3
# Check linting/formatting of workspace files.
- run: npx nx format:check
working-directory: client

# Lint/test/build any apps and libs that have been impacted by the diff.
- run: npx nx affected --target=lint --parallel=3
working-directory: client
# Test/build any apps and libs that have been impacted by the diff.
- run: npx nx affected --target=test --parallel=3 --ci --code-coverage
working-directory: client
- run: npx nx affected --target=build --parallel=3
working-directory: client

React_NX_linting:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Setup Node.js for use with actions
uses: actions/setup-node@v4
with:
node-version: 20
cache: npm

- run: npm ci
working-directory: client

- uses: nrwl/nx-set-shas@v3

# Check linting/formatting of workspace files.
- run: npx nx format:check
working-directory: client

# Lint any apps and libs that have been impacted by the diff.
- run: npx nx affected --target=lint --parallel=3
working-directory: client
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ settings.json
designsafe/apps/rapid/static/designsafe/apps/rapid/build/bundle.*
designsafe/apps/geo/static/designsafe/apps/geo/build/bundle.*
designsafe/static/build/
designsafe/static/react-assets/
designsafe/templates/react-assets.html
# designsafe/static/styles/base.*
/static

Expand Down
2 changes: 1 addition & 1 deletion .pylintrc
Original file line number Diff line number Diff line change
Expand Up @@ -583,7 +583,7 @@ ignored-checks-for-mixins=no-member,
# List of class names for which member attributes should not be checked (useful
# for classes with dynamically set attributes). This supports the use of
# qualified names.
ignored-classes=optparse.Values,thread._local,_thread._local,argparse.Namespace
ignored-classes=optparse.Values,thread._local,_thread._local,argparse.Namespace,Tapis

# Show a hint with possible names when a member name was not found. The aspect
# of finding the hint is based on edit distance.
Expand Down
18 changes: 15 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,23 @@
.PHONY: build
build:
docker-compose -f ./conf/docker/docker-compose.yml build
docker compose -f ./conf/docker/docker-compose.yml build

.PHONY: build-dev
build-dev:
docker compose -f ./conf/docker/docker-compose-dev.yml build

.PHONY: start
start:
docker-compose -f ./conf/docker/docker-compose-dev.all.debug.yml up
docker compose -f ./conf/docker/docker-compose-dev.all.debug.yml up

.PHONY: stop
stop:
docker-compose -f ./conf/docker/docker-compose-dev.all.debug.yml down
docker compose -f ./conf/docker/docker-compose-dev.all.debug.yml down

.PHONY: start-m1
start-m1:
docker compose -f ./conf/docker/docker-compose-dev.all.debug.m1.yml up

.PHONY: stop-m1
stop-m1:
docker compose -f ./conf/docker/docker-compose-dev.all.debug.m1.yml down
54 changes: 27 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,34 +40,37 @@ If you are on a Mac or a Windows machine, the recommended method is to install
- `AGAVE_*`: should be set to enable Agave API integration (authentication, etc.)
- `RT_*`: should be set to enable ticketing

Make copies of [rabbitmq.sample.env](conf/env_files/rabbitmq.sample.env) and [mysql.sample.env](conf/env_files/mysql.sample.env),
then rename them to `rabbitmq.env` and `mysql.env`.

Make copies of [mysql.sample.cnf](conf/mysql.sample.cnf), [redis.sample.conf](conf/redis.sample.conf),
and [rabbitmq.sample.conf](conf/rabbitmq.sample.conf), then rename them to `mysql.cnf`, `redis.conf`, and `rabbitmq.conf`.
Make a copy of [rabbitmq.sample.env](conf/env_files/rabbitmq.sample.env)
then rename it to `rabbitmq.env`.

Make a copy of [external_resource_secrets.sample.py](designsafe/settings/external_resource_secrets.sample.py)
and rename it to `external_resource_secrets.py`.

3. Build the containers and frontend package
3. Build the containers and frontend packages

```
$ make build
```
or
```
$ docker-compose -f conf/docker/docker-compose.yml build
```
1. Containers:
```sh
make build-dev
```
or
```sh
docker-compose -f conf/docker/docker-compose-dev.yml build
```

These lines install the node packages required for DesignSafe,
and build the frontend package.
```
$ npm ci
$ npm run build
```
2. Angular Frontend + static assets:
```sh
npm ci
docker run -v `pwd`:`pwd` -w `pwd` -it node:16 /bin/bash -c "npm run build"
```

If you are working with the frontend code and want it to automatically update,
use `npm run dev` rather than `npm run build` to have it build upon saving the file.
**Note:** If you are working with the frontend code and want it to automatically update, use `npm run dev` rather than `npm run build` to have it build upon saving the file.

3. React Frontend (in another terminal):
```sh
cd client
npm ci
npm run start
```

4. Start local containers

Expand All @@ -82,7 +85,7 @@ If you are on a Mac or a Windows machine, the recommended method is to install
```
$ docker exec -it des_django bash
$ ./manage.py migrate
$ ./manage.py collectstatic -i demo
$ ./manage.py collectstatic --ignore demo --no-input
$ ./manage.py createsuperuser
```

Expand Down Expand Up @@ -225,8 +228,8 @@ $ docker-compose -f conf/docker/docker-compose-dev.all.debug.yml up
$ npm run dev
```
When using this compose file, your Agave Client should be configured with a `callback_url`
of `http://$DOCKER_HOST_IP:8000/auth/agave/callback/`.
When using this compose file, your Tapis Client should be configured with a `callback_url`
of `http://$DOCKER_HOST_IP:8000/auth/tapis/callback/`.
For developing some services, e.g. Box.com integration, https support is required. To
enable an Nginx http proxy run using the [`docker-compose-http.yml`](docker-compose-http.yml)
Expand All @@ -238,9 +241,6 @@ $ docker-compose -f docker-compose-http.yml build
$ docker-compose -f docker-compose-http.yml up
```
When using this compose file, your Agave Client should be configured with a `callback_url`
of `https://$DOCKER_HOST_IP/auth/agave/callback/`.

### Agave filesystem setup
1. Delete all of the old metadata objects using this command:
Expand Down
5 changes: 0 additions & 5 deletions bin/build_client.sh

This file was deleted.

7 changes: 0 additions & 7 deletions bin/dumpdata.sh

This file was deleted.

10 changes: 0 additions & 10 deletions bin/loaddata.sh

This file was deleted.

1 change: 0 additions & 1 deletion bin/mysql.sh

This file was deleted.

2 changes: 1 addition & 1 deletion bin/run-celery-debug.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@
#
celery -A designsafe beat -l info --pidfile= --schedule=/tmp/celerybeat-schedule &
celery -A designsafe worker -l info --autoscale=15,5 -Q indexing,files -n designsafe_worker01 &
celery -A designsafe worker -l info --autoscale=10,3 -Q default,api -n designsafe_worker02
celery -A designsafe worker -l info --autoscale=10,3 -Q default,api,onboarding -n designsafe_worker02
6 changes: 0 additions & 6 deletions bin/run-celery-dev.sh

This file was deleted.

2 changes: 1 addition & 1 deletion bin/run-celery.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
# Run Celery as the DesignSafe Community Account
celery -A designsafe beat -l info --pidfile= --schedule=/tmp/celerybeat-schedule &
celery -A designsafe worker -l info --autoscale=15,5 -Q indexing,files -n designsafe_worker01 &
celery -A designsafe worker -l info --autoscale=10,3 -Q default,api -n designsafe_worker02
celery -A designsafe worker -l info --autoscale=10,3 -Q default,api,onboarding -n designsafe_worker02
3 changes: 0 additions & 3 deletions bin/run-django.sh

This file was deleted.

6 changes: 0 additions & 6 deletions bin/run-flower.sh

This file was deleted.

18 changes: 0 additions & 18 deletions bin/run-tests.sh

This file was deleted.

3 changes: 0 additions & 3 deletions bin/run-uwsgi.sh

This file was deleted.

Loading

0 comments on commit 3cdb782

Please sign in to comment.