Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tools should be deployed directly to the back end #50

Open
jjcarver opened this issue Jun 23, 2021 · 1 comment
Open

Tools should be deployed directly to the back end #50

jjcarver opened this issue Jun 23, 2021 · 1 comment
Labels
enhancement New feature or request

Comments

@jjcarver
Copy link
Collaborator

jjcarver commented Jun 23, 2021

Currently, the workflow deployment scripts deploy both workflow XML files and tools together, directly to one of the ProteoSAFe front end web servers, by taking advantage of the /data/cluster/ mount that is present on all such servers. This is not an appropriate approach since tools are back end artifacts and should not be deployed directly a to a front end server.

Ideally the workflow deployment scripts should deploy tools directly to the ProteoSAFe back end (i.e. ccms-login-temp.ucsd.edu). This is the where all back end mounts are officially accessed, and is the only place where we can safely rely on any cross-server symlink relationships being safely resolvable.

@jjcarver jjcarver added the enhancement New feature or request label Jun 23, 2021
@jjcarver
Copy link
Collaborator Author

Currently, the workflow deployment scripts tightly couple workflow XML files (which, from ProteoSAFe's perspective, are front end web server artifacts) and their associated tools (which are back end cluster artifacts). It is sensible to associate these logically (i.e. by keeping them together in the same repository), but from a deployment perspective any back end artifact (e.g. tools) should ideally be deployed directly to the back end, not the front end.

Right now we deploy both workflow XMLs and tools together by taking advantage of a file system mounted on all production ProteoSAFe web servers - /data/cluster/. This file system is mounted as a convenience on all web servers since it points to that web server's back end "delegate" file system. However, this is considered a convenience since this file system contains back end artifacts and thus its presence is strictly unnecessary on any front end server, and consequently it should not be relied upon for any production services.

Normally this would just be a philosophical nitpick that we could safely ignore, since we are going to keep the /data/cluster/ mounts on all ProteoSAFe web servers and therefore they can be relied upon in practice. However, a recent attempt to deploy a workflow to proteomics3 illustrated a more practical problem - proteomics2 and proteomics3 are supposed to have the exact same tools mount, which we accomplish via symlinks, but these only work on the cluster since proteomics2's back end is not mounted on proteomics3. Thus any attempt to deploy tools directly to proteomics3 will fail.

We could work around this with the current deployment scripts by either never deploying workflows directly to proteomics3, or by mounting proteomics2's back end on proteomics3 so that the symlinks work there. We will not do the latter since this defeats the whole purpose of ProteoSAFe's distributed design - then server front ends and back ends would be tightly coupled via some ugly interrelation existing only to make an imperfect solution work. For now we will just have to go with the former.

In the long term, however, these kinds of special server relationships can be cleanly supported as long as we make sure to deploy all workflow artifacts where they are supposed to go - XMLs to the front end, tools to the back end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant