initial pass at arch diagrams both abstract and implementation #1245
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| from diagrams import Cluster, Diagram, Edge | ||
| from diagrams.onprem.vcs import Github | ||
| from diagrams.onprem.ci import GithubActions, TravisCI, DroneCI, CircleCI | ||
| from diagrams.azure.devops import Pipelines | ||
| from diagrams.generic.storage import Storage | ||
| from diagrams.generic.place import Datacenter | ||
| from diagrams.generic.compute import Rack as Heroku | ||
|
|
||
| with Diagram(name="Conda-Forge Conceptual", show=False, graph_attr=dict(concentrate='True')): | ||
| staged_recipes = Github(label='staged-recipes') | ||
| with Cluster('Build CIs'): | ||
| cis = [Pipelines(label='Windows, Linux, \nMacOS'), DroneCI(label='aarch64'), TravisCI(label='powerpc'), | ||
| CircleCI('misc')] | ||
|
|
||
| linter = Heroku(label='metadata linter') | ||
| rerender = GithubActions(label='feedstock rerender') | ||
| create_feedstock = GithubActions(label='create feedstock repo') | ||
| autotick = GithubActions(label='autotick bot') | ||
|
|
||
| new_feedstock = Github(label='new feedstock') | ||
| anaconda_server = Storage(label='Anaconda Servers') | ||
| pinnings = Github(label='ABI pins') | ||
|
|
||
| internet = Datacenter(label='new version numbers') | ||
| validation = Heroku(label='pkg validation') | ||
|
|
||
| admin_migrations = CircleCI('admin migrations') | ||
|
|
||
|
|
||
| staged_recipes >> cis >> staged_recipes | ||
| staged_recipes >> linter >> staged_recipes | ||
| staged_recipes >> create_feedstock >> new_feedstock >> cis >> validation >> anaconda_server | ||
|
|
||
| new_feedstock >> linter >> new_feedstock | ||
| rerender >> new_feedstock | ||
|
|
||
| autotick >> new_feedstock | ||
| pinnings >> autotick | ||
| internet >> autotick | ||
|
|
||
| admin_migrations >> new_feedstock | ||
|
|
||
| with Diagram(name="Conda-Forge Implementation", show=False, graph_attr=dict(concentrate='True')): | ||
| staged_recipes = Github(label='staged-recipes') | ||
| new_feedstock = Github(label='new feedstock') | ||
| pinnings = Github(label='ABI pins') | ||
|
|
||
| with Cluster('Build CIs'): | ||
| cis = [Pipelines(label='Windows, Linux, \nMacOS'), DroneCI(label='aarch64'), TravisCI(label='powerpc'), | ||
| CircleCI('misc')] | ||
|
|
||
| with Cluster('Heroku Webservices'): | ||
| linter = Heroku(label='metadata linter') | ||
| validation = Heroku(label='pkg validation') | ||
| other_cmds = Heroku(label='webservice commands') | ||
|
|
||
| rerender = GithubActions(label='feedstock rerender') | ||
| other_cmds >> rerender | ||
|
|
||
| create_feedstock = GithubActions(label='create feedstock repo') | ||
| autotick = GithubActions(label='autotick bot') | ||
|
|
||
| anaconda_server = Storage(label='Anaconda Servers') | ||
|
There was a problem hiding this comment. I'm not sure if this is too much detail but I think this should contain more detail about how the channel is implemented. Perhaps three components:
There was a problem hiding this comment. This is all stuff anaconda maintains and so we shouldn't have it here. To us anaconda.org is a free service with a big latency for uploads. There was a problem hiding this comment. I think it's useful to be here so someone can look at this and understand what is might be wrong with conda-forge for a given symptom. Questions and issues with the CDN sync common enough that I think it's worthwhile acknowledging it exists in the conda-forge documentation. Again, I don't feel that strongly about it if others disagree. There was a problem hiding this comment. sure - we can put a big anaconda box around things There was a problem hiding this comment. I'm actually not certain of the anaconda structure. I'm happy to put it in if someone can tell me what the pieces are and how they connect (on the implementation diagram, I think as a concept the channel is just a sink for all of our stuff and a source for users (and ourselves I guess)) |
||
|
|
||
| internet = Datacenter(label='new version numbers') | ||
|
|
||
| admin_migrations = CircleCI('admin migrations') | ||
|
|
||
| staged_recipes >> cis >> staged_recipes | ||
| staged_recipes >> linter >> staged_recipes | ||
| staged_recipes >> create_feedstock >> new_feedstock >> cis >> validation >> anaconda_server | ||
|
|
||
| new_feedstock >> linter >> new_feedstock | ||
| rerender >> new_feedstock | ||
|
|
||
| autotick >> new_feedstock | ||
| pinnings >> autotick | ||
| internet >> autotick | ||
|
|
||
| admin_migrations >> new_feedstock | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if package validation needed for the conceptual diagram as I see it as more of an implementation detail. Perhaps "Anaconda Servers" should be "conda channel"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We maintain the validation with infrastructure we control so it needs to be here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree it needs to be the "Conda-Forge Implementation" diagram but I'm not sure it needs to be in the "Conda-Forge Conceptual" diagram though I don't feel that strongly about it if others disagree.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah sure agreed!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, validation might be an independent concept, since it isn't formally owned by any of the other pieces. Also the fact that we validate our packages is important to the running of conda-forge, since is a barrier to some potential attacks. I would call the implementation detail that this is run off of the heroku server for instance. I'm happy to be turned around on this though.