-
Notifications
You must be signed in to change notification settings - Fork 389
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(docs): generate docs from docstrings #5131
base: v2
Are you sure you want to change the base?
feat(docs): generate docs from docstrings #5131
Conversation
This is a fairly major change to the docs and generates documentation for all public methods, attributes and variables in AWS Powertools. This is achieved using `mkdocstrings`, which replaces instances of ```markdown ::: some.module.path ``` with the documentation of the specified module. As a side effect of generating the documentation, an `objects.inv` file is also generated. From a quick purview of the generated docs, they seem pretty good as the underlying code appears well documented; however, this probably needs to be double checked. Especially for cases where the Numpy docstring style is not being followed which may result in errors generating the docs. Ref: aws-powertools#5125 Signed-off-by: JP-Ellis <[email protected]>
Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need. |
Quality Gate passedIssues Measures |
Hey @JP-Ellis thanks for working in this PR and creating a reproduicle outcome from what we can expect. Let me split down my feedback to make easy we agree on the next steps. pdoc3 vs mkdocstringsWe already have an API documentation created by pdoc3, which works well. Personally, I like it; however, I see much more value in using Testing this PR2 - I tested this PR locally and I was able to create the Current limitations/problems3 - Even if we agreeing to migrate from We should not render these attributes The code is completely broken here We should not have these special characters in the examples Making a smooth transitionI'm thinking about making a smooth transition from Enabling a new Ruff rule5 - We can use this opportunity to progressively enable a new Ruff linter rule: https://docs.astral.sh/ruff/rules/#pydocstyle-d. We didn't enable that because our docstrings and comments need to be revamped. Next stepsSo, based on what I explained before, I suggest the next steps are: 1 - Modify this PR to add just one utility and make all the changes we need to make to the mkdocstrings + docstrings configuration. I suggest we start with BatchProcessing as the codebase is not complex and we can do an excellent job on it that will serve as a base for the others. Please let me know what you think and if you would like to make these changes and go through some back-and-forth review process. If you don't have time to do this —we know everyone is very busy— I can handle it and ping you for reviews and feedback. Please let me know your thoughts. Again, thanks a lot for kicking out this to make our documentation even better. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Leaving a comment to Request Changes.
Amazing feedback! I should have made it clear at the start that this is meant to be a first proof-of-concept and step towards MkDocstrings. I didn't want to invest too much time in case this wasn't going to be adopted. I'm glad to hear this is something you are keen on! To answer some of your more specific feedback.
I'll be honest, I completely missed that. But I'm glad to hear this was in the pipeline already!
Completely agree. When I personally use this, I actually create a small Python scripts that is invoked by Mkdocs to automatically generate the Markdown files on-the-fly. There are a couple of pros and cons:
Yeah, that's completely fair. A per-utility approach would certainly be possible. Since you already have the Note The below docs are in the process of being transitioned and may contain some formatting issues.
Absolutely a fantastic thing to add. Just be aware that Ruff doesn't necessarily enforce all docstring rules all the time as it tries to be smart about which lints are applicable to which rules. As a concrete example in repos I maintain, Ruff will detect if there is a mismatch between the args documented and the args in the function; but it will not raise a lint if the docstring has no args at all. This can be problematic if the args are in fact documented but in a style which Ruff did not expect. With the pace of Ruff's development, this may have changed over the past months, but that is something to be aware of.
Sounds good! I won't get much time this week to contribute, but hopefully this weekend or next week I should get a bit of time :) |
Hey @JP-Ellis! Thanks for the feedback and clarifying things about this PoC. I am writing additional feedback based on your answers.
Yes, we should avoid doing this. My observation about the number of files is because it makes it difficult to review the PR and we can lose attention with too many files. That's why I suggested dividing this into small PRs by utility, which will make our work easier.
Perfect, this is a really good idea.
Hmmm, I didn't know that. I'll spend some time checking this out and see what limitations/caveats we should be aware of and deal with "manually" when doing this work.
Perfect! Take your time, no rush at all and will be amazing to make this change and improve our documentation. Thanks for taking the time to do this and let's make it happen 😄 |
Issue number: #5125
Summary
Changes
This is a fairly major change to the docs and generates documentation for all public methods, attributes and variables in AWS Powertools.
This is achieved using
mkdocstrings
, which replaces instances ofwith the documentation of the specified module.
As a side effect of generating the documentation, an
objects.inv
file is also generated.From a quick purview of the generated docs, they seem pretty good as the underlying code appears well documented; however, this probably needs to be double checked. Especially for cases where the Numpy docstring style is not being followed which may result in errors generating the docs.
User experience
There is no change to the usage of AWS Powertools.
There will be a significant change to the documentation in that the classes/functions/variables will be made public. Most editors would already be presenting this information to developers already, but this just makes it discoverable online.
The main benefit here is that other docs which want to refer to a specific class or function from AWS Powertools can now do so using the
objects.inv
file generated.Checklist
If your change doesn't seem to apply, please leave them unchecked.
Acknowledgment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.