Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: a common interface for the set of operations shared between Table and PartitionedTableProxy #6162

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

jmao-denver
Copy link
Contributor

@jmao-denver jmao-denver commented Oct 1, 2024

Fixes #5233

Verified that auto complete still works correctly, static type check also works
pydocs is changed, not ideal but not sure if we can do better.

@jmao-denver jmao-denver added this to the 0.37.0 milestone Oct 1, 2024
@jmao-denver jmao-denver self-assigned this Oct 1, 2024
@@ -68,7 +68,7 @@
# if we allow sphinx to generate type hints for signatures (default), it would make the generated doc cluttered and hard to read
autodoc_typehints = 'none'
autoclass_content = 'both'

autodoc_default_options = {'inherited-members': False}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have to see the rendered docs to have a view on this config.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, I like where this is going. There are a few areas I have concerns on, but I don't have the data to assess or comment on:

  1. What do the rendered pydocs look like?
  2. I assume TableOperations will also be used in pydeephaven. That is not illustrated here. Are you going to duplicate the code there, use a git symlink, create a new python package, etc.
  3. There are a few methods that are not supported in some cases (e.g. see https://deephaven.io/core/docs/reference/table-operations/partitioned-tables/proxy/#available-methods). What is the plan? Do they get an unsupported exception if they are used? Do we do something else? etc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't like how the docs were rewritten for the refactor. See below. This just sounds like two sets of docs were stacked on top of each other. It needs a rewording or something to make it ok. I'm sure we are not the first ones to deal with this, so I'm sure there are established strategies that are better.

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. The underlying implementations of the operations are quite different between the client API and the server API. So what they can share is an abstract interface not a generic concrete class as written here. The ABC is useful to a certain degree but because the client and the server are not on parity with supported ops, we will need to add a number of 'NotImplemented' exceptions in the implementation classes. I feel the work belongs to the epic of achieving parity between the client and the server.
  2. The approach I think makes sense is to keep the non-shared methods as they are. This works with auto-completion and wouldn't surprise the users with an 'NotImplemented' exception at runtime

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create common super "interface" for pydeephaven.Table, deephaven.Table, and deephaven.PartitionedTableProxy
2 participants