Implement Pagination Features for FHIRClient and Bundle Classes #174
Conversation
I believe this is because of some bad indenting in Note that manually editing
I'm down for whatever - that last option is probably easiest for just getting something that works. And maybe the solution also depends on whether we think this feature "belongs" in fhir-parser. Just wanted to flag the close relationship these two repos have. |
|
@mikix Thank you. I'll have more time this week and hope to finish this PR. |
LanaNYC
force-pushed
the
feat/pagination-Bundle
branch
from
9a69be0
to
8f4a32b
Compare
LanaNYC
changed the title
…next to Bundle, and moved pagination logic to _utils.py
LanaNYC
force-pushed
the
feat/pagination-Bundle
branch
from
8f4a32b
to
08a8535
Compare
| raise e | ||
|
|
||
|
|
||
| def iter_pages(first_bundle: 'Bundle') -> Iterable['Bundle']: |
There was a problem hiding this comment.
Does this type hinting work without fully specifying it? Like 'fhirclient.models.bundle.Bundle'? If so, great. If not and that seems too wordy for you, I think you could also do something like:
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from fhirclient.models.bundle import Bundle
| ("type", "type", str, False, None, True), | ||
| ]) | ||
| return js | ||
| js = super(Bundle, self).elementProperties() |
There was a problem hiding this comment.
This section feels like accidental over-indentation?
| def __iter__(self): | ||
| """ Makes the Bundle itself an iterator by returning an iterator over its entries. """ | ||
| if self.entry is None: | ||
| self._entry_iter = iter([]) | ||
| else: | ||
| self._entry_iter = iter(self.entry) | ||
| return self | ||
|
|
||
| def __next__(self): | ||
| """ Returns the next BundleEntry in the Bundle's entry list using the internal iterator. """ | ||
| # return next(self._entry_iter) | ||
|
|
||
| if not hasattr(self, '_entry_iter'): | ||
| self.__iter__() | ||
| return next(self._entry_iter) |
There was a problem hiding this comment.
nit: could this whole thing just be:
def __iter__(self):
return iter(self.entry or [])
Like, what's the advantage of making Bundle an iterator itself instead of just an iterable? (Honest question, I'm not deep in the iterable weeds)
There was a problem hiding this comment.
Also... having read both PRs now - maybe adding direct iteration support to Bundle is not worth the effort? Like the code in _util.py in this PR could handle iterating on entries and we wouldn't need any fhir-parser changes after all?
That said! I think it's cool to have the direct iteration. But is a new direction for us - adding custom cool Python features to individual classes. Which is nice! But if you want to avoid that complexity here, I'd be happy with all this iteration logic just living as fhirsearch-specific logic.
There was a problem hiding this comment.
(I know I sent you down the path of fhir-parser - so this is probably annoying to hear - I am now just thinking that everyone is likely going to get their Bundles from fhirsearch anyway... I guess it shows up in other places... So direct iteration is still useful... But I think in my head, the iteration would be the pagination itself - which is annoying to do oneself and thus helpful for us to provide. But if the iteration is over the entries, and we do pagination all in fhirsearch, maybe this bit of synctatic sugar can be left off for now, since it reduces the PR complexity a fair bit.)
There was a problem hiding this comment.
And to be clear, iterating over the entries makes sense too - my initial thought was over the pages, but entries feels more natural (for entry in bundle:). So I'm not advocating for changing the iteration to pagination. I'm just realizing that iterating over entries is a small bit of syntactic sugar instead of a big one.
| if not first_bundle: | ||
| return iter([]) | ||
| yield first_bundle | ||
|
|
||
| def perform_resources(self, server): |
There was a problem hiding this comment.
nit: while we're here, maybe add -> list['Resource']: on this method, which should help differentiate how this is different than the _iter version.
(And probably add -> 'Bundle': to perform() too.)
|
|
||
|
|
||
| # Use forward references to avoid circular imports | ||
| def perform_iter(self, server) -> Iterator['Resource']: |
There was a problem hiding this comment.
The way this is written, it will iterate on BundleEntry objects, yeah? But we actually want Bundles.
Should this be re-written as return iter_pages(self.perform(server)) (and change the return hint to Iterator['Bundle']?
| # Old method with deprecation warning | ||
| warnings.warn( | ||
| "perform_resources() is deprecated and will be removed in a future release. " | ||
| "Please use perform_resources_iter() instead.", | ||
| DeprecationWarning, | ||
| ) |
There was a problem hiding this comment.
Fun - I almost never use the warnings module.
I would personally say that we should deprecate perform too - any search can yield multiple Bundles, and not treating a search that way from the get go is likely to encourage buggy code.
When we remove perform and perform_resources, I was thinking we might just rename the iter version into the base versions. Which maybe deserves a different phrasing than "will be removed" -- but maybe that's a bad plan, to cause "needless" name churn. The _iter isn't hurting anyone. So I guess ignore this paragraph. 😄
| bundle = self.perform(server) | ||
| resources = [] | ||
| if bundle is not None and bundle.entry is not None: |
There was a problem hiding this comment.
nit: this code could just be return list(self.perform_resources_iter()) yeah? (just to reduce duplicated code paths, and to emphasize that the method is now just a "worse" version of the _iter one)
There was a problem hiding this comment.
Actually, that change would be a behavior change - but I think an important one? Previously, this did no pagination. But that just means that we'd throw resources on the floor with no indication we had done so or way to work around that.
So I think switching this method to use pagination under the hood is a valuable fix.
| if not first_bundle or not first_bundle.entry: | ||
| return iter([]) |
There was a problem hiding this comment.
This check isn't needed, is it? perform can't return None and the not .entry check is handled by __iter__.
This method could be written as the following, I think?
for bundle in self.perform_iter(server):
for entry in bundle:
yield entry.resource
There was a problem hiding this comment.
Thank you for adding these tests, love to see it ❤️
|
@mikix Thank you for detailed feedback. I'll look at everything this weekend. |
Overview
This pull request aims to address pagination functionalities as outlined in issue #172. The implementation includes the following key changes:
FHIRSearchwith iterators.__iter__and__next__methods to theBundleclass to handle pagination._utils.pyfor better modularity and reuse.Cross-Reference: This PR depends on changes in the
fhir-parserrepository that modify the Bundle class to support generator behavior.Please see the related PR in the
fhir-parserrepository: smart-on-fhir/fhir-parser#53.Backporting: Changes were first introduced in
client-pyby modifying the Bundle class directly. This was backported tofhir-parserto maintain consistency and extend generator behavior to Bundle across the libraries.