First request for Mongo FDW #26
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It'd have conflicted with the submodule.
Slightly better dependency management, easier to update, etc.
To help transition users to the new way.
This'll let people use whatever protocol (git, ssh, http, etc.) they want when checking out the project and the submodule will automatically use the same. See [this blog post][]. [this blog post]: http://blog.tremily.us/posts/Relative_submodules/
Also added code for outputing BSON dates to "MongoDB Extended JSON" [1] [1] http://docs.mongodb.org/manual/reference/mongodb-extended-json/
This will avoid multiple connection and disconnection to MongoDB Server. FDW's Options functions separated from "mongo_fdw.c" and placed in "option.c" file. The newly created connection's pooling functions placed into "connection.c" file.
causing the server to crash while query is used in any container, i.e in PlpgSQl function.
FDW-134, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Ajay Pal.
Sometimes getting a sorted result from MongoDB server is slower than performing a sort locally. To have that flexibility add a GUC named mongo_fdw.enable_order_by_pushdown to control the ORDER BY push-down. However, it is set by super users only. The default is true. FDW-134, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Ajay Pal.
Wherever applicable, perform LIMIT and OFFSET operations on the remote server. This reduces network traffic between local PostgreSQL and remote MongoDB servers. Due to limitations on the MongoDB side, we only support Const nodes of type bigint, which allows us to have only 64-bit integer values. FDW-131, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Kashif Zeeshan, and a few cosmetic changes by Jeevan Chalke.
FDW-131, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Kashif Zeeshan
checking. In the case of nested join queries, the existing logic may not correctly propagate the status of the enable_aggregate_pushdown option to the stage where we are accessing the aggregation path. Fix that by saving the status of enable_aggregate_pushdown in MongoFdwRelationInfo, which is updated for every output rel at various stages. It can then be accessed at the time of grouping operation. FDW-560, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Kashif Zeeshan.
We were pushing the ORDER BY clauses to the remote side without verifying whether the sort operator was safe to ship. It resulted in a wrong output when the sort operator isn't default for the sort expression's type. Fix the same. FDW-564, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke.
Commit 234c8cf misplaced the header file into the wrong .c file. Fix the same. Vaibhav Dalvi.
Vaibhav Dalvi.
… level. Along with the enable_join_pushdown server/table level option, add the same named guc to control join push-down at the session level. This allows the user to control the push-down per the query/data pattern. Default is true. FDW-558, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Ajay Pal.
Standard Support for both EDB Postgres Advanced Server 10 and PostgreSQL 10 is already ended. Thus, adjust Makefile so that we restrict compilation of mongo_fdw code against v10. Update the README accordingly. Also, clean up the code for the same. FDW-588, Vaibhav Dalvi, reviewed by Sravan Velagandula.
Jeevan Chalke.
Sometimes, rare though, performing grouping on a remote server is slower than doing it locally. Thus provide a session-level guc named enable_aggregate_pushdown to control the aggregation pushdown. Default is true. FDW-559, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Kashif Zeeshan.
For some queries, we have observed that sorting on a remote server is very slow. Thus we have added a guc named enable_order_by_pushdown. However, if we want to disable/enable remote server sorting at the table or server level, then it can't be done with the guc. Thus add the server/table level option to control the remote sorting. The table-level value of the option takes precedence over the server-level option value. Default is true. FDW-589, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Kashif Zeeshan.
Reported on GitHub through issue #58 by simon-wolf. FDW-631, Vaibhav Dalvi, reviewed by Jeevan Chalke.
Code changes involve adjusting the make_restrictinfo() call per PostgreSQL's upstream changes. Matches reltarget entries only on varno/varattno. It also takes into account the new RTEPermissionInfo stuff. Beginning with v16, fs_relids includes the rangetable indexes of outer joins, if any were involved in this join. The new field fs_base_relids includes only base relation indexes and thus mimics fs_relids's old semantics. Need to adjust the fdw code to account for this. FDW-603, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Ajay Pal, a few changes by Jeevan Chalke.
I forgot to include them in the previous commit, i.e. 0919481. FDW-603, Vaibhav Dalvi.
FDW-626, Vaibhav Dalvi, reviewed by Jeevan Chalke.
The execution of mongodb_init.sh is compulsory before running regression, and it is missed to mention it in the README file. Also, update a few comments in test cases. FDW-600, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke.
MongoDB stopped supporting this driver version for a few years now, and we have already given deprecation warnings in the previous releases. So, clean up the related code. FDW-664, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Ajay Pal.
The main changes include moving the installation steps to the newly created INSTALL.md file. Other changes in README.md are for unifying it with FDW documentation template. Reported on GitHub through issue #168 by mkgrgis (Михаил). FDW-665, initial patch by mkgrgis, further revised and improved by Vaibhav Dalvi.
Tests involving MIN()/MAX() aggregates sometimes take a shortcut path due to hardware differences, resulting in the explain plan changes and, thus causing regression failures. To get a consistent result, change those to SUM()/AVG() aggregates or add a relevant grouping clause. The patch also tweaks the startup cost for the ordered paths so that ORDER BY will get pushed for a few more queries. FDW-598, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke, tested by Ajay Pal.
FDW-676, Suraj Kharage.
Standard Support for both EDB Postgres Advanced Server 11 and PostgreSQL 11 is already ended. Thus, adjust Makefile so that we restrict compilation of mongo_fdw code against v11. Update the README accordingly. Also, clean up the code for the same. FDW-673, Suraj Kharage, reviewed by Sravan Velagandula, tested by Ajaykumar Pal.
Code changes involve adjusting calls to some server-side functions due to their signature changes. FDW-682, Suraj Kharage, reviewed by Jeevan Chalke.
This allows building the extension from the source's contrib directory without the need for pg_config availability in the path. Jeevan Chalke.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
First request for Mongo FDW