First request for Mongo FDW #26

It'd have conflicted with the submodule.

Slightly better dependency management, easier to update, etc.

To help transition users to the new way.

This'll let people use whatever protocol (git, ssh, http, etc.) they want when checking out the project and the submodule will automatically use the same. See [this blog post][]. [this blog post]: http://blog.tremily.us/posts/Relative_submodules/

Conflicts: README

Also added code for outputing BSON dates to "MongoDB Extended JSON" [1] [1] http://docs.mongodb.org/manual/reference/mongodb-extended-json/

This will avoid multiple connection and disconnection to MongoDB Server. FDW's Options functions separated from "mongo_fdw.c" and placed in "option.c" file. The newly created connection's pooling functions placed into "connection.c" file.

causing the server to crash while query is used in any container, i.e in PlpgSQl function.

…ot "name".

…goDB.

Conflicts: Makefile README.md mongo_fdw.c mongo_query.c

Support JSON Read

r Please enter the commit message for your changes. Lines starting

file to avoid conflict with other libraries.

read/write binary data (bytea)

updating .gitmodules adding META_DRIVER flag to Makefile.meta fixing bson_iterator type reference fixing BSON error reporting adding readPreference support forgot to change header file fixing more reference to mongo_get_connection using readPreference) fixing option for read_preference fixing includes fixing includes removing submodule for meta driver in favor of pkg-config fixing legacy driver iter methods fixing string iterator fixing regex iter fixing code iter type fixing missing method for meta driver fixing binary iterator fixing JsonDump fixing method signature fixing binary iterator fixing BsonAppendBinary setting slaveOk=true for when readPreference is provided fixing readPreference fixing SLAVE_OK forgot to delete some unused code trying to fix collection count patch to make count be able to read from slave fixing count method references trying to fix count method still tring to work around mongo driver bugs still struggling still struggling sigh finally! getting rid of some useless stuff Adding documentation for installing meta driver Fixing formatting

support write text[] (TEXTARRAYOID)

… hot and port.

Conflicts: connection.c mongo_fdw.c mongo_fdw.h option.c

Adding support for libmongoc-1.0 and libbson-1.0

"Merge pull request #12 from mjburghoffer/master"

…of MongoDB.

Read/Write JSON

Merge pull request #13 from the truongsinh / master - Disable the JSON Support for MongoDB's Meta (master branch) because it need to be implemented. - Fix the compilation errors in case of Meta Driver. - Fix the formatting issue introduce by the this pull request Merge pull request #13 from the truongsinh / master - README changes

because it is not working for Meta Driver.

…iver.

stable version of Mongo-C driver and json lib using wget.

The script assumed that the current working dir, `.` is in the PATH. Now a full relative path is used to allow executing configure on systems where this is not the case.

JSON datatype was previously only supported with Mongo's legacy driver. The wrapper for meta driver was not providing implementation of JsonToBsonAppendElement function. This patch adds complete implementation of this function.

Fix autogen.sh to help building mongo-c

should recurse the FillTupleSlot function.

Implement BsonIterSubObject for the meta driver. Fixes #34 and #38.

This feature allows the user to retrieve the full mongo document in a single JSON column. The user would need to add a __doc json column to the foreign table and mongo_fdw will fetch the entire mongo document in this column.This feature only supported in MongoC Meta driver.

…h script.

Added support for ssl in meta driver.

The patch supports other popular types too like text, varchar etc and stops types that can crash like tsquery or produce strange results like numeric. Test cases are added as a part of the patch.

Issue (#72) Provide support for JSONB as type of special __doc column

Test cases are attached with the patch.

Issue (#64) Add support for push down of parameterized where clause

…ation.

Issue #77 - Support for authentication database in the CREATE SERVER

Issue (#88) - Add PostgreSQL version 10 to all Makefiles

Update json-c to 0.12.1

Change version in control file Add sql file for 1.1 & sql file to upgrade from 1.0 to 1.1 Add new sql files in Makefile Add a function to get .so version select mongo_fdw_version(); mongo_fdw_version ------------------- 50100 (1 row)

[RFC] Bump version of mongo-c-driver

Don't destroy BSON object returned by mongoc_cursor_next function.

Issue - #105: Fix crash while selecting data.

Commit fixes the issues which caused compilation error when compiled against PG/EPAS 11.

Upstream commit 8237f27b504ff1d1e2da7ae4c81a7f72ea0e0e3e removed get_relid_attribute_name and added missing_ok get_attname so it can serve the purpose of get_relid_attribute_name. This commit replace get_relid_attribute_name() to get_attname().

Update the json-c library release version and remove the unnecessary file to patch json-c.

Issue - (#113): Fix compilation error of the json-c library.

Add $(LIBJSON)/strerror_override.o to $LIBJSON_OBJS

Remove unnecessary cd into a subdir, followed by a cd back up

Commit fixes the issues which caused compilation error when compiled against PG/EPAS 12.

Earlier commit missed to change the Makefile.meta and .legacy for the v12 support. So this commit fix the same as well as fix few other warning due to v12 API changes.

This was broken as it's not using the correct APIs for fetch the tuple descriptor attributes.

FDW-83, Vaibhav Dalvi, reviewed by Suraj Kharage.

FDW-104, Vaibhav Dalvi, further revised by Jeevan Chalke, reviewed by Rajkumar Raghuwanshi.

This fixes the warnings and errors appearing when compiled with EPAS 13. FDW-102, Vaibhav Dalvi, reviewed by Suraj Kharage.

Like we have it in the database server, set per file type attributes in .gitattributes to fine-tune whitespace checks. However, this file has only the required attributes needed for mongo_fdw. Jeevan Chalke

Earlier, mongo_fdw was using an ongoing connection even after the connection parameters were changed by ALTER SERVER or ALTER USER MAPPING commands. Fix it by watching for a catcache invals on these catalogs and re-establish the connection when these entries change, similar to that of postgres_fdw. FDW-108, Vaibhav Dalvi, reviewed by Suraj Kharage.

FDW-83, Vaibhav Dalvi, reviewed by Suraj Kharage.

It includes - renaming few function names, and variables to match with the postgres_fdw naming conventions. - rearranging code. - removing unnecessary initializations. - removing unnecessary header files. - avoiding unnecessary variables and minimizing their scope. - auditing palloc0() calls, preferring to palloc() whenever possible. - comments, and error message improvements. - indentation, and white-spacing adjustments. FDW-83, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke.

Found when compiled against PostgreSQL. Jeevan Chalke.

mongoc_client_new() function does not return NULL when it fails to connect with the server. So we need to ping the server to make sure that connection is reachable with the parameters given. Along the way, move the connection establishment logic from MongoExecForeignInsert/Update/Delete functions to the MongoBeginForeignModify() function so that we won't try to ping the server for every insert/update/delete operations. FDW-127, Vaibhav Dalvi, reviewed by Suraj Kharage.

Earlier it was giving an error for meta driver and crashing for the legacy driver. Fix those. MongoAcquireSampleRows() function was calling MongoBeginForeignScan() to get a few relevant details by passing fake scanState which was wrong. Change that entire logic so that the relevant details are fetched directly inside MongoAcquireSampleRows() itself. FDW-158, Jeevan Chalke, reviewed by Vaibhav Dalvi, an earlier version of the patch posted by Vaibhav Dalvi is reviewed by Suraj Kharage, but I have completely revised it.

If the testcase fails, it becomes difficult to identify which area got broken. Having separate testcase files helps a bit. Also, added an ORDER BY clause at missing places to ensure unique output order. Along the way, through mongodb_init.sh importing a few data into the Mongo server which will be used in the testcases. FDW-143, Rajkumar Raghuwanshi, reviewed by Vaibhav Dalvi and Jeevan Chalke, few cosmetic changes by me.

It removes a few obsolete link references and rephrases some contents. FDW-140, Vaibhav Dalvi, reviewed by Suraj Kharage, further adjustments by me.

Meta driver APIs expect port value in unsigned short type which resulted in a compilation warning on some gcc version as we were passing an int to it. So change that to fix the compilation warnings. The legacy driver, however, accepts an int, so we cast that value to int while passing to those APIs. Now as we have changed the type to uint16, added a strict check making sure that the value for the port provided is within the ushort range. FDW-199, Vaibhav Dalvi, reviewed and test-case added by me.

In commit 9b4843b, I have failed to update the server_options_1.out. Fix that. Reported and patch by Vaibhav Dalvi.

correctly. Sub-select or correlated queries that use a parameterized plan uses ReScanForeignScan API. However, this API is not correctly rescanning the data. In the case of parameters, we need to recreate the Mongo cursor that fetches the data again. The patch does the same. In passing, refactor some code in this area so that Mongo cursor is created from MongoIterateForeignScan() that is called after Begin and ReScan APIs. However, create a Mongo cursor only if it was not created already. FDW-103, Vaibhav Dalvi, reviewed by Suraj Kharage.

To check which columns are needed to be output, we create a column list from the target list. However, we are only adding the plain Var nodes appearing in the target list. Thus, if a target list has an explicit casting, then it won't appear as a plain Var node and thus not added in the column list, resulting in returning NULL for them. Fix that by correctly pulling out the Vars from the expression appearing in the target list by using pull_var_clause(). This enables us to give a correct result even if we have expressions other than an explicit cast like a function call or operators. Reported on GitHub through issues #133 by Maksim Volkau (dadhi). FDW-197, Vaibhav Dalvi, reviewed by Suraj Kharage

COPY FROM and/or foreign partition routing code path in core assumes that FDW has BeginForeignInsert() APIs present and thus later executes ExecForeignInsert(). However, mongo_fdw does not support routable foreign-table partitions and/or executing COPY FROM on foreign tables and thus do not have BeginForeignInsert() API implemented. But as it has ExecForeignInsert() API, it gets called for these operations and results in the server crash. To fix this, add the BeginForeignInsert() API that throws an error. Also, add EndForeignInsert() similar to the Begin API. The issue was originally reported on MySQL's GitHub through #208. FDW-226, Suraj Kharage, reviewed by Jeevan Chalke.

In get_baserel_parampathinfo(), there is an Assert() which verifies that if rel has LATERAL refs, then every path for it should have lateral_relids. However, in MongoGetForeignPaths() while calling the create_foreignscan_path() we were passing NULL to the required_outer parameter which is passed to the get_baserel_parampathinfo() and thus we end-up getting an assertion failure. Fix that by correctly passing baserel->lateral_relids to the create_foreignscan_path(). FDW-249, Vaibhav Dalvi, reviewed by me.

FDW-227, Kashif Zeeshan, reviewed by Rajkumar Raghuwanshi.

Update autogen.sh to fetch mongo-c-driver 1.17.3 and json-c 0.15. This allows us to support MongoDB server 4.4. Earlier, mongo-c-driver (1.9.5) was supporting MongoDB 3.6, which is quite old, so upgrade to the latest one. Installation steps in autogen.sh needs some adjustments, as these new versions now use CMake. Since with this patch, as we moved to the newest mongo-c-driver, we need to adjust some codes to get rid of the deprecated code. FDW-238, Vaibhav Dalvi, reviewed by Jeevan Chalke, tested by Rajkumar Raghuwanshi.

When building the mongo-c driver, autogen script also builds json-c. However, with commit 6f409d7 we build the latest version of json-c which uses cmake 3+. The json-c version 0.15 includes the xlocale.h file which on some platform included through other system headers too resulting into compilation failures. As json-c includes xlocale.h if HAVE_XLOCALE_H is set, provide a way to disable it using an environment variable. So on platforms like ppc64le, where it is included through the json-c file, it can be disabled by exporting JSONC_CFLAGS="-DHAVE_XLOCALE_H=0" and only included through the system header. Reported by Varsha Mehtre, fix suggested by Sandeep Thakkar. FDW-270, Vaibhav Dalvi, reviewed by me.

Fetching them results in a server crash. Also, they are PostgreSQL specific, so don't fetch them. Throw an error when used in a query, instead. FDW-262, Vaibhav Dalvi, reviewed by me.

Earlier, we are finding the pushable OpExpr by calling the function ApplicableOpExpressionList(). Change that to the postgres_fdw way of having a mongo_is_foreign_expr() function and a foreign_expr_walker(). This is cleaner and in-line with PostgreSQL that can be extended easily. This refactoring allowed is to push down the clauses having a Var node wrapped in the RelabelType. Also, for no reason but just for simplicity, we were re-evaluating restriction clauses that are pushed down to MongoDB again on the local side. Remove that altogether which adds to some performance improvement. FDW-194, Vaibhav Dalvi, reviewed by Suraj Kharage, further improvements by Jeevan Chalke.

Earlier we were doing the same, but commit fce56c6 broke that. Fix it. FDW-297, Vaibhav Dalvi, reviewed by Jeevan Chalke.

Standard Support for both EDB Postgres Advanced Server 9.5 and PostgreSQL 9.5 ended in February 2021. Adjust Makefile so that we restrict compilation of mongo_fdw code against v9.5. Update the README accordingly. Vaibhav Dalvi

This reverts commit 9b798ec.

Earlier, if any of the statements fails, autogen.sh script was simply continuing with the next statement, and in the end, returns 0 as if nothing failed. Fix it so that script does fail upon any error and returns a non-zero status code. Reported on GitHub through issues #150 by Andreas Scherbaum (andreasscherbaum). FDW-379, Vaibhav Dalvi, reviewed by Jeevan Ladhe, tested by Rajkumar Raghuwanshi.

The autogen.sh is using cmake3 to compile the jsonc library and mongoc driver. However, it is observed that some of the platforms do not have that, and thus script fails. So, if cmake3 is not present on the system, then fall back to cmake and use that. Note that if the CMake version is below the required one (i.e. 3.1), then the script will fail. Reported on GitHub through issues #149 by Andreas Scherbaum (andreasscherbaum). FDW-379, Vaibhav Dalvi, reviewed by Jeevan Ladhe, tested by Rajkumar Raghuwanshi.

This fixes the errors, warnings and regression test failures appearing when compiled with PostgreSQL 14 and EDB Postgres Advanced Server 14. FDW-366, Vaibhav Dalvi, reviewed by Suraj Kharage.

The regression tests currently have connection parameters like host, port, user, password are hard-coded in each of the test sql file. If someone has to run the regressions against a MongoDB server other than localhost, and different credentials, one has to change all the sql files manually. Change these variables to read from respective environment variables in the sql files. These environment variables are already set to default values in mongodb_init.sh script. Update the README accordingly. Also, a test retrieving user mapping and server options expects these hard coded values in the output and fails when the regression runs against different MongoDB server with different credentials. Fix them too by writing a plpgsql block. FDW-323, Vaibhav Dalvi, reviewed by Suraj Kharage.

The HINT messages in connection_validation.out vary for different MongoDB versions as well as platforms. Avoid these hint messages by setting the verbosity to terse, so that we can get consistent output across platforms and irrespective of the MongoDB version being used. FDW-403, Vaibhav Dalvi, reviwed by Jeevan Ladhe.

The function mongo_get_option_value() iterates over all the options and returns the value of the option requested as a c-string. For the boolean options ssl and weak_cert_validation also mongo_get_options() calls the same function mongo_get_option_value() and they end up being true always, as any string assigned to bool results in true. To fix this correctly, in mongo_get_options() retrieve the value using defGetBoolean() so that we get only boolean value. But, with the current implementation to retrieve every single option one needs to iterate over the entire list of options separately, leading to a quadratic complexity. Instead, fix this by iterating over the options list just once and remembering all the options on the go by retrieving their respective values using respective defGet*() APIs, rather than using the custom function mongo_get_option_value() which always returns a c-string. FDW-316, Vaibhav Dalvi, reviewed by Suraj Kharage, reported by Kashif Zeeshan.

FDW-331, Vaibhav Dalvi, reviewed by Kashif Zeeshan.

autogen.sh helps in avoiding manual installation and automates the download and installation of the appropriate drivers and libraries including mongo-c and json-c driver. But, these drivers get installed at the default locations e.g. /usr/lib/, and user might not have permission to write in this directory. This results in autogen.sh failure. Use environment variables MONGOC_INSTALL_DIR and JSONC_INSTALL_DIR in autogen.sh to set CMAKE_INSTALL_PREFIX to allow custom directory installation for these drivers where the user might have permission. Reported on GitHub through issue #152 by user pgloader. FDW-406, Vaibhav Dalvi, reviewed by Jeevan Ladhe, tested by Rajkumar Raghuwanshi.

MongoGetForeignPlan() builds the list of columns used for projection and restriction clauses in scan_var_list. Pass this list to ColumnList() to avoid duplication of work, and use it to retrieve a list of foreign table columns. Also, rename ColumnList() to mongo_get_column_list(). FDW-391, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Ladhe.

PostgreSQL allows user to refer the complete row as a single column by means of whole row reference. This internally is referred by a reserved varattno zero, currently this is ignored and hence no data is returned when a foreign table is referred with whole row reference. Add functionality to build the list of all the foreign table columns when whole row reference is requested to query the remote server. FDW-391, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Ladhe, tested by Rajkumar Raghuwanshi.

QueryDocument() adds the filters to the MongoDB query. However, for MongoAcquireSampleRows(), there are no quals and thus NO need to call QueryDocument(). Fix that. Also, the list_concat() function has changed in v13, so add a mongo_list_concat() wrapper which calls the correct form depending on the version. Along the way, improve some comments and indentation. FDW-138, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke, with additional improvements by me.

If we have a join between two foreign tables from the same remote server, push that join down to the remote server instead of fetching all the rows for both the tables and performing a join locally. This will have a very significant win in many practical cases. Since it is very unclear that which of the operators are supported on the MongoDB servers, currently join involving only relational and arithmetic operators in join-clauses are pushed down to avoid any potential join failure. Also, only INNER and LEFT/RIGHT OUTER joins are supported, and not FULL, SEMI, and ANTI joins. Moreover, only joins between two tables are pushed down and not when either inner or outer relation is the join itself. We use an aggregate pipeline with the lookup operator. Also, the MongoDB server returns join results as an array, and thus to get in a tuple form we use the unwind operator. This patch also supports the partition-wise join push-down when the table is partitioned. FDW-138, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke, with additional improvements by me. Tested by Rajkumar Raghuwanshi.

The required MongoDB operators are not present (or complex to implement), so don't support join pushdown there. Also, the MongoDB community stopped supporting this version(0.8) of the driver. FDW-138, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke. Tested by Rajkumar Raghuwanshi.

As various version has slight output change, need to add multiple alternate files for the newly added test case. FDW-138, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke. Tested by Rajkumar Raghuwanshi.

Some join queries when evaluated on the remote server take more time than evaluating a join locally. Thus, add an option to control the join push-down to avoid such performance issues. This option can be set at the server level or the table level. If any of the tables involved in the join has this option set to false, then the join will not be pushed down. The table-level value of the option takes precedence over the server-level option value. Default is true. FDW-445, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke.

Vaibhav Dalvi

For invalid host and port testing, we were using the IP address "127.0.0.5" (with default Mongo port). However, on the rhel8 platform, we observed that the same IP is used as loopback IP for the localhost and thus was not testing the stuff we wanted to test and resulted in a regression. So to avoid failure on such platforms, change the address to "127.0.0.10" and port to 9999 instead. FDW-423, Rajkumar Raghuwanshi, reviewed and further fixed by Vaibhav Dalvi, tested by Varsha Mehtre.

FDW-467, Vaibhav Dalvi.

Earlier, the default was the legacy driver. However, Meta is more active and used prominently, thus switching default compilation to the Meta driver. To compile using legacy driver, copy the Makefile.legacy to Makefile. FDW-455, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Kashif Zeeshan.

changes Improve README by adding detailed instructions for mongo-c and json-c library installation. Add instructions for both installation modes - using script and manual steps. Also, add description for SSL options. FDW-412, Vaibhav Dalvi, reviewed by Suraj Kharage and steps verified by Rajkumar Raghuwanshi.

We were estimating the rows by calling ForeignTableDocumentCount() which is costly, and there is no way to avoid that. Thus add a new server-level option use_remote_estimate so that we will estimate the actual rows only when it is set to true. Default is false. FDW-464, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Kashif Zeeshan.

We have overlooked it. Fix it like other types. FDW-427, Vaibhav Dalvi, reviewed by Suraj Kharage, further revised by Jeevan Chalke, tested by Ajay Pal.

If foreign table type is int and remote data is of type double, then the earlier code was returning zero, and vice-versa. Fix it by checking for the input type and converting that into an appropriate destination type. Reported on GitHub through issue #95 by Rick Otten (rotten). Fix verified by GitHub user mariosnic. FDW-418, Vaibhav Dalvi, reviewed by Suraj Kharage and me, tested by Kashif Zeeshan.

It becomes easy to follow and consistent across all the files. FDW-454, Vaibhav Dalvi, reviewed by Suraj Kharage.

Standard Support for both EDB Postgres Advanced Server 9.6 and PostgreSQL 9.6 ended in February 2022. Adjust Makefile so that we restrict compilation of mongo_fdw code against v9.6. Update the README accordingly. Also, cleanup the code for the same. FDW-488, Vaibhav Dalvi, reviewed by Suraj Kharage.

For the array element, in the meta driver, we should be appending the value to the child document. However, mistakenly we were adding the value to the main query document leading to the server crash. Fix the same by correctly passing the child document to bsonAppendUTF8(). Reported on GitHub through issue #155 by zhang699. FDW-466, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Ajay Pal.

For update/delete operations, we rely on a unique value, and for that, we use the "_id" column, which is supposed to be non-null. However, the user can insert a NULL value to this column on the MongoDB side. And as we don't have handling for the same, we ended up with a server crash along the way. Fix it by correctly appending the NULL value to this column in the update/delete query. Along the way, tighten the checks around the "_id" column. We enforce that the first column of the table must be named "_id" and of type "NAME". Also, on the MongoDB side, it should be of the "ObjectId" type. Moreover, we are not inserting any value for the "_id" column by ignoring the passed-in value. Since the MongoDB server inserts the unique value for the missing "_id" column, we are ignoring it. Emit the debug1 log message for the same. FDW-481, Vaibhav Dalvi, reviewed by Suraj Kharage and me, tested by Ajay Pal.

Code changes involves replacing pg_atoi() with strtol(). FDW-489, Vaibhav Dalvi, reviewed by Suraj Kharage and me.

We were using zero, which is not realistic. Thus add some dummy numbers, those are not realistic, but better than zero. And helps in regression runs. FDW-512, Vaibhav Dalvi, reviewed by Suraj Kharage.

…hes. This involves renaming a few variable names and structure names so that they can be used for general purposes. Also, remove some unused code. FDW-137, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke, tested by Kashif Zeeshan.

Push aggregates to the remote MongoDB server instead of fetching all of the rows and aggregating them locally. This gives a very good performance boost for the cases where aggregates can be pushed down. The push-down is currently limited to aggregate functions min, max, sum, avg, and count. Also, aggregate filters and orders are not pushed down as MongoDB does not support them. FDW-137, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke, further changes, mostly cosmetic, by me, tested by Kashif Zeeshan.

Add new test file aggregate_pushdown.sql for testing aggregate push-down feature. Add multiple alternate output files to adjust with different outputs across server versions. FDW-137, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke, tested by Kashif Zeeshan.

Considering the complexity of generating the aggregation pipeline, add an option to control the push-down to avoid any issues. This option can be set at the server level or the table level. If any of the tables involved in the join has this option set to false, then the join will not be pushed down. The table-level value of the option takes precedence over the server-level option value. Default is true. FDW-137, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke, tested by Kashif Zeeshan.

The legacy driver is too old, and MongoDB also stopped supporting it. Given that all the new feature needs a meta driver, having a legacy driver leads to complexity. Add a warning for the same in autogen.sh if someone tries building it, and also, put a note in the README. FDW-471, Vaibhav Dalvi, reviewed by Suraj Kharage.

FDW-450, Vaibhav Dalvi.

Reported on GitHub through pull request #153 by Danny Hermes (dhermes). FDW-536, Danny Hermes, reviewed and further changes by Vaibhav Dalvi.

While preparing a query filter in mongo_query_document(), we have missed considering the Param node in a comparison expression. And thus, was not preparing the query correctly, leading to the server crash. We do have this handling for the equality operator, though. Fix the same. Reported on GitHub through issue #160 by pludov. FDW-529, Vaibhav Dalvi, reviewed and further revised by Jeevan Chalke.

With these improvements, we now support recursive operator expressions, boolean expressions, Relabel types, and Var's on both sides of an operator. FDW-526, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke, tested by Ajay Pal.

Commit a5b12ba missed that. Vaibhav Dalvi.

If possible, consider a path that adds the ORDER BY clause to the remote SQL so that we get the ordered result set from the foreign server itself. It might help us to have an efficient merge join. NULLs behavior is opposite on the MongoDB server. Thus to get an equivalent result, we push ORDER BY with either ASC NULLS FIRST or DESC NULLS LAST to the remote server. Moreover, as MongoDB sorts only on fields, only column names in ORDER BY expressions are pushed down. FDW-134, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Ajay Pal, and a few cosmetic changes by Jeevan Chalke.

FDW-134, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Ajay Pal.

Sometimes getting a sorted result from MongoDB server is slower than performing a sort locally. To have that flexibility add a GUC named mongo_fdw.enable_order_by_pushdown to control the ORDER BY push-down. However, it is set by super users only. The default is true. FDW-134, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Ajay Pal.

Wherever applicable, perform LIMIT and OFFSET operations on the remote server. This reduces network traffic between local PostgreSQL and remote MongoDB servers. Due to limitations on the MongoDB side, we only support Const nodes of type bigint, which allows us to have only 64-bit integer values. FDW-131, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Kashif Zeeshan, and a few cosmetic changes by Jeevan Chalke.

FDW-131, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Kashif Zeeshan

checking. In the case of nested join queries, the existing logic may not correctly propagate the status of the enable_aggregate_pushdown option to the stage where we are accessing the aggregation path. Fix that by saving the status of enable_aggregate_pushdown in MongoFdwRelationInfo, which is updated for every output rel at various stages. It can then be accessed at the time of grouping operation. FDW-560, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Kashif Zeeshan.

We were pushing the ORDER BY clauses to the remote side without verifying whether the sort operator was safe to ship. It resulted in a wrong output when the sort operator isn't default for the sort expression's type. Fix the same. FDW-564, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke.

Commit 234c8cf misplaced the header file into the wrong .c file. Fix the same. Vaibhav Dalvi.

Vaibhav Dalvi.

… level. Along with the enable_join_pushdown server/table level option, add the same named guc to control join push-down at the session level. This allows the user to control the push-down per the query/data pattern. Default is true. FDW-558, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Ajay Pal.

Standard Support for both EDB Postgres Advanced Server 10 and PostgreSQL 10 is already ended. Thus, adjust Makefile so that we restrict compilation of mongo_fdw code against v10. Update the README accordingly. Also, clean up the code for the same. FDW-588, Vaibhav Dalvi, reviewed by Sravan Velagandula.

Jeevan Chalke.

Sometimes, rare though, performing grouping on a remote server is slower than doing it locally. Thus provide a session-level guc named enable_aggregate_pushdown to control the aggregation pushdown. Default is true. FDW-559, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Kashif Zeeshan.

For some queries, we have observed that sorting on a remote server is very slow. Thus we have added a guc named enable_order_by_pushdown. However, if we want to disable/enable remote server sorting at the table or server level, then it can't be done with the guc. Thus add the server/table level option to control the remote sorting. The table-level value of the option takes precedence over the server-level option value. Default is true. FDW-589, Vaibhav Dalvi, reviewed by Sravan Velagandula, tested by Kashif Zeeshan.

Reported on GitHub through issue #58 by simon-wolf. FDW-631, Vaibhav Dalvi, reviewed by Jeevan Chalke.

Code changes involve adjusting the make_restrictinfo() call per PostgreSQL's upstream changes. Matches reltarget entries only on varno/varattno. It also takes into account the new RTEPermissionInfo stuff. Beginning with v16, fs_relids includes the rangetable indexes of outer joins, if any were involved in this join. The new field fs_base_relids includes only base relation indexes and thus mimics fs_relids's old semantics. Need to adjust the fdw code to account for this. FDW-603, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Ajay Pal, a few changes by Jeevan Chalke.

I forgot to include them in the previous commit, i.e. 0919481. FDW-603, Vaibhav Dalvi.

FDW-626, Vaibhav Dalvi, reviewed by Jeevan Chalke.

The execution of mongodb_init.sh is compulsory before running regression, and it is missed to mention it in the README file. Also, update a few comments in test cases. FDW-600, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke.

MongoDB stopped supporting this driver version for a few years now, and we have already given deprecation warnings in the previous releases. So, clean up the related code. FDW-664, Vaibhav Dalvi, reviewed by Suraj Kharage, tested by Ajay Pal.

The main changes include moving the installation steps to the newly created INSTALL.md file. Other changes in README.md are for unifying it with FDW documentation template. Reported on GitHub through issue #168 by mkgrgis (Михаил). FDW-665, initial patch by mkgrgis, further revised and improved by Vaibhav Dalvi.

Tests involving MIN()/MAX() aggregates sometimes take a shortcut path due to hardware differences, resulting in the explain plan changes and, thus causing regression failures. To get a consistent result, change those to SUM()/AVG() aggregates or add a relevant grouping clause. The patch also tweaks the startup cost for the ordered paths so that ORDER BY will get pushed for a few more queries. FDW-598, Vaibhav Dalvi, reviewed by Suraj Kharage and Jeevan Chalke, tested by Ajay Pal.

FDW-676, Suraj Kharage.

Standard Support for both EDB Postgres Advanced Server 11 and PostgreSQL 11 is already ended. Thus, adjust Makefile so that we restrict compilation of mongo_fdw code against v11. Update the README accordingly. Also, clean up the code for the same. FDW-673, Suraj Kharage, reviewed by Sravan Velagandula, tested by Ajaykumar Pal.

Code changes involve adjusting calls to some server-side functions due to their signature changes. FDW-682, Suraj Kharage, reviewed by Jeevan Chalke.

This allows building the extension from the source's contrib directory without the need for pg_config availability in the path. Jeevan Chalke.

Commits on May 15, 2014

Code style changes.

x746e committed May 15, 2014

Configuration menu

View commit details

Copy full SHA for 6b1afd2

Browse repository at this point

Copy the full SHA

6b1afd2 View commit details

Browse the repository at this point in the history

Commits on Jun 5, 2014

README file changes

ibrarahmed1974 committed Jun 5, 2014

Configuration menu

View commit details

Copy full SHA for bce5fc3

Browse repository at this point

Copy the full SHA

bce5fc3 View commit details

Browse the repository at this point in the history

Commits on Oct 29, 2014

Update README.md

Matthew Burghoffer committed Oct 29, 2014

Configuration menu

View commit details

Copy full SHA for 851f954

Browse repository at this point

Copy the full SHA

851f954 View commit details

Browse the repository at this point in the history

Commits on Nov 6, 2014

Added missing file "Makefile.legacy"

ibrarahmad committed Nov 6, 2014

Configuration menu

View commit details

Copy full SHA for 502efca

Browse repository at this point

Copy the full SHA

502efca View commit details

Browse the repository at this point in the history

Commits on Feb 26, 2015

JSON-C compilation error patch.

ibrarahmad committed Feb 26, 2015

Configuration menu

View commit details

Copy full SHA for 0550351

Browse repository at this point

Copy the full SHA

0550351 View commit details

Browse the repository at this point in the history

Commits on Dec 3, 2015

Added PostgreSQL 9.5 support.

ibrarahmad committed Dec 3, 2015

Configuration menu

View commit details

Copy full SHA for 4078a89

Browse repository at this point

Copy the full SHA

4078a89 View commit details

Browse the repository at this point in the history

Commits on May 23, 2016

README update

ibrarahmad committed May 23, 2016

Configuration menu

View commit details

Copy full SHA for 4b2947b

Browse repository at this point

Copy the full SHA

4b2947b View commit details

Browse the repository at this point in the history

Commits on Dec 19, 2016

README update.

ibrarahmad committed Dec 19, 2016

Configuration menu

View commit details

Copy full SHA for e0e11ee

Browse repository at this point

Copy the full SHA

e0e11ee View commit details

Browse the repository at this point in the history

Commits on Mar 10, 2017

Update json-c to 0.12.1

dtheodor committed Mar 10, 2017

Configuration menu

View commit details

Copy full SHA for 9dfa6ae

Browse repository at this point

Copy the full SHA

9dfa6ae View commit details

Browse the repository at this point in the history

Commits on May 10, 2018

Bump version of mongo-c-driver

jjlakis committed May 10, 2018

Configuration menu

View commit details

Copy full SHA for e16476e

Browse repository at this point

Copy the full SHA

e16476e View commit details

Browse the repository at this point in the history

Commits on Aug 3, 2020

Stamp 5.2.7.

jeevanchalke committed Aug 3, 2020

Configuration menu

View commit details

Copy full SHA for aaeb6c7

Browse repository at this point

Copy the full SHA

aaeb6c7 View commit details

Browse the repository at this point in the history

Commits on Oct 21, 2020

Update LICENSE file.

jeevanchalke committed Oct 21, 2020

Configuration menu

View commit details

Copy full SHA for 9b798ec

Browse repository at this point

Copy the full SHA

9b798ec View commit details

Browse the repository at this point in the history

Commits on Oct 27, 2020

Stamp 5.2.8.

jeevanchalke committed Oct 27, 2020

Configuration menu

View commit details

Copy full SHA for b4c1d76

Browse repository at this point

Copy the full SHA

b4c1d76 View commit details

Browse the repository at this point in the history

Commits on Feb 9, 2021

Update EDB copyrights for 2021.

jeevanchalke committed Feb 9, 2021

Configuration menu

View commit details

Copy full SHA for 654780f

Browse repository at this point

Copy the full SHA

654780f View commit details

Browse the repository at this point in the history

Commits on Jun 15, 2021

Stamp 5.2.9.

jeevanchalke committed Jun 15, 2021

Configuration menu

View commit details

Copy full SHA for ceb03dc

Browse repository at this point

Copy the full SHA

ceb03dc View commit details

Browse the repository at this point in the history

Commits on Sep 15, 2021

Stamp 5.2.10.

jeevanchalke committed Sep 15, 2021

Configuration menu

View commit details

Copy full SHA for 3205c5b

Browse repository at this point

Copy the full SHA

3205c5b View commit details

Browse the repository at this point in the history

Commits on Nov 18, 2021

Stamp 5.3.0.

jeevanchalke committed Nov 18, 2021

Configuration menu

View commit details

Copy full SHA for 562d2d2

Browse repository at this point

Copy the full SHA

562d2d2 View commit details

Browse the repository at this point in the history

Commits on May 19, 2022

Stamp 5.4.0.

jeevanchalke committed May 19, 2022

Configuration menu

View commit details

Copy full SHA for 3e2860b

Browse repository at this point

Copy the full SHA

3e2860b View commit details

Browse the repository at this point in the history

Commits on Dec 14, 2022

Stamp 5.5.0.

jeevanchalke committed Dec 14, 2022

Configuration menu

View commit details

Copy full SHA for c250cbd

Browse repository at this point

Copy the full SHA

c250cbd View commit details

Browse the repository at this point in the history

Commits on Jul 14, 2023

Stamp 5.5.1.

jeevanchalke committed Jul 14, 2023

Configuration menu

View commit details

Copy full SHA for 9bfc78d

Browse repository at this point

Copy the full SHA

9bfc78d View commit details

Browse the repository at this point in the history

Commits on Jul 4, 2024

Update README

jeevanchalke committed Jul 4, 2024

Configuration menu

View commit details

Copy full SHA for 5396a8b

Browse repository at this point

Copy the full SHA

5396a8b View commit details

Browse the repository at this point in the history

Commits on Jul 11, 2024

Stamp 5.5.2.

jeevanchalke committed Jul 11, 2024

Configuration menu

View commit details

Copy full SHA for 1236440

Browse repository at this point

Copy the full SHA

1236440 View commit details

Browse the repository at this point in the history

First request for Mongo FDW #26

Are you sure you want to change the base?

First request for Mongo FDW #26

Commits on Jan 23, 2014

Commits on May 13, 2014

Commits on May 14, 2014

Commits on May 15, 2014

Commits on May 25, 2014

Commits on Jun 2, 2014

Commits on Jun 4, 2014

Commits on Jun 5, 2014

Commits on Jun 6, 2014

Commits on Jun 9, 2014

Commits on Jun 24, 2014

Commits on Jul 10, 2014

Commits on Sep 24, 2014

Commits on Sep 25, 2014

Commits on Sep 30, 2014

Commits on Oct 1, 2014

Commits on Oct 2, 2014

Commits on Oct 20, 2014

Commits on Oct 29, 2014

Commits on Oct 30, 2014

Commits on Nov 3, 2014

Commits on Nov 4, 2014

Commits on Nov 5, 2014

Commits on Nov 6, 2014

Commits on Nov 7, 2014

Commits on Nov 11, 2014

Commits on Nov 14, 2014

Commits on Feb 25, 2015

Commits on Feb 26, 2015

Commits on Feb 27, 2015

Commits on Aug 25, 2015

Commits on Sep 11, 2015

Commits on Oct 20, 2015

Commits on Nov 28, 2015

Commits on Nov 29, 2015

Commits on Dec 3, 2015

Commits on Jan 18, 2016

Commits on Jan 28, 2016

Commits on Jan 29, 2016

Commits on May 23, 2016

Commits on Oct 25, 2016

Commits on Dec 15, 2016

Commits on Dec 17, 2016

Commits on Dec 19, 2016

Commits on Jan 2, 2017

Commits on Feb 13, 2017

Commits on Feb 14, 2017

Commits on Feb 21, 2017

Commits on Feb 22, 2017

Commits on Mar 8, 2017

Commits on Mar 10, 2017

Commits on Apr 22, 2017

Commits on Oct 15, 2017

Commits on Dec 1, 2017

Commits on Dec 12, 2017

Commits on Jan 3, 2018

Commits on Mar 13, 2018

Commits on Mar 14, 2018

Commits on Mar 21, 2018

Commits on May 10, 2018

Commits on May 14, 2018

Commits on Aug 7, 2018

Commits on Aug 20, 2018

Commits on Oct 25, 2018

Commits on Oct 26, 2018

Commits on Nov 1, 2018

Commits on Dec 10, 2018

Commits on Dec 20, 2018

Commits on Jan 11, 2019

Commits on Jan 12, 2019

Commits on Jan 25, 2019

Commits on Mar 31, 2019

Commits on Sep 27, 2019

Commits on May 6, 2020

Commits on May 29, 2020

Commits on Jun 17, 2020

Commits on Jun 23, 2020