Decoding BLOB / CLOB resources #18
Comments
|
I'm attempting to reproduce this, but it works, at least with a simple table (one made by the ibm_db2 test suite) on IBM i and LUW. <?
$conn = new PDO("ibm:[...]", "", "");
$stmt = $conn->prepare("select * from blob_stream");
$res = $stmt->execute();
while (($row = $stmt->fetch()) != false) {
$stream = $row["MY_BLOB"];
$contents = stream_get_contents($stream);
var_dump($contents);
} |
|
(I should mention I've tested this w/ PHP 8.2 on i and PHP 8.1 on Linux+LUW) |
|
FWIW I re-read the issue and I think I saw an implication it's about the lifetimes of the LOB stream. Unfortunately, without a sample, it's hard to reproduce and determine what/why. Especially since it's PHP 5.x era, and a lot of things have changed since then. |
|
Thanks for your response and sorry for the delay. |
|
A case that reproduces would be useful. I'm not familiar enough with Doctrine to tell what SQL it generates. |
|
I think this would be the right translation from Doctrine to raw SQL/PDO: <?php
$conn = new PDO("ibm:DRIVER={IBM DB2 ODBC DRIVER};DATABASE=sample;HOSTNAME=localhost;PORT=60000;PROTOCOL=TCPIP", "db2inst1", "password", [PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION]);
$stmt = $conn->prepare("drop table blob_table");
$res = $stmt->execute();
$stmt = $conn->prepare("create table blob_table (id integer, clobcolumn clob, blobcolumn blob)");
$res = $stmt->execute();
$longBlob = str_repeat('x', 4 * 8192);
$stmt = $conn->prepare("insert into blob_table values (?, ?, ?)");
$stmt->bindValue(1, 1, PDO::PARAM_INT);
$stmt->bindValue(2, "ignored", PDO::PARAM_STR);
$stmt->bindValue(3, fopen('data://text/plain,' . $longBlob, 'r'), PDO::PARAM_LOB);
$res = $stmt->execute();I don't get a crash or userland exception from this. (edit: LUW, Linux amd64, PHP 8.1) |
|
Yes, that's exactly what I'm trying to reproduce, without errors: https://github.com/phansys/pecl-database-pdo_ibm/actions/runs/4186046386/jobs/7253998644. |
|
What would be the best way to run that test from dbal? I have the branch checked out, but the setup for CI looks very complex and I haven't figured out how to untangle it and just run the DB2 relevant tests against PDO_IBM only (...and successfully). |
|
(That is, I'm running |
|
I'd say your approach is enough to achieve a very similar setup. Only two things to consider on the
|
|
Hmmm, running <?xml version="1.0" encoding="utf-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="../../../vendor/phpunit/phpunit/phpunit.xsd"
colors="true"
beStrictAboutOutputDuringTests="true"
beStrictAboutTodoAnnotatedTests="true"
failOnRisky="true"
failOnWarning="true"
convertDeprecationsToExceptions="true"
>
<php>
<ini name="error_reporting" value="-1" />
<ini name="extension" value="/home/calvin/src/pecl-database-pdo_ibm/modules/pdo_ibm.so" />
<ini name="zend.assertions" value="1" />
<var name="db_driver" value="pdo_ibm"/>
<var name="db_host" value="127.0.0.1"/>
<var name="db_user" value="db2inst1"/>
<!--
<var name="db_password" value="Doctrine2018"/>
<var name="db_dbname" value="doctrine"/>
-->
<var name="db_password" value="password"/>
<var name="db_dbname" value="sample"/>
</php>
<testsuites>
<testsuite name="Doctrine DBAL Test Suite">
<!--<directory>../../../tests</directory>-->
<directory>../../../tests/Functional</directory>
</testsuite>
</testsuites>
<coverage>
<include>
<directory suffix=".php">../../../src</directory>
</include>
</coverage>
</phpunit>...I'm getting |
|
What if you filter only the |
|
That cuts the number of tests down to a manageable number, though I still need to figure out why it's not finding PDO_IBM. |
|
What is the message you're receiving? Is it from Doctrine or from PDO? |
|
Mix of both; PDO and Doctrine: |
|
For what you share, I guess is not being loaded. You could use |
|
Oh, you have to run PHPUnit with PHP having those options set, since its interface for that can't change Now I'm getting |
|
Yeah, default port wasn't proper. Now it's |
|
In case you need it, you may specify the |
|
After remembering how to create a database in LUW (can you tell I'm an i user?), I can reproduce this with the slightly unwieldy command line |
|
I ran the test once and now I can't run any of them again 🤨 |
|
Possibly that's because the value there is a stream resource created by |
|
No, it affects the other tests too. The stack trace is hinting to me it might be something about fetching table metadata? |
|
If I followed the flow correctly, I think the value passed there as argument 2 to |
|
It's some kind of stream resource, but if I fetch it, it becomes NULL and returns nothing 🤨 If I check what Per |
|
oh that's really funny diff --git a/src/Schema/DB2SchemaManager.php b/src/Schema/DB2SchemaManager.php
index 0308d69b5..7f582edde 100644
--- a/src/Schema/DB2SchemaManager.php
+++ b/src/Schema/DB2SchemaManager.php
@@ -100,6 +100,9 @@ class DB2SchemaManager extends AbstractSchemaManager
$default = null;
+ if (is_resource($tableColumn['default'])) {
+ $tableColumn['default'] = stream_get_contents($tableColumn['default']);
+ }
if ($tableColumn['default'] !== null && $tableColumn['default'] !== 'NULL') {
$default = $tableColumn['default'];
|
|
rebuilding from PHP master with |
Looks like stuff in stmt_res is getting clobbered? |
|
I wonder if valgrind would help find the clobbering |
|
Valgrind reports nothing. I have a thick slathering of printf something like this so far: ...kinda hinting at lifetimes. Curiously, the LOB stream create calls LOB stream read with a null buffer - and I noticed the LOB handling code is almost completely different on i: https://github.com/php/pecl-database-pdo_ibm/blob/master/ibm_statement.c#L143-L168 ...and that does handle a null buffer. The question is, why is it different for LUW vs. i? |
|
Regarding the differences between LUW and i, IIUC, IBM seems to suggest different drivers for these platforms:
But since I'm not familiar with the IBM products, I can not be sure if these are just 2 possible variants. |
|
i has a (quirky and different) SQL/CLI implementation that's mostly compatible, which is why the #ifdef is around, but this case seems really curious to me, considering the massive behaviour difference (especially when the LUW case also seemingly triggers those behaviours). |
|
I think it's lifetimes. Putting From the looks of it, the LOB should add a reference to the statement, and we should be keeping track of statements in general. fwiw, more printf debugging slurry, so you can kinda see the logic where I go with it: |
Otherwise, the statement can be freed before the LOB can be used. The strategy used is like how pdo_oci8 does it (hold ref to handle) and not like pdo_pgsql does it (hold a hashmap of handles). Related to GH-18
|
I have the (linked by GH) PR that makes it no longer crash, but it still has some issues, it seems... |
As doctrine/dbal#5924 is an attempt to restore support for this extension, maybe there is something missing to consider. I make this comment in order to prevent taking that PR as a solid base for the final implementation. |
I'll note that at least for IBM i, we ('we' being IBM and Seiden Group) don't recommend PDO_IBM and ibm_db2 for any new applications, but rather ODBC with the IBM i ODBC driver instead. That notice is on the PECL package for ibm_db2 at least, but I should probably make it more obvious there and here. We're still maintaining it for existing legacy users on IBM i though. Of course, I don't know how IBM feels about it for Db2 LUW users. Not to say any issues found in PDO_IBM and ibm_db2 shouldn't be fixed, of course. Including this one. |
|
The recommendation is for connections made from the same server or even for remote connections? In my use case, I need to connect to a DB2 for i database from a dockerized Debian mounted on a Centos instance 😵💫 In the
In the IBM support docs, it seems to recommend ODBC just for applications running on the same database server:
|
|
I think there's some politicking going on - using the Db2 LUW driver to connect for Db2 on i requires the Db2 Connect LPP on i, which I think got sold to Rocket Software. IBM itself maintains the ODBC driver for i, which runs on i/Winddows/Linux and doesn't need any extra licensed products. Again, for LUW client to LUW server, none of this is relevant. |
|
Oh, I see. Thank you very much for your explanation. Off topic:I think that's why I can use BTW, using the IBM i Access ODBC approach, I can connect against the DB2 for i server, but not anymore with the Docker container: |
|
@phansys The IBM i team recommends using IBM i ODBC driver (and associated middleware, eg. pdo_odbc, etc) to connect to IBM i databases. We recommend against using ibm_db2 / pdo_ibm unless you're running on IBM i and only then for existing applications that haven't been migrated to ODBC.
I'm pretty sure this doc is really just meant for PHP users who run on IBM i and not making any recommendations for PHP users on other platforms connecting to IBM i. FWIW, I think nearly all IBM i customers running PHP do so on IBM i (I don't know of a single customer that doesn't, or at least runs on a different platform and then connects to IBM i). I'll get the wording improved to be clear that we recommend ODBC in all cases. |
|
Working on the Doctrine driver, I found this suspicious definition: public function getBlobTypeDeclarationSQL(array $column)
{
// todo blob(n) with $column['length'];
return 'BLOB(1M)';
}The same behavior exists with |
I'd like to know what is the status of https://bugs.php.net/bug.php?id=59158.
Is the issue still present? Is there something I can do in order to collaborate with the fix?
doctrine/dbal dropped the support for this extension because that problem, and I'd like to restore that support.
Thank you!
The text was updated successfully, but these errors were encountered: