issue fetching email

[mjac1521]

Summary

I have an email account with more than 15 000 emails in the Sent folder. I create a web interface similar to gmail using Laminas IMAP to fetch the emails from postfix wish use Dovecot to serve the IMAP service.

Current behavior

When I try to access the folder Sent in my interface it take more than 1 minute to show me the emails. I am using pagination so to show me the email in page 1 it take more than a minute, the same for page 2 and so on. I believe every time I hit a page it is reading the whole 15 000 emails to only show me 20 in a page.

How to reproduce

Adding 15 0000 emails o more to a folder. In my case the folder is Sent.

Expected behavior

Laminas-IMAP should fetch emails in a matter of seconds

Sincerely Maykel

[froschdesign]

@mjac1521

I believe every time I hit a page it is reading the whole 15 000 emails to only show me 20 in a page.

Unfortunately we don't know your code, so it would be helpful if you could show us how your query / pagination looks like.

[mjac1521]

Hi Frank,

Thank you so much in advance for taking care of this so sun.
Please find bellow the code we use to get the email and do the pagination

$mail = new Imap([
      'host'     => config('env.HOST'),
      'port'     => config('env.PORT'),
      'user'     => $mailbox->local_part,
      'password' => md5($pass->password),
      'ssl'   => config('env.ENCRY'),
      'novalidatecert' => config('env.VERIFY'),
]);

$cantidad =  $mail->countMessages();
$ini = ($page * 10) + 1;
$end = (((($page * 10) + 10) <= $cantidad ) ? ($page * 10) + 10 : $cantidad);

for($j = $ini - 1; $j < $end; $j++) {
   $mail[$j]->getContent();
}

Sincerely Maykel

[Ocramius]

From your logic, I can see that AbstractStorage#offsetGet($j) will be called:

laminas-mail/src/Storage/AbstractStorage.php

Lines 211 to 221 in b9c896e

	/**
	* ArrayAccess::offsetGet()
	*
	* @param int $id
	* @return \Laminas\Mail\Storage\Message message object
	*/
	#[ReturnTypeWillChange]
	public function offsetGet($id)
	{
	return $this->getMessage($id);
	}

That will in turn go to Imap#getMessage():

laminas-mail/src/Storage/Imap.php

Lines 104 to 122 in b9c896e

	/**
	* Fetch a message
	*
	* @param int $id number of message
	* @return Message
	* @throws Protocol\Exception\RuntimeException
	*/
	public function getMessage($id)
	{
	$data = $this->protocol->fetch(['FLAGS', 'RFC822.HEADER'], $id);
	$header = $data['RFC822.HEADER'];
	$flags = [];
	foreach ($data['FLAGS'] as $flag) {
	$flags[] = static::$knownFlags[$flag] ?? $flag;
	}
	return new $this->messageClass(['handler' => $this, 'id' => $id, 'headers' => $header, 'flags' => $flags]);
	}

Could it be that counting messages is the slow part?

laminas-mail/src/Storage/Imap.php

Lines 59 to 87 in b9c896e

	/**
	* Count messages all messages in current box
	*
	* @param null $flags
	* @throws Exception\RuntimeException
	* @throws Protocol\Exception\RuntimeException
	* @return int number of messages
	*/
	public function countMessages($flags = null)
	{
	if (! $this->currentFolder) {
	throw new Exception\RuntimeException('No selected folder to count');
	}
	if ($flags === null) {
	return count($this->protocol->search(['ALL']));
	}
	$params = [];
	foreach ((array) $flags as $flag) {
	if (isset(static::$searchFlags[$flag])) {
	$params[] = static::$searchFlags[$flag];
	} else {
	$params[] = 'KEYWORD';
	$params[] = $this->protocol->escapeString($flag);
	}
	}
	return count($this->protocol->search($params));
	}

I'd suggest writing a test to verify the issue and sending a patch, @mjac1521.

[mjac1521]

Hi Marcos,
Thank you for your help. Is like you said the issue should be counting so many emails, I have accounts with a couple of emails and those accounts work perfect.

Sincerely Maykel

[Ocramius]

Right, what I'm asking for is if you can design an integration test for the laminas/laminas-mail test suite 😁

We can't design this for you, given availability of contributors.