Zim format detection does not work in 2.12.1 and 2.12.2

[Self-Perfection]

⚠️ This issue respects the following points: ⚠️

• This is a bug. Not a question or feature request.
• The topic is not already reported at Issues. (I've searched it).
• Markor is up to date. See Releases for the latest version. Updates are available from F-Droid and GitHub.
• The bug is still present in the latest development version (git master). (Please download and try the test version of Markor, named Marder. Don't worry; Markor and Marder appear as completely separate applications. You can install both side-by-side, and Markor settings are not touched. In case the issue is resolved there, you don't need to create a bug report. The change will be part of the next Markor update.)

Description

Zim files are handled as plain text files. There is no format parsing, zim wiki headers are not hidden, tags are not handled.

Steps to reproduce

1. Open zim wiki file
2. It is shown as plain text

Information

Android version: irrelevant
Device: irrelevant
App Version: 2.12.2

Source

F-Droid

Format / File type

Wikitext, Zim

Additional info / Log

-

[harshad1]

Also solved in #2088 #2307

Edit. Sorry, wrong pr linked

[Self-Perfection]

Also solved in #2088

That PR is closed a year ago but the issue is a recent regression. I suppose it was introduced about a month or two ago

[fredericjacob]

I can confirm that with the latest version Zim files are not detected as Zim any more, just as plaintext.
Zim pages are in fact *.txt, but they start with the "header" lines e.g.

Content-Type: text/x-zim-wiki Wiki-Format: zim 0.6 Creation-Date: 2019-01-14T23:41:51+01:00

Probably this file content check for *.txt was broken somehow, so Markor just treats them as plain text files due to the ending.

If #2307 fixes the issue already, nice! :)

[fredericjacob]

@harshad1 I just took a look into #2307 , and if I didn't overlook something, I think it won't fix the issue, unfortunately.
I analyzed the problem a bit (just looked at the Github code via Octodroid on the smartphone, so I didn't test anything), and I think I found the reason for the problem.

It seems that this commit introduced the bug:
f35383a

In Document.java, you removed the if-elseif-Statements with a loop iteration over the newly introduced FormatRegistry.FORMATS:

The problem here is: Just using the lower-case filename fnlower in isFileOutOfThisFormat(fnlower) in a generic way is not feasible. In the original code most calls used fnlower as the argument, but Zim, Embedbinary and Orgmode used getPath(): isFileOutOfThisFormat(getPath()). And using the full path of the file is important, because Markor needs it to open the file, look into the content and decide if it contains a Zim header (don't know about the other two formats, but probably they need to be checked somehow too - so there could be similar bugs).
Moreover, the order of the file format checks is important: Originally, plaintext format was only used as fallback if no other format matches before. With the changed code, plaintext is at position 2 and Zim at position 4, so Zim files (ending *.txt just as plaintext) don't have any chance of being matched as Zim.
But like I said, it is necessary to also look into the file contents in addition to checking the line ending to decide if a file is Zim or just an "ordinary" plaintext file, so changing the order is not enough.

[harshad1]

Ah. You're right. I will fix this today

[harshad1]

I have this fixed and pushed up in #2307

Please test.

[fredericjacob]

@harshad1 Ah I saw that you already changed the order of the formats in a previous commit. 👍
Could work now with the change to the file path instead of just the lower case filename.

I just have my smartphone right now and cannot build the APK for testing, however, and I don't even have an Android dev environment set up on my computer at this time... ^^
But there's a valid Zim page under samples/Zim-Sample-Notebook/Root_Page.txt, so you could check if highlighting works if you open this file.

Maybe also @Self-Perfection could do a test if it works now.

Additionally I found a suspicious line in WikitextTextConverter - but this seems to have been there before in a similar form, so probably not exactly related to the issue here:
return Arrays.asList(new String[]{".wikitext"}).contains(ext);
I don't know what this ".wikitext" string could be - if it is supposed to be a file ending, then it's wrong, there is no such file ending as ".wikitext" in Zim, just ".txt". (The term "Wikitext" in general is not related to Zim in any way and not a good naming for the specific format IMO, but that's a different story.)

[Self-Perfection]

Sorry, I can not test. Compiling from source code is beyond my capabilities now.

[gsantner]

testbuilsa of prs are at the github actions tab.

[Self-Perfection]

testbuilsa of prs are at the github actions tab.

👍 Wow. Much CI. Such technology.

So I tested apk from #2307 and it indeed restored zim format handling

[fredericjacob]

Nice to know that the CI pipelines includes test apks. 👍
I tried it out as well and can confirm: the zim format detection works.

[harshad1]

Marking this done as the fix is in #2307