Is --addNamespaces="100" no longer working? (to add Wikipedia's portal pages to ZIM files for schools!)

[holta]

See --addNamespaces="100" at the bottom of these 2 recipes:

• https://farm.openzim.org/recipes/wikipedia_en_all/config
• https://farm.openzim.org/recipes/wikipedia_en_all_maxi/config

I believe this is supposed to add Wikipedia's portal pages to a ZIM file, and if I'm understanding @kelson42 this was working in the past. Questions:

• Is this parameter no longer working?
• If so, any idea why?
• Or, am I looking in the wrong place when trying to find English Wikipedia's ~540 portal pages within https://library.kiwix.org/viewer#wikipedia_en_all_maxi_2022-05/ ?

Schools are always urgently ask for far more colorful pages so as to animate Wikipedia-centric learning — among younger people especially — who commonly turn against Wikipedia (possibly for life?) When elders too often present Wikipedia as text-heavy + ponderous 🐢

So the question is: Can colorful + youth-friendly portal pages like the following (more graphical i.e. with much less text, very intentionally more inviting!) be made available + easily findable offline — ideally in many of the 87 Wikipedia languages[*] that offer portals?

• https://en.wikipedia.org/wiki/Portal:Ancient_Egypt
• https://en.wikipedia.org/wiki/Portal:Biology
• https://en.wikipedia.org/wiki/Portal:Caribbean
• https://en.wikipedia.org/wiki/Portal:Comics
• https://en.wikipedia.org/wiki/Portal:COVID-19

[*] 87 languages appear to have Wikipedia portals, as indicated in the top-right of https://en.wikipedia.org/wiki/Wikipedia:Contents/Portals

Brief code context: https://github.com/openzim/mwoffliner/search?q=addnamespaces

[kelson42]

@pavel-karatsiuba please verify this ticket is valid, I’m really surprised it does not work?

[holta]

Thank you @pavel-karatsiuba:

1. Discoverability (how to find these 540 portal pages) is the key UX question if you have suggestion(s) for teachers? (After we hopefully find them!)

2. Side Question: Since --addNamespaces does not yet work with --articleList, is it possible to include all portal pages by listing them manually (e.g. listing all 540 portal pages) within --articleList ?

CONTEXT: Students need these colorful/introductory pages in ZIM files like the 1 Million article Wikipedia here: #1756

[kelson42]

@pavel-karatsiuba Please die at start if --articleList and --addNamespace are specified together

[kelson42]

Given ZIM file has indeed not the Portal pages but current version of MWoffliner seems to work fine when tested on a smaller wiki and with another namespace. We should clarify here what goes wrong because even new versions wikipedia_en_all_mini_2023-01.zim miss them. But https://library.kiwix.org/viewer#wikipedia_ru_all_maxi/A/%D0%9F%D0%BE%D1%80%D1%82%D0%B0%D0%BB:%D0%9C%D0%BE%D1%80%D1%81%D0%BA%D0%BE%D0%B9_%D0%BF%D0%BE%D1%80%D1%82%D0%B0%D0%BB has the portals... so it seems somehow to be a specific problem with WPEN.

[kelson42]

I would propose to wait https://farm.openzim.org/pipeline/d51dec8e90efcd0ad3f9be36 is over before continuing the investigation.

[holta]

Progress 🙏

Then, very compact/quick/revealing test cases will hopefully be possible after that 😄

[Inbefortus]

From the most recent English Wikipedia ZIM:

[kelson42]

@Inbefortus So basically there is not problem (anymore). Thx you for the check. I will merge the new automated test and close the ticket.

[holta]

@Inbefortus great news: Would you happen to know why Portals are now restored?

1. Discoverability (how to find these 540 portal pages) is the key UX question if you have suggestion(s) for teachers? (After we hopefully find them!)

A very partial answer to the above question is below — unfortunately 540,497 pages is a flood rather than a clean UX — but still this is a start:

https://library.kiwix.org/viewer#search?books.name=wikipedia_en_all_maxi_2023-02&pattern=Portal%3A

[Inbefortus]

great news: Would you happen to know why Portals are now restored?

@holta I'm not even sure if portals ever existed in the first place. I don't recall them being in the 2021-03 ZIM. They must have been added around the end of 2022 or something like that.

[kelson42]

great news: Would you happen to know why Portals are now restored?

@holta I'm not even sure if portals ever existed in the first place. I don't recall them being in the 2021-03 ZIM. They must have been added around the end of 2022 or something like that.

This is actually the only logical explanation, but I don't remember and really woukd really wonder about that. As long as we can not track precisely changes on Zimfarm, we will face this kind of problem. See openzim/zimfarm#498