Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bad results crawling mantine docs ?t=props #156

Open
Dimfred opened this issue Oct 11, 2024 · 3 comments
Open

Bad results crawling mantine docs ?t=props #156

Dimfred opened this issue Oct 11, 2024 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@Dimfred
Copy link

Dimfred commented Oct 11, 2024

Hey thx for the lib :)

Playing around with it trying to crawl: https://mantine.dev/core/button/?t=props
If you have a quick answer why it doesn't work, that would be great, else I'll probably take a look the next days.
Cheers

@Dimfred Dimfred changed the title Can't crawl mantine docs Bad results crawling mantine docs ?t=props Oct 11, 2024
@Marisa2312
Copy link

__

@unclecode unclecode self-assigned this Oct 12, 2024
@unclecode unclecode added the bug Something isn't working label Oct 12, 2024
@unclecode
Copy link
Owner

Hi @Dimfred I am currently working on version 0.3.6. While testing the links you provided with this version, it appears to be functioning as expected at my end. However, I'm not sure what issue you're experiencing. Nevertheless, there are two options. By Monday or Tuesday, we will update the library in Pypi, but for now, you can pull the latest version from the branch "0.3.6". If you're encountering any specific error messages, I would appreciate it if you could share them with me. Thank you for trying our library.

[LOG] 🌤️  Warming up the AsyncWebCrawler
[LOG] 🌞 AsyncWebCrawler is ready to crawl
[LOG] 🕸️ Crawling https://mantine.dev/core/button/?t=props using AsyncPlaywrightCrawlerStrategy...
[LOG] ✅ Crawled https://mantine.dev/core/button/?t=props successfully!
[LOG] 🚀 Crawling done for https://mantine.dev/core/button/?t=props, success: True, time taken: 2.69 seconds
[LOG] 🚀 Content extracted for https://mantine.dev/core/button/?t=props, success: True, time taken: 0.09 seconds
[LOG] 🔥 Extracting semantic blocks for https://mantine.dev/core/button/?t=props, Strategy: AsyncWebCrawler
[LOG] 🚀 Extraction done for https://mantine.dev/core/button/?t=props, time taken: 0.10 seconds.

@Dimfred
Copy link
Author

Dimfred commented Oct 12, 2024

I will try your version and check the results. I don't have issues accessing the link or something, the content is just not there, like the main parts which are present on that page are missing (the table with the props).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants